From patchwork Sun Mar 27 07:55:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12792755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F951C433FE for ; Sun, 27 Mar 2022 07:56:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232063AbiC0H5q (ORCPT ); Sun, 27 Mar 2022 03:57:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233512AbiC0H5p (ORCPT ); Sun, 27 Mar 2022 03:57:45 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF963D2 for ; Sun, 27 Mar 2022 00:56:07 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A7E97B80CC9 for ; Sun, 27 Mar 2022 07:56:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9324C340EC; Sun, 27 Mar 2022 07:56:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648367765; bh=MPGsw+DG33IwDM8iKgBmxMH5V5e939YsNdnUfwLNqjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=trqhVjGuNgkvaxf8m2LHZOWmsH2xkjIctIvn/oiW0b1jkLpum98CJ0UrsP3/XMsob 2wP0Eqe7tdqiOPeLe5Bm19IiMoVhrIwuipzVo8v/ul5eOmMALICzUOYmNrTI9QTQzQ +tfcT8NvsO2nhvsObhw2NNtJNiYHIBM66k3D5GTzT2EhxCcqXhe5Y+YI+QsGuLm/G/ Tl62+/d6UXdUm4wOaU95VkgstWqIQwTdmvVnCY02zE5xhvK8+YS99deTYB+xyZdNd1 HzC9Vw1TZtMpU7vycBlRG+iwGcdb/rXw9s82E8x7bmkp8QaQzkS1TXI+jh458Reph8 QHQuTCR7lwYxQ== From: Leon Romanovsky To: Jason Gunthorpe Cc: Aharon Landau , linux-rdma@vger.kernel.org, Maor Gottlieb , Mark Zhang , Shay Drory Subject: [PATCH rdma-rc 1/3] RDMA/mlx5: Don't remove cache MRs when a delay is needed Date: Sun, 27 Mar 2022 10:55:46 +0300 Message-Id: <73c3fef51ee2bb50ae240306472d9562ba05916f.1648366974.git.leonro@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Don't remove MRs from the cache if need to delay the removal. Fixes: b9358bdbc713 ("RDMA/mlx5: Fix locking in MR cache work queue") Signed-off-by: Aharon Landau Reviewed-by: Shay Drory Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mr.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 8a87a2f074e4..4a01a0ad7c90 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -562,9 +562,10 @@ static void __cache_work_func(struct mlx5_cache_ent *ent) spin_lock_irq(&ent->lock); if (ent->disabled) goto out; - if (need_delay) + if (need_delay) { queue_delayed_work(cache->wq, &ent->dwork, 300 * HZ); - remove_cache_mr_locked(ent); + goto out; + } queue_adjust_cache_locked(ent); } out: From patchwork Sun Mar 27 07:55:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12792753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22B8DC433EF for ; Sun, 27 Mar 2022 07:56:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229999AbiC0H5k (ORCPT ); Sun, 27 Mar 2022 03:57:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229531AbiC0H5i (ORCPT ); Sun, 27 Mar 2022 03:57:38 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8729DD2 for ; Sun, 27 Mar 2022 00:56:00 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 39374B80BAA for ; Sun, 27 Mar 2022 07:55:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 786D9C340EC; Sun, 27 Mar 2022 07:55:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648367758; bh=XN4g2Sxxsq6w09a2Og/3Oktz3XHX0uhKTvkrBHZ5xUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y6fDXVdMoxKu1GJgjWuwKi/jJdPuDZFPsAMz73+3OnAinVLQYJLmXRO70Y+9ne0SE d+QN53vaIfYCJZfuC8VhHXMLyXc69Zzo4lC0Ys5FCyAmVcshGyPAypo01xoLqjOTIj lwj6m9WYgwS76q11csrNxgIn+gkPvyxx/FX0YxNoBiFkj9CgQYEy9dSbzhCV4X2U2D IFvM+g1mPALGcV/7IkVxcv3fidA3lOUkQdZa3T6rWgjJcQYn7ehBTP3etLbdoRMCSG 1DL6xgDLW77LHPZvyYkva3K8LUR5ho43WkyWg6qgu1+72t+amBtqb5EcoALXWfBmA6 6Fo5EEI+WJGfA== From: Leon Romanovsky To: Jason Gunthorpe Cc: Aharon Landau , linux-rdma@vger.kernel.org, Maor Gottlieb , Mark Zhang , Shay Drory Subject: [PATCH rdma-rc 2/3] RDMA/mlx5: Add a missing update of cache->last_add Date: Sun, 27 Mar 2022 10:55:47 +0300 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Update cache->last_add when returning an MR to the cache so that the cache work won't remove it. Fixes: b9358bdbc713 ("RDMA/mlx5: Fix locking in MR cache work queue") Signed-off-by: Aharon Landau Reviewed-by: Shay Drory Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mr.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 4a01a0ad7c90..1eef74b952b6 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -614,6 +614,7 @@ static void mlx5_mr_cache_free(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr) { struct mlx5_cache_ent *ent = mr->cache_ent; + WRITE_ONCE(dev->cache.last_add, jiffies); spin_lock_irq(&ent->lock); list_add_tail(&mr->list, &ent->head); ent->available_mrs++; From patchwork Sun Mar 27 07:55:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 12792754 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D736FC433EF for ; Sun, 27 Mar 2022 07:56:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229531AbiC0H5m (ORCPT ); Sun, 27 Mar 2022 03:57:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232063AbiC0H5l (ORCPT ); Sun, 27 Mar 2022 03:57:41 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A569198 for ; Sun, 27 Mar 2022 00:56:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E1C89B80BAA for ; Sun, 27 Mar 2022 07:56:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26CBDC340EC; Sun, 27 Mar 2022 07:56:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648367761; bh=wN9CiNnMGdRrwuQuTj/9a+jyX/ChqbStpMdKIyRVY9Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tX+YOp6Lsmg5fhTu9loIMUJRsafAozxsasSm94TkhmsUHPzQOlUy6TVPXVhbu27Mi F0vjDlAADXBDrgpCFV50A75FhVqVlPnpEjsrunkSAbg/LPxcFeY9kOg5fwvcTZi2zr /sU9HT7RnOH9IpFQu58GPazW+5xEk9Jz2KC6hwjDBsSk6rAoNNjwq5Ch9iKrJl37cX OWud6z7VW40J4oW1fUW+v71wTAWcnajwO/NsRRrGDagRGNdn25IbXADyqqCujLcbRU 85RZTVOOFmPaRiJmrj57a4a88w55beIdrEfdphmcX2FzKSxyTuLLc2kkNVwgZQjM0n pmM/Fy21zC6uw== From: Leon Romanovsky To: Jason Gunthorpe Cc: Mark Zhang , Aharon Landau , linux-rdma@vger.kernel.org, Maor Gottlieb , Shay Drory Subject: [PATCH rdma-rc 3/3] IB/cm: Cancel mad on the DREQ event when the state is MRA_REP_RCVD Date: Sun, 27 Mar 2022 10:55:48 +0300 Message-Id: <2e8c8487bb5f7d3706683cc3719612c19702385f.1648366974.git.leonro@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mark Zhang On the passive side when the DREQ event comes, if the current state is MRA_REP_RCVD, it needs to cancel the MAD before enter the DREQ_RCVD and TIMEWAIT state, otherwise the destroy_id may block until this mad will reach timeout. Fixes: a977049dacde ("[PATCH] IB: Add the kernel CM implementation") Signed-off-by: Mark Zhang Reviewed-by: Maor Gottlieb Signed-off-by: Leon Romanovsky --- drivers/infiniband/core/cm.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index c903b74f46a4..add21a82c428 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -2824,6 +2824,7 @@ static int cm_dreq_handler(struct cm_work *work) switch (cm_id_priv->id.state) { case IB_CM_REP_SENT: case IB_CM_DREQ_SENT: + case IB_CM_MRA_REP_RCVD: ib_cancel_mad(cm_id_priv->msg); break; case IB_CM_ESTABLISHED: @@ -2831,8 +2832,6 @@ static int cm_dreq_handler(struct cm_work *work) cm_id_priv->id.lap_state == IB_CM_MRA_LAP_RCVD) ib_cancel_mad(cm_id_priv->msg); break; - case IB_CM_MRA_REP_RCVD: - break; case IB_CM_TIMEWAIT: atomic_long_inc(&work->port->counters[CM_RECV_DUPLICATES] [CM_DREQ_COUNTER]);