From patchwork Wed May 11 02:30:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12845701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 994B6C433F5 for ; Wed, 11 May 2022 02:23:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239385AbiEKCXk (ORCPT ); Tue, 10 May 2022 22:23:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234897AbiEKCXh (ORCPT ); Tue, 10 May 2022 22:23:37 -0400 Received: from heian.cn.fujitsu.com (mail.cn.fujitsu.com [183.91.158.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 978BC15A747; Tue, 10 May 2022 19:23:36 -0700 (PDT) IronPort-Data: A9a23:Ysxpea8N60b4liqVHXTFDrUDWXyTJUtcMsCJ2f8bfWQNrUp2gTYGzWFOWG2EOv7ZZDOmfd0jPoq29hhX6pCDx4IyHldlrnsFo1Bi85ScXYvDRqvT04J+FuWaFQQ/qZx2huDodKjYdVeB4Ef9WlTdhSMkj/vQHOKlULes1h1ZHmeIdg9w0HqPpMZp2uaEsfDha++8kYuaT//3YTdJ6BYoWo4g0J9vnTs01BjEVJz0iXRlDRxDlAe2e3D4l/vzL4npR5fzatE88uJX24/+IL+FEmPxp3/BC/uulPD1b08LXqXPewOJjxK6WYD72l4b+HN0if19aZLwam8O49mNt9Rw2tVMt525Ty8nI6/NhP8AFRJfFkmSOIUfouOefSLj6pf7I0ruNiGEL+9VJFMnP58J+LwvWTlm+vkRKTRLZReG78qywbSmWqxvi94lIc3DIowSoDdjwCvfAPJgRorMK43O5NlFzHIqisVHNejRatBfajd1ahnEJRpVNT8/Cp0xtPWpi2HyNTZRwG95D4JfD3P7lVQ3ieaydoGOPIHieCmcpW7AzkquwogzKkty2ASj9Ae4 IronPort-HdrOrdr: A9a23:WlayW6jrwaR9bVuBQuUQZAHsg3BQXuYji2hC6mlwRA09TyX4rbHLoB1/73LJYVkqNk3I5urrBEDtexLhHP1OkOws1NWZLWrbUQKTRekM0WKI+UyDJ8SRzI5g/JYlW61/Jfm1NlJikPv9iTPSL/8QhPWB74Ck7N2z80tQ X-IronPort-AV: E=Sophos;i="5.88,333,1635177600"; d="scan'208";a="124142433" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 11 May 2022 10:23:32 +0800 Received: from G08CNEXMBPEKD05.g08.fujitsu.local (unknown [10.167.33.204]) by cn.fujitsu.com (Postfix) with ESMTP id B12624D1718C; Wed, 11 May 2022 10:23:27 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD05.g08.fujitsu.local (10.167.33.204) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Wed, 11 May 2022 10:23:27 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Wed, 11 May 2022 10:23:25 +0800 From: Li Zhijian To: Zhu Yanjun , Jason Gunthorpe , CC: , Li Zhijian Subject: [PATCH v2 1/2] RDMA/rxe: Update wqe_index for each wqe error completion Date: Wed, 11 May 2022 10:30:29 +0800 Message-ID: <20220511023030.229212-2-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220511023030.229212-1-lizhijian@fujitsu.com> References: <20220511023030.229212-1-lizhijian@fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: B12624D1718C.AAF3E X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Previously, if user space keeps sending abnormal wqe, queue.prod will keep increasing while queue.index doesn't. Once queue.index==queue.prod in next round, req_next_wqe() will treat queue as empty. In such case, no new completion would be generated. Update wqe_index for each wqe completion so that req_next_wqe() can get next wqe properly. Signed-off-by: Li Zhijian --- V2: Fix typos in commit logs drivers/infiniband/sw/rxe/rxe_req.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index a0d5e57f73c1..8bdd0b6b578f 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -773,6 +773,8 @@ int rxe_requester(void *arg) if (ah) rxe_put(ah); err: + /* update wqe_index for each wqe completion */ + qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; __rxe_do_task(&qp->comp.task); From patchwork Wed May 11 02:30:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12845700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ACAEC433F5 for ; Wed, 11 May 2022 02:23:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229448AbiEKCXh (ORCPT ); Tue, 10 May 2022 22:23:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233187AbiEKCXf (ORCPT ); Tue, 10 May 2022 22:23:35 -0400 Received: from heian.cn.fujitsu.com (mail.cn.fujitsu.com [183.91.158.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E991315A747; Tue, 10 May 2022 19:23:33 -0700 (PDT) IronPort-Data: A9a23:2Yt6IazYVmULPJPjVhl6t+fuxCrEfRIJ4+MujC/XYbTApGl03jQFzzFJDW/Sa/mKZWXxfop2PIS/9E0PusOAzdY1HQtv/xmBbVoQ95OdWo7xwmQcns+qBpSaChohtq3yU/GYRCwPZiKa9kfF3oTJ9yEmj/nSHuOkUYYoBwgqLeNaYHZ44f5cs75h6mJYqYDR7zKl4bsekeWGULOW82Ic3lYv1k62gEgHUMIeF98vlgdWifhj5DcynpSOZX4VDfnZw3DQGuG4EgMmLtsvwo1V/kuBl/ssItij1LjmcEwWWaOUNg+L4pZUc/H6xEEc+WppieBmXBYfQR4/ZzGhm9FjyNRPtJW2YQk0PKzQg/lbWB5de817FfQcpOabeyTk66R/yGWDKRMA2c5GHlA0L5waoL4vWUlB8PUZLHYGaRXrr+a3xq+rD+phnMIuKOH1M44F/HJt1zfUCbAhW5+ra6HL48JImS08g8lmA/nTfYwaZCBpYRCGZAdAUn8VB50WjualnnS5eDQwlb4/jcLb+ECKlEoojuera4GTJ7S3qQxuth7wjgr7E67RW3n27OCi9Ac= IronPort-HdrOrdr: A9a23:zgUm9K3MGoEoUuCrA2BiGwqjBI4kLtp133Aq2lEZdPU1SL39qynKppkmPHDP5gr5J0tLpTntAsi9qBDnhPtICOsqTNSftWDd0QPGEGgI1/qB/9SPIU3D398Y/aJhXow7M9foEGV95PyQ3CCIV/om3/mLmZrFudvj X-IronPort-AV: E=Sophos;i="5.88,333,1635177600"; d="scan'208";a="124142432" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 11 May 2022 10:23:32 +0800 Received: from G08CNEXMBPEKD06.g08.fujitsu.local (unknown [10.167.33.206]) by cn.fujitsu.com (Postfix) with ESMTP id 3B0CD4D17192; Wed, 11 May 2022 10:23:28 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD06.g08.fujitsu.local (10.167.33.206) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Wed, 11 May 2022 10:23:26 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Wed, 11 May 2022 10:23:26 +0800 From: Li Zhijian To: Zhu Yanjun , Jason Gunthorpe , CC: , Li Zhijian Subject: [PATCH v2 2/2] RDMA/rxe: Generate error completion for error requester state Date: Wed, 11 May 2022 10:30:30 +0800 Message-ID: <20220511023030.229212-3-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220511023030.229212-1-lizhijian@fujitsu.com> References: <20220511023030.229212-1-lizhijian@fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 3B0CD4D17192.AC456 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org SoftRoCE always returns success when user space is posting a new wqe where it usually just enqueues a wqe. Once the requester state becomes QP_STATE_ERROR, we should generate error completion for all subsequent wqe. So the user is able to poll the completion event to check if the former wqe is handled correctly. Here we check QP_STATE_ERROR after req_next_wqe() so that the completion can associate with its wqe. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_req.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 8bdd0b6b578f..ed6a486c4343 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -624,7 +624,7 @@ int rxe_requester(void *arg) rxe_get(qp); next_wqe: - if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR)) + if (unlikely(!qp->valid)) goto exit; if (unlikely(qp->req.state == QP_STATE_RESET)) { @@ -646,6 +646,14 @@ int rxe_requester(void *arg) if (unlikely(!wqe)) goto exit; + if (qp->req.state == QP_STATE_ERROR) { + /* + * Generate an error completion so that user space is able to + * poll this completion. + */ + goto err; + } + if (wqe->mask & WR_LOCAL_OP_MASK) { ret = rxe_do_local_ops(qp, wqe); if (unlikely(ret))