From patchwork Tue Apr 12 02:29:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12809967 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37985C433EF for ; Tue, 12 Apr 2022 02:22:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244968AbiDLCYw (ORCPT ); Mon, 11 Apr 2022 22:24:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243459AbiDLCYu (ORCPT ); Mon, 11 Apr 2022 22:24:50 -0400 Received: from heian.cn.fujitsu.com (mail.cn.fujitsu.com [183.91.158.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E21E62E9D2; Mon, 11 Apr 2022 19:22:34 -0700 (PDT) IronPort-Data: A9a23:ytdQmaA2u98BCBVW//Dhw5YqxClBgxIJ4g17XOLfBAXrhTIl02YGyWVLWDqOPv2LZ2akLotxaozg/UoAsMOAx9UxeLYW3SszFioV86IpJjg4wn/YZnrUdouaJK5ex512huLocYZkHhcwmj/3auK79SMkjPnRLlbBILWs1h5ZFFYMpBgJ2UoLd94R2uaEsPDha++/kYqaT/73ZDdJ7wVJ3lc8sMpvnv/AUMPa41v0tnRmDRxCUcS3e3M9VPrzLonpR5f0rxU9IwK0ewrD5OnREmLx9BFrBM6nk6rgbwsBRbu60Qqm0yIQAvb9xEMZ4HFaPqUTbZLwbW9TiieJntJwwdNlu4GySBsyI+vHn+F1vxxwSnslYf0WpOWXSZS4mYnJp6HcSFP+0vd8HUNsZdVA0ulyCGBKs/cfLVglahGFmvLzw7+hTORortosIdOtP44FvHxkizbDAp4ORZHFXrWP/9Nd1R8uic1UW/XTfcwUbXxodhuoSx9ANX8FCZ8mkaGjjxHCn5dwwL6OjfNvpTGNk0oqi/6wWOc5s+eiHa199nt0bEqal4ghPiwnCQ== IronPort-HdrOrdr: A9a23:6EyDRqvw/Yzs9+eQQw1pcvwj7skDStV00zEX/kB9WHVpm62j5qSTdZEguCMc5wx+ZJheo7q90cW7IE80lqQFhLX5X43SPzUO0VHARO5fBODZsl/d8kPFltJ15ONJdqhSLJnKB0FmsMCS2mKFOudl7N6Z0K3Av4vj80s= X-IronPort-AV: E=Sophos;i="5.88,333,1635177600"; d="scan'208";a="123488428" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 12 Apr 2022 10:22:31 +0800 Received: from G08CNEXMBPEKD04.g08.fujitsu.local (unknown [10.167.33.201]) by cn.fujitsu.com (Postfix) with ESMTP id EA3E74D16FC6; Tue, 12 Apr 2022 10:22:27 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD04.g08.fujitsu.local (10.167.33.201) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 12 Apr 2022 10:22:27 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 12 Apr 2022 10:22:26 +0800 From: Li Zhijian To: , , CC: , Li Zhijian Subject: [PATCH 1/3] RDMA/rxe: Remove useless parameters for update_state() Date: Tue, 12 Apr 2022 10:29:01 +0800 Message-ID: <20220412022903.574238-1-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-yoursite-MailScanner-ID: EA3E74D16FC6.A02E7 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org wqe was not used by update_state() so far. aaaf62e06623 ("RDMA/rxe: Remove useless argument for update_state()") just did a partial fixes. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_req.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 5f7348b11268..bf7493bab9b9 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -525,8 +525,7 @@ static void rollback_state(struct rxe_send_wqe *wqe, qp->req.psn = rollback_psn; } -static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe, - struct rxe_pkt_info *pkt) +static void update_state(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { qp->req.opcode = pkt->opcode; @@ -753,7 +752,7 @@ int rxe_requester(void *arg) goto err; } - update_state(qp, wqe, &pkt); + update_state(qp, &pkt); goto next_wqe; From patchwork Tue Apr 12 02:29:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12809965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1E1FC433F5 for ; Tue, 12 Apr 2022 02:22:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242026AbiDLCYt (ORCPT ); Mon, 11 Apr 2022 22:24:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229921AbiDLCYs (ORCPT ); Mon, 11 Apr 2022 22:24:48 -0400 Received: from heian.cn.fujitsu.com (mail.cn.fujitsu.com [183.91.158.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 97C8F2E9CC; Mon, 11 Apr 2022 19:22:32 -0700 (PDT) IronPort-Data: A9a23:p4k5aK0V+0KdDT+H6fbD5Qpzkn2cJEfYwER7XOPLsXnJ1Dh302ACnDMfC2mDaanbNGf0fYh/PI+w8B4B65/Ry9U2QQE+nZ1PZygU8JKaX7x1DatR0xu6d5SFFAQ+hyknQoGowPscEzmM9n9BDpC79SMmjfvQH+KlYAL5EnsZqTFMGX5JZS1Ly7ZRbr5A2bBVMivV0T/Ai5S31GyNh1aYBlkpB5er83uDihhdVAQw5TTSbdgT1LPXeuJ84Jg3fcldJFOgKmVY83LTegrN8F251juxExYFAdXjnKv5c1ERX/jZOg3mZnh+AvDk20Yd4HdplPtT2Pk0MC+7jx2NnsJxyddMvJqYRxorP7HXhaIWVBww/yRWZPcdpO+XfSLg2SCU5wicG5f2+N18HUMkLI9Cor4vKW5L/P0cbjsKa3irg+Ow3aL+SeR2gMknBNfkMZlZuXx6yzzdS/E8TvjrR6TM+M8dxjs1j+hQEvvEIckUczxiaFLHeRInElUYB7osneqwiz/0elVlRPi9zUYsyzGLilUvj/62a5yIEuFmjP59xi6wzl8qNUylav3CCOGi9A== IronPort-HdrOrdr: A9a23:kC50MajHjCrR/nL6ql6SZt9JT3BQXuYji2hC6mlwRA09TyX4rbHLoB1/73LJYVkqNk3I5urrBEDtexLhHP1OkOws1NWZLWrbUQKTRekM0WKI+UyDJ8SRzI5g/JYlW61/Jfm1NlJikPv9iTPSL/8QhPWB74Ck7N2z80tQ X-IronPort-AV: E=Sophos;i="5.88,333,1635177600"; d="scan'208";a="123488427" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 12 Apr 2022 10:22:31 +0800 Received: from G08CNEXMBPEKD05.g08.fujitsu.local (unknown [10.167.33.204]) by cn.fujitsu.com (Postfix) with ESMTP id 72EE44D16FCF; Tue, 12 Apr 2022 10:22:28 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD05.g08.fujitsu.local (10.167.33.204) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 12 Apr 2022 10:22:29 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 12 Apr 2022 10:22:26 +0800 From: Li Zhijian To: , , CC: , Li Zhijian Subject: [PATCH 2/3] RDMA/rxe: Update wqe_index for each wqe error completion Date: Tue, 12 Apr 2022 10:29:02 +0800 Message-ID: <20220412022903.574238-2-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220412022903.574238-1-lizhijian@fujitsu.com> References: <20220412022903.574238-1-lizhijian@fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 72EE44D16FCF.A0068 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Previously, if user space keep sending abnormal wqe, qeueu.prod will keep increasing while queue.index doesn't. Once queue.index==qeueu.prod in next round, req_next_wqe() will treat queue is empty. In such case, no new completion would be generated. Update wqe_index for each wqe completion so that req_next_wqe() can get next wqe properly. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_req.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index bf7493bab9b9..c369bebaf02e 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -760,6 +760,8 @@ int rxe_requester(void *arg) if (ah) rxe_put(ah); err: + /* update wqe_index for each wqe completion */ + qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; __rxe_do_task(&qp->comp.task); From patchwork Tue Apr 12 02:29:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12809966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B07BDC433FE for ; Tue, 12 Apr 2022 02:22:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229921AbiDLCYv (ORCPT ); Mon, 11 Apr 2022 22:24:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242867AbiDLCYu (ORCPT ); Mon, 11 Apr 2022 22:24:50 -0400 Received: from heian.cn.fujitsu.com (mail.cn.fujitsu.com [183.91.158.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 348042E6AC; Mon, 11 Apr 2022 19:22:34 -0700 (PDT) IronPort-Data: A9a23:ZrcepaxftCmDi76JAIt6t+fuxCrEfRIJ4+MujC/XYbTApD0g3mZWn2sdDGGDOfuKZzD8f4hwbN7ipEoG6sDXzoViHQtv/xmBbVoQ95OdWo7xwmQcns+qBpSaChohtq3yU/GYRCwPZiKa9kfF3oTJ9yEmj/nSHuOkUYYoBwgqLeNaYHZ44f5cs75h6mJYqYDR7zKl4bsekeWGULOW82Ic3lYv1k62gEgHUMIeF98vlgdWifhj5DcynpSOZX4VDfnZw3DQGuG4EgMmLtsvwo1V/kuBl/ssItij1LjmcEwWWaOUNg+L4pZUc/H6xEEc+WppieBmXBYfQR4/ZzGhm9FjyNRPtJW2YQk0PKzQg/lbWB5de817FfQco+OXcSTl4KR/yGWDKRMA2c5GHlA0L5waoL4vWUlB8PUZLHYGaRXrr+a3xq+rD+phnMIuKOH1M44F/HJt1zfUCbAhW5+ra6HL48JImS08g8lmA/nTfYwaZCBpYRCGZAdAUn8VB50WjualnnS5eDQwlb4/jcLb+ECKlEoojuera4GTJ7S3qQxuth7wjgr7E67RXnn27OCi9Ac= IronPort-HdrOrdr: A9a23:6Vz3O6O+rII0NMBcTv2jsMiBIKoaSvp037BL7TEUdfUxSKGlfq+V8sjzqiWftN98YhAdcLO7Scy9qBHnhP1ICOAqVN/MYOCMghrLEGgN1+vf6gylMyj/28oY7q14bpV5YeeaMXFKyer8/ym0euxN/OW6 X-IronPort-AV: E=Sophos;i="5.88,333,1635177600"; d="scan'208";a="123488429" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 12 Apr 2022 10:22:31 +0800 Received: from G08CNEXMBPEKD06.g08.fujitsu.local (unknown [10.167.33.206]) by cn.fujitsu.com (Postfix) with ESMTP id DA88E4D17165; Tue, 12 Apr 2022 10:22:28 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD06.g08.fujitsu.local (10.167.33.206) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 12 Apr 2022 10:22:27 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 12 Apr 2022 10:22:27 +0800 From: Li Zhijian To: , , CC: , Li Zhijian Subject: [PATCH 3/3] RDMA/rxe: Generate error completion for error requester state Date: Tue, 12 Apr 2022 10:29:03 +0800 Message-ID: <20220412022903.574238-3-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220412022903.574238-1-lizhijian@fujitsu.com> References: <20220412022903.574238-1-lizhijian@fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: DA88E4D17165.AE8A7 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org SoftRoCE always returns success when user space is posting a new wqe where it usually just euqueues a wqe. Once requester state becomes QP_STATE_ERROR, we should generate error completion for all subsequent wqe. So user is able to poll the completion event to check if the former wqe is handled correctly. Here we check QP_STATE_ERROR after req_next_wqe() so that the completion can associate with its wqe. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_req.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index c369bebaf02e..9b35f6528d73 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -611,7 +611,7 @@ int rxe_requester(void *arg) rxe_get(qp); next_wqe: - if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR)) + if (unlikely(!qp->valid)) goto exit; if (unlikely(qp->req.state == QP_STATE_RESET)) { @@ -633,6 +633,14 @@ int rxe_requester(void *arg) if (unlikely(!wqe)) goto exit; + if (qp->req.state == QP_STATE_ERROR) { + /* + * Generate an error completion so that user space is able to + * poll this completion. + */ + goto err; + } + if (wqe->mask & WR_LOCAL_OP_MASK) { ret = rxe_do_local_ops(qp, wqe); if (unlikely(ret))