From patchwork Wed Jul 20 08:56:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12923689 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE171C43334 for ; Wed, 20 Jul 2022 08:56:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229523AbiGTI4i (ORCPT ); Wed, 20 Jul 2022 04:56:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbiGTI4h (ORCPT ); Wed, 20 Jul 2022 04:56:37 -0400 Received: from mail1.bemta34.messagelabs.com (mail1.bemta34.messagelabs.com [195.245.231.4]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE8AD20F73 for ; Wed, 20 Jul 2022 01:56:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1658307395; i=@fujitsu.com; bh=WWvTr9oZhgTy9NzyKlfCg+FUhx6T8/rIpXev7udUMt4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=xKKgWMEKkyMMplSOBTwZVfwxTmAsJj2QAHZnJXFCaCDPskPRamzTPPtEPehPdQejp MyhB/CaIEHSGYZ2J9xnrnJkxfsWxx7fH+5F4N4ewHFNf6XH2q9Zs1LYMKyKinqUZim YhhL1qMVwYY4MaNa2pHS+ubD8r2lgOkidbVCRfzZUN1oH77udyYk1fqoZMS9It0+t/ uwwN66OC19mpwZu6/gOVcopb+ZgNZ8HmYmYQiJJnOnp489l1PJYcvSh04bvfeiMWKk UyOYcgZpQoKEWAtvafIO6c9QDyRIWVkGbMX8puvnKw3GFg6PjFadSGL7M6z82yZirY rdleXdjDam4zA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmphleJIrShJLcpLzFFi42Kxs+FI1HU6fD3 JYNFLKYtjKzezWCz48JbFYuaME4wWU34tZbZ4dqiXxeLL1GnMFsennGN3YPfYOesuu8emVZ1s HjsfWnosbJjK7PHx6S0Wj8+b5Dy2fr7NEsAexZqZl5RfkcCaserPB/aC6+wVz9/8ZmlgvM3Wx cjFISSwhVHiX+NHRghnOZPE14k/gDKcQM4+Ron9e3xAbDYBDYl7LTcZQWwRgeuMEo+2yILYzA K+EhfPH2YCsYUFEiS2/DjEDmKzCKhKrH/8kRXE5hVwkdj9fj8ziC0hoCAx5eF7MJtTwFViytu 9QDYH0C4XiS+vUyHKBSVOznzCAjFeQuLgixdQrYoSRzr/skDYFRKvD1+CiqtJXD23iXkCo+As JO2zkLQvYGRaxWidVJSZnlGSm5iZo2toYKBraGiqa2wEZJroJVbpJuqlluqWpxaX6BrpJZYX6 6UWF+sVV+Ym56To5aWWbGIERk5KsULGDsbvK3/qHWKU5GBSEuW1WXw9SYgvKT+lMiOxOCO+qD QntfgQowwHh5IEb/VBoJxgUWp6akVaZg4wimHSEhw8SiK8uiBp3uKCxNzizHSI1ClGRSlx3kc gCQGQREZpHlwbLHFcYpSVEuZlZGBgEOIpSC3KzSxBlX/FKM7BqCTM++sA0BSezLwSuOmvgBYz AS2e5HQFZHFJIkJKqoHp6Plnl6dVChWI73c98fNHtcmFNsX37fafd8w/tGChzMy7OvJfI//1e Og2FhT0fJgZKqdmsZ+LJTLL4drjxY4GU52/np8jq/5RvW1peafSE22VrDWH6nauy4xtZY/61/ a//cwhc5X8j3MCPhzkkyta8PPmdT+VA4k29wV+bpN4MoP3aPrd0OMiIrqn1z3wZmO69ODK1T0 R5fdb6iZcdr3v4lfDOPvN93DpzYGJa6plg7l/XftR+m/dg0lqB35veCpmx1DEaHmR+9Pf1xuP s35xfeUieeOGw3zdsu93A5pONkQE8E6Yf9tZnTH835VdAskVF7gvss41f6ndL8U0w5L18A7Hq fLTdkzVbVR++CrlhRJLcUaioRZzUXEiAAgfxb6XAwAA X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-5.tower-571.messagelabs.com!1658307394!160602!1 X-Originating-IP: [62.60.8.97] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 4467 invoked from network); 20 Jul 2022 08:56:34 -0000 Received: from unknown (HELO n03ukasimr01.n03.fujitsu.local) (62.60.8.97) by server-5.tower-571.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Jul 2022 08:56:34 -0000 Received: from n03ukasimr01.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTP id 19C79100191; Wed, 20 Jul 2022 09:56:34 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTPS id 0CF32100190; Wed, 20 Jul 2022 09:56:34 +0100 (BST) Received: from centos-smtp.localdomain (10.167.226.45) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 20 Jul 2022 09:56:30 +0100 From: Li Zhijian To: Yanjun Zhu , Jason Gunthorpe , "Haakon Bugge" , , Bob Pearson , Leon Romanovsky CC: Cheng Xu , Li Zhijian Subject: [PATCH RESEND for-next v6 1/4] RDMA/rxe: Update wqe_index for each wqe error completion Date: Wed, 20 Jul 2022 04:56:05 -0400 Message-ID: <1658307368-1851-2-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> References: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.45] X-ClientProxiedBy: G08CNEXCHPEKD08.g08.fujitsu.local (10.167.33.83) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Previously, if user space keeps sending abnormal wqe, queue.index will keep increasing while qp->req.wqe_index doesn't. Once qp->req.wqe_index==queue.index in next round, req_next_wqe() will treat queue as empty. In such case, no new completion would be generated. Update wqe_index for each wqe completion so that req_next_wqe() can get next wqe properly. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_req.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index dec4ddaca70f..c187deeb6e6b 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -760,6 +760,8 @@ int rxe_requester(void *arg) if (ah) rxe_put(ah); err: + /* update wqe_index for each wqe completion */ + qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; __rxe_do_task(&qp->comp.task); From patchwork Wed Jul 20 08:56:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12923690 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BE85C43334 for ; Wed, 20 Jul 2022 08:56:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229561AbiGTI4o (ORCPT ); Wed, 20 Jul 2022 04:56:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229539AbiGTI4m (ORCPT ); Wed, 20 Jul 2022 04:56:42 -0400 Received: from mail1.bemta37.messagelabs.com (mail1.bemta37.messagelabs.com [85.158.142.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A69367CB3 for ; Wed, 20 Jul 2022 01:56:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1658307399; i=@fujitsu.com; bh=zB2ocOaaMAzpH4otIshXdWK3XaHkkri7AmHyrGMo6Fs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TJBJLIKqKqniwgdO1pb2oEaT3klpIMeqUWpF/mmY1S0h9q2D4CePDEO8gO7PrsVay YsnA3FoCN/zUcor2JHo1gWuihNf7JXWeg9wZBVepdIdHxYZ6X8dIQt8yFUbDHPhe26 p0k8P2ms+PuEevhrvwpZTE1rfA0IHJBLH6FOpyhOeFojj7D1W8I/Db8JhLe5n9LtEL lXedUzlF3sPuw+0Wbk3IbeXpcEug6YSAgmiMTKkK7QJQoAEi338GmEyZOh02T+22Bt kIYnEh63a0/bGFDiag6/Pkuzctglb0wApUJTJ5RNIkW92VZLb9nDi/BRU+FJryny8a jy+OGksyomQxA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprAKsWRWlGSWpSXmKPExsViZ8OxWdf98PU kg1l72CyOrdzMYrHgw1sWi5kzTjBaTPm1lNni2aFeFosvU6cxWxyfco7dgd1j56y77B6bVnWy eex8aOmxsGEqs8fHp7dYPD5vkvPY+vk2SwB7FGtmXlJ+RQJrxvfV8QXt3BWbv3UyNzAu5uxi5 OIQEtjIKLFy5V2mLkZOIGcxk0TjK3uIxD5GiXnTD7CDJNgENCTutdxkBLFFBK4zSjzaIgtiMw v4Slw8fxisWVggQ+Lt6rUsIDaLgKrEyZY5bCA2r4CLxJx9t8B6JQQUJKY8fM8MYnMKuEpMebs XyOYAWuYi8eV1KkS5oMTJmU9YIMZLSBx88YIZolVR4kjnXxYIu0Li9eFLUHE1iavnNjFPYBSc haR9FpL2BYxMqxhtkooy0zNKchMzc3QNDQx0DQ1NgbShrqGpXmKVbqJeaqluXn5RSYauoV5ie bFeanGxXnFlbnJOil5easkmRmDkpBSn8u9gvLHvl94hRkkOJiVRXpvF15OE+JLyUyozEosz4o tKc1KLDzHKcHAoSfBWHwTKCRalpqdWpGXmAKMYJi3BwaMkwqsLkuYtLkjMLc5Mh0idYtTlOL9 z/15mIZa8/LxUKXHeRyBFAiBFGaV5cCNgCeUSo6yUMC8jAwODEE9BalFuZgmq/CtGcQ5GJWHe XweApvBk5pXAbXoFdAQT0BGTnK6AHFGSiJCSamCKVZ62akl21UVZ6VkK69/qLQrezHOSad8+s +4bd2qD2wXlfq2KXfuKYVuX8Ia2OzX3vu7b1BRglCMXzyG35sCOCiPGRTJGQhlSi3WEzJUWed 21yQt0Zyv9e2v1heAJN+reHu3+rXfgTJ6U831li7gkA9+6fxKcvx7dX7puU6jUTMGwDz6fpwf oCZgys1/VV27WP7KzZtrbogq/SS7s2TP+OZ2I8tc98TRPRSFofp7tsWkPBPmjrus6X5yb3Cza 851dw8TVz1dupY1UbVT2ju3z211PV+1t4DOb8dhs383SDNsN6fz6y052B8sLhCyQalkpeMD+Y ekR7m8aH3WLL/Af/MBgzhYQkfnOJku8T4mlOCPRUIu5qDgRAGbkOP2jAwAA X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-9.tower-732.messagelabs.com!1658307398!318012!1 X-Originating-IP: [62.60.8.179] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 8051 invoked from network); 20 Jul 2022 08:56:39 -0000 Received: from unknown (HELO n03ukasimr04.n03.fujitsu.local) (62.60.8.179) by server-9.tower-732.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Jul 2022 08:56:39 -0000 Received: from n03ukasimr04.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr04.n03.fujitsu.local (Postfix) with ESMTP id 6FD95150; Wed, 20 Jul 2022 09:56:38 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr04.n03.fujitsu.local (Postfix) with ESMTPS id 61C8C7B; Wed, 20 Jul 2022 09:56:38 +0100 (BST) Received: from centos-smtp.localdomain (10.167.226.45) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 20 Jul 2022 09:56:34 +0100 From: Li Zhijian To: Yanjun Zhu , Jason Gunthorpe , "Haakon Bugge" , , Bob Pearson , Leon Romanovsky CC: Cheng Xu , Li Zhijian Subject: [PATCH RESEND for-next v6 2/4] RDMA/rxe: Generate error completion for error requester QP state Date: Wed, 20 Jul 2022 04:56:06 -0400 Message-ID: <1658307368-1851-3-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> References: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.45] X-ClientProxiedBy: G08CNEXCHPEKD08.g08.fujitsu.local (10.167.33.83) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org As per IBTA specification, all subsequent WQEs while QP is in error state should be completed with a flush error. Here we check QP_STATE_ERROR after req_next_wqe() so that rxe_completer() has chance to be called where it will set CQ state to FLUSH ERROR and the completion can associate with its WQE. Signed-off-by: Li Zhijian --- V5: parentheses issue # Cheng Xu V4: check QP ERROR before QP RESET # Bob V3: unlikely() optimization # Cheng Xu update commit log # Haakon Bugge --- drivers/infiniband/sw/rxe/rxe_req.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index c187deeb6e6b..cbb2ce2d7b50 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -611,9 +611,20 @@ int rxe_requester(void *arg) return -EAGAIN; next_wqe: - if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR)) + if (unlikely(!qp->valid)) goto exit; + if (unlikely(qp->req.state == QP_STATE_ERROR)) { + wqe = req_next_wqe(qp); + if (wqe) + /* + * Generate an error completion for error qp state + */ + goto err; + else + goto exit; + } + if (unlikely(qp->req.state == QP_STATE_RESET)) { qp->req.wqe_index = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); From patchwork Wed Jul 20 08:56:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12923691 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBA0DCCA480 for ; Wed, 20 Jul 2022 08:56:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229744AbiGTI4r (ORCPT ); Wed, 20 Jul 2022 04:56:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbiGTI4q (ORCPT ); Wed, 20 Jul 2022 04:56:46 -0400 Received: from mail1.bemta32.messagelabs.com (mail1.bemta32.messagelabs.com [195.245.230.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EB9267CA5 for ; Wed, 20 Jul 2022 01:56:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1658307402; i=@fujitsu.com; bh=GwK0f5+nazGfLHGdhdDbn1nqYies5I7If8cq+o9I8XM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=y26ov6UWL0+TdHi3PBC27luxacdsTv81D1id+jTQwPSCAjQGRNIT0UoR54tJfTT6E iMf4O270nayNk/El5UKDfcbB5DXWAbyvm70uxcrETZ3Kp+pn0vbhpFtW4ULPC1KABI iG/bV6dMxUPTnNAZC1fYFAt+aHtW38w0p5PmDklvHEbTOs4xxtn9dr67lyeZhpJPmm n9ApMvlu5oUXH88i+MG7Q77pMcwk7c4dyGsjKV0pb/2pw+HbCQndLMFCDgRoc90oOR aqBr1CkPLvxbABSWsxpyCb+thCXCIrhmtuJIuSusdafKRqF0A+WQe3il4nlxH9TFZy o4AmhUkKSz0LQ== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupnleJIrShJLcpLzFFi42Kxs+GYpOt1+Hq Swb/ljBbHVm5msVjw4S2LxcwZJxgtpvxaymzx7FAvi8WXqdOYLY5POcfuwO6xc9Zddo9NqzrZ PHY+tPRY2DCV2ePj01ssHp83yXls/XybJYA9ijUzLym/IoE14+/UvSwFV2QrHn/azN7A2CfZx cjFISSwhVHizYuNLBDOciaJ9/0fmSCcfYwST1sms3UxcnKwCWhI3Gu5yQhiiwhcZZQ4el4HxG YW0JRY8WE/E4gtLBAjcexYP1ANBweLgKrEl1nFIGFeAReJrsP/2EFsCQEFiSkP3zOD2JwCrhJ T3u5lBikXAqr58joVolxQ4uTMJywQ0yUkDr54wQzRqihxpPMvC4RdIfH68CWouJrE1XObmCcw Cs5C0j4LSfsCRqZVjFZJRZnpGSW5iZk5uoYGBrqGhqa6QNLURC+xSjdRL7VUtzy1uETXUC+xv FgvtbhYr7gyNzknRS8vtWQTIzBuUooZZu1gnNX3U+8QoyQHk5Ior83i60lCfEn5KZUZicUZ8U WlOanFhxhlODiUJHirDwLlBItS01Mr0jJzgDEMk5bg4FES4dUFSfMWFyTmFmemQ6ROMepyTJ3 9bz+zEEtefl6qlDjvI5AiAZCijNI8uBGwdHKJUVZKmJeRgYFBiKcgtSg3swRV/hWjOAejkjDv rwNAU3gy80rgNr0COoIJ6IhJTldAjihJREhJNTCdmGmXI/Zgyo0PEf8bJDau/a5a7CTz6uxH3 4tnV//eeOzpvn+1eSuvBiSLrZv2+czPQ6WXdF+VPQuaePvg5QOiW79/+6a8U+HVzEv7Y9gNpm +Y+cXGk0vEbtPh0iqNAvYim8O6exc/bGjjUc3h7qpNXlm91bRoSc0Edj+mutumvvwlWvrSfUv e3D/GK9E1e95VnevtpxNF5523NK50rPmy/vLfv6dubEtSlsmNP3BkXp3XnPsrhMJ2/fPkDOBw PMzqFeDC69GkUPBPR76Bn5f5Z5tEVcaONZZ3i1hkbkb8T90V/Gp5cVhB//4V2Symen/19dlCn f5JX8z5u+vOpr/7O84UuV7kFFeLWTnj5QxvJZbijERDLeai4kQAuCC56aIDAAA= X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-9.tower-587.messagelabs.com!1658307401!46125!1 X-Originating-IP: [62.60.8.146] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 29843 invoked from network); 20 Jul 2022 08:56:41 -0000 Received: from unknown (HELO n03ukasimr02.n03.fujitsu.local) (62.60.8.146) by server-9.tower-587.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Jul 2022 08:56:41 -0000 Received: from n03ukasimr02.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr02.n03.fujitsu.local (Postfix) with ESMTP id 96E071000C2; Wed, 20 Jul 2022 09:56:41 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr02.n03.fujitsu.local (Postfix) with ESMTPS id 8999A100077; Wed, 20 Jul 2022 09:56:41 +0100 (BST) Received: from centos-smtp.localdomain (10.167.226.45) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 20 Jul 2022 09:56:38 +0100 From: Li Zhijian To: Yanjun Zhu , Jason Gunthorpe , Haakon Bugge , , Bob Pearson , Leon Romanovsky CC: Cheng Xu Subject: [PATCH RESEND for-next v6 3/4] RDMA/rxe: Split qp state for requester and completer Date: Wed, 20 Jul 2022 04:56:07 -0400 Message-ID: <1658307368-1851-4-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> References: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.45] X-ClientProxiedBy: G08CNEXCHPEKD08.g08.fujitsu.local (10.167.33.83) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Bob Pearson Currently the requester can continue to process send wqes after an local qp operation error is detected because the setting of the qp state to the error state is deferred until later. This patch splits the qp state for the completer and requester into two separate states and sets qp->req.state = QP_STATE_ERROR as soon as the error is detected before another wqe can be executed. Signed-off-by: Bob Pearson --- V4: new patch --- drivers/infiniband/sw/rxe/rxe_comp.c | 6 +++--- drivers/infiniband/sw/rxe/rxe_qp.c | 5 +++++ drivers/infiniband/sw/rxe/rxe_req.c | 1 + drivers/infiniband/sw/rxe/rxe_verbs.h | 1 + 4 files changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index da3a398053b8..0b68630a3e49 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -565,10 +565,10 @@ int rxe_completer(void *arg) if (!rxe_get(qp)) return -EAGAIN; - if (!qp->valid || qp->req.state == QP_STATE_ERROR || - qp->req.state == QP_STATE_RESET) { + if (!qp->valid || qp->comp.state == QP_STATE_ERROR || + qp->comp.state == QP_STATE_RESET) { rxe_drain_resp_pkts(qp, qp->valid && - qp->req.state == QP_STATE_ERROR); + qp->comp.state == QP_STATE_ERROR); ret = -EAGAIN; goto done; } diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index f69e3ff095f8..becdcfdfc49b 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -228,6 +228,7 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, QUEUE_TYPE_FROM_CLIENT); qp->req.state = QP_STATE_RESET; + qp->comp.state = QP_STATE_RESET; qp->req.opcode = -1; qp->comp.opcode = -1; @@ -488,6 +489,7 @@ static void rxe_qp_reset(struct rxe_qp *qp) /* move qp to the reset state */ qp->req.state = QP_STATE_RESET; + qp->comp.state = QP_STATE_RESET; qp->resp.state = QP_STATE_RESET; /* let state machines reset themselves drain work and packet queues @@ -550,6 +552,7 @@ void rxe_qp_error(struct rxe_qp *qp) { qp->req.state = QP_STATE_ERROR; qp->resp.state = QP_STATE_ERROR; + qp->comp.state = QP_STATE_ERROR; qp->attr.qp_state = IB_QPS_ERR; /* drain work and packet queues */ @@ -687,6 +690,7 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, pr_debug("qp#%d state -> INIT\n", qp_num(qp)); qp->req.state = QP_STATE_INIT; qp->resp.state = QP_STATE_INIT; + qp->comp.state = QP_STATE_INIT; break; case IB_QPS_RTR: @@ -697,6 +701,7 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, case IB_QPS_RTS: pr_debug("qp#%d state -> RTS\n", qp_num(qp)); qp->req.state = QP_STATE_READY; + qp->comp.state = QP_STATE_READY; break; case IB_QPS_SQD: diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index cbb2ce2d7b50..35a249727435 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -774,6 +774,7 @@ int rxe_requester(void *arg) /* update wqe_index for each wqe completion */ qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; + qp->req.state = QP_STATE_ERROR; __rxe_do_task(&qp->comp.task); exit: diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 628e40c1714b..ab899aee72e4 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -128,6 +128,7 @@ struct rxe_req_info { }; struct rxe_comp_info { + enum rxe_qp_state state; u32 psn; int opcode; int timeout; From patchwork Wed Jul 20 08:56:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12923692 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51389C433EF for ; Wed, 20 Jul 2022 08:56:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229449AbiGTI4x (ORCPT ); Wed, 20 Jul 2022 04:56:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229832AbiGTI4t (ORCPT ); Wed, 20 Jul 2022 04:56:49 -0400 Received: from mail1.bemta37.messagelabs.com (mail1.bemta37.messagelabs.com [85.158.142.2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13BBE6BC27 for ; Wed, 20 Jul 2022 01:56:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1658307406; i=@fujitsu.com; bh=exwYpHRudrNXVQdZIG8MptKqNawZdKpVdTio4sV4GlA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=G2D+ar4Uf8Y0x28dWv2DMBcGq9VAbRqa57lxsL8Eghi6Z8pDlEoGjM8QXc3LdQUwr P/mEmFhUzEsPeMFOq3re/LolGGcSgAuqrnNQacsqx/pC1cyfHdhHbvSOmN+3CPQZrj zmIaDQjq88E/WeR1DToIIM0IT5YQPc1hVfBeSRk+s2ctRAKhqq+/edufAps78eW7Wt xgIPSp+di6eBCVuFYZkgNk4SEc6AjY9KkPPt+qwKb//nOWuAG415qSpLVDjzAXhI5X uG6TeeV+PgRP+YJFeOpcHTRWD9vtxi6diJKwhjWro5+igPZP1xq8Dx3tEHZ6M9u+3u KQ4ySVEhfQABw== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprIKsWRWlGSWpSXmKPExsViZ8ORqOt7+Hq SQcM/TYtjKzezWCz48JbFYuaME4wWU34tZbZ4dqiXxeLL1GnMFsennGN3YPfYOesuu8emVZ1s HjsfWnosbJjK7PHx6S0Wj8+b5Dy2fr7NEsAexZqZl5RfkcCa8XfPLuaCT6wVDxe9Ym9g/MbSx cjFISSwhVHi1PR7jBDOciaJ97MmMEE4+xglvt5Yzt7FyMnBJqAhca/lJiOILSJwnVHi0RZZEJ tZwFfi4vnDQA0cHMICrhKL5lmChFkEVCWur9jCAmLzCrhINH/eAWZLCChITHn4nhnE5gQqn/J 2LzNIqxBQzZfXqRDlghInZz5hgZguIXHwxQtmiFZFiSOdf6HGVEi8PnwJKq4mcfXcJuYJjIKz kLTPQtK+gJFpFaNNUlFmekZJbmJmjq6hgYGuoaGprqWZrqGxkV5ilW6iXmqpbl5+UUmGrqFeY nmxXmpxsV5xZW5yTopeXmrJJkZg7KQUJ7TsYLy975feIUZJDiYlUV6bxdeThPiS8lMqMxKLM+ KLSnNSiw8xynBwKEnwVh8EygkWpaanVqRl5gDjGCYtwcGjJMKrC5LmLS5IzC3OTIdInWLU5Zg 3+99+ZiGWvPy8VClx3kcgRQIgRRmleXAjYCnlEqOslDAvIwMDgxBPQWpRbmYJqvwrRnEORiVh 3l8HgKbwZOaVwG16BXQEE9ARk5yugBxRkoiQkmpgSr293NyGi1PqrOz1P5wTEvw+M17z9XCLK GS+2ZNYW7JnabuHS1mJorl47TR7HaUJZyfrGaxgXOen5n/R7tgyt80CTxcflTwcoKu6b0bj0n juR0L91/znJb1X4uqfL9GkffSQtjpjsgnz8j/K9bzTKqwSmjq2nFnRdGXzpDvFx1z/MuhETLz 51e3Lsg1yL8NOnJ3RvN/jCvuHx8ed43fuj/z2+ZTfLfYNvu+fGxtxPKxVeDXr+0v3rrv8f778 /9yivXvuJuaN28792br9Y+zxZYuFVgexl/6dcPr5E+VKk7B1R+6+Y3t0XU5iS1O6dvyNk2VXF jLOLV5vub/X4tadI3sU6iwXLXjdO5N9jc79wo9KLMUZiYZazEXFiQCBdQHKpAMAAA== X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-7.tower-728.messagelabs.com!1658307405!45336!1 X-Originating-IP: [62.60.8.97] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15051 invoked from network); 20 Jul 2022 08:56:45 -0000 Received: from unknown (HELO n03ukasimr01.n03.fujitsu.local) (62.60.8.97) by server-7.tower-728.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Jul 2022 08:56:45 -0000 Received: from n03ukasimr01.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTP id 0C38E100190; Wed, 20 Jul 2022 09:56:45 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTPS id 00007100043; Wed, 20 Jul 2022 09:56:44 +0100 (BST) Received: from centos-smtp.localdomain (10.167.226.45) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Wed, 20 Jul 2022 09:56:41 +0100 From: Li Zhijian To: Yanjun Zhu , Jason Gunthorpe , "Haakon Bugge" , , Bob Pearson , Leon Romanovsky CC: Cheng Xu , Li Zhijian Subject: [PATCH RESEND for-next v6 4/4] RDMA/rxe: Fix typo in comment Date: Wed, 20 Jul 2022 04:56:08 -0400 Message-ID: <1658307368-1851-5-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> References: <1658307368-1851-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.45] X-ClientProxiedBy: G08CNEXCHPEKD08.g08.fujitsu.local (10.167.33.83) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Fix a spelling mistake Signed-off-by: Li Zhijian Reviewed-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_task.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c index 0c4db5bb17d7..c9b80410cd5b 100644 --- a/drivers/infiniband/sw/rxe/rxe_task.c +++ b/drivers/infiniband/sw/rxe/rxe_task.c @@ -67,7 +67,7 @@ void rxe_do_task(struct tasklet_struct *t) cont = 1; break; - /* soneone tried to run the task since the last time we called + /* someone tried to run the task since the last time we called * func, so we will call one more time regardless of the * return value */