From patchwork Wed Mar 2 13:25:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "D. Wythe" X-Patchwork-Id: 12766011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB4E0C433FE for ; Wed, 2 Mar 2022 13:25:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242179AbiCBN0A (ORCPT ); Wed, 2 Mar 2022 08:26:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240476AbiCBN0A (ORCPT ); Wed, 2 Mar 2022 08:26:00 -0500 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E7DB1D0E1; Wed, 2 Mar 2022 05:25:15 -0800 (PST) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R411e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=alibuda@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0V62tJZM_1646227512; Received: from localhost(mailfrom:alibuda@linux.alibaba.com fp:SMTPD_---0V62tJZM_1646227512) by smtp.aliyun-inc.com(127.0.0.1); Wed, 02 Mar 2022 21:25:13 +0800 From: "D. Wythe" To: kgraul@linux.ibm.com Cc: kuba@kernel.org, davem@davemloft.net, netdev@vger.kernel.org, linux-s390@vger.kernel.org, linux-rdma@vger.kernel.org Subject: [PATCH net v3 0/2] fix unexpected SMC_CLC_DECL_ERR_REGRMB error Date: Wed, 2 Mar 2022 21:25:10 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: "D. Wythe" We can easily trigger the SMC_CLC_DECL_ERR_REGRMB exception within following script: server: smc_run nginx client: smc_run ./wrk -c 2000 -t 8 -d 20 http://smc-server And we can clearly see that this error is also divided into two types: 1. 0x09990003 2. 0x05000000/0x09990003 Which has the same root causes, but the immediate causes vary. The root cause of this issues is that remove connections from link group is not synchronous with add/delete rtoken entry, which means that even the number of connections is less that SMC_RMBS_PER_LGR_MAX, it does not mean that the connection can register rtoken successfully later. In other words, the rtoken entry may released, This will cause an unexpected SMC_CLC_DECL_ERR_REGRMB to be reported, and then this SMC connections have to fallback to TCP. This patch set handles two types of SMC_CLC_DECL_ERR_REGRMB exceptions from different perspectives. Patch 1: fix the 0x05000000/0x09990003 error. Patch 2: fix the 0x09990003 error. After those patches, there is no SMC_CLC_DECL_ERR_REGRMB exceptions in my test case any more. v1 -> v2: - add bugfix patch for SMC_CLC_DECL_ERR_REGRMB cause by server side v2 -> v3: - fix incorrect mail thread D. Wythe (2): net/smc: fix unexpected SMC_CLC_DECL_ERR_REGRMB error generated by client net/smc: fix unexpected SMC_CLC_DECL_ERR_REGRMB error cause by server net/smc/smc_core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)