From patchwork Fri Oct 27 05:41:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13438113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0BC1C25B48 for ; Fri, 27 Oct 2023 05:42:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232148AbjJ0FmW (ORCPT ); Fri, 27 Oct 2023 01:42:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229590AbjJ0FmV (ORCPT ); Fri, 27 Oct 2023 01:42:21 -0400 Received: from esa1.hc1455-7.c3s2.iphmx.com (esa1.hc1455-7.c3s2.iphmx.com [207.54.90.47]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FAAC1A7; Thu, 26 Oct 2023 22:42:19 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6600,9927,10875"; a="137517268" X-IronPort-AV: E=Sophos;i="6.03,255,1694703600"; d="scan'208";a="137517268" Received: from unknown (HELO yto-r2.gw.nic.fujitsu.com) ([218.44.52.218]) by esa1.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2023 14:42:11 +0900 Received: from yto-m4.gw.nic.fujitsu.com (yto-nat-yto-m4.gw.nic.fujitsu.com [192.168.83.67]) by yto-r2.gw.nic.fujitsu.com (Postfix) with ESMTP id 7D2C1D6186; Fri, 27 Oct 2023 14:42:08 +0900 (JST) Received: from kws-ab3.gw.nic.fujitsu.com (kws-ab3.gw.nic.fujitsu.com [192.51.206.21]) by yto-m4.gw.nic.fujitsu.com (Postfix) with ESMTP id AF8D5D3F3B; Fri, 27 Oct 2023 14:42:07 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by kws-ab3.gw.nic.fujitsu.com (Postfix) with ESMTP id 43BD72007471E; Fri, 27 Oct 2023 14:42:07 +0900 (JST) Received: from FNSTPC.g08.fujitsu.local (unknown [10.167.226.45]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 682921A0070; Fri, 27 Oct 2023 13:42:06 +0800 (CST) From: Li Zhijian To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org Cc: linux-kernel@vger.kernel.org, rpearsonhpe@gmail.com, matsuda-daisuke@fujitsu.com, bvanassche@acm.org, Li Zhijian Subject: [PATCH RFC 1/2] RDMA/rxe: don't allow registering !PAGE_SIZE mr Date: Fri, 27 Oct 2023 13:41:53 +0800 Message-ID: <20231027054154.2935054-1-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-TM-AS-Product-Ver: IMSS-9.1.0.1417-9.0.0.1002-27960.005 X-TM-AS-User-Approved-Sender: Yes X-TMASE-Version: IMSS-9.1.0.1417-9.0.1002-27960.005 X-TMASE-Result: 10--2.709700-10.000000 X-TMASE-MatchedRID: 362T4rQL1OeMCPk/lvFUOp13aYUE0ivy2FA7wK9mP9dHpEd1UrzmFcyY +KNsy3oNEtuR7TIdaKXplZ0MI+xAXtAy2LXTI7g3EXjPIvKd74BMkOX0UoduuZBz1ZAU7t9T5s7 uzWaKyAzi8zVgXoAltr8hWd4lAsFlC24oEZ6SpSmcfuxsiY4QFAusc5H1417CXea3jCa2pjxA+d r4vevoQ6AYfsZo2EAGB3zb8qgE6aUTkfXrC6tp/rLvgdRTF0C9JtAM880Fc9lxfsHyeP4fXJsNE GpLafrrLM/nEDLP056e+TDiyH/49wxfkLAfkNNSaAZk0sEcY14= X-TMASE-SNAP-Result: 1.821001.0001-0-1-22:0,33:0,34:0-0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org mr->page_list only encodes *page without page offset, when page_size != PAGE_SIZE, we cannot restore the address with a wrong page_offset. Note that this patch will break some ULPs that try to register 4K MR when PAGE_SIZE is not 4K. SRP and nvme over RXE is known to be impacted. Signed-off-by: Li Zhijian Reviewed-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index f54042e9aeb2..61a136ea1d91 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -234,6 +234,12 @@ int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sgl, struct rxe_mr *mr = to_rmr(ibmr); unsigned int page_size = mr_page_size(mr); + if (page_size != PAGE_SIZE) { + rxe_info_mr(mr, "Unsupported MR with page_size %u, expect %lu\n", + page_size, PAGE_SIZE); + return -EOPNOTSUPP; + } + mr->nbuf = 0; mr->page_shift = ilog2(page_size); mr->page_mask = ~((u64)page_size - 1); From patchwork Fri Oct 27 05:41:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 13438112 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB60FC25B47 for ; Fri, 27 Oct 2023 05:42:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230178AbjJ0FmS (ORCPT ); Fri, 27 Oct 2023 01:42:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229590AbjJ0FmS (ORCPT ); Fri, 27 Oct 2023 01:42:18 -0400 Received: from esa12.hc1455-7.c3s2.iphmx.com (esa12.hc1455-7.c3s2.iphmx.com [139.138.37.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 810171A7; Thu, 26 Oct 2023 22:42:13 -0700 (PDT) X-IronPort-AV: E=McAfee;i="6600,9927,10875"; a="117100430" X-IronPort-AV: E=Sophos;i="6.03,255,1694703600"; d="scan'208";a="117100430" Received: from unknown (HELO yto-r3.gw.nic.fujitsu.com) ([218.44.52.219]) by esa12.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2023 14:42:10 +0900 Received: from yto-m2.gw.nic.fujitsu.com (yto-nat-yto-m2.gw.nic.fujitsu.com [192.168.83.65]) by yto-r3.gw.nic.fujitsu.com (Postfix) with ESMTP id E15D1D5026; Fri, 27 Oct 2023 14:42:08 +0900 (JST) Received: from kws-ab4.gw.nic.fujitsu.com (kws-ab4.gw.nic.fujitsu.com [192.51.206.22]) by yto-m2.gw.nic.fujitsu.com (Postfix) with ESMTP id 290E9DA222; Fri, 27 Oct 2023 14:42:08 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by kws-ab4.gw.nic.fujitsu.com (Postfix) with ESMTP id B236370B0B; Fri, 27 Oct 2023 14:42:07 +0900 (JST) Received: from FNSTPC.g08.fujitsu.local (unknown [10.167.226.45]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 2475A1A0073; Fri, 27 Oct 2023 13:42:07 +0800 (CST) From: Li Zhijian To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org Cc: linux-kernel@vger.kernel.org, rpearsonhpe@gmail.com, matsuda-daisuke@fujitsu.com, bvanassche@acm.org, Li Zhijian Subject: [PATCH RFC 2/2] RDMA/rxe: set RXE_PAGE_SIZE_CAP to PAGE_SIZE Date: Fri, 27 Oct 2023 13:41:54 +0800 Message-ID: <20231027054154.2935054-2-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231027054154.2935054-1-lizhijian@fujitsu.com> References: <20231027054154.2935054-1-lizhijian@fujitsu.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-TM-AS-Product-Ver: IMSS-9.1.0.1417-9.0.0.1002-27960.005 X-TM-AS-User-Approved-Sender: Yes X-TMASE-Version: IMSS-9.1.0.1417-9.0.1002-27960.005 X-TMASE-Result: 10--5.753700-10.000000 X-TMASE-MatchedRID: S8AxUE03d7BTorztX3wKRAmyVrMCuJ9SwTlc9CcHMZerwqxtE531VCzy bVqWyY2ND+LZhHM9RwrRpRbL7Yjxh7QIlEvYJcRNfa4B0jQGGeoogaQhRNNEvpsoi2XrUn/J8m+ hzBStanvIM9mETCO70yAHAopEd76vN6erXqkXaWNUTRnbdqHqdCXwULxDsmJEWzsYPewyFk5QGf k+LPMB6g== X-TMASE-SNAP-Result: 1.821001.0001-0-1-22:0,33:0,34:0-0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org RXE_PAGE_SIZE_CAP means the MR page size supported by RXE. However in current RXE implementation, only PAGE_SIZE MR works well. So change it to PAGE_SIZE only. ULPs such as SRP calculating the page size according to this attribute get worked again with this change. Signed-off-by: Li Zhijian Reviewed-by: Bart Van Assche Acked-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_param.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index d2f57ead78ad..b1cf1e1c0ce1 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -38,7 +38,7 @@ static inline enum ib_mtu eth_mtu_int_to_enum(int mtu) /* default/initial rxe device parameter settings */ enum rxe_device_param { RXE_MAX_MR_SIZE = -1ull, - RXE_PAGE_SIZE_CAP = 0xfffff000, + RXE_PAGE_SIZE_CAP = PAGE_SIZE, RXE_MAX_QP_WR = DEFAULT_MAX_VALUE, RXE_DEVICE_CAP_FLAGS = IB_DEVICE_BAD_PKEY_CNTR | IB_DEVICE_BAD_QKEY_CNTR