From patchwork Wed Jan 5 22:12:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79C19C433EF for ; Wed, 5 Jan 2022 05:49:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237554AbiAEFtu (ORCPT ); Wed, 5 Jan 2022 00:49:50 -0500 Received: from mga07.intel.com ([134.134.136.100]:27100 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236188AbiAEFtu (ORCPT ); Wed, 5 Jan 2022 00:49:50 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="305725903" X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="305725903" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 21:49:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="760688607" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by fmsmga006.fm.intel.com with ESMTP; 04 Jan 2022 21:49:48 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 1/5] RDMA/core: Calculate UDP source port based on flow label or lqpn/rqpn Date: Wed, 5 Jan 2022 17:12:33 -0500 Message-Id: <20220105221237.2659462-2-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105221237.2659462-1-yanjun.zhu@linux.dev> References: <20220105221237.2659462-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Calculate and set UDP source port based on the flow label. If flow label is not defined in GRH then calculate it based on lqpn/rqpn. Signed-off-by: Zhu Yanjun Reviewed-by: Leon Romanovsky --- include/rdma/ib_verbs.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 6e9ad656ecb7..69d883f7fb41 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -4749,6 +4749,23 @@ static inline u32 rdma_calc_flow_label(u32 lqpn, u32 rqpn) return (u32)(v & IB_GRH_FLOWLABEL_MASK); } +/** + * rdma_get_udp_sport - Calculate and set UDP source port based on the flow + * label. If flow label is not defined in GRH then + * calculate it based on lqpn/rqpn. + * + * @fl: flow label from GRH + * @lqpn: local qp number + * @rqpn: remote qp number + */ +static inline u16 rdma_get_udp_sport(u32 fl, u32 lqpn, u32 rqpn) +{ + if (!fl) + fl = rdma_calc_flow_label(lqpn, rqpn); + + return rdma_flow_label_to_udp_sport(fl); +} + const struct ib_port_immutable* ib_port_immutable_read(struct ib_device *dev, unsigned int port); #endif /* IB_VERBS_H */ From patchwork Wed Jan 5 22:12:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 244C1C433EF for ; Wed, 5 Jan 2022 05:49:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237566AbiAEFtw (ORCPT ); Wed, 5 Jan 2022 00:49:52 -0500 Received: from mga07.intel.com ([134.134.136.100]:27100 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237564AbiAEFtw (ORCPT ); Wed, 5 Jan 2022 00:49:52 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="305725905" X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="305725905" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 21:49:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="760688615" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by fmsmga006.fm.intel.com with ESMTP; 04 Jan 2022 21:49:50 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 2/5] RDMA/hns: Replace get_udp_sport with rdma_get_udp_sport Date: Wed, 5 Jan 2022 17:12:34 -0500 Message-Id: <20220105221237.2659462-3-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105221237.2659462-1-yanjun.zhu@linux.dev> References: <20220105221237.2659462-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Several drivers have the same function xxx_get_udp_sport. So this function is moved to ib_verbs.h. Signed-off-by: Zhu Yanjun Reviewed-by: Leon Romanovsky --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index eb0defa80d0d..cb795663b813 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -4488,14 +4488,6 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp, return 0; } -static inline u16 get_udp_sport(u32 fl, u32 lqpn, u32 rqpn) -{ - if (!fl) - fl = rdma_calc_flow_label(lqpn, rqpn); - - return rdma_flow_label_to_udp_sport(fl); -} - static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, u32 *dip_idx) { @@ -4712,8 +4704,8 @@ static int hns_roce_v2_set_path(struct ib_qp *ibqp, } hr_reg_write(context, QPC_UDPSPN, - is_udp ? get_udp_sport(grh->flow_label, ibqp->qp_num, - attr->dest_qp_num) : 0); + is_udp ? rdma_get_udp_sport(grh->flow_label, ibqp->qp_num, + attr->dest_qp_num) : 0); hr_reg_clear(qpc_mask, QPC_UDPSPN); From patchwork Wed Jan 5 22:12:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5601FC433EF for ; Wed, 5 Jan 2022 05:49:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236188AbiAEFty (ORCPT ); Wed, 5 Jan 2022 00:49:54 -0500 Received: from mga07.intel.com ([134.134.136.100]:27100 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237564AbiAEFty (ORCPT ); Wed, 5 Jan 2022 00:49:54 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="305725914" X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="305725914" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 21:49:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="760688622" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by fmsmga006.fm.intel.com with ESMTP; 04 Jan 2022 21:49:52 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 3/5] RDMA/irdma: Make the source udp port vary Date: Wed, 5 Jan 2022 17:12:35 -0500 Message-Id: <20220105221237.2659462-4-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105221237.2659462-1-yanjun.zhu@linux.dev> References: <20220105221237.2659462-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Get the source udp port number for a QP based on the grh.flow_label or lqpn/rqrpn. This provides a better spread of traffic across NIC RX queues. Signed-off-by: Zhu Yanjun Reviewed-by: Leon Romanovsky Acked-by: Shiraz Saleem --- drivers/infiniband/hw/irdma/verbs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 8cd5f9261692..8234a101e752 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -1170,6 +1170,10 @@ int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr, udp_info->ttl = attr->ah_attr.grh.hop_limit; udp_info->flow_label = attr->ah_attr.grh.flow_label; udp_info->tos = attr->ah_attr.grh.traffic_class; + udp_info->src_port = rdma_get_udp_sport( + udp_info->flow_label, + ibqp->qp_num, + roce_info->dest_qp); irdma_qp_rem_qos(&iwqp->sc_qp); dev->ws_remove(iwqp->sc_qp.vsi, ctx_info->user_pri); ctx_info->user_pri = rt_tos2priority(udp_info->tos); From patchwork Wed Jan 5 22:12:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E045C433F5 for ; Wed, 5 Jan 2022 05:49:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237564AbiAEFt5 (ORCPT ); Wed, 5 Jan 2022 00:49:57 -0500 Received: from mga07.intel.com ([134.134.136.100]:27100 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237570AbiAEFt4 (ORCPT ); Wed, 5 Jan 2022 00:49:56 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="305725926" X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="305725926" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 21:49:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="760688629" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by fmsmga006.fm.intel.com with ESMTP; 04 Jan 2022 21:49:54 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 4/5] RDMA/rxe: Use the standard method to produce udp source port Date: Wed, 5 Jan 2022 17:12:36 -0500 Message-Id: <20220105221237.2659462-5-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105221237.2659462-1-yanjun.zhu@linux.dev> References: <20220105221237.2659462-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Use the standard method to produce udp source port. Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_verbs.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 0aa0d7e52773..42fa81b455de 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -469,6 +469,12 @@ static int rxe_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, if (err) goto err1; + if ((mask & IB_QP_AV) && (attr->ah_attr.ah_flags & IB_AH_GRH)) + qp->src_port = rdma_get_udp_sport(attr->ah_attr.grh.flow_label, + qp->ibqp.qp_num, + qp->attr.dest_qp_num); + + return 0; err1: From patchwork Wed Jan 5 22:12:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 12704051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B7B9C433EF for ; Wed, 5 Jan 2022 05:49:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237571AbiAEFt6 (ORCPT ); Wed, 5 Jan 2022 00:49:58 -0500 Received: from mga07.intel.com ([134.134.136.100]:27100 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237567AbiAEFt6 (ORCPT ); Wed, 5 Jan 2022 00:49:58 -0500 X-IronPort-AV: E=McAfee;i="6200,9189,10217"; a="305725932" X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="305725932" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 21:49:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,262,1635231600"; d="scan'208";a="760688637" Received: from intel-obmc.bj.intel.com (HELO intel-71.bj.intel.com) ([10.238.154.71]) by fmsmga006.fm.intel.com with ESMTP; 04 Jan 2022 21:49:56 -0800 From: yanjun.zhu@linux.dev To: liangwenpeng@huawei.com, jgg@ziepe.ca, mustafa.ismail@intel.com, shiraz.saleem@intel.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, yanjun.zhu@linux.dev Subject: [PATCH 5/5] RDMA/rxe: Remove the redundant randomization for UDP source port Date: Wed, 5 Jan 2022 17:12:37 -0500 Message-Id: <20220105221237.2659462-6-yanjun.zhu@linux.dev> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220105221237.2659462-1-yanjun.zhu@linux.dev> References: <20220105221237.2659462-1-yanjun.zhu@linux.dev> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Since the UDP source port is modified in rxe_modify_qp, the randomization for UDP source port is redundant in this function. So remove it. Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_qp.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 54b8711321c1..84d6ffe7350a 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -210,15 +210,9 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, return err; qp->sk->sk->sk_user_data = qp; - /* pick a source UDP port number for this QP based on - * the source QPN. this spreads traffic for different QPs - * across different NIC RX queues (while using a single - * flow for a given QP to maintain packet order). - * the port number must be in the Dynamic Ports range - * (0xc000 - 0xffff). + /* Source UDP port number for this QP is modified in rxe_qp_modify. */ - qp->src_port = RXE_ROCE_V2_SPORT + - (hash_32_generic(qp_num(qp), 14) & 0x3fff); + qp->src_port = RXE_ROCE_V2_SPORT; qp->sq.max_wr = init->cap.max_send_wr; /* These caps are limited by rxe_qp_chk_cap() done by the caller */