From patchwork Thu Jun 8 10:50:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naveen Mamindlapalli X-Patchwork-Id: 13271940 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44681C2F6 for ; Thu, 8 Jun 2023 10:50:24 +0000 (UTC) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE65A269A; Thu, 8 Jun 2023 03:50:22 -0700 (PDT) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3587nH50023576; Thu, 8 Jun 2023 03:50:17 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=L2TeqzzSAJW3Y0ujjwLs+3OlJYXWv6M+88djxLDuzzk=; b=b0nljOoXhMD9NayzOdT2KlvF5u1a0ivMgpbneDuDdvw8cKag8fNYHIrFkWaC2YyU+JRO fMq7WryeKpaLAr4aeJEfC/OZHWyfXsxOiC7NwOOUcoMaJnARFZIGf4S+/zO1nxBeQvXG bAJ6zI+7/6kWRwt8h9XUslR+My3FciwEsp2iGd5kKb4LyyPW/59zgzo5j2OdA0ZQC/ux v4eibrsIIAOI+7qrGCfeufhewSAFJ14NCos3NvMjxrmHf9E3wC4dE+mbVq6O/1muk7wx sEIdWRm3lorXBSPKvQowVLCtam8R/DLJYG3yXooYyQjcktijf1wvb0bmmoTETRQKqdhc aQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3r329c2c30-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2023 03:50:16 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 8 Jun 2023 03:50:15 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 8 Jun 2023 03:50:15 -0700 Received: from #hyd1583.marvell.com (unknown [10.29.37.44]) by maili.marvell.com (Postfix) with ESMTP id DF6573F707F; Thu, 8 Jun 2023 03:50:12 -0700 (PDT) From: Naveen Mamindlapalli To: , , , , , , CC: Naveen Mamindlapalli Subject: [net-next PATCH 1/6] octeontx2-af: Add devlink option to adjust mcam high prio zone entries Date: Thu, 8 Jun 2023 16:20:02 +0530 Message-ID: <20230608105007.26924-2-naveenm@marvell.com> X-Mailer: git-send-email 2.39.0.198.ga38d39a4c5 In-Reply-To: <20230608105007.26924-1-naveenm@marvell.com> References: <20230608105007.26924-1-naveenm@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: gT39MCYMJMLnozTCBMUbCjhRItGLTfBo X-Proofpoint-ORIG-GUID: gT39MCYMJMLnozTCBMUbCjhRItGLTfBo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-08_07,2023-06-08_01,2023-05-22_02 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org The NPC MCAM entries are currently divided into three priority zones in AF driver: high, mid, and low. The high priority zone and low priority zone take up 1/8th (each) of the available MCAM entries, and remaining going to the mid priority zone. The current allocation scheme may not meet certain requirements, such as when a requester needs more high priority zone entries than are reserved. This patch adds a devlink configurable option to increase the number of high priority zone entries that can be allocated by requester. The max number of entries that can be reserved for high priority usage is 100% of available MCAM entries. Usage: 1) Change high priority zone percentage to 75%: devlink -p dev param set pci/0002:01:00.0 name npc_mcam_high_zone_percent \ value 75 cmode runtime 2) Read high priority zone percentage: devlink -p dev param show pci/0002:01:00.0 name npc_mcam_high_zone_percent The devlink set configuration is only permitted when no MCAM entries are assigned, i.e., all MCAM entries are free, indicating that no PF/VF driver is loaded. So user must unload/unbind PF/VF driver/devices before modifying the high priority zone percentage. Signed-off-by: Naveen Mamindlapalli Signed-off-by: Sunil Goutham --- .../marvell/octeontx2/af/rvu_devlink.c | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c index bda1a6fa2ec4..64b8beaa7247 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c @@ -1438,6 +1438,7 @@ enum rvu_af_dl_param_id { RVU_AF_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, RVU_AF_DEVLINK_PARAM_ID_DWRR_MTU, RVU_AF_DEVLINK_PARAM_ID_NPC_EXACT_FEATURE_DISABLE, + RVU_AF_DEVLINK_PARAM_ID_NPC_MCAM_ZONE_PERCENT, }; static int rvu_af_npc_exact_feature_get(struct devlink *devlink, u32 id, @@ -1494,6 +1495,67 @@ static int rvu_af_npc_exact_feature_validate(struct devlink *devlink, u32 id, return -EFAULT; } +static int rvu_af_dl_npc_mcam_high_zone_percent_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct rvu_devlink *rvu_dl = devlink_priv(devlink); + struct rvu *rvu = rvu_dl->rvu; + struct npc_mcam *mcam; + u32 percent; + + mcam = &rvu->hw->mcam; + percent = (mcam->hprio_count * 100) / mcam->bmap_entries; + ctx->val.vu8 = (u8)percent; + + return 0; +} + +static int rvu_af_dl_npc_mcam_high_zone_percent_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct rvu_devlink *rvu_dl = devlink_priv(devlink); + struct rvu *rvu = rvu_dl->rvu; + struct npc_mcam *mcam; + u32 percent; + + percent = ctx->val.vu8; + mcam = &rvu->hw->mcam; + mcam->hprio_count = (mcam->bmap_entries * percent) / 100; + mcam->hprio_end = mcam->hprio_count; + mcam->lprio_count = (mcam->bmap_entries - mcam->hprio_count) / 2; + mcam->lprio_start = mcam->bmap_entries - mcam->lprio_count; + + return 0; +} + +static int rvu_af_dl_npc_mcam_high_zone_percent_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + struct rvu_devlink *rvu_dl = devlink_priv(devlink); + struct rvu *rvu = rvu_dl->rvu; + struct npc_mcam *mcam; + + /* The percent of high prio zone must range from 12% to 100% of unreserved mcam space */ + if (val.vu8 < 12 || val.vu8 > 100) { + NL_SET_ERR_MSG_MOD(extack, + "mcam high zone percent must be between 12% to 100%"); + return -EINVAL; + } + + /* Do not allow user to modify the high priority zone entries while mcam entries + * have already been assigned. + */ + mcam = &rvu->hw->mcam; + if (mcam->bmap_fcnt < mcam->bmap_entries) { + NL_SET_ERR_MSG_MOD(extack, + "mcam entries have already been assigned, can't resize"); + return -EPERM; + } + + return 0; +} + static const struct devlink_param rvu_af_dl_params[] = { DEVLINK_PARAM_DRIVER(RVU_AF_DEVLINK_PARAM_ID_DWRR_MTU, "dwrr_mtu", DEVLINK_PARAM_TYPE_U32, @@ -1506,6 +1568,12 @@ static const struct devlink_param rvu_af_dl_params[] = { rvu_af_npc_exact_feature_get, rvu_af_npc_exact_feature_disable, rvu_af_npc_exact_feature_validate), + DEVLINK_PARAM_DRIVER(RVU_AF_DEVLINK_PARAM_ID_NPC_MCAM_ZONE_PERCENT, + "npc_mcam_high_zone_percent", DEVLINK_PARAM_TYPE_U8, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + rvu_af_dl_npc_mcam_high_zone_percent_get, + rvu_af_dl_npc_mcam_high_zone_percent_set, + rvu_af_dl_npc_mcam_high_zone_percent_validate), }; /* Devlink switch mode */ From patchwork Thu Jun 8 10:50:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naveen Mamindlapalli X-Patchwork-Id: 13271943 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A2D810974 for ; Thu, 8 Jun 2023 10:50:36 +0000 (UTC) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61D102D57; Thu, 8 Jun 2023 03:50:34 -0700 (PDT) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35838QYX023519; Thu, 8 Jun 2023 03:50:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Ax/yq4NvinpMK8sJ/K+gDkMxqN6eTkspj5OjglVD590=; b=Axo9Eo7fdadz1X/0vNv7dhNZglsIvJHge3iqS2RhCy5oMTUJc6S4DfBZW4W/P9N87N/C gNO9nnLKWRsW6egNKe/jf++REG5u8WQsdCwnJgBZNnhf4GKsnxLAnh3uuDI0Byl4Wt8J X9il6lKXBL3AE0Ieh35Mx6q+8pXuWkEtCSzpenEVnauH7p5F816+UOBCly8aVIKDxa6z 9Mq9UlyCisMTg83dYa1uphW69rB4vhhqVjJC7oz0tlJ+t15hKTUiOCtPXPGRWD4GFx4n ExbaY/Ft50ZlLAQ47y8383zjfl0/r7N7UOQ4vl3+Ckdn8MvyEMOtRaooT5obDJ3kf33n Xw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3r30eu27cx-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2023 03:50:21 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 8 Jun 2023 03:50:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 8 Jun 2023 03:50:18 -0700 Received: from #hyd1583.marvell.com (unknown [10.29.37.44]) by maili.marvell.com (Postfix) with ESMTP id CCC183F70B6; Thu, 8 Jun 2023 03:50:15 -0700 (PDT) From: Naveen Mamindlapalli To: , , , , , , CC: Kiran Kumar K , Geetha sowjanya , Naveen Mamindlapalli Subject: [net-next PATCH 2/6] octeontx2-af: extend RSS supported offload types Date: Thu, 8 Jun 2023 16:20:03 +0530 Message-ID: <20230608105007.26924-3-naveenm@marvell.com> X-Mailer: git-send-email 2.39.0.198.ga38d39a4c5 In-Reply-To: <20230608105007.26924-1-naveenm@marvell.com> References: <20230608105007.26924-1-naveenm@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: zB_6fweu-mb1lf41PfabjLNfBp-D11ag X-Proofpoint-GUID: zB_6fweu-mb1lf41PfabjLNfBp-D11ag X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-08_07,2023-06-08_01,2023-05-22_02 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org From: Kiran Kumar K Add support to select L3 SRC or DST only, L4 SRC or DST only for RSS calculation. AF consumer may have requirement as we can select only SRC or DST data for RSS calculation in L3, L4 layers. With this requirement there will be following combinations, IPV[4,6]_SRC_ONLY, IPV[4,6]_DST_ONLY, [TCP,UDP,SCTP]_SRC_ONLY, [TCP,UDP,SCTP]_DST_ONLY. So, instead of creating a bit for each combination, we are using upper 4 bits (31:28) in the flow_key_cfg to represent the SRC, DST selection. 31 => L3_SRC, 30 => L3_DST, 29 => L4_SRC, 28 => L4_DST. These won't be part of flow_cfg, so that we don't need to change the existing ABI. Signed-off-by: Kiran Kumar K Signed-off-by: Geetha sowjanya Signed-off-by: Naveen Mamindlapalli --- .../net/ethernet/marvell/octeontx2/af/mbox.h | 6 ++ .../ethernet/marvell/octeontx2/af/rvu_nix.c | 57 +++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h index d2584ebb7a70..43968f0b6218 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -1074,6 +1074,8 @@ struct nix_vtag_config_rsp { */ }; +#define NIX_FLOW_KEY_TYPE_L3_L4_MASK (~(0xf << 28)) + struct nix_rss_flowkey_cfg { struct mbox_msghdr hdr; int mcam_index; /* MCAM entry index to modify */ @@ -1099,6 +1101,10 @@ struct nix_rss_flowkey_cfg { #define NIX_FLOW_KEY_TYPE_IPV4_PROTO BIT(21) #define NIX_FLOW_KEY_TYPE_AH BIT(22) #define NIX_FLOW_KEY_TYPE_ESP BIT(23) +#define NIX_FLOW_KEY_TYPE_L4_DST_ONLY BIT(28) +#define NIX_FLOW_KEY_TYPE_L4_SRC_ONLY BIT(29) +#define NIX_FLOW_KEY_TYPE_L3_DST_ONLY BIT(30) +#define NIX_FLOW_KEY_TYPE_L3_SRC_ONLY BIT(31) u32 flowkey_cfg; /* Flowkey types selected */ u8 group; /* RSS context or group */ }; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c index 6b8747ebc08c..d40bde3948b4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c @@ -3287,6 +3287,7 @@ static int set_flowkey_fields(struct nix_rx_flowkey_alg *alg, u32 flow_cfg) struct nix_rx_flowkey_alg *field; struct nix_rx_flowkey_alg tmp; u32 key_type, valid_key; + u32 l3_l4_src_dst; int l4_key_offset = 0; if (!alg) @@ -3314,6 +3315,15 @@ static int set_flowkey_fields(struct nix_rx_flowkey_alg *alg, u32 flow_cfg) * group_member - Enabled when protocol is part of a group. */ + /* Last 4 bits (31:28) are reserved to specify SRC, DST + * selection for L3, L4 i.e IPV[4,6]_SRC, IPV[4,6]_DST, + * [TCP,UDP,SCTP]_SRC, [TCP,UDP,SCTP]_DST + * 31 => L3_SRC, 30 => L3_DST, 29 => L4_SRC, 28 => L4_DST + */ + l3_l4_src_dst = flow_cfg; + /* Reset these 4 bits, so that these won't be part of key */ + flow_cfg &= NIX_FLOW_KEY_TYPE_L3_L4_MASK; + keyoff_marker = 0; max_key_off = 0; group_member = 0; nr_field = 0; key_off = 0; field_marker = 1; field = &tmp; max_bit_pos = fls(flow_cfg); @@ -3351,6 +3361,22 @@ static int set_flowkey_fields(struct nix_rx_flowkey_alg *alg, u32 flow_cfg) } field->hdr_offset = 12; /* SIP offset */ field->bytesm1 = 7; /* SIP + DIP, 8 bytes */ + + /* Only SIP */ + if (l3_l4_src_dst & NIX_FLOW_KEY_TYPE_L3_SRC_ONLY) + field->bytesm1 = 3; /* SIP, 4 bytes */ + + if (l3_l4_src_dst & NIX_FLOW_KEY_TYPE_L3_DST_ONLY) { + /* Both SIP + DIP */ + if (field->bytesm1 == 3) { + field->bytesm1 = 7; /* SIP + DIP, 8B */ + } else { + /* Only DIP */ + field->hdr_offset = 16; /* DIP off */ + field->bytesm1 = 3; /* DIP, 4 bytes */ + } + } + field->ltype_mask = 0xF; /* Match only IPv4 */ keyoff_marker = false; break; @@ -3364,6 +3390,22 @@ static int set_flowkey_fields(struct nix_rx_flowkey_alg *alg, u32 flow_cfg) } field->hdr_offset = 8; /* SIP offset */ field->bytesm1 = 31; /* SIP + DIP, 32 bytes */ + + /* Only SIP */ + if (l3_l4_src_dst & NIX_FLOW_KEY_TYPE_L3_SRC_ONLY) + field->bytesm1 = 15; /* SIP, 16 bytes */ + + if (l3_l4_src_dst & NIX_FLOW_KEY_TYPE_L3_DST_ONLY) { + /* Both SIP + DIP */ + if (field->bytesm1 == 15) { + /* SIP + DIP, 32 bytes */ + field->bytesm1 = 31; + } else { + /* Only DIP */ + field->hdr_offset = 24; /* DIP off */ + field->bytesm1 = 15; /* DIP,16 bytes */ + } + } field->ltype_mask = 0xF; /* Match only IPv6 */ break; case NIX_FLOW_KEY_TYPE_TCP: @@ -3379,6 +3421,21 @@ static int set_flowkey_fields(struct nix_rx_flowkey_alg *alg, u32 flow_cfg) field->lid = NPC_LID_LH; field->bytesm1 = 3; /* Sport + Dport, 4 bytes */ + if (l3_l4_src_dst & NIX_FLOW_KEY_TYPE_L4_SRC_ONLY) + field->bytesm1 = 1; /* SRC, 2 bytes */ + + if (l3_l4_src_dst & NIX_FLOW_KEY_TYPE_L4_DST_ONLY) { + /* Both SRC + DST */ + if (field->bytesm1 == 1) { + /* SRC + DST, 4 bytes */ + field->bytesm1 = 3; + } else { + /* Only DIP */ + field->hdr_offset = 2; /* DST off */ + field->bytesm1 = 1; /* DST, 2 bytes */ + } + } + /* Enum values for NPC_LID_LD and NPC_LID_LG are same, * so no need to change the ltype_match, just change * the lid for inner protocols From patchwork Thu Jun 8 10:50:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naveen Mamindlapalli X-Patchwork-Id: 13271941 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6FB41094D for ; Thu, 8 Jun 2023 10:50:31 +0000 (UTC) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C252CE6C; Thu, 8 Jun 2023 03:50:29 -0700 (PDT) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3588EvXt023828; Thu, 8 Jun 2023 03:50:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Vv7HiX1LYJQPoGFMza8lr/qn0akT9GHlrSMcoOdYePg=; b=DiEPPnwWcThk7XYkyRTi8IpfIdEDQTYoGoUBhAdbo7t27WuZfELABl+AjB01fcdwECst kX/N1J/Sazgm0XeALcSLJ+s+LfWayRdon6IprUSzJ2AD5oBKZtTnVkFMaD18ZY9OVMhK gLjzQcVfyebD8hUr1qF1OoS5hEl5x2iYvxnhKoMkrjcYDU/846MUtzq4FRuiJcE+yjDx i55RZg1FdCDvhKywV91Zj+ZQ5CWbXKFqBAl9qwupgnKYWQgQEN52LkUUaYinmy1rJMdK j+ilLqbfuM+WJmEiD1m5cf7Gpj0U7bVv1xFAQFE+OUo3Gn0/aeRwLzaxU/Ar3+PNnF/4 /Q== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3r329c2c34-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2023 03:50:23 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 8 Jun 2023 03:50:21 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 8 Jun 2023 03:50:21 -0700 Received: from #hyd1583.marvell.com (unknown [10.29.37.44]) by maili.marvell.com (Postfix) with ESMTP id 40CD63F707C; Thu, 8 Jun 2023 03:50:19 -0700 (PDT) From: Naveen Mamindlapalli To: , , , , , , CC: Geetha sowjanya , Naveen Mamindlapalli Subject: [net-next PATCH 3/6] octeontx2-af: cn10k: Set NIX DWRR MTU for CN10KB silicon Date: Thu, 8 Jun 2023 16:20:04 +0530 Message-ID: <20230608105007.26924-4-naveenm@marvell.com> X-Mailer: git-send-email 2.39.0.198.ga38d39a4c5 In-Reply-To: <20230608105007.26924-1-naveenm@marvell.com> References: <20230608105007.26924-1-naveenm@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: RQinqoKWA7bNOIMazj95o584NnEWLeMd X-Proofpoint-ORIG-GUID: RQinqoKWA7bNOIMazj95o584NnEWLeMd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-08_07,2023-06-08_01,2023-05-22_02 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org From: Sunil Goutham The DWRR MTU config added for SDP and RPM/LBK links on CN10K silicon is further extended on CK10KB silicon variant and made it configurable. Now there are 4 DWRR MTU config to choose while setting transmit scheduler's RR_WEIGHT. Here we are reserving one config for each of RPM, SDP and LBK. NIXX_AF_DWRR_MTUX(0) ---> RPM NIXX_AF_DWRR_MTUX(1) ---> SDP NIXX_AF_DWRR_MTUX(2) ---> LBK PF/VF drivers can choose the DWRR_MTU to be used by setting SMQX_CFG[pkt_link_type] to one of above. TLx_SCHEDULE[RR_WEIGHT] is to be as configured 'quantum / 2^DWRR_MTUX[MTU]'. DWRR_MTU of each link is exposed to PF/VF drivers via mailbox for RR_WEIGHT calculation. Signed-off-by: Sunil Goutham Signed-off-by: Geetha sowjanya Signed-off-by: Naveen Mamindlapalli --- .../ethernet/marvell/octeontx2/af/common.h | 7 +++ .../net/ethernet/marvell/octeontx2/af/mbox.h | 4 +- .../net/ethernet/marvell/octeontx2/af/rvu.h | 2 + .../marvell/octeontx2/af/rvu_devlink.c | 6 ++- .../ethernet/marvell/octeontx2/af/rvu_nix.c | 44 ++++++++++++++++--- .../ethernet/marvell/octeontx2/af/rvu_reg.h | 3 +- .../marvell/octeontx2/nic/otx2_common.c | 18 +++++++- .../marvell/octeontx2/nic/otx2_common.h | 1 + 8 files changed, 73 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/common.h b/drivers/net/ethernet/marvell/octeontx2/af/common.h index 8931864ee110..b2d9dac7030b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/common.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/common.h @@ -145,6 +145,13 @@ enum nix_scheduler { #define TXSCH_TL1_DFLT_RR_PRIO (0x1ull) #define CN10K_MAX_DWRR_WEIGHT 16384 /* Weight is 14bit on CN10K */ +/* Don't change the order as on CN10K (except CN10KB) + * SMQX_CFG[SDP] value should be 1 for SDP flows. + */ +#define SMQ_LINK_TYPE_RPM 0 +#define SMQ_LINK_TYPE_SDP 1 +#define SMQ_LINK_TYPE_LBK 2 + /* Min/Max packet sizes, excluding FCS */ #define NIC_HW_MIN_FRS 40 #define NIC_HW_MAX_FRS 9212 diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h index 43968f0b6218..f233f98cbeea 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -1237,7 +1237,9 @@ struct nix_hw_info { u16 min_mtu; u32 rpm_dwrr_mtu; u32 sdp_dwrr_mtu; - u64 rsvd[16]; /* Add reserved fields for future expansion */ + u32 lbk_dwrr_mtu; + u32 rsvd32[1]; + u64 rsvd[15]; /* Add reserved fields for future expansion */ }; struct nix_bandprof_alloc_req { diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index 7f0a64731c67..54c3fa815e37 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -344,6 +344,7 @@ struct hw_cap { bool per_pf_mbox_regs; /* PF mbox specified in per PF registers ? */ bool programmable_chans; /* Channels programmable ? */ bool ipolicer; + bool nix_multiple_dwrr_mtu; /* Multiple DWRR_MTU to choose from */ bool npc_hash_extract; /* Hash extract enabled ? */ bool npc_exact_match_enabled; /* Exact match supported ? */ }; @@ -785,6 +786,7 @@ int nix_aq_context_read(struct rvu *rvu, struct nix_hw *nix_hw, struct nix_cn10k_aq_enq_rsp *aq_rsp, u16 pcifunc, u8 ctype, u32 qidx); int rvu_get_nix_blkaddr(struct rvu *rvu, u16 pcifunc); +int nix_get_dwrr_mtu_reg(struct rvu_hwinfo *hw, int smq_link_type); u32 convert_dwrr_mtu_to_bytes(u8 dwrr_mtu); u32 convert_bytes_to_dwrr_mtu(u32 bytes); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c index 64b8beaa7247..2a35bd5d26d3 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c @@ -1413,7 +1413,8 @@ static int rvu_af_dl_dwrr_mtu_set(struct devlink *devlink, u32 id, u64 dwrr_mtu; dwrr_mtu = convert_bytes_to_dwrr_mtu(ctx->val.vu32); - rvu_write64(rvu, BLKADDR_NIX0, NIX_AF_DWRR_RPM_MTU, dwrr_mtu); + rvu_write64(rvu, BLKADDR_NIX0, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_RPM), dwrr_mtu); return 0; } @@ -1428,7 +1429,8 @@ static int rvu_af_dl_dwrr_mtu_get(struct devlink *devlink, u32 id, if (!rvu->hw->cap.nix_common_dwrr_mtu) return -EOPNOTSUPP; - dwrr_mtu = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_DWRR_RPM_MTU); + dwrr_mtu = rvu_read64(rvu, BLKADDR_NIX0, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_RPM)); ctx->val.vu32 = convert_dwrr_mtu_to_bytes(dwrr_mtu); return 0; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c index d40bde3948b4..74793dd7d895 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c @@ -191,6 +191,18 @@ struct nix_hw *get_nix_hw(struct rvu_hwinfo *hw, int blkaddr) return NULL; } +int nix_get_dwrr_mtu_reg(struct rvu_hwinfo *hw, int smq_link_type) +{ + if (hw->cap.nix_multiple_dwrr_mtu) + return NIX_AF_DWRR_MTUX(smq_link_type); + + if (smq_link_type == SMQ_LINK_TYPE_SDP) + return NIX_AF_DWRR_SDP_MTU; + + /* Here it's same reg for RPM and LBK */ + return NIX_AF_DWRR_RPM_MTU; +} + u32 convert_dwrr_mtu_to_bytes(u8 dwrr_mtu) { dwrr_mtu &= 0x1FULL; @@ -3125,10 +3137,16 @@ static int nix_setup_txschq(struct rvu *rvu, struct nix_hw *nix_hw, int blkaddr) } /* Setup a default value of 8192 as DWRR MTU */ - if (rvu->hw->cap.nix_common_dwrr_mtu) { - rvu_write64(rvu, blkaddr, NIX_AF_DWRR_RPM_MTU, + if (rvu->hw->cap.nix_common_dwrr_mtu || + rvu->hw->cap.nix_multiple_dwrr_mtu) { + rvu_write64(rvu, blkaddr, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_RPM), convert_bytes_to_dwrr_mtu(8192)); - rvu_write64(rvu, blkaddr, NIX_AF_DWRR_SDP_MTU, + rvu_write64(rvu, blkaddr, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_LBK), + convert_bytes_to_dwrr_mtu(8192)); + rvu_write64(rvu, blkaddr, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_SDP), convert_bytes_to_dwrr_mtu(8192)); } @@ -3226,19 +3244,28 @@ int rvu_mbox_handler_nix_get_hw_info(struct rvu *rvu, struct msg_req *req, rsp->min_mtu = NIC_HW_MIN_FRS; - if (!rvu->hw->cap.nix_common_dwrr_mtu) { + if (!rvu->hw->cap.nix_common_dwrr_mtu && + !rvu->hw->cap.nix_multiple_dwrr_mtu) { /* Return '1' on OTx2 */ rsp->rpm_dwrr_mtu = 1; rsp->sdp_dwrr_mtu = 1; + rsp->lbk_dwrr_mtu = 1; return 0; } - dwrr_mtu = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_DWRR_RPM_MTU); + /* Return DWRR_MTU for TLx_SCHEDULE[RR_WEIGHT] config */ + dwrr_mtu = rvu_read64(rvu, blkaddr, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_RPM)); rsp->rpm_dwrr_mtu = convert_dwrr_mtu_to_bytes(dwrr_mtu); - dwrr_mtu = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_DWRR_SDP_MTU); + dwrr_mtu = rvu_read64(rvu, blkaddr, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_SDP)); rsp->sdp_dwrr_mtu = convert_dwrr_mtu_to_bytes(dwrr_mtu); + dwrr_mtu = rvu_read64(rvu, blkaddr, + nix_get_dwrr_mtu_reg(rvu->hw, SMQ_LINK_TYPE_LBK)); + rsp->lbk_dwrr_mtu = convert_dwrr_mtu_to_bytes(dwrr_mtu); + return 0; } @@ -4305,8 +4332,11 @@ static void rvu_nix_setup_capabilities(struct rvu *rvu, int blkaddr) * Check if HW uses a common MTU for all DWRR quantum configs. * On OcteonTx2 this register field is '0'. */ - if (((hw_const >> 56) & 0x10) == 0x10) + if ((((hw_const >> 56) & 0x10) == 0x10) && !(hw_const & BIT_ULL(61))) hw->cap.nix_common_dwrr_mtu = true; + + if (hw_const & BIT_ULL(61)) + hw->cap.nix_multiple_dwrr_mtu = true; } static int rvu_nix_block_init(struct rvu *rvu, struct nix_hw *nix_hw) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h index 0e0d536645ac..6b0105c895a4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h @@ -271,7 +271,8 @@ #define NIX_AF_DEBUG_NPC_RESP_DATAX(a) (0x680 | (a) << 3) #define NIX_AF_SMQX_CFG(a) (0x700 | (a) << 16) #define NIX_AF_SQM_DBG_CTL_STATUS (0x750) -#define NIX_AF_DWRR_SDP_MTU (0x790) +#define NIX_AF_DWRR_SDP_MTU (0x790) /* All CN10K except CN10KB */ +#define NIX_AF_DWRR_MTUX(a) (0x790 | (a) << 16) /* Only for CN10KB */ #define NIX_AF_DWRR_RPM_MTU (0x7A0) #define NIX_AF_PSE_CHANNEL_LEVEL (0x800) #define NIX_AF_PSE_SHAPER_CFG (0x810) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 88f8772a61cd..10faa50c876b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -8,6 +8,7 @@ #include #include #include +#include #include "otx2_reg.h" #include "otx2_common.h" @@ -616,6 +617,10 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl, int prio, bool txschq_for req->regval[0] = ((u64)pfvf->tx_max_pktlen << 8) | OTX2_MIN_MTU; req->regval[0] |= (0x20ULL << 51) | (0x80ULL << 39) | (0x2ULL << 36); + /* Set link type for DWRR MTU selection on CN10K silicons */ + if (!is_dev_otx2(pfvf->pdev)) + req->regval[0] |= FIELD_PREP(GENMASK_ULL(58, 57), + (u64)hw->smq_link_type); req->num_regs++; /* MDQ config */ parent = schq_list[NIX_TXSCH_LVL_TL4][prio]; @@ -1734,6 +1739,17 @@ void otx2_set_cints_affinity(struct otx2_nic *pfvf) } } +static u32 get_dwrr_mtu(struct otx2_nic *pfvf, struct nix_hw_info *hw) +{ + if (is_otx2_lbkvf(pfvf->pdev)) { + pfvf->hw.smq_link_type = SMQ_LINK_TYPE_LBK; + return hw->lbk_dwrr_mtu; + } + + pfvf->hw.smq_link_type = SMQ_LINK_TYPE_RPM; + return hw->rpm_dwrr_mtu; +} + u16 otx2_get_max_mtu(struct otx2_nic *pfvf) { struct nix_hw_info *rsp; @@ -1763,7 +1779,7 @@ u16 otx2_get_max_mtu(struct otx2_nic *pfvf) max_mtu = rsp->max_mtu - 8 - OTX2_ETH_HLEN; /* Also save DWRR MTU, needed for DWRR weight calculation */ - pfvf->hw.dwrr_mtu = rsp->rpm_dwrr_mtu; + pfvf->hw.dwrr_mtu = get_dwrr_mtu(pfvf, rsp); if (!pfvf->hw.dwrr_mtu) pfvf->hw.dwrr_mtu = 1; } diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 5bee3c3a7ce4..aa07069cd9dc 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -209,6 +209,7 @@ struct otx2_hw { u16 txschq_list[NIX_TXSCH_LVL_CNT][MAX_TXSCHQ_PER_FUNC]; u16 matchall_ipolicer; u32 dwrr_mtu; + u8 smq_link_type; /* HW settings, coalescing etc */ u16 rx_chan_base; From patchwork Thu Jun 8 10:50:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naveen Mamindlapalli X-Patchwork-Id: 13271942 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FBA3BA2E for ; Thu, 8 Jun 2023 10:50:36 +0000 (UTC) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1839D2737; Thu, 8 Jun 2023 03:50:34 -0700 (PDT) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35838QYY023519; Thu, 8 Jun 2023 03:50:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=aLCLPKdHSPqIYFI/3Akos3aGAHBSGwD5lr/j0WX1JO4=; b=j0felJ9k9i1rzzwsL4rLk4oadzwcRkOf907VK6SPyqy+dWAe8bmcnZUTf8NoEWC78LjU 2ELd19MH1Wu/pUPWV6HuA3nHBgtEnycAXoujw+V6s3IsWHoutYHGEHIiIVIkljQFpfnO qU+YeS/o7yrzypwEsSJcU/icusR/AU+KbKjwM+B7H7nUAstjZ6e7NMvNX3qWjyP/sVcT 7Xs70H/RoIZaKtJLTnEF68XH9pkhy/yAsVG+cKU4RMTwLwsQVlzqd2kH+5ZaBXNpd5q1 eJsdbUd5ig7RRJ/u9ulBEAVQo9QVw+wIvd1oF6mahETiIfkAlpX5xnoX4b/lDdhzzzSi gA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3r30eu27d8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2023 03:50:27 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 8 Jun 2023 03:50:24 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 8 Jun 2023 03:50:25 -0700 Received: from #hyd1583.marvell.com (unknown [10.29.37.44]) by maili.marvell.com (Postfix) with ESMTP id 69BB23F707E; Thu, 8 Jun 2023 03:50:22 -0700 (PDT) From: Naveen Mamindlapalli To: , , , , , , CC: Subbaraya Sundeep , Naveen Mamindlapalli Subject: [net-next PATCH 4/6] octeontx2-af: Enable LBK links only when switch mode is on. Date: Thu, 8 Jun 2023 16:20:05 +0530 Message-ID: <20230608105007.26924-5-naveenm@marvell.com> X-Mailer: git-send-email 2.39.0.198.ga38d39a4c5 In-Reply-To: <20230608105007.26924-1-naveenm@marvell.com> References: <20230608105007.26924-1-naveenm@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: F3XTPZ394zVy4PKVxqa49HKSE4Lfn5Rx X-Proofpoint-GUID: F3XTPZ394zVy4PKVxqa49HKSE4Lfn5Rx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-08_07,2023-06-08_01,2023-05-22_02 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org From: Subbaraya Sundeep Currently, all the TL3_TL2 nodes are being configured to enable switch LBK channel 63 in them. Instead enable them only when switch mode is enabled. Signed-off-by: Subbaraya Sundeep Signed-off-by: Naveen Mamindlapalli --- .../net/ethernet/marvell/octeontx2/af/rvu.h | 2 ++ .../ethernet/marvell/octeontx2/af/rvu_nix.c | 11 +++++------ .../ethernet/marvell/octeontx2/af/rvu_switch.c | 18 ++++++++++++++++++ 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index 54c3fa815e37..78c796fb2bb4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -789,6 +789,8 @@ int rvu_get_nix_blkaddr(struct rvu *rvu, u16 pcifunc); int nix_get_dwrr_mtu_reg(struct rvu_hwinfo *hw, int smq_link_type); u32 convert_dwrr_mtu_to_bytes(u8 dwrr_mtu); u32 convert_bytes_to_dwrr_mtu(u32 bytes); +void rvu_nix_tx_tl2_cfg(struct rvu *rvu, int blkaddr, u16 pcifunc, + struct nix_txsch *txsch, bool enable); /* NPC APIs */ void rvu_npc_freemem(struct rvu *rvu); diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c index 74793dd7d895..842ee9909af4 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c @@ -2394,17 +2394,19 @@ static int nix_txschq_cfg_read(struct rvu *rvu, struct nix_hw *nix_hw, return 0; } -static void rvu_nix_tx_tl2_cfg(struct rvu *rvu, int blkaddr, - u16 pcifunc, struct nix_txsch *txsch) +void rvu_nix_tx_tl2_cfg(struct rvu *rvu, int blkaddr, u16 pcifunc, + struct nix_txsch *txsch, bool enable) { struct rvu_hwinfo *hw = rvu->hw; int lbk_link_start, lbk_links; u8 pf = rvu_get_pf(pcifunc); int schq; + u64 cfg; if (!is_pf_cgxmapped(rvu, pf)) return; + cfg = enable ? (BIT_ULL(12) | RVU_SWITCH_LBK_CHAN) : 0; lbk_link_start = hw->cgx_links; for (schq = 0; schq < txsch->schq.max; schq++) { @@ -2418,8 +2420,7 @@ static void rvu_nix_tx_tl2_cfg(struct rvu *rvu, int blkaddr, rvu_write64(rvu, blkaddr, NIX_AF_TL3_TL2X_LINKX_CFG(schq, lbk_link_start + - lbk_links), - BIT_ULL(12) | RVU_SWITCH_LBK_CHAN); + lbk_links), cfg); } } @@ -2525,8 +2526,6 @@ int rvu_mbox_handler_nix_txschq_cfg(struct rvu *rvu, rvu_write64(rvu, blkaddr, reg, regval); } - rvu_nix_tx_tl2_cfg(rvu, blkaddr, pcifunc, - &nix_hw->txsch[NIX_TXSCH_LVL_TL2]); return 0; } diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_switch.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_switch.c index 3392487f6b47..592b317f4637 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_switch.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_switch.c @@ -8,6 +8,17 @@ #include #include "rvu.h" +static void rvu_switch_enable_lbk_link(struct rvu *rvu, u16 pcifunc, bool enable) +{ + struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc); + struct nix_hw *nix_hw; + + nix_hw = get_nix_hw(rvu->hw, pfvf->nix_blkaddr); + /* Enable LBK links with channel 63 for TX MCAM rule */ + rvu_nix_tx_tl2_cfg(rvu, pfvf->nix_blkaddr, pcifunc, + &nix_hw->txsch[NIX_TXSCH_LVL_TL2], enable); +} + static int rvu_switch_install_rx_rule(struct rvu *rvu, u16 pcifunc, u16 chan_mask) { @@ -52,6 +63,8 @@ static int rvu_switch_install_tx_rule(struct rvu *rvu, u16 pcifunc, u16 entry) if (!test_bit(NIXLF_INITIALIZED, &pfvf->flags)) return 0; + rvu_switch_enable_lbk_link(rvu, pcifunc, true); + lbkid = pfvf->nix_blkaddr == BLKADDR_NIX0 ? 0 : 1; ether_addr_copy(req.packet.dmac, pfvf->mac_addr); eth_broadcast_addr((u8 *)&req.mask.dmac); @@ -218,6 +231,9 @@ void rvu_switch_disable(struct rvu *rvu) "Reverting RX rule for PF%d failed(%d)\n", pf, err); + /* Disable LBK link */ + rvu_switch_enable_lbk_link(rvu, pcifunc, false); + rvu_get_pf_numvfs(rvu, pf, &numvfs, NULL); for (vf = 0; vf < numvfs; vf++) { pcifunc = pf << 10 | ((vf + 1) & 0x3FF); @@ -226,6 +242,8 @@ void rvu_switch_disable(struct rvu *rvu) dev_err(rvu->dev, "Reverting RX rule for PF%dVF%d failed(%d)\n", pf, vf, err); + + rvu_switch_enable_lbk_link(rvu, pcifunc, false); } } From patchwork Thu Jun 8 10:50:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naveen Mamindlapalli X-Patchwork-Id: 13271944 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B644111187 for ; Thu, 8 Jun 2023 10:50:41 +0000 (UTC) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E770E2D69; Thu, 8 Jun 2023 03:50:36 -0700 (PDT) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35887B8X023585; Thu, 8 Jun 2023 03:50:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=3yaqcdBg41d7mhWAoxcGyVDU3b5DBL/sd2gMwajYnRY=; b=KIXCVREOZ/ZZ/GFs7DJsZquDuxUby+JLkS67P82Vj7oZEn+ZdHbWLSenFogiBfIv69xA aV6ahY3h6nsZNHClsSXPgEnQkVPweiWFrok+XQIJ/1cBRJF8VocpAHmRU2PUYtJbCSLU RmBtY4ZJwGk9g44vVIE7b4toVM5erKM/UifzgVr8MvTMFC2Id8Up+A2vwvmuMxfrxXQH pO7zkRitrYzwBvESOvPo9nTVzLyBCj6W9dxqN11iYPh9sVFg1JVacLb6++ny7e83H+wR 03nz4TTrXO3oHhE0QlzroZumKOkc2Eio1wEkiwtkoCWWd7nfJKZteM5cn5ihmVPiChdY 5A== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3r329c2c3c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2023 03:50:30 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 8 Jun 2023 03:50:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 8 Jun 2023 03:50:28 -0700 Received: from #hyd1583.marvell.com (unknown [10.29.37.44]) by maili.marvell.com (Postfix) with ESMTP id 90D623F707F; Thu, 8 Jun 2023 03:50:25 -0700 (PDT) From: Naveen Mamindlapalli To: , , , , , , CC: Nithin Dabilpuram , Jerin Jacob Kollanukkaran , Naveen Mamindlapalli Subject: [net-next PATCH 5/6] octeontx2-af: add option to toggle DROP_RE enable in rx cfg Date: Thu, 8 Jun 2023 16:20:06 +0530 Message-ID: <20230608105007.26924-6-naveenm@marvell.com> X-Mailer: git-send-email 2.39.0.198.ga38d39a4c5 In-Reply-To: <20230608105007.26924-1-naveenm@marvell.com> References: <20230608105007.26924-1-naveenm@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-GUID: 6QVY8VUU-nGZ68CMqLXeJgI5jlPPdlD4 X-Proofpoint-ORIG-GUID: 6QVY8VUU-nGZ68CMqLXeJgI5jlPPdlD4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-08_07,2023-06-08_01,2023-05-22_02 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org From: Nithin Dabilpuram Add option to toggle DROP_RE bit in rx cfg mbox. This helps in modifying the config runtime as opposed to setting available via nix_lf_alloc() mbox at NIX LF init time. Signed-off-by: Nithin Dabilpuram Signed-off-by: Jerin Jacob Kollanukkaran Signed-off-by: Sunil Kovvuri Goutham Signed-off-by: Naveen Mamindlapalli --- drivers/net/ethernet/marvell/octeontx2/af/mbox.h | 1 + drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h index f233f98cbeea..3f30db28eff1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/mbox.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/mbox.h @@ -1151,6 +1151,7 @@ struct nix_rx_cfg { struct mbox_msghdr hdr; #define NIX_RX_OL3_VERIFY BIT(0) #define NIX_RX_OL4_VERIFY BIT(1) +#define NIX_RX_DROP_RE BIT(2) u8 len_verify; /* Outer L3/L4 len check */ #define NIX_RX_CSUM_OL4_VERIFY BIT(0) u8 csum_verify; /* Outer L4 checksum verification */ diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c index 842ee9909af4..18a146e9c4ef 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c @@ -4130,6 +4130,11 @@ int rvu_mbox_handler_nix_set_rx_cfg(struct rvu *rvu, struct nix_rx_cfg *req, else cfg &= ~BIT_ULL(40); + if (req->len_verify & NIX_RX_DROP_RE) + cfg |= BIT_ULL(32); + else + cfg &= ~BIT_ULL(32); + if (req->csum_verify & BIT(0)) cfg |= BIT_ULL(37); else From patchwork Thu Jun 8 10:50:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naveen Mamindlapalli X-Patchwork-Id: 13271945 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70F9811187 for ; Thu, 8 Jun 2023 10:50:48 +0000 (UTC) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4816E2D41; Thu, 8 Jun 2023 03:50:42 -0700 (PDT) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35838QYa023519; Thu, 8 Jun 2023 03:50:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=4cyWdPq2awzrKVLCMzcQ/M2cjI+AAxSFQEvtCVM6Dcs=; b=hYc2uF4ZdZTgPl2LZb6QLgXuuBO4Av2jrUTN6WvX6LzYM5j2uEw7Mzql1iXSrECn70fD WquqpCSuOw/pkCYI1I/FsmRQ/7HnhK6AqS/Xk7j7ZHmWaK4ywlONSLXjkonmldk0mX36 s6hCytiCbc0fdMOJkyvDdILkii1ZPjg/rr9a5CvAptI6yjBJkCTffhGrthLvJ6OYczEN K5lDhMcW50HUjVf7v/plk3Djour8YxvYy8oLBdK0uXh3OTphrEbImPFPw/ycLwY0nDQg F9eDqob/0CVSda4WCkEvO91v0zMVWlDvlxI0fAkjgVUSHTPgVn5MZI1fcYSnNRerb0by GA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3r30eu27dp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2023 03:50:33 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 8 Jun 2023 03:50:31 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 8 Jun 2023 03:50:31 -0700 Received: from #hyd1583.marvell.com (unknown [10.29.37.44]) by maili.marvell.com (Postfix) with ESMTP id 01CD33F707C; Thu, 8 Jun 2023 03:50:28 -0700 (PDT) From: Naveen Mamindlapalli To: , , , , , , CC: Naveen Mamindlapalli Subject: [net-next PATCH 6/6] octeontx2-af: Set XOFF on other child transmit schedulers during SMQ flush Date: Thu, 8 Jun 2023 16:20:07 +0530 Message-ID: <20230608105007.26924-7-naveenm@marvell.com> X-Mailer: git-send-email 2.39.0.198.ga38d39a4c5 In-Reply-To: <20230608105007.26924-1-naveenm@marvell.com> References: <20230608105007.26924-1-naveenm@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: FxoRhPXoPdjRf6ap7frjR4m4FBxdzeVJ X-Proofpoint-GUID: FxoRhPXoPdjRf6ap7frjR4m4FBxdzeVJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-08_07,2023-06-08_01,2023-05-22_02 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org When multiple transmit scheduler queues feed a TL1 transmit link, the SMQ flush initiated on a low priority queue might get stuck when a high priority queue fully subscribes the transmit link. This inturn effects interface teardown. To avoid this, temporarily XOFF all TL1's other immediate child transmit scheduler queues and also clear any rate limit configuration on all the scheduler queues in SMQ(flush) hierarchy. Signed-off-by: Naveen Mamindlapalli Signed-off-by: Sunil Goutham --- .../net/ethernet/marvell/octeontx2/af/rvu.h | 16 +++ .../ethernet/marvell/octeontx2/af/rvu_nix.c | 130 +++++++++++++++++- 2 files changed, 144 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index 78c796fb2bb4..8a825b983320 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -283,6 +283,22 @@ struct nix_mark_format { u32 *cfg; }; +/* smq(flush) to tl1 cir/pir info */ +struct nix_smq_tree_ctx { + u64 cir_off; + u64 cir_val; + u64 pir_off; + u64 pir_val; +}; + +/* smq flush context */ +struct nix_smq_flush_ctx { + int smq; + u16 tl1_schq; + u16 tl2_schq; + struct nix_smq_tree_ctx smq_tree_ctx[NIX_TXSCH_LVL_CNT]; +}; + struct npc_pkind { struct rsrc_bmap rsrc; u32 *pfchan_map; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c index 18a146e9c4ef..eb5c11d06d7f 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c @@ -2062,9 +2062,121 @@ int rvu_mbox_handler_nix_txsch_alloc(struct rvu *rvu, return rc; } +static void nix_smq_flush_fill_ctx(struct rvu *rvu, int blkaddr, int smq, + struct nix_smq_flush_ctx *smq_flush_ctx) +{ + struct nix_smq_tree_ctx *smq_tree_ctx; + u64 parent_off, regval; + u16 schq; + int lvl; + + smq_flush_ctx->smq = smq; + + schq = smq; + for (lvl = NIX_TXSCH_LVL_SMQ; lvl <= NIX_TXSCH_LVL_TL1; lvl++) { + smq_tree_ctx = &smq_flush_ctx->smq_tree_ctx[lvl]; + if (lvl == NIX_TXSCH_LVL_TL1) { + smq_flush_ctx->tl1_schq = schq; + smq_tree_ctx->cir_off = NIX_AF_TL1X_CIR(schq); + smq_tree_ctx->pir_off = 0; + smq_tree_ctx->pir_val = 0; + parent_off = 0; + } else if (lvl == NIX_TXSCH_LVL_TL2) { + smq_flush_ctx->tl2_schq = schq; + smq_tree_ctx->cir_off = NIX_AF_TL2X_CIR(schq); + smq_tree_ctx->pir_off = NIX_AF_TL2X_PIR(schq); + parent_off = NIX_AF_TL2X_PARENT(schq); + } else if (lvl == NIX_TXSCH_LVL_TL3) { + smq_tree_ctx->cir_off = NIX_AF_TL3X_CIR(schq); + smq_tree_ctx->pir_off = NIX_AF_TL3X_PIR(schq); + parent_off = NIX_AF_TL3X_PARENT(schq); + } else if (lvl == NIX_TXSCH_LVL_TL4) { + smq_tree_ctx->cir_off = NIX_AF_TL4X_CIR(schq); + smq_tree_ctx->pir_off = NIX_AF_TL4X_PIR(schq); + parent_off = NIX_AF_TL4X_PARENT(schq); + } else if (lvl == NIX_TXSCH_LVL_MDQ) { + smq_tree_ctx->cir_off = NIX_AF_MDQX_CIR(schq); + smq_tree_ctx->pir_off = NIX_AF_MDQX_PIR(schq); + parent_off = NIX_AF_MDQX_PARENT(schq); + } + /* save cir/pir register values */ + smq_tree_ctx->cir_val = rvu_read64(rvu, blkaddr, smq_tree_ctx->cir_off); + if (smq_tree_ctx->pir_off) + smq_tree_ctx->pir_val = rvu_read64(rvu, blkaddr, smq_tree_ctx->pir_off); + + /* get parent txsch node */ + if (parent_off) { + regval = rvu_read64(rvu, blkaddr, parent_off); + schq = (regval >> 16) & 0x1FF; + } + } +} + +static void nix_smq_flush_enadis_xoff(struct rvu *rvu, int blkaddr, + struct nix_smq_flush_ctx *smq_flush_ctx, bool enable) +{ + struct nix_txsch *txsch; + struct nix_hw *nix_hw; + u64 regoff; + int tl2; + + nix_hw = get_nix_hw(rvu->hw, blkaddr); + if (!nix_hw) + return; + + /* loop through all TL2s with matching PF_FUNC */ + txsch = &nix_hw->txsch[NIX_TXSCH_LVL_TL2]; + for (tl2 = 0; tl2 < txsch->schq.max; tl2++) { + /* skip the smq(flush) TL2 */ + if (tl2 == smq_flush_ctx->tl2_schq) + continue; + /* skip unused TL2s */ + if (TXSCH_MAP_FLAGS(txsch->pfvf_map[tl2]) & NIX_TXSCHQ_FREE) + continue; + /* skip if PF_FUNC doesn't match */ + if ((TXSCH_MAP_FUNC(txsch->pfvf_map[tl2]) & ~RVU_PFVF_FUNC_MASK) != + (TXSCH_MAP_FUNC(txsch->pfvf_map[smq_flush_ctx->tl2_schq] & + ~RVU_PFVF_FUNC_MASK))) + continue; + /* enable/disable XOFF */ + regoff = NIX_AF_TL2X_SW_XOFF(tl2); + if (enable) + rvu_write64(rvu, blkaddr, regoff, 0x1); + else + rvu_write64(rvu, blkaddr, regoff, 0x0); + } +} + +static void nix_smq_flush_enadis_rate(struct rvu *rvu, int blkaddr, + struct nix_smq_flush_ctx *smq_flush_ctx, bool enable) +{ + u64 cir_off, pir_off, cir_val, pir_val; + struct nix_smq_tree_ctx *smq_tree_ctx; + int lvl; + + for (lvl = NIX_TXSCH_LVL_SMQ; lvl <= NIX_TXSCH_LVL_TL1; lvl++) { + smq_tree_ctx = &smq_flush_ctx->smq_tree_ctx[lvl]; + cir_off = smq_tree_ctx->cir_off; + cir_val = smq_tree_ctx->cir_val; + pir_off = smq_tree_ctx->pir_off; + pir_val = smq_tree_ctx->pir_val; + + if (enable) { + rvu_write64(rvu, blkaddr, cir_off, cir_val); + if (lvl != NIX_TXSCH_LVL_TL1) + rvu_write64(rvu, blkaddr, pir_off, pir_val); + } else { + rvu_write64(rvu, blkaddr, cir_off, 0x0); + if (lvl != NIX_TXSCH_LVL_TL1) + rvu_write64(rvu, blkaddr, pir_off, 0x0); + } + } +} + static int nix_smq_flush(struct rvu *rvu, int blkaddr, int smq, u16 pcifunc, int nixlf) { + struct nix_smq_flush_ctx *smq_flush_ctx; int pf = rvu_get_pf(pcifunc); u8 cgx_id = 0, lmac_id = 0; int err, restore_tx_en = 0; @@ -2077,6 +2189,14 @@ static int nix_smq_flush(struct rvu *rvu, int blkaddr, lmac_id, true); } + /* XOFF all TL2s whose parent TL1 matches SMQ tree TL1 */ + smq_flush_ctx = kzalloc(sizeof(*smq_flush_ctx), GFP_KERNEL); + if (!smq_flush_ctx) + return -ENOMEM; + nix_smq_flush_fill_ctx(rvu, blkaddr, smq, smq_flush_ctx); + nix_smq_flush_enadis_xoff(rvu, blkaddr, smq_flush_ctx, true); + nix_smq_flush_enadis_rate(rvu, blkaddr, smq_flush_ctx, false); + cfg = rvu_read64(rvu, blkaddr, NIX_AF_SMQX_CFG(smq)); /* Do SMQ flush and set enqueue xoff */ cfg |= BIT_ULL(50) | BIT_ULL(49); @@ -2091,8 +2211,14 @@ static int nix_smq_flush(struct rvu *rvu, int blkaddr, err = rvu_poll_reg(rvu, blkaddr, NIX_AF_SMQX_CFG(smq), BIT_ULL(49), true); if (err) - dev_err(rvu->dev, - "NIXLF%d: SMQ%d flush failed\n", nixlf, smq); + dev_info(rvu->dev, + "NIXLF%d: SMQ%d flush failed, txlink might be busy\n", + nixlf, smq); + + /* clear XOFF on TL2s */ + nix_smq_flush_enadis_rate(rvu, blkaddr, smq_flush_ctx, true); + nix_smq_flush_enadis_xoff(rvu, blkaddr, smq_flush_ctx, false); + kfree(smq_flush_ctx); rvu_cgx_enadis_rx_bp(rvu, pf, true); /* restore cgx tx state */