From patchwork Thu Feb 13 01:12:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972654 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB7D47DA6C for ; Thu, 13 Feb 2025 01:13:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409208; cv=none; b=C1XXPx1Oi5FYulKeXMRf6JfBWS4YdJa8v+2KbFBPAiETCOjpk+IseiH2yZVYY92+4o0K8Ve28wD/l/0vWFOqc0seJleuha9vqyQ5tf6seTwKmbYJ3aIREEtL4TuamPFqCSEZQtcJbCs1jxZfqsi1ni5jMv7u3oU1+Qa0oZe7mdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409208; c=relaxed/simple; bh=IkC5DrSpwBzYVtxuuXtzRppK51yx82yjGQY9KwH7jcs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OQDQAxT4gNEHQRh3PCbGRCfA9QUXmqKbOZHLx0ITsynvtpjqixXqQtrZsVvU7yK9bDw/5I9/H7Irp1VDN3jhRmngBFtluHZdNGqpWqN5gFIF7g3WVtr6jZ2AaF3NuRjMUw0gix8tUtuzHCRESTh6CERoUDfkXjhF/4F1i+d0NBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=LRxB+RgI; arc=none smtp.client-ip=209.85.210.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="LRxB+RgI" Received: by mail-ot1-f41.google.com with SMTP id 46e09a7af769-726cdf7541eso158204a34.1 for ; Wed, 12 Feb 2025 17:13:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409206; x=1740014006; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Xs0HqGMlTz4yprx8RCINcEsYohiRoMzfTFTjyXVzrPs=; b=LRxB+RgIp8cQZ18AkrK0eM0fcsWze2PYT/5Cr7GZf7siPyjminFvht90ttQOmEK5PA bvgqwvjqTTVjFxmmFsuPKfW6tuUjB9JCsXnqg1ywBamTzLHsk0OoFhvTea1xGERT4A7M yvQDr1BRS2GaWLwAjOfByZqGui8jGlz9cbWZM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409206; x=1740014006; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xs0HqGMlTz4yprx8RCINcEsYohiRoMzfTFTjyXVzrPs=; b=Nt/caCMkjlOyO0r/2pc5Tg7hSOQNbVaDns2UJpDw8mxnzGvCYr71rcymPdrloVcWwo fSXqZb4H7tUfEhIS56iG4J2TOruUFEsRLGjZBYkLr6Jtql4LVRgkX6ujWhq/+PA0wRpJ xU7y0Nvf6UVcB72dAm5/009UCd48OwbdOO+0dfhv1iaJj7TN8fXIKjYc4ePB4z85zlrb pQkfrK7mJaAVAqcEfQ+P8k4HvIBvzG+mkFpkau42RBjeXmJZmdibhwj2e74OpuRy0XPA VZ+jAzx0PJVLHoNHxrrRnJQrm+AJX1c3wTZgBzsOp7TGpOUabNNSFaapY//O5tHyv0+F zVpA== X-Gm-Message-State: AOJu0YwkSqk0lqKnbhbH0jltFwPI376toabZggk9c2sS54uDGJRUpBvk vi5CniYn7CXDEVz6UxCFSo5szvQDan2l5FaJ2AVngPgIqRntZS/ufU2uyyp4Rg== X-Gm-Gg: ASbGncuLcPaQDGRHbUkDY7tXXLWr4qwkEuyn086PkDc0ToVuiBpZfdIejROeC+C/WEI qZrBljgHxFjqyFd1Le+3u/DvsBt62usPI4B6Uf8d8Dofo4/g0tcEY3EbM99qjPnjXTiFTAr1IIG btEzjhcZJdwFtVcNLC2fwPt4c5IsUinCOyDkkwOrIUGAYzbKdMV/6rGECdJpWTsTD2jVuPlvyGv QxGh6cR5Svr4nL3M18ybIDhcHEZ+c11M1eusjP+WYiUXzMkwYcdTdV7n8S3fsGAPPZ72jGzAMYl JptCl/vRf1T961JwnYNIqEfkI3BK2Pw+38NTQoRdA4bxCjGfbNWP52ASYFWZmYszYj4= X-Google-Smtp-Source: AGHT+IF/90JA6hBCg9mTROhXerY/WnHdDUfq+jLIX9BjVR5CxSdS8gMp97PBgBoUcYKw4CfxvBzv7g== X-Received: by 2002:a05:6830:6e8c:b0:71e:15f7:1a3a with SMTP id 46e09a7af769-726fefd3a0emr603731a34.2.1739409205848; Wed, 12 Feb 2025 17:13:25 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:24 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Somnath Kotur Subject: [PATCH net-next v5 01/11] bnxt_en: Set NPAR 1.2 support when registering with firmware Date: Wed, 12 Feb 2025 17:12:29 -0800 Message-ID: <20250213011240.1640031-2-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org NPAR (Network interface card partitioning)[1] 1.2 adds a transparent VLAN tag for all packets between the NIC and the switch. Because of that, RX VLAN acceleration cannot be supported for any additional host configured VLANs. The driver has to acknowledge that it can support no RX VLAN acceleration and set the NPAR 1.2 supported flag when registering with the FW. Otherwise, the FW call will fail and the driver will abort on these NPAR 1.2 NICs with this error: bnxt_en 0000:26:00.0 (unnamed net_device) (uninitialized): hwrm req_type 0x1d seq id 0xb error 0x2 [1] https://techdocs.broadcom.com/us/en/storage-and-ethernet-connectivity/ethernet-nic-controllers/bcm957xxx/adapters/introduction/features/network-partitioning-npar.html Reviewed-by: Somnath Kotur Reviewed-by: Michal Swiatkowski Signed-off-by: Michael Chan --- v4: Fix typo in NPAR and improve the description of NPAR --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7b8b5b39c7bb..fc870c104c56 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -5565,6 +5565,8 @@ int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp, unsigned long *bmap, int bmap_size, if (bp->fw_cap & BNXT_FW_CAP_ERROR_RECOVERY) flags |= FUNC_DRV_RGTR_REQ_FLAGS_ERROR_RECOVERY_SUPPORT | FUNC_DRV_RGTR_REQ_FLAGS_MASTER_SUPPORT; + if (bp->fw_cap & BNXT_FW_CAP_NPAR_1_2) + flags |= FUNC_DRV_RGTR_REQ_FLAGS_NPAR_1_2_SUPPORT; req->flags = cpu_to_le32(flags); req->ver_maj_8b = DRV_VER_MAJ; req->ver_min_8b = DRV_VER_MIN; @@ -8365,6 +8367,7 @@ static int bnxt_hwrm_func_qcfg(struct bnxt *bp) switch (resp->port_partition_type) { case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_0: + case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_2: case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR1_5: case FUNC_QCFG_RESP_PORT_PARTITION_TYPE_NPAR2_0: bp->port_partition_type = resp->port_partition_type; @@ -9529,6 +9532,8 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp) bp->fw_cap |= BNXT_FW_CAP_HOT_RESET_IF; if (BNXT_PF(bp) && (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_FW_LIVEPATCH_SUPPORTED)) bp->fw_cap |= BNXT_FW_CAP_LIVEPATCH; + if (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_NPAR_1_2_SUPPORTED) + bp->fw_cap |= BNXT_FW_CAP_NPAR_1_2; if (BNXT_PF(bp) && (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_DFLT_VLAN_TPID_PCP_SUPPORTED)) bp->fw_cap |= BNXT_FW_CAP_DFLT_VLAN_TPID_PCP; if (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_BS_V2_SUPPORTED) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 2373f423a523..db0be469a3db 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2492,6 +2492,7 @@ struct bnxt { #define BNXT_FW_CAP_CFA_RFS_RING_TBL_IDX_V3 BIT_ULL(39) #define BNXT_FW_CAP_VNIC_RE_FLUSH BIT_ULL(40) #define BNXT_FW_CAP_SW_MAX_RESOURCE_LIMITS BIT_ULL(41) + #define BNXT_FW_CAP_NPAR_1_2 BIT_ULL(42) u32 fw_dbg_cap; From patchwork Thu Feb 13 01:12:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972655 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C81BF12D758 for ; Thu, 13 Feb 2025 01:13:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409210; cv=none; b=oDEAqO7UkLCr8yp8oXwBz2ZGTK+EbYqCbN3f7IY9T21fFOT+KEA4JXxzq4OTzzXPpbIugGWOYUZ2sKh4ADBmPLJpU0Z7GgUvP+/u+2Gt1TrvzMEL/Oe/y4h2Pc5htIDY51g0NyQj66c6clR0Y8ZF0PnXy4E4ooCqFe+YTn3OcIU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409210; c=relaxed/simple; bh=798C9bVl/2f9vmMwMenfvnnbRYCzOvXYML/rPtxBmWQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ElXYbNMc7/51LR5Qhc1dLnRrOSugEqDYXLeH5eeSQ9Jxc2YXxoSZY0DgZL55gj4AjmOwOzQpdM5n4gMaS+KjDry24ksvG5Bjg9U4TL9K0f5EbbJ6aL9CTNMYC+gmiOSIhE2rnWEpuj4XHOihZE2SgFefYDDlHKNLHDvKIOS7rAI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=gqYGk9Ix; arc=none smtp.client-ip=209.85.210.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="gqYGk9Ix" Received: by mail-ot1-f48.google.com with SMTP id 46e09a7af769-726f989dc12so214201a34.0 for ; Wed, 12 Feb 2025 17:13:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409208; x=1740014008; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BGlMPNT2LmAYPrCmYwn48Iozoi1TKQ77bAj5ygSrkAE=; b=gqYGk9IxAX1NyUMTWy211MJh2bpXU9y0OEk+Fn7Jab1IeQ2tBb5xgUSdbkXA+Xy3Y3 XFWLp0SSf3+rqs5rFBb0qiNZRNAUkLdx3JIfpSoNecoNHRHaNDH9FQ0AgS4xR9o34BuE SnCPvACk5fGU/XjqYjr1lTFJ071a2xVIOwEaY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409208; x=1740014008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BGlMPNT2LmAYPrCmYwn48Iozoi1TKQ77bAj5ygSrkAE=; b=P7M29aD0kq/vJpklNCCBEfrH1IXBIUY11b5H9XQnju+7EJBNDseRP1DyYUPjzkszQY QDiVFL9fK7O48DqENoL7Yec0tuGoazChrgDyPRv0AnAui1YOFjPA06uvvxRWkdb9+bj8 iiNrErf6f09QCtPTGVeoz7jyuPqGPQ2ukzGxo2kT5IpX0NGj5SoTH0tigUpocLt0djx6 1I5TFGJTaevMBLlp8cokavGwYyx6kUP1aHl6m3zwNRSKF3REIGGwQ3i/YyzDO4AD0WnJ oTa1eBgVD5RqBRg/itZTfMgvJUSz/tp27+QgL7Cr7d/q9GaNc4QQ9CgkD96In6GI9/l2 xK2A== X-Gm-Message-State: AOJu0YyR75BtFHHItkD/+LfpfZJxiQC9QRzKgbxLJMM/gJCvxoTPn3JK idtdv+dpUYwEQ2kUJU6RqMXF96kNebDf/+qxNmZa7h0qgyy+popkEOGcQVEm9g== X-Gm-Gg: ASbGnctaf80frC8hwSG/BSe4dWSQT4vsRg6bxHjvJl2W6jEWbCPRu4gZCyK0itKH0Lh RopRfFs1eDQYFdqLkuRrLFOeyn3NEvJjfmsesyfyfteterULd/JFXbvoaUcdthJI69e5+exSWfM CqOhu5AzaezXvF4m/HC8eFt1Jmxj5CBwS/yLGyY9ztqenaW++ZqAso8B95NBo4wMt9qeVdMTtEE QwVoYtBws4Ze2HqMZsKbvzFUp0kcnEFBzsOpdMydauyl4UPxQ4FBO9UmV+B+Md0J7rSnaaoqptW JPx4sUNqhlfbngrKjY3sVf2+b2S1j9pzJ2S8BF2ovck9fCjBpJd+wec3CSMPALRDykU= X-Google-Smtp-Source: AGHT+IERHtrdym+a+md7axIghGcxq3RiGaTbvxFTMe5x1uxPrJ/XnrL+3vA0wTIn/nEGPAckIupSWQ== X-Received: by 2002:a05:6808:3505:b0:3f3:bd0b:e0c7 with SMTP id 5614622812f47-3f3d8da2607mr756128b6e.8.1739409207796; Wed, 12 Feb 2025 17:13:27 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:26 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Ajit Khaparde Subject: [PATCH net-next v5 02/11] bnxt_en: Refactor completion ring allocation logic for P5_PLUS chips Date: Wed, 12 Feb 2025 17:12:30 -0800 Message-ID: <20250213011240.1640031-3-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Add a new bnxt_hwrm_cp_ring_alloc_p5() function to handle allocating one completion ring on P5_PLUS chips. This simplifies the existing code and will be useful later in the series. Reviewed-by: Ajit Khaparde Reviewed-by: Michal Swiatkowski Signed-off-by: Michael Chan --- v2: Use const for a variable --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 44 +++++++++++------------ 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index fc870c104c56..0e16ea823fbd 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7199,6 +7199,25 @@ static int bnxt_hwrm_rx_agg_ring_alloc(struct bnxt *bp, return 0; } +static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp, + struct bnxt_cp_ring_info *cpr) +{ + const u32 type = HWRM_RING_ALLOC_CMPL; + struct bnxt_napi *bnapi = cpr->bnapi; + struct bnxt_ring_struct *ring; + u32 map_idx = bnapi->index; + int rc; + + ring = &cpr->cp_ring_struct; + ring->handle = BNXT_SET_NQ_HDL(cpr); + rc = hwrm_ring_alloc_send_msg(bp, ring, type, map_idx); + if (rc) + return rc; + bnxt_set_db(bp, &cpr->cp_db, type, map_idx, ring->fw_ring_id); + bnxt_db_cq(bp, &cpr->cp_db, cpr->cp_raw_cons); + return 0; +} + static int bnxt_hwrm_ring_alloc(struct bnxt *bp) { bool agg_rings = !!(bp->flags & BNXT_FLAG_AGG_RINGS); @@ -7242,19 +7261,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp) u32 map_idx; if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { - struct bnxt_cp_ring_info *cpr2 = txr->tx_cpr; - struct bnxt_napi *bnapi = txr->bnapi; - u32 type2 = HWRM_RING_ALLOC_CMPL; - - ring = &cpr2->cp_ring_struct; - ring->handle = BNXT_SET_NQ_HDL(cpr2); - map_idx = bnapi->index; - rc = hwrm_ring_alloc_send_msg(bp, ring, type2, map_idx); + rc = bnxt_hwrm_cp_ring_alloc_p5(bp, txr->tx_cpr); if (rc) goto err_out; - bnxt_set_db(bp, &cpr2->cp_db, type2, map_idx, - ring->fw_ring_id); - bnxt_db_cq(bp, &cpr2->cp_db, cpr2->cp_raw_cons); } ring = &txr->tx_ring_struct; map_idx = i; @@ -7274,20 +7283,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp) if (!agg_rings) bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { - struct bnxt_cp_ring_info *cpr2 = rxr->rx_cpr; - struct bnxt_napi *bnapi = rxr->bnapi; - u32 type2 = HWRM_RING_ALLOC_CMPL; - struct bnxt_ring_struct *ring; - u32 map_idx = bnapi->index; - - ring = &cpr2->cp_ring_struct; - ring->handle = BNXT_SET_NQ_HDL(cpr2); - rc = hwrm_ring_alloc_send_msg(bp, ring, type2, map_idx); + rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr); if (rc) goto err_out; - bnxt_set_db(bp, &cpr2->cp_db, type2, map_idx, - ring->fw_ring_id); - bnxt_db_cq(bp, &cpr2->cp_db, cpr2->cp_raw_cons); } } From patchwork Thu Feb 13 01:12:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972656 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0A4213BAF1 for ; Thu, 13 Feb 2025 01:13:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409215; cv=none; b=P8oRLCBF9cwUwhItnkoMzLl4UXm3kz1V1KcvOobrxzDcH6DqKOE7k7p9gbho8Bu6RPzwH4xkwLhelnccFlhSiLIkOChBp/0ala9ujFrynI9F+6kHcI7dIbZxuJhSCe983FoEuHJkAfQR7z+SNZfEq4r1n7rIC5P4MFQOniBtL9s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409215; c=relaxed/simple; bh=N4tPBq96MNEhuP9E/BsZZBcc9KAk6IUhpymzyYeQMxk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LC3Hosua+dXtkez8fbTjdbSN1b5ot0Kf7C7UA4uLwr2stTpV87DxNwsMrwbFgh4RVmMksJXYqOF+tp+pESgmuvqrRPPWVF9F5IDGiVVWV+rQnBrUAeeEQdla056uY0DewPt6cFLpljg6RC5MSe2SMgq5QnItj4ZuZ0yvLF3Ufjw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=X0YRHgGT; arc=none smtp.client-ip=209.85.210.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="X0YRHgGT" Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-726f943b6f4so130045a34.0 for ; Wed, 12 Feb 2025 17:13:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409210; x=1740014010; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cAnNmS1PpxNhoASBZ/NEPmOmzVWH3c4fcg+geDwoOGo=; b=X0YRHgGTLJN31hETlnG1AE1XjdCTITPaQEvP9JSxHjj4Jvl6hj0QAIzIxNqec0ELFR 34GDQ99CjPjQur2gR4mVHLwbd1m1VIqODtgyRVmt6c56ByNhJfbd/GCfLVswXqiwR3PX RGuD+sAxDGvV6hvWnQWEoQBaYfTWoWzkqLmCA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409210; x=1740014010; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cAnNmS1PpxNhoASBZ/NEPmOmzVWH3c4fcg+geDwoOGo=; b=h6+5rywojO8H9U/+o/LajdM1tGDNXfcsStLqrpyyLj0NMvUitQJN5R30109/Pr2nFp FC5SAqOJR5w3R/nlETl9nXZTQkdfxWUMpPSNx4lKFqiZHbE6VFFClgbtrSCRb0izLuwm op80gDM2nN6S91xCLOCgG262BKvb57T5ooE00NglhbhPtohiHALQQf6RvZquaRX6caBo 00onMEQ8GvxDGzrXJXgkxWyNpw6sz12B7C9qpc8r4FzJnwa37YZHM/T4lJWO8zmk7ykV /nTJHJF4LMFdT5Z2aOSyjQSM5dVNiv7g3j4PWU7itb16YmSCdklZou7yYDa/lS9IWK5Y HKow== X-Gm-Message-State: AOJu0YzPK59NLV0ajSC3/so8cnW4fgOhqIXkAYJarY2AlWQunLFzyJTD vIAB6RyEOiDf53VTgzsOoG2mBx2UKPgcV5EcWsdP+u3E6SQxyjA+Gp/VcXYH8w== X-Gm-Gg: ASbGncsoKdYKRQvwLtmHVIHKe4gugWB1zvYmsOGpWprMpVDvBWVnuNlKMnvH0xA5eYH kjscC1DEkN6Zy6x+k3GYHJGoFbnfhEwuHbg7BGraJ+Pmv4GJUAA9cQ7mFXne7KHi6vyNDSYI7co QstDfdqBAB14SEv2hAr9Y3Q3pbw9dzyIEraICw+hXHeCB1lnkYNkkmpQz/l6VV9y4ge86gZWH+U p/TdsdjuQlLIJdkj4Y2Jxa4uSdoqVZthVV3BUhEkVTQ1CiStXY+lCnDTHtqY66gH3K5exrajLaa cBLDwitX3iJgOAu+KA3ArD4X9yxzaQFTFmet4GhAg7uGBp2qHp5Sx4Jm+PqdQwgckgQ= X-Google-Smtp-Source: AGHT+IHOmzYRB3mHEOl2Yb2lOg3rtjKR5mQBbkRYGnYTmXjixbczwuwVoP2jtQ8Ib08grBd/0/1sag== X-Received: by 2002:a05:6830:f8a:b0:718:7c3:f86a with SMTP id 46e09a7af769-726fe758cebmr857175a34.6.1739409209809; Wed, 12 Feb 2025 17:13:29 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:28 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Ajit Khaparde Subject: [PATCH net-next v5 03/11] bnxt_en: Refactor TX ring allocation logic Date: Wed, 12 Feb 2025 17:12:31 -0800 Message-ID: <20250213011240.1640031-4-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Add a new bnxt_hwrm_tx_ring_alloc() function to handle allocating a transmit ring. This will be useful later in the series. Reviewed-by: Ajit Khaparde Reviewed-by: Michal Swiatkowski Signed-off-by: Michael Chan --- v2: Use const for a variable --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 0e16ea823fbd..8ab7345acb0a 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7218,6 +7218,20 @@ static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp, return 0; } +static int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, + struct bnxt_tx_ring_info *txr, u32 tx_idx) +{ + struct bnxt_ring_struct *ring = &txr->tx_ring_struct; + const u32 type = HWRM_RING_ALLOC_TX; + int rc; + + rc = hwrm_ring_alloc_send_msg(bp, ring, type, tx_idx); + if (rc) + return rc; + bnxt_set_db(bp, &txr->tx_db, type, tx_idx, ring->fw_ring_id); + return 0; +} + static int bnxt_hwrm_ring_alloc(struct bnxt *bp) { bool agg_rings = !!(bp->flags & BNXT_FLAG_AGG_RINGS); @@ -7254,23 +7268,17 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp) } } - type = HWRM_RING_ALLOC_TX; for (i = 0; i < bp->tx_nr_rings; i++) { struct bnxt_tx_ring_info *txr = &bp->tx_ring[i]; - struct bnxt_ring_struct *ring; - u32 map_idx; if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { rc = bnxt_hwrm_cp_ring_alloc_p5(bp, txr->tx_cpr); if (rc) goto err_out; } - ring = &txr->tx_ring_struct; - map_idx = i; - rc = hwrm_ring_alloc_send_msg(bp, ring, type, map_idx); + rc = bnxt_hwrm_tx_ring_alloc(bp, txr, i); if (rc) goto err_out; - bnxt_set_db(bp, &txr->tx_db, type, map_idx, ring->fw_ring_id); } for (i = 0; i < bp->rx_nr_rings; i++) { From patchwork Thu Feb 13 01:12:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972657 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2259781AC8 for ; Thu, 13 Feb 2025 01:13:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409215; cv=none; b=FzR7egoAZ2T6S7+qjScw0DJrovI5KOadUQ4PBb+zF7KgC2ayne7RWBe9C4G6kWqr2AcFG2RlgC+5BchGLxZtS1bBAKWMhtPy/eVdbDbummPB2EfVeAHdTDr9xii0hC2Tk/WLdHR3zmkEsZQ8xgOchddF7bNPBiwSrimPjPNhRjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409215; c=relaxed/simple; bh=3Wv6fRui6JUppkVHVPDhuyoEourXEdFIz2Ar68Mhcgo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mefaDLiAJhyWQM7E7ghm5bNISN4/QZGGLZ4BNNDeEFLnj587UEI00h490rGahGa0MB5Tu3YDoZAGxFK8IfBsvFqVoSuMQV/XgPPe7uDb87iyQh6DhCKimuo2MAjH7vNzHEiSjga5pxaDr+8wubOHptb1iWr3MZexKpxVEzDTwAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=h6xR1/3t; arc=none smtp.client-ip=209.85.210.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="h6xR1/3t" Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-726ed359c56so318513a34.3 for ; Wed, 12 Feb 2025 17:13:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409213; x=1740014013; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tYbE7VuTl9BMV8ls+ftV23iDp1ujBGiTUNoHs2AfpuI=; b=h6xR1/3t6N66dwTLwLGvzi5TDsV0DqDmjh3gJp/Jjm5Dnech4O9rCh85/aQrsqd4F1 vsre7b/JmQbRqv5PRdyjH+6sR1A1gO9cZPDMFQ4e1A5+8ehfLM7LIthHU+TpbQG01FD+ J4aKrfHnvZDrPgpkUjrcdLjX24VPaQO8S+ot8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409213; x=1740014013; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tYbE7VuTl9BMV8ls+ftV23iDp1ujBGiTUNoHs2AfpuI=; b=Lsrd+pYMQzm2qIReIkF8av5M2ERY3FOKZ62TVHOCxvrs7noV214vsrSOgbl6BecynD rtBjNrI3AAWPia5PtC0Uv6Br6LW+f+uHPbD35vgmY2aTDp8L+6WfC2Lfcvlgyi49skZG q1WMoDh1JTpy8h1/v6AKHTNELfXYkz0Bd0juqrqK46HNymaoYecO4LrEqtqW14DZs5Sv WX55pWrIHo4MpEJYAJ1jBfbTbwZUPVt07ixoT4goIpVOSIMlbSs+/dg76wpLMsU7mIBA YTc7q0SgLNhIf65R2nzN5Mu2MjNGDHQf2CrCCuUzVErm8oVqOEBW5S0+WjN6clRFoACx SkNw== X-Gm-Message-State: AOJu0Yzb4SEtskt6v4n+2xUs25mtdx05eYZxIPs1wjG1+WnHinSsQQO+ Zw0aW8craLrkt5jFvwIFzOR/AiBORXq4ys+Le7GkQBbKZd+GvfzrWHAH0x5KMw== X-Gm-Gg: ASbGnct55nsII08vgMwJ3btJLz++zXIbEqm5Zom8nE5cZIVTexhYAV+OzG5hGsvcZqr F8jB/GDpviZ1RaOzK/x6ADT4EjHRBuFu7Ppbf50KqO6iR1L9EtmOvjuL1YIGnF3k2YyLA10xLrK Xq+QSQRQTzWPPKNAvqTiPjY7c1CBwyWEN/0uxJssqc+tIBlO2mwdcfNFeCWV5Xboq0kF1x9ar4n O5of2xmhTGSbpG/1xKPwFQdViS8R4lLemvOomJjpJfTPhUIPiMjEO7EqfEZRbKe9BVsK/YcEKSj QsABL3HmHWpKZz/RjyBL4rO+B3MjeBz/ejGJGu1Umip9ALNbBHQIz1I/yMmYpYGgeFA= X-Google-Smtp-Source: AGHT+IH1X0NNE3pqbZwjNuV2aealYqQRH0pfK9uL7qenIbiPX003kYK+5F5tzCO/7XdOl4XScNL+Ag== X-Received: by 2002:a05:6830:3148:b0:727:bf:e433 with SMTP id 46e09a7af769-72700bfe60cmr183967a34.19.1739409211762; Wed, 12 Feb 2025 17:13:31 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:30 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Somnath Kotur , Kalesh AP Subject: [PATCH net-next v5 04/11] bnxt_en: Refactor completion ring free routine Date: Wed, 12 Feb 2025 17:12:32 -0800 Message-ID: <20250213011240.1640031-5-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Somnath Kotur Add a wrapper routine to free L2 completion rings. This will be useful later in the series. Reviewed-by: Kalesh AP Reviewed-by: Michal Swiatkowski Signed-off-by: Somnath Kotur Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 26 ++++++++++++++--------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 8ab7345acb0a..52d4dc222759 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7405,6 +7405,20 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, bp->grp_info[grp_idx].agg_fw_ring_id = INVALID_HW_RING_ID; } +static void bnxt_hwrm_cp_ring_free(struct bnxt *bp, + struct bnxt_cp_ring_info *cpr) +{ + struct bnxt_ring_struct *ring; + + ring = &cpr->cp_ring_struct; + if (ring->fw_ring_id == INVALID_HW_RING_ID) + return; + + hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_L2_CMPL, + INVALID_HW_RING_ID); + ring->fw_ring_id = INVALID_HW_RING_ID; +} + static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) { u32 type; @@ -7450,17 +7464,9 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) struct bnxt_ring_struct *ring; int j; - for (j = 0; j < cpr->cp_ring_count && cpr->cp_ring_arr; j++) { - struct bnxt_cp_ring_info *cpr2 = &cpr->cp_ring_arr[j]; + for (j = 0; j < cpr->cp_ring_count && cpr->cp_ring_arr; j++) + bnxt_hwrm_cp_ring_free(bp, &cpr->cp_ring_arr[j]); - ring = &cpr2->cp_ring_struct; - if (ring->fw_ring_id == INVALID_HW_RING_ID) - continue; - hwrm_ring_free_send_msg(bp, ring, - RING_FREE_REQ_RING_TYPE_L2_CMPL, - INVALID_HW_RING_ID); - ring->fw_ring_id = INVALID_HW_RING_ID; - } ring = &cpr->cp_ring_struct; if (ring->fw_ring_id != INVALID_HW_RING_ID) { hwrm_ring_free_send_msg(bp, ring, type, From patchwork Thu Feb 13 01:12:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972658 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BF4284D29 for ; Thu, 13 Feb 2025 01:13:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409217; cv=none; b=mHSZ4vtBIn5sJpfSn5dzmHiEHHCirpBDuRqCkkZPP82SZH9zZp/ishABNjpjMgaPmi8X7KQaQXLyzOUUVMJQfZxlva0yKrpEnMKSAVOeagIpU1bMgj41PayvzTjB33FucgAScrZZWuTsnuts5xpnLmGhCcwRZwTwPQPqQszlhGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409217; c=relaxed/simple; bh=WMm7lF8i3aT0wkgzKRQ9Rv0++wE/D/rYcS6YOf4Fbdc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WGHpX0bRphZD/OGK/gz6+xb7NWgUKhtwL9EbQwCXDBwW4NmhjaAP5yrydSZ2MRoFHAI5HCrJJjVFtr7/Ee9R6n/3QVR+N5Vy7DDNdSe24R+oXun1eGvAu/U+Iwj9DOtkBnwVgJMMr5jVY2RtWwsGEKHoegCOad4KuWSNG5qT3Q4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=Jq01dXZ0; arc=none smtp.client-ip=209.85.210.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="Jq01dXZ0" Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-71fbb0d035dso252996a34.2 for ; Wed, 12 Feb 2025 17:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409214; x=1740014014; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ycu8HxRfHwBK04zHzX+xCqlv0tMM5LUbOD+7kw9MqnQ=; b=Jq01dXZ0Jr9gYMteAZBf0yUyZshALAy1j66VN2iIjmT1y5MZqQ+lJ+Pp9QC0QJO1cn SkadkYy6qr9sfpChUJ17tZpDgBG8uegU6NZnNIIO2+AhVymZ1F653Ua/X3BouHAiUKDQ fmoGl8/++2HY4hrN36rFKL3ZBrXwS7bXqMLUk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409214; x=1740014014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ycu8HxRfHwBK04zHzX+xCqlv0tMM5LUbOD+7kw9MqnQ=; b=YlEz76llKxF/5LFmKcePaEBZ6NzhajaA5B+Wvkr3dGWukkK3VmaAkRLfm8qbdLX8q+ mTNMVgV07xt/gWHALqGJrgEbQTqFcwMq42SMh3WoviLAibUlM+Pd9WTMwwPDn+D++0b1 mbWbrzjm0jjjL2MghYVLnaN719bjJwuN92QUQw8YoliPP3dYreg/RnD3yVZu8BJsG8xT Sq+wRVc5pxTaX5cLz5cdRmCW7ZSvF8f81bLG3CrZW2Ub3JpXZq0u94ySSNLfCRI5FiaX Vj6NRJteOK0pKjbwBLUXTW3S09Ut4InDFpT+BaIIh3RGSwob4OCsK0g6WN/mzCd2IrP6 Mj1w== X-Gm-Message-State: AOJu0YyezyntrEE392LwoBwYnlwXTxSANZDBT6Wvu0UfJZ0ZLtYHk8XQ 3DYAnT11Mgtholm+iTP6iyPvNnFa68hukjRToqJEEX16r5hCcEqxTbUjK038SA== X-Gm-Gg: ASbGnctVgCHsimBf2QaltYFEqB12PsxpgSgdnyQinzKLYWiLCpxmQJMHMZ1BrR2Z9+x kFlI20fZHsB4ubcd2+0v2J7ngO3l4gpPtwxOr7Po7ttIqPcr7CbVAhxyD7a2bqj3PvV4rbgWyzj tMx26uSc2g3dqrMsnrcVcgXtHzLnDQKkzbx4hPdWBiQdt5+oeQc8lXXGFx7LOnbn+TfxqoEKpkL zM3tBzep6jImVdyDBp77Q891QSLKJwLcsiSbc2SC0/uUcxOBtteQ7fNs+rIh1MqW9AA9ZOh3LCp ayBY4xuIM/UP36EkK+awLVt88u/78GSH4wk0wczMBQR07kogTgwx7bd6jy3xfTw5FdU= X-Google-Smtp-Source: AGHT+IFSdHICmV54tOV6uEADJTXrP2+XqjZbh2QsJybypqYDTy0v1wwAKnRrApErddhA8OoV2l62Ag== X-Received: by 2002:a05:6830:3150:b0:71e:1fc7:252 with SMTP id 46e09a7af769-726fe7dc129mr812084a34.20.1739409212974; Wed, 12 Feb 2025 17:13:32 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:32 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Somnath Kotur Subject: [PATCH net-next v5 05/11] bnxt_en: Refactor bnxt_free_tx_rings() to free per TX ring Date: Wed, 12 Feb 2025 17:12:33 -0800 Message-ID: <20250213011240.1640031-6-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Somnath Kotur Modify bnxt_free_tx_rings() to free the skbs per TX ring. This will be useful later in the series. Reviewed-by: Michal Swiatkowski Signed-off-by: Somnath Kotur Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 115 ++++++++++++---------- 1 file changed, 61 insertions(+), 54 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 52d4dc222759..453f52648145 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3314,74 +3314,81 @@ static int bnxt_poll_p5(struct napi_struct *napi, int budget) return work_done; } -static void bnxt_free_tx_skbs(struct bnxt *bp) +static void bnxt_free_one_tx_ring_skbs(struct bnxt *bp, + struct bnxt_tx_ring_info *txr, int idx) { int i, max_idx; struct pci_dev *pdev = bp->pdev; - if (!bp->tx_ring) - return; - max_idx = bp->tx_nr_pages * TX_DESC_CNT; - for (i = 0; i < bp->tx_nr_rings; i++) { - struct bnxt_tx_ring_info *txr = &bp->tx_ring[i]; - int j; - if (!txr->tx_buf_ring) + for (i = 0; i < max_idx;) { + struct bnxt_sw_tx_bd *tx_buf = &txr->tx_buf_ring[i]; + struct sk_buff *skb; + int j, last; + + if (idx < bp->tx_nr_rings_xdp && + tx_buf->action == XDP_REDIRECT) { + dma_unmap_single(&pdev->dev, + dma_unmap_addr(tx_buf, mapping), + dma_unmap_len(tx_buf, len), + DMA_TO_DEVICE); + xdp_return_frame(tx_buf->xdpf); + tx_buf->action = 0; + tx_buf->xdpf = NULL; + i++; continue; + } - for (j = 0; j < max_idx;) { - struct bnxt_sw_tx_bd *tx_buf = &txr->tx_buf_ring[j]; - struct sk_buff *skb; - int k, last; - - if (i < bp->tx_nr_rings_xdp && - tx_buf->action == XDP_REDIRECT) { - dma_unmap_single(&pdev->dev, - dma_unmap_addr(tx_buf, mapping), - dma_unmap_len(tx_buf, len), - DMA_TO_DEVICE); - xdp_return_frame(tx_buf->xdpf); - tx_buf->action = 0; - tx_buf->xdpf = NULL; - j++; - continue; - } + skb = tx_buf->skb; + if (!skb) { + i++; + continue; + } - skb = tx_buf->skb; - if (!skb) { - j++; - continue; - } + tx_buf->skb = NULL; - tx_buf->skb = NULL; + if (tx_buf->is_push) { + dev_kfree_skb(skb); + i += 2; + continue; + } - if (tx_buf->is_push) { - dev_kfree_skb(skb); - j += 2; - continue; - } + dma_unmap_single(&pdev->dev, + dma_unmap_addr(tx_buf, mapping), + skb_headlen(skb), + DMA_TO_DEVICE); - dma_unmap_single(&pdev->dev, - dma_unmap_addr(tx_buf, mapping), - skb_headlen(skb), - DMA_TO_DEVICE); + last = tx_buf->nr_frags; + i += 2; + for (j = 0; j < last; j++, i++) { + int ring_idx = i & bp->tx_ring_mask; + skb_frag_t *frag = &skb_shinfo(skb)->frags[j]; - last = tx_buf->nr_frags; - j += 2; - for (k = 0; k < last; k++, j++) { - int ring_idx = j & bp->tx_ring_mask; - skb_frag_t *frag = &skb_shinfo(skb)->frags[k]; - - tx_buf = &txr->tx_buf_ring[ring_idx]; - dma_unmap_page( - &pdev->dev, - dma_unmap_addr(tx_buf, mapping), - skb_frag_size(frag), DMA_TO_DEVICE); - } - dev_kfree_skb(skb); + tx_buf = &txr->tx_buf_ring[ring_idx]; + dma_unmap_page(&pdev->dev, + dma_unmap_addr(tx_buf, mapping), + skb_frag_size(frag), DMA_TO_DEVICE); } - netdev_tx_reset_queue(netdev_get_tx_queue(bp->dev, i)); + dev_kfree_skb(skb); + } + netdev_tx_reset_queue(netdev_get_tx_queue(bp->dev, idx)); +} + +static void bnxt_free_tx_skbs(struct bnxt *bp) +{ + int i; + + if (!bp->tx_ring) + return; + + for (i = 0; i < bp->tx_nr_rings; i++) { + struct bnxt_tx_ring_info *txr = &bp->tx_ring[i]; + + if (!txr->tx_buf_ring) + continue; + + bnxt_free_one_tx_ring_skbs(bp, txr, i); } } From patchwork Thu Feb 13 01:12:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972659 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71C8F1411EB for ; Thu, 13 Feb 2025 01:13:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409218; cv=none; b=TP6OT5mLj9iLHgIpUGv3ollo4UixVBHeqlLE2xuFmMlV5pulJ0XVYrQ5OTiwr259HdSslB3+hKwOfgYqOOhAwvMBtzD452p2s6hvUU6F6SX5ZPu8JKDLO2Eoa8TMSjvfqkGunmOIq9+4wpXSiP3yq8MUh8QkTLQGfL7Jv8bDGTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409218; c=relaxed/simple; bh=I9Fd3w7iBdsQw7RR7W5KoX06nwuqLqZBEZ0GhvatebA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ir38JkVqj/xgeOBLqh/ZBnrSBdiC9BotifjQ4a5HuTupEMfJuPcn7OVJspQVufs2ouojWHVgs6vkrLvw7TpWzKrhMzJyb0QNkU/wfNyXYILWXOIx/OIRqh7SIzsUoIcksvIFJcDTEA3SPy8miAyjKX8e3Dpvu/es49bUnOdopfA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=QQv5VjhG; arc=none smtp.client-ip=209.85.210.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="QQv5VjhG" Received: by mail-ot1-f42.google.com with SMTP id 46e09a7af769-726cdf7541eso158252a34.1 for ; Wed, 12 Feb 2025 17:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409214; x=1740014014; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jPidmFB0PDNsFwHUYhUbpXpckFku0salq0TmYXgX8xo=; b=QQv5VjhGpDU8uHLuitWLe089Ksz6hwabpqnOZks2XHnEbMloiMAyEeeR/+W8C++zq7 8aKgPXRSBKA8ryd7l/vKImFbtN1fj2buWtPTkqskNKc1eUfF0vjeOSXhS3Gn9YO8VLZY VoSyF+f0cf0Wr7zeLiX+P/BPno8TH1KgJ7l+4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409214; x=1740014014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jPidmFB0PDNsFwHUYhUbpXpckFku0salq0TmYXgX8xo=; b=LRgbxfrmY2wzdAfUZq9j1iFDdpcznN1IXerskGqJrV7dB7W7rOajtkwQn7Ii0120TH 3Sxor6zv668Xnc69QAZa2FYNh4x8cPNYEioq1eo1yrW4qdmAoK4Cjh0DPHNzLfI9Wdu0 vDdVHb/iSitcp4TdMRoXWOpRatTKDxSiF8B3Cnyp3Iw2SHtfw0+t+E9283ynWc2mkbpW 4VKcwgawozMfvVT/fkxREArKjv7bTX9XH8s20Wr2nhq43W51QEUe63zwVeQDKUfXf7dO VLsUCQ7k8ChuJT2QljGh7RsFGhohFpMbdY1BJaqEahE4S+waIR5Y3vFPItbY2+6j0NEZ o5ug== X-Gm-Message-State: AOJu0YyFJrdxLCzvVtcr/yoE8oBaNOwE+dKl94XGkaL+IsWPMvpq4JFf +v9AfTfWj92mUEdodtu1d55Xl+84Ei+FbMkvhJiHxItDg8fxxnoWi6BrL3G8uQ== X-Gm-Gg: ASbGncsN/PFMiyh22Jfij4ci+oYE9nbRjbL9SUDcCRK0KrGj3mArOM4GSOHfQDoEmi9 Ya063fnq8j54q/Ms3DJnQEmRFa1vXmXW7ErfYx4yToBGH4wMAVfg93RxsgVJwVUpEeXfE6Es9Ju JLBmmigFDfhk2oIAOzcVD2/E/ik+5zoDy4RVSdmhwOeDpKtbzqRYtAiIb+pzIati8HHdi0djWKu rvBaIYVLZLTPzPun3Re/pncWZ7L+DF3clF7rmUKlFrkaBsE3gu0OsTTyvrDW/8ystgShlB4Rr4K KSjuH9RMtjgk+xHjQ0VhIr5aE+2YSJelGOLy40QLgtxqJ6d4ZBorhnaFJMj83DbXb2g= X-Google-Smtp-Source: AGHT+IH6rJK+CwS3G3Lw2JD7SlcxemFyzRe20++6kikxNZgxFXW1MH+C0DMK4Q/v+SvWx4e4rNC+ag== X-Received: by 2002:a05:6830:f8a:b0:710:a425:d6b8 with SMTP id 46e09a7af769-726ff1629d0mr606734a34.14.1739409214200; Wed, 12 Feb 2025 17:13:34 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:33 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Hongguang Gao , Ajit Khaparde Subject: [PATCH net-next v5 06/11] bnxt_en: Refactor RX/RX AGG ring parameters setup for P5_PLUS Date: Wed, 12 Feb 2025 17:12:34 -0800 Message-ID: <20250213011240.1640031-7-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org There is some common code for setting up RX and RX AGG ring allocation parameters for P5_PLUS chips. Refactor the logic into a new function. Reviewed-by: Hongguang Gao Reviewed-by: Ajit Khaparde Reviewed-by: Michal Swiatkowski Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 58 +++++++++++------------ 1 file changed, 28 insertions(+), 30 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 453f52648145..ac63d3feaa1d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -6944,6 +6944,28 @@ static void bnxt_hwrm_ring_grp_free(struct bnxt *bp) hwrm_req_drop(bp, req); } +static void bnxt_set_rx_ring_params_p5(struct bnxt *bp, u32 ring_type, + struct hwrm_ring_alloc_input *req, + struct bnxt_ring_struct *ring) +{ + struct bnxt_ring_grp_info *grp_info = &bp->grp_info[ring->grp_idx]; + u32 enables = RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID; + + if (ring_type == HWRM_RING_ALLOC_AGG) { + req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX_AGG; + req->rx_ring_id = cpu_to_le16(grp_info->rx_fw_ring_id); + req->rx_buf_size = cpu_to_le16(BNXT_RX_PAGE_SIZE); + enables |= RING_ALLOC_REQ_ENABLES_RX_RING_ID_VALID; + } else { + req->rx_buf_size = cpu_to_le16(bp->rx_buf_use_size); + if (NET_IP_ALIGN == 2) + req->flags = + cpu_to_le16(RING_ALLOC_REQ_FLAGS_RX_SOP_PAD); + } + req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx); + req->enables |= cpu_to_le32(enables); +} + static int hwrm_ring_alloc_send_msg(struct bnxt *bp, struct bnxt_ring_struct *ring, u32 ring_type, u32 map_index) @@ -6995,37 +7017,13 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp, break; } case HWRM_RING_ALLOC_RX: - req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX; - req->length = cpu_to_le32(bp->rx_ring_mask + 1); - if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { - u16 flags = 0; - - /* Association of rx ring with stats context */ - grp_info = &bp->grp_info[ring->grp_idx]; - req->rx_buf_size = cpu_to_le16(bp->rx_buf_use_size); - req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx); - req->enables |= cpu_to_le32( - RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID); - if (NET_IP_ALIGN == 2) - flags = RING_ALLOC_REQ_FLAGS_RX_SOP_PAD; - req->flags = cpu_to_le16(flags); - } - break; case HWRM_RING_ALLOC_AGG: - if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { - req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX_AGG; - /* Association of agg ring with rx ring */ - grp_info = &bp->grp_info[ring->grp_idx]; - req->rx_ring_id = cpu_to_le16(grp_info->rx_fw_ring_id); - req->rx_buf_size = cpu_to_le16(BNXT_RX_PAGE_SIZE); - req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx); - req->enables |= cpu_to_le32( - RING_ALLOC_REQ_ENABLES_RX_RING_ID_VALID | - RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID); - } else { - req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX; - } - req->length = cpu_to_le32(bp->rx_agg_ring_mask + 1); + req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX; + req->length = (ring_type == HWRM_RING_ALLOC_RX) ? + cpu_to_le32(bp->rx_ring_mask + 1) : + cpu_to_le32(bp->rx_agg_ring_mask + 1); + if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) + bnxt_set_rx_ring_params_p5(bp, ring_type, req, ring); break; case HWRM_RING_ALLOC_CMPL: req->ring_type = RING_ALLOC_REQ_RING_TYPE_L2_CMPL; From patchwork Thu Feb 13 01:12:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972660 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E991142E77 for ; Thu, 13 Feb 2025 01:13:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409218; cv=none; b=PE0HThCGh4YbRGz0LcfQaBr7R4V1cjMNux5DubQekJTEFjrFOwCYL/x+evL0w1CyGh+Ph+QfWSij3u4BhutgszV09ufQxQHt9xygLLg2npx2VTQAAZc84w7HdFYmuKM+G306VGcrtUWQ2wswdHkkbSyR+47+PBG44vs+PlIR3Ng= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409218; c=relaxed/simple; bh=/Z40j/J42ZiQ048j2vF3aU41dOC1PM7eOBd+ElrAFQM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FD5+vliWvQP6c3grciepRnc7IWT1WIaquj5I/kJ9ynS1UBoil3uqqCdMZOZ2nAFjHOd2Q5754rwbq/8CUQFcwg2Dd7QHgSbjmsBlx+mqaXnrU5WwjXEPdbIXEz8bq3j56+M3ybwV6SEJEhG7umeG8W6o0s7f9p26xUBJ25aP5l0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=RW4JOfEQ; arc=none smtp.client-ip=209.85.210.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="RW4JOfEQ" Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-726ef4cba96so111664a34.2 for ; Wed, 12 Feb 2025 17:13:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409216; x=1740014016; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=I4ASapAG1egWbtPE1ui+0PISpvbEH/BCpm+FdWZFb9A=; b=RW4JOfEQjZyiMb7+IqnyzZaBnbEMpAwoLNOJnnp2VyVHiq928kELstTfYFMCC96WaU QDh1tuNLIDF4W64ZJs1CcGMbyoLJdpHN6/HVie4ZjGSrv1GvSra55573ZeBkkxmRkMtw CTaoe6HgxECiyVoNolTBhjjsNgcy4FmsrRTmo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409216; x=1740014016; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I4ASapAG1egWbtPE1ui+0PISpvbEH/BCpm+FdWZFb9A=; b=pAHc9oHy8DnSGBDy/2pMgXZqbVp9mS4SPOa9GXTtlnv+HT97kMVB0VoAWFxMdegrDu PhkE/7jfdpbZ6/+f7Bd9f33+X6/DLz6OJd+4sMSnT0uwlnxODsPSTKfBwrWB5RveUZ0C tfDst2TuO6w1WDGgVH67TgelnEYaFGL8mxFR6uryIHzebsB22UIK48A3+AKkQowyYC87 LSzFBkIGn59296sDin3VYJ4YVhfjSgZ/pzhqV81Ia6OitN7bSufXxF3PQF+G2zfU/7lN IiLrndb6TsYwGH0yYbdqBxPqAMbIAVOSPwSeXq8wsI2rdaaabF1rPLdK8GaCS0PoMZjN 70vA== X-Gm-Message-State: AOJu0YxTYWgYpE3d5Qmbnz0nwMim/f/ZWG9AIewJZ6svNYil7X/CIbtD kYA0diCB8WU5WYkijC+YrbzxQRWus0sTC5m9RdtbhTyge6OQLJCvNUf65/UWiA== X-Gm-Gg: ASbGncvdi2D4BXAxMcEDFEAPXXPuc0k0FzH7An6zYmgOzMVk6YoOnJ5kZjnYwWf7fBo yokcO7LnDd+y+jrxSGRxPPtTK3Qolg1Oq6upQOLgVu5xmJNmLmBiPrM31BPbvadEs+i7b4ULXZ1 d/RQ1mb/xH6CmzQt/Igi24qgruRdORg730nZ5JKwVgwHzA8JdcPSQ575+ZSx15Jx2K/Sz7Rjr1d N6Xt2ZK5ILTEPZjmcFROJ5mkWGV1XGzjL6GBHhitfE9XsiyKyIK1hFWdIfU3j76c7MFHbc82otx rEu0vnFAo+qPCHfbpisj3bMch3HoT+xNmeDo2qskfKXNOlLi6whiID0+hY4yuEjg/xQ= X-Google-Smtp-Source: AGHT+IEeZ6LN6Abwgwk4biXUtOiHDFSPWdweHEAJnI4CDPzSMTmxS4K9vFtnpMUpTcHwgGI1COGBJQ== X-Received: by 2002:a05:6830:6487:b0:718:8dc:a5e with SMTP id 46e09a7af769-726f1c67cbcmr3737408a34.9.1739409216275; Wed, 12 Feb 2025 17:13:36 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:35 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Hongguang Gao , Ajit Khaparde Subject: [PATCH net-next v5 07/11] bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings Date: Wed, 12 Feb 2025 17:12:35 -0800 Message-ID: <20250213011240.1640031-8-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Newer firmware can use the NQ ring ID associated with each RX/RX AGG ring to enable PCIe Steering Tags on P5_PLUS chips. When allocating RX/RX AGG rings, pass along NQ ring ID for the firmware to use. This information helps optimize DMA writes by directing them to the cache closer to the CPU consuming the data, potentially improving the processing speed. This change is backward-compatible with older firmware, which will simply disregard the information. Reviewed-by: Hongguang Gao Reviewed-by: Ajit Khaparde Reviewed-by: Michal Swiatkowski Signed-off-by: Andy Gospodarek Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index ac63d3feaa1d..c6cf575af53f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -6949,7 +6949,8 @@ static void bnxt_set_rx_ring_params_p5(struct bnxt *bp, u32 ring_type, struct bnxt_ring_struct *ring) { struct bnxt_ring_grp_info *grp_info = &bp->grp_info[ring->grp_idx]; - u32 enables = RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID; + u32 enables = RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID | + RING_ALLOC_REQ_ENABLES_NQ_RING_ID_VALID; if (ring_type == HWRM_RING_ALLOC_AGG) { req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX_AGG; @@ -6963,6 +6964,7 @@ static void bnxt_set_rx_ring_params_p5(struct bnxt *bp, u32 ring_type, cpu_to_le16(RING_ALLOC_REQ_FLAGS_RX_SOP_PAD); } req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx); + req->nq_ring_id = cpu_to_le16(grp_info->cp_fw_ring_id); req->enables |= cpu_to_le32(enables); } From patchwork Thu Feb 13 01:12:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972661 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60B0A14831C for ; Thu, 13 Feb 2025 01:13:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409221; cv=none; b=bmvXI5822LlrIf6OPmnq3nsTEXmsQApPztT50p5hVxqnW88m1YZprhI62wI2WJUNWCJ38NYf+lTwd2SQqcMDFnDT6GKvqzitrIhUno+utEM1af2EJ/X/4JJKJpwFUNyXVlblZFSFuiEQGoV6EQ+VYNlunwOYSdUOZOTcQHhP+rs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409221; c=relaxed/simple; bh=hpi06z8m58JGcyJPDP4kwpNgO2LiYOdW9xnp7xS7QfE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MXBM8R20yFmHzpiourQB1C8z6/7D20UA5/YA8VNLISp+rx17vWvrQZrVqFXuzOArh6UO0OJ1qQ2gElCFPOlcITCPy8maT1ViyfJG9isGPE1rH/kWIkktRNnTW1lXq48hgu1OxNUJliBvrMfQ7XAIw5qVKQxm692Ic9LcqW3mlQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=MpUYoqhd; arc=none smtp.client-ip=209.85.210.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="MpUYoqhd" Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-726819aa651so156140a34.1 for ; Wed, 12 Feb 2025 17:13:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409218; x=1740014018; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rMZaiXwf/CDrdAxd9HUemEUI3z0hjO8BGgDx5GB81as=; b=MpUYoqhd1mPHPzqSwFyexyGox6a75/q2TO42GFgoEIjU6wIIFC/EMLkk2WSJv1F6hU sJ+a4XWZSNh8vwwb56krQkS3fcrDa6Vxu8z7KFdZ6rdA5/YwPpp8Lpyp9brRnIy2AQ3J Mk2bIVtImOHA3ostrg0ml5zUSN3pPSRNkdt+0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409218; x=1740014018; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rMZaiXwf/CDrdAxd9HUemEUI3z0hjO8BGgDx5GB81as=; b=cX8DwY7fv0/Zs2PN48KSu68m4UUSXADUrDkMRu2waaMAN2PqBSwV1FNTbIX2K21qPO ngEjTugvyLHq1IA3pvDqFA/Pc4Dec22kwTcY+3fCapRaRhWu6bXo5rr28NivPddX2IRf wDKzocOTbTl74iK1dIEPVoXuqSZnIgKfKWO/aURLE8iCVBtua7F8MEQdhCm9Sbq5Oaeg uFbHVJRY4P+B2mugaIFtZx/O/vYHN06Z30eaPwlK+gTeD/tQo/lHLWMwa3upuVUzpJof D1K+4F5KvT2tUqLC4+40aSsKIrHO+cjW7Oesh9V2FirY3T2u4dSgqa5TuWM7LoIIbG60 MX5g== X-Gm-Message-State: AOJu0YxhEMxD2oGm8gS7XH3Qs9apckl7yMmihV72c7BOkhjcJZCrsiLy AXfuuNbQszU+eLDbqZNJDmL3N0VEgs0/e9WEKtp5BpViuYMqxTHbe86l/zj3Ug== X-Gm-Gg: ASbGnct2964UvfuWeylhUGEMh0U4mdI75YZhHeGzTiXhZ6zeerZd4bev3DqtG0hyER3 SfJVBjwU22iNbm6yC9jcnFgBl9XiQfBpmPF3WLcjElEMwBFWtJdOSaqlq9UjP1cuTdrgYZFYEA9 O4IiOmNf88NobAlLlA+ztp8Nfwaes0Gm8ll0WoIOXd12DtyLzORZrbCeHBXseNNbGLdu6OLuoEw 2Ni4dK9Y39b+i+WBDXKOk+pcRGJU42x6ckPqfO+9ie+pX0qUDFSUJKqilEyu3IWkgc9Qrj3c86e 6VskB7Pn0DvgrIR8pCP7HofA4HayBGCo1EQj4Fh6SARhlK9RX4eXcTsC2V3AKrlTdOA= X-Google-Smtp-Source: AGHT+IGbt0wrTSzAw3Ps/iywBOo476FnZCfBwYcrEOaZF5dNtIZfSqdCM/7akXOTE+XQY+WamPOIbQ== X-Received: by 2002:a05:6830:6f84:b0:726:ef21:ef33 with SMTP id 46e09a7af769-726f1c36eb3mr4014913a34.4.1739409218413; Wed, 12 Feb 2025 17:13:38 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:37 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Somnath Kotur , David Wei Subject: [PATCH net-next v5 08/11] bnxt_en: Reallocate RX completion ring for TPH support Date: Wed, 12 Feb 2025 17:12:36 -0800 Message-ID: <20250213011240.1640031-9-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Somnath Kotur In order to program the correct Steering Tag during an IRQ affinity change, we need to free/re-allocate the RX completion ring during queue_restart. If TPH is enabled, call FW to free the Rx completion ring and clear the ring entries in queue_stop(). Re-allocate it in queue_start() if TPH is enabled. Note that TPH mode is not enabled in this patch and will be enabled later in the patch series. While modifying bnxt_queue_start(), remove the unnecessary zeroing of rxr->rx_next_cons. It gets overwritten by the clone in bnxt_queue_start(). Remove the rx_reset counter increment since restart is not reset. Add comment to clarify that the ring allocations in queue_start should never fail. Reviewed-by: Michal Swiatkowski Signed-off-by: Somnath Kotur Signed-off-by: Michael Chan --- Cc: David Wei v5: Remove reset counter increment. Add comment to clarify ring alloc should always succeed in this codepath. v3: Only free/allocate the RX cmpl ring when TPH is enabled. --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 38 +++++++++++++++++++---- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 ++ 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index c6cf575af53f..04980718c287 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7426,6 +7426,19 @@ static void bnxt_hwrm_cp_ring_free(struct bnxt *bp, ring->fw_ring_id = INVALID_HW_RING_ID; } +static void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr) +{ + struct bnxt_ring_struct *ring = &cpr->cp_ring_struct; + int i, size = ring->ring_mem.page_size; + + cpr->cp_raw_cons = 0; + cpr->toggle = 0; + + for (i = 0; i < bp->cp_nr_pages; i++) + if (cpr->cp_desc_ring[i]) + memset(cpr->cp_desc_ring[i], 0, size); +} + static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) { u32 type; @@ -15623,7 +15636,6 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx) { struct bnxt *bp = netdev_priv(dev); struct bnxt_rx_ring_info *rxr, *clone; - struct bnxt_cp_ring_info *cpr; struct bnxt_vnic_info *vnic; int i, rc; @@ -15642,20 +15654,27 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx) bnxt_copy_rx_ring(bp, rxr, clone); + /* All rings have been reserved and previously allocated. + * Reallocating with the same parameters should never fail. + */ rc = bnxt_hwrm_rx_ring_alloc(bp, rxr); if (rc) return rc; + + if (bp->tph_mode) { + rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr); + if (rc) + goto err_free_hwrm_rx_ring; + } + rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr); if (rc) - goto err_free_hwrm_rx_ring; + goto err_free_hwrm_cp_ring; bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); if (bp->flags & BNXT_FLAG_AGG_RINGS) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); - cpr = &rxr->bnapi->cp_ring; - cpr->sw_stats->rx.rx_resets++; - for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) { vnic = &bp->vnic_info[i]; @@ -15672,6 +15691,9 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx) return 0; +err_free_hwrm_cp_ring: + if (bp->tph_mode) + bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr); err_free_hwrm_rx_ring: bnxt_hwrm_rx_ring_free(bp, rxr, false); return rc; @@ -15696,11 +15718,15 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx) cancel_work_sync(&rxr->bnapi->cp_ring.dim.work); bnxt_hwrm_rx_ring_free(bp, rxr, false); bnxt_hwrm_rx_agg_ring_free(bp, rxr, false); - rxr->rx_next_cons = 0; page_pool_disable_direct_recycling(rxr->page_pool); if (bnxt_separate_head_pool()) page_pool_disable_direct_recycling(rxr->head_pool); + if (bp->tph_mode) { + bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr); + bnxt_clear_one_cp_ring(bp, rxr->rx_cpr); + } + memcpy(qmem, rxr, sizeof(*rxr)); bnxt_init_rx_ring_struct(bp, qmem); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index db0be469a3db..4e20878e7714 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2410,6 +2410,8 @@ struct bnxt { u8 max_q; u8 num_tc; + u8 tph_mode; + unsigned int current_interval; #define BNXT_TIMER_INTERVAL HZ From patchwork Thu Feb 13 01:12:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972662 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f47.google.com (mail-ot1-f47.google.com [209.85.210.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BF4714A605 for ; Thu, 13 Feb 2025 01:13:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409222; cv=none; b=YA5IcLvH3JKQjZvajYPNLiHDKlU+m3D2o8fJRvgp1DNInpWFwAFgwFgoFktIt3kqFJhaPyOn8ERPGOwXtAJ3XAaOzN5cWNzAVGzMATG8mRbeppgv6Y3tGd0XUtTwQ4qpUALIcq6mZLYMMz0AEDnP7uw16LBjymftgy/RXenPrdY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409222; c=relaxed/simple; bh=IXKovOctHJnLdiAOUUqeOrUEhuY/G8oC4zb+ENO9NYs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SnnNUd655fYWpCkbrp93DY5jAK2rqdoNXvSDnAJWf2PbF6EmnXM2WB4VINuEQSak+rTX3CVMoGUHhffOlFobNXXfqFCfAaZEEEtKbR/lHHUGP2icIOTPcBWurP015jWwJQLJf/1+TLXfvoLaRm+LmTh0Mc3QXBKyDNFpPmw8XyE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=NdDUiqIx; arc=none smtp.client-ip=209.85.210.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="NdDUiqIx" Received: by mail-ot1-f47.google.com with SMTP id 46e09a7af769-72700b5056bso75565a34.0 for ; Wed, 12 Feb 2025 17:13:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409220; x=1740014020; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q/WGpVdKkhB4PCRJWhKILMrZMw+zO2YjcWfapXVtE7s=; b=NdDUiqIxzHXSY9+dsDy7vke7tqbKpr55bGQ80TNF9vsslrnWWbb1PlO+sJCpPj/JXE ShBXHPrO2EXjGAPKKq11IicARm8GzToYbbPVfDLQK8xCXKiFbJkMQgUu1HcNzGhZdwdA 5rV4oyhYdfAc4+tlIy9Ur0Njw0kCXu52xD+0s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409220; x=1740014020; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q/WGpVdKkhB4PCRJWhKILMrZMw+zO2YjcWfapXVtE7s=; b=mXm5fPq7ckrWReG09AH6u1E57EtMZ3RS81PbeZyBVLtKsAqLT0WP/Canbp7hp0gw6n CGNqGtH3epzNu0CILOYJAKg5qQr7QZeD8tm2ACBmTpsxg22ucd2iIYiTWGU4gmTiw64d OaP+iVBaoSeptLz4NMebH8ySC/nDaAlGoUqZdlSxLIlF+jbP9nkvVqntExttUquTZ+a7 5gWGwG9q4WjXzZmdZCDMlpgTIdbPDhlm9wjOOZTsT+4E48p/h8KOdvZwbAH0/h6mHfcn 2b6qInVJwtHq3wap16+ysOCbt1JvzgGPcv6JMhibyHHICy7R3NEmAEZxXJAUfkAIuisc F5oA== X-Gm-Message-State: AOJu0Yym5a+HnwMfXSGqf2/46maS22DiaTDKc58V6g+uM8uHc4MgQKf7 4Z0HFdLYaDx5n/ap8VWawWZL9Ubu3jQoiYoq45OEq6Sg4BVmiNpRn2zb7fvm/g== X-Gm-Gg: ASbGnctToQFRGH/H/emzRyWP6ssNkuRBlqpTW85CXKlPPSAouwJvMhaXGcBby+17A6A NqDeLLGOgo6s1gyeXrxtApxzlgHGnjBXEGroI6+MpTSewCreWMambxVkXE/unqKwkQRBPBeVAAa BDHs/Vsoa6lW4yOLqCKgggpCvhXZ1eYcg2n2XtgnPJ8731x8ltJKNeoJLrg5WkE+A2fngT1upoO 1MSjwexbKV3iGqsU1caz8LCW06wSlK+Uq51coydnLEE8L44nfRMKUP9gYjjnKxQL1vLZ5k3CvhW Zn+AXaJ4yTILfLbh/6l763MHWwiEfAAsyFsufKdut4Gu5W0FzY77dM/bTHrso2xIfds= X-Google-Smtp-Source: AGHT+IEoDImTiKmjDyWclHRHK7JTX+CsMIjl7XEmp8Kyyzz7Va4GUD6fWPwMUwlE6EAC8lh8HwCRzA== X-Received: by 2002:a9d:6009:0:b0:71d:e923:6d2f with SMTP id 46e09a7af769-726feef7e2cmr679939a34.13.1739409220494; Wed, 12 Feb 2025 17:13:40 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:39 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Ajit Khaparde Subject: [PATCH net-next v5 09/11] bnxt_en: Refactor TX ring free logic Date: Wed, 12 Feb 2025 17:12:37 -0800 Message-ID: <20250213011240.1640031-10-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Add a new bnxt_hwrm_tx_ring_free() function to handle freeing a HW transmit ring. The new function will also be used in the next patch to free the TX ring in queue_stop. Reviewed-by: Ajit Khaparde Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 33 +++++++++++++---------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 04980718c287..2d0d9ac8e2c4 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7368,6 +7368,23 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp, return 0; } +static void bnxt_hwrm_tx_ring_free(struct bnxt *bp, + struct bnxt_tx_ring_info *txr, + bool close_path) +{ + struct bnxt_ring_struct *ring = &txr->tx_ring_struct; + u32 cmpl_ring_id; + + if (ring->fw_ring_id == INVALID_HW_RING_ID) + return; + + cmpl_ring_id = close_path ? bnxt_cp_ring_for_tx(bp, txr) : + INVALID_HW_RING_ID; + hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_TX, + cmpl_ring_id); + ring->fw_ring_id = INVALID_HW_RING_ID; +} + static void bnxt_hwrm_rx_ring_free(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, bool close_path) @@ -7447,20 +7464,8 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) if (!bp->bnapi) return; - for (i = 0; i < bp->tx_nr_rings; i++) { - struct bnxt_tx_ring_info *txr = &bp->tx_ring[i]; - struct bnxt_ring_struct *ring = &txr->tx_ring_struct; - - if (ring->fw_ring_id != INVALID_HW_RING_ID) { - u32 cmpl_ring_id = bnxt_cp_ring_for_tx(bp, txr); - - hwrm_ring_free_send_msg(bp, ring, - RING_FREE_REQ_RING_TYPE_TX, - close_path ? cmpl_ring_id : - INVALID_HW_RING_ID); - ring->fw_ring_id = INVALID_HW_RING_ID; - } - } + for (i = 0; i < bp->tx_nr_rings; i++) + bnxt_hwrm_tx_ring_free(bp, &bp->tx_ring[i], close_path); bnxt_cancel_dim(bp); for (i = 0; i < bp->rx_nr_rings; i++) { From patchwork Thu Feb 13 01:12:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972663 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 840ED13A3EC for ; Thu, 13 Feb 2025 01:13:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409225; cv=none; b=hKwWP84b3DsAnuFlEmzfNxqJX4dMRwDDAAOblRmKjwrVLQ2X0qsej2iAm34YyuhiqgzPZYWFpLsr55x511vWs/yZgtRw148tm8tSrp1LFH6o4amlc17gBj0F0OuUhLBuE//pnyw3RuT3l4puoB4botwdda4hwCH3Hr8SHca2f00= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409225; c=relaxed/simple; bh=7yFFQYYH4+IZ/hODe5AzffCuzF/F+agehrnk2mzt25A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=R0YuoeFmyt90Gkchcq1bi9XFigms92NNzYlf9lMf+U053De8cMNTy1DEHWKhZ82EIzRuUuKkIu0hOjgW39T+u/SoGX8Bl4FQrNJAe9f6q1RBsUYsb7oHWLhY9NrK082BHBzBKJfdxI3RFcS+UtbBO9P5aNKGle0M+p7wf9P085U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=Bw3kk9G2; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="Bw3kk9G2" Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5fc8f74d397so210013eaf.3 for ; Wed, 12 Feb 2025 17:13:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409222; x=1740014022; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/6ahPr3qNlrtPz7WNKqwlq2nEMj89hzFEsFf8OUr3YU=; b=Bw3kk9G2sWz2w80FGhmNO8/VK1JS8oCPure49XY7sD1YhgXxc517Tsj8uOrI6mSnUS eTQLxeS4TOzmIVQqUFxMXwpgUYUEXw96mvjBS12i8sWVHOhVMJlWGTMGS2V1UTr8jMO5 wTqd2W/irGL9Qx0/2DRHuyp0ZKNOMFLrOhxQ4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409222; x=1740014022; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/6ahPr3qNlrtPz7WNKqwlq2nEMj89hzFEsFf8OUr3YU=; b=swLGmSpwrLEVyMo6H0K+WYjKECJ2yiLS/JJTgrns6PaYxHTZioL5UGsHTgSZp6MD4D q4uSqR56KKSYO3WSB979KgpY+oWoDeuBFvMRbgDeg2EdLtXrhjfQfhLfrsDMTJoOtFE8 mUyQ8kKSdLvZwXswKxL5OFMMk7xFkK/0GNwBKfN2GR9214ggDgNR6kUazPMavG4qvSzN QtuoogODZ2OG/Vsp9VyJYuPC85ymWIzvdf7WcK8BX5gVKOvE0b12TfB6bFNueEjXKmzU 2UchGdlTxo7MflxPGecjdzlzj+5HqXLPu6xgbTt3zicl/WrpL0MkC+uqXMAn/wXf9KHV PCuA== X-Gm-Message-State: AOJu0Yxwme9mgF0EjGnthBY38HQtFz7grVsBQ5oT2rc3hjpzNY7X/R2Q wy7Ot8UVcHQgLB/SCykbGkmSGqdfdq14BcpgFm/eU57ty8YrUdCdV68pg1H0Bw== X-Gm-Gg: ASbGncvl3I/GYPJvEBJKTgEhmG+nDmS1ReSnm3JLvvll0ZYuJP79rlZ6N1NRhm68LPJ Kn1KcmNDsilMFgK6L9I4s4CEtYfXfdduFdIxjlA3n3wfipPYyuqrjYJprkEIPKGC7g+M/VwzaH7 R/nFdn44tLDDszL416KA8CCa1WZMPFH7CDZm7uAF8ME99rDXh5oc8KrkPYlZhRT9NEEaHar+rM3 oL00LTZ+XIkGbXRvdLClXNK9jEkKQyq5+Qai6lwAiDgaRtdz/HlW0UGjAg1wPBEaYhdN5I1mcM3 mlro34f/pQuLfbsqxBQubMqxrjWV7Inm1IabJSFBvg2NUlt6be7MKPMWGE7+sofDoJY= X-Google-Smtp-Source: AGHT+IGaF25b+KFHyJdESkI1lKfFaiWVAkjVXb86JdAaIYwH+sNp0zFhuYVUvq1XHhGmfmzLqhqGpg== X-Received: by 2002:a05:6820:2221:b0:5fc:b059:7597 with SMTP id 006d021491bc7-5fcb0597a2cmr608862eaf.1.1739409222571; Wed, 12 Feb 2025 17:13:42 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:41 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Somnath Kotur , Ajit Khaparde , David Wei Subject: [PATCH net-next v5 10/11] bnxt_en: Extend queue stop/start for TX rings Date: Wed, 12 Feb 2025 17:12:38 -0800 Message-ID: <20250213011240.1640031-11-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Somnath Kotur In order to use queue_stop/queue_start to support the new Steering Tags, we need to free the TX ring and TX completion ring if it is a combined channel with TX/RX sharing the same NAPI. Otherwise TX completions will not have the updated Steering Tag. If TPH is not enabled, we just stop the TX ring without freeing the TX/TX cmpl rings. With that we can now add napi_disable() and napi_enable() during queue_stop()/ queue_start(). This will guarantee that NAPI will stop processing the completion entries in case there are additional pending entries in the completion rings after queue_stop(). There could be some NQEs sitting unprocessed while NAPI is disabled thereby leaving the NQ unarmed. Explicitly re-arm the NQ after napi_enable() in queue start so that NAPI will resume properly. Error handling in bnxt_queue_start() requires a reset. If a TX ring cannot be allocated or initialized properly, it will cause TX timeout. The reset will also free any partially allocated rings. We don't expect to hit this error path because re-allocating previously reserved and allocated rings with the same parameters should never fail. Reviewed-by: Ajit Khaparde Reviewed-by: Michal Swiatkowski Signed-off-by: Somnath Kotur Signed-off-by: Michael Chan --- Cc: David Wei v5: Remove reset counter increment. Add comments about napi_disable(). Add comments that ring alloc should always succed in this code path. v3: Fix build bot warning. Only free TX/TX cmpl rings when TPH is enabled. v2: Add reset for error handling in queue_start(). Fix compile error. Discussion about adding napi_disable()/napi_enable(): https://lore.kernel.org/netdev/5336d624-8d8b-40a6-b732-b020e4a119a2@davidwei.uk/#t --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 119 ++++++++++++++++++++-- 1 file changed, 110 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 2d0d9ac8e2c4..1997cdbd5801 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -11279,6 +11279,78 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init) return 0; } +static void bnxt_tx_queue_stop(struct bnxt *bp, int idx) +{ + struct bnxt_tx_ring_info *txr; + struct netdev_queue *txq; + struct bnxt_napi *bnapi; + int i; + + bnapi = bp->bnapi[idx]; + bnxt_for_each_napi_tx(i, bnapi, txr) { + WRITE_ONCE(txr->dev_state, BNXT_DEV_STATE_CLOSING); + synchronize_net(); + + if (!(bnapi->flags & BNXT_NAPI_FLAG_XDP)) { + txq = netdev_get_tx_queue(bp->dev, txr->txq_index); + if (txq) { + __netif_tx_lock_bh(txq); + netif_tx_stop_queue(txq); + __netif_tx_unlock_bh(txq); + } + } + + if (!bp->tph_mode) + continue; + + bnxt_hwrm_tx_ring_free(bp, txr, true); + bnxt_hwrm_cp_ring_free(bp, txr->tx_cpr); + bnxt_free_one_tx_ring_skbs(bp, txr, txr->txq_index); + bnxt_clear_one_cp_ring(bp, txr->tx_cpr); + } +} + +static int bnxt_tx_queue_start(struct bnxt *bp, int idx) +{ + struct bnxt_tx_ring_info *txr; + struct netdev_queue *txq; + struct bnxt_napi *bnapi; + int rc, i; + + bnapi = bp->bnapi[idx]; + /* All rings have been reserved and previously allocated. + * Reallocating with the same parameters should never fail. + */ + bnxt_for_each_napi_tx(i, bnapi, txr) { + if (!bp->tph_mode) + goto start_tx; + + rc = bnxt_hwrm_cp_ring_alloc_p5(bp, txr->tx_cpr); + if (rc) + return rc; + + rc = bnxt_hwrm_tx_ring_alloc(bp, txr, false); + if (rc) + return rc; + + txr->tx_prod = 0; + txr->tx_cons = 0; + txr->tx_hw_cons = 0; +start_tx: + WRITE_ONCE(txr->dev_state, 0); + synchronize_net(); + + if (bnapi->flags & BNXT_NAPI_FLAG_XDP) + continue; + + txq = netdev_get_tx_queue(bp->dev, txr->txq_index); + if (txq) + netif_tx_start_queue(txq); + } + + return 0; +} + static void bnxt_free_irq(struct bnxt *bp) { struct bnxt_irq *irq; @@ -15641,7 +15713,9 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx) { struct bnxt *bp = netdev_priv(dev); struct bnxt_rx_ring_info *rxr, *clone; + struct bnxt_cp_ring_info *cpr; struct bnxt_vnic_info *vnic; + struct bnxt_napi *bnapi; int i, rc; rxr = &bp->rx_ring[idx]; @@ -15659,27 +15733,39 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx) bnxt_copy_rx_ring(bp, rxr, clone); + bnapi = rxr->bnapi; + cpr = &bnapi->cp_ring; + /* All rings have been reserved and previously allocated. * Reallocating with the same parameters should never fail. */ rc = bnxt_hwrm_rx_ring_alloc(bp, rxr); if (rc) - return rc; + goto err_reset; if (bp->tph_mode) { rc = bnxt_hwrm_cp_ring_alloc_p5(bp, rxr->rx_cpr); if (rc) - goto err_free_hwrm_rx_ring; + goto err_reset; } rc = bnxt_hwrm_rx_agg_ring_alloc(bp, rxr); if (rc) - goto err_free_hwrm_cp_ring; + goto err_reset; bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); if (bp->flags & BNXT_FLAG_AGG_RINGS) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); + if (bp->flags & BNXT_FLAG_SHARED_RINGS) { + rc = bnxt_tx_queue_start(bp, idx); + if (rc) + goto err_reset; + } + + napi_enable(&bnapi->napi); + bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons); + for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) { vnic = &bp->vnic_info[i]; @@ -15696,11 +15782,12 @@ static int bnxt_queue_start(struct net_device *dev, void *qmem, int idx) return 0; -err_free_hwrm_cp_ring: - if (bp->tph_mode) - bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr); -err_free_hwrm_rx_ring: - bnxt_hwrm_rx_ring_free(bp, rxr, false); +err_reset: + netdev_err(bp->dev, "Unexpected HWRM error during queue start rc: %d\n", + rc); + napi_enable(&bnapi->napi); + bnxt_db_nq_arm(bp, &cpr->cp_db, cpr->cp_raw_cons); + bnxt_reset_task(bp, true); return rc; } @@ -15708,7 +15795,9 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx) { struct bnxt *bp = netdev_priv(dev); struct bnxt_rx_ring_info *rxr; + struct bnxt_cp_ring_info *cpr; struct bnxt_vnic_info *vnic; + struct bnxt_napi *bnapi; int i; for (i = 0; i <= BNXT_VNIC_NTUPLE; i++) { @@ -15720,17 +15809,29 @@ static int bnxt_queue_stop(struct net_device *dev, void *qmem, int idx) /* Make sure NAPI sees that the VNIC is disabled */ synchronize_net(); rxr = &bp->rx_ring[idx]; - cancel_work_sync(&rxr->bnapi->cp_ring.dim.work); + bnapi = rxr->bnapi; + cpr = &bnapi->cp_ring; + cancel_work_sync(&cpr->dim.work); bnxt_hwrm_rx_ring_free(bp, rxr, false); bnxt_hwrm_rx_agg_ring_free(bp, rxr, false); page_pool_disable_direct_recycling(rxr->page_pool); if (bnxt_separate_head_pool()) page_pool_disable_direct_recycling(rxr->head_pool); + if (bp->flags & BNXT_FLAG_SHARED_RINGS) + bnxt_tx_queue_stop(bp, idx); + + /* Disable NAPI now after freeing the rings because HWRM_RING_FREE + * completion is handled in NAPI to guarantee no more DMA on that ring + * after seeing the completion. + */ + napi_disable(&bnapi->napi); + if (bp->tph_mode) { bnxt_hwrm_cp_ring_free(bp, rxr->rx_cpr); bnxt_clear_one_cp_ring(bp, rxr->rx_cpr); } + bnxt_db_nq(bp, &cpr->cp_db, cpr->cp_raw_cons); memcpy(qmem, rxr, sizeof(*rxr)); bnxt_init_rx_ring_struct(bp, qmem); From patchwork Thu Feb 13 01:12:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 13972664 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FC00153598 for ; Thu, 13 Feb 2025 01:13:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409227; cv=none; b=cIKL57ZKwjyoNiZ7SoOp5uNAVzArOgufQdWFkYAC8oVAPpzCjlOzDGBnHXtkmUyd+782vN8toPBBSD+69Ra/uLKG+LRsOfLY92BlYuVv9R2sy0rsVsgIUBdm+uamOQD8e0g+xMZz+fuiAfI4VbJ/drKJkm8AhkxJWaZL0pnsj1A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739409227; c=relaxed/simple; bh=t1ZTNMpHicfVMyIowy0DXLMxgE2fqOtZdkWdpAHwIUI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s8A+9kJb9HMtACqomVVqXon8c00w1EPauq/btpv08fm2C5m+7bWnEF5aMsK8BOKRv52JHI8K0GS/ABbTZt7pmRbFcXz8vhAOH7rUYzG11nxQQMTLwRbdv8jT/Hgo1owfNE9IxNBRg8wOwTwbjGDXQTnH7x3U/+4RvDzdnutT8f4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=DRGo163u; arc=none smtp.client-ip=209.85.210.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="DRGo163u" Received: by mail-ot1-f43.google.com with SMTP id 46e09a7af769-726f1de8190so118268a34.3 for ; Wed, 12 Feb 2025 17:13:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1739409224; x=1740014024; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4R/yPIFFTjU1z1BbTkCQhD01Z9CT3ll1m6Xp1u1mhmY=; b=DRGo163uBszmPi9AnYk+KW+doS9ICEzsfkslc6UhhJ0sXzBUQJAVqOr67biStp/D4U slMUpltAvwuBAvLRYA5qmx4ovBzDg+4VMwEigw2/IkI8cRdIi4V87lTaUb7/sC6QiEej BupWYHDJwPBW46RwdDDzaTvLnzFtB5DOtAb/U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739409224; x=1740014024; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4R/yPIFFTjU1z1BbTkCQhD01Z9CT3ll1m6Xp1u1mhmY=; b=NkkNfjwKZr8EwtD53Sj/SX5AvHrqdnE5+n4NIzwjKVr857IIeMFd2u5EHl9OYs2Ygz yp0/x/6NUQ79jnFicSLHLKOU8eZRtQTi7ZlVkIQ/StyBR/VZkbI3HlbjJPcvVQF8dIQ6 AkhlHVIxt2OCxnM0RYguXnEtwhIMF77CLQ4Vag7xflMreeXjEV+BxhumGowCDAqTzrCk OZlLkSHeEJ3HlwPkEobq1RbSCOaJl6gCHCwFX8jevrn98GIP+bLkM3/IrDLBu3W5fqy/ 4R9LuSr39SWMygOtPJKcMtuNe5XqMR1HB++0ljgz+hNc+P6GLgI3BdphIZAGl8n8MzZf 5eRw== X-Gm-Message-State: AOJu0YyVD1F4nuo1ufK74SnoLyaUR6j2nJpisXoMIG5edcgwWtn2J8IZ 27YcOsTxPaY+HVqn1yqvPt3M4nBkJ7zpDsnhbnDECiVAglwAX98IplxpjxvOrg== X-Gm-Gg: ASbGncvZakyH4XbIurZYX3Yju9jh0Tyn1doKvPxst1qHqg0RtHVcljcw/8pTRJ0Ta6/ Vkpo5nXQ691T3aCtireBvClNW+1uNBQ7LC/wz90xpZO0Tv6p9LaqDbdQnHyCGhvxHwXqBDl8rgE ken3T//uj+FeI0k2RCUY/3BOS4Ku2l2vvy4+nFCl8bQH6PQCg93TvL8I4RkAiw3mnqcp5xGHmK/ yQXaCeBHtoQoV2Qjzqem/l0yJBJZJLtM0wKiBNxb5aUkcTyNO2N36ITHCJ1bCUDVgWOEBBs+lNe GaXvpcy7pQOvBLq+1tIuFqCGhDJdksEfblb/0nVJQkdB/U8xmq0q8l/Nmdsq6BTuAuo= X-Google-Smtp-Source: AGHT+IFDgEgUU+aBUSTGDH1M5Hq/KLaD64Zz7BNNWUzXGYbKnm4egPysQsPnv6ceNlR5OFNcSV1DMg== X-Received: by 2002:a05:6830:3786:b0:71d:fb64:b606 with SMTP id 46e09a7af769-726fe7eea07mr924552a34.23.1739409223871; Wed, 12 Feb 2025 17:13:43 -0800 (PST) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-727001cdb70sm195967a34.13.2025.02.12.17.13.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 17:13:43 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com, michal.swiatkowski@linux.intel.com, helgaas@kernel.org, horms@kernel.org, Manoj Panicker , Somnath Kotur , Wei Huang , Ajit Khaparde Subject: [PATCH net-next v5 11/11] bnxt_en: Add TPH support in BNXT driver Date: Wed, 12 Feb 2025 17:12:39 -0800 Message-ID: <20250213011240.1640031-12-michael.chan@broadcom.com> X-Mailer: git-send-email 2.43.4 In-Reply-To: <20250213011240.1640031-1-michael.chan@broadcom.com> References: <20250213011240.1640031-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Manoj Panicker Add TPH support to the Broadcom BNXT device driver. This allows the driver to utilize TPH functions for retrieving and configuring Steering Tags when changing interrupt affinity. With compatible NIC firmware, network traffic will be tagged correctly with Steering Tags, resulting in significant memory bandwidth savings and other advantages as demonstrated by real network benchmarks on TPH-capable platforms. Co-developed-by: Somnath Kotur Signed-off-by: Somnath Kotur Co-developed-by: Wei Huang Signed-off-by: Wei Huang Signed-off-by: Manoj Panicker Reviewed-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Signed-off-by: Michael Chan --- Cc: Bjorn Helgaas v3: Add MODULE_IMPORT_NS("NETDEV_INTERNAL") Previous driver series fixing rtnl_lock and empty release function: https://lore.kernel.org/netdev/20241115200412.1340286-1-wei.huang2@amd.com/ v5 of the PCI series using netdev_rx_queue_restart(): https://lore.kernel.org/netdev/20240916205103.3882081-5-wei.huang2@amd.com/ v1 of the PCI series using open/close: https://lore.kernel.org/netdev/20240509162741.1937586-9-wei.huang2@amd.com/ --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 106 ++++++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 5 + 2 files changed, 111 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 1997cdbd5801..15c57a06ecaf 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -55,6 +55,8 @@ #include #include #include +#include +#include #include "bnxt_hsi.h" #include "bnxt.h" @@ -76,6 +78,7 @@ #define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \ NETIF_MSG_TX_ERR) +MODULE_IMPORT_NS("NETDEV_INTERNAL"); MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Broadcom NetXtreme network driver"); @@ -11351,6 +11354,83 @@ static int bnxt_tx_queue_start(struct bnxt *bp, int idx) return 0; } +static void bnxt_irq_affinity_notify(struct irq_affinity_notify *notify, + const cpumask_t *mask) +{ + struct bnxt_irq *irq; + u16 tag; + int err; + + irq = container_of(notify, struct bnxt_irq, affinity_notify); + + if (!irq->bp->tph_mode) + return; + + cpumask_copy(irq->cpu_mask, mask); + + if (irq->ring_nr >= irq->bp->rx_nr_rings) + return; + + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM, + cpumask_first(irq->cpu_mask), &tag)) + return; + + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag)) + return; + + rtnl_lock(); + if (netif_running(irq->bp->dev)) { + err = netdev_rx_queue_restart(irq->bp->dev, irq->ring_nr); + if (err) + netdev_err(irq->bp->dev, + "RX queue restart failed: err=%d\n", err); + } + rtnl_unlock(); +} + +static void bnxt_irq_affinity_release(struct kref *ref) +{ + struct irq_affinity_notify *notify = + container_of(ref, struct irq_affinity_notify, kref); + struct bnxt_irq *irq; + + irq = container_of(notify, struct bnxt_irq, affinity_notify); + + if (!irq->bp->tph_mode) + return; + + if (pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, 0)) { + netdev_err(irq->bp->dev, + "Setting ST=0 for MSIX entry %d failed\n", + irq->msix_nr); + return; + } +} + +static void bnxt_release_irq_notifier(struct bnxt_irq *irq) +{ + irq_set_affinity_notifier(irq->vector, NULL); +} + +static void bnxt_register_irq_notifier(struct bnxt *bp, struct bnxt_irq *irq) +{ + struct irq_affinity_notify *notify; + + irq->bp = bp; + + /* Nothing to do if TPH is not enabled */ + if (!bp->tph_mode) + return; + + /* Register IRQ affinity notifier */ + notify = &irq->affinity_notify; + notify->irq = irq->vector; + notify->notify = bnxt_irq_affinity_notify; + notify->release = bnxt_irq_affinity_release; + + irq_set_affinity_notifier(irq->vector, notify); +} + static void bnxt_free_irq(struct bnxt *bp) { struct bnxt_irq *irq; @@ -11373,11 +11453,18 @@ static void bnxt_free_irq(struct bnxt *bp) free_cpumask_var(irq->cpu_mask); irq->have_cpumask = 0; } + + bnxt_release_irq_notifier(irq); + free_irq(irq->vector, bp->bnapi[i]); } irq->requested = 0; } + + /* Disable TPH support */ + pcie_disable_tph(bp->pdev); + bp->tph_mode = 0; } static int bnxt_request_irq(struct bnxt *bp) @@ -11397,6 +11484,12 @@ static int bnxt_request_irq(struct bnxt *bp) #ifdef CONFIG_RFS_ACCEL rmap = bp->dev->rx_cpu_rmap; #endif + + /* Enable TPH support as part of IRQ request */ + rc = pcie_enable_tph(bp->pdev, PCI_TPH_ST_IV_MODE); + if (!rc) + bp->tph_mode = PCI_TPH_ST_IV_MODE; + for (i = 0, j = 0; i < bp->cp_nr_rings; i++) { int map_idx = bnxt_cp_num_to_irq_num(bp, i); struct bnxt_irq *irq = &bp->irq_tbl[map_idx]; @@ -11420,8 +11513,11 @@ static int bnxt_request_irq(struct bnxt *bp) if (zalloc_cpumask_var(&irq->cpu_mask, GFP_KERNEL)) { int numa_node = dev_to_node(&bp->pdev->dev); + u16 tag; irq->have_cpumask = 1; + irq->msix_nr = map_idx; + irq->ring_nr = i; cpumask_set_cpu(cpumask_local_spread(i, numa_node), irq->cpu_mask); rc = irq_update_affinity_hint(irq->vector, irq->cpu_mask); @@ -11431,6 +11527,16 @@ static int bnxt_request_irq(struct bnxt *bp) irq->vector); break; } + + bnxt_register_irq_notifier(bp, irq); + + /* Init ST table entry */ + if (pcie_tph_get_cpu_st(irq->bp->pdev, TPH_MEM_TYPE_VM, + cpumask_first(irq->cpu_mask), + &tag)) + continue; + + pcie_tph_set_st_entry(irq->bp->pdev, irq->msix_nr, tag); } } return rc; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 4e20878e7714..e85b5ce94f58 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1234,6 +1234,11 @@ struct bnxt_irq { u8 have_cpumask:1; char name[IFNAMSIZ + BNXT_IRQ_NAME_EXTRA]; cpumask_var_t cpu_mask; + + struct bnxt *bp; + int msix_nr; + int ring_nr; + struct irq_affinity_notify affinity_notify; }; #define HWRM_RING_ALLOC_TX 0x1