From patchwork Thu Jan 9 10:32:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shinas Rasheed X-Patchwork-Id: 13932337 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39572201017; Thu, 9 Jan 2025 10:32:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.148.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736418764; cv=none; b=Mwl5YN7a9hC50fUr3fIL4EEEj9n260Djm3SlKN0hXtfNGBrZQIf0AfYoy4mj0Hb0yKZG9pRkiM6tntxhYn7v9Qdv2M18GriJ4GGgAzJtevygB1YvKMTatu7OLiaPfd43s1BhskS7beMg7Jy59n9BiJTokFhKE/856VCxPUWhAOU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736418764; c=relaxed/simple; bh=FHiFxfddVY4Pw92WR1BZd4w57EVpaf+7sa03OwQVaoA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iOhgQtvPr1zgL3YH6cwhzrw+dZLFesJ1w7jNX2BFsoYbj8JU2S0BhsnzFhSoN2I0r0YrzmGwgfJA3n8mJvl2LDhQXCFnE6OIEVagcPu0Q8+/Yw5KTaM2NohaAQh1N787n7LoIboc1vuN/VKuZL9cB2fNK3NhLvWFbH9vGtJ9xOI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com; spf=pass smtp.mailfrom=marvell.com; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b=c/DW2dN0; arc=none smtp.client-ip=67.231.148.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=marvell.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="c/DW2dN0" Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5099MBJx031734; Thu, 9 Jan 2025 02:32:27 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=N RCTvT/DSQyr+sAcxJ8/PN8n8rTpNkRVFViXZKL6/oI=; b=c/DW2dN0REItg2/x2 KXG+nCT4EpQspR2nwWbKa6bNZjl39AD+7jRmjojdafRYhZwcK24Au3XuuGEwu4wB vxcCUNPyXTN5ddos8cn3RgpA3habqIc3UUMZv8nnj3355QsvXr5iNjS0IcmGBpVS Gs/8TNaEj/WH/mM5zPdvNJsU2NIujYmYLuNbDfHVkcIoCZUosoridsylszcWDlZn snm1834/23vP0tTmlIwRjKc/0dlcHGSwBHuP2Zq9S0eMEwmJBt1uaykU2e7AXn3u 3xoDrQAMjVOhAYj7Ys6cxJV8dqIOXfLxerR/GC7RMBT0rzVJDr3RYlhU6SXNLDmP z32gA== Received: from dc6wp-exch02.marvell.com ([4.21.29.225]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 442brv04qn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 09 Jan 2025 02:32:27 -0800 (PST) Received: from DC6WP-EXCH02.marvell.com (10.76.176.209) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 9 Jan 2025 02:32:26 -0800 Received: from maili.marvell.com (10.69.176.80) by DC6WP-EXCH02.marvell.com (10.76.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Thu, 9 Jan 2025 02:32:26 -0800 Received: from ubuntu-PowerEdge-T110-II.sclab.marvell.com (unknown [10.106.27.86]) by maili.marvell.com (Postfix) with ESMTP id 829BB3F7072; Thu, 9 Jan 2025 02:32:25 -0800 (PST) From: Shinas Rasheed To: , CC: , , , , , , , , , "Shinas Rasheed" , Veerasenareddy Burru , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , "Paolo Abeni" Subject: [PATCH net v5 1/4] octeon_ep: update tx/rx stats locally for persistence Date: Thu, 9 Jan 2025 02:32:17 -0800 Message-ID: <20250109103221.2544467-2-srasheed@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250109103221.2544467-1-srasheed@marvell.com> References: <20250109103221.2544467-1-srasheed@marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: u_eMjDuxqmauPFsVt-Fjsleqk4z91hT6 X-Proofpoint-GUID: u_eMjDuxqmauPFsVt-Fjsleqk4z91hT6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Patchwork-Delegate: kuba@kernel.org Update tx/rx stats locally, so that ndo_get_stats64() can use that and not rely on per queue resources to obtain statistics. The latter used to cause race conditions when the device stopped. Signed-off-by: Shinas Rasheed --- V5: - Patch introduced .../marvell/octeon_ep/octep_ethtool.c | 41 ++++++++----------- .../ethernet/marvell/octeon_ep/octep_main.c | 37 ++++++++--------- .../ethernet/marvell/octeon_ep/octep_main.h | 11 +++++ .../net/ethernet/marvell/octeon_ep/octep_rx.c | 12 +++--- .../net/ethernet/marvell/octeon_ep/octep_rx.h | 4 +- .../net/ethernet/marvell/octeon_ep/octep_tx.c | 7 ++-- .../net/ethernet/marvell/octeon_ep/octep_tx.h | 4 +- 7 files changed, 60 insertions(+), 56 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_ethtool.c b/drivers/net/ethernet/marvell/octeon_ep/octep_ethtool.c index 4f4d58189118..79d66426c1da 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_ethtool.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_ethtool.c @@ -150,17 +150,14 @@ octep_get_ethtool_stats(struct net_device *netdev, iface_rx_stats, iface_tx_stats); - for (q = 0; q < oct->num_oqs; q++) { - struct octep_iq *iq = oct->iq[q]; - struct octep_oq *oq = oct->oq[q]; - - tx_packets += iq->stats.instr_completed; - tx_bytes += iq->stats.bytes_sent; - tx_busy_errors += iq->stats.tx_busy; - - rx_packets += oq->stats.packets; - rx_bytes += oq->stats.bytes; - rx_alloc_errors += oq->stats.alloc_failures; + for (q = 0; q < oct->num_ioq_stats; q++) { + tx_packets += oct->stats_iq[q].instr_completed; + tx_bytes += oct->stats_iq[q].bytes_sent; + tx_busy_errors += oct->stats_iq[q].tx_busy; + + rx_packets += oct->stats_oq[q].packets; + rx_bytes += oct->stats_oq[q].bytes; + rx_alloc_errors += oct->stats_oq[q].alloc_failures; } i = 0; data[i++] = rx_packets; @@ -198,22 +195,18 @@ octep_get_ethtool_stats(struct net_device *netdev, data[i++] = iface_rx_stats->err_pkts; /* Per Tx Queue stats */ - for (q = 0; q < oct->num_iqs; q++) { - struct octep_iq *iq = oct->iq[q]; - - data[i++] = iq->stats.instr_posted; - data[i++] = iq->stats.instr_completed; - data[i++] = iq->stats.bytes_sent; - data[i++] = iq->stats.tx_busy; + for (q = 0; q < oct->num_ioq_stats; q++) { + data[i++] = oct->stats_iq[q].instr_posted; + data[i++] = oct->stats_iq[q].instr_completed; + data[i++] = oct->stats_iq[q].bytes_sent; + data[i++] = oct->stats_iq[q].tx_busy; } /* Per Rx Queue stats */ - for (q = 0; q < oct->num_oqs; q++) { - struct octep_oq *oq = oct->oq[q]; - - data[i++] = oq->stats.packets; - data[i++] = oq->stats.bytes; - data[i++] = oq->stats.alloc_failures; + for (q = 0; q < oct->num_ioq_stats; q++) { + data[i++] = oct->stats_oq[q].packets; + data[i++] = oct->stats_oq[q].bytes; + data[i++] = oct->stats_oq[q].alloc_failures; } } diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_main.c b/drivers/net/ethernet/marvell/octeon_ep/octep_main.c index 549436efc204..c5fdedd3ea78 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_main.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_main.c @@ -822,7 +822,7 @@ static inline int octep_iq_full_check(struct octep_iq *iq) if (unlikely(IQ_INSTR_SPACE(iq) > OCTEP_WAKE_QUEUE_THRESHOLD)) { netif_start_subqueue(iq->netdev, iq->q_no); - iq->stats.restart_cnt++; + iq->stats->restart_cnt++; return 0; } @@ -960,7 +960,7 @@ static netdev_tx_t octep_start_xmit(struct sk_buff *skb, wmb(); /* Ring Doorbell to notify the NIC of new packets */ writel(iq->fill_cnt, iq->doorbell_reg); - iq->stats.instr_posted += iq->fill_cnt; + iq->stats->instr_posted += iq->fill_cnt; iq->fill_cnt = 0; return NETDEV_TX_OK; @@ -991,33 +991,30 @@ static netdev_tx_t octep_start_xmit(struct sk_buff *skb, static void octep_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *stats) { - u64 tx_packets, tx_bytes, rx_packets, rx_bytes; struct octep_device *oct = netdev_priv(netdev); int q; + oct->iface_tx_stats.pkts = 0; + oct->iface_tx_stats.octs = 0; + oct->iface_rx_stats.pkts = 0; + oct->iface_rx_stats.octets = 0; + for (q = 0; q < oct->num_ioq_stats; q++) { + oct->iface_tx_stats.pkts += oct->stats_iq[q].instr_completed; + oct->iface_tx_stats.octs += oct->stats_iq[q].bytes_sent; + oct->iface_rx_stats.pkts += oct->stats_oq[q].packets; + oct->iface_rx_stats.octets += oct->stats_oq[q].bytes; + } + if (netif_running(netdev)) octep_ctrl_net_get_if_stats(oct, OCTEP_CTRL_NET_INVALID_VFID, &oct->iface_rx_stats, &oct->iface_tx_stats); - tx_packets = 0; - tx_bytes = 0; - rx_packets = 0; - rx_bytes = 0; - for (q = 0; q < oct->num_oqs; q++) { - struct octep_iq *iq = oct->iq[q]; - struct octep_oq *oq = oct->oq[q]; - - tx_packets += iq->stats.instr_completed; - tx_bytes += iq->stats.bytes_sent; - rx_packets += oq->stats.packets; - rx_bytes += oq->stats.bytes; - } - stats->tx_packets = tx_packets; - stats->tx_bytes = tx_bytes; - stats->rx_packets = rx_packets; - stats->rx_bytes = rx_bytes; + stats->tx_packets = oct->iface_tx_stats.pkts; + stats->tx_bytes = oct->iface_tx_stats.octs; + stats->rx_packets = oct->iface_rx_stats.pkts; + stats->rx_bytes = oct->iface_rx_stats.octets; stats->multicast = oct->iface_rx_stats.mcast_pkts; stats->rx_errors = oct->iface_rx_stats.err_pkts; stats->collisions = oct->iface_tx_stats.xscol; diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_main.h b/drivers/net/ethernet/marvell/octeon_ep/octep_main.h index fee59e0e0138..231ff863310b 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_main.h +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_main.h @@ -257,11 +257,22 @@ struct octep_device { /* Pointers to Octeon Tx queues */ struct octep_iq *iq[OCTEP_MAX_IQ]; + /* Per iq stats */ + struct octep_iq_stats stats_iq[OCTEP_MAX_IQ]; + /* Rx queues (OQ: Output Queue) */ u16 num_oqs; /* Pointers to Octeon Rx queues */ struct octep_oq *oq[OCTEP_MAX_OQ]; + /* Number oq stats preserved + * This number would remain constant when device goes down + * This will be updated when device comes back up + */ + u16 num_ioq_stats; + /* Per oq stats */ + struct octep_oq_stats stats_oq[OCTEP_MAX_OQ]; + /* Hardware port number of the PCIe interface */ u16 pcie_port; diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c b/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c index 8af75cb37c3e..a7337f391a21 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_rx.c @@ -87,7 +87,7 @@ static int octep_oq_refill(struct octep_device *oct, struct octep_oq *oq) page = dev_alloc_page(); if (unlikely(!page)) { dev_err(oq->dev, "refill: rx buffer alloc failed\n"); - oq->stats.alloc_failures++; + oq->stats->alloc_failures++; break; } @@ -98,7 +98,7 @@ static int octep_oq_refill(struct octep_device *oct, struct octep_oq *oq) "OQ-%d buffer refill: DMA mapping error!\n", oq->q_no); put_page(page); - oq->stats.alloc_failures++; + oq->stats->alloc_failures++; break; } oq->buff_info[refill_idx].page = page; @@ -134,6 +134,7 @@ static int octep_setup_oq(struct octep_device *oct, int q_no) oq->netdev = oct->netdev; oq->dev = &oct->pdev->dev; oq->q_no = q_no; + oq->stats = &oct->stats_oq[q_no]; oq->max_count = CFG_GET_OQ_NUM_DESC(oct->conf); oq->ring_size_mask = oq->max_count - 1; oq->buffer_size = CFG_GET_OQ_BUF_SIZE(oct->conf); @@ -262,6 +263,7 @@ int octep_setup_oqs(struct octep_device *oct) dev_dbg(&oct->pdev->dev, "Successfully setup OQ(RxQ)-%d.\n", i); } + oct->num_ioq_stats = oct->num_oqs; return 0; oq_setup_err: @@ -443,7 +445,7 @@ static int __octep_oq_process_rx(struct octep_device *oct, if (!skb) { octep_oq_drop_rx(oq, buff_info, &read_idx, &desc_used); - oq->stats.alloc_failures++; + oq->stats->alloc_failures++; continue; } skb_reserve(skb, data_offset); @@ -494,8 +496,8 @@ static int __octep_oq_process_rx(struct octep_device *oct, oq->host_read_idx = read_idx; oq->refill_count += desc_used; - oq->stats.packets += pkt; - oq->stats.bytes += rx_bytes; + oq->stats->packets += pkt; + oq->stats->bytes += rx_bytes; return pkt; } diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_rx.h b/drivers/net/ethernet/marvell/octeon_ep/octep_rx.h index 3b08e2d560dc..b4696c93d0e6 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_rx.h +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_rx.h @@ -186,8 +186,8 @@ struct octep_oq { */ u8 __iomem *pkts_sent_reg; - /* Statistics for this OQ. */ - struct octep_oq_stats stats; + /* Pointer to statistics for this OQ. */ + struct octep_oq_stats *stats; /* Packets pending to be processed */ u32 pkts_pending; diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c index 06851b78aa28..08ee90013fef 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.c @@ -81,9 +81,9 @@ int octep_iq_process_completions(struct octep_iq *iq, u16 budget) } iq->pkts_processed += compl_pkts; - iq->stats.instr_completed += compl_pkts; - iq->stats.bytes_sent += compl_bytes; - iq->stats.sgentry_sent += compl_sg; + iq->stats->instr_completed += compl_pkts; + iq->stats->bytes_sent += compl_bytes; + iq->stats->sgentry_sent += compl_sg; iq->flush_index = fi; netdev_tx_completed_queue(iq->netdev_q, compl_pkts, compl_bytes); @@ -187,6 +187,7 @@ static int octep_setup_iq(struct octep_device *oct, int q_no) iq->netdev = oct->netdev; iq->dev = &oct->pdev->dev; iq->q_no = q_no; + iq->stats = &oct->stats_iq[q_no]; iq->max_count = CFG_GET_IQ_NUM_DESC(oct->conf); iq->ring_size_mask = iq->max_count - 1; iq->fill_threshold = CFG_GET_IQ_DB_MIN(oct->conf); diff --git a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h index 875a2c34091f..58fb39dda977 100644 --- a/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h +++ b/drivers/net/ethernet/marvell/octeon_ep/octep_tx.h @@ -170,8 +170,8 @@ struct octep_iq { */ u16 flush_index; - /* Statistics for this input queue. */ - struct octep_iq_stats stats; + /* Pointer to statistics for this input queue. */ + struct octep_iq_stats *stats; /* Pointer to the Virtual Base addr of the input ring. */ struct octep_tx_desc_hw *desc_ring;