From patchwork Wed Oct 6 07:18:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Subbaraya Sundeep X-Patchwork-Id: 12538761 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E490BC433EF for ; Wed, 6 Oct 2021 07:19:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CDA1561154 for ; Wed, 6 Oct 2021 07:19:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237399AbhJFHU6 (ORCPT ); Wed, 6 Oct 2021 03:20:58 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:42224 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236829AbhJFHU5 (ORCPT ); Wed, 6 Oct 2021 03:20:57 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1961wQYU028888; Wed, 6 Oct 2021 00:19:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=p+XHXftOTAllZG7o/Jjk0yxhGs+BKI2zNRFCCLZ/8n0=; b=DcWO27Y+RIYtskmUimmKrQH/vVBNKEovrSpeh+OKer5AXLaJW20O7nLYCmGSdD8bPOm5 cl/A9BzMxGanFZb2+8+wdJWh1aKDnl0dS9BbFv9S1CCENEJ1ip6N10tolzE8vuTRISGJ HHIzqdCYbe1KiIgMahCIkZQIY/oV6IZxcJq1eJwzacz2jgBov3/Zs7mizVN67llZeKf+ 1QQbgfDflMGLwOt1xuwX/OAWmHUbB/osdTaprwdepxYX9btI0XUOqMfIm6668SmruFDs psvntP0s9adif4DuyilBUxhJ0jRSqdKajG4MSO+lO8pPv82sX2vMpRyRHyjeIEXg/ZBY KA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bgr1qkj84-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 06 Oct 2021 00:19:01 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 6 Oct 2021 00:19:00 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 6 Oct 2021 00:19:00 -0700 Received: from hyd1358.marvell.com (unknown [10.29.37.11]) by maili.marvell.com (Postfix) with ESMTP id E08553F7054; Wed, 6 Oct 2021 00:18:57 -0700 (PDT) From: Subbaraya Sundeep To: , , CC: , , , Subbaraya Sundeep Subject: [net-next PATCH v2 1/3] octeontx2-pf: Simplify the receive buffer size calculation Date: Wed, 6 Oct 2021 12:48:44 +0530 Message-ID: <1633504726-30751-2-git-send-email-sbhatta@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1633504726-30751-1-git-send-email-sbhatta@marvell.com> References: <1633504726-30751-1-git-send-email-sbhatta@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 7seUOW0NMr0DQ7ZMokJ4KFQSMHk3-7_I X-Proofpoint-GUID: 7seUOW0NMr0DQ7ZMokJ4KFQSMHk3-7_I X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-05_06,2021-10-04_01,2020-04-07_01 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org This patch separates the logic of configuring hardware maximum transmit frame size and receive frame size. This simplifies the logic to calculate receive buffer size and using cqe descriptor of different size. Also additional size of skb_shared_info structure is allocated for each receive buffer pointer given to hardware which is not necessary. Hence change the size calculation to remove the size of skb_shared_info. Add a check for array out of bounds while adding fragments to the network stack. Signed-off-by: Subbaraya Sundeep Signed-off-by: Hariprasad Kelam Signed-off-by: Sunil Goutham --- drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c | 2 +- .../ethernet/marvell/octeontx2/nic/otx2_common.c | 10 +++----- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 2 +- .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 15 +++++------ .../net/ethernet/marvell/octeontx2/nic/otx2_txrx.c | 30 +++++++++++++++------- .../net/ethernet/marvell/octeontx2/nic/otx2_txrx.h | 4 +-- 6 files changed, 35 insertions(+), 28 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c index 95f21df..fd4f083 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c @@ -88,7 +88,7 @@ int cn10k_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura) aq->sq.ena = 1; /* Only one SMQ is allocated, map all SQ's to that SMQ */ aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0]; - aq->sq.smq_rr_weight = mtu_to_dwrr_weight(pfvf, pfvf->max_frs); + aq->sq.smq_rr_weight = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); aq->sq.default_chan = pfvf->hw.tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */ aq->sq.sqb_aura = sqb_aura; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 0c89eb8..66da31f 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -231,7 +231,7 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) return -ENOMEM; } - req->maxlen = pfvf->max_frs; + req->maxlen = pfvf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; err = otx2_sync_mbox_msg(&pfvf->mbox); mutex_unlock(&pfvf->mbox.lock); @@ -590,7 +590,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) u64 schq, parent; u64 dwrr_val; - dwrr_val = mtu_to_dwrr_weight(pfvf, pfvf->max_frs); + dwrr_val = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); req = otx2_mbox_alloc_msg_nix_txschq_cfg(&pfvf->mbox); if (!req) @@ -603,9 +603,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) /* Set topology e.t.c configuration */ if (lvl == NIX_TXSCH_LVL_SMQ) { req->reg[0] = NIX_AF_SMQX_CFG(schq); - req->regval[0] = ((pfvf->netdev->max_mtu + OTX2_ETH_HLEN) << 8) - | OTX2_MIN_MTU; - + req->regval[0] = ((u64)pfvf->tx_max_pktlen << 8) | OTX2_MIN_MTU; req->regval[0] |= (0x20ULL << 51) | (0x80ULL << 39) | (0x2ULL << 36); req->num_regs++; @@ -800,7 +798,7 @@ int otx2_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura) aq->sq.ena = 1; /* Only one SMQ is allocated, map all SQ's to that SMQ */ aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0]; - aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->max_frs); + aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); aq->sq.default_chan = pfvf->hw.tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */ aq->sq.sqb_aura = sqb_aura; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index a05f6bd..61e5281 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -326,7 +326,7 @@ struct otx2_nic { struct net_device *netdev; struct dev_hw_ops *hw_ops; void *iommu_domain; - u16 max_frs; + u16 tx_max_pktlen; u16 rbsize; /* Receive buffer size */ #define OTX2_FLAG_RX_TSTAMP_ENABLED BIT_ULL(0) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index f24e920..1e0d0c9c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -1312,16 +1312,14 @@ static int otx2_get_rbuf_size(struct otx2_nic *pf, int mtu) * NIX transfers entire data using 6 segments/buffers and writes * a CQE_RX descriptor with those segment addresses. First segment * has additional data prepended to packet. Also software omits a - * headroom of 128 bytes and sizeof(struct skb_shared_info) in - * each segment. Hence the total size of memory needed - * to receive a packet with 'mtu' is: + * headroom of 128 bytes in each segment. Hence the total size of + * memory needed to receive a packet with 'mtu' is: * frame size = mtu + additional data; - * memory = frame_size + (headroom + struct skb_shared_info size) * 6; + * memory = frame_size + headroom * 6; * each receive buffer size = memory / 6; */ frame_size = mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; - total_size = frame_size + (OTX2_HEAD_ROOM + - OTX2_DATA_ALIGN(sizeof(struct skb_shared_info))) * 6; + total_size = frame_size + OTX2_HEAD_ROOM * 6; rbuf_size = total_size / 6; return ALIGN(rbuf_size, 2048); @@ -1343,7 +1341,8 @@ static int otx2_init_hw_resources(struct otx2_nic *pf) hw->sqpool_cnt = hw->tot_tx_queues; hw->pool_cnt = hw->rqpool_cnt + hw->sqpool_cnt; - pf->max_frs = pf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; + /* Maximum hardware supported transmit length */ + pf->tx_max_pktlen = pf->netdev->max_mtu + OTX2_ETH_HLEN; pf->rbsize = otx2_get_rbuf_size(pf, pf->netdev->mtu); @@ -1807,7 +1806,7 @@ static netdev_tx_t otx2_xmit(struct sk_buff *skb, struct net_device *netdev) /* Check for minimum and maximum packet length */ if (skb->len <= ETH_HLEN || - (!skb_shinfo(skb)->gso_size && skb->len > pf->max_frs)) { + (!skb_shinfo(skb)->gso_size && skb->len > pf->tx_max_pktlen)) { dev_kfree_skb(skb); return NETDEV_TX_OK; } diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index 2d2b673..0cc6353 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -181,8 +181,9 @@ static void otx2_set_rxtstamp(struct otx2_nic *pfvf, skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(tsns); } -static void otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb, - u64 iova, int len, struct nix_rx_parse_s *parse) +static bool otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb, + u64 iova, int len, struct nix_rx_parse_s *parse, + int qidx) { struct page *page; int off = 0; @@ -203,11 +204,22 @@ static void otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb, } page = virt_to_page(va); - skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, - va - page_address(page) + off, len - off, pfvf->rbsize); + if (likely(skb_shinfo(skb)->nr_frags < MAX_SKB_FRAGS)) { + skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, + va - page_address(page) + off, + len - off, pfvf->rbsize); + + otx2_dma_unmap_page(pfvf, iova - OTX2_HEAD_ROOM, + pfvf->rbsize, DMA_FROM_DEVICE); + return true; + } - otx2_dma_unmap_page(pfvf, iova - OTX2_HEAD_ROOM, - pfvf->rbsize, DMA_FROM_DEVICE); + /* If more than MAX_SKB_FRAGS fragments are received then + * give back those buffer pointers to hardware for reuse. + */ + pfvf->hw_ops->aura_freeptr(pfvf, qidx, iova & ~0x07ULL); + + return false; } static void otx2_set_rxhash(struct otx2_nic *pfvf, @@ -349,9 +361,9 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, seg_addr = &sg->seg_addr; seg_size = (void *)sg; for (seg = 0; seg < sg->segs; seg++, seg_addr++) { - otx2_skb_add_frag(pfvf, skb, *seg_addr, seg_size[seg], - parse); - cq->pool_ptrs++; + if (otx2_skb_add_frag(pfvf, skb, *seg_addr, + seg_size[seg], parse, cq->cq_idx)) + cq->pool_ptrs++; } start += sizeof(*sg); } diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h index 5c05774..f1a04cf 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h @@ -39,9 +39,7 @@ #define RCV_FRAG_LEN(x) \ ((RCV_FRAG_LEN1(x) < 2048) ? 2048 : RCV_FRAG_LEN1(x)) -#define DMA_BUFFER_LEN(x) \ - ((x) - OTX2_HEAD_ROOM - \ - OTX2_DATA_ALIGN(sizeof(struct skb_shared_info))) +#define DMA_BUFFER_LEN(x) ((x) - OTX2_HEAD_ROOM) /* IRQ triggered when NIX_LF_CINTX_CNT[ECOUNT] * is equal to this value. From patchwork Wed Oct 6 07:18:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Subbaraya Sundeep X-Patchwork-Id: 12538763 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AB43C433FE for ; Wed, 6 Oct 2021 07:19:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F16F61154 for ; Wed, 6 Oct 2021 07:19:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237443AbhJFHVA (ORCPT ); Wed, 6 Oct 2021 03:21:00 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:57316 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236829AbhJFHU6 (ORCPT ); Wed, 6 Oct 2021 03:20:58 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19620stE030285; Wed, 6 Oct 2021 00:19:04 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=lsTPQJbq1BoWC63f6Z/OJhplQKO6S+lgfgJHE06Md1E=; b=XIxObnTuSfKmZSSDRYMPuW0Lz5dNznVJHKTbBkExZbU9U4u9mjXtpgsK/HjAhfucxwbV 2ypEdo43/XOcwIauRK1FBC5lKC2L+cFCEF6jin0GoOoCAAeJxDl/xmRpPgyxZrpUh4o8 x7ny/5+Sba7mpzZVgFXIRo/b2vq8DBBiDfvmCgWFeXLfbiFQ/zVx0R4ZIZuO/LtwkPYJ u1v5m1bIy1LdFDMkc4mxSuxA7j2nUcFvGT4n29zx7TCuEhuWEH/hnF+h4MNe7MXM6GnN SMpbmnmAkjwXWzyzi43GvrQMahbBJQnh2DVjrtmo2J1Y1wOjOjvhLmGD66dUPzpgHaHp 4w== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bgr1qkj8j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 06 Oct 2021 00:19:04 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 6 Oct 2021 00:19:02 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 6 Oct 2021 00:19:02 -0700 Received: from hyd1358.marvell.com (unknown [10.29.37.11]) by maili.marvell.com (Postfix) with ESMTP id 8B6BC3F704C; Wed, 6 Oct 2021 00:19:00 -0700 (PDT) From: Subbaraya Sundeep To: , , CC: , , , Subbaraya Sundeep Subject: [net-next PATCH v2 2/3] octeontx2-pf: Add devlink param to vary cqe size Date: Wed, 6 Oct 2021 12:48:45 +0530 Message-ID: <1633504726-30751-3-git-send-email-sbhatta@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1633504726-30751-1-git-send-email-sbhatta@marvell.com> References: <1633504726-30751-1-git-send-email-sbhatta@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: SYh-PMQHljueHYNjaU4dTSFHtn80f_VU X-Proofpoint-GUID: SYh-PMQHljueHYNjaU4dTSFHtn80f_VU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-05_06,2021-10-04_01,2020-04-07_01 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Completion Queue Entry(CQE) is a descriptor written by hardware to notify software about the send and receive completion status. The CQE can be of size 128 or 512 bytes. A 512 bytes CQE can hold more receive fragments pointers compared to 128 bytes CQE. This patch adds devlink param to change CQE descriptor size. Signed-off-by: Subbaraya Sundeep Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.c | 10 +++- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 + .../ethernet/marvell/octeontx2/nic/otx2_devlink.c | 56 ++++++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 2 + .../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 2 + 5 files changed, 69 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 66da31f..3777f41 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -222,8 +222,11 @@ EXPORT_SYMBOL(otx2_set_mac_address); int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) { struct nix_frs_cfg *req; + u16 maxlen; int err; + maxlen = otx2_get_max_mtu(pfvf) + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; + mutex_lock(&pfvf->mbox.lock); req = otx2_mbox_alloc_msg_nix_set_hw_frs(&pfvf->mbox); if (!req) { @@ -233,6 +236,9 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) req->maxlen = pfvf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; + if (is_otx2_lbkvf(pfvf->pdev)) + req->maxlen = maxlen; + err = otx2_sync_mbox_msg(&pfvf->mbox); mutex_unlock(&pfvf->mbox.lock); return err; @@ -1036,7 +1042,7 @@ int otx2_config_nix(struct otx2_nic *pfvf) struct nix_lf_alloc_rsp *rsp; int err; - pfvf->qset.xqe_size = NIX_XQESZ_W16 ? 128 : 512; + pfvf->qset.xqe_size = pfvf->hw.xqe_size; /* Get memory to put this msg */ nixlf = otx2_mbox_alloc_msg_nix_lf_alloc(&pfvf->mbox); @@ -1049,7 +1055,7 @@ int otx2_config_nix(struct otx2_nic *pfvf) nixlf->cq_cnt = pfvf->qset.cq_cnt; nixlf->rss_sz = MAX_RSS_INDIR_TBL_SIZE; nixlf->rss_grps = MAX_RSS_GROUPS; - nixlf->xqe_sz = NIX_XQESZ_W16; + nixlf->xqe_sz = pfvf->hw.xqe_size == 128 ? NIX_XQESZ_W16 : NIX_XQESZ_W64; /* We don't know absolute NPA LF idx attached. * AF will replace 'RVU_DEFAULT_PF_FUNC' with * NPA LF attached to this RVU PF/VF. diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 61e5281..6e0d1ac 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -177,6 +177,7 @@ struct otx2_hw { u16 pool_cnt; u16 rqpool_cnt; u16 sqpool_cnt; + u16 xqe_size; /* NPA */ u32 stack_pg_ptrs; /* No of ptrs per stack page */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c index 777a270..98450e1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c @@ -64,9 +64,60 @@ static int otx2_dl_mcam_count_get(struct devlink *devlink, u32 id, return 0; } +static int otx2_dl_cqe_size_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + if (val.vu16 != 128 && val.vu16 != 512) { + NL_SET_ERR_MSG_MOD(extack, + "Only 128 or 512 byte descriptor allowed"); + return -EINVAL; + } + + return 0; +} + +static int otx2_dl_cqe_size_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + struct net_device *netdev; + int err = 0; + bool if_up; + + rtnl_lock(); + + netdev = pfvf->netdev; + if_up = netif_running(netdev); + if (if_up) + netdev->netdev_ops->ndo_stop(netdev); + + pfvf->hw.xqe_size = ctx->val.vu16; + + if (if_up) + err = netdev->netdev_ops->ndo_open(netdev); + + rtnl_unlock(); + + return err; +} + +static int otx2_dl_cqe_size_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + + ctx->val.vu16 = pfvf->hw.xqe_size; + + return 0; +} + enum otx2_dl_param_id { OTX2_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, OTX2_DEVLINK_PARAM_ID_MCAM_COUNT, + OTX2_DEVLINK_PARAM_ID_CQE_SIZE, }; static const struct devlink_param otx2_dl_params[] = { @@ -75,6 +126,11 @@ static const struct devlink_param otx2_dl_params[] = { BIT(DEVLINK_PARAM_CMODE_RUNTIME), otx2_dl_mcam_count_get, otx2_dl_mcam_count_set, otx2_dl_mcam_count_validate), + DEVLINK_PARAM_DRIVER(OTX2_DEVLINK_PARAM_ID_CQE_SIZE, + "completion_descriptor_size", DEVLINK_PARAM_TYPE_U16, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + otx2_dl_cqe_size_get, otx2_dl_cqe_size_set, + otx2_dl_cqe_size_validate), }; /* Devlink OPs */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 1e0d0c9c..8618cf7 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -2624,6 +2624,8 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id) hw->tx_queues = qcount; hw->tot_tx_queues = qcount; hw->max_queues = qcount; + /* Use CQE of 128 byte descriptor size by default */ + hw->xqe_size = 128; num_vec = pci_msix_vec_count(pdev); hw->irq_name = devm_kmalloc_array(&hw->pdev->dev, num_vec, NAME_SIZE, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 980219a..672be05 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -587,6 +587,8 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id) hw->tx_queues = qcount; hw->max_queues = qcount; hw->tot_tx_queues = qcount; + /* Use CQE of 128 byte descriptor size by default */ + hw->xqe_size = 128; hw->irq_name = devm_kmalloc_array(&hw->pdev->dev, num_vec, NAME_SIZE, GFP_KERNEL); From patchwork Wed Oct 6 07:18:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Subbaraya Sundeep X-Patchwork-Id: 12538765 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64468C433F5 for ; Wed, 6 Oct 2021 07:19:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47F6C61100 for ; Wed, 6 Oct 2021 07:19:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237484AbhJFHVF (ORCPT ); Wed, 6 Oct 2021 03:21:05 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:2336 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237419AbhJFHVA (ORCPT ); Wed, 6 Oct 2021 03:21:00 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19620stG030285; Wed, 6 Oct 2021 00:19:07 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=Um9s9/TLHNrNMcB2c9SFF6AaxMvdu82WG4m2JK9f+8k=; b=VrcajbmX/jKMX84LNBl0BNHgP3sUJciDONtZVgyHOGX5YynAlJpG+uFKsdRPHqr9DzQc wtcuqjsgqPS5RMURLSiqveIfO1bWZM7b5XqwoJF8m4+iekPAvwiQXo7+eK6sx68fLEHz 0N6H5qMVkTe6UcMUthpFKeQX8w0TO/jh62z7VCnhzPcyQaQTPhWK8NBdvHTZ/k38gff6 oNDlm7XmSiayA6m6c7K0oFMIF/a3zjDIIe7Z5P5o2xEzIrfhORpLV42oyb9QPi4/8QES DTkPhkxFCxTlXqh8U6hfh/lBrrH4WC2fCxZRCDT/eACDl5PDVcFEoTJAZyJpU9oPK+zH Ww== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bgr1qkj8j-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 06 Oct 2021 00:19:06 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Wed, 6 Oct 2021 00:19:05 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 6 Oct 2021 00:19:05 -0700 Received: from hyd1358.marvell.com (unknown [10.29.37.11]) by maili.marvell.com (Postfix) with ESMTP id 3283E3F7054; Wed, 6 Oct 2021 00:19:02 -0700 (PDT) From: Subbaraya Sundeep To: , , CC: , , , Subbaraya Sundeep Subject: [net-next PATCH v2 3/3] octeontx2-pf: Add devlink param to vary rbuf size Date: Wed, 6 Oct 2021 12:48:46 +0530 Message-ID: <1633504726-30751-4-git-send-email-sbhatta@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1633504726-30751-1-git-send-email-sbhatta@marvell.com> References: <1633504726-30751-1-git-send-email-sbhatta@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 4z5b_9Kc0B4epfjW-ZUM4jQpFAavb4SP X-Proofpoint-GUID: 4z5b_9Kc0B4epfjW-ZUM4jQpFAavb4SP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-05_06,2021-10-04_01,2020-04-07_01 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The size of receive buffers for receiving packets is calculated based on the mtu of interface. This patch adds devlink parameter to utilize user given size for receive buffers instead. By changing CQE descriptor size and receive buffer sizes the number of buffer pointers used by hardware for a large packet can be configured. Signed-off-by: Subbaraya Sundeep Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 + .../ethernet/marvell/octeontx2/nic/otx2_devlink.c | 60 ++++++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 7 +++ .../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 5 ++ 4 files changed, 73 insertions(+) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 6e0d1ac..f885abe 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -178,6 +178,7 @@ struct otx2_hw { u16 rqpool_cnt; u16 sqpool_cnt; u16 xqe_size; + u16 rbuf_fixed_size; /* NPA */ u32 stack_pg_ptrs; /* No of ptrs per stack page */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c index 98450e1..0fc7e32 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c @@ -64,6 +64,60 @@ static int otx2_dl_mcam_count_get(struct devlink *devlink, u32 id, return 0; } +static int otx2_dl_rbuf_size_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + /* Hardware supports max size of 32k for a receive buffer + * and 1536 is typical ethernet frame size. + */ + if (val.vu16 < 1536 || val.vu16 > 32768) { + NL_SET_ERR_MSG_MOD(extack, + "Receive buffer range is 1536 - 32768"); + return -EINVAL; + } + + return 0; +} + +static int otx2_dl_rbuf_size_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + struct net_device *netdev; + int err = 0; + bool if_up; + + rtnl_lock(); + + netdev = pfvf->netdev; + if_up = netif_running(netdev); + if (if_up) + netdev->netdev_ops->ndo_stop(netdev); + + pfvf->hw.rbuf_fixed_size = ALIGN(ctx->val.vu16, OTX2_ALIGN) + + OTX2_HEAD_ROOM; + + if (if_up) + err = netdev->netdev_ops->ndo_open(netdev); + + rtnl_unlock(); + + return err; +} + +static int otx2_dl_rbuf_size_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + + ctx->val.vu16 = pfvf->hw.rbuf_fixed_size; + + return 0; +} + static int otx2_dl_cqe_size_validate(struct devlink *devlink, u32 id, union devlink_param_value val, struct netlink_ext_ack *extack) @@ -118,6 +172,7 @@ enum otx2_dl_param_id { OTX2_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, OTX2_DEVLINK_PARAM_ID_MCAM_COUNT, OTX2_DEVLINK_PARAM_ID_CQE_SIZE, + OTX2_DEVLINK_PARAM_ID_RBUF_SIZE, }; static const struct devlink_param otx2_dl_params[] = { @@ -131,6 +186,11 @@ static const struct devlink_param otx2_dl_params[] = { BIT(DEVLINK_PARAM_CMODE_RUNTIME), otx2_dl_cqe_size_get, otx2_dl_cqe_size_set, otx2_dl_cqe_size_validate), + DEVLINK_PARAM_DRIVER(OTX2_DEVLINK_PARAM_ID_RBUF_SIZE, + "receive_buffer_size", DEVLINK_PARAM_TYPE_U16, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + otx2_dl_rbuf_size_get, otx2_dl_rbuf_size_set, + otx2_dl_rbuf_size_validate), }; /* Devlink OPs */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 8618cf7..1be524d3 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -65,6 +65,10 @@ static int otx2_change_mtu(struct net_device *netdev, int new_mtu) netdev_info(netdev, "Changing MTU from %d to %d\n", netdev->mtu, new_mtu); netdev->mtu = new_mtu; + /* Modify receive buffer size based on MTU and do not + * use the fixed size set. + */ + pf->hw.rbuf_fixed_size = 0; if (if_up) err = otx2_open(netdev); @@ -1306,6 +1310,9 @@ static int otx2_get_rbuf_size(struct otx2_nic *pf, int mtu) int total_size; int rbuf_size; + if (pf->hw.rbuf_fixed_size) + return pf->hw.rbuf_fixed_size; + /* The data transferred by NIX to memory consists of actual packet * plus additional data which has timestamp and/or EDSA/HIGIG2 * headers if interface is configured in corresponding modes. diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 672be05..229d051 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -439,6 +439,7 @@ static void otx2vf_do_set_rx_mode(struct work_struct *work) static int otx2vf_change_mtu(struct net_device *netdev, int new_mtu) { + struct otx2_nic *vf = netdev_priv(netdev); bool if_up = netif_running(netdev); int err = 0; @@ -448,6 +449,10 @@ static int otx2vf_change_mtu(struct net_device *netdev, int new_mtu) netdev_info(netdev, "Changing MTU from %d to %d\n", netdev->mtu, new_mtu); netdev->mtu = new_mtu; + /* Modify receive buffer size based on MTU and do not + * use the fixed size set. + */ + vf->hw.rbuf_fixed_size = 0; if (if_up) err = otx2vf_open(netdev);