From patchwork Tue Nov 15 12:07:36 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 9429565 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 77B5160484 for ; Tue, 15 Nov 2016 12:10:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6046D2874B for ; Tue, 15 Nov 2016 12:10:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 51055286F3; Tue, 15 Nov 2016 12:10:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DF5A1286F3 for ; Tue, 15 Nov 2016 12:10:19 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1c6cX8-0002Cw-H7; Tue, 15 Nov 2016 12:08:14 +0000 Received: from mail-pg0-x244.google.com ([2607:f8b0:400e:c05::244]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1c6cX0-000205-EO for linux-arm-kernel@lists.infradead.org; Tue, 15 Nov 2016 12:08:09 +0000 Received: by mail-pg0-x244.google.com with SMTP id e9so11376588pgc.1 for ; Tue, 15 Nov 2016 04:07:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=PwqD51Dwu0nmS4AUgUlCBuaCf6XvEV2JSeCPmqhICaU=; b=bZJdBCso+F7pMMsibxg4PdHt8FG2tods5QEH/wJoQc5OkSG0aywtOKQ6TcpDdeiw2a OsgaHLxYLjKXeO3SSbndvAnCwqUiSWy5ojZ1qcmJiKMGisuySQ73NeS8x59kYnNzfxIU V0y0ZoQ61W7DwiAa9t+bwhz6TByF6kcm4dbpWp/JizRHx1MkJN0M5zkcezq9z6yPiHPD wwOfi5E5SqtNn0yZuh1jqnT7G3vETI10vBV/fxQpuKJhEohRa0RRUX6CmusglpxxexAw x2k2NsNh28TA/jzj29KKhswZuHgIHBeoqCnVI3Z9oOia57q9T4pJdFsjJyB9Dj7e5a0w aJgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=PwqD51Dwu0nmS4AUgUlCBuaCf6XvEV2JSeCPmqhICaU=; b=T2j9USI4rxYSBgpMlIsV9Sx/aUV8GBXL6PecGkicx0B+CM/Y8iLl1qcaU0H4r9Hbps ul1Ms5p9LsfSAdoCS+Si+7qvmqG5WwC3+fkJ7/44SD/bTMgwtrC2FCVEb5fYgg9JrMBR XHeBeDCA499lS1iV8cTSeOpMk/dlahp7saHDeCDTU5D/HF11dul0+j0ZkwTcprr9K2FH fOMa84yV8ZWMr3f27CpEbpWCfsf4uAVi3YBGugidn8ZOaweGQHVr2md1zeMtj38mPrR6 2xbz7LO0b+3dipzAXLIzAA4K3EJwD76vgaFfG3FE4SiC8xwXgTfi3iv26CjgDfE7CIjm DZWw== X-Gm-Message-State: ABUngvdHxUvKMBsNyNggqneNYI6rsR8Z8V5xMoymzJj7RqqknzcFwxnNDZz1GnVHu2SK0Q== X-Received: by 10.98.71.218 with SMTP id p87mr46461710pfi.125.1479211665365; Tue, 15 Nov 2016 04:07:45 -0800 (PST) Received: from machine421.in.caveonetworks.com ([14.140.2.178]) by smtp.googlemail.com with ESMTPSA id s3sm38706100pfe.27.2016.11.15.04.07.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 15 Nov 2016 04:07:44 -0800 (PST) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Subject: [PATCH v2 2/5] net: thunderx: Program LMAC credits based on MTU Date: Tue, 15 Nov 2016 17:37:36 +0530 Message-Id: <1479211656-14316-1-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20161115_040806_706416_C39AA687 X-CRM114-Status: GOOD ( 20.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sunil Goutham , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sunil Goutham Programming LMAC credits taking 9K frame size by default is incorrect as for an interface which is one of the many on the same BGX/QLM no of credits available will be less as Tx FIFO will be divided across all interfaces. So let's say a BGX with 40G interface and another BGX with multiple 10G, bandwidth of 10G interfaces will be effected when traffic is running on both 40G and 10G interfaces simultaneously. This patch fixes this issue by programming credits based on netdev's MTU. Also fixed configuring MTU to HW and added CQE counter for pkts which exceed this value. Signed-off-by: Sunil Goutham --- drivers/net/ethernet/cavium/thunder/nic.h | 3 +- drivers/net/ethernet/cavium/thunder/nic_main.c | 36 +++++++++++++------- drivers/net/ethernet/cavium/thunder/nic_reg.h | 1 + drivers/net/ethernet/cavium/thunder/nicvf_main.c | 38 ++++++++++++---------- drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 3 ++ drivers/net/ethernet/cavium/thunder/nicvf_queues.h | 2 ++ 6 files changed, 53 insertions(+), 30 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nic.h b/drivers/net/ethernet/cavium/thunder/nic.h index 3042610..cd2d379 100644 --- a/drivers/net/ethernet/cavium/thunder/nic.h +++ b/drivers/net/ethernet/cavium/thunder/nic.h @@ -47,7 +47,7 @@ /* Min/Max packet size */ #define NIC_HW_MIN_FRS 64 -#define NIC_HW_MAX_FRS 9200 /* 9216 max packet including FCS */ +#define NIC_HW_MAX_FRS 9190 /* Excluding L2 header and FCS */ /* Max pkinds */ #define NIC_MAX_PKIND 16 @@ -282,7 +282,6 @@ struct nicvf { u8 node; u8 cpi_alg; - u16 mtu; bool link_up; u8 duplex; u32 speed; diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c b/drivers/net/ethernet/cavium/thunder/nic_main.c index 2bbf4cb..85c9e62 100644 --- a/drivers/net/ethernet/cavium/thunder/nic_main.c +++ b/drivers/net/ethernet/cavium/thunder/nic_main.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "nic_reg.h" #include "nic.h" @@ -260,18 +261,31 @@ static void nic_get_bgx_stats(struct nicpf *nic, struct bgx_stats_msg *bgx) /* Update hardware min/max frame size */ static int nic_update_hw_frs(struct nicpf *nic, int new_frs, int vf) { - if ((new_frs > NIC_HW_MAX_FRS) || (new_frs < NIC_HW_MIN_FRS)) { - dev_err(&nic->pdev->dev, - "Invalid MTU setting from VF%d rejected, should be between %d and %d\n", - vf, NIC_HW_MIN_FRS, NIC_HW_MAX_FRS); + int bgx, lmac, lmac_cnt; + u64 lmac_credits; + + if ((new_frs > NIC_HW_MAX_FRS) || (new_frs < NIC_HW_MIN_FRS)) return 1; - } - new_frs += ETH_HLEN; - if (new_frs <= nic->pkind.maxlen) - return 0; - nic->pkind.maxlen = new_frs; - nic_reg_write(nic, NIC_PF_PKIND_0_15_CFG, *(u64 *)&nic->pkind); + bgx = NIC_GET_BGX_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); + lmac = NIC_GET_LMAC_FROM_VF_LMAC_MAP(nic->vf_lmac_map[vf]); + lmac += bgx * MAX_LMAC_PER_BGX; + + new_frs += VLAN_ETH_HLEN + ETH_FCS_LEN + 4; + + /* Update corresponding LMAC credits */ + lmac_cnt = bgx_get_lmac_count(nic->node, bgx); + lmac_credits = nic_reg_read(nic, NIC_PF_LMAC_0_7_CREDIT + (lmac * 8)); + lmac_credits &= ~(0xFFFFFULL << 12); + lmac_credits |= (((((48 * 1024) / lmac_cnt) - new_frs) / 16) << 12); + nic_reg_write(nic, NIC_PF_LMAC_0_7_CREDIT + (lmac * 8), lmac_credits); + + /* Enforce MTU in HW + * This config is supported only from 88xx pass 2.0 onwards. + */ + if (!pass1_silicon(nic->pdev)) + nic_reg_write(nic, + NIC_PF_LMAC_0_7_CFG2 + (lmac * 8), new_frs); return 0; } @@ -464,7 +478,7 @@ static int nic_init_hw(struct nicpf *nic) /* PKIND configuration */ nic->pkind.minlen = 0; - nic->pkind.maxlen = NIC_HW_MAX_FRS + ETH_HLEN; + nic->pkind.maxlen = NIC_HW_MAX_FRS + VLAN_ETH_HLEN + ETH_FCS_LEN + 4; nic->pkind.lenerr_en = 1; nic->pkind.rx_hdr = 0; nic->pkind.hdr_sl = 0; diff --git a/drivers/net/ethernet/cavium/thunder/nic_reg.h b/drivers/net/ethernet/cavium/thunder/nic_reg.h index edf779f..80d4633 100644 --- a/drivers/net/ethernet/cavium/thunder/nic_reg.h +++ b/drivers/net/ethernet/cavium/thunder/nic_reg.h @@ -106,6 +106,7 @@ #define NIC_PF_MPI_0_2047_CFG (0x210000) #define NIC_PF_RSSI_0_4097_RQ (0x220000) #define NIC_PF_LMAC_0_7_CFG (0x240000) +#define NIC_PF_LMAC_0_7_CFG2 (0x240100) #define NIC_PF_LMAC_0_7_SW_XOFF (0x242000) #define NIC_PF_LMAC_0_7_CREDIT (0x244000) #define NIC_PF_CHAN_0_255_TX_CFG (0x400000) diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index 45a13f7..8f83361 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -1189,6 +1189,17 @@ int nicvf_stop(struct net_device *netdev) return 0; } +static int nicvf_update_hw_max_frs(struct nicvf *nic, int mtu) +{ + union nic_mbx mbx = {}; + + mbx.frs.msg = NIC_MBOX_MSG_SET_MAX_FRS; + mbx.frs.max_frs = mtu; + mbx.frs.vf_id = nic->vf_id; + + return nicvf_send_msg_to_pf(nic, &mbx); +} + int nicvf_open(struct net_device *netdev) { int err, qidx; @@ -1196,8 +1207,6 @@ int nicvf_open(struct net_device *netdev) struct queue_set *qs = nic->qs; struct nicvf_cq_poll *cq_poll = NULL; - nic->mtu = netdev->mtu; - netif_carrier_off(netdev); err = nicvf_register_misc_interrupt(nic); @@ -1248,9 +1257,12 @@ int nicvf_open(struct net_device *netdev) if (nic->sqs_mode) nicvf_get_primary_vf_struct(nic); - /* Configure receive side scaling */ - if (!nic->sqs_mode) + /* Configure receive side scaling and MTU */ + if (!nic->sqs_mode) { nicvf_rss_init(nic); + if (nicvf_update_hw_max_frs(nic, netdev->mtu)) + goto cleanup; + } err = nicvf_register_interrupts(nic); if (err) @@ -1297,17 +1309,6 @@ int nicvf_open(struct net_device *netdev) return err; } -static int nicvf_update_hw_max_frs(struct nicvf *nic, int mtu) -{ - union nic_mbx mbx = {}; - - mbx.frs.msg = NIC_MBOX_MSG_SET_MAX_FRS; - mbx.frs.max_frs = mtu; - mbx.frs.vf_id = nic->vf_id; - - return nicvf_send_msg_to_pf(nic, &mbx); -} - static int nicvf_change_mtu(struct net_device *netdev, int new_mtu) { struct nicvf *nic = netdev_priv(netdev); @@ -1318,10 +1319,13 @@ static int nicvf_change_mtu(struct net_device *netdev, int new_mtu) if (new_mtu < NIC_HW_MIN_FRS) return -EINVAL; + netdev->mtu = new_mtu; + + if (!netif_running(netdev)) + return 0; + if (nicvf_update_hw_max_frs(nic, new_mtu)) return -EINVAL; - netdev->mtu = new_mtu; - nic->mtu = new_mtu; return 0; } diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c index a4fc501..f0e0ca6 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c @@ -1530,6 +1530,9 @@ int nicvf_check_cqe_tx_errs(struct nicvf *nic, case CQ_TX_ERROP_SUBDC_ERR: stats->tx.subdesc_err++; break; + case CQ_TX_ERROP_MAX_SIZE_VIOL: + stats->tx.max_size_exceeded++; + break; case CQ_TX_ERROP_IMM_SIZE_OFLOW: stats->tx.imm_size_oflow++; break; diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.h b/drivers/net/ethernet/cavium/thunder/nicvf_queues.h index 869f338..8f4718e 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.h +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.h @@ -158,6 +158,7 @@ enum CQ_TX_ERROP_E { CQ_TX_ERROP_DESC_FAULT = 0x10, CQ_TX_ERROP_HDR_CONS_ERR = 0x11, CQ_TX_ERROP_SUBDC_ERR = 0x12, + CQ_TX_ERROP_MAX_SIZE_VIOL = 0x13, CQ_TX_ERROP_IMM_SIZE_OFLOW = 0x80, CQ_TX_ERROP_DATA_SEQUENCE_ERR = 0x81, CQ_TX_ERROP_MEM_SEQUENCE_ERR = 0x82, @@ -177,6 +178,7 @@ struct cmp_queue_stats { u64 desc_fault; u64 hdr_cons_err; u64 subdesc_err; + u64 max_size_exceeded; u64 imm_size_oflow; u64 data_seq_err; u64 mem_seq_err;