From patchwork Wed Dec 7 05:26:16 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 9463963 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 992C260459 for ; Wed, 7 Dec 2016 05:27:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CB3028478 for ; Wed, 7 Dec 2016 05:27:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 818AC284CF; Wed, 7 Dec 2016 05:27:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2AC4284E0 for ; Wed, 7 Dec 2016 05:27:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752298AbcLGF10 (ORCPT ); Wed, 7 Dec 2016 00:27:26 -0500 Received: from mail-pg0-f54.google.com ([74.125.83.54]:33013 "EHLO mail-pg0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752336AbcLGF0z (ORCPT ); Wed, 7 Dec 2016 00:26:55 -0500 Received: by mail-pg0-f54.google.com with SMTP id 3so157789361pgd.0 for ; Tue, 06 Dec 2016 21:26:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=i7GwHWEG2gy6vELRd5gT8gN/JLGrc51RjtoJOUqLSuM=; b=L4yxU6ixWn8gfopHoMWoUPLFkFKkl9vCnmroc1XuKiwGqXHOlCmAA1a942OSUcDkMU r50Kubdfm4DZyKlcMfZ/VT8OwTNKeMX4/029ziFcCo6NAiMGocu8JUu5gxsXCruBjfjU 78EIy58zuxJwmo9iDQ+K2OV6GQwhIPWq0VdbM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=i7GwHWEG2gy6vELRd5gT8gN/JLGrc51RjtoJOUqLSuM=; b=lU9s8ANIoOGrXxaKTAHdH2RySaLqoH7GWBc7e830X7BBjfe4aAkFP+tgdWi/j0SYjh hUdamxFU0SqK7IgLpLc/Jcd4lblsDol7FEj8lWqKELpi2CyFBsCZQ2hTspSHtc6TZ3SA qKsI/+7x8kJAy1W/kMVYOKbg2Vfj5doHxsFXY4W7JV+w7kiOIgrCKX8TBS6/QvnUCVbJ mZLuZ+s4a+VZeuBxPsHt3SP81DI+42OXPefSzM3iOw133KqXBtvei8YZigSz7M2zeYeH QzkHMWzAFsLKFGIMV3s6hW8xjYd4HJAO2OqwSP8sS5QmU+lc4A7ya99/9kwxnoxP0Ozg qHwQ== X-Gm-Message-State: AKaTC00lnGa+wPQIA6MZDAwvbM0mcMm9K+D/7pqB47QC3v7NgCdRdjFNYsOF+b0cuFNL9b88 X-Received: by 10.84.137.1 with SMTP id 1mr141171460plm.8.1481088414655; Tue, 06 Dec 2016 21:26:54 -0800 (PST) Received: from localhost.broadcom.com ([192.19.255.250]) by smtp.gmail.com with ESMTPSA id x90sm38605509pfk.73.2016.12.06.21.26.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Dec 2016 21:26:54 -0800 (PST) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, selvin.xavier@broadcom.com, somnath.kotur@broadcom.com, dledford@redhat.com, linux-rdma@vger.kernel.org Subject: [PATCH net-next v2 2/7] bnxt_en: Enable MSIX early in bnxt_init_one(). Date: Wed, 7 Dec 2016 00:26:16 -0500 Message-Id: <1481088381-30411-3-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1481088381-30411-1-git-send-email-michael.chan@broadcom.com> References: <1481088381-30411-1-git-send-email-michael.chan@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To better support the new RDMA driver, we need to move pci_enable_msix() from bnxt_open() to bnxt_init_one(). This way, MSIX vectors are available to the RDMA driver whether the network device is up or down. Part of the existing bnxt_setup_int_mode() function is now refactored into a new bnxt_init_int_mode(). bnxt_init_int_mode() is called during bnxt_init_one() to enable MSIX. The remaining logic in bnxt_setup_int_mode() to map the IRQs to the completion rings is called during bnxt_open(). v2: Fixed compile warning when CONFIG_BNXT_SRIOV is not set. Signed-off-by: Somnath Kotur Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 183 +++++++++++++++++++----------- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 115 insertions(+), 69 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 269e757..da302eb 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -4743,6 +4743,80 @@ static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max, return 0; } +static void bnxt_setup_msix(struct bnxt *bp) +{ + const int len = sizeof(bp->irq_tbl[0].name); + struct net_device *dev = bp->dev; + int tcs, i; + + tcs = netdev_get_num_tc(dev); + if (tcs > 1) { + bp->tx_nr_rings_per_tc = bp->tx_nr_rings / tcs; + if (bp->tx_nr_rings_per_tc == 0) { + netdev_reset_tc(dev); + bp->tx_nr_rings_per_tc = bp->tx_nr_rings; + } else { + int i, off, count; + + bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tcs; + for (i = 0; i < tcs; i++) { + count = bp->tx_nr_rings_per_tc; + off = i * count; + netdev_set_tc_queue(dev, i, count, off); + } + } + } + + for (i = 0; i < bp->cp_nr_rings; i++) { + char *attr; + + if (bp->flags & BNXT_FLAG_SHARED_RINGS) + attr = "TxRx"; + else if (i < bp->rx_nr_rings) + attr = "rx"; + else + attr = "tx"; + + snprintf(bp->irq_tbl[i].name, len, "%s-%s-%d", dev->name, attr, + i); + bp->irq_tbl[i].handler = bnxt_msix; + } +} + +static void bnxt_setup_inta(struct bnxt *bp) +{ + const int len = sizeof(bp->irq_tbl[0].name); + + if (netdev_get_num_tc(bp->dev)) + netdev_reset_tc(bp->dev); + + snprintf(bp->irq_tbl[0].name, len, "%s-%s-%d", bp->dev->name, "TxRx", + 0); + bp->irq_tbl[0].handler = bnxt_inta; +} + +static int bnxt_setup_int_mode(struct bnxt *bp) +{ + int rc; + + if (bp->flags & BNXT_FLAG_USING_MSIX) + bnxt_setup_msix(bp); + else + bnxt_setup_inta(bp); + + rc = bnxt_set_real_num_queues(bp); + return rc; +} + +static unsigned int bnxt_get_max_func_irqs(struct bnxt *bp) +{ +#if defined(CONFIG_BNXT_SRIOV) + if (BNXT_VF(bp)) + return bp->vf.max_irqs; +#endif + return bp->pf.max_irqs; +} + void bnxt_set_max_func_irqs(struct bnxt *bp, unsigned int max_irqs) { #if defined(CONFIG_BNXT_SRIOV) @@ -4753,16 +4827,12 @@ void bnxt_set_max_func_irqs(struct bnxt *bp, unsigned int max_irqs) bp->pf.max_irqs = max_irqs; } -static int bnxt_setup_msix(struct bnxt *bp) +static int bnxt_init_msix(struct bnxt *bp) { - struct msix_entry *msix_ent; - struct net_device *dev = bp->dev; int i, total_vecs, rc = 0, min = 1; - const int len = sizeof(bp->irq_tbl[0].name); - - bp->flags &= ~BNXT_FLAG_USING_MSIX; - total_vecs = bp->cp_nr_rings; + struct msix_entry *msix_ent; + total_vecs = bnxt_get_max_func_irqs(bp); msix_ent = kcalloc(total_vecs, sizeof(struct msix_entry), GFP_KERNEL); if (!msix_ent) return -ENOMEM; @@ -4783,8 +4853,10 @@ static int bnxt_setup_msix(struct bnxt *bp) bp->irq_tbl = kcalloc(total_vecs, sizeof(struct bnxt_irq), GFP_KERNEL); if (bp->irq_tbl) { - int tcs; + for (i = 0; i < total_vecs; i++) + bp->irq_tbl[i].vector = msix_ent[i].vector; + bp->total_irqs = total_vecs; /* Trim rings based upon num of vectors allocated */ rc = bnxt_trim_rings(bp, &bp->rx_nr_rings, &bp->tx_nr_rings, total_vecs, min == 1); @@ -4792,43 +4864,10 @@ static int bnxt_setup_msix(struct bnxt *bp) goto msix_setup_exit; bp->tx_nr_rings_per_tc = bp->tx_nr_rings; - tcs = netdev_get_num_tc(dev); - if (tcs > 1) { - bp->tx_nr_rings_per_tc = bp->tx_nr_rings / tcs; - if (bp->tx_nr_rings_per_tc == 0) { - netdev_reset_tc(dev); - bp->tx_nr_rings_per_tc = bp->tx_nr_rings; - } else { - int i, off, count; + bp->cp_nr_rings = (min == 1) ? + max_t(int, bp->tx_nr_rings, bp->rx_nr_rings) : + bp->tx_nr_rings + bp->rx_nr_rings; - bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tcs; - for (i = 0; i < tcs; i++) { - count = bp->tx_nr_rings_per_tc; - off = i * count; - netdev_set_tc_queue(dev, i, count, off); - } - } - } - bp->cp_nr_rings = total_vecs; - - for (i = 0; i < bp->cp_nr_rings; i++) { - char *attr; - - bp->irq_tbl[i].vector = msix_ent[i].vector; - if (bp->flags & BNXT_FLAG_SHARED_RINGS) - attr = "TxRx"; - else if (i < bp->rx_nr_rings) - attr = "rx"; - else - attr = "tx"; - - snprintf(bp->irq_tbl[i].name, len, - "%s-%s-%d", dev->name, attr, i); - bp->irq_tbl[i].handler = bnxt_msix; - } - rc = bnxt_set_real_num_queues(bp); - if (rc) - goto msix_setup_exit; } else { rc = -ENOMEM; goto msix_setup_exit; @@ -4838,52 +4877,54 @@ static int bnxt_setup_msix(struct bnxt *bp) return 0; msix_setup_exit: - netdev_err(bp->dev, "bnxt_setup_msix err: %x\n", rc); + netdev_err(bp->dev, "bnxt_init_msix err: %x\n", rc); + kfree(bp->irq_tbl); + bp->irq_tbl = NULL; pci_disable_msix(bp->pdev); kfree(msix_ent); return rc; } -static int bnxt_setup_inta(struct bnxt *bp) +static int bnxt_init_inta(struct bnxt *bp) { - int rc; - const int len = sizeof(bp->irq_tbl[0].name); - - if (netdev_get_num_tc(bp->dev)) - netdev_reset_tc(bp->dev); - bp->irq_tbl = kcalloc(1, sizeof(struct bnxt_irq), GFP_KERNEL); - if (!bp->irq_tbl) { - rc = -ENOMEM; - return rc; - } + if (!bp->irq_tbl) + return -ENOMEM; + + bp->total_irqs = 1; bp->rx_nr_rings = 1; bp->tx_nr_rings = 1; bp->cp_nr_rings = 1; bp->tx_nr_rings_per_tc = bp->tx_nr_rings; bp->flags |= BNXT_FLAG_SHARED_RINGS; bp->irq_tbl[0].vector = bp->pdev->irq; - snprintf(bp->irq_tbl[0].name, len, - "%s-%s-%d", bp->dev->name, "TxRx", 0); - bp->irq_tbl[0].handler = bnxt_inta; - rc = bnxt_set_real_num_queues(bp); - return rc; + return 0; } -static int bnxt_setup_int_mode(struct bnxt *bp) +static int bnxt_init_int_mode(struct bnxt *bp) { int rc = 0; if (bp->flags & BNXT_FLAG_MSIX_CAP) - rc = bnxt_setup_msix(bp); + rc = bnxt_init_msix(bp); if (!(bp->flags & BNXT_FLAG_USING_MSIX) && BNXT_PF(bp)) { /* fallback to INTA */ - rc = bnxt_setup_inta(bp); + rc = bnxt_init_inta(bp); } return rc; } +static void bnxt_clear_int_mode(struct bnxt *bp) +{ + if (bp->flags & BNXT_FLAG_USING_MSIX) + pci_disable_msix(bp->pdev); + + kfree(bp->irq_tbl); + bp->irq_tbl = NULL; + bp->flags &= ~BNXT_FLAG_USING_MSIX; +} + static void bnxt_free_irq(struct bnxt *bp) { struct bnxt_irq *irq; @@ -4902,10 +4943,6 @@ static void bnxt_free_irq(struct bnxt *bp) free_irq(irq->vector, bp->bnapi[i]); irq->requested = 0; } - if (bp->flags & BNXT_FLAG_USING_MSIX) - pci_disable_msix(bp->pdev); - kfree(bp->irq_tbl); - bp->irq_tbl = NULL; } static int bnxt_request_irq(struct bnxt *bp) @@ -6695,6 +6732,7 @@ static void bnxt_remove_one(struct pci_dev *pdev) cancel_work_sync(&bp->sp_task); bp->sp_event = 0; + bnxt_clear_int_mode(bp); bnxt_hwrm_func_drv_unrgtr(bp); bnxt_free_hwrm_resources(bp); bnxt_dcb_free(bp); @@ -6990,10 +7028,14 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) if (rc) goto init_err; - rc = register_netdev(dev); + rc = bnxt_init_int_mode(bp); if (rc) goto init_err; + rc = register_netdev(dev); + if (rc) + goto init_err_clr_int; + netdev_info(dev, "%s found at mem %lx, node addr %pM\n", board_info[ent->driver_data].name, (long)pci_resource_start(pdev, 0), dev->dev_addr); @@ -7002,6 +7044,9 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) return 0; +init_err_clr_int: + bnxt_clear_int_mode(bp); + init_err: pci_iounmap(pdev, bp->bar0); pci_release_regions(pdev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 8327d0d..1461355 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1024,6 +1024,7 @@ struct bnxt { #define BNXT_STATE_FN_RST_DONE 2 struct bnxt_irq *irq_tbl; + int total_irqs; u8 mac_addr[ETH_ALEN]; #ifdef CONFIG_BNXT_DCB