From patchwork Wed Dec 21 11:41:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 9482935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 38BDD60772 for ; Wed, 21 Dec 2016 11:43:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18AA6283FC for ; Wed, 21 Dec 2016 11:43:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0D96128402; Wed, 21 Dec 2016 11:43:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CF81283FC for ; Wed, 21 Dec 2016 11:43:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757116AbcLULnA (ORCPT ); Wed, 21 Dec 2016 06:43:00 -0500 Received: from mail-wm0-f41.google.com ([74.125.82.41]:37826 "EHLO mail-wm0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757257AbcLULm5 (ORCPT ); Wed, 21 Dec 2016 06:42:57 -0500 Received: by mail-wm0-f41.google.com with SMTP id t79so156259883wmt.0 for ; Wed, 21 Dec 2016 03:42:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yfVrBPRKLoh3i4OxAxpwZuMnfbjfi+FMLQWTl7IRSa8=; b=A5hglWahuSVuY8FUPFYM3ubSyfxO3WRpcasg0vc3gnttal6nDWHlbgaCcoLVfo5Eua CgXBNkzNZy0M77lktXDtyS+U3oV926fcQIM3SrsWTLEswj6dK+EqZujVfeJeEru6uPT2 rNmr3ww7zutsyKlMnjNLwPMgG4xVllFMj3xLQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yfVrBPRKLoh3i4OxAxpwZuMnfbjfi+FMLQWTl7IRSa8=; b=h8uG4tBaWV3mG1omRlhb/yQm1T5KLanjyBGoKV5q2xH8FyxBO0iDu1Nl6umgfps7Uq rDauS4BKS0zRqhbvzPucirRvi/KcEsnJXunTwAIlkEcZrDdxqaVVgLSFeZqTYWnxvfkk H06kO3xGGXXvzuZPaKDjw3O8e0kI14bAbd9Igct2n5pcK1MhuNAp3aMCfS8Ypo1xrFK3 y5dPjZkkWc36RiJpP8CpI37Mqfizht7na9aLBHdiJIJu8IFHIhWKv6im087lX7AOpRqK wp++Ei93fOgLMlrRxizQVoyEEBLpMMElkVEwcGUAmYnEQr3H+/qIwr507ZNnf/Wakk8V w5jg== X-Gm-Message-State: AIkVDXKgAiY35U8LwI90/aXmKtlfNTwEk11SEGGu4X5FUxMh7kYJPffh7xFBrc5sgbQVwrMR X-Received: by 10.28.129.81 with SMTP id c78mr4608201wmd.94.1482320575296; Wed, 21 Dec 2016 03:42:55 -0800 (PST) Received: from dhcp-10-192-206-197.iig.avagotech.net ([192.19.239.250]) by smtp.gmail.com with ESMTPSA id t194sm2997257wmd.1.2016.12.21.03.42.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 21 Dec 2016 03:42:54 -0800 (PST) From: Selvin Xavier To: dledford@redhat.com, linux-rdma@vger.kernel.org Cc: netdev@vger.kernel.org, michael.chan@broadcom.com, Selvin Xavier , Eddie Wai , Devesh Sharma , Somnath Kotur , Sriharsha Basavapatna Subject: [PATCH for bnxt_re V4 05/21] RDMA/bnxt_re: Adding Notification Queue support Date: Wed, 21 Dec 2016 03:41:54 -0800 Message-Id: <1482320530-5344-6-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1482320530-5344-1-git-send-email-selvin.xavier@broadcom.com> References: <1482320530-5344-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Completion Notifcations are handled by Notification Queue (NQ). This patch configures the NQs. Also, configures the Door bell page mapping v3: Fixes some sparse warnings related to endianness checks v4: Change include file names Signed-off-by: Eddie Wai Signed-off-by: Devesh Sharma Signed-off-by: Somnath Kotur Signed-off-by: Sriharsha Basavapatna Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 8 ++ drivers/infiniband/hw/bnxt_re/main.c | 52 +++++++++- drivers/infiniband/hw/bnxt_re/qplib_fp.c | 161 ++++++++++++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/qplib_fp.h | 60 +++++++++++ drivers/infiniband/hw/bnxt_re/qplib_res.h | 6 ++ 5 files changed, 286 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 78e95a5..8d2f42f 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -60,6 +60,8 @@ struct bnxt_re_work { #define BNXT_RE_MIN_MSIX 2 #define BNXT_RE_MAX_MSIX 16 #define BNXT_RE_AEQ_IDX 0 +#define BNXT_RE_NQ_IDX 1 + struct bnxt_re_dev { struct ib_device ibdev; struct list_head list; @@ -78,9 +80,15 @@ struct bnxt_re_dev { int id; + /* FP Notification Queue (CQ & SRQ) */ + struct tasklet_struct nq_task; + /* RCFW Channel */ struct bnxt_qplib_rcfw rcfw; + /* NQ */ + struct bnxt_qplib_nq nq; + /* Device Resources */ struct bnxt_qplib_dev_attr dev_attr; struct bnxt_qplib_ctx qplib_ctx; diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index a8d0f9b..bef654e 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -568,6 +568,9 @@ static int bnxt_re_aeq_handler(struct bnxt_qplib_rcfw *rcfw, static void bnxt_re_cleanup_res(struct bnxt_re_dev *rdev) { + if (rdev->nq.hwq.max_elements) + bnxt_qplib_disable_nq(&rdev->nq); + if (rdev->qplib_res.rcfw) bnxt_qplib_cleanup_res(&rdev->qplib_res); } @@ -578,11 +581,32 @@ static int bnxt_re_init_res(struct bnxt_re_dev *rdev) bnxt_qplib_init_res(&rdev->qplib_res); + if (rdev->msix_entries[BNXT_RE_NQ_IDX].vector <= 0) + return -EINVAL; + + rc = bnxt_qplib_enable_nq(rdev->en_dev->pdev, &rdev->nq, + rdev->msix_entries[BNXT_RE_NQ_IDX].vector, + rdev->msix_entries[BNXT_RE_NQ_IDX].db_offset, + NULL, + NULL); + + if (rc) + dev_err(rdev_to_dev(rdev), "Failed to enable NQ: %#x", rc); + return rc; } static void bnxt_re_free_res(struct bnxt_re_dev *rdev, bool lock_wait) { + if (rdev->nq.hwq.max_elements) { + bnxt_re_net_ring_free(rdev, rdev->nq.ring_id, lock_wait); + bnxt_qplib_free_nq(&rdev->nq); + } + if (rdev->qplib_res.dpi_tbl.max) { + bnxt_qplib_dealloc_dpi(&rdev->qplib_res, + &rdev->qplib_res.dpi_tbl, + &rdev->dpi_privileged); + } if (rdev->qplib_res.rcfw) { bnxt_qplib_free_res(&rdev->qplib_res); rdev->qplib_res.rcfw = NULL; @@ -604,8 +628,34 @@ static int bnxt_re_alloc_res(struct bnxt_re_dev *rdev) if (rc) goto fail; - return 0; + rc = bnxt_qplib_alloc_dpi(&rdev->qplib_res.dpi_tbl, + &rdev->dpi_privileged, + rdev); + if (rc) + goto fail; + rdev->nq.hwq.max_elements = BNXT_RE_MAX_CQ_COUNT + + BNXT_RE_MAX_SRQC_COUNT + 2; + rc = bnxt_qplib_alloc_nq(rdev->en_dev->pdev, &rdev->nq); + if (rc) { + dev_err(rdev_to_dev(rdev), + "Failed to allocate NQ memory: %#x", rc); + goto fail; + } + rc = bnxt_re_net_ring_alloc + (rdev, rdev->nq.hwq.pbl[PBL_LVL_0].pg_map_arr, + rdev->nq.hwq.pbl[rdev->nq.hwq.level].pg_count, + HWRM_RING_ALLOC_CMPL, BNXT_QPLIB_NQE_MAX_CNT - 1, + rdev->msix_entries[BNXT_RE_NQ_IDX].ring_idx, + &rdev->nq.ring_id); + if (rc) { + dev_err(rdev_to_dev(rdev), + "Failed to allocate NQ ring: %#x", rc); + goto free_nq; + } + return 0; +free_nq: + bnxt_qplib_free_nq(&rdev->nq); fail: rdev->qplib_res.rcfw = NULL; return rc; diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c index 36c4b81..a33960b 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c @@ -35,3 +35,164 @@ * * Description: Fast Path Operators */ + +#include +#include +#include +#include +#include +#include + +#include "roce_hsi.h" + +#include "qplib_res.h" +#include "qplib_rcfw.h" +#include "qplib_sp.h" +#include "qplib_fp.h" + +static void bnxt_qplib_service_nq(unsigned long data) +{ + struct bnxt_qplib_nq *nq = (struct bnxt_qplib_nq *)data; + struct bnxt_qplib_hwq *hwq = &nq->hwq; + struct nq_base *nqe, **nq_ptr; + u32 sw_cons, raw_cons; + u16 type; + int budget = nq->budget; + + /* Service the NQ until empty */ + raw_cons = hwq->cons; + while (budget--) { + sw_cons = HWQ_CMP(raw_cons, hwq); + nq_ptr = (struct nq_base **)hwq->pbl_ptr; + nqe = &nq_ptr[NQE_PG(sw_cons)][NQE_IDX(sw_cons)]; + if (!NQE_CMP_VALID(nqe, raw_cons, hwq->max_elements)) + break; + + type = le16_to_cpu(nqe->info10_type) & NQ_BASE_TYPE_MASK; + switch (type) { + case NQ_BASE_TYPE_CQ_NOTIFICATION: + break; + case NQ_BASE_TYPE_DBQ_EVENT: + break; + default: + dev_warn(&nq->pdev->dev, + "QPLIB: nqe with type = 0x%x not handled", + type); + break; + } + raw_cons++; + } + if (hwq->cons != raw_cons) { + hwq->cons = raw_cons; + NQ_DB_REARM(nq->bar_reg_iomem, hwq->cons, hwq->max_elements); + } +} + +static irqreturn_t bnxt_qplib_nq_irq(int irq, void *dev_instance) +{ + struct bnxt_qplib_nq *nq = dev_instance; + struct bnxt_qplib_hwq *hwq = &nq->hwq; + struct nq_base **nq_ptr; + u32 sw_cons; + + /* Prefetch the NQ element */ + sw_cons = HWQ_CMP(hwq->cons, hwq); + nq_ptr = (struct nq_base **)nq->hwq.pbl_ptr; + prefetch(&nq_ptr[NQE_PG(sw_cons)][NQE_IDX(sw_cons)]); + + /* Fan out to CPU affinitized kthreads? */ + tasklet_schedule(&nq->worker); + + return IRQ_HANDLED; +} + +void bnxt_qplib_disable_nq(struct bnxt_qplib_nq *nq) +{ + /* Make sure the HW is stopped! */ + synchronize_irq(nq->vector); + tasklet_disable(&nq->worker); + tasklet_kill(&nq->worker); + + if (nq->requested) { + free_irq(nq->vector, nq); + nq->requested = false; + } + if (nq->bar_reg_iomem) + iounmap(nq->bar_reg_iomem); + nq->bar_reg_iomem = NULL; + + nq->cqn_handler = NULL; + nq->srqn_handler = NULL; + nq->vector = 0; +} + +int bnxt_qplib_enable_nq(struct pci_dev *pdev, struct bnxt_qplib_nq *nq, + int msix_vector, int bar_reg_offset, + int (*cqn_handler)(struct bnxt_qplib_nq *nq, + void *), + int (*srqn_handler)(struct bnxt_qplib_nq *nq, + void *, u8 event)) +{ + resource_size_t nq_base; + int rc; + + nq->pdev = pdev; + nq->vector = msix_vector; + + nq->cqn_handler = cqn_handler; + + nq->srqn_handler = srqn_handler; + + tasklet_init(&nq->worker, bnxt_qplib_service_nq, (unsigned long)nq); + + nq->requested = false; + rc = request_irq(nq->vector, bnxt_qplib_nq_irq, 0, "bnxt_qplib_nq", nq); + if (rc) { + dev_err(&nq->pdev->dev, + "Failed to request IRQ for NQ: %#x", rc); + bnxt_qplib_disable_nq(nq); + goto fail; + } + nq->requested = true; + nq->bar_reg = NQ_CONS_PCI_BAR_REGION; + nq->bar_reg_off = bar_reg_offset; + nq_base = pci_resource_start(pdev, nq->bar_reg); + if (!nq_base) { + rc = -ENOMEM; + goto fail; + } + nq->bar_reg_iomem = ioremap_nocache(nq_base + nq->bar_reg_off, 4); + if (!nq->bar_reg_iomem) { + rc = -ENOMEM; + goto fail; + } + NQ_DB_REARM(nq->bar_reg_iomem, nq->hwq.cons, nq->hwq.max_elements); + + return 0; +fail: + bnxt_qplib_disable_nq(nq); + return rc; +} + +void bnxt_qplib_free_nq(struct bnxt_qplib_nq *nq) +{ + if (nq->hwq.max_elements) + bnxt_qplib_free_hwq(nq->pdev, &nq->hwq); +} + +int bnxt_qplib_alloc_nq(struct pci_dev *pdev, struct bnxt_qplib_nq *nq) +{ + nq->pdev = pdev; + if (!nq->hwq.max_elements || + nq->hwq.max_elements > BNXT_QPLIB_NQE_MAX_CNT) + nq->hwq.max_elements = BNXT_QPLIB_NQE_MAX_CNT; + + if (bnxt_qplib_alloc_init_hwq(nq->pdev, &nq->hwq, NULL, 0, + &nq->hwq.max_elements, + BNXT_QPLIB_MAX_NQE_ENTRY_SIZE, 0, + PAGE_SIZE, HWQ_TYPE_L2_CMPL)) + return -ENOMEM; + + nq->budget = 8; + return 0; +} diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.h b/drivers/infiniband/hw/bnxt_re/qplib_fp.h index 0983465..25a4494 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.h @@ -39,4 +39,64 @@ #ifndef __BNXT_QPLIB_FP_H__ #define __BNXT_QPLIB_FP_H__ +#define BNXT_QPLIB_MAX_NQE_ENTRY_SIZE sizeof(struct nq_base) + +#define NQE_CNT_PER_PG (PAGE_SIZE / BNXT_QPLIB_MAX_NQE_ENTRY_SIZE) +#define NQE_MAX_IDX_PER_PG (NQE_CNT_PER_PG - 1) +#define NQE_PG(x) (((x) & ~NQE_MAX_IDX_PER_PG) / NQE_CNT_PER_PG) +#define NQE_IDX(x) ((x) & NQE_MAX_IDX_PER_PG) + +#define NQE_CMP_VALID(hdr, raw_cons, cp_bit) \ + (!!(le32_to_cpu((hdr)->info63_v[0]) & NQ_BASE_V) == \ + !((raw_cons) & (cp_bit))) + +#define BNXT_QPLIB_NQE_MAX_CNT (128 * 1024) + +#define NQ_CONS_PCI_BAR_REGION 2 +#define NQ_DB_KEY_CP (0x2 << CMPL_DOORBELL_KEY_SFT) +#define NQ_DB_IDX_VALID CMPL_DOORBELL_IDX_VALID +#define NQ_DB_IRQ_DIS CMPL_DOORBELL_MASK +#define NQ_DB_CP_FLAGS_REARM (NQ_DB_KEY_CP | \ + NQ_DB_IDX_VALID) +#define NQ_DB_CP_FLAGS (NQ_DB_KEY_CP | \ + NQ_DB_IDX_VALID | \ + NQ_DB_IRQ_DIS) +#define NQ_DB_REARM(db, raw_cons, cp_bit) \ + writel(NQ_DB_CP_FLAGS_REARM | ((raw_cons) & ((cp_bit) - 1)), db) +#define NQ_DB(db, raw_cons, cp_bit) \ + writel(NQ_DB_CP_FLAGS | ((raw_cons) & ((cp_bit) - 1)), db) + +struct bnxt_qplib_nq { + struct pci_dev *pdev; + + int vector; + int budget; + bool requested; + struct tasklet_struct worker; + struct bnxt_qplib_hwq hwq; + + u16 bar_reg; + u16 bar_reg_off; + u16 ring_id; + void __iomem *bar_reg_iomem; + + int (*cqn_handler) + (struct bnxt_qplib_nq *nq, + void *cq); + int (*srqn_handler) + (struct bnxt_qplib_nq *nq, + void *srq, + u8 event); +}; + +void bnxt_qplib_disable_nq(struct bnxt_qplib_nq *nq); +int bnxt_qplib_enable_nq(struct pci_dev *pdev, struct bnxt_qplib_nq *nq, + int msix_vector, int bar_reg_offset, + int (*cqn_handler)(struct bnxt_qplib_nq *nq, + void *cq), + int (*srqn_handler)(struct bnxt_qplib_nq *nq, + void *srq, + u8 event)); +void bnxt_qplib_free_nq(struct bnxt_qplib_nq *nq); +int bnxt_qplib_alloc_nq(struct pci_dev *pdev, struct bnxt_qplib_nq *nq); #endif /* __BNXT_QPLIB_FP_H__ */ diff --git a/drivers/infiniband/hw/bnxt_re/qplib_res.h b/drivers/infiniband/hw/bnxt_re/qplib_res.h index ce122cf..571feda 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_res.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_res.h @@ -193,6 +193,12 @@ int bnxt_qplib_alloc_init_hwq(struct pci_dev *pdev, struct bnxt_qplib_hwq *hwq, struct scatterlist *sl, int nmap, u32 *elements, u32 elements_per_page, u32 aux, u32 pg_size, enum bnxt_qplib_hwq_type hwq_type); +int bnxt_qplib_alloc_dpi(struct bnxt_qplib_dpi_tbl *dpit, + struct bnxt_qplib_dpi *dpi, + void *app); +int bnxt_qplib_dealloc_dpi(struct bnxt_qplib_res *res, + struct bnxt_qplib_dpi_tbl *dpi_tbl, + struct bnxt_qplib_dpi *dpi); void bnxt_qplib_cleanup_res(struct bnxt_qplib_res *res); int bnxt_qplib_init_res(struct bnxt_qplib_res *res); void bnxt_qplib_free_res(struct bnxt_qplib_res *res);