From patchwork Fri Feb 10 11:19:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 9566591 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5D0E760216 for ; Fri, 10 Feb 2017 11:29:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 496AA2853D for ; Fri, 10 Feb 2017 11:29:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3E10728552; Fri, 10 Feb 2017 11:29:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83AF928559 for ; Fri, 10 Feb 2017 11:29:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753066AbdBJL2x (ORCPT ); Fri, 10 Feb 2017 06:28:53 -0500 Received: from mail-qt0-f173.google.com ([209.85.216.173]:35505 "EHLO mail-qt0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753040AbdBJL2v (ORCPT ); Fri, 10 Feb 2017 06:28:51 -0500 Received: by mail-qt0-f173.google.com with SMTP id x49so31418116qtc.2 for ; Fri, 10 Feb 2017 03:28:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Aas9m1gsAnV1J/ZDNVdjYSEs7yvnyeOR+vWW7iDmVlA=; b=RmEiX/wKiL03zseKSG4swI9XuMj1QtA4aFvoRBxp/Ng23cqZVQLR+ZsOg/CkJA3Meh mP+RL+T4x7HIhEWX8MTol715zKeAvJ+xihinuDVgclQAFdhmz918zIzNQ4mKPnd9fH/q xmg7ASQ3NiLjp1y5lvsoPVOuIGl6F2aZA3/3w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Aas9m1gsAnV1J/ZDNVdjYSEs7yvnyeOR+vWW7iDmVlA=; b=RY+hDY3vUvzL2mm2Pe61iGMUdgIioPF6o5QoLwLZyJwQIdETpW0zqtnxu0Wth9LC+c TqrAWTkdHpON0NxBua6XKMHqcMP/JDXCE8hiLDArvWnqzuEYa9bgFDlieusQkci61ctY +cPFnUS/bIuVJdxC6RPoBSRgAGTz0BFx2tzpVKf4yGdK7jsAkOkxmguof3oNTLxeErgq E9X2oGreLv2jzQ8m+110I5dxmn7UlNW/Si0TBT31QxkROnmK0iO5xTiDIMCfgyP1h3Ey EbM33sdqaLEyMhyhe0/vjRAI3Q/c5EVX47jXOXkV64KEH527TUjpdxEq61S89fMFuMbF Izuw== X-Gm-Message-State: AMke39lnj9YCgdSJrUAxcfeoGNFzNfEtJq+3yT7SrR+5WhFsnZGyWNkj2C052hBhQvvYRful X-Received: by 10.200.35.246 with SMTP id r51mr7231572qtr.183.1486725680453; Fri, 10 Feb 2017 03:21:20 -0800 (PST) Received: from dhcp-10-192-206-197.iig.avagotech.net ([192.19.239.250]) by smtp.gmail.com with ESMTPSA id z23sm1211412qkb.30.2017.02.10.03.21.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Feb 2017 03:21:19 -0800 (PST) From: Selvin Xavier To: dledford@redhat.com, linux-rdma@vger.kernel.org Cc: netdev@vger.kernel.org, Selvin Xavier , Eddie Wai , Devesh Sharma , Somnath Kotur , Sriharsha Basavapatna Subject: [PATCH V5 for-next 18/21] RDMA/bnxt_re: Support for DCB Date: Fri, 10 Feb 2017 03:19:50 -0800 Message-Id: <1486725593-9872-19-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1486725593-9872-1-git-send-email-selvin.xavier@broadcom.com> References: <1486725593-9872-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch queries the configured RoCE APP Priority on the host using the dcbnl API and programs the RoCE FW with the corresponding Traffic Class(es) for the priority. v2: Fixed some sparse warning and cleanup of function bnxt_re_query_hwrm_pri2cos v3: Adds bnxt_qplib_map_tc2cos as a part of this patch. Uses ROCE_V2_UDP_DPORT instead of BNXT_RE_ROCE_V2_PORT_NO. v5: Uses ETH_P_IBOE macro for RoCE ethertype Signed-off-by: Eddie Wai Signed-off-by: Devesh Sharma Signed-off-by: Somnath Kotur Signed-off-by: Sriharsha Basavapatna Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 3 + drivers/infiniband/hw/bnxt_re/main.c | 141 +++++++++++++++++++++++++++++++ drivers/infiniband/hw/bnxt_re/qplib_sp.c | 37 ++++++++ drivers/infiniband/hw/bnxt_re/qplib_sp.h | 1 + 4 files changed, 182 insertions(+) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 5032ca1..ebf7be8 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -93,6 +93,9 @@ struct bnxt_re_dev { int id; + struct delayed_work worker; + u8 cur_prio_map; + /* FP Notification Queue (CQ & SRQ) */ struct tasklet_struct nq_task; diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index c84691a..92a217b 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -44,8 +44,10 @@ #include #include #include +#include #include #include +#include #include #include @@ -726,6 +728,50 @@ static void bnxt_re_dispatch_event(struct ib_device *ibdev, struct ib_qp *qp, ib_dispatch_event(&ib_event); } +#define HWRM_QUEUE_PRI2COS_QCFG_INPUT_FLAGS_IVLAN 0x02 +static int bnxt_re_query_hwrm_pri2cos(struct bnxt_re_dev *rdev, u8 dir, + u64 *cid_map) +{ + struct hwrm_queue_pri2cos_qcfg_input req = {0}; + struct bnxt *bp = netdev_priv(rdev->netdev); + struct hwrm_queue_pri2cos_qcfg_output resp; + struct bnxt_en_dev *en_dev = rdev->en_dev; + struct bnxt_fw_msg fw_msg; + u32 flags = 0; + u8 *qcfgmap, *tmp_map; + int rc = 0, i; + + if (!cid_map) + return -EINVAL; + + memset(&fw_msg, 0, sizeof(fw_msg)); + bnxt_re_init_hwrm_hdr(rdev, (void *)&req, + HWRM_QUEUE_PRI2COS_QCFG, -1, -1); + flags |= (dir & 0x01); + flags |= HWRM_QUEUE_PRI2COS_QCFG_INPUT_FLAGS_IVLAN; + req.flags = cpu_to_le32(flags); + req.port_id = bp->pf.port_id; + + bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, + sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + if (rc) + return rc; + + if (resp.queue_cfg_info) { + dev_warn(rdev_to_dev(rdev), + "Asymmetric cos queue configuration detected"); + dev_warn(rdev_to_dev(rdev), + " on device, QoS may not be fully functional\n"); + } + qcfgmap = &resp.pri0_cos_queue_id; + tmp_map = (u8 *)cid_map; + for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) + tmp_map[i] = qcfgmap[i]; + + return rc; +} + static bool bnxt_re_is_qp1_or_shadow_qp(struct bnxt_re_dev *rdev, struct bnxt_re_qp *qp) { @@ -757,6 +803,80 @@ static void bnxt_re_dev_stop(struct bnxt_re_dev *rdev) mutex_unlock(&rdev->qp_lock); } +static u32 bnxt_re_get_priority_mask(struct bnxt_re_dev *rdev) +{ + u32 prio_map = 0, tmp_map = 0; + struct net_device *netdev; + struct dcb_app app; + + netdev = rdev->netdev; + + memset(&app, 0, sizeof(app)); + app.selector = IEEE_8021QAZ_APP_SEL_ETHERTYPE; + app.protocol = ETH_P_IBOE; + tmp_map = dcb_ieee_getapp_mask(netdev, &app); + prio_map = tmp_map; + + app.selector = IEEE_8021QAZ_APP_SEL_DGRAM; + app.protocol = ROCE_V2_UDP_DPORT; + tmp_map = dcb_ieee_getapp_mask(netdev, &app); + prio_map |= tmp_map; + + if (!prio_map) + prio_map = -EFAULT; + return prio_map; +} + +static void bnxt_re_parse_cid_map(u8 prio_map, u8 *cid_map, u16 *cosq) +{ + u16 prio; + u8 id; + + for (prio = 0, id = 0; prio < 8; prio++) { + if (prio_map & (1 << prio)) { + cosq[id] = cid_map[prio]; + id++; + if (id == 2) /* Max 2 tcs supported */ + break; + } + } +} + +static int bnxt_re_setup_qos(struct bnxt_re_dev *rdev) +{ + u8 prio_map = 0; + u64 cid_map; + int rc; + + /* Get priority for roce */ + rc = bnxt_re_get_priority_mask(rdev); + if (rc < 0) + return rc; + prio_map = (u8)rc; + + if (prio_map == rdev->cur_prio_map) + return 0; + rdev->cur_prio_map = prio_map; + /* Get cosq id for this priority */ + rc = bnxt_re_query_hwrm_pri2cos(rdev, 0, &cid_map); + if (rc) { + dev_warn(rdev_to_dev(rdev), "no cos for p_mask %x\n", prio_map); + return rc; + } + /* Parse CoS IDs for app priority */ + bnxt_re_parse_cid_map(prio_map, (u8 *)&cid_map, rdev->cosq); + + /* Config BONO. */ + rc = bnxt_qplib_map_tc2cos(&rdev->qplib_res, rdev->cosq); + if (rc) { + dev_warn(rdev_to_dev(rdev), "no tc for cos{%x, %x}\n", + rdev->cosq[0], rdev->cosq[1]); + return rc; + } + + return 0; +} + static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) { int i, rc; @@ -768,6 +888,9 @@ static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) /* Cleanup ib dev */ bnxt_re_unregister_ib(rdev); } + if (test_and_clear_bit(BNXT_RE_FLAG_QOS_WORK_REG, &rdev->flags)) + cancel_delayed_work(&rdev->worker); + bnxt_re_cleanup_res(rdev); bnxt_re_free_res(rdev, lock_wait); @@ -810,6 +933,16 @@ static void bnxt_re_set_resource_limits(struct bnxt_re_dev *rdev) rdev->dev_attr.tqm_alloc_reqs[i]; } +/* worker thread for polling periodic events. Now used for QoS programming*/ +static void bnxt_re_worker(struct work_struct *work) +{ + struct bnxt_re_dev *rdev = container_of(work, struct bnxt_re_dev, + worker.work); + + bnxt_re_setup_qos(rdev); + schedule_delayed_work(&rdev->worker, msecs_to_jiffies(30000)); +} + static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) { int i, j, rc; @@ -894,6 +1027,14 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) goto fail; } + rc = bnxt_re_setup_qos(rdev); + if (rc) + pr_info("RoCE priority not yet configured\n"); + + INIT_DELAYED_WORK(&rdev->worker, bnxt_re_worker); + set_bit(BNXT_RE_FLAG_QOS_WORK_REG, &rdev->flags); + schedule_delayed_work(&rdev->worker, msecs_to_jiffies(30000)); + /* Register ib dev */ rc = bnxt_re_register_ib(rdev); if (rc) { diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.c b/drivers/infiniband/hw/bnxt_re/qplib_sp.c index a936c48..7b31ecc 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_sp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.c @@ -799,3 +799,40 @@ int bnxt_qplib_free_fast_reg_page_list(struct bnxt_qplib_res *res, bnxt_qplib_free_hwq(res->pdev, &frpl->hwq); return 0; } + +int bnxt_qplib_map_tc2cos(struct bnxt_qplib_res *res, u16 *cids) +{ + struct bnxt_qplib_rcfw *rcfw = res->rcfw; + struct cmdq_map_tc_to_cos req; + struct creq_map_tc_to_cos_resp *resp; + u16 cmd_flags = 0; + int tleft; + + RCFW_CMD_PREP(req, MAP_TC_TO_COS, cmd_flags); + req.cos0 = cpu_to_le16(cids[0]); + req.cos1 = cpu_to_le16(cids[1]); + + resp = bnxt_qplib_rcfw_send_message(rcfw, (void *)&req, NULL, 0); + if (!resp) { + dev_err(&res->pdev->dev, "QPLIB: SP: MAP_TC2COS send failed"); + return -EINVAL; + } + + tleft = bnxt_qplib_rcfw_block_for_resp(rcfw, le16_to_cpu(req.cookie)); + if (!tleft) { + dev_err(&res->pdev->dev, "QPLIB: SP: MAP_TC2COS timed out"); + return -ETIMEDOUT; + } + + if (resp->status || + le16_to_cpu(resp->cookie) != le16_to_cpu(req.cookie)) { + dev_err(&res->pdev->dev, "QPLIB: SP: MAP_TC2COS failed "); + dev_err(&res->pdev->dev, + "QPLIB: with status 0x%x cmdq 0x%x resp 0x%x", + resp->status, le16_to_cpu(req.cookie), + le16_to_cpu(resp->cookie)); + return -EINVAL; + } + + return 0; +} diff --git a/drivers/infiniband/hw/bnxt_re/qplib_sp.h b/drivers/infiniband/hw/bnxt_re/qplib_sp.h index 18c94c8..1442a61 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_sp.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_sp.h @@ -156,4 +156,5 @@ int bnxt_qplib_alloc_fast_reg_page_list(struct bnxt_qplib_res *res, struct bnxt_qplib_frpl *frpl, int max); int bnxt_qplib_free_fast_reg_page_list(struct bnxt_qplib_res *res, struct bnxt_qplib_frpl *frpl); +int bnxt_qplib_map_tc2cos(struct bnxt_qplib_res *res, u16 *cids); #endif /* __BNXT_QPLIB_SP_H__*/