From patchwork Fri Aug 9 09:40:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086111 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D8ED14F7 for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E62028C5B for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 52F5628C62; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0B8F28C5B for ; Fri, 9 Aug 2019 09:45:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730233AbfHIJp2 (ORCPT ); Fri, 9 Aug 2019 05:45:28 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:4210 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726152AbfHIJp2 (ORCPT ); Fri, 9 Aug 2019 05:45:28 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id BB39030FE410BB3BE2B9; Fri, 9 Aug 2019 17:45:26 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:19 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 1/9] RDMA/hns: Logic optimization of wc_flags Date: Fri, 9 Aug 2019 17:40:58 +0800 Message-ID: <1565343666-73193-2-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Weihang Li We should set IB_WC_WITH_VLAN only when VLAN is enabled. In addition, move setting of IB_WC_WITH_SMAC below setting of wc->smac. Signed-off-by: Weihang Li --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 87a1574..7a14f0b 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -2860,15 +2860,16 @@ static int hns_roce_v2_poll_one(struct hns_roce_cq *hr_cq, wc->smac[5] = roce_get_field(cqe->byte_28, V2_CQE_BYTE_28_SMAC_5_M, V2_CQE_BYTE_28_SMAC_5_S); + wc->wc_flags |= IB_WC_WITH_SMAC; if (roce_get_bit(cqe->byte_28, V2_CQE_BYTE_28_VID_VLD_S)) { wc->vlan_id = (u16)roce_get_field(cqe->byte_28, V2_CQE_BYTE_28_VID_M, V2_CQE_BYTE_28_VID_S); + wc->wc_flags |= IB_WC_WITH_VLAN; } else { wc->vlan_id = 0xffff; } - wc->wc_flags |= (IB_WC_WITH_VLAN | IB_WC_WITH_SMAC); wc->network_hdr_type = roce_get_field(cqe->byte_28, V2_CQE_BYTE_28_PORT_TYPE_M, V2_CQE_BYTE_28_PORT_TYPE_S); From patchwork Fri Aug 9 09:40:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086119 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDC7E6C5 for ; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE12E28C52 for ; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C279128C5E; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3762C28C52 for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405764AbfHIJp3 (ORCPT ); Fri, 9 Aug 2019 05:45:29 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:4212 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405976AbfHIJp3 (ORCPT ); Fri, 9 Aug 2019 05:45:29 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id BF1AC63219FC138F2EA9; Fri, 9 Aug 2019 17:45:26 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:20 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 2/9] RDMA/hns: Bugfix for creating qp attached to srq Date: Fri, 9 Aug 2019 17:40:59 +0800 Message-ID: <1565343666-73193-3-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When create a qp and attached to srq, rq will no longer be used and the members of rq will be set zero. As a result, the wrid of rq will not be allocated and used. Fixes: 926a01dc000d ("RDMA/hns: Add QP operations support for hip08 SoC") Signed-off-by: Lijun Ou --- drivers/infiniband/hw/hns/hns_roce_qp.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c index b729f8e..5f23c09 100644 --- a/drivers/infiniband/hw/hns/hns_roce_qp.c +++ b/drivers/infiniband/hw/hns/hns_roce_qp.c @@ -853,12 +853,19 @@ static int hns_roce_create_qp_common(struct hns_roce_dev *hr_dev, } hr_qp->sq.wrid = kcalloc(hr_qp->sq.wqe_cnt, sizeof(u64), - GFP_KERNEL); - hr_qp->rq.wrid = kcalloc(hr_qp->rq.wqe_cnt, sizeof(u64), - GFP_KERNEL); - if (!hr_qp->sq.wrid || !hr_qp->rq.wrid) { + GFP_KERNEL); + if (ZERO_OR_NULL_PTR(hr_qp->sq.wrid)) { ret = -ENOMEM; - goto err_wrid; + goto err_get_bufs; + } + + if (hr_qp->rq.wqe_cnt) { + hr_qp->rq.wrid = kcalloc(hr_qp->rq.wqe_cnt, sizeof(u64), + GFP_KERNEL); + if (ZERO_OR_NULL_PTR(hr_qp->rq.wrid)) { + ret = -ENOMEM; + goto err_sq_wrid; + } } } @@ -944,8 +951,8 @@ static int hns_roce_create_qp_common(struct hns_roce_dev *hr_dev, hns_roce_qp_has_rq(init_attr)) hns_roce_db_unmap_user(uctx, &hr_qp->rdb); } else { - kfree(hr_qp->sq.wrid); - kfree(hr_qp->rq.wrid); + if (hr_qp->rq.wqe_cnt) + kfree(hr_qp->rq.wrid); } err_sq_dbmap: @@ -956,6 +963,10 @@ static int hns_roce_create_qp_common(struct hns_roce_dev *hr_dev, hns_roce_qp_has_sq(init_attr)) hns_roce_db_unmap_user(uctx, &hr_qp->sdb); +err_sq_wrid: + if (!udata) + kfree(hr_qp->sq.wrid); + err_get_bufs: hns_roce_free_buf_list(buf_list, hr_qp->region_cnt); From patchwork Fri Aug 9 09:41:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086117 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63DAB14F7 for ; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5062A28C5B for ; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4475728C62; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCEA228C61 for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2406053AbfHIJpa (ORCPT ); Fri, 9 Aug 2019 05:45:30 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:4211 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405773AbfHIJp3 (ORCPT ); Fri, 9 Aug 2019 05:45:29 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id B71F23BEB48BEA89C8A9; Fri, 9 Aug 2019 17:45:26 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:20 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 3/9] RDMA/hns: Completely release qp resources when hw err Date: Fri, 9 Aug 2019 17:41:00 +0800 Message-ID: <1565343666-73193-4-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yangyang Li Even if no response from hardware, make sure that qp related resources are completely released. Signed-off-by: Yangyang Li --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 7a14f0b..0409851 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -4562,16 +4562,14 @@ static int hns_roce_v2_destroy_qp_common(struct hns_roce_dev *hr_dev, { struct hns_roce_cq *send_cq, *recv_cq; struct ib_device *ibdev = &hr_dev->ib_dev; - int ret; + int ret = 0; if (hr_qp->ibqp.qp_type == IB_QPT_RC && hr_qp->state != IB_QPS_RESET) { /* Modify qp to reset before destroying qp */ ret = hns_roce_v2_modify_qp(&hr_qp->ibqp, NULL, 0, hr_qp->state, IB_QPS_RESET); - if (ret) { + if (ret) ibdev_err(ibdev, "modify QP to Reset failed.\n"); - return ret; - } } send_cq = to_hr_cq(hr_qp->ibqp.send_cq); @@ -4627,7 +4625,7 @@ static int hns_roce_v2_destroy_qp_common(struct hns_roce_dev *hr_dev, kfree(hr_qp->rq_inl_buf.wqe_list); } - return 0; + return ret; } static int hns_roce_v2_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) @@ -4637,11 +4635,9 @@ static int hns_roce_v2_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) int ret; ret = hns_roce_v2_destroy_qp_common(hr_dev, hr_qp, udata); - if (ret) { + if (ret) ibdev_err(&hr_dev->ib_dev, "Destroy qp 0x%06lx failed(%d)\n", hr_qp->qpn, ret); - return ret; - } if (hr_qp->ibqp.qp_type == IB_QPT_GSI) kfree(hr_to_hr_sqp(hr_qp)); From patchwork Fri Aug 9 09:41:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086115 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C06D6C5 for ; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C77528C5B for ; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 10B0A28C63; Fri, 9 Aug 2019 09:45:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD52028C5B for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405976AbfHIJpa (ORCPT ); Fri, 9 Aug 2019 05:45:30 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:4209 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2406053AbfHIJp3 (ORCPT ); Fri, 9 Aug 2019 05:45:29 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id B2FE31DE1C64D207FA77; Fri, 9 Aug 2019 17:45:26 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:20 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 4/9] RDMA/hns: Modify pi vlaue when cq overflows Date: Fri, 9 Aug 2019 17:41:01 +0800 Message-ID: <1565343666-73193-5-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yangyang Li When exiting "for loop", the actual value of pi will be increased by 1, which is compatible with the next calculation. But when pi is equal to "ci + hr_cq-> ib_cq.cqe", the "break" was called and the pi is actual value, it will lead one cqe still existing, so the "==" should be modify to ">". Signed-off-by: Yangyang Li --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 0409851..c6b55a1 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -2407,7 +2407,7 @@ static void __hns_roce_v2_cq_clean(struct hns_roce_cq *hr_cq, u32 qpn, for (prod_index = hr_cq->cons_index; get_sw_cqe_v2(hr_cq, prod_index); ++prod_index) { - if (prod_index == hr_cq->cons_index + hr_cq->ib_cq.cqe) + if (prod_index > hr_cq->cons_index + hr_cq->ib_cq.cqe) break; } From patchwork Fri Aug 9 09:41:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086113 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B006C1709 for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB3628C5B for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93B7928C62; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 16CA128C61 for ; Fri, 9 Aug 2019 09:45:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726152AbfHIJp3 (ORCPT ); Fri, 9 Aug 2019 05:45:29 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:55368 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405764AbfHIJp2 (ORCPT ); Fri, 9 Aug 2019 05:45:28 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id DE3FFEB024A772756ADD; Fri, 9 Aug 2019 17:45:26 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:20 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 5/9] RDMA/hns: Bugfix for slab-out-of-bounds when unloading hip08 driver Date: Fri, 9 Aug 2019 17:41:02 +0800 Message-ID: <1565343666-73193-6-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Xi Wang kasan will report a BUG when run command 'rmmod hns_roce_hw_v2', the calltrace is as follows: ================================================================== BUG: KASAN: slab-out-of-bounds in hns_roce_table_mhop_put+0x584/0x828 [hns_roce] Read of size 8 at addr ffff802185e08300 by task rmmod/270 Call trace: dump_backtrace+0x0/0x1e8 show_stack+0x14/0x20 dump_stack+0xc4/0xfc print_address_description+0x60/0x270 __kasan_report+0x164/0x1b8 kasan_report+0xc/0x18 __asan_load8+0x84/0xa8 hns_roce_table_mhop_put+0x584/0x828 [hns_roce] hns_roce_table_put+0x174/0x1a0 [hns_roce] hns_roce_mr_free+0x124/0x210 [hns_roce] hns_roce_dereg_mr+0x90/0xb8 [hns_roce] ib_dealloc_pd_user+0x60/0xf0 ib_mad_port_close+0x128/0x1d8 ib_mad_remove_device+0x94/0x118 remove_client_context+0xa0/0xe0 disable_device+0xfc/0x1c0 __ib_unregister_device+0x60/0xe0 ib_unregister_device+0x24/0x38 hns_roce_exit+0x3c/0x138 [hns_roce] __hns_roce_hw_v2_uninit_instance.isra.30+0x28/0x50 [hns_roce_hw_v2] hns_roce_hw_v2_uninit_instance+0x44/0x60 [hns_roce_hw_v2] hclge_uninit_client_instance+0x15c/0x238 [hclge] hnae3_uninit_client_instance+0x84/0xa8 [hnae3] hnae3_unregister_client+0x84/0x158 [hnae3] hns_roce_hw_v2_exit+0x14/0x20 [hns_roce_hw_v2] __arm64_sys_delete_module+0x20c/0x308 el0_svc_handler+0xbc/0x210 el0_svc+0x8/0xc Allocated by task 255: __kasan_kmalloc.isra.0+0xd0/0x180 kasan_kmalloc+0xc/0x18 __kmalloc+0x16c/0x328 hns_roce_init_hem_table+0x20c/0x428 [hns_roce] hns_roce_init+0x214/0xfe0 [hns_roce] __hns_roce_hw_v2_init_instance+0x284/0x330 [hns_roce_hw_v2] hns_roce_hw_v2_init_instance+0xd0/0x1b8 [hns_roce_hw_v2] hclge_init_roce_client_instance+0x180/0x310 [hclge] hclge_init_client_instance+0xcc/0x508 [hclge] hnae3_init_client_instance.part.3+0x3c/0x80 [hnae3] hnae3_register_client+0x134/0x1a8 [hnae3] 0xffff200009c00014 do_one_initcall+0x9c/0x3e0 do_init_module+0xd4/0x2d8 load_module+0x3284/0x3690 __se_sys_init_module+0x274/0x308 __arm64_sys_init_module+0x40/0x50 el0_svc_handler+0xbc/0x210 el0_svc+0x8/0xc Freed by task 0: (stack is not available) The buggy address belongs to the object at ffff802185e06300 which belongs to the cache kmalloc-8k of size 8192 The buggy address is located 0 bytes to the right of 8192-byte region [ffff802185e06300, ffff802185e08300) The buggy address belongs to the page: page:ffff7fe008617800 refcount:1 mapcount:0 mapping:ffff802340020e00 index:0x0 compound_mapcount: 0 flags: 0x5fffe00000010200(slab|head) raw: 5fffe00000010200 dead000000000100 dead000000000200 ffff802340020e00 raw: 0000000000000000 00000000803e003e 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff802185e08200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ffff802185e08280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >ffff802185e08300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ^ ffff802185e08380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ffff802185e08400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ================================================================== Disabling lock debugging due to kernel taint Fixes: a25d13cbe816 ("RDMA/hns: Add the interfaces to support multi hop addressing for the contexts in hip08") Signed-off-by: Xi Wang --- drivers/infiniband/hw/hns/hns_roce_hem.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hem.c b/drivers/infiniband/hw/hns/hns_roce_hem.c index 0268c7a..f2c4fef 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hem.c +++ b/drivers/infiniband/hw/hns/hns_roce_hem.c @@ -85,12 +85,13 @@ bool hns_roce_check_whether_mhop(struct hns_roce_dev *hr_dev, u32 type) } static bool hns_roce_check_hem_null(struct hns_roce_hem **hem, u64 start_idx, - u32 bt_chunk_num) + u32 bt_chunk_num, u64 hem_max_num) { - int i; + u64 check_max_num = start_idx + bt_chunk_num; + u64 i; - for (i = 0; i < bt_chunk_num; i++) - if (hem[start_idx + i]) + for (i = start_idx; (i < check_max_num) && (i < hem_max_num); i++) + if (hem[i]) return false; return true; @@ -496,6 +497,12 @@ static int hns_roce_table_mhop_get(struct hns_roce_dev *hr_dev, return -EINVAL; } + if (unlikely(hem_idx >= table->num_hem)) { + dev_err(dev, "Table %d exceed hem limt idx = %llu,max = %lu!\n", + table->type, hem_idx, table->num_hem); + return -EINVAL; + } + mutex_lock(&table->mutex); if (table->hem[hem_idx]) { @@ -732,7 +739,7 @@ static void hns_roce_table_mhop_put(struct hns_roce_dev *hr_dev, if (check_whether_bt_num_2(table->type, hop_num)) { start_idx = mhop.l0_idx * chunk_ba_num; if (hns_roce_check_hem_null(table->hem, start_idx, - chunk_ba_num)) { + chunk_ba_num, table->num_hem)) { if (table->type < HEM_TYPE_MTT && hr_dev->hw->clear_hem(hr_dev, table, obj, 0)) dev_warn(dev, "Clear HEM base address failed.\n"); @@ -746,7 +753,7 @@ static void hns_roce_table_mhop_put(struct hns_roce_dev *hr_dev, start_idx = mhop.l0_idx * chunk_ba_num * chunk_ba_num + mhop.l1_idx * chunk_ba_num; if (hns_roce_check_hem_null(table->hem, start_idx, - chunk_ba_num)) { + chunk_ba_num, table->num_hem)) { if (hr_dev->hw->clear_hem(hr_dev, table, obj, 1)) dev_warn(dev, "Clear HEM base address failed.\n"); From patchwork Fri Aug 9 09:41:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086125 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81EF014D5 for ; Fri, 9 Aug 2019 09:45:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71CEC28C52 for ; Fri, 9 Aug 2019 09:45:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 662E528C5E; Fri, 9 Aug 2019 09:45:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EDA0A28C5B for ; Fri, 9 Aug 2019 09:45:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2406120AbfHIJpe (ORCPT ); Fri, 9 Aug 2019 05:45:34 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:55538 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405773AbfHIJpe (ORCPT ); Fri, 9 Aug 2019 05:45:34 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id C585D7AD63B6C701E89A; Fri, 9 Aug 2019 17:45:31 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:21 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 6/9] RDMA/hns: bugfix for slab-out-of-bounds when loading hip08 driver Date: Fri, 9 Aug 2019 17:41:03 +0800 Message-ID: <1565343666-73193-7-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Xi Wang kasan will report a BUG when run command 'insmod hns_roce_hw_v2.ko', the calltrace is as follows: ================================================================== BUG: KASAN: slab-out-of-bounds in hns_roce_v2_init_eq_table+0x1324/0x1948 [hns_roce_hw_v2] Read of size 8 at addr ffff8020e7a10608 by task insmod/256 CPU: 0 PID: 256 Comm: insmod Tainted: G O 5.2.0-rc4 #1 Hardware name: Huawei D06 /D06, BIOS Hisilicon D06 UEFI RC0 Call trace: dump_backtrace+0x0/0x1e8 show_stack+0x14/0x20 dump_stack+0xc4/0xfc print_address_description+0x60/0x270 __kasan_report+0x164/0x1b8 kasan_report+0xc/0x18 __asan_load8+0x84/0xa8 hns_roce_v2_init_eq_table+0x1324/0x1948 [hns_roce_hw_v2] hns_roce_init+0xf8/0xfe0 [hns_roce] __hns_roce_hw_v2_init_instance+0x284/0x330 [hns_roce_hw_v2] hns_roce_hw_v2_init_instance+0xd0/0x1b8 [hns_roce_hw_v2] hclge_init_roce_client_instance+0x180/0x310 [hclge] hclge_init_client_instance+0xcc/0x508 [hclge] hnae3_init_client_instance.part.3+0x3c/0x80 [hnae3] hnae3_register_client+0x134/0x1a8 [hnae3] hns_roce_hw_v2_init+0x14/0x10000 [hns_roce_hw_v2] do_one_initcall+0x9c/0x3e0 do_init_module+0xd4/0x2d8 load_module+0x3284/0x3690 __se_sys_init_module+0x274/0x308 __arm64_sys_init_module+0x40/0x50 el0_svc_handler+0xbc/0x210 el0_svc+0x8/0xc Allocated by task 256: __kasan_kmalloc.isra.0+0xd0/0x180 kasan_kmalloc+0xc/0x18 __kmalloc+0x16c/0x328 hns_roce_v2_init_eq_table+0x764/0x1948 [hns_roce_hw_v2] hns_roce_init+0xf8/0xfe0 [hns_roce] __hns_roce_hw_v2_init_instance+0x284/0x330 [hns_roce_hw_v2] hns_roce_hw_v2_init_instance+0xd0/0x1b8 [hns_roce_hw_v2] hclge_init_roce_client_instance+0x180/0x310 [hclge] hclge_init_client_instance+0xcc/0x508 [hclge] hnae3_init_client_instance.part.3+0x3c/0x80 [hnae3] hnae3_register_client+0x134/0x1a8 [hnae3] hns_roce_hw_v2_init+0x14/0x10000 [hns_roce_hw_v2] do_one_initcall+0x9c/0x3e0 do_init_module+0xd4/0x2d8 load_module+0x3284/0x3690 __se_sys_init_module+0x274/0x308 __arm64_sys_init_module+0x40/0x50 el0_svc_handler+0xbc/0x210 el0_svc+0x8/0xc Freed by task 0: (stack is not available) The buggy address belongs to the object at ffff8020e7a10600 which belongs to the cache kmalloc-128 of size 128 The buggy address is located 8 bytes inside of 128-byte region [ffff8020e7a10600, ffff8020e7a10680) The buggy address belongs to the page: page:ffff7fe00839e840 refcount:1 mapcount:0 mapping:ffff802340020200 index:0x0 flags: 0x5fffe00000000200(slab) raw: 5fffe00000000200 dead000000000100 dead000000000200 ffff802340020200 raw: 0000000000000000 0000000081000100 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff8020e7a10500: 00 00 00 00 00 00 00 00 fc fc fc fc fc fc fc fc ffff8020e7a10580: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc >ffff8020e7a10600: 00 fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ^ ffff8020e7a10680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ffff8020e7a10700: 00 fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ================================================================== Disabling lock debugging due to kernel taint Fixes: a5073d6054f7 ("RDMA/hns: Add eq support of hip08") Signed-off-by: Xi Wang --- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index c6b55a1..d33341e 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -5543,7 +5543,8 @@ static int hns_roce_mhop_alloc_eq(struct hns_roce_dev *hr_dev, break; } eq->cur_eqe_ba = eq->buf_dma[0]; - eq->nxt_eqe_ba = eq->buf_dma[1]; + if (ba_num > 1) + eq->nxt_eqe_ba = eq->buf_dma[1]; } else if (mhop_num == 2) { /* alloc L1 BT and buf */ @@ -5584,7 +5585,8 @@ static int hns_roce_mhop_alloc_eq(struct hns_roce_dev *hr_dev, break; } eq->cur_eqe_ba = eq->buf_dma[0]; - eq->nxt_eqe_ba = eq->buf_dma[1]; + if (ba_num > 1) + eq->nxt_eqe_ba = eq->buf_dma[1]; } eq->l0_last_num = i + 1; From patchwork Fri Aug 9 09:41:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086121 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4ED146C5 for ; Fri, 9 Aug 2019 09:45:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BC5328C52 for ; Fri, 9 Aug 2019 09:45:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2E42828C5E; Fri, 9 Aug 2019 09:45:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D6DC528C52 for ; Fri, 9 Aug 2019 09:45:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2406043AbfHIJpd (ORCPT ); Fri, 9 Aug 2019 05:45:33 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:55516 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405773AbfHIJpd (ORCPT ); Fri, 9 Aug 2019 05:45:33 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id C1C37E7F778F3AB9B2A5; Fri, 9 Aug 2019 17:45:31 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:21 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 7/9] RDMA/hns: Remove unuseful member Date: Fri, 9 Aug 2019 17:41:04 +0800 Message-ID: <1565343666-73193-8-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Lang Cheng For struct hns_roce_cmdq, toggle is unused and delete it. Signed-off-by: Lang Cheng --- drivers/infiniband/hw/hns/hns_roce_cmd.c | 1 - drivers/infiniband/hw/hns/hns_roce_device.h | 1 - 2 files changed, 2 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_cmd.c b/drivers/infiniband/hw/hns/hns_roce_cmd.c index 0cd09bf..ade26fa 100644 --- a/drivers/infiniband/hw/hns/hns_roce_cmd.c +++ b/drivers/infiniband/hw/hns/hns_roce_cmd.c @@ -211,7 +211,6 @@ int hns_roce_cmd_init(struct hns_roce_dev *hr_dev) mutex_init(&hr_dev->cmd.hcr_mutex); sema_init(&hr_dev->cmd.poll_sem, 1); hr_dev->cmd.use_events = 0; - hr_dev->cmd.toggle = 1; hr_dev->cmd.max_cmds = CMD_MAX_NUM; hr_dev->cmd.pool = dma_pool_create("hns_roce_cmd", dev, HNS_ROCE_MAILBOX_SIZE, diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index 8c4b120..32465f5 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -617,7 +617,6 @@ struct hns_roce_cmdq { * close device, switch into poll mode(non event mode) */ u8 use_events; - u8 toggle; }; struct hns_roce_cmd_mailbox { From patchwork Fri Aug 9 09:41:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086123 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 46B326C5 for ; Fri, 9 Aug 2019 09:45:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 376B828C52 for ; Fri, 9 Aug 2019 09:45:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2BDF328C61; Fri, 9 Aug 2019 09:45:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A016228C52 for ; Fri, 9 Aug 2019 09:45:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2406136AbfHIJpe (ORCPT ); Fri, 9 Aug 2019 05:45:34 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:55556 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2406120AbfHIJpd (ORCPT ); Fri, 9 Aug 2019 05:45:33 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id C9C25159F539E5631454; Fri, 9 Aug 2019 17:45:31 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:21 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 8/9] RDMA/hns: Kernel notify usr space to stop ring db Date: Fri, 9 Aug 2019 17:41:05 +0800 Message-ID: <1565343666-73193-9-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yangyang Li In the reset scenario, if the kernel receives the reset signal, it needs to notify the user space to stop ring doorbell. Signed-off-by: Yangyang Li --- drivers/infiniband/hw/hns/hns_roce_device.h | 4 +++ drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 52 ++++++++++++++++++++++++++++- drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 4 +++ drivers/infiniband/hw/hns/hns_roce_main.c | 22 ++++++------ 4 files changed, 70 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index 32465f5..be65fce 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -268,6 +268,8 @@ enum { #define PAGE_ADDR_SHIFT 12 +#define HNS_ROCE_IS_RESETTING 1 + struct hns_roce_uar { u64 pfn; unsigned long index; @@ -1043,6 +1045,8 @@ struct hns_roce_dev { u32 odb_offset; dma_addr_t tptr_dma_addr; /* only for hw v1 */ u32 tptr_size; /* only for hw v1 */ + struct page *reset_page; /* store reset state */ + void *reset_kaddr; /* addr of reset page */ const struct hns_roce_hw *hw; void *priv; struct workqueue_struct *irq_workq; diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index d33341e..138e5a8 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -1867,17 +1867,49 @@ static void hns_roce_free_link_table(struct hns_roce_dev *hr_dev, link_tbl->table.map); } +static int hns_roce_v2_get_reset_page(struct hns_roce_dev *hr_dev) +{ + hr_dev->reset_page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!hr_dev->reset_page) + return -ENOMEM; + + hr_dev->reset_kaddr = vmap(&hr_dev->reset_page, 1, VM_MAP, PAGE_KERNEL); + if (!hr_dev->reset_kaddr) + goto err_with_vmap; + + return 0; + +err_with_vmap: + put_page(hr_dev->reset_page); + return -ENOMEM; +} + +static void hns_roce_v2_put_reset_page(struct hns_roce_dev *hr_dev) +{ + vunmap(hr_dev->reset_kaddr); + hr_dev->reset_kaddr = NULL; + put_page(hr_dev->reset_page); + hr_dev->reset_page = NULL; +} + static int hns_roce_v2_init(struct hns_roce_dev *hr_dev) { struct hns_roce_v2_priv *priv = hr_dev->priv; int qpc_count, cqc_count; int ret, i; + ret = hns_roce_v2_get_reset_page(hr_dev); + if (ret) { + dev_err(hr_dev->dev, + "reset state init failed, ret = %d.\n", ret); + return ret; + } + /* TSQ includes SQ doorbell and ack doorbell */ ret = hns_roce_init_link_table(hr_dev, TSQ_LINK_TABLE); if (ret) { dev_err(hr_dev->dev, "TSQ init failed, ret = %d.\n", ret); - return ret; + goto err_tsq_init_failed; } ret = hns_roce_init_link_table(hr_dev, TPQ_LINK_TABLE); @@ -1923,6 +1955,9 @@ static int hns_roce_v2_init(struct hns_roce_dev *hr_dev) err_tpq_init_failed: hns_roce_free_link_table(hr_dev, &priv->tsq); +err_tsq_init_failed: + hns_roce_v2_put_reset_page(hr_dev); + return ret; } @@ -1935,6 +1970,7 @@ static void hns_roce_v2_exit(struct hns_roce_dev *hr_dev) hns_roce_free_link_table(hr_dev, &priv->tpq); hns_roce_free_link_table(hr_dev, &priv->tsq); + hns_roce_v2_put_reset_page(hr_dev); } static int hns_roce_query_mbox_status(struct hns_roce_dev *hr_dev) @@ -6434,6 +6470,19 @@ static void hns_roce_hw_v2_uninit_instance(struct hnae3_handle *handle, handle->rinfo.instance_state = HNS_ROCE_STATE_NON_INIT; } + +static void hns_roce_v2_reset_notify_user(struct hns_roce_dev *hr_dev) +{ + struct hns_roce_v2_reset_state *state; + + state = (struct hns_roce_v2_reset_state *) hr_dev->reset_kaddr; + + state->reset_state = HNS_ROCE_IS_RESETTING; + + /* Ensure reset state was flushed in memory */ + wmb(); +} + static int hns_roce_hw_v2_reset_notify_down(struct hnae3_handle *handle) { struct hns_roce_dev *hr_dev; @@ -6454,6 +6503,7 @@ static int hns_roce_hw_v2_reset_notify_down(struct hnae3_handle *handle) hr_dev->is_reset = true; hr_dev->active = false; hr_dev->dis_db = true; + hns_roce_v2_reset_notify_user(hr_dev); event.event = IB_EVENT_DEVICE_FATAL; event.device = &hr_dev->ib_dev; diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h index 58931b5..392bc03 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h @@ -1622,6 +1622,10 @@ struct hns_roce_link_table_entry { #define HNS_ROCE_LINK_TABLE_NXT_PTR_S 20 #define HNS_ROCE_LINK_TABLE_NXT_PTR_M GENMASK(31, 20) +struct hns_roce_v2_reset_state { + u32 reset_state; /* stored to use in user space */ +}; + struct hns_roce_v2_priv { struct hnae3_handle *handle; struct hns_roce_v2_cmq cmq; diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c index 1e4ba48..1bda7a5 100644 --- a/drivers/infiniband/hw/hns/hns_roce_main.c +++ b/drivers/infiniband/hw/hns/hns_roce_main.c @@ -360,18 +360,18 @@ static int hns_roce_mmap(struct ib_ucontext *context, PAGE_SIZE, pgprot_noncached(vma->vm_page_prot)); - /* vm_pgoff: 1 -- TPTR */ + /* vm_pgoff: 1 -- TPTR(hw v1), reset_page(hw v2) */ case 1: - if (!hr_dev->tptr_dma_addr || !hr_dev->tptr_size) - return -EINVAL; - /* - * FIXME: using io_remap_pfn_range on the dma address returned - * by dma_alloc_coherent is totally wrong. - */ - return rdma_user_mmap_io(context, vma, - hr_dev->tptr_dma_addr >> PAGE_SHIFT, - hr_dev->tptr_size, - vma->vm_page_prot); + if (hr_dev->tptr_dma_addr && hr_dev->tptr_size) + return rdma_user_mmap_io(context, vma, + hr_dev->tptr_dma_addr >> PAGE_SHIFT, + hr_dev->tptr_size, + vma->vm_page_prot); + + if (hr_dev->reset_page) + return remap_pfn_range(vma, vma->vm_start, + page_to_pfn(hr_dev->reset_page), + PAGE_SIZE, vma->vm_page_prot); default: return -EINVAL; From patchwork Fri Aug 9 09:41:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 11086161 X-Patchwork-Delegate: dledford@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D7CE14E5 for ; Fri, 9 Aug 2019 10:01:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F047928C0C for ; Fri, 9 Aug 2019 10:01:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E491128C61; Fri, 9 Aug 2019 10:01:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65DD328C0C for ; Fri, 9 Aug 2019 10:01:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726164AbfHIKBY (ORCPT ); Fri, 9 Aug 2019 06:01:24 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:49852 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726152AbfHIKBY (ORCPT ); Fri, 9 Aug 2019 06:01:24 -0400 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id CE9A3AAEFD90F4BBB567; Fri, 9 Aug 2019 17:45:31 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.439.0; Fri, 9 Aug 2019 17:45:21 +0800 From: Lijun Ou To: , CC: , , Subject: [PATCH for-next 9/9] RDMA/hns: Copy some information of AV to user Date: Fri, 9 Aug 2019 17:41:06 +0800 Message-ID: <1565343666-73193-10-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1565343666-73193-1-git-send-email-oulijun@huawei.com> References: <1565343666-73193-1-git-send-email-oulijun@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When the driver support UD transport in user mode, it needs to create the Address Handle(AH) and transfer Address Vector to The hardware. The Address Vector includes the destination mac and vlan inforation and it will be generated from the kernel driver. As a result, we can copy this information to user through ib_copy_to_udata function. Signed-off-by: Lijun Ou --- drivers/infiniband/hw/hns/hns_roce_ah.c | 22 ++++++++++++++++++---- drivers/infiniband/hw/hns/hns_roce_device.h | 3 ++- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 ++- drivers/infiniband/hw/hns/hns_roce_main.c | 7 +++++++ include/uapi/rdma/hns-abi.h | 7 +++++++ 5 files changed, 36 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_ah.c b/drivers/infiniband/hw/hns/hns_roce_ah.c index cdd2ac2..eee53be 100644 --- a/drivers/infiniband/hw/hns/hns_roce_ah.c +++ b/drivers/infiniband/hw/hns/hns_roce_ah.c @@ -31,6 +31,7 @@ */ #include +#include #include #include #include "hns_roce_device.h" @@ -43,13 +44,14 @@ int hns_roce_create_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr, u32 flags, struct ib_udata *udata) { struct hns_roce_dev *hr_dev = to_hr_dev(ibah->device); + struct hns_roce_ib_create_ah_resp resp = {}; const struct ib_gid_attr *gid_attr; struct device *dev = hr_dev->dev; struct hns_roce_ah *ah = to_hr_ah(ibah); u16 vlan_tag = 0xffff; const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr); - bool vlan_en = false; - int ret; + u8 vlan_en = 0; + int ret = 0; gid_attr = ah_attr->grh.sgid_attr; ret = rdma_read_gid_l2_fields(gid_attr, &vlan_tag, NULL); @@ -60,7 +62,7 @@ int hns_roce_create_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr, memcpy(ah->av.mac, ah_attr->roce.dmac, ETH_ALEN); if (vlan_tag < VLAN_CFI_MASK) { - vlan_en = true; + vlan_en = 1; vlan_tag |= (rdma_ah_get_sl(ah_attr) & HNS_ROCE_VLAN_SL_BIT_MASK) << HNS_ROCE_VLAN_SL_SHIFT; @@ -80,7 +82,19 @@ int hns_roce_create_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr, memcpy(ah->av.dgid, grh->dgid.raw, HNS_ROCE_GID_SIZE); ah->av.sl_tclass_flowlabel = cpu_to_le32(rdma_ah_get_sl(ah_attr) << - HNS_ROCE_SL_SHIFT); + i HNS_ROCE_SL_SHIFT); + + if (udata) { + memcpy(resp.dmac, ah_attr->roce.dmac, ETH_ALEN); + resp.vlan = vlan_tag; + resp.vlan_en = vlan_en; + ret = ib_copy_to_udata(udata, &resp, + min(udata->outlen, sizeof(resp))); + if (ret) { + kfree(ah); + return ret; + } + } return 0; } diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index be65fce..e703912 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -217,6 +217,7 @@ enum { HNS_ROCE_CAP_FLAG_FRMR = BIT(8), HNS_ROCE_CAP_FLAG_QP_FLOW_CTRL = BIT(9), HNS_ROCE_CAP_FLAG_ATOMIC = BIT(10), + HNS_ROCE_CAP_FLAG_UD = BIT(11), }; enum hns_roce_mtt_type { @@ -578,7 +579,7 @@ struct hns_roce_av { u8 dgid[HNS_ROCE_GID_SIZE]; u8 mac[ETH_ALEN]; __le16 vlan; - bool vlan_en; + u8 vlan_en; }; struct hns_roce_ah { diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 138e5a8..71166e7 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -1653,7 +1653,8 @@ static int hns_roce_v2_profile(struct hns_roce_dev *hr_dev) if (hr_dev->pci_dev->revision == 0x21) { caps->flags |= HNS_ROCE_CAP_FLAG_ATOMIC | HNS_ROCE_CAP_FLAG_SRQ | - HNS_ROCE_CAP_FLAG_QP_FLOW_CTRL; + HNS_ROCE_CAP_FLAG_QP_FLOW_CTRL | + HNS_ROCE_CAP_FLAG_UD; caps->num_qpc_timer = HNS_ROCE_V2_MAX_QPC_TIMER_NUM; caps->qpc_timer_entry_sz = HNS_ROCE_V2_QPC_TIMER_ENTRY_SZ; diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c index 1bda7a5..ba9594c 100644 --- a/drivers/infiniband/hw/hns/hns_roce_main.c +++ b/drivers/infiniband/hw/hns/hns_roce_main.c @@ -550,6 +550,13 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev) if (ret) return ret; } + + if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_UD) { + ib_dev->uverbs_cmd_mask |= + (1ULL << IB_USER_VERBS_CMD_CREATE_AH) | + (1ULL << IB_USER_VERBS_CMD_DESTROY_AH); + } + ret = ib_register_device(ib_dev, "hns_%d"); if (ret) { dev_err(dev, "ib_register_device failed!\n"); diff --git a/include/uapi/rdma/hns-abi.h b/include/uapi/rdma/hns-abi.h index eb76b38..6a2d994 100644 --- a/include/uapi/rdma/hns-abi.h +++ b/include/uapi/rdma/hns-abi.h @@ -80,4 +80,11 @@ struct hns_roce_ib_alloc_pd_resp { __u32 pdn; }; +struct hns_roce_ib_create_ah_resp { + __u8 dmac[6]; + __u16 vlan; + __u8 vlan_en; + __u8 reserved[7]; +}; + #endif /* HNS_ABI_USER_H */