From patchwork Fri Dec 29 11:26:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yixian Liu X-Patchwork-Id: 10136977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C8F5860318 for ; Fri, 29 Dec 2017 10:39:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C17A42DEEE for ; Fri, 29 Dec 2017 10:39:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B52032DF07; Fri, 29 Dec 2017 10:39:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 259BA2DEEE for ; Fri, 29 Dec 2017 10:39:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754600AbdL2Kjc (ORCPT ); Fri, 29 Dec 2017 05:39:32 -0500 Received: from szxga04-in.huawei.com ([45.249.212.190]:3209 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754719AbdL2Kjb (ORCPT ); Fri, 29 Dec 2017 05:39:31 -0500 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 811B7ACA49BE2; Fri, 29 Dec 2017 18:39:17 +0800 (CST) Received: from localhost.localdomain (10.67.212.75) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.361.1; Fri, 29 Dec 2017 18:39:18 +0800 From: Yixian Liu To: , CC: , Subject: [PATCH v4 for-next 1/2] RDMA/hns: Add detailed comments for mb() call Date: Fri, 29 Dec 2017 19:26:18 +0800 Message-ID: <1514546779-206832-2-git-send-email-liuyixian@huawei.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1514546779-206832-1-git-send-email-liuyixian@huawei.com> References: <1514546779-206832-1-git-send-email-liuyixian@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.212.75] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds more detailed comments when we call the memory barrier function, such as rmb, wmb and mb. Three mb() callers are deleted since they are unnecessary. Signed-off-by: Yixian Liu Signed-off-by: Lijun Ou Signed-off-by: Wei Hu (Xavier) Reviewed-by: Leon Romanovsky --- v3 -> v4: 1. Delete two mb() call since they are unnecessary. v3: 1. Subsitute rmb() with dma_rmb() according to Jason's comment, since using dma_rmb() is enough. --- drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 19 ++++++++++++------- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 26 ++++++++++---------------- 2 files changed, 22 insertions(+), 23 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c index 6100ace..73f187e 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c @@ -3963,8 +3963,6 @@ static void set_eq_cons_index_v1(struct hns_roce_eq *eq, int req_not) { roce_raw_write((eq->cons_index & HNS_ROCE_V1_CONS_IDX_M) | (req_not << eq->log_entries), eq->doorbell); - /* Memory barrier */ - mb(); } static void hns_roce_v1_wq_catas_err_handle(struct hns_roce_dev *hr_dev, @@ -4156,13 +4154,16 @@ static int hns_roce_v1_aeq_int(struct hns_roce_dev *hr_dev, int event_type; while ((aeqe = next_aeqe_sw_v1(eq))) { + + /* Make sure we read the AEQ entry after we have checked the + * ownership bit + */ + dma_rmb(); + dev_dbg(dev, "aeqe = %p, aeqe->asyn.event_type = 0x%lx\n", aeqe, roce_get_field(aeqe->asyn, HNS_ROCE_AEQE_U32_4_EVENT_TYPE_M, HNS_ROCE_AEQE_U32_4_EVENT_TYPE_S)); - /* Memory barrier */ - rmb(); - event_type = roce_get_field(aeqe->asyn, HNS_ROCE_AEQE_U32_4_EVENT_TYPE_M, HNS_ROCE_AEQE_U32_4_EVENT_TYPE_S); @@ -4260,8 +4261,12 @@ static int hns_roce_v1_ceq_int(struct hns_roce_dev *hr_dev, u32 cqn; while ((ceqe = next_ceqe_sw_v1(eq))) { - /* Memory barrier */ - rmb(); + + /* Make sure we read CEQ entry after we have checked the + * ownership bit + */ + dma_rmb(); + cqn = roce_get_field(ceqe->comp, HNS_ROCE_CEQE_CEQE_COMP_CQN_M, HNS_ROCE_CEQE_CEQE_COMP_CQN_S); diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 04281d0..68dc718 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -3199,10 +3199,6 @@ static void set_eq_cons_index_v2(struct hns_roce_eq *eq) (eq->cons_index & HNS_ROCE_V2_CONS_IDX_M)); hns_roce_write64_k(doorbell, eq->doorbell); - - /* Memory barrier */ - mb(); - } static void hns_roce_v2_wq_catas_err_handle(struct hns_roce_dev *hr_dev, @@ -3384,8 +3380,11 @@ static int hns_roce_v2_aeq_int(struct hns_roce_dev *hr_dev, int event_type; while ((aeqe = next_aeqe_sw_v2(eq))) { - /* Memory barrier */ - rmb(); + + /* Make sure we read AEQ entry after we have checked the + * ownership bit + */ + dma_rmb(); event_type = roce_get_field(aeqe->asyn, HNS_ROCE_V2_AEQE_EVENT_TYPE_M, @@ -3500,8 +3499,11 @@ static int hns_roce_v2_ceq_int(struct hns_roce_dev *hr_dev, while ((ceqe = next_ceqe_sw_v2(eq))) { - /* Memory barrier */ - rmb(); + /* Make sure we read CEQ entry after we have checked the + * ownership bit + */ + dma_rmb(); + cqn = roce_get_field(ceqe->comp, HNS_ROCE_V2_CEQE_COMP_CQN_M, HNS_ROCE_V2_CEQE_COMP_CQN_S); @@ -3556,9 +3558,6 @@ static irqreturn_t hns_roce_v2_msix_interrupt_abn(int irq, void *dev_id) roce_set_bit(int_st, HNS_ROCE_V2_VF_INT_ST_AEQ_OVERFLOW_S, 1); roce_write(hr_dev, ROCEE_VF_ABN_INT_ST_REG, int_st); - /* Memory barrier */ - mb(); - roce_set_bit(int_en, HNS_ROCE_V2_VF_ABN_INT_EN_S, 1); roce_write(hr_dev, ROCEE_VF_ABN_INT_EN_REG, int_en); @@ -3569,9 +3568,6 @@ static irqreturn_t hns_roce_v2_msix_interrupt_abn(int irq, void *dev_id) roce_set_bit(int_st, HNS_ROCE_V2_VF_INT_ST_BUS_ERR_S, 1); roce_write(hr_dev, ROCEE_VF_ABN_INT_ST_REG, int_st); - /* Memory barrier */ - mb(); - roce_set_bit(int_en, HNS_ROCE_V2_VF_ABN_INT_EN_S, 1); roce_write(hr_dev, ROCEE_VF_ABN_INT_EN_REG, int_en); @@ -3582,8 +3578,6 @@ static irqreturn_t hns_roce_v2_msix_interrupt_abn(int irq, void *dev_id) roce_set_bit(int_st, HNS_ROCE_V2_VF_INT_ST_OTHER_ERR_S, 1); roce_write(hr_dev, ROCEE_VF_ABN_INT_ST_REG, int_st); - /* Memory barrier */ - mb(); roce_set_bit(int_en, HNS_ROCE_V2_VF_ABN_INT_EN_S, 1); roce_write(hr_dev, ROCEE_VF_ABN_INT_EN_REG, int_en);