From patchwork Thu Sep 20 17:08:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devesh Sharma X-Patchwork-Id: 10608239 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C02314DA for ; Thu, 20 Sep 2018 17:09:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB2642E259 for ; Thu, 20 Sep 2018 17:09:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF1112E25F; Thu, 20 Sep 2018 17:09:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C12322E259 for ; Thu, 20 Sep 2018 17:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728997AbeITWxZ (ORCPT ); Thu, 20 Sep 2018 18:53:25 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:40580 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726193AbeITWxY (ORCPT ); Thu, 20 Sep 2018 18:53:24 -0400 Received: by mail-wr1-f67.google.com with SMTP id y8-v6so6520231wrh.7 for ; Thu, 20 Sep 2018 10:08:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TM9ewyNVC7zUzXF5PqhW3Uv0LjjZ2WKk9uV2hty3eEs=; b=JlRz9+8DzV33IcI8Sc7x9DUQUO5Yu5lA74fprkoMn/YcNUz7UA+xmkm0LsTC64c/YC Q0f+wa76yPyn+Pb4tlwSj1F3XCedtw1XZpyP2nez+S39ysQcy991+o24IvB2m85g2i04 7HuTve+g9ueFOtfdxI8W8f7YXxjMM7sNq/KWs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TM9ewyNVC7zUzXF5PqhW3Uv0LjjZ2WKk9uV2hty3eEs=; b=nw9HMMT/BYXB8vE5XrAnhT0OXukTam7Pio6m4fNxnyS3p6WKH7ypnsuFtQcteBXeVd /Vv5BSyw9CGMHmV7TVrO3IyBRqEtM26hM/M/rlDVizvjaTaaVuJ+0cD/+I8gP6Yoaqe0 rLziAKXi19fAPXV4avF8rC0p/20HfZ2PdXBVFA9cBBIhP/blVGqLr4cMpj+0uY1lbfTS k2wuk5Sri+p2ewPliRmxkPvb/+1gLtmddXRbKGpPRaV/d+fhf1NYAKyCRXyb2eoXO87P yu4m0AgzDgjS308/oBvHbtgGGLrjWlJanyzCdwXEnQsI1cDesOWoIObcDK5A4zIZnzEH sihw== X-Gm-Message-State: APzg51CvQu3snEg5j2nyBuxQyZj7upgsxmjli8s1PM+whol4JraJVXOR jP/6Z0d648BeHAQpr6xG2VBcDg== X-Google-Smtp-Source: ANB0VdYVmwAamlQtA3Hxf64uGRF5bFLEUPN28eWPhDDX4EhgLrf9qwsOYltw84kar9Xh8nIQ9H93BA== X-Received: by 2002:adf:ed88:: with SMTP id c8-v6mr34480220wro.264.1537463336579; Thu, 20 Sep 2018 10:08:56 -0700 (PDT) Received: from neo00-el73.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id v133-v6sm3458232wma.36.2018.09.20.10.08.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 10:08:55 -0700 (PDT) From: Devesh Sharma To: jgg@mellanox.com, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, NMoreyChaisemartin@suse.de, Devesh Sharma , JD Zheng , Jonathan Richardson Subject: [PATCH rdma-core 2/4] bnxt_re/lib: Remove db_lock around doorbell ring Date: Thu, 20 Sep 2018 13:08:32 -0400 Message-Id: <1537463314-7807-3-git-send-email-devesh.sharma@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1537463314-7807-1-git-send-email-devesh.sharma@broadcom.com> References: <1537463314-7807-1-git-send-email-devesh.sharma@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove the lock around the doorbell. It isn't necessary and was a bottleneck for multithreaded clients. Performance of spdk improves by 40% with it removed. Signed-off-by: JD Zheng Signed-off-by: Jonathan Richardson Signed-off-by: Devesh Sharma --- providers/bnxt_re/db.c | 2 -- providers/bnxt_re/main.c | 1 - providers/bnxt_re/main.h | 1 - providers/bnxt_re/verbs.c | 2 -- 4 files changed, 6 deletions(-) diff --git a/providers/bnxt_re/db.c b/providers/bnxt_re/db.c index a79f871..2896ceb 100644 --- a/providers/bnxt_re/db.c +++ b/providers/bnxt_re/db.c @@ -44,11 +44,9 @@ static void bnxt_re_ring_db(struct bnxt_re_dpi *dpi, { __le64 *dbval; - pthread_spin_lock(&dpi->db_lock); dbval = (__le64 *)&hdr->indx; udma_to_device_barrier(); iowrite64(dpi->dbpage, dbval); - pthread_spin_unlock(&dpi->db_lock); } static void bnxt_re_init_db_hdr(struct bnxt_re_db_hdr *hdr, uint32_t indx, diff --git a/providers/bnxt_re/main.c b/providers/bnxt_re/main.c index 54e3cc3..a1ba06a 100644 --- a/providers/bnxt_re/main.c +++ b/providers/bnxt_re/main.c @@ -167,7 +167,6 @@ static void bnxt_re_free_context(struct ibv_context *ibvctx) * allocated in this context. */ if (cntx->udpi.dbpage && cntx->udpi.dbpage != MAP_FAILED) { - pthread_spin_destroy(&cntx->udpi.db_lock); munmap(cntx->udpi.dbpage, dev->pg_size); cntx->udpi.dbpage = NULL; } diff --git a/providers/bnxt_re/main.h b/providers/bnxt_re/main.h index 16f55f9..0b5c749 100644 --- a/providers/bnxt_re/main.h +++ b/providers/bnxt_re/main.h @@ -59,7 +59,6 @@ struct bnxt_re_dpi { __u32 dpindx; __u64 *dbpage; - pthread_spinlock_t db_lock; }; struct bnxt_re_pd { diff --git a/providers/bnxt_re/verbs.c b/providers/bnxt_re/verbs.c index 9ce1454..8e9fd8c 100644 --- a/providers/bnxt_re/verbs.c +++ b/providers/bnxt_re/verbs.c @@ -110,8 +110,6 @@ struct ibv_pd *bnxt_re_alloc_pd(struct ibv_context *ibvctx) (void)ibv_cmd_dealloc_pd(&pd->ibvpd); goto out; } - pthread_spin_init(&cntx->udpi.db_lock, - PTHREAD_PROCESS_PRIVATE); } return &pd->ibvpd;