From patchwork Wed May 10 10:45:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 9719665 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D023F60236 for ; Wed, 10 May 2017 10:46:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C407328576 for ; Wed, 10 May 2017 10:46:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B7A062857F; Wed, 10 May 2017 10:46:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5169A28576 for ; Wed, 10 May 2017 10:46:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752285AbdEJKqY (ORCPT ); Wed, 10 May 2017 06:46:24 -0400 Received: from mail-qt0-f171.google.com ([209.85.216.171]:33462 "EHLO mail-qt0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752606AbdEJKqX (ORCPT ); Wed, 10 May 2017 06:46:23 -0400 Received: by mail-qt0-f171.google.com with SMTP id t26so23927283qtg.0 for ; Wed, 10 May 2017 03:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/5VhSKE3R3xRGoEI6IM4RAWl2BwKxHPvpkA0Hw9Ss6Y=; b=cleumJ00PlrjsrMoGYxv2UrcDUgZwKvMYQSJ5l/faAbPQDejwXFUz+o/Kye3ZqaaOE KACJ3Q+bN1x7iTGLyMh/f6MTWHacdsVNYwg+M13ocRrMmW4NnOjj4EpkKWoklioVakZ+ kRjttt5BLAubER31uB0G8iiEsq5tYBVFjJlEQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/5VhSKE3R3xRGoEI6IM4RAWl2BwKxHPvpkA0Hw9Ss6Y=; b=hG0rk1jSdejz8Hf/Cvebb6cM+XTjOnN87gNiFFmGXjW/UpZ92RNAiEeuGh79zvU58W dzFozX/Sjc1BEiA44K60RxEP1S/m2p0t2JumAwyiKL7nlNyxPld6nmm+dbmCsildd6IS BEO/YLkrK4Z/D6JA2p5kmOQjt3/j5Rxlpeb6eVW/hvbvJWI5WTB/hPElZQcc5YmODtCa vmxwVavjOqcUPzkWRvV5QyyE+p/VkQa3wY3YetuwyEgkVeS+e19QOyiRF6ZNLr0oTD9/ efnOL/sDHQqMCrC/2AtS5qvlEM5fOkf7kymFn1M4nlNQC3f9gvI8yDOeX5Fs2B5tpBi+ UZbQ== X-Gm-Message-State: AODbwcABcIWL3BMWhfv9Q6KBaqenGaYluzwE8MB2ntQklYKIpssfkbKc +xfeEhqx8MfZzDPq X-Received: by 10.200.3.134 with SMTP id t6mr5195520qtg.238.1494413182872; Wed, 10 May 2017 03:46:22 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net ([192.19.239.250]) by smtp.gmail.com with ESMTPSA id k10sm1868531qke.3.2017.05.10.03.46.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 May 2017 03:46:22 -0700 (PDT) From: Selvin Xavier To: dledford@redhat.com, linux-rdma@vger.kernel.org Cc: Devesh Sharma , Kalesh AP , Selvin Xavier Subject: [PATCH for-next 10/14] RDMA/bnxt_re: Fix RQE posting logic Date: Wed, 10 May 2017 03:45:35 -0700 Message-Id: <1494413139-11883-11-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1494413139-11883-1-git-send-email-selvin.xavier@broadcom.com> References: <1494413139-11883-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Devesh Sharma This patch adds code to ring RQ Doorbell aggressively so that the adapter can DMA RQ buffers sooner, instead of DMA all WQEs in the post_recv WR list together at the end of the post_recv verb. Also use spinlock to serialize RQ posting Signed-off-by: Kalesh AP Signed-off-by: Devesh Sharma Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 ++ drivers/infiniband/hw/bnxt_re/ib_verbs.c | 18 +++++++++++++++++- drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 277c2da..12950ec 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -58,6 +58,8 @@ #define BNXT_RE_UD_QP_HW_STALL 0x400000 +#define BNXT_RE_RQ_WQE_THRESHOLD 32 + struct bnxt_re_work { struct work_struct work; unsigned long event; diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 525f4b0..5a8b17e 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -1225,6 +1225,7 @@ struct ib_qp *bnxt_re_create_qp(struct ib_pd *ib_pd, qp->ib_qp.qp_num = qp->qplib_qp.id; spin_lock_init(&qp->sq_lock); + spin_lock_init(&qp->rq_lock); if (udata) { struct bnxt_re_qp_resp resp; @@ -2256,7 +2257,10 @@ int bnxt_re_post_recv(struct ib_qp *ib_qp, struct ib_recv_wr *wr, struct bnxt_re_qp *qp = container_of(ib_qp, struct bnxt_re_qp, ib_qp); struct bnxt_qplib_swqe wqe; int rc = 0, payload_sz = 0; + unsigned long flags; + u32 count = 0; + spin_lock_irqsave(&qp->rq_lock, flags); while (wr) { /* House keeping */ memset(&wqe, 0, sizeof(wqe)); @@ -2285,9 +2289,21 @@ int bnxt_re_post_recv(struct ib_qp *ib_qp, struct ib_recv_wr *wr, *bad_wr = wr; break; } + + /* Ring DB if the RQEs posted reaches a threshold value */ + if (++count >= BNXT_RE_RQ_WQE_THRESHOLD) { + bnxt_qplib_post_recv_db(&qp->qplib_qp); + count = 0; + } + wr = wr->next; } - bnxt_qplib_post_recv_db(&qp->qplib_qp); + + if (count) + bnxt_qplib_post_recv_db(&qp->qplib_qp); + + spin_unlock_irqrestore(&qp->rq_lock, flags); + return rc; } diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h index 381e4e9..93539e3 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h @@ -72,6 +72,7 @@ struct bnxt_re_qp { struct bnxt_re_dev *rdev; struct ib_qp ib_qp; spinlock_t sq_lock; /* protect sq */ + spinlock_t rq_lock; /* protect rq */ struct bnxt_qplib_qp qplib_qp; struct ib_umem *sumem; struct ib_umem *rumem;