From patchwork Wed May 17 09:19:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 9730511 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5F4C5602DB for ; Wed, 17 May 2017 09:21:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B09326E39 for ; Wed, 17 May 2017 09:21:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FD19286D2; Wed, 17 May 2017 09:21:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E262A26E39 for ; Wed, 17 May 2017 09:21:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753728AbdEQJVT (ORCPT ); Wed, 17 May 2017 05:21:19 -0400 Received: from mail-qt0-f177.google.com ([209.85.216.177]:34897 "EHLO mail-qt0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753763AbdEQJVS (ORCPT ); Wed, 17 May 2017 05:21:18 -0400 Received: by mail-qt0-f177.google.com with SMTP id v27so4525886qtg.2 for ; Wed, 17 May 2017 02:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lUhOoeEqNN+XmCxAWQ1sK/bOrLsSVTXxFQON7uB4f+Y=; b=hHTWybmBjnUfI46dI2lxfxb0gSxPsStm5JqXy0N7FU1ruU9h6NLY94hTmCWfRJjpwx elwIdc0cv7ayUwp/NXuhzCIM/ZOxLz3f2LOpXEYMxQGFpOfCTG4iRV5Da4vaBGf7imwB z+BuBFSkIOWZYPv/dDVlZZHKxNygR1YM11A6w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lUhOoeEqNN+XmCxAWQ1sK/bOrLsSVTXxFQON7uB4f+Y=; b=C3TuAExULqU0IpQg8d3Pr6hY/Un2MaePpnkqyV2ALkEB22P138J2rkwbESfzRiJ0+o LXQNmf4ifJIQyyyuqOtIzKMghLdenYvgMKDa9nCAkrPM97PTZdKrNe92BghYnD9artkQ CvNn/AXO79hOuianJ4gptfr5sLKgtaUnKGDe+VPBr6nHlpS8uWTKN1UHrtX91ji8VbNZ jgu/nXAkdOPMr1aaDaW9lHN7wo6XDteezq0Qr27g+A0GtKLnCZRwUoLShiYJ+7U5XqYZ H5xmWiBKEHUsK0uk2Kjmj7ssmHIF+bLf3t4j5bD+baWPF7f/qH9i9belTdWVGzaSacns ZUMA== X-Gm-Message-State: AODbwcCEIU+tR7K05J+xVcVF3EFTJSStTQN6hwO0MSaZGrOx4ZZamrm+ XDA8RzLJSKVd5Aor X-Received: by 10.237.59.168 with SMTP id r37mr2189359qte.246.1495012877163; Wed, 17 May 2017 02:21:17 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net ([192.19.239.250]) by smtp.gmail.com with ESMTPSA id t136sm998431qke.40.2017.05.17.02.21.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 May 2017 02:21:16 -0700 (PDT) From: Selvin Xavier To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Devesh Sharma , Kalesh AP , Selvin Xavier Subject: [PATCH V2 for-next 10/15] RDMA/bnxt_re: Fix RQE posting logic Date: Wed, 17 May 2017 02:19:46 -0700 Message-Id: <1495012791-5053-11-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1495012791-5053-1-git-send-email-selvin.xavier@broadcom.com> References: <1495012791-5053-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Devesh Sharma This patch adds code to ring RQ Doorbell aggressively so that the adapter can DMA RQ buffers sooner, instead of DMA all WQEs in the post_recv WR list together at the end of the post_recv verb. Also use spinlock to serialize RQ posting Signed-off-by: Kalesh AP Signed-off-by: Devesh Sharma Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 ++ drivers/infiniband/hw/bnxt_re/ib_verbs.c | 18 +++++++++++++++++- drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 277c2da..12950ec 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -58,6 +58,8 @@ #define BNXT_RE_UD_QP_HW_STALL 0x400000 +#define BNXT_RE_RQ_WQE_THRESHOLD 32 + struct bnxt_re_work { struct work_struct work; unsigned long event; diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 9848ab9..f967255 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -1224,6 +1224,7 @@ struct ib_qp *bnxt_re_create_qp(struct ib_pd *ib_pd, qp->ib_qp.qp_num = qp->qplib_qp.id; spin_lock_init(&qp->sq_lock); + spin_lock_init(&qp->rq_lock); if (udata) { struct bnxt_re_qp_resp resp; @@ -2255,7 +2256,10 @@ int bnxt_re_post_recv(struct ib_qp *ib_qp, struct ib_recv_wr *wr, struct bnxt_re_qp *qp = container_of(ib_qp, struct bnxt_re_qp, ib_qp); struct bnxt_qplib_swqe wqe; int rc = 0, payload_sz = 0; + unsigned long flags; + u32 count = 0; + spin_lock_irqsave(&qp->rq_lock, flags); while (wr) { /* House keeping */ memset(&wqe, 0, sizeof(wqe)); @@ -2284,9 +2288,21 @@ int bnxt_re_post_recv(struct ib_qp *ib_qp, struct ib_recv_wr *wr, *bad_wr = wr; break; } + + /* Ring DB if the RQEs posted reaches a threshold value */ + if (++count >= BNXT_RE_RQ_WQE_THRESHOLD) { + bnxt_qplib_post_recv_db(&qp->qplib_qp); + count = 0; + } + wr = wr->next; } - bnxt_qplib_post_recv_db(&qp->qplib_qp); + + if (count) + bnxt_qplib_post_recv_db(&qp->qplib_qp); + + spin_unlock_irqrestore(&qp->rq_lock, flags); + return rc; } diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h index 381e4e9..93539e3 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h @@ -72,6 +72,7 @@ struct bnxt_re_qp { struct bnxt_re_dev *rdev; struct ib_qp ib_qp; spinlock_t sq_lock; /* protect sq */ + spinlock_t rq_lock; /* protect rq */ struct bnxt_qplib_qp qplib_qp; struct ib_umem *sumem; struct ib_umem *rumem;