From patchwork Tue Jun 15 18:31:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devesh Sharma X-Patchwork-Id: 12322649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CABE8C48BE8 for ; Tue, 15 Jun 2021 18:32:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ADDF261185 for ; Tue, 15 Jun 2021 18:32:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231260AbhFOSeM (ORCPT ); Tue, 15 Jun 2021 14:34:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231667AbhFOSeL (ORCPT ); Tue, 15 Jun 2021 14:34:11 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44FF1C061574 for ; Tue, 15 Jun 2021 11:32:06 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id e1so8909084pld.13 for ; Tue, 15 Jun 2021 11:32:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version; bh=7bTY5xpEWMmzT80HcFQslEC986NYpK1Kqp4NGiKWwME=; b=EEGbZvpKMIUMjIO1O4G3O+eTYlvr8ILwawB7pErzHqIH/CV5ZAEBC4Z1gjrgdEJzmm ngNFhdzHsY+fvhSW+OO5A8Ugvl7uSYsMYEmq4JiZDOQplyQZO5wtCaG7YgijgS3sZvFw 3vOHhTbEEHUq/h4Tb7yUHEabMgXi56zsiK6BE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version; bh=7bTY5xpEWMmzT80HcFQslEC986NYpK1Kqp4NGiKWwME=; b=I61lb0nivHmVobhpo7C/jwGHcLveiUe2EUXcCqfr3dMDle5QYwJDYQ3T28M9zRFd9P TT8MkPOmciUJcv7KcpAvROmxxR6/PcShYc8/+THc5LGtIKYIMdbWZ+HlX0r3fcId6vK5 vnjnobiLwaHg1vyP2z2sQ+SC8OrWCROeM1lxJ04MNedPB9NUDyNb67Pgr43guRL2ppFe ANAD+KArkYUabt1la/t7FsaPbb0DY/wULR+rcyVSzAT4br5GhIovfU88XiE+kg5gZqgA pByPAnoHkVXdvmsF29QzwkKtAs58SfMzlL9Wwn6M+HmMGjP/hJOcaqtPtdlk4xzIpA7d CusQ== X-Gm-Message-State: AOAM533LAghL9RrI9ZxltpHTBSqtgAJVAHHev4oKBTutqlb4z7InwESo bJQ9pvoVCovSu449d1x9sBD4O20RkcFfTu6alqLGsEtUZ8W8lJAqhsRJuifUwxBlhPlvRZHBG8R kE2CCINclzZHO7tSsWhA9kSzCGDBqwtdwWuLLyoMrOwHFPxDmsLAy9iBKXR83qd+54wwD0PiqSl NuCUlChA== X-Google-Smtp-Source: ABdhPJzQluRJpJX19B1Ntw1dSXSDDuipDyRQvZMg5NkHk0akn4jFsW8dRrZj3THES6TywjfxXDGh2w== X-Received: by 2002:a17:902:f283:b029:10d:e7f3:725 with SMTP id k3-20020a170902f283b029010de7f30725mr3802501plc.45.1623781924830; Tue, 15 Jun 2021 11:32:04 -0700 (PDT) Received: from dev01.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id i125sm3803823pfc.7.2021.06.15.11.32.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Jun 2021 11:32:04 -0700 (PDT) From: Devesh Sharma To: linux-rdma@vger.kernel.org Cc: Devesh Sharma Subject: [PATCH V5 rdma-core 3/5] bnxt_re/lib: add a function to initialize software queue Date: Wed, 16 Jun 2021 00:01:46 +0530 Message-Id: <20210615183148.1179042-4-devesh.sharma@broadcom.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210615183148.1179042-1-devesh.sharma@broadcom.com> References: <20210615183148.1179042-1-devesh.sharma@broadcom.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Splitting the shadow software queue initialization into a separate function. Same is being called for both RQ and SQ during create QP. Signed-off-by: Devesh Sharma --- providers/bnxt_re/main.h | 2 ++ providers/bnxt_re/verbs.c | 65 ++++++++++++++++++++++++--------------- 2 files changed, 43 insertions(+), 24 deletions(-) diff --git a/providers/bnxt_re/main.h b/providers/bnxt_re/main.h index dc8166f2..71a449e3 100644 --- a/providers/bnxt_re/main.h +++ b/providers/bnxt_re/main.h @@ -96,6 +96,8 @@ struct bnxt_re_wrid { uint64_t wrid; uint32_t bytes; int next_idx; + uint32_t st_slot_idx; + uint8_t slots; uint8_t sig; }; diff --git a/providers/bnxt_re/verbs.c b/providers/bnxt_re/verbs.c index 11c01574..b07628bb 100644 --- a/providers/bnxt_re/verbs.c +++ b/providers/bnxt_re/verbs.c @@ -847,9 +847,27 @@ static void bnxt_re_free_queues(struct bnxt_re_qp *qp) bnxt_re_free_aligned(qp->jsqq->hwque); } +static int bnxt_re_alloc_init_swque(struct bnxt_re_joint_queue *jqq, int nwr) +{ + int indx; + + jqq->swque = calloc(nwr, sizeof(struct bnxt_re_wrid)); + if (!jqq->swque) + return -ENOMEM; + jqq->start_idx = 0; + jqq->last_idx = nwr - 1; + for (indx = 0; indx < nwr; indx++) + jqq->swque[indx].next_idx = indx + 1; + jqq->swque[jqq->last_idx].next_idx = 0; + jqq->last_idx = 0; + + return 0; +} + static int bnxt_re_alloc_queues(struct bnxt_re_qp *qp, struct ibv_qp_init_attr *attr, - uint32_t pg_size) { + uint32_t pg_size) +{ struct bnxt_re_psns_ext *psns_ext; struct bnxt_re_wrid *swque; struct bnxt_re_queue *que; @@ -857,22 +875,23 @@ static int bnxt_re_alloc_queues(struct bnxt_re_qp *qp, uint32_t psn_depth; uint32_t psn_size; int ret, indx; + uint32_t nswr; que = qp->jsqq->hwque; que->stride = bnxt_re_get_sqe_sz(); /* 8916 adjustment */ - que->depth = roundup_pow_of_two(attr->cap.max_send_wr + 1 + - BNXT_RE_FULL_FLAG_DELTA); - que->diff = que->depth - attr->cap.max_send_wr; + nswr = roundup_pow_of_two(attr->cap.max_send_wr + 1 + + BNXT_RE_FULL_FLAG_DELTA); + que->diff = nswr - attr->cap.max_send_wr; /* psn_depth extra entries of size que->stride */ psn_size = bnxt_re_is_chip_gen_p5(qp->cctx) ? sizeof(struct bnxt_re_psns_ext) : sizeof(struct bnxt_re_psns); - psn_depth = (que->depth * psn_size) / que->stride; - if ((que->depth * psn_size) % que->stride) + psn_depth = (nswr * psn_size) / que->stride; + if ((nswr * psn_size) % que->stride) psn_depth++; - que->depth += psn_depth; + que->depth = nswr + psn_depth; /* PSN-search memory is allocated without checking for * QP-Type. Kenrel driver do not map this memory if it * is UD-qp. UD-qp use this memory to maintain WC-opcode. @@ -884,44 +903,42 @@ static int bnxt_re_alloc_queues(struct bnxt_re_qp *qp, /* exclude psns depth*/ que->depth -= psn_depth; /* start of spsn space sizeof(struct bnxt_re_psns) each. */ - psns = (que->va + que->stride * que->depth); + psns = (que->va + que->stride * nswr); psns_ext = (struct bnxt_re_psns_ext *)psns; - swque = calloc(que->depth, sizeof(struct bnxt_re_wrid)); - if (!swque) { + + ret = bnxt_re_alloc_init_swque(qp->jsqq, nswr); + if (ret) { ret = -ENOMEM; goto fail; } - for (indx = 0 ; indx < que->depth; indx++, psns++) + swque = qp->jsqq->swque; + for (indx = 0; indx < nswr; indx++, psns++) swque[indx].psns = psns; if (bnxt_re_is_chip_gen_p5(qp->cctx)) { - for (indx = 0 ; indx < que->depth; indx++, psns_ext++) { + for (indx = 0; indx < nswr; indx++, psns_ext++) { swque[indx].psns_ext = psns_ext; swque[indx].psns = (struct bnxt_re_psns *)psns_ext; } } - qp->jsqq->swque = swque; - - qp->cap.max_swr = que->depth; + qp->cap.max_swr = nswr; pthread_spin_init(&que->qlock, PTHREAD_PROCESS_PRIVATE); if (qp->jrqq) { que = qp->jrqq->hwque; que->stride = bnxt_re_get_rqe_sz(); - que->depth = roundup_pow_of_two(attr->cap.max_recv_wr + 1); - que->diff = que->depth - attr->cap.max_recv_wr; + nswr = roundup_pow_of_two(attr->cap.max_recv_wr + 1); + que->depth = nswr; + que->diff = nswr - attr->cap.max_recv_wr; ret = bnxt_re_alloc_aligned(que, pg_size); if (ret) goto fail; - pthread_spin_init(&que->qlock, PTHREAD_PROCESS_PRIVATE); /* For RQ only bnxt_re_wri.wrid is used. */ - qp->jrqq->swque = calloc(que->depth, - sizeof(struct bnxt_re_wrid)); - if (!qp->jrqq->swque) { - ret = -ENOMEM; + ret = bnxt_re_alloc_init_swque(qp->jrqq, nswr); + if (ret) goto fail; - } - qp->cap.max_rwr = que->depth; + pthread_spin_init(&que->qlock, PTHREAD_PROCESS_PRIVATE); + qp->cap.max_rwr = nswr; } return 0;