From patchwork Wed Jun 16 20:25:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devesh Sharma X-Patchwork-Id: 12325867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFF4EC48BE6 for ; Wed, 16 Jun 2021 20:25:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C8AC0613C7 for ; Wed, 16 Jun 2021 20:25:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233454AbhFPU1w (ORCPT ); Wed, 16 Jun 2021 16:27:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233390AbhFPU1v (ORCPT ); Wed, 16 Jun 2021 16:27:51 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33AFFC061574 for ; Wed, 16 Jun 2021 13:25:45 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id x10so1728296plg.3 for ; Wed, 16 Jun 2021 13:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version; bh=kzJvwukIxRyzqc5C232Pbi6x2tnUOcBuLA1BpOoc/cw=; b=PhRQbTrK0FkfI92FCWOSjFmqBzixTwdtU7+xue92Od8RyU8ktuC+bBM/Y9uTC6O0Is 2KMaOJWvN392HLU1LTEq4w7k1OciuwvjTfKHbxWdK93FvDAklqDYOUt6maoP1gKKR9/Z FcWWFxdPs020J6nMlxnL0FDLdVZazMG2egoms= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version; bh=kzJvwukIxRyzqc5C232Pbi6x2tnUOcBuLA1BpOoc/cw=; b=DdTZWjHqcnqMcKeruSK20Wy+vzwH220cxMxtt4ETNGoBZTU8e7sFsK6x2nEKrukxQS elTT5CC5rAgvG9UDzfqyZSLN6SqbwJkKZTTyoi2S+PSgv/PpoLj2WaMxN8SPYf7oANv6 hzkI3QQAWT7GsuHHWotr/V/PaiIl76qeCGH0CyQZRzp/LH+WHKZa6ESw1ur3+cUvvsWa Ik7nA2hfoeiKEtvboaFe6p/rOcUZaDIFFS48xR2NxX4sdOSVQVqSBaaXDhLMm09UYcjL Mio02SZeZ/O8WDxg1vUR99T3if4ZYboHo7PQfP5j2d2Rjg6m/ArvhXcra6e3LALTDQ8J ajMg== X-Gm-Message-State: AOAM530Zc+l2jVO+dmpqfspnSjfKiAHSBt6NkU8w4QeexkzBgm8/vye5 37xpNGbU7I57S4D1lfUUu6D2GR+TiSboeF36sByaj3pIKgeTZAYrUWUDv6BXF8fneOZpUIEFlu9 DUzMRpgX8HUMV6NXiPDze35Vzf4WCUwtLA7vJ9L5TZunfkwbVvULCU6xbf/RbwHJa5uh6B+PzsU H8dm2sYQ== X-Google-Smtp-Source: ABdhPJwiE0Dyql93Lg9/zWmypB7DTsjDmEHd9QkwGl4ONUEqITuCiz7Is9l3lqULOPGUsiFxyc3wzA== X-Received: by 2002:a17:90b:3802:: with SMTP id mq2mr1614641pjb.79.1623875144195; Wed, 16 Jun 2021 13:25:44 -0700 (PDT) Received: from dev01.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id s24sm6236561pju.9.2021.06.16.13.25.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Jun 2021 13:25:43 -0700 (PDT) From: Devesh Sharma To: linux-rdma@vger.kernel.org Cc: Devesh Sharma Subject: [PATCH V6 rdma-core 3/5] bnxt_re/lib: add a function to initialize software queue Date: Thu, 17 Jun 2021 01:55:22 +0530 Message-Id: <20210616202524.1185195-4-devesh.sharma@broadcom.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210616202524.1185195-1-devesh.sharma@broadcom.com> References: <20210616202524.1185195-1-devesh.sharma@broadcom.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Splitting the shadow software queue initialization into a separate function. Same is being called for both RQ and SQ during create QP. Signed-off-by: Devesh Sharma --- providers/bnxt_re/main.h | 2 ++ providers/bnxt_re/verbs.c | 65 ++++++++++++++++++++++++--------------- 2 files changed, 43 insertions(+), 24 deletions(-) diff --git a/providers/bnxt_re/main.h b/providers/bnxt_re/main.h index dc8166f2..71a449e3 100644 --- a/providers/bnxt_re/main.h +++ b/providers/bnxt_re/main.h @@ -96,6 +96,8 @@ struct bnxt_re_wrid { uint64_t wrid; uint32_t bytes; int next_idx; + uint32_t st_slot_idx; + uint8_t slots; uint8_t sig; }; diff --git a/providers/bnxt_re/verbs.c b/providers/bnxt_re/verbs.c index 51216129..81bf09c1 100644 --- a/providers/bnxt_re/verbs.c +++ b/providers/bnxt_re/verbs.c @@ -847,9 +847,27 @@ static void bnxt_re_free_queues(struct bnxt_re_qp *qp) bnxt_re_free_aligned(qp->jsqq->hwque); } +static int bnxt_re_alloc_init_swque(struct bnxt_re_joint_queue *jqq, int nwr) +{ + int indx; + + jqq->swque = calloc(nwr, sizeof(struct bnxt_re_wrid)); + if (!jqq->swque) + return -ENOMEM; + jqq->start_idx = 0; + jqq->last_idx = nwr - 1; + for (indx = 0; indx < nwr; indx++) + jqq->swque[indx].next_idx = indx + 1; + jqq->swque[jqq->last_idx].next_idx = 0; + jqq->last_idx = 0; + + return 0; +} + static int bnxt_re_alloc_queues(struct bnxt_re_qp *qp, struct ibv_qp_init_attr *attr, - uint32_t pg_size) { + uint32_t pg_size) +{ struct bnxt_re_psns_ext *psns_ext; struct bnxt_re_wrid *swque; struct bnxt_re_queue *que; @@ -857,22 +875,23 @@ static int bnxt_re_alloc_queues(struct bnxt_re_qp *qp, uint32_t psn_depth; uint32_t psn_size; int ret, indx; + uint32_t nswr; que = qp->jsqq->hwque; que->stride = bnxt_re_get_sqe_sz(); /* 8916 adjustment */ - que->depth = roundup_pow_of_two(attr->cap.max_send_wr + 1 + - BNXT_RE_FULL_FLAG_DELTA); - que->diff = que->depth - attr->cap.max_send_wr; + nswr = roundup_pow_of_two(attr->cap.max_send_wr + 1 + + BNXT_RE_FULL_FLAG_DELTA); + que->diff = nswr - attr->cap.max_send_wr; /* psn_depth extra entries of size que->stride */ psn_size = bnxt_re_is_chip_gen_p5(qp->cctx) ? sizeof(struct bnxt_re_psns_ext) : sizeof(struct bnxt_re_psns); - psn_depth = (que->depth * psn_size) / que->stride; - if ((que->depth * psn_size) % que->stride) + psn_depth = (nswr * psn_size) / que->stride; + if ((nswr * psn_size) % que->stride) psn_depth++; - que->depth += psn_depth; + que->depth = nswr + psn_depth; /* PSN-search memory is allocated without checking for * QP-Type. Kenrel driver do not map this memory if it * is UD-qp. UD-qp use this memory to maintain WC-opcode. @@ -884,44 +903,42 @@ static int bnxt_re_alloc_queues(struct bnxt_re_qp *qp, /* exclude psns depth*/ que->depth -= psn_depth; /* start of spsn space sizeof(struct bnxt_re_psns) each. */ - psns = (que->va + que->stride * que->depth); + psns = (que->va + que->stride * nswr); psns_ext = (struct bnxt_re_psns_ext *)psns; - swque = calloc(que->depth, sizeof(struct bnxt_re_wrid)); - if (!swque) { + + ret = bnxt_re_alloc_init_swque(qp->jsqq, nswr); + if (ret) { ret = -ENOMEM; goto fail; } - for (indx = 0 ; indx < que->depth; indx++, psns++) + swque = qp->jsqq->swque; + for (indx = 0; indx < nswr; indx++, psns++) swque[indx].psns = psns; if (bnxt_re_is_chip_gen_p5(qp->cctx)) { - for (indx = 0 ; indx < que->depth; indx++, psns_ext++) { + for (indx = 0; indx < nswr; indx++, psns_ext++) { swque[indx].psns_ext = psns_ext; swque[indx].psns = (struct bnxt_re_psns *)psns_ext; } } - qp->jsqq->swque = swque; - - qp->cap.max_swr = que->depth; + qp->cap.max_swr = nswr; pthread_spin_init(&que->qlock, PTHREAD_PROCESS_PRIVATE); if (qp->jrqq) { que = qp->jrqq->hwque; que->stride = bnxt_re_get_rqe_sz(); - que->depth = roundup_pow_of_two(attr->cap.max_recv_wr + 1); - que->diff = que->depth - attr->cap.max_recv_wr; + nswr = roundup_pow_of_two(attr->cap.max_recv_wr + 1); + que->depth = nswr; + que->diff = nswr - attr->cap.max_recv_wr; ret = bnxt_re_alloc_aligned(que, pg_size); if (ret) goto fail; - pthread_spin_init(&que->qlock, PTHREAD_PROCESS_PRIVATE); /* For RQ only bnxt_re_wri.wrid is used. */ - qp->jrqq->swque = calloc(que->depth, - sizeof(struct bnxt_re_wrid)); - if (!qp->jrqq->swque) { - ret = -ENOMEM; + ret = bnxt_re_alloc_init_swque(qp->jrqq, nswr); + if (ret) goto fail; - } - qp->cap.max_rwr = que->depth; + pthread_spin_init(&que->qlock, PTHREAD_PROCESS_PRIVATE); + qp->cap.max_rwr = nswr; } return 0;