From patchwork Tue Mar 19 17:00:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 13596931 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0D765224D0; Tue, 19 Mar 2024 17:00:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710867627; cv=none; b=YU1UIUGtTeysQeCePyiIAsSCcwIemDn/KIh5vRSC51VrlametPUpPkh2jl8BXv5pjWjX0UZo+l1qgiYnp+95Nb8VjzWS0SVx61bEvbmULtbVmIvJSO6ij5E96GvdWqWlZQXwxs34yCGSpFWEyNRxsNKLO5ujOA1I7JLDZqmVKCY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710867627; c=relaxed/simple; bh=UBzp2+L5DZCxzkUt0BgNm18wsMT0W9aAtI+VywexiCs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=eW+LGppRqVcVT13os04qFQhC4N4vj6jbQw7dPwXd6iAJ+5stO8XvWqH9fZp+QBJQ71bN80O0fA6A205JqdzKBcvAzl0ZJlQY2b0OwUSyvj9iZ7NhRKTmlHCvVYsD8JXnRhexwjJEi4CVrZNTvHeomNLS/SyKbvekW1JocKd741Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=i5yvrjRO; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="i5yvrjRO" Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id 9D2F120B74C4; Tue, 19 Mar 2024 10:00:25 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9D2F120B74C4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1710867625; bh=KQpc5b+IMDR1rbzinuEbYLwQYGgKN1HCmxsKitzyTkA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i5yvrjROVnqBo1R2ENur/kTnwZ5bhwTIrcB3Yilmx70tqWflrD0+vj0Y21aIh/Rg+ 6fDaD0dfwZzvKpOtTX1U8i8O3SXvnSWz59GfIB9wqMI06gqihkdpurYOaeA6TXKWUm dUDMDJoBtz62Lrm/yC9MGK2oFfLa2Ji733YBW9j8= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v2 4/4] RDMA/mana_ib: Use struct mana_ib_queue for RAW QPs Date: Tue, 19 Mar 2024 10:00:13 -0700 Message-Id: <1710867613-4798-5-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1710867613-4798-1-git-send-email-kotaranov@linux.microsoft.com> References: <1710867613-4798-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Konstantin Taranov Use struct mana_ib_queue and its helpers for RAW QPs Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/mana_ib.h | 13 +++---- drivers/infiniband/hw/mana/qp.c | 54 ++++++++-------------------- 2 files changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index a8953ee80..5d9a6e7cb 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -91,15 +91,16 @@ struct mana_ib_cq { u32 comp_vector; }; +struct mana_ib_raw_sq { + /* Work queue info */ + struct mana_ib_queue queue; + mana_handle_t tx_object; +}; + struct mana_ib_qp { struct ib_qp ibqp; - /* Work queue info */ - struct ib_umem *sq_umem; - int sqe; - u64 sq_gdma_region; - u64 sq_id; - mana_handle_t tx_object; + struct mana_ib_raw_sq sq; /* The port on the IB device, starting with 1 */ u32 port; diff --git a/drivers/infiniband/hw/mana/qp.c b/drivers/infiniband/hw/mana/qp.c index f606caa75..5818b665e 100644 --- a/drivers/infiniband/hw/mana/qp.c +++ b/drivers/infiniband/hw/mana/qp.c @@ -297,7 +297,6 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd, struct mana_obj_spec cq_spec = {}; struct mana_port_context *mpc; struct net_device *ndev; - struct ib_umem *umem; struct mana_eq *eq; int eq_vec; u32 port; @@ -346,32 +345,15 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd, ibdev_dbg(&mdev->ib_dev, "ucmd sq_buf_addr 0x%llx port %u\n", ucmd.sq_buf_addr, ucmd.port); - umem = ib_umem_get(ibpd->device, ucmd.sq_buf_addr, ucmd.sq_buf_size, - IB_ACCESS_LOCAL_WRITE); - if (IS_ERR(umem)) { - err = PTR_ERR(umem); - ibdev_dbg(&mdev->ib_dev, - "Failed to get umem for create qp-raw, err %d\n", - err); - goto err_free_vport; - } - qp->sq_umem = umem; - - err = mana_ib_create_zero_offset_dma_region(mdev, qp->sq_umem, - &qp->sq_gdma_region); + err = mana_ib_create_queue(mdev, ucmd.sq_buf_addr, ucmd.sq_buf_size, &qp->sq.queue); if (err) { ibdev_dbg(&mdev->ib_dev, - "Failed to create dma region for create qp-raw, %d\n", - err); - goto err_release_umem; + "Failed to create queue for create qp-raw, err %d\n", err); + goto err_free_vport; } - ibdev_dbg(&mdev->ib_dev, - "create_dma_region ret %d gdma_region 0x%llx\n", - err, qp->sq_gdma_region); - /* Create a WQ on the same port handle used by the Ethernet */ - wq_spec.gdma_region = qp->sq_gdma_region; + wq_spec.gdma_region = qp->sq.queue.gdma_region; wq_spec.queue_size = ucmd.sq_buf_size; cq_spec.gdma_region = send_cq->queue.gdma_region; @@ -382,19 +364,19 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd, cq_spec.attached_eq = eq->eq->id; err = mana_create_wq_obj(mpc, mpc->port_handle, GDMA_SQ, &wq_spec, - &cq_spec, &qp->tx_object); + &cq_spec, &qp->sq.tx_object); if (err) { ibdev_dbg(&mdev->ib_dev, "Failed to create wq for create raw-qp, err %d\n", err); - goto err_destroy_dma_region; + goto err_destroy_queue; } /* The GDMA regions are now owned by the WQ object */ - qp->sq_gdma_region = GDMA_INVALID_DMA_REGION; + qp->sq.queue.gdma_region = GDMA_INVALID_DMA_REGION; send_cq->queue.gdma_region = GDMA_INVALID_DMA_REGION; - qp->sq_id = wq_spec.queue_index; + qp->sq.queue.id = wq_spec.queue_index; send_cq->queue.id = cq_spec.queue_index; /* Create CQ table entry */ @@ -404,9 +386,9 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd, ibdev_dbg(&mdev->ib_dev, "ret %d qp->tx_object 0x%llx sq id %llu cq id %llu\n", err, - qp->tx_object, qp->sq_id, send_cq->queue.id); + qp->sq.tx_object, qp->sq.queue.id, send_cq->queue.id); - resp.sqid = qp->sq_id; + resp.sqid = qp->sq.queue.id; resp.cqid = send_cq->queue.id; resp.tx_vp_offset = pd->tx_vp_offset; @@ -425,13 +407,10 @@ static int mana_ib_create_qp_raw(struct ib_qp *ibqp, struct ib_pd *ibpd, gc->cq_table[send_cq->queue.id] = NULL; err_destroy_wq_obj: - mana_destroy_wq_obj(mpc, GDMA_SQ, qp->tx_object); + mana_destroy_wq_obj(mpc, GDMA_SQ, qp->sq.tx_object); -err_destroy_dma_region: - mana_ib_gd_destroy_dma_region(mdev, qp->sq_gdma_region); - -err_release_umem: - ib_umem_release(umem); +err_destroy_queue: + mana_ib_destroy_queue(mdev, &qp->sq.queue); err_free_vport: mana_ib_uncfg_vport(mdev, pd, port); @@ -505,12 +484,9 @@ static int mana_ib_destroy_qp_raw(struct mana_ib_qp *qp, struct ib_udata *udata) mpc = netdev_priv(ndev); pd = container_of(ibpd, struct mana_ib_pd, ibpd); - mana_destroy_wq_obj(mpc, GDMA_SQ, qp->tx_object); + mana_destroy_wq_obj(mpc, GDMA_SQ, qp->sq.tx_object); - if (qp->sq_umem) { - mana_ib_gd_destroy_dma_region(mdev, qp->sq_gdma_region); - ib_umem_release(qp->sq_umem); - } + mana_ib_destroy_queue(mdev, &qp->sq.queue); mana_ib_uncfg_vport(mdev, pd, qp->port);