From patchwork Tue Mar 20 21:18:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10297949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9B50D600F6 for ; Tue, 20 Mar 2018 21:19:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B48D2969A for ; Tue, 20 Mar 2018 21:19:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7FF902969F; Tue, 20 Mar 2018 21:19:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCE2E2969A for ; Tue, 20 Mar 2018 21:19:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751464AbeCTVTJ (ORCPT ); Tue, 20 Mar 2018 17:19:09 -0400 Received: from mail-wm0-f67.google.com ([74.125.82.67]:40658 "EHLO mail-wm0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751411AbeCTVTE (ORCPT ); Tue, 20 Mar 2018 17:19:04 -0400 Received: by mail-wm0-f67.google.com with SMTP id t6so6215062wmt.5 for ; Tue, 20 Mar 2018 14:19:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9JcTnLUSQOqYZSbcxZCGZw8YV4vVEY6JyBoK56GczBE=; b=EWMcBdkpLjU4az+uI1NLJpM5rTQyIfnBJdvJSfYy6Hxd20KvPS80WHYwrj48bvOdNU P5NEo4KyHqiLa5N7MQcXwKp3tUIR27Bl6SvNUmpKVLGdvVnzVOpewH1OeKLfCUsl8jG4 mfmb6AtA0IDBQDG+RbnbqwVpnXVWyZ9bfhRJexiRw6wGrXnPTifT6Z2bX8Y7y64CG0TC Q0sPdY+KUSqjQ3O+GcYWLh7xkDd+9IWcwj+SgKlGGPwoBJ7THT0MX98lqYndeq+ga/ef l5bM91yBGhetv+mSmquveA2A5Gww7xzprUOwB90nqO4EXy/D9phb0bgEOAiKzQ2hLimF b+nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9JcTnLUSQOqYZSbcxZCGZw8YV4vVEY6JyBoK56GczBE=; b=k95t2MN/mM6eD/nhCfA8SSrtPY1ZZzT1BrN3THd6U8eTNynGAF7+4mu1y2wlLTlIdW bah9uTIab8gBqmCNI1BLD4yiD5emPZaOBub54hHWkSR7+T3mxceLkMvyGBLJqTGQ4X6U +/E0sJePYDz53hXaKLaZ4Iour9mSrZjPeAaQn8XN6KR8yPGXEV3ayWrfxUAMgx63Q84K yV3Dd8yKc8QUXOBlSJSwmyovKfx2bjAqb3oqioczaWGmhSV3kKaCOz+OsxE2M6p0h01D S71PTpkW1hFlNFZDD2VLt5snYJ9euJ9/ubNS6CZmhWTI1d8S/5xrr1DA25vw0Da/T4Ao rjqw== X-Gm-Message-State: AElRT7FFiUNK/YmnWyRoSR2M1L5pGU0IUZ7mRqQhKRKF9fvXT5yGhumf i5FVYWvGsfXnlOQtQp/pfBEllnhIg4Y= X-Google-Smtp-Source: AG47ELtNKuLOGFLv0yw8BPL4snKgyGlzyxG2RTF2KJMK+rXBQs3Xr3pQKpGzhD2B8cxt+Nk61wV7ew== X-Received: by 10.28.230.68 with SMTP id d65mr909050wmh.13.1521580743127; Tue, 20 Mar 2018 14:19:03 -0700 (PDT) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id p21sm3385506wmc.0.2018.03.20.14.18.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Mar 2018 14:18:59 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.86_2) (envelope-from ) id 1eyOen-0002eW-2E; Tue, 20 Mar 2018 15:18:57 -0600 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Yishai Hadas , Matan Barak Subject: [PATCH rdma-core 04/20] mlx4: Move mlx4 to exclusively use the kernel uapi headers Date: Tue, 20 Mar 2018 15:18:34 -0600 Message-Id: <20180320211850.10090-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180320211850.10090-1-jgg@ziepe.ca> References: <20180320211850.10090-1-jgg@ziepe.ca> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Using the driver support macros we can now directly create the required driver structs automatically. A few name changes are required to make this work consistently with the macros. Signed-off-by: Jason Gunthorpe --- kernel-headers/CMakeLists.txt | 1 + providers/mlx4/mlx4-abi.h | 149 ++++++++---------------------------------- providers/mlx4/mlx4.c | 2 +- providers/mlx4/srq.c | 2 +- providers/mlx4/verbs.c | 26 ++++---- 5 files changed, 44 insertions(+), 136 deletions(-) diff --git a/kernel-headers/CMakeLists.txt b/kernel-headers/CMakeLists.txt index 365b4c57cd7095..9787c62794cc06 100644 --- a/kernel-headers/CMakeLists.txt +++ b/kernel-headers/CMakeLists.txt @@ -56,6 +56,7 @@ endfunction() # Transform the kernel ABIs used by the providers rdma_kernel_provider_abi( rdma/ib_user_verbs.h + rdma/mlx4-abi.h rdma/qedr-abi.h rdma/rdma_user_rxe.h ) diff --git a/providers/mlx4/mlx4-abi.h b/providers/mlx4/mlx4-abi.h index 33453abb43b082..e1d8327e7dde4a 100644 --- a/providers/mlx4/mlx4-abi.h +++ b/providers/mlx4/mlx4-abi.h @@ -35,131 +35,40 @@ #include #include +#include #define MLX4_UVERBS_MIN_ABI_VERSION 2 #define MLX4_UVERBS_MAX_ABI_VERSION 4 #define MLX4_UVERBS_NO_DEV_CAPS_ABI_VERSION 3 -struct mlx4_alloc_ucontext_resp_v3 { - struct ib_uverbs_get_context_resp ibv_resp; - __u32 qp_tab_size; - __u16 bf_reg_size; - __u16 bf_regs_per_page; -}; - -struct mlx4_alloc_ucontext_resp { - struct ib_uverbs_get_context_resp ibv_resp; - __u32 dev_caps; - __u32 qp_tab_size; - __u16 bf_reg_size; - __u16 bf_regs_per_page; - __u32 cqe_size; -}; - -struct mlx4_alloc_pd_resp { - struct ib_uverbs_alloc_pd_resp ibv_resp; - __u32 pdn; - __u32 reserved; -}; - -struct mlx4_create_cq { - struct ibv_create_cq ibv_cmd; - __u64 buf_addr; - __u64 db_addr; -}; - -struct mlx4_create_cq_resp { - struct ib_uverbs_create_cq_resp ibv_resp; - __u32 cqn; - __u32 reserved; -}; - -struct mlx4_create_cq_ex { - struct ibv_create_cq_ex ibv_cmd; - __u64 buf_addr; - __u64 db_addr; -}; - -struct mlx4_create_cq_resp_ex { - struct ib_uverbs_ex_create_cq_resp ibv_resp; - __u32 cqn; - __u32 reserved; -}; - -struct mlx4_resize_cq { - struct ibv_resize_cq ibv_cmd; - __u64 buf_addr; -}; - -struct mlx4_query_device_ex_resp { - struct ib_uverbs_ex_query_device_resp ibv_resp; - __u32 comp_mask; - __u32 response_length; - __u64 hca_core_clock_offset; - __u32 max_inl_recv_sz; - /* Explicitly align the response to u64 */ - __u32 reserved; - struct mlx4_ib_rss_caps rss_caps; /* vendor data channel */ - struct mlx4_ib_tso_caps tso_caps; -}; - -struct mlx4_query_device_ex { - struct ibv_query_device_ex ibv_cmd; -}; - -struct mlx4_create_srq { - struct ibv_create_srq ibv_cmd; - __u64 buf_addr; - __u64 db_addr; -}; - -struct mlx4_create_xsrq { - struct ibv_create_xsrq ibv_cmd; - __u64 buf_addr; - __u64 db_addr; -}; - -struct mlx4_create_srq_resp { - struct ib_uverbs_create_srq_resp ibv_resp; - __u32 srqn; - __u32 reserved; -}; - -struct mlx4_create_qp { - struct ibv_create_qp ibv_cmd; - __u64 buf_addr; - __u64 db_addr; - __u8 log_sq_bb_count; - __u8 log_sq_stride; - __u8 sq_no_prefetch; /* was reserved in ABI 2 */ - __u8 reserved; - __u32 inl_recv_sz; -}; - -struct mlx4_create_qp_ex_rss { - struct ibv_create_qp_ex ibv_cmd; - struct mlx4_ib_create_qp_rss drv_ex; -}; - -struct mlx4_create_qp_ex { - struct ibv_create_qp_ex ibv_cmd; - struct mlx4_ib_create_qp drv_ex; -}; - -struct mlx4_create_qp_resp_ex { - struct ib_uverbs_ex_create_qp_resp ibv_resp; -}; - -struct mlx4_create_wq { - struct ibv_create_wq ibv_cmd; - struct mlx4_ib_create_wq drv; -}; - -struct mlx4_modify_wq { - struct ibv_modify_wq ibv_cmd; - __u32 comp_mask; - __u32 reserved; -}; +DECLARE_DRV_CMD(mlx4_alloc_pd, IB_USER_VERBS_CMD_ALLOC_PD, + empty, mlx4_ib_alloc_pd_resp); +DECLARE_DRV_CMD(mlx4_create_cq, IB_USER_VERBS_CMD_CREATE_CQ, + mlx4_ib_create_cq, mlx4_ib_create_cq_resp); +DECLARE_DRV_CMD(mlx4_create_cq_ex, IB_USER_VERBS_EX_CMD_CREATE_CQ, + mlx4_ib_create_cq, mlx4_ib_create_cq_resp); +DECLARE_DRV_CMD(mlx4_create_qp, IB_USER_VERBS_CMD_CREATE_QP, + mlx4_ib_create_qp, empty); +DECLARE_DRV_CMD(mlx4_create_qp_ex, IB_USER_VERBS_EX_CMD_CREATE_QP, + mlx4_ib_create_qp, empty); +DECLARE_DRV_CMD(mlx4_create_qp_ex_rss, IB_USER_VERBS_EX_CMD_CREATE_QP, + mlx4_ib_create_qp_rss, empty); +DECLARE_DRV_CMD(mlx4_create_srq, IB_USER_VERBS_CMD_CREATE_SRQ, + mlx4_ib_create_srq, mlx4_ib_create_srq_resp); +DECLARE_DRV_CMD(mlx4_create_wq, IB_USER_VERBS_EX_CMD_CREATE_WQ, + mlx4_ib_create_wq, empty); +DECLARE_DRV_CMD(mlx4_create_xsrq, IB_USER_VERBS_CMD_CREATE_XSRQ, + mlx4_ib_create_srq, mlx4_ib_create_srq_resp); +DECLARE_DRV_CMD(mlx4_alloc_ucontext_v3, IB_USER_VERBS_CMD_GET_CONTEXT, + empty, mlx4_ib_alloc_ucontext_resp_v3); +DECLARE_DRV_CMD(mlx4_alloc_ucontext, IB_USER_VERBS_CMD_GET_CONTEXT, + empty, mlx4_ib_alloc_ucontext_resp); +DECLARE_DRV_CMD(mlx4_modify_wq, IB_USER_VERBS_EX_CMD_MODIFY_WQ, + mlx4_ib_modify_wq, empty); +DECLARE_DRV_CMD(mlx4_query_device_ex, IB_USER_VERBS_EX_CMD_QUERY_DEVICE, + empty, mlx4_uverbs_ex_query_device_resp); +DECLARE_DRV_CMD(mlx4_resize_cq, IB_USER_VERBS_CMD_RESIZE_CQ, + mlx4_ib_resize_cq, empty); #endif /* MLX4_ABI_H */ diff --git a/providers/mlx4/mlx4.c b/providers/mlx4/mlx4.c index 9652762099699f..43e730f833df41 100644 --- a/providers/mlx4/mlx4.c +++ b/providers/mlx4/mlx4.c @@ -171,7 +171,7 @@ static struct verbs_context *mlx4_alloc_context(struct ibv_device *ibdev, struct ibv_get_context cmd; struct mlx4_alloc_ucontext_resp resp; int i; - struct mlx4_alloc_ucontext_resp_v3 resp_v3; + struct mlx4_alloc_ucontext_v3_resp resp_v3; __u16 bf_reg_size; struct mlx4_device *dev = to_mdev(ibdev); struct verbs_context *verbs_ctx; diff --git a/providers/mlx4/srq.c b/providers/mlx4/srq.c index f9cb0dd4f3df52..00d9a70d9edb61 100644 --- a/providers/mlx4/srq.c +++ b/providers/mlx4/srq.c @@ -234,7 +234,7 @@ struct ibv_srq *mlx4_create_xrc_srq(struct ibv_context *context, struct ibv_srq_init_attr_ex *attr_ex) { struct mlx4_create_xsrq cmd; - struct mlx4_create_srq_resp resp; + struct mlx4_create_xsrq_resp resp; struct mlx4_srq *srq; int ret; diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c index 034f7c4381e9b1..52ec553aed283e 100644 --- a/providers/mlx4/verbs.c +++ b/providers/mlx4/verbs.c @@ -439,7 +439,7 @@ static int mlx4_cmd_create_cq_ex(struct ibv_context *context, struct mlx4_cq *cq) { struct mlx4_create_cq_ex cmd; - struct mlx4_create_cq_resp_ex resp; + struct mlx4_create_cq_ex_resp resp; int ret; cmd.buf_addr = (uintptr_t) cq->buf.buf; @@ -763,21 +763,21 @@ static int mlx4_cmd_create_qp_ex_rss(struct ibv_context *context, struct mlx4_qp *qp) { struct mlx4_create_qp_ex_rss cmd_ex = {}; - struct mlx4_create_qp_resp_ex resp; + struct mlx4_create_qp_ex_resp resp; int ret; if (attr->rx_hash_conf.rx_hash_key_len != - sizeof(cmd_ex.drv_ex.rx_hash_key)) { + sizeof(cmd_ex.rx_hash_key)) { errno = ENOTSUP; return errno; } - cmd_ex.drv_ex.rx_hash_fields_mask = + cmd_ex.rx_hash_fields_mask = attr->rx_hash_conf.rx_hash_fields_mask; - cmd_ex.drv_ex.rx_hash_function = + cmd_ex.rx_hash_function = attr->rx_hash_conf.rx_hash_function; - memcpy(cmd_ex.drv_ex.rx_hash_key, attr->rx_hash_conf.rx_hash_key, - sizeof(cmd_ex.drv_ex.rx_hash_key)); + memcpy(cmd_ex.rx_hash_key, attr->rx_hash_conf.rx_hash_key, + sizeof(cmd_ex.rx_hash_key)); ret = ibv_cmd_create_qp_ex2(context, &qp->verbs_qp, sizeof(qp->verbs_qp), attr, @@ -827,15 +827,13 @@ static int mlx4_cmd_create_qp_ex(struct ibv_context *context, struct mlx4_qp *qp) { struct mlx4_create_qp_ex cmd_ex; - struct mlx4_create_qp_resp_ex resp; + struct mlx4_create_qp_ex_resp resp; int ret; memset(&cmd_ex, 0, sizeof(cmd_ex)); *ibv_create_cq_ex_to_reg(&cmd_ex.ibv_cmd) = cmd->ibv_cmd.core_payload; - memcpy(&cmd_ex.drv_ex, &cmd->buf_addr, - offsetof(typeof(*cmd), sq_no_prefetch) + - sizeof(cmd->sq_no_prefetch) - sizeof(cmd->ibv_cmd)); + cmd_ex.drv_payload = cmd->drv_payload; ret = ibv_cmd_create_qp_ex2(context, &qp->verbs_qp, sizeof(qp->verbs_qp), attr, @@ -1457,11 +1455,11 @@ struct ibv_wq *mlx4_create_wq(struct ibv_context *context, goto err_free; *qp->db = 0; - cmd.drv.db_addr = (uintptr_t)qp->db; + cmd.db_addr = (uintptr_t)qp->db; - cmd.drv.buf_addr = (uintptr_t)qp->buf.buf; + cmd.buf_addr = (uintptr_t)qp->buf.buf; - cmd.drv.log_range_size = ctx->log_wqs_range_sz; + cmd.log_range_size = ctx->log_wqs_range_sz; pthread_mutex_lock(&to_mctx(context)->qp_table_mutex);