From patchwork Mon Dec 7 20:45:37 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 7790101 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id F0251BEEE1 for ; Mon, 7 Dec 2015 20:45:48 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 08BE1204EA for ; Mon, 7 Dec 2015 20:45:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E332F204B5 for ; Mon, 7 Dec 2015 20:45:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933027AbbLGUpl (ORCPT ); Mon, 7 Dec 2015 15:45:41 -0500 Received: from mga01.intel.com ([192.55.52.88]:4138 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933064AbbLGUpj (ORCPT ); Mon, 7 Dec 2015 15:45:39 -0500 Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP; 07 Dec 2015 12:45:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,396,1444719600"; d="scan'208";a="9114516" Received: from sedona.ch.intel.com ([143.182.228.65]) by fmsmga004.fm.intel.com with ESMTP; 07 Dec 2015 12:45:39 -0800 Received: from phlsvsles11.ph.intel.com (phlsvsles11.ph.intel.com [10.228.195.43]) by sedona.ch.intel.com (8.13.6/8.14.3/Standard MailSET/Hub) with ESMTP id tB7Kjbme028466; Mon, 7 Dec 2015 13:45:38 -0700 Received: from phlsvslse11.ph.intel.com (localhost [127.0.0.1]) by phlsvsles11.ph.intel.com with ESMTP id tB7Kjbho011169; Mon, 7 Dec 2015 15:45:37 -0500 Subject: [PATCH 36/37] IB/rdmavt: Add pkey support To: dledford@redhat.com From: Dennis Dalessandro Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn , Ira Weiny Date: Mon, 07 Dec 2015 15:45:37 -0500 Message-ID: <20151207204536.8144.51093.stgit@phlsvslse11.ph.intel.com> In-Reply-To: <20151207204046.8144.18752.stgit@phlsvslse11.ph.intel.com> References: <20151207204046.8144.18752.stgit@phlsvslse11.ph.intel.com> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add pkey table in rdi per port data structure. Also bring in related pkey functions. Drivers will still be responsible for allocating and maintaining the pkey table. However they need to tell rdmavt where to find the pkey table. We can not move the pkey table up into rdmavt because drivers need to manipulate this long before registering with it. Reviewed-by: Ira Weiny Reviewed-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/sw/rdmavt/vt.c | 46 ++++++++++++++++++++++++------------- include/rdma/rdma_vt.h | 38 +++++++++++++++++++++++++++---- 2 files changed, 64 insertions(+), 20 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c index ae0a1fe..d541b05 100644 --- a/drivers/infiniband/sw/rdmavt/vt.c +++ b/drivers/infiniband/sw/rdmavt/vt.c @@ -160,6 +160,17 @@ static int rvt_query_pkey(struct ib_device *ibdev, u8 port, u16 index, * lock, if a stale value is read and sent to the user so be it there is * no way to protect against that anyway. */ + struct rvt_dev_info *rdi = ib_to_rvt(ibdev); + int port_index; + + if (index >= rvt_get_npkeys(rdi)) + return -EINVAL; + + port_index = port - 1; /* IB ports start at 1 our array at 0 */ + if ((port_index < 0) || (port_index >= rdi->dparms.nports)) + return -EINVAL; + + *pkey = rvt_get_pkey(rdi, port_index, index); return 0; } @@ -233,19 +244,6 @@ int rvt_register_device(struct rvt_dev_info *rdi) return -EINVAL; } - if (!rdi->dparms.nports) { - rvt_pr_err(rdi, "Driver says it has no ports.\n"); - return -EINVAL; - } - - rdi->ports = kcalloc(rdi->dparms.nports, - sizeof(struct rvt_ibport **), - GFP_KERNEL); - if (!rdi->ports) { - rvt_pr_err(rdi, "Could not allocate port mem.\n"); - return -ENOMEM; - } - /* Once we get past here we can use the rvt_pr macros */ rvt_mmap_init(rdi); @@ -360,9 +358,25 @@ EXPORT_SYMBOL(rvt_unregister_device); * Keep track of a list of ports. No need to have a detach port. * They persist until the driver goes away. */ -void rvt_attach_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, - int portnum) +int rvt_init_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, + int portnum, u16 *pkey_table) { + if (!rdi->dparms.nports) { + rvt_pr_err(rdi, "Driver says it has no ports.\n"); + return -EINVAL; + } + + rdi->ports = kcalloc(rdi->dparms.nports, + sizeof(struct rvt_ibport **), + GFP_KERNEL); + if (!rdi->ports) { + rvt_pr_err(rdi, "Could not allocate port mem.\n"); + return -ENOMEM; + } + rdi->ports[portnum] = port; + rdi->ports[portnum]->pkey_table = pkey_table; + + return 0; } -EXPORT_SYMBOL(rvt_attach_port); +EXPORT_SYMBOL(rvt_init_port); diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h index 5681a54..afcc819 100644 --- a/include/rdma/rdma_vt.h +++ b/include/rdma/rdma_vt.h @@ -72,6 +72,8 @@ #define RVT_FLAG_QP_INIT_DRIVER BIT(2) #define RVT_FLAG_CQ_INIT_DRIVER BIT(3) +#define RVT_MAX_PKEY_VALUES 16 + struct rvt_ibport { struct rvt_qp __rcu *qp[2]; struct ib_mad_agent *send_agent; /* agent for SMI (traps) */ @@ -128,6 +130,14 @@ struct rvt_ibport { void *priv; /* driver private data */ + /* + * The pkey table is allocated and maintained by the driver. Drivers + * need to have access to this before registering with rdmav. However + * rdmavt will need access to it so drivers need to proviee this during + * the attach port API call. + */ + u16 *pkey_table; + /* TODO: Move sm_ah and smi_ah into here as well*/ }; @@ -181,6 +191,7 @@ struct rvt_driver_params { int qpn_res_start; int qpn_res_end; int nports; + int npkeys; u8 qos_shift; }; @@ -244,8 +255,6 @@ struct rvt_dev_info { struct rvt_mregion __rcu *dma_mr; struct rvt_lkey_table lk_table; - /* PKey Table goes here */ - /* Driver specific helper functions */ struct rvt_driver_provided driver_f; @@ -288,11 +297,32 @@ static inline struct rvt_srq *ibsrq_to_rvtsrq(struct ib_srq *ibsrq) return container_of(ibsrq, struct rvt_srq, ibsrq); } +static inline unsigned rvt_get_npkeys(struct rvt_dev_info *rdi) +{ + /* + * All ports have same number of pkeys. + */ + return rdi->dparms.npkeys; +} + +/* + * Return the indexed PKEY from the port PKEY table. + */ +static inline u16 rvt_get_pkey(struct rvt_dev_info *rdi, + int port_index, + unsigned index) +{ + if (index >= rvt_get_npkeys(rdi)) + return 0; + else + return rdi->ports[port_index]->pkey_table[index]; +} + int rvt_register_device(struct rvt_dev_info *rvd); void rvt_unregister_device(struct rvt_dev_info *rvd); int rvt_check_ah(struct ib_device *ibdev, struct ib_ah_attr *ah_attr); -void rvt_attach_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, - int portnum); +int rvt_init_port(struct rvt_dev_info *rdi, struct rvt_ibport *port, + int portnum, u16 *pkey_table); int rvt_rkey_ok(struct rvt_qp *qp, struct rvt_sge *sge, u32 len, u64 vaddr, u32 rkey, int acc); int rvt_lkey_ok(struct rvt_lkey_table *rkt, struct rvt_pd *pd,