From patchwork Mon Jul 20 08:01:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gal Pressman X-Patchwork-Id: 11673195 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92ED913A4 for ; Mon, 20 Jul 2020 08:01:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B69D20709 for ; Mon, 20 Jul 2020 08:01:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="m9u6UVIL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726666AbgGTIBk (ORCPT ); Mon, 20 Jul 2020 04:01:40 -0400 Received: from smtp-fw-4101.amazon.com ([72.21.198.25]:14369 "EHLO smtp-fw-4101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726015AbgGTIBk (ORCPT ); Mon, 20 Jul 2020 04:01:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1595232099; x=1626768099; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WCD+SaI/Zc2qPh/Zx+uAq6cQgp/p2lIgcDYvudjBgMg=; b=m9u6UVIL1znCDFhNYjS55AoiTlFqHKUaXa3wUHtp4KQpuMxKLw1f+t9H dMQj1cHS/2+D7FqpJa+E4rX/xM5h9vtYzooYMUN6KZxkZpdPy/KdSIfg5 nHHw2lPqpn4BGfSV+BXGwSwQQp85Zw+CHBrE+24xoJEYlkddbGkAWRYzz E=; IronPort-SDR: EV60Mwb6qaQHOV5zBFSz1RgZe6j/ZqrPqknWk8TGgqs/+IpoQaAti8iIQMS/yjmspjpB1gwmMy UgUMUmyGA32w== X-IronPort-AV: E=Sophos;i="5.75,374,1589241600"; d="scan'208";a="42881973" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2b-a7fdc47a.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-4101.iad4.amazon.com with ESMTP; 20 Jul 2020 08:01:38 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2b-a7fdc47a.us-west-2.amazon.com (Postfix) with ESMTPS id BBF6DC0597; Mon, 20 Jul 2020 08:01:36 +0000 (UTC) Received: from EX13D19EUB002.ant.amazon.com (10.43.166.78) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:36 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D19EUB002.ant.amazon.com (10.43.166.78) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:34 +0000 Received: from 8c85908914bf.ant.amazon.com (10.1.212.12) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 20 Jul 2020 08:01:31 +0000 From: Gal Pressman To: Jason Gunthorpe , Doug Ledford CC: , Alexander Matushevsky , Gal Pressman , Daniel Kranzdorf , Firas JahJah Subject: [PATCH for-next v2 1/4] RDMA/efa: Expose maximum TX doorbell batch Date: Mon, 20 Jul 2020 11:01:10 +0300 Message-ID: <20200720080113.13055-2-galpress@amazon.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200720080113.13055-1-galpress@amazon.com> References: <20200720080113.13055-1-galpress@amazon.com> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The device reports the maximum number of bytes to be written before ringing the doorbell (zero means unlimited). This patch queries the max batch size and reports it back to the userspace library. Reviewed-by: Daniel Kranzdorf Reviewed-by: Firas JahJah Signed-off-by: Gal Pressman --- drivers/infiniband/hw/efa/efa_admin_cmds_defs.h | 11 +++++++++++ drivers/infiniband/hw/efa/efa_com_cmd.c | 1 + drivers/infiniband/hw/efa/efa_com_cmd.h | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 1 + include/uapi/rdma/efa-abi.h | 4 +++- 5 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h index bef2bd291054..03e7388af06e 100644 --- a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h +++ b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h @@ -632,6 +632,17 @@ struct efa_admin_feature_queue_attr_desc { /* Maximum number of SGEs for a single RDMA read WQE */ u16 max_wr_rdma_sges; + + /* + * Maximum number of bytes that can be written to SQ between two + * consecutive doorbells (in units of 64B). Driver must ensure that only + * complete WQEs are written to queue before issuing a doorbell. + * Examples: max_tx_batch=16 and WQE size = 64B, means up to 16 WQEs can + * be written to SQ between two consecutive doorbells. max_tx_batch=11 + * and WQE size = 128B, means up to 5 WQEs can be written to SQ between + * two consecutive doorbells. Zero means unlimited. + */ + u16 max_tx_batch; }; struct efa_admin_feature_aenq_desc { diff --git a/drivers/infiniband/hw/efa/efa_com_cmd.c b/drivers/infiniband/hw/efa/efa_com_cmd.c index fabd8df2e78f..53cfde5c43d8 100644 --- a/drivers/infiniband/hw/efa/efa_com_cmd.c +++ b/drivers/infiniband/hw/efa/efa_com_cmd.c @@ -480,6 +480,7 @@ int efa_com_get_device_attr(struct efa_com_dev *edev, result->max_llq_size = resp.u.queue_attr.max_llq_size; result->sub_cqs_per_cq = resp.u.queue_attr.sub_cqs_per_cq; result->max_wr_rdma_sge = resp.u.queue_attr.max_wr_rdma_sges; + result->max_tx_batch = resp.u.queue_attr.max_tx_batch; err = efa_com_get_feature(edev, &resp, EFA_ADMIN_NETWORK_ATTR); if (err) { diff --git a/drivers/infiniband/hw/efa/efa_com_cmd.h b/drivers/infiniband/hw/efa/efa_com_cmd.h index 41ce4a476ee6..8df2a26d57d4 100644 --- a/drivers/infiniband/hw/efa/efa_com_cmd.h +++ b/drivers/infiniband/hw/efa/efa_com_cmd.h @@ -127,6 +127,7 @@ struct efa_com_get_device_attr_result { u16 max_sq_sge; u16 max_rq_sge; u16 max_wr_rdma_sge; + u16 max_tx_batch; u8 db_bar; }; diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c index 08313f7c73bc..f49d14cebe4a 100644 --- a/drivers/infiniband/hw/efa/efa_verbs.c +++ b/drivers/infiniband/hw/efa/efa_verbs.c @@ -1525,6 +1525,7 @@ int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata) resp.sub_cqs_per_cq = dev->dev_attr.sub_cqs_per_cq; resp.inline_buf_size = dev->dev_attr.inline_buf_size; resp.max_llq_size = dev->dev_attr.max_llq_size; + resp.max_tx_batch = dev->dev_attr.max_tx_batch; if (udata && udata->outlen) { err = ib_copy_to_udata(udata, &resp, diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h index 53b6e2036a9b..10781763da37 100644 --- a/include/uapi/rdma/efa-abi.h +++ b/include/uapi/rdma/efa-abi.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) */ /* - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All rights reserved. + * Copyright 2018-2020 Amazon.com, Inc. or its affiliates. All rights reserved. */ #ifndef EFA_ABI_USER_H @@ -31,6 +31,8 @@ struct efa_ibv_alloc_ucontext_resp { __u16 sub_cqs_per_cq; __u16 inline_buf_size; __u32 max_llq_size; /* bytes */ + __u16 max_tx_batch; /* units of 64 bytes */ + __u8 reserved_90[6]; }; struct efa_ibv_alloc_pd_resp { From patchwork Mon Jul 20 08:01:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gal Pressman X-Patchwork-Id: 11673199 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F48E13A4 for ; Mon, 20 Jul 2020 08:01:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68AAF20709 for ; Mon, 20 Jul 2020 08:01:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="ZzGcBYfS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727087AbgGTIBq (ORCPT ); Mon, 20 Jul 2020 04:01:46 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:24074 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726015AbgGTIBq (ORCPT ); Mon, 20 Jul 2020 04:01:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1595232106; x=1626768106; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DMKjB8xCfBzBCkKNfpgxia5k3f3wlJiB4oaoIdPpvAs=; b=ZzGcBYfSSiB3mONCXQ/vyI4IBIyghFMYtpLthEBuyvNnu4xXbMvNQXe7 z8nD5BZVUdIna5dTgRm1P0PYisf74jGuod+VKZpj8LM/BOgWDiVYUlC1K H3FeZ8OLC2YR+v3wsgeg/Pk1BiQffssCZMSnD/IG0r04gw6Gxwk0yjoDR A=; IronPort-SDR: D8CCVleMABBALdXvil6Pafh/Qj5I4f4YYNa4BlUcrauRRQSpVj2do2LVsy/9CZRwGrhoxym60U cYXiumI1reVA== X-IronPort-AV: E=Sophos;i="5.75,374,1589241600"; d="scan'208";a="42772455" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2a-53356bf6.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 20 Jul 2020 08:01:45 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2a-53356bf6.us-west-2.amazon.com (Postfix) with ESMTPS id BE8AEA1F1B; Mon, 20 Jul 2020 08:01:43 +0000 (UTC) Received: from EX13D02EUC001.ant.amazon.com (10.43.164.92) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:43 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D02EUC001.ant.amazon.com (10.43.164.92) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:42 +0000 Received: from 8c85908914bf.ant.amazon.com (10.1.212.12) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 20 Jul 2020 08:01:38 +0000 From: Gal Pressman To: Jason Gunthorpe , Doug Ledford CC: , Alexander Matushevsky , Gal Pressman , Firas JahJah , "Shadi Ammouri" Subject: [PATCH for-next v2 2/4] RDMA/efa: Expose minimum SQ size Date: Mon, 20 Jul 2020 11:01:11 +0300 Message-ID: <20200720080113.13055-3-galpress@amazon.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200720080113.13055-1-galpress@amazon.com> References: <20200720080113.13055-1-galpress@amazon.com> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The device reports the minimum SQ size required for creation. This patch queries the min SQ size and reports it back to the userspace library. Reviewed-by: Firas JahJah Reviewed-by: Shadi Ammouri Signed-off-by: Gal Pressman --- drivers/infiniband/hw/efa/efa_admin_cmds_defs.h | 4 ++-- drivers/infiniband/hw/efa/efa_com_cmd.c | 1 + drivers/infiniband/hw/efa/efa_com_cmd.h | 1 + drivers/infiniband/hw/efa/efa_verbs.c | 1 + include/uapi/rdma/efa-abi.h | 3 ++- 5 files changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h index 03e7388af06e..5484b08bbc5d 100644 --- a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h +++ b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h @@ -606,8 +606,8 @@ struct efa_admin_feature_queue_attr_desc { /* Number of sub-CQs to be created for each CQ */ u16 sub_cqs_per_cq; - /* MBZ */ - u16 reserved; + /* Minimum number of WQEs per SQ */ + u16 min_sq_depth; /* Maximum number of SGEs (buffers) allowed for a single send WQE */ u16 max_wr_send_sges; diff --git a/drivers/infiniband/hw/efa/efa_com_cmd.c b/drivers/infiniband/hw/efa/efa_com_cmd.c index 53cfde5c43d8..6ac23627f65a 100644 --- a/drivers/infiniband/hw/efa/efa_com_cmd.c +++ b/drivers/infiniband/hw/efa/efa_com_cmd.c @@ -481,6 +481,7 @@ int efa_com_get_device_attr(struct efa_com_dev *edev, result->sub_cqs_per_cq = resp.u.queue_attr.sub_cqs_per_cq; result->max_wr_rdma_sge = resp.u.queue_attr.max_wr_rdma_sges; result->max_tx_batch = resp.u.queue_attr.max_tx_batch; + result->min_sq_depth = resp.u.queue_attr.min_sq_depth; err = efa_com_get_feature(edev, &resp, EFA_ADMIN_NETWORK_ATTR); if (err) { diff --git a/drivers/infiniband/hw/efa/efa_com_cmd.h b/drivers/infiniband/hw/efa/efa_com_cmd.h index 8df2a26d57d4..190bac23f585 100644 --- a/drivers/infiniband/hw/efa/efa_com_cmd.h +++ b/drivers/infiniband/hw/efa/efa_com_cmd.h @@ -128,6 +128,7 @@ struct efa_com_get_device_attr_result { u16 max_rq_sge; u16 max_wr_rdma_sge; u16 max_tx_batch; + u16 min_sq_depth; u8 db_bar; }; diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c index f49d14cebe4a..26102ab333b2 100644 --- a/drivers/infiniband/hw/efa/efa_verbs.c +++ b/drivers/infiniband/hw/efa/efa_verbs.c @@ -1526,6 +1526,7 @@ int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata) resp.inline_buf_size = dev->dev_attr.inline_buf_size; resp.max_llq_size = dev->dev_attr.max_llq_size; resp.max_tx_batch = dev->dev_attr.max_tx_batch; + resp.min_sq_wr = dev->dev_attr.min_sq_depth; if (udata && udata->outlen) { err = ib_copy_to_udata(udata, &resp, diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h index 10781763da37..7ef2306f8dd4 100644 --- a/include/uapi/rdma/efa-abi.h +++ b/include/uapi/rdma/efa-abi.h @@ -32,7 +32,8 @@ struct efa_ibv_alloc_ucontext_resp { __u16 inline_buf_size; __u32 max_llq_size; /* bytes */ __u16 max_tx_batch; /* units of 64 bytes */ - __u8 reserved_90[6]; + __u16 min_sq_wr; + __u8 reserved_a0[4]; }; struct efa_ibv_alloc_pd_resp { From patchwork Mon Jul 20 08:01:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gal Pressman X-Patchwork-Id: 11673201 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A222313A4 for ; Mon, 20 Jul 2020 08:01:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A07D20702 for ; Mon, 20 Jul 2020 08:01:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="smS+ZstH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726520AbgGTIBz (ORCPT ); Mon, 20 Jul 2020 04:01:55 -0400 Received: from smtp-fw-4101.amazon.com ([72.21.198.25]:14447 "EHLO smtp-fw-4101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726015AbgGTIBy (ORCPT ); Mon, 20 Jul 2020 04:01:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1595232114; x=1626768114; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nd5cTpzvBEW3bME0QgeJ7poe/+hUeRst1fyzSB7rY54=; b=smS+ZstHMAFS+QKFVvsFYm0cRv2r4piXQqMEVpTrpHmBA7/SUHEOJ5t3 0Wz2yTYaHjvvLdkWsfGn4Um3+rF7+MVLaCdHf0JVIVKT/6vIlBgEJvJN5 qBSrI2z9/S4zSOfv4m8nXC5m525TFEhJyAH/8nQpLtiRXvedCiVgOughh g=; IronPort-SDR: WWfOUMbHNLnQfeAKrZI1T4jIHs62VmgCJmXmR3gaowdGd9Vd7RRKZckgsIFsB9pCCcjMwUl76E sCEo3ozFDR8w== X-IronPort-AV: E=Sophos;i="5.75,374,1589241600"; d="scan'208";a="42882035" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2c-1968f9fa.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-4101.iad4.amazon.com with ESMTP; 20 Jul 2020 08:01:53 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2c-1968f9fa.us-west-2.amazon.com (Postfix) with ESMTPS id 57A8EA403F; Mon, 20 Jul 2020 08:01:52 +0000 (UTC) Received: from EX13D22EUA003.ant.amazon.com (10.43.165.210) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:51 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D22EUA003.ant.amazon.com (10.43.165.210) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:50 +0000 Received: from 8c85908914bf.ant.amazon.com (10.1.212.12) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 20 Jul 2020 08:01:46 +0000 From: Gal Pressman To: Jason Gunthorpe , Doug Ledford CC: , Alexander Matushevsky , Gal Pressman , Shadi Ammouri , Yossi Leybovich Subject: [PATCH for-next v2 3/4] RDMA/efa: User/kernel compatibility handshake mechanism Date: Mon, 20 Jul 2020 11:01:12 +0300 Message-ID: <20200720080113.13055-4-galpress@amazon.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200720080113.13055-1-galpress@amazon.com> References: <20200720080113.13055-1-galpress@amazon.com> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce a mechanism that performs an handshake between the userspace provider and kernel driver which verifies that the user supports all required features in order to operate correctly. The handshake verifies the needed functionality by comparing the reported device caps and the provider caps. If the device reports a non-zero capability the appropriate comp mask is required from the userspace provider in order to allocate the context. Reviewed-by: Shadi Ammouri Reviewed-by: Yossi Leybovich Signed-off-by: Gal Pressman Reported-by: kernel test robot --- drivers/infiniband/hw/efa/efa_verbs.c | 60 +++++++++++++++++++++++++++ include/uapi/rdma/efa-abi.h | 10 +++++ 2 files changed, 70 insertions(+) diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c index 26102ab333b2..f60bf9ce656f 100644 --- a/drivers/infiniband/hw/efa/efa_verbs.c +++ b/drivers/infiniband/hw/efa/efa_verbs.c @@ -1501,11 +1501,59 @@ static int efa_dealloc_uar(struct efa_dev *dev, u16 uarn) return efa_com_dealloc_uar(&dev->edev, ¶ms); } +#define DEFINE_GET_DEV_ATTR_FUNC(_attr) \ + bool dev_attr_##_attr(const struct efa_dev *dev) \ + { \ + return dev->dev_attr._attr; \ + } + +#define DEFINE_CHECK_COMP_FUNC(_attr, _mask) \ + bool check_comp_##_attr(const struct efa_dev *dev, u32 comp_mask) \ + { \ + return !dev_attr_##_attr(dev) || (comp_mask & (_mask)); \ + } + +#define DEFINE_COMP_HANDSHAKE(_attr, _mask) \ + { \ + .attr = #_attr, \ + .check_comp = ({ \ + DEFINE_GET_DEV_ATTR_FUNC(_attr) \ + DEFINE_CHECK_COMP_FUNC(_attr, _mask) \ + check_comp_##_attr; \ + }), \ + } + +int efa_user_comp_handshake(const struct ib_ucontext *ibucontext, + const struct efa_ibv_alloc_ucontext_cmd *cmd) +{ + struct { + char *attr; + bool (*check_comp)(const struct efa_dev *dev, u32 comp_mask); + } user_comp_handshakes[] = { + DEFINE_COMP_HANDSHAKE(max_tx_batch, EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH), + DEFINE_COMP_HANDSHAKE(min_sq_depth, EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR), + }; + struct efa_dev *dev = to_edev(ibucontext->device); + int i; + + for (i = 0; i < ARRAY_SIZE(user_comp_handshakes); i++) { + if (!user_comp_handshakes[i].check_comp(dev, cmd->comp_mask)) { + ibdev_dbg(&dev->ibdev, + "Userspace handshake failed for %s attribute\n", + user_comp_handshakes[i].attr); + return -EOPNOTSUPP; + } + } + + return 0; +} + int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata) { struct efa_ucontext *ucontext = to_eucontext(ibucontext); struct efa_dev *dev = to_edev(ibucontext->device); struct efa_ibv_alloc_ucontext_resp resp = {}; + struct efa_ibv_alloc_ucontext_cmd cmd = {}; struct efa_com_alloc_uar_result result; int err; @@ -1514,6 +1562,18 @@ int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata) * we will ack input fields in our response. */ + err = ib_copy_from_udata(&cmd, udata, + min(sizeof(cmd), udata->inlen)); + if (err) { + ibdev_dbg(&dev->ibdev, + "Cannot copy udata for alloc_ucontext\n"); + goto err_out; + } + + err = efa_user_comp_handshake(ibucontext, &cmd); + if (err) + goto err_out; + err = efa_com_alloc_uar(&dev->edev, &result); if (err) goto err_out; diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h index 7ef2306f8dd4..507a2862bedb 100644 --- a/include/uapi/rdma/efa-abi.h +++ b/include/uapi/rdma/efa-abi.h @@ -20,6 +20,16 @@ * hex bit offset of the field. */ +enum { + EFA_ALLOC_UCONTEXT_CMD_COMP_TX_BATCH = 1 << 0, + EFA_ALLOC_UCONTEXT_CMD_COMP_MIN_SQ_WR = 1 << 1, +}; + +struct efa_ibv_alloc_ucontext_cmd { + __u32 comp_mask; + __u8 reserved_20[4]; +}; + enum efa_ibv_user_cmds_supp_udata { EFA_USER_CMDS_SUPP_UDATA_QUERY_DEVICE = 1 << 0, EFA_USER_CMDS_SUPP_UDATA_CREATE_AH = 1 << 1, From patchwork Mon Jul 20 08:01:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gal Pressman X-Patchwork-Id: 11673203 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E49C713A4 for ; Mon, 20 Jul 2020 08:02:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C38FB20702 for ; Mon, 20 Jul 2020 08:02:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="RCoklrLg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727097AbgGTICB (ORCPT ); Mon, 20 Jul 2020 04:02:01 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:18716 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726015AbgGTICB (ORCPT ); Mon, 20 Jul 2020 04:02:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1595232121; x=1626768121; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EXBkKvxEbJsQ96m+qS4fije8AJ7HuRS1Zj4zUOqcY00=; b=RCoklrLgYFR2+91beeiWBzNlJx6iIndPs1+D4XJI4Rp6UMy1x9ZhNAeK Ix0gvX/9Zvs4u7E26Cy23sCDs74aSlYu/pVSgNwzDa/wap3dfERxLA4Ig yKafP8Fp90UbetueY6R/jsgSRi7Gdo64zm8FaT4vZBJG0M/AzscW/FB9X w=; IronPort-SDR: 7d7ygNPQ5+PmELQpRbbgJj55hXnWRGlJ/zome3lQ+dV1Zme92nXWSrYBtAoefNFqFkQQMaTJVj 6KdJ7VqDZ9nA== X-IronPort-AV: E=Sophos;i="5.75,374,1589241600"; d="scan'208";a="61016680" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2b-5bdc5131.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 20 Jul 2020 08:01:59 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2b-5bdc5131.us-west-2.amazon.com (Postfix) with ESMTPS id 26844A0803; Mon, 20 Jul 2020 08:01:58 +0000 (UTC) Received: from EX13D22EUA003.ant.amazon.com (10.43.165.210) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:57 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D22EUA003.ant.amazon.com (10.43.165.210) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 20 Jul 2020 08:01:56 +0000 Received: from 8c85908914bf.ant.amazon.com (10.1.212.12) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 20 Jul 2020 08:01:52 +0000 From: Gal Pressman To: Jason Gunthorpe , Doug Ledford CC: , Alexander Matushevsky , Gal Pressman , Shadi Ammouri , Yossi Leybovich Subject: [PATCH for-next v2 4/4] RDMA/efa: Add EFA 0xefa1 PCI ID Date: Mon, 20 Jul 2020 11:01:13 +0300 Message-ID: <20200720080113.13055-5-galpress@amazon.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200720080113.13055-1-galpress@amazon.com> References: <20200720080113.13055-1-galpress@amazon.com> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add support for 0xefa1 devices. Reviewed-by: Shadi Ammouri Reviewed-by: Yossi Leybovich Signed-off-by: Gal Pressman --- drivers/infiniband/hw/efa/efa_main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/efa/efa_main.c b/drivers/infiniband/hw/efa/efa_main.c index 82145574c928..92d701146320 100644 --- a/drivers/infiniband/hw/efa/efa_main.c +++ b/drivers/infiniband/hw/efa/efa_main.c @@ -12,10 +12,12 @@ #include "efa.h" -#define PCI_DEV_ID_EFA_VF 0xefa0 +#define PCI_DEV_ID_EFA0_VF 0xefa0 +#define PCI_DEV_ID_EFA1_VF 0xefa1 static const struct pci_device_id efa_pci_tbl[] = { - { PCI_VDEVICE(AMAZON, PCI_DEV_ID_EFA_VF) }, + { PCI_VDEVICE(AMAZON, PCI_DEV_ID_EFA0_VF) }, + { PCI_VDEVICE(AMAZON, PCI_DEV_ID_EFA1_VF) }, { } };