From patchwork Fri Aug 18 16:44:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9909605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2FDD460386 for ; Fri, 18 Aug 2017 16:45:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 22D0128CCB for ; Fri, 18 Aug 2017 16:45:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1768328D22; Fri, 18 Aug 2017 16:45:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 853E828CCB for ; Fri, 18 Aug 2017 16:45:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752311AbdHRQpj (ORCPT ); Fri, 18 Aug 2017 12:45:39 -0400 Received: from mail-sn1nam02on0061.outbound.protection.outlook.com ([104.47.36.61]:27840 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752533AbdHRQpi (ORCPT ); Fri, 18 Aug 2017 12:45:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=mbCf+jLrgkHneg1JwEg+lnbCMBMKmtUgCTLJ30e7Irw=; b=dS9AvSlfsvOKZJmJpESlWxfv+6SymJh6voV/S9I4Elip0dDae6EaDTSb9nonkKrx44kQZiWEenkfwOPJew1Fc5LirycuyKzF54QdxTohBE6ZqKCSRh55a+WTFGFsHQJ/Xkca4zKi8L2lFS4Sw1imtPIY9D7g+MteORQIpzqLrl8= Received: from promb-2s-dhcp95-136.eng.vmware.com (208.91.1.34) by DM5PR05MB3516.namprd05.prod.outlook.com (10.174.242.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1362.12; Fri, 18 Aug 2017 16:45:35 +0000 From: Adit Ranadive To: dledford@redhat.com, linux-rdma@vger.kernel.org Cc: Adit Ranadive , pv-drivers@vmware.com Subject: [PATCH for-next 2/3] RDMA/vmw_pvrdma: Update device query parameters Date: Fri, 18 Aug 2017 09:44:48 -0700 Message-Id: <6eb31ebec69c1d28b83cb44e84cdb01ebdec3827.1503073950.git.aditr@vmware.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: MWHPR15CA0052.namprd15.prod.outlook.com (10.174.254.14) To DM5PR05MB3516.namprd05.prod.outlook.com (10.174.242.145) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 85d83b43-df6a-4963-c0d0-08d4e6588cd9 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DM5PR05MB3516; X-Microsoft-Exchange-Diagnostics: 1; DM5PR05MB3516; 3:G/qYSY48Ep/jQQ4TDTlcSbLnuzM7+z7edvL6f/waw483OC8jFe8JC5iqxaEf3A4x3OO6TK+55SD3xq3QatFtdD3Glf+C7y7asPOJj5PfzO6kslZWVL6g4/gL1gF5O7A2sLv9gAYk1WEI2JUGlpoj5+CoJruOdmeB5T/DLef424H7J8J2+uCUtMKBdsa9sm4wI0rSzI8gOd+A5RnvBF+FT/IlSTZ0e6ewOO8oV5SnMoYhlJAl1pzYyxvszkd5jsBm; 25:R8UlJmgh7DQGlxoTKXDmp/4TU5tFc1FpWc8oxhJ1qBoTL+TqUC3g5ziRd+uH6kUZ8i7VXFG6CBZkxMblyDDXqE5+z93kN6+xnIPYKyYxyP7VYcfHXUCP6hHQCP1Nv2suw9FwJ7ReAsNOPGaNJRBbtNfx/XTKd4MCfWEi8auORBkGvSVaiwkU7yzSWgl03h/riUwQm1TlCehGT3RxFg/7DV4lgC/vPDnmF5ewj8XW1UXHEkSxClmpi8tDhMbJbObV7dUXAAxqc2c8ZaP9RSM8EmVOQ7Ph4Mi9rchL5NIKNohbJbLjFv4F56hyoGuz1aNi1rAyR0RsoT+bq/hS8+f5hQ==; 31:wfepOoRQ8muvZf9nvnBsDGKO/dX1G+0g3INUnejR/h16eBoL05RiDZbmqySzmqXPL561zdwu+ktwhth4F/HYaPb6CAb0vXjQTsFFWizWPOu4fCMD4yED4tLBivHwnuLdKb5Z1jRaMGByGDQs6l6rVMI0xtrWvNy4sv0GI7Kx3DthYto9YrE96CBBvbboit8oZYCHK5NyZ22ocRoXCW6nEyKIQAUfauuMgRoZOzrtjlY= X-MS-TrafficTypeDiagnostic: DM5PR05MB3516: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; X-Microsoft-Exchange-Diagnostics: 1; DM5PR05MB3516; 20:AO3PKWexwuQaD13tWVCiAx50ymr9bor0nPzraHFFgOpAsrmPUErAuM5oRkvw1HMeslZxFErSliD5GLiKPKJMGmBbAyOv9pzVyFg8N2wLpFu9wISl6/L0YdDS3bodvp82dxJSrL92mGnB6Sr/n9jO1C1Xfag4VLZ60LF4cjnxVvmbpYpQThsWzdnV13sh7cenCrDB+NHMy+vTF2ZgdnUdAPbNYrCtX39MHAqZV7u5ah0qKwval1w3Frqfx02rG9B/zsGKxD12nO6+HVLciCO+CQHWSjpJqWFXIpfOdujqk4tzR8p0FECORBhQQ6T9G+LL5T4M15FAG8gp6U/gr9LY6+kLW5+a81rUDpQPuY3kjitQQtwnklSJRSLbaNKZDcRVLJCuum6Vz5GhbE2fQnHAU+DXWausw512KGEvcXIvCGsTRTGUi8ix3TJhD3RlgbyoM9eqmkorYHTBBCLwIIWKxd9JFemntDCb20P4CT/G+oCxVP5in7zLiKKFNHcSxkVw; 4:pK6VNgPNT6ZZoH7l6fxsjCzqY5jvwn0siyNGMs+e9RrhFA3F8lSSedKrmnrNdnho7H3ZuFVq5+FNU2/w8T6hAwo3FwqgvdVwjer+a+2V5sP920AhNT7kBbTS6KDsG7nlI6BUXz61ln7fKJ5JTgIhZDnaXT+E0JfUWtqrim0L8cyxi+S7NkkF3FmezgeuJ0H9Dopyv8oDeKrFx0SBHlguyHxCXbM4/IdvbzYVJZsbdRRdkeq8VQstXveVjYoD4P2cH9hRtYztq7zaX3B04DUzh0E4kwpZojeWohP0E21oxdo= X-Exchange-Antispam-Report-Test: UriScan:(61668805478150); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6041248)(20161123562025)(20161123560025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR05MB3516; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR05MB3516; X-Forefront-PRVS: 040359335D X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7370300001)(4630300001)(6009001)(199003)(189002)(66066001)(47776003)(5003940100001)(68736007)(101416001)(6486002)(53936002)(305945005)(7736002)(50226002)(97736004)(81166006)(8676002)(189998001)(81156014)(86362001)(50466002)(48376002)(33646002)(42186005)(2906002)(118296001)(76176999)(15650500001)(50986999)(36756003)(5660300001)(3846002)(2950100002)(110136004)(6666003)(107886003)(6116002)(106356001)(105586002)(478600001)(25786009)(4326008)(7350300001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR05MB3516; H:promb-2s-dhcp95-136.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR05MB3516; 23:41QuGaOzRMJHo+LTl7q1LOPfegR9GIC0E8M8WvKgO?= =?us-ascii?Q?+NoJ5a+CFIugOA79UJDjhWXkZRWZZhfp0a7XCp7mMTbkEX6k6YUkvDw/Klir?= =?us-ascii?Q?KsUsYv6tjzGpSV19Ig4teqqZK9EKwlJt5q2zKXpREBW5ExEy+ZV7SYmSi94t?= =?us-ascii?Q?7APanKHsnAd4cNdc1MN3j6fkslZAVTGfc1xiUblEBYe7mnz2G9ficoF0cZp0?= =?us-ascii?Q?L0EMQYDrZ4mSQpghtv14DKdpY1mEmYwMU2dWLe48gsc/WbwOOfvV5ZL628yj?= =?us-ascii?Q?ubtILfWTqz84e9+KgyZjfjJwsCG79AUmZpdRGOwh/4EHEZMjv41gIHFXE3zn?= =?us-ascii?Q?DDLukA+CngatfWRTquUeOdqmby/7p+wpG0sCUgANyMSNJZGyfsDzA/lQP7K3?= =?us-ascii?Q?QDtvB/b3UPVGuv2WgL5xDKG5relwo4kA4xxkX4kdrvPjy92Pgw8Wgpt18P38?= =?us-ascii?Q?5GiPc3MvCh+UatgNnmvrHPBmVGVHfZ0m1aruzaR6JGoFJcP32V+MGsi3hIi7?= =?us-ascii?Q?sDMKeqPZr4vUQZ8/i4SMcJYDSdkfhpBs1rorM/E0R8gvbwZZwgK0lWWAlukQ?= =?us-ascii?Q?ga81vBEkiZnssw70aInXdtP7OlaqEnF7NimbUyKnwb+7/z92SDW8xs8LJ3Te?= =?us-ascii?Q?qFRbTcJF0y1i/8L5WNCSLqFZEypb1VAlJqlqvzuKW43MRNpvqYp/wtBLf1+r?= =?us-ascii?Q?GdH0el00720Jkv2U2QkaoIwCi3aQilJRwINhrXlahogwAtXTylrE9TWg9CGC?= =?us-ascii?Q?IQ7M0TovVumP+EtRG/V13iITwbFEjT0dBUFa5+68p8tP1We/jD6jeSD712Xa?= =?us-ascii?Q?Ha9HvYPw3rJiUUcvCGnNJvEvNRgK2KWuPs1w68wxVbSEMN1PWSUoJQsTaEcJ?= =?us-ascii?Q?3FvQzeLuzyXzUm3MVqDaZQsFEqLdDFzpeOMTxuqA6LJQnq174kLnVChUkJWR?= =?us-ascii?Q?ewMK4P+3H5NTRmYBcCnpUhguzTE4ZUbkeGDb3ou+CTf7pm23y71XGfJwwuKG?= =?us-ascii?Q?evYdg3f8t0DQiAa8FB1R6DZi1HSQJRSvDwxFnOdkHK+STn7O2g7IwVpaUNYE?= =?us-ascii?Q?fZ6Gamm9Su32OeM3Fnv0ubmzGJESLa93uHt3zBeJDub2TYOZQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR05MB3516; 6:Z3aIk0rOeuLGzzs9pKRjnIa7Nd4Iuat1GSh7B9vD4ZPRxIS3oWRwGY1zfwDNCb9AXKMeVBYT091pjPZicrIy20skYVfN4RGxeJ68Ph3faJuF1OqOzxv8DR7W/GjaP6UEJi+hnhjwD/7uMjnVnjg8t1vvAbYpV/JeMwNYgzEd/BgGV08QzPltT7g2ziOM9pp9aR9pa2F9QVKCc19ziYAn2cm8HpF2F/gOo/znFJzmn8OHmHNmLckWg4nd0EcQnuKnklGVWmF67hepORbBrvRR8wZO4nHX+ovI5CtBhUoXXuze+HyKDrjMAqEGJZlLw4j/OguJYg+2xOIkmeZ1BA9oww==; 5:HNA31so0j3t4uF9UXZ0AydAXdpr9CIf0Fe2hLidr5w8hjmi5qUum26AtsaycwMYmiYS3zXpCKMZ2fxEk8K9f5rcGsy7VzfQvsQmtkcfoVeaWau0h1vIIuXZ4K2lnRBBsmmXg3JyxmbEO3tqRTj/Y6w==; 24:NymJT/USTF27imR0Co8Ye3GU2++OEU7RD6F/XjWOtV25VOrfoj1rG3rEb20/vjdSPW60mZZGtxGU8i0lijyhoFt6ZFM4qk41BGi6288Z/A4=; 7:GfB5yzUvlTfkeWxrUY2JXPNTs+c+2Ztz8qM0kNI5TZLM5O610mz6+9fmZ/QQkHDmoME5BWTL5V1059z9M3kgvLMMY6S6vaLkct/Or919XRQaFZi5hydBVBT1ShbbKfM43/dRNHErBbH7FvwDo6Z8a1JRWx80WNvmjVMBwJKcip8bhc3+L01eDTQ9LniLjdYV1tuy8IJThHzVGiBAANNbmWHFznwQieTtziHjhZudVAw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR05MB3516; 20:/f5vqKqCcjb1So2xmPTOIXKG9m5F6BqVJYZA/VR0S5FP5K8w88wjjF59MdECHmnyYics8cDrx/JNj3IXqOlhxHrwbXh4fIgl71VnSOl9B4ac7EJJq/J/dl83U59nS4eXj2e06WQUuEva8aST24M6LjCfLwaxAWO+RD8ZZJ0BmVI= X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2017 16:45:35.4267 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR05MB3516 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Added support for two device caps - max_sge_rd, max_fast_reg_page_list_len. Simplified some of the RoCEv2 versioning code and added 2 more device cap flags. Acked-by: Bryan Tan Acked-by: Aditya Sarwade Signed-off-by: Adit Ranadive --- drivers/infiniband/hw/vmw_pvrdma/pvrdma.h | 1 + drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h | 19 ++++++++++--------- drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c | 7 +++---- drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c | 9 +++++++++ 4 files changed, 23 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h b/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h index 513a182..ea10155 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma.h @@ -194,6 +194,7 @@ struct pvrdma_dev { void *resp_slot; unsigned long flags; struct list_head device_link; + unsigned int dsr_version; /* Locking and interrupt information. */ spinlock_t cmd_lock; /* Command lock. */ diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h index 71df5d1..d947557 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h @@ -50,7 +50,15 @@ #include "pvrdma_verbs.h" -#define PVRDMA_VERSION 18 +/* + * PVRDMA version macros. Some new features require updates to PVRDMA_VERSION. + * These macros allow us to check for different features if necessary. + */ + +#define PVRDMA_ROCEV1_VERSION 17 +#define PVRDMA_ROCEV2_VERSION 18 +#define PVRDMA_VERSION PVRDMA_ROCEV2_VERSION + #define PVRDMA_BOARD_ID 1 #define PVRDMA_REV_ID 1 @@ -123,13 +131,6 @@ #define PVRDMA_GID_TYPE_FLAG_ROCE_V1 BIT(0) #define PVRDMA_GID_TYPE_FLAG_ROCE_V2 BIT(1) -/* - * PVRDMA version macros. Some new features require updates to PVRDMA_VERSION. - * These macros allow us to check for different features if necessary. - */ - -#define PVRDMA_VERSION_ROCEV2_SUPPORT 18 - enum pvrdma_pci_resource { PVRDMA_PCI_RESOURCE_MSIX, /* BAR0: MSI-X, MMIO. */ PVRDMA_PCI_RESOURCE_REG, /* BAR1: Registers, MMIO. */ @@ -232,7 +233,7 @@ struct pvrdma_device_caps { u8 atomic_ops; /* PVRDMA_ATOMIC_OP_* bits */ u8 bmme_flags; /* FRWR Mem Mgmt Extensions */ u8 gid_types; /* PVRDMA_GID_TYPE_FLAG_ */ - u8 reserved[4]; + u32 max_fast_reg_page_list_len; }; struct pvrdma_ring_page_info { diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c index 6fd5828..ae536a4 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c @@ -729,7 +729,6 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, int ret; unsigned long start; unsigned long len; - unsigned int version; dma_addr_t slot_dma = 0; dev_dbg(&pdev->dev, "initializing driver %s\n", pci_name(pdev)); @@ -826,9 +825,9 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, goto err_unmap_regs; } - version = pvrdma_read_reg(dev, PVRDMA_REG_VERSION); + dev->dsr_version = pvrdma_read_reg(dev, PVRDMA_REG_VERSION); dev_info(&pdev->dev, "device version %d, driver version %d\n", - version, PVRDMA_VERSION); + dev->dsr_version, PVRDMA_VERSION); dev->dsr = dma_alloc_coherent(&pdev->dev, sizeof(*dev->dsr), &dev->dsrbase, GFP_KERNEL); @@ -908,7 +907,7 @@ static int pvrdma_pci_probe(struct pci_dev *pdev, } /* PVRDMA supports RoCE V1 or V2. */ - if (version >= PVRDMA_VERSION_ROCEV2_SUPPORT && + if (dev->dsr_version >= PVRDMA_ROCEV2_VERSION && !((dev->dsr->caps.gid_types & PVRDMA_GID_TYPE_FLAG_ROCE_V1) || (dev->dsr->caps.gid_types & PVRDMA_GID_TYPE_FLAG_ROCE_V2))) { dev_err(&pdev->dev, "driver needs RoCE v1 or v2 support\n"); diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c index 2851704..0b19d1a 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c @@ -83,6 +83,8 @@ int pvrdma_query_device(struct ib_device *ibdev, props->max_qp_wr = dev->dsr->caps.max_qp_wr; props->device_cap_flags = dev->dsr->caps.device_cap_flags; props->max_sge = dev->dsr->caps.max_sge; + props->max_sge_rd = (dev->dsr_version >= PVRDMA_ROCEV2_VERSION) ? + dev->dsr->caps.max_sge_rd : dev->dsr->caps.max_sge; props->max_cq = dev->dsr->caps.max_cq; props->max_cqe = dev->dsr->caps.max_cqe; props->max_mr = dev->dsr->caps.max_mr; @@ -100,9 +102,16 @@ int pvrdma_query_device(struct ib_device *ibdev, if ((dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_LOCAL_INV) && (dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_REMOTE_INV) && (dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_FAST_REG_WR)) { + props->max_fast_reg_page_list_len = + (dev->dsr_version >= PVRDMA_ROCEV2_VERSION) ? + dev->dsr->caps.max_fast_reg_page_list_len : + PVRDMA_MAX_FAST_REG_PAGES; props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; } + props->device_cap_flags |= IB_DEVICE_PORT_ACTIVE_EVENT | + IB_DEVICE_RC_RNR_NAK_GEN; + return 0; }