From patchwork Wed Jul 6 06:14:45 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9215525 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E7B2E607D9 for ; Wed, 6 Jul 2016 06:31:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D601B286FA for ; Wed, 6 Jul 2016 06:31:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CAD9A286FE; Wed, 6 Jul 2016 06:31:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45ED5286FA for ; Wed, 6 Jul 2016 06:31:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751178AbcGFGbe (ORCPT ); Wed, 6 Jul 2016 02:31:34 -0400 Received: from ex13-edg-ou-001.vmware.com ([208.91.0.189]:53393 "EHLO EX13-EDG-OU-001.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750886AbcGFGbc (ORCPT ); Wed, 6 Jul 2016 02:31:32 -0400 Received: from sc9-mailhost3.vmware.com (10.113.161.73) by EX13-EDG-OU-001.vmware.com (10.113.208.155) with Microsoft SMTP Server id 15.0.1156.6; Tue, 5 Jul 2016 23:15:12 -0700 Received: from EX13-CAS-003.vmware.com (ex13-cas-003.vmware.com [10.113.191.53]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id 18F71407B7; Tue, 5 Jul 2016 23:15:28 -0700 (PDT) Received: from EX13-CAS-003.vmware.com (10.113.191.53) by EX13-MBX-000.vmware.com (10.113.191.20) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Tue, 5 Jul 2016 23:15:27 -0700 Received: from NAM03-DM3-obe.outbound.protection.outlook.com (10.113.170.11) by EX13-CAS-003.vmware.com (10.113.191.53) with Microsoft SMTP Server (TLS) id 15.0.1156.6 via Frontend Transport; Tue, 5 Jul 2016 23:15:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=uka9QjQhABmz/BfDGpGSoh5tKRt3KhUQXcbpClGnAf4=; b=rHzrdTmo879P5bTPpgMILTAt9fO9itdMgEOXQssS+Cfb40mK4Jbvod7nUohxKaGndeFTtFWNotEuuJeG1M7Pv1/O3G9G7v6cPUsEL2Epb2Qw4JuE76Ijs7RiMuqrCuGXpMu9oVb2PRAthHlEPN9eqXgUZT/wzQi6W3hpa8meJbw= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; Received: from promb-2s-dhcp95-136.eng.vmware.com (208.91.1.34) by SN2PR0501MB848.namprd05.prod.outlook.com (10.160.14.146) with Microsoft SMTP Server (TLS) id 15.1.528.16; Wed, 6 Jul 2016 06:15:25 +0000 From: Adit Ranadive To: , , CC: Adit Ranadive , , , , Subject: [PATCH v1 12/15] IB/pvrdma: Add functions for Verbs support Date: Tue, 5 Jul 2016 23:14:45 -0700 Message-ID: <1467785688-23229-13-git-send-email-aditr@vmware.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1467785688-23229-1-git-send-email-aditr@vmware.com> References: <1467785688-23229-1-git-send-email-aditr@vmware.com> MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: CO2PR11CA0028.namprd11.prod.outlook.com (10.141.242.166) To SN2PR0501MB848.namprd05.prod.outlook.com (10.160.14.146) X-MS-Office365-Filtering-Correlation-Id: 8c119348-0455-4e62-9488-08d3a564ebbf X-Microsoft-Exchange-Diagnostics: 1; SN2PR0501MB848; 2:WYqhggry4DPIWeA6vBLDAUXv1heQrXCzf+XmZdw9GMn1m2rlx4NNHq11EY5NSgQBysKE/wGF5kI00Dxvju9lSvbIRn6BGtm60vmPXRxQpfs88Zf0napdKhmSiWa1loEC3oU8G84S3HtllbLURbMCk0uxK9vO3tj0wAgVwGbLRV5YgNPQ3Zja7FYLZnzmJ+a9; 3:bp+Uj+2wgXioCRSI9t4FrzIS25zsyElp4AvjTlBU11TIHzVJD4zooH3xKhNRGHwnsItLSxYCWA9vK6ijQ6IvdFxsVhXoHO82/C9zjCBWYlxy4H+W1ssmycPadkoejGcb X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN2PR0501MB848; X-Microsoft-Exchange-Diagnostics: 1; SN2PR0501MB848; 25:JSPRFoQfzv8P3aHQVtwRttOoEaKOW2+TUxlUNwXTwhXv9FPVhshEpyH1TKIt0xDpJoFfIqnxkGzcvkhhF6cHINwK8d9I44MF3JpVwFyugtn1yNuLI3g9OyenF72p/O9ExudLKk6Rneroe3tCjdGRXu0eCozFkZaEkxVnb6f6beyJsOjC2AU03Ry3SV7On95oyIxtk4s+lDHUUooCzVy5igbY/MYQ1MA+NyoBlXg1YrKokwYUHRGRVQQ7rCXud0B6c2BLl5iOpmPAg6fiyABEX9+w2ENpbZ2CoxoK24kxAlWG8bWZWK71p/EvNPp/vkutgJrV/Tmt6VlYfyjiDffquocOcxo7mbCSkY65G5InDYKt9CYH2TswEFpaw9qWAKw/JyJ8xsGfZVKOGWnpvNnQF7FW7dlbEKCTawsveXGEgpNHrp2dMh6SqR6BQLEGULFWHLsFZUjyAJW8fDXFl6wlasLcAGtjOWt4ZqsHKGWgycnly/M0uhI/Zx9zB3GSUZQz6pCBCRtD37TFN0AZK8+1vuQI/POWGHs+04nUyk436UnSe+aoeLs3RrN1SFvTYgB7BwqyS1vV/8t+NzYjVHlIH1k1gmzHdxX6TgCmc2qAmJKmDJ/rTfy88Vcw6MIo4rlY+41msg8JpBfQNn7ZKELPeqUMPTX6j7uTOwrRw7AYkNBGAlVrBC8acgjHFIBMxjsK+i2gYnfkvl582SUOpYJnLlGvQLJBDAowRitIunxQrtddMMSWQGs7ge0qKPq723ZSVAYVA9GZzLm+H3Lq6do1XdR9W/DVQpQ8KRuSUhcG7uzg/fJ7v+p5cGy6Eu87vKK+ X-Microsoft-Exchange-Diagnostics: 1; SN2PR0501MB848; 31:I3747G9iQ5MQZfS99yeDvH22Dz9N3yTbo3d9yyRe43PbrsDphWI4IJmzHogGXh+cw2gXDgChXSRyvEC1svov7ksWMRm3r6Ew6mboTa94gVNBleJkglD2BmXflY5FoRApBV/Mme4X31CroZpM8KZnGsBQpprOngc/3gc6O2a86vgbH0q85Q1QHcZO5qkKo+C2S47I0lgx0CTq0/tU72c6gw==; 20:Gg9cTEGYO6/JuXbVYj0fnyaWF43B7MIKkJwr78C91DA2rB0bQrRHAvb+V0hsF3I+QfwmU0RpQF82zoFVmQM3JkM/CM/IvCBJn+IVL7MKCuj63VYBUCNv62sQyeMRFh6N+7Hrivo96n8jOy1UWvlkXJsae9+CcqKoQ8pNJpp7WJx0bPh1Sf3wBT5Wb3WapVthaS3/S1wq5aA6L0bWF3Yq0Lzxh9RwX0gNn9i8eu/4V2zkaJ2Hdrn7cP3dmobmy6RE; 4:Kx1agklcbEd33Xz7F6ZlvOKfemsDoM2fSSXqpRXtYXh6GvyHaP5P1cv3kCtl7b6ilZhoafSj7QKU25/fS1bi61P8nzjlG1VB+gC7Q0BLcO+FEl+KTJwiZEsbOmxENbRCGKG5R9ZXnd4LHMA8tMzLXC4fDwkWJXwc0xr6lNR1mOJDYxikDvNeEY5TMQytL711dvT8q3PExryYDXuCiEA3Yca205Y4E63GaoLfzsOhUK+ie0ahmW1u2EdKi4qfixvJDCQnI4x2i25EKSYGal0KE2IAWPrM1wKKokTw2vq5dn7++mcGlDJ7MQ/KEMsGRpXhXco18jQRsJPqqHjyyeXhkdMmJsRCGc0lHX2iNIy2wozLcJNEJ2fddEXV2cjwiMmpGP5mHaKYeUeFa7VCiHcA+BzT3YlvjEDJp52XxoyjRkLuqWZfvz6LzJtSX1kG0cx0KNPsd/EkE SkxPiK7fBgL6J5fdfuMca0UAHLPY2j1nIk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(250305191791016)(61668805478150)(22074186197030); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:SN2PR0501MB848; BCL:0; PCL:0; RULEID:; SRVR:SN2PR0501MB848; X-Forefront-PRVS: 0995196AA2 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(81166006)(81156014)(4001450100002)(92566002)(86362001)(19580395003)(4001430100002)(19580405001)(7846002)(7736002)(305945005)(8676002)(105586002)(2201001)(586003)(42186005)(5890100001)(33646002)(106356001)(3846002)(5003940100001)(6116002)(48376002)(36756003)(50466002)(5001770100001)(97736004)(189998001)(229853001)(47776003)(66066001)(76176999)(107886002)(50986999)(68736007)(101416001)(50226002)(4326007)(2950100001)(15975445007)(77096005)(2906002)(7099028)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN2PR0501MB848; H:promb-2s-dhcp95-136.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (EX13-EDG-OU-001.vmware.com: aditr@vmware.com does not designate permitted sender hosts) Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN2PR0501MB848; 23:xugvQ6sHn3UZ875iEp+BK6RJFyRzZ0DR3EbzYKca?= =?us-ascii?Q?O/OAwNjALUYaJiyZWDDgmRpM0C4/EZzxkh4yguXox9T1+84ABn6FF1YbhK40?= =?us-ascii?Q?1LsvE5JILynKzTiQmadTH66U1p4K1B2twFyjfTQEhkwMqmPFY+UjobJADsuf?= =?us-ascii?Q?CP4laIssmzxBjqBfPBDfGRgeOKp9Yh45ZSwql6mWwl3bcn9+CA8TZ1zmntIT?= =?us-ascii?Q?SgBGtxcCcexVowdgfhfL47jRdp/VotO4EDTq7yRJ5jO9Vf14mEhvjpJr6ErL?= =?us-ascii?Q?Bipbm6En/VYML+HD88Xe8JSqYV9JWTK2ce7mBooXrAsJ6wk/2lrR7ahZ3Ma0?= =?us-ascii?Q?JWwy5t26q9smpBGT9thP9WKjj8yxj/T0Ijw+jBPGuj2GR+EX83DTzNIPgzYq?= =?us-ascii?Q?9G2sOy4db5kpHZZFNDvNl4up/GlTNLzwf6J/mefCAEKxnqN4ph7ZFzgIZcDN?= =?us-ascii?Q?okJi88LUbfD8cIdLNV09J52toR8PtDnKRk9v5Uc8O8hu8ABg3l9mGCvkXQfo?= =?us-ascii?Q?pOtR5E/AEq9ei6SpSn7D/TNQhepi3pbhaivSVPK7cbpmZCzpHhGgZOWR00sb?= =?us-ascii?Q?v2A4l/7qGXSaB/v3hGQOeYFfg3X6X0K9hTPB9afx7lSSLEiD3r3ZYSp5i5Q+?= =?us-ascii?Q?Qrh+p78m0ad2P2WGSERvPiBCgKuG5mmss0tVCqX7Ch7Sws39zcQJ8goOQHZA?= =?us-ascii?Q?1JC77kmmN26U9DMgoGA8Rc9btJFLs/BGuTTOiGmw6z3qOr7pg5ZfasLuu9Po?= =?us-ascii?Q?ASuj/xF77dtobFKHTqnWnzFU0L7oQxpKlb1ceL9Dm/Rk6aKY4WP0d0iLrDqL?= =?us-ascii?Q?lO3I+QZbZojO3O23dRnkCKL+h2BQR22W9fKiogjnkz8ivnaQg21GHGxrkGeq?= =?us-ascii?Q?QNmjCxvCSP0uFI333PGwKRIo+al48kInTJJBxLH5lJ6eb6HRma+8YLx9C2Ou?= =?us-ascii?Q?7Kc1o3n8Xy+ODlHXJhejbt3v4sdGF3GpPJUK/4iBzQmsn0xjnSxg+Qi36OGG?= =?us-ascii?Q?4bFFy9+CSwrwax8J17Y/CxLMHDJxIgq7E1TlaZpK0rLZM8rqo8a23/Wk13Me?= =?us-ascii?Q?gHPkkol+a77MH+R2GbgE3RoEFBHQFgzrIwltzUx7eb12GpJQhsogqffpnpYn?= =?us-ascii?Q?YZH5vnU4hwOBFbHmL0vg1yVC5yp3zSJ3xfdJQOuGxUfoRphJiQO5HGFIvQg6?= =?us-ascii?Q?lKaTv9Wk+hMClFJdIX+bxwbmW3ODjCsd4NZa6Si7UODsIYGw/RU0Wcw/IFJ9?= =?us-ascii?Q?REkr5snBk+bEYHZZKkM=3D?= X-Microsoft-Exchange-Diagnostics: 1; SN2PR0501MB848; 6:XG9od0GNd7YIAmqu0rnPpw+NFVyLhYT6+040OP7lh3uB4sxCD1I3LfJumPdPZaDSc+aEpCZDsLY83LF9CUnZ8oO0zlm6BNehQ6O7edhqCC5eKe2GhoiaqwKByx+UbiE2cnjWd0rEazjX0TUld/91IQuTNQsYC3kzf5tj1qJfoWvZxizGHgAYGQgc3WhX0MD+aPdlwA8oaGsmrm0B8ak+3pNu+Px9R2IS5xGdQOPcPWoGTm0pOvrvGfHhQQBz62AjgYDuRarPM6UY56eEk5HEcusHBKbN8ugZ2tGeJAPuE0I=; 5:DdmSye+b47npCL+paA8bbjPmV0emvNJ7wholQXrCjw+UcW3Ap4sQpoe+JDP+sn3BQOU2IwKe7xU/+wUDcOMwWpAePA/9FNb/27yyaVCSjlW1ioWMuHGa/pcGES6kY/9u5xUIWpF3E5Ow9ZJ1YXlrpw==; 24:jyKiGLtBU1pIiBjr/i7Az5/wHd38PYgiV8AHVRVqys1T/7bnHdQtWkpRtOl43MKE0kJIxffBhx1fXDC4VQJ1IAQ4ic9dZ5TnSeAO/39uLzM=; 7:dicY2UtXiFOwcArVk20PkEFajlzBu0rwy9g1bPwdnOY0noiTDzhJCOwXo5U0E5Kb4Wgli+wzsfvFFAA5s2RJHilMS0ZXePLeZ72xM3uH67K+w5l+km5nrmG9/HbnUyRFe0763Qv52nXzoKiDYQWbm8t/cz3XWqBPQlBDY6MvlDq2dqA0jU7x8aCelxCAEPZzY6+/CDntzZkfu2QiH2fMVA1yF5QzuAQq7eKB5iOE9D7EJVysSzGXUAwinx/PfNS2 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN2PR0501MB848; 20:/XTjvLMqhmndr2SqwVyfQHqfANOZSWqRWfxYQ2bi/DMAWCFP3/J2a5KQzVOAQcwOfpr/yU6/1vhFVjejuD9D1sHKZXB9OTWrgA154bOvfXp6dYd61rg6PJWWnVHwLpZcfxrDteOY71D3MchsgTdNLUTL0yMTQQAbPisIqE8Dwik= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2016 06:15:25.1849 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR0501MB848 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch implements the remaining Verbs functions registered with the core RDMA stack. Reviewed-by: Jorgen Hansen Reviewed-by: George Zhang Reviewed-by: Aditya Sarwade Reviewed-by: Bryan Tan Signed-off-by: Adit Ranadive --- drivers/infiniband/hw/pvrdma/pvrdma_verbs.c | 593 ++++++++++++++++++++++++++++ drivers/infiniband/hw/pvrdma/pvrdma_verbs.h | 108 +++++ 2 files changed, 701 insertions(+) create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_verbs.c create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_verbs.h diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_verbs.c b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.c new file mode 100644 index 0000000..bfc5f89 --- /dev/null +++ b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.c @@ -0,0 +1,593 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include +#include + +#include "pvrdma.h" +#include "pvrdma_user.h" + +/** + * pvrdma_query_device - query device + * @ibdev: the device to query + * @props: the device properties + * @uhw: user data + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_device(struct ib_device *ibdev, + struct ib_device_attr *props, + struct ib_udata *uhw) +{ + struct pvrdma_dev *dev = to_vdev(ibdev); + + if (uhw->inlen || uhw->outlen) + return -EINVAL; + + memset(props, 0, sizeof(*props)); + + props->fw_ver = dev->dsr->caps.fw_ver; + props->sys_image_guid = dev->dsr->caps.sys_image_guid; + props->max_mr_size = dev->dsr->caps.max_mr_size; + props->page_size_cap = dev->dsr->caps.page_size_cap; + props->vendor_id = dev->dsr->caps.vendor_id; + props->vendor_part_id = dev->pdev->device; + props->hw_ver = dev->dsr->caps.hw_ver; + props->max_qp = dev->dsr->caps.max_qp; + props->max_qp_wr = dev->dsr->caps.max_qp_wr; + props->device_cap_flags = dev->dsr->caps.device_cap_flags; + props->max_sge = dev->dsr->caps.max_sge; + props->max_sge_rd = dev->dsr->caps.max_sge_rd; + props->max_cq = dev->dsr->caps.max_cq; + props->max_cqe = dev->dsr->caps.max_cqe; + props->max_mr = dev->dsr->caps.max_mr; + props->max_pd = dev->dsr->caps.max_pd; + props->max_qp_rd_atom = dev->dsr->caps.max_qp_rd_atom; + props->max_ee_rd_atom = dev->dsr->caps.max_ee_rd_atom; + props->max_res_rd_atom = dev->dsr->caps.max_res_rd_atom; + props->max_qp_init_rd_atom = dev->dsr->caps.max_qp_init_rd_atom; + props->max_ee_init_rd_atom = dev->dsr->caps.max_ee_init_rd_atom; + props->atomic_cap = + dev->dsr->caps.atomic_ops & + (PVRDMA_ATOMIC_OP_COMP_SWAP | PVRDMA_ATOMIC_OP_FETCH_ADD) ? + IB_ATOMIC_HCA : IB_ATOMIC_NONE; + props->masked_atomic_cap = props->atomic_cap; + props->max_ee = dev->dsr->caps.max_ee; + props->max_rdd = dev->dsr->caps.max_rdd; + props->max_mw = dev->dsr->caps.max_mw; + props->max_raw_ipv6_qp = dev->dsr->caps.max_raw_ipv6_qp; + props->max_raw_ethy_qp = dev->dsr->caps.max_raw_ethy_qp; + props->max_mcast_grp = dev->dsr->caps.max_mcast_grp; + props->max_mcast_qp_attach = dev->dsr->caps.max_mcast_qp_attach; + props->max_total_mcast_qp_attach = + dev->dsr->caps.max_total_mcast_qp_attach; + props->max_ah = dev->dsr->caps.max_ah; + props->max_fmr = dev->dsr->caps.max_fmr; + props->max_map_per_fmr = dev->dsr->caps.max_map_per_fmr; + props->max_srq = dev->dsr->caps.max_srq; + props->max_srq_wr = dev->dsr->caps.max_srq_wr; + props->max_srq_sge = dev->dsr->caps.max_srq_sge; + props->max_fast_reg_page_list_len = 0; + props->max_pkeys = dev->dsr->caps.max_pkeys; + props->local_ca_ack_delay = dev->dsr->caps.local_ca_ack_delay; + if ((dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_LOCAL_INV) && + (dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_REMOTE_INV) && + (dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_FAST_REG_WR)) { + props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; + } + + return 0; +} + +/** + * pvrdma_query_port - query device port attributes + * @ibdev: the device to query + * @port: the port number + * @props: the device properties + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_port(struct ib_device *ibdev, u8 port, + struct ib_port_attr *props) +{ + struct pvrdma_dev *dev = to_vdev(ibdev); + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_query_port *cmd = &req.query_port; + struct pvrdma_cmd_query_port_resp *resp = &rsp.query_port_resp; + int err; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_QUERY_PORT; + cmd->port_num = port; + + err = pvrdma_cmd_post(dev, &req, true, &rsp); + if (err < 0 || rsp.hdr.ack != PVRDMA_CMD_QUERY_PORT_RESP) { + dev_warn(&dev->pdev->dev, "could not query port\n"); + return -EFAULT; + } + + memset(props, 0, sizeof(*props)); + + props->state = pvrdma_port_state_to_ib(resp->attrs.state); + props->max_mtu = pvrdma_mtu_to_ib(resp->attrs.max_mtu); + props->active_mtu = pvrdma_mtu_to_ib(resp->attrs.active_mtu); + props->gid_tbl_len = resp->attrs.gid_tbl_len; + props->port_cap_flags = + pvrdma_port_cap_flags_to_ib(resp->attrs.port_cap_flags); + props->max_msg_sz = resp->attrs.max_msg_sz; + props->bad_pkey_cntr = resp->attrs.bad_pkey_cntr; + props->qkey_viol_cntr = resp->attrs.qkey_viol_cntr; + props->pkey_tbl_len = resp->attrs.pkey_tbl_len; + props->lid = resp->attrs.lid; + props->sm_lid = resp->attrs.sm_lid; + props->lmc = resp->attrs.lmc; + props->max_vl_num = resp->attrs.max_vl_num; + props->sm_sl = resp->attrs.sm_sl; + props->subnet_timeout = resp->attrs.subnet_timeout; + props->init_type_reply = resp->attrs.init_type_reply; + props->active_width = pvrdma_port_width_to_ib(resp->attrs.active_width); + props->active_speed = pvrdma_port_speed_to_ib(resp->attrs.active_speed); + props->phys_state = resp->attrs.phys_state; + + return 0; +} + +/** + * pvrdma_query_gid - query device gid + * @ibdev: the device to query + * @port: the port number + * @index: the index + * @gid: the device gid value + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_gid(struct ib_device *ibdev, u8 port, int index, + union ib_gid *gid) +{ + struct pvrdma_dev *dev = to_vdev(ibdev); + + if (index >= dev->dsr->caps.gid_tbl_len) + return -EINVAL; + + memcpy(gid, &dev->sgid_tbl[index], sizeof(union ib_gid)); + + return 0; +} + +/** + * pvrdma_query_pkey - query device port's P_Key table + * @ibdev: the device to query + * @port: the port number + * @index: the index + * @pkey: the device P_Key value + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_pkey(struct ib_device *ibdev, u8 port, u16 index, + u16 *pkey) +{ + int err = 0; + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_query_pkey *cmd = &req.query_pkey; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_QUERY_PKEY; + cmd->port_num = port; + cmd->index = index; + + err = pvrdma_cmd_post(to_vdev(ibdev), &req, true, &rsp); + if (err < 0 || rsp.hdr.ack != PVRDMA_CMD_QUERY_PKEY_RESP) { + struct pvrdma_dev *dev = to_vdev(ibdev); + + dev_warn(&dev->pdev->dev, "could not query device pkey\n"); + return -EFAULT; + } + + *pkey = rsp.query_pkey_resp.pkey; + + return 0; +} + +enum rdma_link_layer pvrdma_port_link_layer(struct ib_device *ibdev, + u8 port) +{ + return IB_LINK_LAYER_ETHERNET; +} + +int pvrdma_modify_device(struct ib_device *ibdev, int mask, + struct ib_device_modify *props) +{ + unsigned long flags; + int ret; + + if (mask & ~(IB_DEVICE_MODIFY_SYS_IMAGE_GUID | + IB_DEVICE_MODIFY_NODE_DESC)) { + struct pvrdma_dev *dev = to_vdev(ibdev); + + dev_warn(&dev->pdev->dev, "unsupported device modify mask\n"); + ret = -EOPNOTSUPP; + goto err_out; + } + + if (mask & IB_DEVICE_MODIFY_NODE_DESC) { + spin_lock_irqsave(&to_vdev(ibdev)->desc_lock, flags); + memcpy(ibdev->node_desc, props->node_desc, 64); + spin_unlock_irqrestore(&to_vdev(ibdev)->desc_lock, flags); + } + + if (mask & IB_DEVICE_MODIFY_SYS_IMAGE_GUID) { + mutex_lock(&to_vdev(ibdev)->port_mutex); + to_vdev(ibdev)->sys_image_guid = + cpu_to_be64(props->sys_image_guid); + mutex_unlock(&to_vdev(ibdev)->port_mutex); + } + + return 0; + +err_out: + return ret; +} + +/** + * pvrdma_modify_port - modify device port attributes + * @ibdev: the device to modify + * @port: the port number + * @mask: attributes to modify + * @props: the device properties + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_modify_port(struct ib_device *ibdev, u8 port, int mask, + struct ib_port_modify *props) +{ + struct ib_port_attr attr; + struct pvrdma_dev *vdev = to_vdev(ibdev); + int ret; + + mutex_lock(&vdev->port_mutex); + ret = pvrdma_query_port(ibdev, port, &attr); + if (ret) + goto err_out; + + vdev->port_cap_mask |= props->set_port_cap_mask; + vdev->port_cap_mask &= ~props->clr_port_cap_mask; + + if (mask & IB_PORT_SHUTDOWN) + vdev->ib_active = false; + + if (mask & (IB_PORT_INIT_TYPE | IB_PORT_RESET_QKEY_CNTR)) { + ret = -EOPNOTSUPP; + goto err_out; + } + + ret = 0; + +err_out: + mutex_unlock(&vdev->port_mutex); + return ret; +} + +/** + * pvrdma_alloc_ucontext - allocate ucontext + * @ibdev: the IB device + * @udata: user data + * + * @return: the ib_ucontext pointer on success, otherwise errno. + */ +struct ib_ucontext *pvrdma_alloc_ucontext(struct ib_device *ibdev, + struct ib_udata *udata) +{ + struct pvrdma_dev *vdev = to_vdev(ibdev); + struct pvrdma_ucontext *context; + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_create_uc *cmd = &req.create_uc; + struct pvrdma_cmd_create_uc_resp *resp = &rsp.create_uc_resp; + struct pvrdma_alloc_ucontext_resp uresp; + int ret; + + if (!vdev->ib_active) + return ERR_PTR(-EAGAIN); + + context = kmalloc(sizeof(*context), GFP_KERNEL); + if (!context) + return ERR_PTR(-ENOMEM); + + context->dev = vdev; + ret = pvrdma_uar_alloc(vdev, &context->uar); + if (ret) { + kfree(context); + return ERR_PTR(-ENOMEM); + } + + /* get ctx_handle from host */ + memset(cmd, 0, sizeof(*cmd)); + cmd->pfn = context->uar.pfn; + cmd->hdr.cmd = PVRDMA_CMD_CREATE_UC; + ret = pvrdma_cmd_post(vdev, &req, true, &rsp); + if (ret < 0 || rsp.hdr.ack != PVRDMA_CMD_CREATE_UC_RESP) { + struct pvrdma_dev *dev = to_vdev(ibdev); + + dev_warn(&dev->pdev->dev, "could not create ucontext\n"); + pvrdma_uar_free(vdev, &context->uar); + kfree(context); + return ERR_PTR(-EFAULT); + } + + context->ctx_handle = resp->ctx_handle; + + /* copy back to user */ + uresp.qp_tab_size = vdev->dsr->caps.max_qp; + ret = ib_copy_to_udata(udata, &uresp, sizeof(uresp)); + if (ret) { + pvrdma_uar_free(vdev, &context->uar); + context->ibucontext.device = ibdev; + pvrdma_dealloc_ucontext(&context->ibucontext); + return ERR_PTR(-EFAULT); + } + + return &context->ibucontext; +} + +/** + * pvrdma_dealloc_ucontext - deallocate ucontext + * @ibcontext: the ucontext + * + * @return: 0 on success, otherwise errno. + */ +int pvrdma_dealloc_ucontext(struct ib_ucontext *ibcontext) +{ + struct pvrdma_ucontext *context = to_vucontext(ibcontext); + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_destroy_uc *cmd = &req.destroy_uc; + int ret; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_DESTROY_UC; + cmd->ctx_handle = context->ctx_handle; + + ret = pvrdma_cmd_post(context->dev, &req, false, &rsp); + if (ret < 0) { + dev_warn(&context->dev->pdev->dev, + "destroy ucontext failed\n"); + /* Just do it */ + } + pvrdma_uar_free(to_vdev(ibcontext->device), &context->uar); + kfree(context); + + return ret; +} + +/** + * pvrdma_mmap - create mmap region + * @ibcontext: the user context + * @vma: the VMA + * + * @return: 0 on success, otherwise errno. + */ +int pvrdma_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma) +{ + struct pvrdma_ucontext *context = to_vucontext(ibcontext); + unsigned long start = vma->vm_start; + unsigned long size = vma->vm_end - vma->vm_start; + unsigned long offset = vma->vm_pgoff << PAGE_SHIFT; + + dev_dbg(&context->dev->pdev->dev, "create mmap region\n"); + + if ((size != PAGE_SIZE) || (offset & ~PAGE_MASK)) { + dev_warn(&context->dev->pdev->dev, + "invalid params for mmap region\n"); + return -EINVAL; + } + + /* Map UAR to kernel space, VM_LOCKED? */ + vma->vm_flags |= VM_DONTCOPY | VM_DONTEXPAND; + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + if (io_remap_pfn_range(vma, start, context->uar.pfn, size, + vma->vm_page_prot)) + return -EAGAIN; + + return 0; +} + +/** + * pvrdma_alloc_pd - allocate protection domain + * @ibdev: the IB device + * @context: user context + * @udata: user data + * + * @return: the ib_pd protection domain pointer on success, otherwise errno. + */ +struct ib_pd *pvrdma_alloc_pd(struct ib_device *ibdev, + struct ib_ucontext *context, + struct ib_udata *udata) +{ + struct pvrdma_pd *pd; + struct pvrdma_dev *dev = to_vdev(ibdev); + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_create_pd *cmd = &req.create_pd; + struct pvrdma_cmd_create_pd_resp *resp = &rsp.create_pd_resp; + int ret; + + /* Check allowed max pds */ + if (!atomic_add_unless(&dev->num_pds, 1, dev->dsr->caps.max_pd)) + return ERR_PTR(-EINVAL); + + pd = kmalloc(sizeof(*pd), GFP_KERNEL); + if (!pd) { + atomic_dec(&dev->num_pds); + return ERR_PTR(-ENOMEM); + } + + pd->priviledged = !context; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_CREATE_PD; + cmd->ctx_handle = (context) ? to_vucontext(context)->ctx_handle : 0; + ret = pvrdma_cmd_post(dev, &req, true, &rsp); + if (ret < 0 || resp->hdr.ack != PVRDMA_CMD_CREATE_PD_RESP) { + dev_warn(&dev->pdev->dev, + "failed to allocate protection domain\n"); + kfree(pd); + atomic_dec(&dev->num_pds); + return ERR_PTR(-EFAULT); + } + + pd->pd_handle = resp->pd_handle; + pd->pdn = resp->pd_handle; + + if (context) { + if (ib_copy_to_udata(udata, &pd->pdn, sizeof(__u32))) { + dev_warn(&dev->pdev->dev, + "failed to copy back protection domain\n"); + pvrdma_dealloc_pd(&pd->ibpd); + atomic_dec(&dev->num_pds); + return ERR_PTR(-EFAULT); + } + } + + /* u32 pd handle */ + return &pd->ibpd; +} + +/** + * pvrdma_dealloc_pd - deallocate protection domain + * @pd: the protection domain to be released + * + * @return: 0 on success, otherwise errno. + */ +int pvrdma_dealloc_pd(struct ib_pd *pd) +{ + struct pvrdma_dev *dev = to_vdev(pd->device); + union pvrdma_cmd_req req; + struct pvrdma_cmd_destroy_pd *cmd = &req.destroy_pd; + int ret; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_DESTROY_PD; + cmd->pd_handle = to_vpd(pd)->pd_handle; + + ret = pvrdma_cmd_post(dev, &req, false, NULL); + if (ret) + dev_warn(&dev->pdev->dev, + "could not dealloc protection domain\n"); + + kfree(to_vpd(pd)); + atomic_dec(&dev->num_pds); + + return 0; +} + +/** + * pvrdma_create_ah - create an address handle + * @pd: the protection domain + * @ah_attr: the attributes of the AH + * + * @return: the ib_ah pointer on success, oterwise errno. + */ +struct ib_ah *pvrdma_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr) +{ + struct pvrdma_dev *dev = to_vdev(pd->device); + struct pvrdma_ah *ah; + enum rdma_link_layer ll; + + if (!(ah_attr->ah_flags & IB_AH_GRH)) + return ERR_PTR(-EINVAL); + + ll = rdma_port_get_link_layer(pd->device, ah_attr->port_num); + + if (ll != IB_LINK_LAYER_ETHERNET || + rdma_is_multicast_addr((struct in6_addr *)ah_attr->grh.dgid.raw)) + return ERR_PTR(-EINVAL); + + if (!atomic_add_unless(&dev->num_ahs, 1, dev->dsr->caps.max_ah)) + return ERR_PTR(-EINVAL); + + ah = kzalloc(sizeof(*ah), GFP_KERNEL); + if (!ah) { + atomic_dec(&dev->num_ahs); + return ERR_PTR(-ENOMEM); + } + + ah->av.port_pd = to_vpd(pd)->pd_handle | (ah_attr->port_num << 24); + ah->av.src_path_bits = ah_attr->src_path_bits; + ah->av.src_path_bits |= 0x80; + ah->av.gid_index = ah_attr->grh.sgid_index; + ah->av.hop_limit = ah_attr->grh.hop_limit; + ah->av.sl_tclass_flowlabel = (ah_attr->grh.traffic_class << 20) | + ah_attr->grh.flow_label; + memcpy(ah->av.dgid, ah_attr->grh.dgid.raw, 16); + memcpy(ah->av.dmac, ah_attr->dmac, 6); + + ah->ibah.device = pd->device; + ah->ibah.pd = pd; + ah->ibah.uobject = NULL; + + return &ah->ibah; +} + +/** + * pvrdma_destroy_ah - destroy an address handle + * @ah: the address handle to destroyed + * + * @return: 0 on success. + */ +int pvrdma_destroy_ah(struct ib_ah *ah) +{ + struct pvrdma_dev *dev = to_vdev(ah->device); + + kfree(to_vah(ah)); + atomic_dec(&dev->num_ahs); + + return 0; +} diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_verbs.h b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.h new file mode 100644 index 0000000..e52fbe5 --- /dev/null +++ b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.h @@ -0,0 +1,108 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __PVRDMA_VERBS_H__ +#define __PVRDMA_VERBS_H__ + +int pvrdma_query_device(struct ib_device *ibdev, + struct ib_device_attr *props, + struct ib_udata *udata); +int pvrdma_query_port(struct ib_device *ibdev, u8 port, + struct ib_port_attr *props); +int pvrdma_query_gid(struct ib_device *ibdev, u8 port, + int index, union ib_gid *gid); +int pvrdma_query_pkey(struct ib_device *ibdev, u8 port, + u16 index, u16 *pkey); +enum rdma_link_layer pvrdma_port_link_layer(struct ib_device *ibdev, + u8 port); +int pvrdma_modify_device(struct ib_device *ibdev, int mask, + struct ib_device_modify *props); +int pvrdma_modify_port(struct ib_device *ibdev, u8 port, + int mask, struct ib_port_modify *props); +int pvrdma_mmap(struct ib_ucontext *context, struct vm_area_struct *vma); +struct ib_ucontext *pvrdma_alloc_ucontext(struct ib_device *ibdev, + struct ib_udata *udata); +int pvrdma_dealloc_ucontext(struct ib_ucontext *context); +struct ib_pd *pvrdma_alloc_pd(struct ib_device *ibdev, + struct ib_ucontext *context, + struct ib_udata *udata); +int pvrdma_dealloc_pd(struct ib_pd *ibpd); +struct ib_mr *pvrdma_get_dma_mr(struct ib_pd *pd, int acc); +struct ib_mr *pvrdma_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, + u64 virt_addr, int access_flags, + struct ib_udata *udata); +int pvrdma_dereg_mr(struct ib_mr *mr); +struct ib_mr *pvrdma_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type, + u32 max_num_sg); +int pvrdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, + int sg_nents, unsigned int *sg_offset); +int pvrdma_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); +int pvrdma_resize_cq(struct ib_cq *ibcq, int entries, + struct ib_udata *udata); +struct ib_cq *pvrdma_create_cq(struct ib_device *ibdev, + const struct ib_cq_init_attr *attr, + struct ib_ucontext *context, + struct ib_udata *udata); +int pvrdma_resize_cq(struct ib_cq *ibcq, int entries, + struct ib_udata *udata); +int pvrdma_destroy_cq(struct ib_cq *cq); +int pvrdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); +int pvrdma_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); +struct ib_ah *pvrdma_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr); +int pvrdma_destroy_ah(struct ib_ah *ah); +struct ib_qp *pvrdma_create_qp(struct ib_pd *pd, + struct ib_qp_init_attr *init_attr, + struct ib_udata *udata); +int pvrdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, + int attr_mask, struct ib_udata *udata); +int pvrdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, + int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr); +int pvrdma_destroy_qp(struct ib_qp *qp); +int pvrdma_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, + struct ib_send_wr **bad_wr); +int pvrdma_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr, + struct ib_recv_wr **bad_wr); + +#endif /* __PVRDMA_VERBS_H__ */