From patchwork Sat Sep 24 23:21:29 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adit Ranadive X-Patchwork-Id: 9349347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 75064601C2 for ; Sat, 24 Sep 2016 23:23:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 624E6290C3 for ; Sat, 24 Sep 2016 23:23:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 56AA8290CB; Sat, 24 Sep 2016 23:23:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 02D98290C3 for ; Sat, 24 Sep 2016 23:23:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030206AbcIXXWf (ORCPT ); Sat, 24 Sep 2016 19:22:35 -0400 Received: from ex13-edg-ou-001.vmware.com ([208.91.0.189]:24765 "EHLO EX13-EDG-OU-001.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030220AbcIXXWX (ORCPT ); Sat, 24 Sep 2016 19:22:23 -0400 Received: from sc9-mailhost3.vmware.com (10.113.161.73) by EX13-EDG-OU-001.vmware.com (10.113.208.155) with Microsoft SMTP Server id 15.0.1156.6; Sat, 24 Sep 2016 16:21:56 -0700 Received: from EX13-CAS-001.vmware.com (ex13-cas-001.vmware.com [10.113.191.51]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id D824C404B5; Sat, 24 Sep 2016 16:22:22 -0700 (PDT) Received: from EX13-CAS-003.vmware.com (10.113.191.53) by EX13-MBX-020.vmware.com (10.113.191.40) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Sat, 24 Sep 2016 16:22:22 -0700 Received: from NAM03-CO1-obe.outbound.protection.outlook.com (10.113.170.11) by EX13-CAS-003.vmware.com (10.113.191.53) with Microsoft SMTP Server (TLS) id 15.0.1156.6 via Frontend Transport; Sat, 24 Sep 2016 16:22:22 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=onevmw.onmicrosoft.com; s=selector1-vmware-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=uGwWw8ijfgrblNBMpnpV+PXOq6coF3VLCEbz+YljUPU=; b=YCwP2ZfkZ++MnnxmoZ+K1aDj3sxzgFCrI1kJ5KsF1VFgZLAC9Np7+4CUg/reAm2WshLhKxgVyWr32ZIoO52liSxTsUht9kNTayduE+rrAPteg3KCG9rWJ8TTovBWSj2NBFjx/Irb0lWARXzgNQEYDvrz2OMWG1FLyQKhYySyTkI= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aditr@vmware.com; Received: from promb-2s-dhcp95-136.eng.vmware.com (208.91.1.34) by CO2PR0501MB837.namprd05.prod.outlook.com (10.141.244.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.639.2; Sat, 24 Sep 2016 23:22:14 +0000 From: Adit Ranadive To: , , CC: Adit Ranadive , , , , , , Subject: [PATCH v5 05/16] IB/pvrdma: Add functions for Verbs support Date: Sat, 24 Sep 2016 16:21:29 -0700 Message-ID: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [208.91.1.34] X-ClientProxiedBy: CY1PR18CA0027.namprd18.prod.outlook.com (10.163.31.37) To CO2PR0501MB837.namprd05.prod.outlook.com (10.141.244.147) X-MS-Office365-Filtering-Correlation-Id: 1224f29b-5cbf-4bbd-be68-08d3e4d19f47 X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 2:pUQ4OARZRTEQzyJk/ckTY6H4Rl5Uvtgh2zyPhWb4ckPw4zFlGIlM4v9Xl7OJYbmLUogGsHS4fBL90mUZ0lZ+se7u2yWYpXhKSmyDRHUauyf4vIzuu6DSEevQhBabOSiuGz0zJ1vraZVDfpgAefzr/vctInw+PMntXbFmfH5gAhyBIPFdAuSCQp0InNEKkJvT; 3:DSRxF+5efmyXON54cMBce/ghCYZytdUSYAho0FVLfTp6AZ4kosxeGawPixm+jctfBZbO2A0/i+i0Q3mJPDor49H9/7LN07f+cFEFXyqhJ7AVmF0gDRctlQzNbY4HlD43 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CO2PR0501MB837; X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 25:nSszAZYQxXXIKGVTahYS21k6+VP40Vc5nY/sLNBzqjDVx00IkbVdouSAslE02c4pnJRwWM9xsrraD3P3Gup0JZWAWpLi7/+4ZpJN5mJ3HvqggLFCCOaVK/jMgh04CrxTb1boNbN9OvetuWPLd5ZQkbr6zl7SO0S+eBWYrL3NKyiQGYOsoPxkgbe4kiqBSGIt2adr2esWXAIUQG7KMEGPufhhATeqEC+4F6/ZnacQaWkpj5BQQiGTXeaNhipFcMeq0MjtJ0SRBa0vPXxVxAwlBFn6Z56VsC2Eok3RfVldhys4KuwV4aLgawZevxqAkiS6sBbk3BHkgi6NCk2SWjVRSo4s2u8Vx4pA50Z5BEuKlxsmm9CCY0o3Ic+CjFacHIoEdbnhcWHexskKL+W/Vi5aQfBaJI5ZOXLqqfjmymSajssTXrKtEr5PLQ7KGLei/mmtQ4333OgncqDgjclTN4/dh3LVxntDzu5z6KJFpnB6ghyeUqVQtuwnA9lAA8/8Yx1gD7w8llJxFyKA28EZkrybbU3Fe9dO6dOglZ/HrCvpxhs3eH2Yia7AsFUqZnK15b/EDGdoWG2ou1jH9NNPjDr7EoTR78JFSm+x09u+zMgduG2YKir7OKOsFKDQ7jSGbBdbesY3ZuAhJdIxQlYqrohp+XlS85+16J5CNNvtIbRSgXsEM430NhCwabdqxAygo0QmTOrePgFCdexs9wbx76Dg6fqQp33NrXqemSh31MneQjTYPHAqTbnbhWZSy+EZXBTn9Mln5UtXJjfTsp+YByRY/nxLNOXfUUlLJTAggDzFBQMzdUa+E/TUHM4ozmvHbjQMo7YOqAAord1cjjJsERRmxQ== X-LD-Processed: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 31:PuZDRar4hWC9sHkvzE3DU/wYrh1pt47cm3BifdRaMkyjTGgTKheYUiHl9ANiJJFvDEhYu7Sf44Y59VLYiEjSzd1p73WvBD5dKWDwWPX/L5C/94KwUqfFSmNuzHNamI9LhIO+ZOa+Poe6SEieXfiq0g80nnzAaFkrcd5V6eyz9N6Tt8OFB8pQDVOL3IWIkvwVs60SrUMguuPsoPdcobQt9WXPr0S99OZYn00K70dYPZU=; 20:geDpOHiJGViCLVqBAVOSMpJvZ9HWMlwVKf9mgK0KMemd5oIeAqUZwm4LnIMTswrwnSES5PB2QfIEIBLoJTGZyJaKIl18CahlAhfjGzXUyeYibIMs8y30ajN7KpK6WDi7gKCpdGf8F1RYksQS95Ak2NcBRtrbJzc1mWIiqjz7QtIJGH1BS9Fjmu1vn+FwVXxBYY6h+ZwfUNosRR9C5EJHSFJQiB/gsQbJ6TCvyZJnvHM0SQJ69hOZ+xan8MTuwnEIPr17/DadS5tTg4t9VFfqw9xapCtVfSCu0Vb/j+4fW5XFXW4gYJN69999tDBboR4Rf0BQw9C6HispuTiwGXHu++VT4F8Tqv0qMpkay6g3B7w37/FFM8GHnq6mLdXWOvhm/nunoRCTgxDmLlSwXL7DwTv8RPlmn9ElietxLTfVcQWEWMnTk50buPMsWjiyBZMf3TrvlVXyzUj3Tv7WuRzVOQ/YY+Xf6QGa053BLEjfqRrSQNE2K2ZZFFYVajS39Msc X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(250305191791016)(61668805478150)(22074186197030); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:CO2PR0501MB837; BCL:0; PCL:0; RULEID:; SRVR:CO2PR0501MB837; X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 4:BOOdygl2YdkzPAY3GRs2hm6LukJT41z0lQA57WAP+y04Cd8ELiokqY/CUgeDx+3CVszDduClroFbRmWmgN8m2dg8KPidEoUdxCI9rHOC8yI1oNLg89CcA9EBw9ndWtY1z3Lo1OwJYo4IO0TmEoaOrdikTkcRsUDmUfkN6ItFq6nXKHgeCdqwIP9d+vP+4lGiSDSWicTd1U3rIuGF3XQOL6abzkXS1EdIWO+MrUX6rddHHoeccbKUI42iuWZ+uj8KxdNvrv6WwJMxknADvJZX1+eB4FNG8jk65tW4+6eM3IV2g72MIjGjrNMzj4EudeMfGLWmILoldRnf/oNRTj4mY3rb5OUR7iGaZsfwg63iqH0PGbD2epcxWetdYqlGu83qGXsP3r/30+V2WNggL+0XllsiimoQ26R8tl/M0z65svZ8RFWuMblW3oNlWt27BC0/BJQzzKuQK/hnufX8+O5bEhJTKTTRA03m1zz8zVfT0d3umpICYRf8KlGNgyzePL+C X-Forefront-PRVS: 0075CB064E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(107886002)(77096005)(97736004)(4326007)(106356001)(105586002)(15975445007)(76176999)(586003)(6862003)(118296001)(50986999)(2950100002)(33646002)(6116002)(5001770100001)(19580395003)(5660300001)(19580405001)(92566002)(2906002)(3846002)(47776003)(189998001)(50466002)(86362001)(48376002)(2201001)(81156014)(81166006)(101416001)(8676002)(50226002)(5003940100001)(36756003)(7736002)(7846002)(305945005)(229853001)(42186005)(68736007)(66066001)(4001430100002)(7099028)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:CO2PR0501MB837; H:promb-2s-dhcp95-136.eng.vmware.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (EX13-EDG-OU-001.vmware.com: aditr@vmware.com does not designate permitted sender hosts) Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CO2PR0501MB837; 23:3/8V0giEPRHF53o9AvOzcrt0xmOCnkEEhEftbryC?= =?us-ascii?Q?96ZEq6jVqnDC3qqTBA9CvUieJXIl9Yb+CDPAYYhyAN5+rSz5/t3NAeZ4NWud?= =?us-ascii?Q?bSAYddhbwQzf3WfSiILHzRsotJVNrXghq124KIKuQ6KeDvF1pUlnKcQoEXnL?= =?us-ascii?Q?lrnuA9RSexWmfien+glO2NnAYMepP215BPzm83tdwko1lAg/OmWne8ohgFfH?= =?us-ascii?Q?B6KcUbsCGp6bIUp2pU79eep6qMuHgHuhy7ozEULy8YmZwYEPv3ta/vBiaiJi?= =?us-ascii?Q?T+oW4kFvB1NUZ22ibHKB+Lr8AIsvE7n/0vGDbEy2xSeQ5BD+L7U0I5TT2buq?= =?us-ascii?Q?yNb0k3EWwyjLEUnjMK6wSigo/6TdjqMHeoI2efksNmjjAeW/J5IurZDHrsV6?= =?us-ascii?Q?l2RbRl7kkriRzKgyyvzjuEjLvfXcAxzuSWB0nasvB4wlxAxVB3veNU7s2O8G?= =?us-ascii?Q?WFPzLq52J0cdkal5BmevjtQSOZxVRknGkHdGRbdZpIe4j0ZVZff+LeCYlr6E?= =?us-ascii?Q?0SGV2WH6JgfouV2VptPt+bnUOl4cBJxRRwbbYxF1kaxTnWjhkE7AnzD2CaYO?= =?us-ascii?Q?BGBo5a8mfd76oWdiIVt2bv3qSZD9ffP1niwOKkjQ0Zq9T7hMlV9LqJs4m+bU?= =?us-ascii?Q?myhsy/TWNCyeS3AjESyRYYwMbID36ZmwIIUMiUZ+3fm33U66JIsCqjkvURud?= =?us-ascii?Q?pe1ZnvPkZ+RTXTmOPBo2IPoA77gx9Jr+yxBDMaQCaVmhDaxFucaOX7Sxy/J3?= =?us-ascii?Q?FgB6yvqeNWpilEMsWNXxOjVaX5y5ay9k0XRevk2u+n/6bGBJ0WVKu98lxg2C?= =?us-ascii?Q?mqDq6DAZWtqj+wrT9qq9ml7PoQBT7T1w7Jdj83u8C98EO0a0DqvrXCdLbk7R?= =?us-ascii?Q?0/0iN3K9UYzv0M2hKMtBUOq772CqYOjhybHnotC8ensrxBWqZ6deawHAclN4?= =?us-ascii?Q?PUaU5z4ck3yklhgr+QhH1AwWrEoa5GGmjoI82MDcclbv4+k4NRI6IrqjvwyZ?= =?us-ascii?Q?DEqxc+zGRLjzefDV5bjLSZkd8rAasevGpDHC8eu2CxP1zpTHmPmd4UsXayFm?= =?us-ascii?Q?7fo1s57lvALLWOyhMeWzvF25HAtsMpSha56w0PP9yuqMC+LvwluPYQTDD+uI?= =?us-ascii?Q?uQn+qJztZletWsqfdlkLhU60JoCSnUxspjTLXkcdV6NwfZgk3l2n++AoI97o?= =?us-ascii?Q?HYLLOaecQSD3U5UmEDx5Y70SBId6c0F+s/TBKmveN0g4MnEDaN6v2shI2A?= =?us-ascii?Q?=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 6:hz+61aZKDZkWrvRNnGHYT0aAp0ynmbtP1gMvThq807zs4wEz1mtz2TstAsLFLESQXaomVk0tQ+XszlkQ81lDp5RvlfMkH9an1ll/Ex2Y6gIaWUe1UXesarT29CnQbKOswK/5JIlVILJ9gO03JHRkjti2QzowP05JrSpjReWPYNaD2PZdi6CkyiR4nbIMWcTdbun/y6wW3H5mt7n4uDrS2z4M2rwJEIlaDVkAEtXXxCbL+uUX4bqnkMl10HwUwmWotIsSaGMgVjFQolu6DV5lKIlCcCd6c0AE0Yd2k/ODcX4=; 5:ClMKvjye6hWJQvHQ+dax3nLeSr83Bj4HmKlIEk0WV8GKtOT/niBzAo2bJ4W5Y7t4pyivLj1hEc5MLEa+naLb60CIy7joiXvMmEvDh5LHKLwAx9iTquqLVqZpxF9TKeDhtmpeZnd+VfRHo/bxZnyNLw==; 24:ixtbkN44DBh08Q7dJW/0tof1t6vZuBaW+eahgW4CBf61xSvK+2DCVocKYmVZfz81khmxSceXpPZprByNmsAUNMoHFOTXZ7GxzDVmw21iahQ=; 7:7hUcclKkBnLpiZtqLdZalc1Fl+5pxV421GwsLmj/JyORCPg6mOtc2pv2TWpG4TJ9rDUCLGHU+bKVL6GX4bsfyvPXmt4mbaQXsVYlBTxXPj6YrLZQVGEvJK9Zaw0CiVxRh7YOFh2VOzyCD+IhVDkFzT7uxBVHMitPnMU3L2gpz2+ntvnGvuTQlCwdfLddy9FjrxvQVIcY/w8TCogT6MVvAq//Q8k0c2X5PiaDEFdnmvm/uEWLlf8OfGAee/MkFytgwWEjh3nJLO82P8UQG/jYs5wokoU/uGFlhS7htRrrX4SK7D3mT7CkMUrcTNda1uFB SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CO2PR0501MB837; 20:QXI3VPEW1bNFboCKUqoLCFcXG/RyTJjYqsglN7vBn7yFDpsDxj1K1uegl/aAO6u3iHa/eSsKk0cHeyLiopWErNn3C9SFWBlOaTgSEfCoB1mhZqhniFo5PEckLriyJMCYXLAK+GqXjVxU2qhkXPsqrRA23EjKGr/M26SgqFwooQc= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2016 23:22:14.8619 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR0501MB837 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch implements the remaining Verbs functions registered with the core RDMA stack. Reviewed-by: Jorgen Hansen Reviewed-by: George Zhang Reviewed-by: Aditya Sarwade Reviewed-by: Bryan Tan Signed-off-by: Adit Ranadive --- Changes v4->v5: - Update include for headers in UAPI folder. - Removed setting any properties that are reported by device as 0. - Simplified modify_port. - PD should be allocated first in kernel then in device. - Update to pvrdma_cmd_post for creating/destroying PD, Query port/device. Changes v3->v4: - Renamed priviledged -> privileged. - Added error numbers for command errors. - Removed unnecessary goto in modify_device. - Moved pd allocation to after command execution. - Removed an incorrect atomic_dec. --- drivers/infiniband/hw/pvrdma/pvrdma_verbs.c | 577 ++++++++++++++++++++++++++++ drivers/infiniband/hw/pvrdma/pvrdma_verbs.h | 108 ++++++ 2 files changed, 685 insertions(+) create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_verbs.c create mode 100644 drivers/infiniband/hw/pvrdma/pvrdma_verbs.h diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_verbs.c b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.c new file mode 100644 index 0000000..a7aef93d --- /dev/null +++ b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.c @@ -0,0 +1,577 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "pvrdma.h" + +/** + * pvrdma_query_device - query device + * @ibdev: the device to query + * @props: the device properties + * @uhw: user data + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_device(struct ib_device *ibdev, + struct ib_device_attr *props, + struct ib_udata *uhw) +{ + struct pvrdma_dev *dev = to_vdev(ibdev); + + if (uhw->inlen || uhw->outlen) + return -EINVAL; + + memset(props, 0, sizeof(*props)); + + props->fw_ver = dev->dsr->caps.fw_ver; + props->sys_image_guid = dev->dsr->caps.sys_image_guid; + props->max_mr_size = dev->dsr->caps.max_mr_size; + props->page_size_cap = dev->dsr->caps.page_size_cap; + props->vendor_id = dev->dsr->caps.vendor_id; + props->vendor_part_id = dev->pdev->device; + props->hw_ver = dev->dsr->caps.hw_ver; + props->max_qp = dev->dsr->caps.max_qp; + props->max_qp_wr = dev->dsr->caps.max_qp_wr; + props->device_cap_flags = dev->dsr->caps.device_cap_flags; + props->max_sge = dev->dsr->caps.max_sge; + props->max_cq = dev->dsr->caps.max_cq; + props->max_cqe = dev->dsr->caps.max_cqe; + props->max_mr = dev->dsr->caps.max_mr; + props->max_pd = dev->dsr->caps.max_pd; + props->max_qp_rd_atom = dev->dsr->caps.max_qp_rd_atom; + props->max_qp_init_rd_atom = dev->dsr->caps.max_qp_init_rd_atom; + props->atomic_cap = + dev->dsr->caps.atomic_ops & + (PVRDMA_ATOMIC_OP_COMP_SWAP | PVRDMA_ATOMIC_OP_FETCH_ADD) ? + IB_ATOMIC_HCA : IB_ATOMIC_NONE; + props->masked_atomic_cap = props->atomic_cap; + props->max_ah = dev->dsr->caps.max_ah; + props->max_pkeys = dev->dsr->caps.max_pkeys; + props->local_ca_ack_delay = dev->dsr->caps.local_ca_ack_delay; + if ((dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_LOCAL_INV) && + (dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_REMOTE_INV) && + (dev->dsr->caps.bmme_flags & PVRDMA_BMME_FLAG_FAST_REG_WR)) { + props->device_cap_flags |= IB_DEVICE_MEM_MGT_EXTENSIONS; + } + + return 0; +} + +/** + * pvrdma_query_port - query device port attributes + * @ibdev: the device to query + * @port: the port number + * @props: the device properties + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_port(struct ib_device *ibdev, u8 port, + struct ib_port_attr *props) +{ + struct pvrdma_dev *dev = to_vdev(ibdev); + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_query_port *cmd = &req.query_port; + struct pvrdma_cmd_query_port_resp *resp = &rsp.query_port_resp; + int err; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_QUERY_PORT; + cmd->port_num = port; + + err = pvrdma_cmd_post(dev, &req, &rsp, PVRDMA_CMD_QUERY_PORT_RESP); + if (err < 0) { + dev_warn(&dev->pdev->dev, + "could not query port, error: %d\n", err); + return err; + } + + memset(props, 0, sizeof(*props)); + + props->state = pvrdma_port_state_to_ib(resp->attrs.state); + props->max_mtu = pvrdma_mtu_to_ib(resp->attrs.max_mtu); + props->active_mtu = pvrdma_mtu_to_ib(resp->attrs.active_mtu); + props->gid_tbl_len = resp->attrs.gid_tbl_len; + props->port_cap_flags = + pvrdma_port_cap_flags_to_ib(resp->attrs.port_cap_flags); + props->max_msg_sz = resp->attrs.max_msg_sz; + props->bad_pkey_cntr = resp->attrs.bad_pkey_cntr; + props->qkey_viol_cntr = resp->attrs.qkey_viol_cntr; + props->pkey_tbl_len = resp->attrs.pkey_tbl_len; + props->lid = resp->attrs.lid; + props->sm_lid = resp->attrs.sm_lid; + props->lmc = resp->attrs.lmc; + props->max_vl_num = resp->attrs.max_vl_num; + props->sm_sl = resp->attrs.sm_sl; + props->subnet_timeout = resp->attrs.subnet_timeout; + props->init_type_reply = resp->attrs.init_type_reply; + props->active_width = pvrdma_port_width_to_ib(resp->attrs.active_width); + props->active_speed = pvrdma_port_speed_to_ib(resp->attrs.active_speed); + props->phys_state = resp->attrs.phys_state; + + return 0; +} + +/** + * pvrdma_query_gid - query device gid + * @ibdev: the device to query + * @port: the port number + * @index: the index + * @gid: the device gid value + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_gid(struct ib_device *ibdev, u8 port, int index, + union ib_gid *gid) +{ + struct pvrdma_dev *dev = to_vdev(ibdev); + + if (index >= dev->dsr->caps.gid_tbl_len) + return -EINVAL; + + memcpy(gid, &dev->sgid_tbl[index], sizeof(union ib_gid)); + + return 0; +} + +/** + * pvrdma_query_pkey - query device port's P_Key table + * @ibdev: the device to query + * @port: the port number + * @index: the index + * @pkey: the device P_Key value + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_query_pkey(struct ib_device *ibdev, u8 port, u16 index, + u16 *pkey) +{ + int err = 0; + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_query_pkey *cmd = &req.query_pkey; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_QUERY_PKEY; + cmd->port_num = port; + cmd->index = index; + + err = pvrdma_cmd_post(to_vdev(ibdev), &req, &rsp, + PVRDMA_CMD_QUERY_PKEY_RESP); + if (err < 0) { + dev_warn(&to_vdev(ibdev)->pdev->dev, + "could not query pkey, error: %d\n", err); + return err; + } + + *pkey = rsp.query_pkey_resp.pkey; + + return 0; +} + +enum rdma_link_layer pvrdma_port_link_layer(struct ib_device *ibdev, + u8 port) +{ + return IB_LINK_LAYER_ETHERNET; +} + +int pvrdma_modify_device(struct ib_device *ibdev, int mask, + struct ib_device_modify *props) +{ + unsigned long flags; + + if (mask & ~(IB_DEVICE_MODIFY_SYS_IMAGE_GUID | + IB_DEVICE_MODIFY_NODE_DESC)) { + dev_warn(&to_vdev(ibdev)->pdev->dev, + "unsupported device modify mask %#x\n", mask); + return -EOPNOTSUPP; + } + + if (mask & IB_DEVICE_MODIFY_NODE_DESC) { + spin_lock_irqsave(&to_vdev(ibdev)->desc_lock, flags); + memcpy(ibdev->node_desc, props->node_desc, 64); + spin_unlock_irqrestore(&to_vdev(ibdev)->desc_lock, flags); + } + + if (mask & IB_DEVICE_MODIFY_SYS_IMAGE_GUID) { + mutex_lock(&to_vdev(ibdev)->port_mutex); + to_vdev(ibdev)->sys_image_guid = + cpu_to_be64(props->sys_image_guid); + mutex_unlock(&to_vdev(ibdev)->port_mutex); + } + + return 0; +} + +/** + * pvrdma_modify_port - modify device port attributes + * @ibdev: the device to modify + * @port: the port number + * @mask: attributes to modify + * @props: the device properties + * + * @return: 0 on success, otherwise negative errno + */ +int pvrdma_modify_port(struct ib_device *ibdev, u8 port, int mask, + struct ib_port_modify *props) +{ + struct ib_port_attr attr; + struct pvrdma_dev *vdev = to_vdev(ibdev); + int ret; + + if (mask & ~IB_PORT_SHUTDOWN) { + dev_warn(&vdev->pdev->dev, + "unsupported port modify mask %#x\n", mask); + return -EOPNOTSUPP; + } + + mutex_lock(&vdev->port_mutex); + ret = pvrdma_query_port(ibdev, port, &attr); + if (ret) + goto out; + + vdev->port_cap_mask |= props->set_port_cap_mask; + vdev->port_cap_mask &= ~props->clr_port_cap_mask; + + if (mask & IB_PORT_SHUTDOWN) + vdev->ib_active = false; + +out: + mutex_unlock(&vdev->port_mutex); + return ret; +} + +/** + * pvrdma_alloc_ucontext - allocate ucontext + * @ibdev: the IB device + * @udata: user data + * + * @return: the ib_ucontext pointer on success, otherwise errno. + */ +struct ib_ucontext *pvrdma_alloc_ucontext(struct ib_device *ibdev, + struct ib_udata *udata) +{ + struct pvrdma_dev *vdev = to_vdev(ibdev); + struct pvrdma_ucontext *context; + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_create_uc *cmd = &req.create_uc; + struct pvrdma_cmd_create_uc_resp *resp = &rsp.create_uc_resp; + struct pvrdma_alloc_ucontext_resp uresp; + int ret; + void *ptr; + + if (!vdev->ib_active) + return ERR_PTR(-EAGAIN); + + context = kmalloc(sizeof(*context), GFP_KERNEL); + if (!context) + return ERR_PTR(-ENOMEM); + + context->dev = vdev; + ret = pvrdma_uar_alloc(vdev, &context->uar); + if (ret) { + kfree(context); + return ERR_PTR(-ENOMEM); + } + + /* get ctx_handle from host */ + memset(cmd, 0, sizeof(*cmd)); + cmd->pfn = context->uar.pfn; + cmd->hdr.cmd = PVRDMA_CMD_CREATE_UC; + ret = pvrdma_cmd_post(vdev, &req, &rsp, PVRDMA_CMD_CREATE_UC_RESP); + if (ret < 0) { + dev_warn(&vdev->pdev->dev, + "could not create ucontext, error: %d\n", ret); + ptr = ERR_PTR(ret); + goto err; + } + + context->ctx_handle = resp->ctx_handle; + + /* copy back to user */ + uresp.qp_tab_size = vdev->dsr->caps.max_qp; + ret = ib_copy_to_udata(udata, &uresp, sizeof(uresp)); + if (ret) { + pvrdma_uar_free(vdev, &context->uar); + context->ibucontext.device = ibdev; + pvrdma_dealloc_ucontext(&context->ibucontext); + return ERR_PTR(-EFAULT); + } + + return &context->ibucontext; + +err: + pvrdma_uar_free(vdev, &context->uar); + kfree(context); + return ptr; +} + +/** + * pvrdma_dealloc_ucontext - deallocate ucontext + * @ibcontext: the ucontext + * + * @return: 0 on success, otherwise errno. + */ +int pvrdma_dealloc_ucontext(struct ib_ucontext *ibcontext) +{ + struct pvrdma_ucontext *context = to_vucontext(ibcontext); + union pvrdma_cmd_req req; + struct pvrdma_cmd_destroy_uc *cmd = &req.destroy_uc; + int ret; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_DESTROY_UC; + cmd->ctx_handle = context->ctx_handle; + + ret = pvrdma_cmd_post(context->dev, &req, NULL, 0); + if (ret < 0) + dev_warn(&context->dev->pdev->dev, + "destroy ucontext failed, error: %d\n", ret); + + /* Free the UAR even if the device command failed */ + pvrdma_uar_free(to_vdev(ibcontext->device), &context->uar); + kfree(context); + + return ret; +} + +/** + * pvrdma_mmap - create mmap region + * @ibcontext: the user context + * @vma: the VMA + * + * @return: 0 on success, otherwise errno. + */ +int pvrdma_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma) +{ + struct pvrdma_ucontext *context = to_vucontext(ibcontext); + unsigned long start = vma->vm_start; + unsigned long size = vma->vm_end - vma->vm_start; + unsigned long offset = vma->vm_pgoff << PAGE_SHIFT; + + dev_dbg(&context->dev->pdev->dev, "create mmap region\n"); + + if ((size != PAGE_SIZE) || (offset & ~PAGE_MASK)) { + dev_warn(&context->dev->pdev->dev, + "invalid params for mmap region\n"); + return -EINVAL; + } + + /* Map UAR to kernel space, VM_LOCKED? */ + vma->vm_flags |= VM_DONTCOPY | VM_DONTEXPAND; + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + if (io_remap_pfn_range(vma, start, context->uar.pfn, size, + vma->vm_page_prot)) + return -EAGAIN; + + return 0; +} + +/** + * pvrdma_alloc_pd - allocate protection domain + * @ibdev: the IB device + * @context: user context + * @udata: user data + * + * @return: the ib_pd protection domain pointer on success, otherwise errno. + */ +struct ib_pd *pvrdma_alloc_pd(struct ib_device *ibdev, + struct ib_ucontext *context, + struct ib_udata *udata) +{ + struct pvrdma_pd *pd; + struct pvrdma_dev *dev = to_vdev(ibdev); + union pvrdma_cmd_req req; + union pvrdma_cmd_resp rsp; + struct pvrdma_cmd_create_pd *cmd = &req.create_pd; + struct pvrdma_cmd_create_pd_resp *resp = &rsp.create_pd_resp; + int ret; + void *ptr; + + /* Check allowed max pds */ + if (!atomic_add_unless(&dev->num_pds, 1, dev->dsr->caps.max_pd)) + return ERR_PTR(-ENOMEM); + + pd = kmalloc(sizeof(*pd), GFP_KERNEL); + if (!pd) { + ptr = ERR_PTR(-ENOMEM); + goto err; + } + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_CREATE_PD; + cmd->ctx_handle = (context) ? to_vucontext(context)->ctx_handle : 0; + ret = pvrdma_cmd_post(dev, &req, &rsp, PVRDMA_CMD_CREATE_PD_RESP); + if (ret < 0) { + dev_warn(&dev->pdev->dev, + "failed to allocate protection domain, error: %d\n", + ret); + ptr = ERR_PTR(ret); + goto freepd; + } + + pd->privileged = !context; + pd->pd_handle = resp->pd_handle; + pd->pdn = resp->pd_handle; + + if (context) { + if (ib_copy_to_udata(udata, &pd->pdn, sizeof(__u32))) { + dev_warn(&dev->pdev->dev, + "failed to copy back protection domain\n"); + pvrdma_dealloc_pd(&pd->ibpd); + return ERR_PTR(-EFAULT); + } + } + + /* u32 pd handle */ + return &pd->ibpd; + +freepd: + kfree(pd); +err: + atomic_dec(&dev->num_pds); + return ptr; +} + +/** + * pvrdma_dealloc_pd - deallocate protection domain + * @pd: the protection domain to be released + * + * @return: 0 on success, otherwise errno. + */ +int pvrdma_dealloc_pd(struct ib_pd *pd) +{ + struct pvrdma_dev *dev = to_vdev(pd->device); + union pvrdma_cmd_req req; + struct pvrdma_cmd_destroy_pd *cmd = &req.destroy_pd; + int ret; + + memset(cmd, 0, sizeof(*cmd)); + cmd->hdr.cmd = PVRDMA_CMD_DESTROY_PD; + cmd->pd_handle = to_vpd(pd)->pd_handle; + + ret = pvrdma_cmd_post(dev, &req, NULL, 0); + if (ret) + dev_warn(&dev->pdev->dev, + "could not dealloc protection domain, error: %d\n", + ret); + + kfree(to_vpd(pd)); + atomic_dec(&dev->num_pds); + + return 0; +} + +/** + * pvrdma_create_ah - create an address handle + * @pd: the protection domain + * @ah_attr: the attributes of the AH + * + * @return: the ib_ah pointer on success, otherwise errno. + */ +struct ib_ah *pvrdma_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr) +{ + struct pvrdma_dev *dev = to_vdev(pd->device); + struct pvrdma_ah *ah; + enum rdma_link_layer ll; + + if (!(ah_attr->ah_flags & IB_AH_GRH)) + return ERR_PTR(-EINVAL); + + ll = rdma_port_get_link_layer(pd->device, ah_attr->port_num); + + if (ll != IB_LINK_LAYER_ETHERNET || + rdma_is_multicast_addr((struct in6_addr *)ah_attr->grh.dgid.raw)) + return ERR_PTR(-EINVAL); + + if (!atomic_add_unless(&dev->num_ahs, 1, dev->dsr->caps.max_ah)) + return ERR_PTR(-ENOMEM); + + ah = kzalloc(sizeof(*ah), GFP_KERNEL); + if (!ah) { + atomic_dec(&dev->num_ahs); + return ERR_PTR(-ENOMEM); + } + + ah->av.port_pd = to_vpd(pd)->pd_handle | (ah_attr->port_num << 24); + ah->av.src_path_bits = ah_attr->src_path_bits; + ah->av.src_path_bits |= 0x80; + ah->av.gid_index = ah_attr->grh.sgid_index; + ah->av.hop_limit = ah_attr->grh.hop_limit; + ah->av.sl_tclass_flowlabel = (ah_attr->grh.traffic_class << 20) | + ah_attr->grh.flow_label; + memcpy(ah->av.dgid, ah_attr->grh.dgid.raw, 16); + memcpy(ah->av.dmac, ah_attr->dmac, 6); + + ah->ibah.device = pd->device; + ah->ibah.pd = pd; + ah->ibah.uobject = NULL; + + return &ah->ibah; +} + +/** + * pvrdma_destroy_ah - destroy an address handle + * @ah: the address handle to destroyed + * + * @return: 0 on success. + */ +int pvrdma_destroy_ah(struct ib_ah *ah) +{ + struct pvrdma_dev *dev = to_vdev(ah->device); + + kfree(to_vah(ah)); + atomic_dec(&dev->num_ahs); + + return 0; +} diff --git a/drivers/infiniband/hw/pvrdma/pvrdma_verbs.h b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.h new file mode 100644 index 0000000..e52fbe5 --- /dev/null +++ b/drivers/infiniband/hw/pvrdma/pvrdma_verbs.h @@ -0,0 +1,108 @@ +/* + * Copyright (c) 2012-2016 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of EITHER the GNU General Public License + * version 2 as published by the Free Software Foundation or the BSD + * 2-Clause License. This program is distributed in the hope that it + * will be useful, but WITHOUT ANY WARRANTY; WITHOUT EVEN THE IMPLIED + * WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + * See the GNU General Public License version 2 for more details at + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html. + * + * You should have received a copy of the GNU General Public License + * along with this program available in the file COPYING in the main + * directory of this source tree. + * + * The BSD 2-Clause License + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __PVRDMA_VERBS_H__ +#define __PVRDMA_VERBS_H__ + +int pvrdma_query_device(struct ib_device *ibdev, + struct ib_device_attr *props, + struct ib_udata *udata); +int pvrdma_query_port(struct ib_device *ibdev, u8 port, + struct ib_port_attr *props); +int pvrdma_query_gid(struct ib_device *ibdev, u8 port, + int index, union ib_gid *gid); +int pvrdma_query_pkey(struct ib_device *ibdev, u8 port, + u16 index, u16 *pkey); +enum rdma_link_layer pvrdma_port_link_layer(struct ib_device *ibdev, + u8 port); +int pvrdma_modify_device(struct ib_device *ibdev, int mask, + struct ib_device_modify *props); +int pvrdma_modify_port(struct ib_device *ibdev, u8 port, + int mask, struct ib_port_modify *props); +int pvrdma_mmap(struct ib_ucontext *context, struct vm_area_struct *vma); +struct ib_ucontext *pvrdma_alloc_ucontext(struct ib_device *ibdev, + struct ib_udata *udata); +int pvrdma_dealloc_ucontext(struct ib_ucontext *context); +struct ib_pd *pvrdma_alloc_pd(struct ib_device *ibdev, + struct ib_ucontext *context, + struct ib_udata *udata); +int pvrdma_dealloc_pd(struct ib_pd *ibpd); +struct ib_mr *pvrdma_get_dma_mr(struct ib_pd *pd, int acc); +struct ib_mr *pvrdma_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, + u64 virt_addr, int access_flags, + struct ib_udata *udata); +int pvrdma_dereg_mr(struct ib_mr *mr); +struct ib_mr *pvrdma_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type, + u32 max_num_sg); +int pvrdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, + int sg_nents, unsigned int *sg_offset); +int pvrdma_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period); +int pvrdma_resize_cq(struct ib_cq *ibcq, int entries, + struct ib_udata *udata); +struct ib_cq *pvrdma_create_cq(struct ib_device *ibdev, + const struct ib_cq_init_attr *attr, + struct ib_ucontext *context, + struct ib_udata *udata); +int pvrdma_resize_cq(struct ib_cq *ibcq, int entries, + struct ib_udata *udata); +int pvrdma_destroy_cq(struct ib_cq *cq); +int pvrdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); +int pvrdma_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); +struct ib_ah *pvrdma_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr); +int pvrdma_destroy_ah(struct ib_ah *ah); +struct ib_qp *pvrdma_create_qp(struct ib_pd *pd, + struct ib_qp_init_attr *init_attr, + struct ib_udata *udata); +int pvrdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, + int attr_mask, struct ib_udata *udata); +int pvrdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, + int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr); +int pvrdma_destroy_qp(struct ib_qp *qp); +int pvrdma_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, + struct ib_send_wr **bad_wr); +int pvrdma_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr, + struct ib_recv_wr **bad_wr); + +#endif /* __PVRDMA_VERBS_H__ */