From patchwork Tue Mar 23 16:14:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A2D3C433E3 for ; Tue, 23 Mar 2021 16:16:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 137B2619C8 for ; Tue, 23 Mar 2021 16:16:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233134AbhCWQPf (ORCPT ); Tue, 23 Mar 2021 12:15:35 -0400 Received: from mail-mw2nam10on2048.outbound.protection.outlook.com ([40.107.94.48]:9569 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233132AbhCWQPP (ORCPT ); Tue, 23 Mar 2021 12:15:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ItPLBrK3bn1IT4x00vSBt5zNbkjHHyKtZVf7hRvB+utrkHsRJxZcayLSkqUt5qPO58rM2di0BROtXTUI2RQ6o791m2ujLXytp84rrn796VtNJ1R5kA+RjAt4pkUoqGe6CCHqffQu2xLWgk44UeqInzoC4XWPj+9ozV8uU8cDAanJgPIOdBDnF8W33JeoF68zti7kLsEc3CH0FzAY+hWEDJL201uPy+l8uc1rSCP+vx7tPDSEXWXvCSPdtWeNeSkkMbsllObPhw7sqPz0Kyx4Dt9Ym3PDXnKxu+HudPB3eJsQF3Th9+zry1Zm4DinCNgBsRYwSBTbsBKc2bPtiZbzoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N0iRN/VSy2CWTtwWkvSUA0AnLu+cLQXXHj8mJqZo+SY=; b=VIO7HS6L9vzl4Oj0IRi0+7yLWmw8iUH49MrOPqwvthqCnqjnLc2Pl1pFI1OwvfYlWLWbX8qhOVuF3A514MMhbkQ1mfKdbjah0sBMjEbOlVsn/7W1EOCT266SbFMXvhJosihKyLwtfH6+qHLYtTawySbj4OijqX2B/YzQI3XWpYtGCiVk6rTEArKg9POBKREuPi/mxADM/N1TLkGscqrDKDchNVk93h+QZAqeLmpgcVTmCKtzvMrAR0xkyAB9uNnoxL2Z+vNXHnZGsWUJgx4ESU77kO5/a+HM1stLC55imoAHw6h5+aSSVpZvpYMJmcWxce6Hm2vPDqnjzL2gWxfTjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N0iRN/VSy2CWTtwWkvSUA0AnLu+cLQXXHj8mJqZo+SY=; b=LKdGbLqhv7pOwf4MA6ma2NeAYFa1qPgnLj+VAwfMgDouqrz0N5Tf1LthzADk+LoMaXL2nTootv6Px1RL4Tm3jk+3WeFsaWTXAh2UyqmrmHJG4iLGkGOSGrYTudHRmMf1JL2YzJ6m1TN6GK9n2FC/C4wTEKmfllsVePIjFn0PHTXpE1fyvCIvUvtcZ5jP4O8KDWBeY6wWYLa2j6LUc5SHB6iTgwG/0IZoIdGemBq28DLdHL4TPoe7ZV9dTsBY+mi3QtdV4zwsW/kFXgIlNpSpxjazyq/PaLg35qtvjagIRp8dRvWiCpUOM+ntPCbRr8f/57Q8L3C3cgd8VoheQhB7Gw== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM5PR12MB2440.namprd12.prod.outlook.com (2603:10b6:4:b6::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Tue, 23 Mar 2021 16:15:08 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:08 +0000 From: Jason Gunthorpe To: Alex Williamson , kvm@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 01/14] vfio: Remove extra put/gets around vfio_device->group Date: Tue, 23 Mar 2021 13:14:53 -0300 Message-Id: <1-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: MN2PR19CA0031.namprd19.prod.outlook.com (2603:10b6:208:178::44) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by MN2PR19CA0031.namprd19.prod.outlook.com (2603:10b6:208:178::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Tue, 23 Mar 2021 16:15:07 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCP-Ae; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cae0e79a-52c2-45b0-c9c7-08d8ee16d376 X-MS-TrafficTypeDiagnostic: DM5PR12MB2440: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3/li3MepplDXlga7QVIvMuQBVwImJl8aRKMcgD1Q4RzyO/249jcZuJv7ZXl8Bi59XM5LFDtEC12HEA5NzBzNjCoTNoRd2DYNvPUHNNbrPCgnTyW75f9X7HuQ9ovtC0tS8oqsl1x3vyuXcg+4m6waPMdUWgshPSuGaY0NoQgfRPDuFK4qawY4nTvArlMvZrAwpw7L/2HjmBBYqonZouGt6LlEzbyE30/+1TZqxieCm2pC5qDfq9Lt4nnd1GdF0fvJ8zVMrLLbKyfRIAaJsFBzpbCbRflmcntIjk1LTFIRnPHzROfKeZ2cgilQmQm+xMgsiXGaZdH44EUORCBNXAg2o0xrA8/A49wf2I4CtFL8U4WDHMY5Qq66AvqAU1UE41OwCZmaUg1W1szBWNyU+ZxIhYIx2q4eYQs+4mVPTQMKxqErLTL4i39T6iKh3RFHAbrLj6lif9zkmaotLawJDEs5HQ6lb6iCDOU9n2EfDK1GtLbM7zczWAtC782V/6bmn1zVvEd63D7p+2DKyzsgiczqsX2ikKmRkvJajXz1+L88lbbINikmTEvjsEHvpvfbpRbkGqcH8gUMtvKrfvNaZBgSngBu+20kVR18ro6SfRTdosoyjStb0goBEaPn/ySt2QLSUF8BjFujnF0VZROFhJEAKtX48BKOn1tVPuxJclM4p2+oryWJxCecKMh9IjnuamIr X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(396003)(376002)(39860400002)(136003)(66476007)(66556008)(186003)(36756003)(6666004)(86362001)(38100700001)(66946007)(4326008)(107886003)(8936002)(54906003)(9746002)(26005)(478600001)(426003)(8676002)(9786002)(83380400001)(2906002)(316002)(5660300002)(2616005)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: DdOGWTl+2osOuO2jpsAZSJQNEZCxvS++1CCeDDwolmtlBMY2hgjREpuyH39Pe81ICZTu6lPim4TrBjs8vvUbVmSsdAV9b/E302nR1yqc8XkIsWe91hODMX7OMDXrxmsDuUwf9ObNkHXRU+Ikjk1/pa/pJajznIrRUIWIruIQtonv0Ydg6MucA1XZCCNxnct4fZBuzOH0s2SO8w7kmTmdvmrMwIxwb0mDjIWw2qUpyskR1Ru00Xhr6JfcSYGWcHlhRiuyy/z0Zc/73T7Y29G/seEw4hpsyXy7WE7h5GfIEgod/h+4Z9LpXDMFswdV0zol7hv40Mvw9mbUaUr5SlZywQ4UIyQLjhVntYov/RuxWnhsXfpDd5b9s4KMNSOQ/7zxH92SmLWDqugDVtod2dCaaPIXY0ClDTWByxBZ/g+yroTOv5xI9tcXmo//PTicuwoN9H7nuuLZTv7F/6wYEpHaKbGL6ioO7YtO76DdSYAdXJJWKTpluJL+Eg6gbTDeq232Gxrw51FGwZYwmCN0Y5jpFJXWnyViTD2NMY49O31H/PUvsl4pfu1E15UUCnFSzTDK6hEflXuGrOmm58n0SCj94R3HEBAMJ/vcHn6gMLT7QDGj5evrB1/ShxhnArLnGV6PxCV1Bl2nqQfnESicFBMHYisG4J4lQBOhvZAP8il75f7teBoDYHHXfohsHkSlTs7wjtIL8QJ2aD4LnkKKcc6HuPu6x9cN+wrfF+q6jCC3h+K1hKLRiw99mnc3SxtoQqJ3JcQ0pkjZqbcKJS8rdryEDK7UkXYPOW3iiZxGe3V/5qF3jWkKbQIxGZTZbjYz4JBuAkJDWeYohrV3KS+04vaq1HyDdv0Fw7ctxFSPSIzLPmXDq1JLziP52HJx94hNTqTQbi5fDT41cAC6GVRAjjDNisd9RD68M5RIVQxFol85VQGNAA8iCEzNR8W2Zi1yNL93fmt3LJT8O5cKnDlJtEvYb7wtFpjwokdhwyclTk9NXPFfZn44Sn3vCJmnB+TLBSmtgTuuzdUi24T8rruxhVhyiOhhUsHQTJjudMVDL21ZnsMarSPqjDRFNeinUl6YSKXQcVa0bSA3WU1t3Dx6Im2BmBpzjBWJO8q6qETq6bDpY4M8q4YmjAqkvZkWknwyCdWZy1S+JynYbTANlM8SFnw8xaFBLutKoduD+ljH7iJoDPt7j9Wi6yxqvp9TMe+cVv8soOf4Z6OhOgl8NLRogNKjPqbLpBswcKLvmzVieYcYXXHJ6SythdzLYp4rAfVcuUfH2hqsd9seQE6GVkSLwnP8Hp/+pm/Uk0Ip62LFXq9/jG3AMvYArZuH5KLtMO+gEDep X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: cae0e79a-52c2-45b0-c9c7-08d8ee16d376 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:08.0731 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3Kl3xHtr1LhdumUQ/z6aXi5XXReij1IF0Nbe12Go8+uYotfh92SO/oXiYLEaAfhE X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2440 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The vfio_device->group value has a get obtained during vfio_add_group_dev() which gets moved from the stack to vfio_device->group in vfio_group_create_device(). The reference remains until we reach the end of vfio_del_group_dev() when it is put back. Thus anything that already has a kref on the vfio_device is guaranteed a valid group pointer. Remove all the extra reference traffic. It is tricky to see, but the get at the start of vfio_del_group_dev() is actually pairing with the put hidden inside vfio_device_put() a few lines below. A later patch merges vfio_group_create_device() into vfio_add_group_dev() which makes the ownership and error flow on the create side easier to follow. Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Reviewed-by: Max Gurtovoy Reviewed-by: Cornelia Huck Reviewed-by: Eric Auger Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 21 ++------------------- 1 file changed, 2 insertions(+), 19 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 38779e6fd80cb4..15d8e678e5563a 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -546,14 +546,12 @@ struct vfio_device *vfio_group_create_device(struct vfio_group *group, kref_init(&device->kref); device->dev = dev; + /* Our reference on group is moved to the device */ device->group = group; device->ops = ops; device->device_data = device_data; dev_set_drvdata(dev, device); - /* No need to get group_lock, caller has group reference */ - vfio_group_get(group); - mutex_lock(&group->device_lock); list_add(&device->group_next, &group->device_list); group->dev_counter++; @@ -585,13 +583,11 @@ void vfio_device_put(struct vfio_device *device) { struct vfio_group *group = device->group; kref_put_mutex(&device->kref, vfio_device_release, &group->device_lock); - vfio_group_put(group); } EXPORT_SYMBOL_GPL(vfio_device_put); static void vfio_device_get(struct vfio_device *device) { - vfio_group_get(device->group); kref_get(&device->kref); } @@ -841,14 +837,6 @@ int vfio_add_group_dev(struct device *dev, vfio_group_put(group); return PTR_ERR(device); } - - /* - * Drop all but the vfio_device reference. The vfio_device holds - * a reference to the vfio_group, which holds a reference to the - * iommu_group. - */ - vfio_group_put(group); - return 0; } EXPORT_SYMBOL_GPL(vfio_add_group_dev); @@ -928,12 +916,6 @@ void *vfio_del_group_dev(struct device *dev) unsigned int i = 0; bool interrupted = false; - /* - * The group exists so long as we have a device reference. Get - * a group reference and use it to scan for the device going away. - */ - vfio_group_get(group); - /* * When the device is removed from the group, the group suddenly * becomes non-viable; the device has a driver (until the unbind @@ -1008,6 +990,7 @@ void *vfio_del_group_dev(struct device *dev) if (list_empty(&group->device_list)) wait_event(group->container_q, !group->container); + /* Matches the get in vfio_group_create_device() */ vfio_group_put(group); return device_data; From patchwork Tue Mar 23 16:14:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EDDAC433E9 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D4E5619C8 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233179AbhCWQPi (ORCPT ); Tue, 23 Mar 2021 12:15:38 -0400 Received: from mail-mw2nam10on2048.outbound.protection.outlook.com ([40.107.94.48]:9569 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233144AbhCWQPR (ORCPT ); Tue, 23 Mar 2021 12:15:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BLba+vfTJZXRaoECPefM+/+1cIWSjO2PF6mT7WwL8Jfrgmqm3BXbQk+8+9DSqcyCQhCYmr8Ypm/IzAkR8v/bcYOsoPq/DnAov4jWBLDO9SKdN7H2PaGTzCLuxMFPzTdI1dv3ysVFGX6X3rVGJXMWeTyHqLdUiKwt15n372LLGRu1U50SQOKkuNxpRUJW1NMDvxt9llgz0jD7GNfaVI6NLV88v3dYML29Cnj34PYGyxQ08q2cFKqqJ5761wGhSU1e/NHsWG6xz7cAdb/ijPiOfY5WMSOo8qvaSb5FUQlkRgNYJjgkcLxmiZCMgJ/uAwlAPVWADbfMA9V4+WYzyjVA9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FBn0P1KzS0KeguBazW8G31BWuQQTBc6PsBlBz9t2yMg=; b=YYBd7HqKubbLA6+dprBAh+ZjSuYWHXIFsR5uFb+xqwzLjZCt4qII/m+q0hEzfWWtgt8kOHcMbbABkHgkINXh7KFotqR6L20JsE/sreca8K5OfXnVz9a53BsP+xTTzd1HX9wJ9qj/Pg3UAOqggsbjDEqb475RpzBjhYOB6B9EQ06GdbiCeXd5llOEDoYmegPYPPcNerQ84idtLkzog+Rw9Nmu6kzIMTWdpI3e3RZwu2k42k2EHvGgzW2rCaWXefa0+FhTmxuThE9sEhbRyZBzhXXBw97sEdjwfJDHaRsRNJsNofvTXYlvqRRZiOiM8WLG2PaqYx6YSD0rlKC0GQie2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FBn0P1KzS0KeguBazW8G31BWuQQTBc6PsBlBz9t2yMg=; b=Xjy++pW/U7ZkL1bdy0PI2XnCgcWB+ri1txEKpcNIXzyreRYyYDUiD8qJl4rf3hpyRoVBRxHB3K5HQ0e230achOzaDr5Ef8mNSB2uO0FUdLaqZR0GVQRop63b11Y+nrHWowRLW6Jnux3F32dratcZzyX9IGxTF0oBtxT7uQkWQ5D4keRF33XEEDA2Zjpru72/Ta/wP4K86Bb8lJwPoQyfATD6RZfDWTOQ3kBkAxFl8IvSa6KBYQ1xrTTffSE5EcmWtlLPGptbKbjx40rbT70SGNxmFSB9ayuoqPZ/TFu8Ig09FnQ/p7kQ6GFahytz6NViV8bmvRI+3CK7fqQ/S8Hv+Q== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM5PR12MB2440.namprd12.prod.outlook.com (2603:10b6:4:b6::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Tue, 23 Mar 2021 16:15:09 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:09 +0000 From: Jason Gunthorpe To: Alex Williamson , kvm@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 02/14] vfio: Simplify the lifetime logic for vfio_device Date: Tue, 23 Mar 2021 13:14:54 -0300 Message-Id: <2-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0195.namprd13.prod.outlook.com (2603:10b6:208:2be::20) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0195.namprd13.prod.outlook.com (2603:10b6:208:2be::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10 via Frontend Transport; Tue, 23 Mar 2021 16:15:07 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCS-CP; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: da5c743d-4a97-443e-9cfd-08d8ee16d38a X-MS-TrafficTypeDiagnostic: DM5PR12MB2440: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O7RJZz8z8d2Huq+BeXib54hC8LU2YevTEnKHpMBpy8AbjLT4zmbJYe5cKvPHniJ0YWRDKps99N+tb4AhnfhUdA8baQHBHem1IAbQonY26c8ucp4mFVrsA5xQKma2MpQOzyS8deVNNHbymDhOGur8jBYQSw4rSOmeZztyTqtW0G+aZqGiXvG2D6T+kmH9FGhC/yS8PLAFk+rRI7B6l3lINZNY7vBJROLBtLyTlGs6LYfZXzzNOWdmEQE25FzdP1ldNc7r0iFO+++Wd/98NDRNPuU/BC+sWVpPitgGbhzjDeJc0ratVscfBi1Fe9CvuRiICd2Dqz0nxP0WnL4pfSwW4RPYfqIWVIkr3+lBMPB2zncR8xkZnbwZ7ODR4JvJHqfjyv9JaRCS0kCmVL+b/t8K7utGvUUmgcvDrCjvRXW4HsuCe06m4r7IqJxprvmxNqnS6VmTVnWehrSehszoMuUWALUOueoO5RaVr373uKGpw+oO2MP5Qp94Xs6VeIqaSsJH1wjRij1lG9aTwRcHQQLRZUEppRKmxB/7KamFiVw1WVHevEEebjGtSMrh4zlDaaA9I0OS97WE53NdFAniWY1yP0cAh/7EZhLcDQ4LfN26T7+i8pkM8tHLO2Hmjt1Vv0ixL7qYKy6Ik41kVWXm2oPn6jbwT9epwDlQGeta9sh8dd+44miMXgPSYAwb27R44c7S X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(396003)(376002)(39860400002)(136003)(66476007)(66556008)(186003)(36756003)(6666004)(86362001)(38100700001)(66946007)(4326008)(107886003)(8936002)(54906003)(9746002)(26005)(478600001)(426003)(8676002)(9786002)(83380400001)(2906002)(316002)(5660300002)(2616005)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: Hg1nsSTUZNQ+aew2t7U0KGs775RXYs8M5UiZ5GyGSJb9TcqkewL8WHf6WNf1ODKhdEHRglMgJVzwX3aeIY0z1DzOtXLQBFoGH2i/WFVEw2IZs00n0yMie0cIrumXsKhx+uciG6h1omqH+7k9nekuzVVjSBEhJKH+qePX/GEQgeWqywesleOc5GnHOgj8aTvI7szBhK+ZvRpwbsM8VYFaOfp4y99oyytJIGe/j1bMohUlEhkeCbVppfdvY4d+IIrVTvFGpK9wcIK3nY4HxIVx7FmpSzaHCxU4hCBhddxZNFLLo5atqhro4JmdK51vp1zQjyFzFKxPrs0BFxiqNrQOekGvHgIyXLqeJcomxfv3dnW7/FmBLP/siOESl1KAPWHbU2XYzUAIolEJAd6eMCpxqWsMJ6qu6JsJrFmCBwK4l5UjtZ5uOBC8czoRcJ1/jwmxVtkp5V4qkeIWab+/VtHobwfeBZOivaC37iWvEvAD4igLxsO7Am0nMxwvYmjfJ45oXTJWoxeL28yg/sZ3ZqWmt8Mzmaqc2KpQwQ8z5SHzgBbX73L94HPG5BSKbDYV7CFbBHLpeQOge8Q5ZzlGjWd/5PC0ilBI/QGIOjjWXVp7jBqZMarXlsqYXI63DfATjJnwmT96/uULTeCwVFLuVnnWND7gidi3zkD6JR5V81oYgov2chRG/iJCcBamKIvZGO0JEaEsdIAE8LFuqK7ocwoMFobyaXRyvDNCwlmc3ohCZzL4E8WhzgKPfe1U7L0rCRBPmRsnHW9uzU4I6Od+wMqg6W3T5Dwtjpn9Vb/ixN7De0iXju4FSkMS+viYFEIUKA2u3mT6iFWeTzZn2hRrxvj2ZxC/uOShoqZHiZ2fH69e+U5RxEJOcDzG22nxtCwyRJc/YXaOwfDu81YYtHy3jUS/Zv1dpwL/bncx7RBJR8f/c0q1Ost2F/8RcaXd19sel2fPEdauPPbA+1qNgTcwGgB940zoEmJBKaJmlZapu2ZP8qAL+lL201vJxHyngtHgbPGU92IaS48EkTB1rl3bA54IDOIe1SLjei1/comAPBR4BzhBLNUl5zj9gtbpxou5YGil389Cd/2kyLeOCbYlm2vA6faGsQgzJ1JpnypCzSXZDupHw32EUhLx7WofqTMsQi9DblhrJPaA60hcAxZAIjfQTCjwBzjsH7lC9UmUnBoB0agxdbGlP9Ot449QuExVuHL5DLnb95iCdpAJDDuyFfQEVoXTaChJodkJC449QUu4Z5G8ajJQ5fhhykf7R45HfBBzsgpKf4MQ8/dGclSKS3D7Nyhq+eIPuPQKs7vPonAGJTROJQKIVXF5eEFpm5VJ+BvN X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: da5c743d-4a97-443e-9cfd-08d8ee16d38a X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:08.2440 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kC2rFQ/jjsXWlIIL0g+Y/C+JDBS7Xl3z46V9HSTKcSKsXzBmKgm2ZixYoi/JIGIQ X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2440 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The vfio_device is using a 'sleep until all refs go to zero' pattern for its lifetime, but it is indirectly coded by repeatedly scanning the group list waiting for the device to be removed on its own. Switch this around to be a direct representation, use a refcount to count the number of places that are blocking destruction and sleep directly on a completion until that counter goes to zero. kfree the device after other accesses have been excluded in vfio_del_group_dev(). This is a fairly common Linux idiom. Due to this we can now remove kref_put_mutex(), which is very rarely used in the kernel. Here it is being used to prevent a zero ref device from being seen in the group list. Instead allow the zero ref device to continue to exist in the device_list and use refcount_inc_not_zero() to exclude it once refs go to zero. This patch is organized so the next patch will be able to alter the API to allow drivers to provide the kfree. Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Reviewed-by: Eric Auger Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 79 ++++++++++++++------------------------------- 1 file changed, 25 insertions(+), 54 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 15d8e678e5563a..32660e8a69ae20 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -46,7 +46,6 @@ static struct vfio { struct mutex group_lock; struct cdev group_cdev; dev_t group_devt; - wait_queue_head_t release_q; } vfio; struct vfio_iommu_driver { @@ -91,7 +90,8 @@ struct vfio_group { }; struct vfio_device { - struct kref kref; + refcount_t refcount; + struct completion comp; struct device *dev; const struct vfio_device_ops *ops; struct vfio_group *group; @@ -544,7 +544,8 @@ struct vfio_device *vfio_group_create_device(struct vfio_group *group, if (!device) return ERR_PTR(-ENOMEM); - kref_init(&device->kref); + refcount_set(&device->refcount, 1); + init_completion(&device->comp); device->dev = dev; /* Our reference on group is moved to the device */ device->group = group; @@ -560,35 +561,17 @@ struct vfio_device *vfio_group_create_device(struct vfio_group *group, return device; } -static void vfio_device_release(struct kref *kref) -{ - struct vfio_device *device = container_of(kref, - struct vfio_device, kref); - struct vfio_group *group = device->group; - - list_del(&device->group_next); - group->dev_counter--; - mutex_unlock(&group->device_lock); - - dev_set_drvdata(device->dev, NULL); - - kfree(device); - - /* vfio_del_group_dev may be waiting for this device */ - wake_up(&vfio.release_q); -} - /* Device reference always implies a group reference */ void vfio_device_put(struct vfio_device *device) { - struct vfio_group *group = device->group; - kref_put_mutex(&device->kref, vfio_device_release, &group->device_lock); + if (refcount_dec_and_test(&device->refcount)) + complete(&device->comp); } EXPORT_SYMBOL_GPL(vfio_device_put); -static void vfio_device_get(struct vfio_device *device) +static bool vfio_device_try_get(struct vfio_device *device) { - kref_get(&device->kref); + return refcount_inc_not_zero(&device->refcount); } static struct vfio_device *vfio_group_get_device(struct vfio_group *group, @@ -598,8 +581,7 @@ static struct vfio_device *vfio_group_get_device(struct vfio_group *group, mutex_lock(&group->device_lock); list_for_each_entry(device, &group->device_list, group_next) { - if (device->dev == dev) { - vfio_device_get(device); + if (device->dev == dev && vfio_device_try_get(device)) { mutex_unlock(&group->device_lock); return device; } @@ -883,9 +865,8 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group, ret = !strcmp(dev_name(it->dev), buf); } - if (ret) { + if (ret && vfio_device_try_get(it)) { device = it; - vfio_device_get(device); break; } } @@ -908,13 +889,13 @@ EXPORT_SYMBOL_GPL(vfio_device_data); * removed. Open file descriptors for the device... */ void *vfio_del_group_dev(struct device *dev) { - DEFINE_WAIT_FUNC(wait, woken_wake_function); struct vfio_device *device = dev_get_drvdata(dev); struct vfio_group *group = device->group; void *device_data = device->device_data; struct vfio_unbound_dev *unbound; unsigned int i = 0; bool interrupted = false; + long rc; /* * When the device is removed from the group, the group suddenly @@ -935,32 +916,18 @@ void *vfio_del_group_dev(struct device *dev) WARN_ON(!unbound); vfio_device_put(device); - - /* - * If the device is still present in the group after the above - * 'put', then it is in use and we need to request it from the - * bus driver. The driver may in turn need to request the - * device from the user. We send the request on an arbitrary - * interval with counter to allow the driver to take escalating - * measures to release the device if it has the ability to do so. - */ - add_wait_queue(&vfio.release_q, &wait); - - do { - device = vfio_group_get_device(group, dev); - if (!device) - break; - + rc = try_wait_for_completion(&device->comp); + while (rc <= 0) { if (device->ops->request) device->ops->request(device_data, i++); - vfio_device_put(device); - if (interrupted) { - wait_woken(&wait, TASK_UNINTERRUPTIBLE, HZ * 10); + rc = wait_for_completion_timeout(&device->comp, + HZ * 10); } else { - wait_woken(&wait, TASK_INTERRUPTIBLE, HZ * 10); - if (signal_pending(current)) { + rc = wait_for_completion_interruptible_timeout( + &device->comp, HZ * 10); + if (rc < 0) { interrupted = true; dev_warn(dev, "Device is currently in use, task" @@ -969,10 +936,13 @@ void *vfio_del_group_dev(struct device *dev) current->comm, task_pid_nr(current)); } } + } - } while (1); + mutex_lock(&group->device_lock); + list_del(&device->group_next); + group->dev_counter--; + mutex_unlock(&group->device_lock); - remove_wait_queue(&vfio.release_q, &wait); /* * In order to support multiple devices per group, devices can be * plucked from the group while other devices in the group are still @@ -992,6 +962,8 @@ void *vfio_del_group_dev(struct device *dev) /* Matches the get in vfio_group_create_device() */ vfio_group_put(group); + dev_set_drvdata(dev, NULL); + kfree(device); return device_data; } @@ -2362,7 +2334,6 @@ static int __init vfio_init(void) mutex_init(&vfio.iommu_drivers_lock); INIT_LIST_HEAD(&vfio.group_list); INIT_LIST_HEAD(&vfio.iommu_drivers_list); - init_waitqueue_head(&vfio.release_q); ret = misc_register(&vfio_dev); if (ret) { From patchwork Tue Mar 23 16:14:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D5B2C433EB for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DCCE619BD for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233186AbhCWQPj (ORCPT ); Tue, 23 Mar 2021 12:15:39 -0400 Received: from mail-co1nam11on2068.outbound.protection.outlook.com ([40.107.220.68]:19553 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233159AbhCWQPT (ORCPT ); Tue, 23 Mar 2021 12:15:19 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ns1rs91FtesS24C27g4Xv1EjBkppLeBPAJA/Eq2iaiWSjy2InDe2HXNCWTv7LOCOAENLaTlaQvZzwI3YkuwJ6VwgJRgLDxLlNg06ETTdAYxWkvfSb5pXErzIwFrmZ5wfrXnxkScgk61D0jRTe21yX2yvemI3WfZY2BVYcDSNNT5Qp1iJoybGIPk0pIuopV+ALA29RzfSE7D0iKdorCTOOWcHRXDNOM0HU4VUlgaj8vFzJx5uAOd05o5mu2aJD2VkYfDmY5YkyRoSO538cbSnM+oM5/0xkzg4V7joX+XVlMogowkuEczhosnR4pWjlvXG/siS5FdVrNJYIw6lnqb7Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+xPBC3xPaI8fc+7REGhZLu3AHPQyC6odvVGpV0xzeNo=; b=LItbQZVVUb9h3z+/R+vMJ5OkkWBsOKwOYBDXAORCWpU9HQK0JV+DLilNk54deeYXK+eoCHrkM+CnmBL42mkkai7GvZacBPLF1suWexURlUzwLPwzGC6fJpecLQuVcEuOwSlZpg4Ytl1soX71LTW1+kCTYDi+TBIh76pjrNIossEbW1BmCrA5fZGRLgOheu7gk9XREDKTgbJmcPiGpc3F3/pkoWf4vHjyG7glX8auZjYG1VbsG0Z2YZht++DECDf5XfDwMzZDHy67rGk3aHDNBDeKzrwTuT8RefvuJFn95AyKBmo+C3ZDra/6xfi8HQQygbRsWsyYc6oRZWkaeqXvgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+xPBC3xPaI8fc+7REGhZLu3AHPQyC6odvVGpV0xzeNo=; b=JA7QU/g5XBtxia12dDlvaCxo+TBfTvB01z7aWxPSfKVrQy00PX5IvxqRk8+UJxmmRy5eChcDodAC69F6/Xcj2SokyqYr7UJ83NPfdSDLrYrd7PWEbZQ8jGDoZpBNRnG3Nxn2qviUxjfMOVUC4cocq2nWI7tXYqKwRwNBw1GP9J5SLhOYazUIWUjCa2nPWt7IobAbR03561Zva2FYEuQD9YfNRugmYGQAYXijq/QWoxKFQncO9kliP6t1/EfL0xOg7fycQqkJ0rHeOFxs1m9qCZVxmz1o6tAoVKMTOvbzXyl2D32p9ZfMjHN64cR5x74Cn4763dsQNvUxGauTG4eT+w== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:13 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:13 +0000 From: Jason Gunthorpe To: Alex Williamson , Jonathan Corbet , kvm@vger.kernel.org, linux-doc@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta , Liu Yi L Subject: [PATCH v3 03/14] vfio: Split creation of a vfio_device into init and register ops Date: Tue, 23 Mar 2021 13:14:55 -0300 Message-Id: <3-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0208.namprd13.prod.outlook.com (2603:10b6:208:2be::33) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0208.namprd13.prod.outlook.com (2603:10b6:208:2be::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.14 via Frontend Transport; Tue, 23 Mar 2021 16:15:10 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCY-E9; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a19d3db3-8609-4dce-cb7f-08d8ee16d4d2 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c0uY+sVyS8SBfiT0GSQ5UHyMDY+3PDrn7vhj7BDRKCtEyGJZ9cAvMMRPd7/jtr9MjA/b+cim0JZA5kHL0HLpUmdCTXaSVeSuafN9UZoarYNQHDn0H4Pg8fiXx6i4t6sH2sRAMI3XY/DcjTwUs88CXfEWMcRWobBV1mA1Xe+5TguIK0A+fbG2pkJLSJxMaj7FgVJW7wpHZmF4tDw2ECAfQXnLGVd2gzXbKrnUTMuhUNHzzjCrmv96L5FdIWMnkaGqtDKxMWCE7WobOYby9jjtAYIV+QxgPm1+96rdouXWmKLTiC7lZRlUvBnn9tDEMioYCxZnDfEHnRM4JSrkm31dli1o1rYCXk4rEwCYEpBHWYTARFEtV1s27omBXQX41bP/O0/EVVOaDHm8qu9zs7Iqgr/05vMfsoxgwhAbQyASXdVxLYCtLd/gShU3ckFT6BklY5lvxSg7lSEo3qFnc04UfTYGPA8EK9bh36Mwc8r7FQ7xv65GlSMhm35HzXq6Z9e+ZqqTPVx2CtwEFZ02U8SNelZZwOoXJ2DsR9F1hv2vyRcxbznFCvKufCcyLO/3Am30TcdwFW9OIQ3t147jLCU10sgwqFamoEnFxJZOZA8aaVfaxulTZTMKSASWdI85t8kvezNEwTzWYwH14ZyJvuIDli2teXwf7se7nuTyIMRrwntDSEB0fxe3pqzIWy7d1nsP X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(30864003)(2616005)(66946007)(66556008)(2906002)(186003)(7416002)(6666004)(83380400001)(426003)(54906003)(110136005)(316002)(8936002)(478600001)(8676002)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: hd/QjxBXufRo+U+3cqDaCl7rMs++NdT3il0W0k/fQfMVc+g+xlaVHyVpHomxTesTq+7LWD6JOrS7l1K9GWdBDU7f+FZAP/K7J0aadwUoL19xsjXR76D82wOBccMpvwxUC1JEBjyfvk85F7szC4CaOIbBZxl3/nqVWdtqIgUKmli570kmX1c7g0sv5jTXwS7fV+8s+6GwaaRaGZL6MLsqT5nAeUBcceifJsJcNHrMyffl0s2bWX0i4Dl7jBSheGUSLXyeh0wTC+ths5ADZyqeYGCIwXUgMM2oPbLF/WnIkVounrhBVyByR7duMavBnu5CJs6Wj5sRVps/DADdeqdQelCEZUTY/bUf3RsxtRPq8TP7/Uv2r9uhixrRIbWjlyUjMT+G4EBxZFFZCDBjV7CTKJVSGSfwwor6/d6ie7O/ePderVjJsXJRwZL7XIOVzHAb7xmXZBoSgy2v5QUSx+5eHphcbzARRNOqx1yvGYUD/Vubb0I5+BkgOsONNSnnK02Q7TgJobQtcGhdkC4RpJuBoZmMX3tcdvkQUC8wQO64ve9IOODvYyi87phX6MVUEKV4pVCQiG1/V4ojikvHDWssVpc2L9GgUVraZhlb293bjdnyOqVZ6Lc2wGA7LjtGGGOu4lRsh1eDCCzNEjqPMOzf0WCgwwCORnoR4mzBF8HXg7ANf/YIAlQQalyNcaYffdbOwwfqVll83FnrMOCDUrj6pDQaQl1ERiZMDTEyKqiKE6iYVVbkJ89lN7E76zCnXNVMfQXZY6py4AWKsTUtet+mQnRXxjiuMdhxHei4Ocd8ExpyOOFKy0zFJYsMcx4OLg/zfAA16r1MwigwFs/OM1TrhKyPW+cAmoY7xPF4Cu1hhYG48bh/hzcxU0rueTlFzRdjj1D8N0Q22QIV3oce6kdf5mjZ9H9a2VZDG8WnqCo+wVbo8rqpVNL2fVTHHlHXMHID+H+84oEC5iKTCL3IfVAay5tHvd51MSP3XRvF+FSQYUD83pBVgw4QYPxIP0hKlkvYe1l+GDCGAwTRrU9cfIP6y9Us8kEvp76dwVdcP2XgjkuOrtyVrr13gZLAF5lw2jquT5Jxsy/edXUPTsYalFATF6dFMdTKEHqFsDXJ5JcqnheooPSF6nY3ir9eSijcB/DPv4Lup1PpJPbVSEu1qd5UccEl0O3Y3E+8HO3mHFRr1XiHIbGyYR6FAMzcGp7UEjgNmDyAvOC6IYf49sHLf8SlHgzlR6Nxf5xVn2mYGJnEQNegmJt80ZtuGDghRZgDlRPizr+lzs6frr2sJn2bVfnbG8rhipsMv23J5t/5wkomg+e5Q5E5Z82xjPX3hDsUTojJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a19d3db3-8609-4dce-cb7f-08d8ee16d4d2 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:10.2289 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IARZt3T9BD1or7Cdfbiachz2NWW69fXKVit3Al6NiwBLtlWcwbxuUzzfV5PIZ1ax X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This makes the struct vfio_device part of the public interface so it can be used with container_of and so forth, as is typical for a Linux subystem. This is the first step to bring some type-safety to the vfio interface by allowing the replacement of 'void *' and 'struct device *' inputs with a simple and clear 'struct vfio_device *' For now the self-allocating vfio_add_group_dev() interface is kept so each user can be updated as a separate patch. The expected usage pattern is driver core probe() function: my_device = kzalloc(sizeof(*mydevice)); vfio_init_group_dev(&my_device->vdev, dev, ops, mydevice); /* other driver specific prep */ vfio_register_group_dev(&my_device->vdev); dev_set_drvdata(dev, my_device); driver core remove() function: my_device = dev_get_drvdata(dev); vfio_unregister_group_dev(&my_device->vdev); /* other driver specific tear down */ kfree(my_device); Allowing the driver to be able to use the drvdata and vfio_device to go to/from its own data. The pattern also makes it clear that vfio_register_group_dev() must be last in the sequence, as once it is called the core code can immediately start calling ops. The init/register gap is provided to allow for the driver to do setup before ops can be called and thus avoid races. Reviewed-by: Christoph Hellwig Reviewed-by: Liu Yi L Reviewed-by: Cornelia Huck Reviewed-by: Max Gurtovoy Reviewed-by: Kevin Tian Reviewed-by: Eric Auger Signed-off-by: Jason Gunthorpe --- Documentation/driver-api/vfio.rst | 31 ++++---- drivers/vfio/vfio.c | 125 ++++++++++++++++-------------- include/linux/vfio.h | 16 ++++ 3 files changed, 99 insertions(+), 73 deletions(-) diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst index f1a4d3c3ba0bb1..d3a02300913a7f 100644 --- a/Documentation/driver-api/vfio.rst +++ b/Documentation/driver-api/vfio.rst @@ -249,18 +249,23 @@ VFIO bus driver API VFIO bus drivers, such as vfio-pci make use of only a few interfaces into VFIO core. When devices are bound and unbound to the driver, -the driver should call vfio_add_group_dev() and vfio_del_group_dev() -respectively:: - - extern int vfio_add_group_dev(struct device *dev, - const struct vfio_device_ops *ops, - void *device_data); - - extern void *vfio_del_group_dev(struct device *dev); - -vfio_add_group_dev() indicates to the core to begin tracking the -iommu_group of the specified dev and register the dev as owned by -a VFIO bus driver. The driver provides an ops structure for callbacks +the driver should call vfio_register_group_dev() and +vfio_unregister_group_dev() respectively:: + + void vfio_init_group_dev(struct vfio_device *device, + struct device *dev, + const struct vfio_device_ops *ops, + void *device_data); + int vfio_register_group_dev(struct vfio_device *device); + void vfio_unregister_group_dev(struct vfio_device *device); + +The driver should embed the vfio_device in its own structure and call +vfio_init_group_dev() to pre-configure it before going to registration. +vfio_register_group_dev() indicates to the core to begin tracking the +iommu_group of the specified dev and register the dev as owned by a VFIO bus +driver. Once vfio_register_group_dev() returns it is possible for userspace to +start accessing the driver, thus the driver should ensure it is completely +ready before calling it. The driver provides an ops structure for callbacks similar to a file operations structure:: struct vfio_device_ops { @@ -276,7 +281,7 @@ similar to a file operations structure:: }; Each function is passed the device_data that was originally registered -in the vfio_add_group_dev() call above. This allows the bus driver +in the vfio_register_group_dev() call above. This allows the bus driver an easy place to store its opaque, private data. The open/release callbacks are issued when a new file descriptor is created for a device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 32660e8a69ae20..2ea430de505b3b 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -89,16 +89,6 @@ struct vfio_group { struct blocking_notifier_head notifier; }; -struct vfio_device { - refcount_t refcount; - struct completion comp; - struct device *dev; - const struct vfio_device_ops *ops; - struct vfio_group *group; - struct list_head group_next; - void *device_data; -}; - #ifdef CONFIG_VFIO_NOIOMMU static bool noiommu __read_mostly; module_param_named(enable_unsafe_noiommu_mode, @@ -532,35 +522,6 @@ static struct vfio_group *vfio_group_get_from_dev(struct device *dev) /** * Device objects - create, release, get, put, search */ -static -struct vfio_device *vfio_group_create_device(struct vfio_group *group, - struct device *dev, - const struct vfio_device_ops *ops, - void *device_data) -{ - struct vfio_device *device; - - device = kzalloc(sizeof(*device), GFP_KERNEL); - if (!device) - return ERR_PTR(-ENOMEM); - - refcount_set(&device->refcount, 1); - init_completion(&device->comp); - device->dev = dev; - /* Our reference on group is moved to the device */ - device->group = group; - device->ops = ops; - device->device_data = device_data; - dev_set_drvdata(dev, device); - - mutex_lock(&group->device_lock); - list_add(&device->group_next, &group->device_list); - group->dev_counter++; - mutex_unlock(&group->device_lock); - - return device; -} - /* Device reference always implies a group reference */ void vfio_device_put(struct vfio_device *device) { @@ -779,14 +740,23 @@ static int vfio_iommu_group_notifier(struct notifier_block *nb, /** * VFIO driver API */ -int vfio_add_group_dev(struct device *dev, - const struct vfio_device_ops *ops, void *device_data) +void vfio_init_group_dev(struct vfio_device *device, struct device *dev, + const struct vfio_device_ops *ops, void *device_data) +{ + init_completion(&device->comp); + device->dev = dev; + device->ops = ops; + device->device_data = device_data; +} +EXPORT_SYMBOL_GPL(vfio_init_group_dev); + +int vfio_register_group_dev(struct vfio_device *device) { + struct vfio_device *existing_device; struct iommu_group *iommu_group; struct vfio_group *group; - struct vfio_device *device; - iommu_group = iommu_group_get(dev); + iommu_group = iommu_group_get(device->dev); if (!iommu_group) return -EINVAL; @@ -805,21 +775,50 @@ int vfio_add_group_dev(struct device *dev, iommu_group_put(iommu_group); } - device = vfio_group_get_device(group, dev); - if (device) { - dev_WARN(dev, "Device already exists on group %d\n", + existing_device = vfio_group_get_device(group, device->dev); + if (existing_device) { + dev_WARN(device->dev, "Device already exists on group %d\n", iommu_group_id(iommu_group)); - vfio_device_put(device); + vfio_device_put(existing_device); vfio_group_put(group); return -EBUSY; } - device = vfio_group_create_device(group, dev, ops, device_data); - if (IS_ERR(device)) { - vfio_group_put(group); - return PTR_ERR(device); - } + /* Our reference on group is moved to the device */ + device->group = group; + + /* Refcounting can't start until the driver calls register */ + refcount_set(&device->refcount, 1); + + mutex_lock(&group->device_lock); + list_add(&device->group_next, &group->device_list); + group->dev_counter++; + mutex_unlock(&group->device_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(vfio_register_group_dev); + +int vfio_add_group_dev(struct device *dev, const struct vfio_device_ops *ops, + void *device_data) +{ + struct vfio_device *device; + int ret; + + device = kzalloc(sizeof(*device), GFP_KERNEL); + if (!device) + return -ENOMEM; + + vfio_init_group_dev(device, dev, ops, device_data); + ret = vfio_register_group_dev(device); + if (ret) + goto err_kfree; + dev_set_drvdata(dev, device); return 0; + +err_kfree: + kfree(device); + return ret; } EXPORT_SYMBOL_GPL(vfio_add_group_dev); @@ -887,11 +886,9 @@ EXPORT_SYMBOL_GPL(vfio_device_data); /* * Decrement the device reference count and wait for the device to be * removed. Open file descriptors for the device... */ -void *vfio_del_group_dev(struct device *dev) +void vfio_unregister_group_dev(struct vfio_device *device) { - struct vfio_device *device = dev_get_drvdata(dev); struct vfio_group *group = device->group; - void *device_data = device->device_data; struct vfio_unbound_dev *unbound; unsigned int i = 0; bool interrupted = false; @@ -908,7 +905,7 @@ void *vfio_del_group_dev(struct device *dev) */ unbound = kzalloc(sizeof(*unbound), GFP_KERNEL); if (unbound) { - unbound->dev = dev; + unbound->dev = device->dev; mutex_lock(&group->unbound_lock); list_add(&unbound->unbound_next, &group->unbound_list); mutex_unlock(&group->unbound_lock); @@ -919,7 +916,7 @@ void *vfio_del_group_dev(struct device *dev) rc = try_wait_for_completion(&device->comp); while (rc <= 0) { if (device->ops->request) - device->ops->request(device_data, i++); + device->ops->request(device->device_data, i++); if (interrupted) { rc = wait_for_completion_timeout(&device->comp, @@ -929,7 +926,7 @@ void *vfio_del_group_dev(struct device *dev) &device->comp, HZ * 10); if (rc < 0) { interrupted = true; - dev_warn(dev, + dev_warn(device->dev, "Device is currently in use, task" " \"%s\" (%d) " "blocked until device is released", @@ -960,11 +957,19 @@ void *vfio_del_group_dev(struct device *dev) if (list_empty(&group->device_list)) wait_event(group->container_q, !group->container); - /* Matches the get in vfio_group_create_device() */ + /* Matches the get in vfio_register_group_dev() */ vfio_group_put(group); +} +EXPORT_SYMBOL_GPL(vfio_unregister_group_dev); + +void *vfio_del_group_dev(struct device *dev) +{ + struct vfio_device *device = dev_get_drvdata(dev); + void *device_data = device->device_data; + + vfio_unregister_group_dev(device); dev_set_drvdata(dev, NULL); kfree(device); - return device_data; } EXPORT_SYMBOL_GPL(vfio_del_group_dev); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index b7e18bde5aa8b3..ad8b579d67d34a 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -15,6 +15,18 @@ #include #include +struct vfio_device { + struct device *dev; + const struct vfio_device_ops *ops; + struct vfio_group *group; + + /* Members below here are private, not for driver use */ + refcount_t refcount; + struct completion comp; + struct list_head group_next; + void *device_data; +}; + /** * struct vfio_device_ops - VFIO bus driver device callbacks * @@ -48,11 +60,15 @@ struct vfio_device_ops { extern struct iommu_group *vfio_iommu_group_get(struct device *dev); extern void vfio_iommu_group_put(struct iommu_group *group, struct device *dev); +void vfio_init_group_dev(struct vfio_device *device, struct device *dev, + const struct vfio_device_ops *ops, void *device_data); +int vfio_register_group_dev(struct vfio_device *device); extern int vfio_add_group_dev(struct device *dev, const struct vfio_device_ops *ops, void *device_data); extern void *vfio_del_group_dev(struct device *dev); +void vfio_unregister_group_dev(struct vfio_device *device); extern struct vfio_device *vfio_device_get_from_dev(struct device *dev); extern void vfio_device_put(struct vfio_device *device); extern void *vfio_device_data(struct vfio_device *device); From patchwork Tue Mar 23 16:14:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE349C433ED for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8401619C1 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233193AbhCWQPk (ORCPT ); Tue, 23 Mar 2021 12:15:40 -0400 Received: from mail-bn7nam10on2089.outbound.protection.outlook.com ([40.107.92.89]:23521 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233155AbhCWQPS (ORCPT ); Tue, 23 Mar 2021 12:15:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V0jUrG5fUqFvNv4DismhgtQ3P+VYLwMYMZNXDVhLp0zqs8Ii9Za+tKyYf4jeGJe7KUvI7sa+WjNpJSqg2Vfbwzzpy7n/Tqim13UQ0LY0H7FE2DR4wpaMkd69Y8VwBXFItVfTREZGAc5bpnIElD0DisrQMNf2Y5mSxwXcgBHNH2XoxQPFlXQJLFuCSnatMkwUoGK8d/vScj7msoXHg+yr+q7Ir2hcAr6z5dxrT7CgSnmE14+V9pk08nzCLHm08GdBVedMTWpiQPVZ0mXIJ1sWLvbWNoZfR1YTGVYNKLdp1qV5LHBDvMLIoVYwjWLPSo19cVRGoqjpYLq/lBAPF/Tirw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/y54WC9a+FVWZaqM3M97wYPuLmnZdtfElYFZssIdjgo=; b=aOP1qojK7dIDU4zX788FSVxY9bggaOowYOyRvW3ly/JKP4EL1XQyP7pjEadX/DCNuz5z17u/8yxIrCn8Z7y4vuUWLYLppz2JL3fCuC7Kq4tslqFx+DobQOtfdCA31oFtvIF0rMsU0K1Xkq+lfr0ixZiAUoBP4RfYLY4UVM2cTInzQF4z4p2tm3Y9TMZyasQyV+Jhflmc/JCa1ItJbzBWizGlYC3dYvJZ39TNetI9zfaqKaDjVhp3PxO+ZzAkdcoNAQAKjOh/KCjCNdIxySsx3fFmTgRzCro3CUYV8pP9oxPyFR0HTBsVYw6sA5TZLldRgxJa2eEMRx32bkCizvpauQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/y54WC9a+FVWZaqM3M97wYPuLmnZdtfElYFZssIdjgo=; b=kvvl068PbSI2i9XMQRjFyxx3Khuz3u4by/IcMJn4l9OUiy7286BdkbeoqJKZjQFTGJdN8QAnd0J06tQFadkBQf36tUnaBvXErdzte53ial1J4A+b0w4+18qJ/kob4VC6+cJfWgLRn98tdEBaorPEl/+OzTqnhFsnrRSBprAAzui98RnEpDllDZOkmZBd84hi9ZX5xGCygVgHvSdXD3u44TuqLlPq/KMJKFVyEhOolyPNiINbse7/xzI0hYkSL1EEFRBJBf6g6sXK8QppqB9gpxp7AajiJHgSi4rfzxKyg4CmQTqsdz237H4/C9113hgKTj1qo0WAGDyZl6ZmSK2qSA== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:13 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:13 +0000 From: Jason Gunthorpe To: Alex Williamson , kvm@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 04/14] vfio/platform: Use vfio_init/register/unregister_group_dev Date: Tue, 23 Mar 2021 13:14:56 -0300 Message-Id: <4-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0189.namprd13.prod.outlook.com (2603:10b6:208:2be::14) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0189.namprd13.prod.outlook.com (2603:10b6:208:2be::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10 via Frontend Transport; Tue, 23 Mar 2021 16:15:09 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCb-G5; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8198eceb-cfa8-4989-3796-08d8ee16d49a X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QaKBvV3wH2egMyXb3DmViamSEDvenug7i7LBw20YVxq3gpCpOWmTX8sUrSOvJrpZ91sYkXAERNM8AtG36a8z/tJjLxGmW3Cu8ohydx5HUmMTUCV+dZQqt1YNiBL/jdPa+7bturuz2oo6gmiaONCnJSB+/+VsWdeKBJB8NkGyg3XywMSNJ/vRdJ1Y3Ssvod20rLNzq8KKjoq/Yqqi/G1zBUl8a6A2Pruvbcs95TC27n4v8MC36rkwzp8f5oqPcB1gm+fW5dM912q8h6vJpcqOdhFV2/GbIf86zsxdjA5m8DJAzXUM4HaX6fl54YDzJSEI3h7GkJdcu/AbdyJidRL+sA3WHqRUymj/OqLwhHeUoeNf/r7ZfZVCkm2B3APg5pKwHvwbyUqkmqFhtoUwI9PN3J5swfWI48PvVQ81C79eXGSHxLinxQ94zDIs2/jb+aNgYujyiHhwNmHPwnaYSsx+cfepz8/ei5Vcym3g1ekvD8Xj68ttSJH4Cj5WDV1vJfoZQ8IgbZMn0LERL4mw2obZmOFqqJX8HCLQMaRcn8cmBE4XNfDUEUJ/bovOPISTD+aAEygkZ57tK3U4+PfEuiwZ3AHELdCQPIynhtZVQK/FWEk1/NNRDJTqwGd4CQdjHmg6DIrFriTstGHzYbr10gvoCT+3DTNIxFkIzq5zsaJdPnacu+7CHlOt55INhMlZJtOi X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(6666004)(83380400001)(426003)(54906003)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: Sxgd2i7ByDCMbO51CQe0jSePAfBp/fbY5bSYlGoN5LjJ5ixCWxBWbQvWfHS5QHrLNk/voHOntFVM1iMOvDyBQGqVe9BXyq+opCBjV7zzH7s3Kn0oi32A9IxDkprMfwtKzfAU24P/1w3E2Fiii9jW6ZPizxRMUEz1fpOLUoijWtMskRNJKsjgONbMbkDVMrbSemTTKLktnK7Cw94asvYjzrEgFbZNbnJk+YJpTh3KjNm0ztTbAlP6ABiUhOOIPP04TlD2JVpZHS0WIXl9SWWO1bJQ7PynsHrApTwQ/Zw1/cuEJ/gtyXlgbyVL/BbjRj1jgmF1EMM0PqUzi8vPjNwsu/gLq9w9g4pPLuQn26TrDwSlRGcButclFp1aL1s5ogMJozJ4K9n/5ePXVCjB3oSjWDjP9oXQyNpjq1lTCBBrWd+6rkhrBchL4K2UlnMl8paVbPnZjgjQAAzP6WEapqEveKmKtWdCoNiPGRTiXpFZWErD8lMPziuoBwVa51sHybLAJVvMmIg/Ezwej7I3DJVHrTcpwiiCFrNThcHN5u7mMQz1EEw8qwQrfOowFHkLMqmJtcpdg/HzANwVUWqLZug5cbtBUIQpDUl7SEngD6IbRkQX7DKWKasDquRrEjq/RRMHCafhQRvWAaQTsGni2tGnH/LtYmg9a4yXW9huGl1/tpXWqXpTNM+WzB4ltSpbH7o5ZsL/Lvx5z+Dpe0XT/wQsj+w4aKOpLPEC5o4Du47kS6LLPyXTll6wyvgQci8MdqIqE2LIrjlLCcvvD1BLS2mlzkLSFxEVZNX9LCpCzUgi2lERBrpyMlrfsx0DlAOMvZNcN06iQL2Kx9YYYAJqwnzS6bmm/1QHTNUx76TrlBoutbOHwQQ0emeS/dR6c37WEm5dThrSEdaPcd1kzl3CroAwAq+6/kb3hPaaJfszhJF7pucKKOU2s4sHkawfqRbFCpGkVbaMQ2b2uyJ7MVQbr8q2TdGuqYApUtdfd48hUpjo+u2TsxnR0+htSAe2wMkF9aWYqjHe4R2jyf6Q9XPvumEwpzByILUJXT2BQYD60SdOvEbXihApsuv8JKP2D00ab+q4vuu65ZO6TJ6c8k6RJJBeB/FeevcSGxV50oK3Mhw5mJDIG6GnQCdcIKfktMzP3cc+EC58YQj1uVLIuxQuIEA3Q2cFDuDn2q747voiWemxb0XrQ8PP0XB1oq4T3jtBXMniIYVflqDhfniHZejk6d9DQ0LmAelwwQvGZcbNIBCy4uEA9K7zmVijV/4yuP5I2T40DV2ZG6h6yuIEW5ypcvMwG7b/srzoNb5cmjw9uVkEEbBJg5xjBbx9B6Il7JukMrqJ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8198eceb-cfa8-4989-3796-08d8ee16d49a X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:09.8072 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7Qt2QF8QzxltuRJloDXkceYmA0H5erdiQqIl5bFlue0jd6NWIBKRWdgIu+pW/t0+ X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org platform already allocates a struct vfio_platform_device with exactly the same lifetime as vfio_device, switch to the new API and embed vfio_device in vfio_platform_device. Reviewed-by: Christoph Hellwig Reviewed-by: Cornelia Huck Acked-by: Eric Auger Tested-by: Eric Auger Signed-off-by: Jason Gunthorpe --- drivers/vfio/platform/vfio_amba.c | 8 ++++--- drivers/vfio/platform/vfio_platform.c | 20 ++++++++-------- drivers/vfio/platform/vfio_platform_common.c | 23 +++++++------------ drivers/vfio/platform/vfio_platform_private.h | 5 ++-- 4 files changed, 25 insertions(+), 31 deletions(-) diff --git a/drivers/vfio/platform/vfio_amba.c b/drivers/vfio/platform/vfio_amba.c index 3626c21501017e..f970eb2a999f29 100644 --- a/drivers/vfio/platform/vfio_amba.c +++ b/drivers/vfio/platform/vfio_amba.c @@ -66,16 +66,18 @@ static int vfio_amba_probe(struct amba_device *adev, const struct amba_id *id) if (ret) { kfree(vdev->name); kfree(vdev); + return ret; } - return ret; + dev_set_drvdata(&adev->dev, vdev); + return 0; } static void vfio_amba_remove(struct amba_device *adev) { - struct vfio_platform_device *vdev = - vfio_platform_remove_common(&adev->dev); + struct vfio_platform_device *vdev = dev_get_drvdata(&adev->dev); + vfio_platform_remove_common(vdev); kfree(vdev->name); kfree(vdev); } diff --git a/drivers/vfio/platform/vfio_platform.c b/drivers/vfio/platform/vfio_platform.c index 9fb6818cea12cb..e4027799a154ff 100644 --- a/drivers/vfio/platform/vfio_platform.c +++ b/drivers/vfio/platform/vfio_platform.c @@ -54,23 +54,21 @@ static int vfio_platform_probe(struct platform_device *pdev) vdev->reset_required = reset_required; ret = vfio_platform_probe_common(vdev, &pdev->dev); - if (ret) + if (ret) { kfree(vdev); - - return ret; + return ret; + } + dev_set_drvdata(&pdev->dev, vdev); + return 0; } static int vfio_platform_remove(struct platform_device *pdev) { - struct vfio_platform_device *vdev; - - vdev = vfio_platform_remove_common(&pdev->dev); - if (vdev) { - kfree(vdev); - return 0; - } + struct vfio_platform_device *vdev = dev_get_drvdata(&pdev->dev); - return -EINVAL; + vfio_platform_remove_common(vdev); + kfree(vdev); + return 0; } static struct platform_driver vfio_platform_driver = { diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c index fb4b385191f288..6eb749250ee41c 100644 --- a/drivers/vfio/platform/vfio_platform_common.c +++ b/drivers/vfio/platform/vfio_platform_common.c @@ -659,8 +659,7 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev, struct iommu_group *group; int ret; - if (!vdev) - return -EINVAL; + vfio_init_group_dev(&vdev->vdev, dev, &vfio_platform_ops, vdev); ret = vfio_platform_acpi_probe(vdev, dev); if (ret) @@ -685,13 +684,13 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev, goto put_reset; } - ret = vfio_add_group_dev(dev, &vfio_platform_ops, vdev); + ret = vfio_register_group_dev(&vdev->vdev); if (ret) goto put_iommu; mutex_init(&vdev->igate); - pm_runtime_enable(vdev->device); + pm_runtime_enable(dev); return 0; put_iommu: @@ -702,19 +701,13 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev, } EXPORT_SYMBOL_GPL(vfio_platform_probe_common); -struct vfio_platform_device *vfio_platform_remove_common(struct device *dev) +void vfio_platform_remove_common(struct vfio_platform_device *vdev) { - struct vfio_platform_device *vdev; - - vdev = vfio_del_group_dev(dev); + vfio_unregister_group_dev(&vdev->vdev); - if (vdev) { - pm_runtime_disable(vdev->device); - vfio_platform_put_reset(vdev); - vfio_iommu_group_put(dev->iommu_group, dev); - } - - return vdev; + pm_runtime_disable(vdev->device); + vfio_platform_put_reset(vdev); + vfio_iommu_group_put(vdev->vdev.dev->iommu_group, vdev->vdev.dev); } EXPORT_SYMBOL_GPL(vfio_platform_remove_common); diff --git a/drivers/vfio/platform/vfio_platform_private.h b/drivers/vfio/platform/vfio_platform_private.h index 289089910643ac..a5ba82c8cbc354 100644 --- a/drivers/vfio/platform/vfio_platform_private.h +++ b/drivers/vfio/platform/vfio_platform_private.h @@ -9,6 +9,7 @@ #include #include +#include #define VFIO_PLATFORM_OFFSET_SHIFT 40 #define VFIO_PLATFORM_OFFSET_MASK (((u64)(1) << VFIO_PLATFORM_OFFSET_SHIFT) - 1) @@ -42,6 +43,7 @@ struct vfio_platform_region { }; struct vfio_platform_device { + struct vfio_device vdev; struct vfio_platform_region *regions; u32 num_regions; struct vfio_platform_irq *irqs; @@ -80,8 +82,7 @@ struct vfio_platform_reset_node { extern int vfio_platform_probe_common(struct vfio_platform_device *vdev, struct device *dev); -extern struct vfio_platform_device *vfio_platform_remove_common - (struct device *dev); +void vfio_platform_remove_common(struct vfio_platform_device *vdev); extern int vfio_platform_irq_init(struct vfio_platform_device *vdev); extern void vfio_platform_irq_cleanup(struct vfio_platform_device *vdev); From patchwork Tue Mar 23 16:14:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52F12C433E2 for ; Tue, 23 Mar 2021 16:16:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31BEF619C3 for ; Tue, 23 Mar 2021 16:16:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233125AbhCWQPd (ORCPT ); Tue, 23 Mar 2021 12:15:33 -0400 Received: from mail-bn7nam10on2089.outbound.protection.outlook.com ([40.107.92.89]:23521 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233127AbhCWQPO (ORCPT ); Tue, 23 Mar 2021 12:15:14 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OHQhHSF/0EgU/CDVrT4XUErMPHtuisKJa4m4tIMWVMzfpbzLJ0a8Zx1qg+GX0ezk+Xz/vuqq6mY8p2s2rQ/kikYJPbcixhiKXuRFvHAGCdZvV8U2k9GlWdlb9v/Lz8SbNL5FhVgZmHJ9+J7amS7bVjOMvQCfZMCdA9+oBQ4K4CjGWNcfjZtB2E6DUQ3HXNLfsdD62CX/yAvHDOaiSSDd4rKx21oueWYfLWk4UdrcONRWMk2iWMeB8nnx2v+KNmFWat+qi55Wa9eSRhKC5c1r7K0+IKGjVp1a11Yim5FqKQqjD09wUNgc9XYNGwBMx6vyE0ud5XouatYa07oxorVjkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k3Qoue6mKiKM6pBM0skuOa/a/7zY0XMpX4DKf8IH8aY=; b=E4uqviuD9CGlnblPQ8OldZ630ysEBCj/f83W+anojmq3zRd6l9x2S2t7QLnK+1S8DuIEZJh1xRwK1z1Xr77E67Azh8qidekUFwHSCz+LuPM7a9pdFey1bWYTeONdy+prF6HRyBrG6XTk9HDLsK0/5rAT/LwV750Djj+EKZdy5FoL7dJTw49IAqBoBhon2ZOMwTbwHEsM5u8eNhNSKx/35jUKYY5k3VExPfS09MozlQRnlz4n6Z/CinuoZ10h/J1dhsjf3idaL6QUnCAtCsHDug//DunvgQDq62ZPDtMW/pWnU1NDwq+e0U15jNxqOm37e4orXWJscB1UsTLltc/FNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k3Qoue6mKiKM6pBM0skuOa/a/7zY0XMpX4DKf8IH8aY=; b=UVq9QEf9b+E4+l7wwRWm7WVoOax0/fKWcKzegoR1zOTK2p7D6jOJxfVXLTx5ToUAZPjJOHKxK9E9ZP2XAw9RQ7ZCSRLivTyvF8dDXWAfz7WS8Nvx538oi9xxNR/oUpSVDYCIB1sF0P4q17MlDPnbL9KjlIcspqGod14EZ+tHqrl+xbc5gPPkrQC33cT4LfBVkwZCr/BMZ0gBApu8LckMb6JfAmFzv/esw1ZQHvvejDnwP4Ke0GTZDJHMCGzKOa1spI/apYmgcJ7kZeL4rs/ETs+AG/s+JYOrATOzUyOAOWLtSH9L5F58XMjIN06qf7J8za61xB0sExH3v0coDF+/4g== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:12 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:10 +0000 From: Jason Gunthorpe To: Cornelia Huck , kvm@vger.kernel.org Cc: Alex Williamson , "Raj, Ashok" , Bharat Bhushan , Dan Williams , Daniel Vetter , Diana Craciun , Eric Auger , Christoph Hellwig , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 05/14] vfio/fsl-mc: Re-order vfio_fsl_mc_probe() Date: Tue, 23 Mar 2021 13:14:57 -0300 Message-Id: <5-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0173.namprd13.prod.outlook.com (2603:10b6:208:2bd::28) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0173.namprd13.prod.outlook.com (2603:10b6:208:2bd::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.9 via Frontend Transport; Tue, 23 Mar 2021 16:15:08 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCg-Ho; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 71b7bca2-b3cc-4ce4-556d-08d8ee16d3e7 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IIvCQGLRA+mDSWFJ3Cqna2o5sS10M/3LvnmpdsDVRtaFnxK+nqBAIhuC3M4anbkmkdQ1wQrZk0T5A1pbFEFRwfGnVH1G3PXnCZSX2gUMosf3wNO/WWzmMEV+Pz+Y5zsNE7mGzYuJi7oAzn7E5L9LZWw4Vy627XbCU4ExsmMzaT8Ocpkk0XwZWtUEVXMZMUoekXyFm+dQEwRtu+tSu47MHafgy1/mXzR1Np2LFR9kSOzKe3+cXfvxWbLqg8G2jXC9NmxcsdBjnOE2D/wRDxtyQloufGmlCEIZsyu8fqlYbQBF4CJZQ2L/QU3tYQBkULmfR/3rqo64omE8emimxoP9lqFrGQIOH2769T7qacnD9itX2dsC3I+Zrb7sLVl9yX/VwLiW+uzTwxDxZ8OugLtybmq2pndzO9USJDHBKBa/OBIVJrd/xTnxNedDwsBrCRL4oZk9xUzZyCI6T7lC4LZooMmXtVnhzsAEXkSohfrcYU6KtfRPt3VN/fMIUgtN2XHfxWNGlPdyFgmMoL3Nb8DFXoDSA74tu7Tc2OENWnLzXeWrVkJayuacEMPnFWZiHgCjoM4k6VTJ8TmoTaMqULv+mlPXawZFcF3gnrFE93UePZYg0bzb+xFv0AUaN2oVfsGTLRWJwo/tY5rJjTH+ZAIynTTbxDcQkv+oiQNHXzFOUl8JR31clktRTrMTaSFF3VIG/5iB6MSeC3Tu0z3fdKyHyg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(7416002)(6666004)(83380400001)(426003)(54906003)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001)(4216001)(169823001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: hfQl7jJEgW30aqAVR5IZCl4HvhMavvGLJTDZYjvcXKu5WgXQFcwwcZ2IuCX/X/Xnye+xPJMGxOU4dePmdll3bTFRiaJTBGKtQCB7tjgBjBdQYScBdHkW3uuYfzv0i6u88NxpMlHSwYeH07s6rajRLuPE1feK8OU5wijT3FJ31Gh9bcyMi7A+MXy98KaPWbabUxQV+vAZ1FxzQ7MzozdFYFmlsynDKIVCkpk72kjYES8OBYzPemHbFcJt6EeslD/UAY2fy52zBKSwFOGzGc3FFivCSNaxEKCloTAKdvtkn1tu1gw1mvVLsddOejxeBNwgHhhS+7XCSxecWeco9UeuGMCWq180wChySOsYa58PCnpSeavC7CT7TZiK5cSXj6lYLTfVEqBoENxY3cBWMqcDbEReIdbUoX/A+aRMNPdG4M82aX8Ck9UdlfSwfs6KCvZmZrVY9MkAmulkYGua38nw7sGul/Y8RrWWX+yRQ2A98h8C+igCOQ6xpy4Lsa/quoHStTlPte9qurnDDygpeS5I6kmCP1WUK8DvQdesCwbQIEbVtToN/APFxb3WE+KHF9jwvyidXnPltR8EkU1o49NKydQo0t6goINJwTRRYfy+7syNCSBLG0Ct+8GwTek4AoPSEl0s6jnVn3KtVrUjikY87Vq1W3hQgOXcEAM0WkZQLJiO3EpAVcVchu0zUUO5hsifhd0WG8oBfI/3grxrdGsHdlqzem0/sXN1vaqwXHVttYilJ3BKhUD1sJMD5sThBSkJFk8Ymt6JrrpRK9EOhCtuF37oQQtNI1KMszSMzGkAbwPoVMCFnS4tApaLoA0DjUxZp0tNL1gtXuYCfGIExGNkictc2HHhlaRoeYzuyx1z0Q1Ou+EhfGlkYBpQZWYcaZoPRdb+SouLlvfofiU+TLPJ6wObc+8aGeChKfdIlUpx0nW6uRIVqMNGQWB27YOOLPmkSOaebMKrfoueJBgmArg5djXeCCsU8C1QbdL5WZai+VPBtRFkNhtZv182JQ8Rw7MUcv6ZE0WqDKfieGMC87WPRRJOd+GZ73aCQSMgi2Az1JQ0znld/O7lw64e3Bc/3FCSooAq4vCMNAL+8RZYmbCjL58RW1ISB67Cm3RdOeKQ7hTWqDbMgOwtDwNdfRQiZJ6ki/MxcNPApl4Wj/auZhdnC06LXo42xGlI+TbFiEaslErpKxIJVcitB7zOO8fvh2rxFMzit400NyvqW6pbuMBhQsdNFjbCxHHrViCdlECEIJzB7MzmM4Svb+nRilaEYcGTJJ072aK7zZF9OkhqID1uNdcm0iPRcKkNtvECXeHZYGXQRj1rV1ZNubaHJGKd4xO/ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 71b7bca2-b3cc-4ce4-556d-08d8ee16d3e7 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:08.7508 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tMBUpRVrW5neZmA+U1TbLGmcWNJaRw68zKguOrYXLWxRIfHrHy0/EgLyE14bWsm9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio_add_group_dev() must be called only after all of the private data in vdev is fully setup and ready, otherwise there could be races with user space instantiating a device file descriptor and starting to call ops. For instance vfio_fsl_mc_reflck_attach() sets vdev->reflck and vfio_fsl_mc_open(), called by fops open, unconditionally derefs it, which will crash if things get out of order. This driver started life with the right sequence, but two commits added stuff after vfio_add_group_dev(). Fixes: 2e0d29561f59 ("vfio/fsl-mc: Add irq infrastructure for fsl-mc devices") Fixes: f2ba7e8c947b ("vfio/fsl-mc: Added lock support in preparation for interrupt handling") Co-developed-by: Diana Craciun OSS Signed-off-by: Jason Gunthorpe --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 74 ++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 27 deletions(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index f27e25112c4037..8722f5effacd44 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -568,23 +568,39 @@ static int vfio_fsl_mc_init_device(struct vfio_fsl_mc_device *vdev) dev_err(&mc_dev->dev, "VFIO_FSL_MC: Failed to setup DPRC (%d)\n", ret); goto out_nc_unreg; } + return 0; + +out_nc_unreg: + bus_unregister_notifier(&fsl_mc_bus_type, &vdev->nb); + return ret; +} +static int vfio_fsl_mc_scan_container(struct fsl_mc_device *mc_dev) +{ + int ret; + + /* non dprc devices do not scan for other devices */ + if (!is_fsl_mc_bus_dprc(mc_dev)) + return 0; ret = dprc_scan_container(mc_dev, false); if (ret) { - dev_err(&mc_dev->dev, "VFIO_FSL_MC: Container scanning failed (%d)\n", ret); - goto out_dprc_cleanup; + dev_err(&mc_dev->dev, + "VFIO_FSL_MC: Container scanning failed (%d)\n", ret); + dprc_remove_devices(mc_dev, NULL, 0); + return ret; } - return 0; +} + +static void vfio_fsl_uninit_device(struct vfio_fsl_mc_device *vdev) +{ + struct fsl_mc_device *mc_dev = vdev->mc_dev; + + if (!is_fsl_mc_bus_dprc(mc_dev)) + return; -out_dprc_cleanup: - dprc_remove_devices(mc_dev, NULL, 0); dprc_cleanup(mc_dev); -out_nc_unreg: bus_unregister_notifier(&fsl_mc_bus_type, &vdev->nb); - vdev->nb.notifier_call = NULL; - - return ret; } static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) @@ -607,29 +623,39 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) } vdev->mc_dev = mc_dev; - - ret = vfio_add_group_dev(dev, &vfio_fsl_mc_ops, vdev); - if (ret) { - dev_err(dev, "VFIO_FSL_MC: Failed to add to vfio group\n"); - goto out_group_put; - } + mutex_init(&vdev->igate); ret = vfio_fsl_mc_reflck_attach(vdev); if (ret) - goto out_group_dev; + goto out_group_put; ret = vfio_fsl_mc_init_device(vdev); if (ret) goto out_reflck; - mutex_init(&vdev->igate); + ret = vfio_add_group_dev(dev, &vfio_fsl_mc_ops, vdev); + if (ret) { + dev_err(dev, "VFIO_FSL_MC: Failed to add to vfio group\n"); + goto out_device; + } + /* + * This triggers recursion into vfio_fsl_mc_probe() on another device + * and the vfio_fsl_mc_reflck_attach() must succeed, which relies on the + * vfio_add_group_dev() above. It has no impact on this vdev, so it is + * safe to be after the vfio device is made live. + */ + ret = vfio_fsl_mc_scan_container(mc_dev); + if (ret) + goto out_group_dev; return 0; -out_reflck: - vfio_fsl_mc_reflck_put(vdev->reflck); out_group_dev: vfio_del_group_dev(dev); +out_device: + vfio_fsl_uninit_device(vdev); +out_reflck: + vfio_fsl_mc_reflck_put(vdev->reflck); out_group_put: vfio_iommu_group_put(group, dev); return ret; @@ -646,16 +672,10 @@ static int vfio_fsl_mc_remove(struct fsl_mc_device *mc_dev) mutex_destroy(&vdev->igate); + dprc_remove_devices(mc_dev, NULL, 0); + vfio_fsl_uninit_device(vdev); vfio_fsl_mc_reflck_put(vdev->reflck); - if (is_fsl_mc_bus_dprc(mc_dev)) { - dprc_remove_devices(mc_dev, NULL, 0); - dprc_cleanup(mc_dev); - } - - if (vdev->nb.notifier_call) - bus_unregister_notifier(&fsl_mc_bus_type, &vdev->nb); - vfio_iommu_group_put(mc_dev->dev.iommu_group, dev); return 0; From patchwork Tue Mar 23 16:14:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDABEC433F2 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD6D7619C6 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233196AbhCWQPn (ORCPT ); Tue, 23 Mar 2021 12:15:43 -0400 Received: from mail-co1nam11on2068.outbound.protection.outlook.com ([40.107.220.68]:19553 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233167AbhCWQPU (ORCPT ); Tue, 23 Mar 2021 12:15:20 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FDoagTyEcJJ08RmITZy/X6hgd1/ov09zTjqr51mdlPZupgQDKfc1YmSJzpjA7U4fV0qIG9cL7Ph3UGvIm2gQd/Wij2x5pdtDze9jrANpRYpKJOlr7Wvyl0m13WM8we/2T3Jc7Yh2sF6FGtetQscay8XFedhsyMXCEWW64CrfH/K8RhnbAMvIJMQP55SQ/RJpecU1zI06nGQJa9zpgH3yMzSbL1fMnDFppyE4tfdXnMasQG0anEUt2dtV64JJXZoqwHo1xB0nEXY82z1WnGqSi888NUS/AueGtVMQjyr3gUXL/0IF16sH8UQ3EwsG+tUTiLU4/N4UikDHaMVCTAIcZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1Au+jz4Mnc+9WPJjKoH1/bA/zwfibxm5kZdV034G6lE=; b=g4SiSUTokZalXZUlpFocC0opE+dzgevcE4+D8dgxrzMcSRmOz1rTZ8+X84CkxvXas5/Qy+wylTUzqgOWucm1ipt15QKyGT3ti0ZgcNNWgPnnmBDd5ONg/xxrSge9EpRF85Qnm2uV3EiCWwOXzLLAxagKkHu54gizCeaV/8l4yEkQPvZuu8gxjzLCeQUHYBf4CilDj2yKZDbk+xRLMd5GbVOXm8IXc+uNlJBsRGRpIkPZVC7v1IjVpZ+I01JW3gF1D2uL7eLQInZm6pejPWfMVHwb8MixyQyjUXKsAocxk3MuQkpc8V4zFgpggPnGsBok9t62S59J/u5nQrAXspxJYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1Au+jz4Mnc+9WPJjKoH1/bA/zwfibxm5kZdV034G6lE=; b=aw80IOFKFscOCZXJDIA2p1L07k1oYp4yqD8XBH5Q7woIMjSD3c3YYX5+NO8uT6zkP46ShnTixRE1/rGOdkXZtDx8yuRo20qTVqZ3Cv2QqQGsdL6QynRFvPFnTgIlKR9MlHchQt0/ljLZFmhrZvSumOOY19XJbcbrOYSDcHjqRYDAkczauZYt1z5KFWSkrYCUaT+QDtMxM19ZpmYtC1v9hnxHqKxvRv6SE12z9BNbZHQNEdFRR/etuemACvWahFPFbwGTmaF6l5FS792KuGmSGBrLIJGPNAS57rZJmi7IMWRXRn2GenQyUljl4i9rnA1xZFbWquwZplDTte9buCcKnQ== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:15 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:15 +0000 From: Jason Gunthorpe To: Alex Williamson , Diana Craciun , kvm@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Christoph Hellwig , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 06/14] vfio/fsl-mc: Use vfio_init/register/unregister_group_dev Date: Tue, 23 Mar 2021 13:14:58 -0300 Message-Id: <6-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0207.namprd13.prod.outlook.com (2603:10b6:208:2be::32) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0207.namprd13.prod.outlook.com (2603:10b6:208:2be::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.14 via Frontend Transport; Tue, 23 Mar 2021 16:15:11 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCj-It; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ace483f6-563e-40a7-fef3-08d8ee16d595 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IwTMi2Z14ZVhTKAXt9IuYxIuHi0eTjeR8aV7xc8eaiyYcWxrrfpuRcHqfOIZ8gFcYvBAJy0hHMBkBcKfuYOSFAZPjs1XLCnnVcBBAsYH5v76VEY6lQYFMoANzFRXbWvgk6BoeHEi71W51d5KI3BASsgPMOg2a7fe1NqlmsY4Le0UiFvxPcQN34xMypdmWPoS99V/8AD6MYFpLiiws1vGhtyE9a7cm/SHWigqzYS6G5j9vA6EQwc2KaM2+g/wyeYi+LtZ9/h0Der3iWUYWC8YHFirZOxjd+3DOAo4UfsrVuoxD1VW+FOqgmRwMgCMUOlKk/dNak44JcMYRV8HevNUqTbDsLcF+jGq9nUTQRwDAyGLY3IJhPG9fMv/0z2lhKy7T45Pnnteu7FBS0bdOMcxBnC3cJtK4OmNWzGxN01m0jW06d2YKNiQeKxt2j/Q+zvumP10jZ42bufLOPQvsQARlfePE4hN2QzKs1e91XyhOZ+i1c9J6K5SHUlQtVGim2+JI6Bxy8qFml+N6bBDQMRkpM+L1PEead8INE/OTFjtk3+zFDafBDDAJcqVtGDI67q6mtJ9PxbTp3qsijl2tOubIqajM4wi4OOacCqpsnB4qF8sjuWY3wVWZjc+gsCtseN5V+R2OSZ+lr3HnX/pXsBnf8w0sCAr9/l7FFfDEUgkY8IlWWKRneDeAtxX7in9NY6V X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(6666004)(83380400001)(426003)(54906003)(110136005)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: KrwwHdrzalwT7w535bjfmbWS8z2Hv5nVS3eNz/VvnzGDQd50XycZ7KP8r3oSjRLdLm1da8SRDM+FZjAswMSN9ejK1S30h2UCBT2BaQzoAwLtTJzF/P6JsRcHW/ETbNyGzMOgDmcSuIgPdp35tqT9zBKqPZ3+rGJ8Mc11BNibmToovDcTFYlKngya676DStZRo9jzslTs1zgBpj3JYpo2arX2MshUd0VfqUcBEUXAGxPWx+yGEI/O0C314SnTZpwYc5MPB3LrERt1BsbnZp5Ozqg9Rg6BeDGFD15C0GvahuOxYo9HGWDi9FUqRE80AXcsioEqDf6CcoNmwjO1OjVWbBJycWoYU7p2D5yi9UV9dneghY9kZKNcjeK53zH4gmo0RXwW+R92ogkxDFtChWNUpY8XiW+nfcEBCGR2z/+FlPfU59+P86scIXfzu0SMa3X/bw0dz20LKK2IoL6o6bNcTCp7d7VTfhJN9Vi1OJS3JF2sPH1o2k5T2p+BfdcWXSTYGlSV/7kRuKNGUotYiqCKfu3SPaiMMdNGnATbsoqDTPjMOIf6S1XKVGDgP3EKNm04bGqOfTvw8bDxTOjDH6SH/Qu0buRJlJIX3LGCLjAKK6NiGoctLv7weCa9LFyNC9mnV7dLnXGxZSq8iUjpqGUY1ZYxAznssh36C4JxRsZGR0TtA5c09V7tyFV79C55ahTHpeKxB7yuy2I615CcEwTzfugDNieP7raU9w6yXbPYA/cWuKCA+0aDbwdtEG+J1mnCMZSUvsSFDpELex8XLDpPGwvT3sq6b0/jZRl6nIWX3RdWrUMMGuLI6oVhSq20ibCUVIjkUXGchmXxQf8x1a0k3lm6gY5vvwpuA8juvJAXnQIECHz4dgNksVMI+vs3+iM1pMuM42KwZDDMYmIlnZ4uNTGZ3xButSGWjKv5la9C9b5amsaIPlKmdG4JpFMX3i5bW11+OBtjcu85D3PecpC618qJ2NF2x+vjgX1vJd3CrcfrHY2LqpAXiYG9/JtkcIRm6AsauS6rJYtEMOqxHcf6fRmL8GBH04VpBVipoUKmBknCuH+VLFnZpFnDmBpgjBg1oIFueDz33PPojYztecIub2O4KBf9yPeWBBs+4nyYn6ZNMp0KrAsmDnUOKTZx0nGHra0XCZrAnZT1iDIMaU4HPO5nr95FJuVdIh0seBXHu6rZhCXVQj/sIO1NeQknRBd7+ziEt/EmiAHKiv56CUZX6OE79SlFPtqxuLdpZdSgg642NIhE6QS8Y7/CCLQBxqZaMyR/SY3I4pfYN9n9g+ms4yngcPt7FQtQiXUm8Wk9jGc9rSHQVWWKTB8XgZhYKnbH X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ace483f6-563e-40a7-fef3-08d8ee16d595 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:11.5422 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yaZZpI++uRCaUNmriY8vJPUGJdkj5NftXnERvGyWXxgEfWWG2ikaqEbB2SWX6QkR X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org fsl-mc already allocates a struct vfio_fsl_mc_device with exactly the same lifetime as vfio_device, switch to the new API and embed vfio_device in vfio_fsl_mc_device. While here remove the devm usage for the vdev, this code is clean and doesn't need devm. Reviewed-by: Christoph Hellwig Reviewed-by: Cornelia Huck Signed-off-by: Jason Gunthorpe --- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 20 +++++++++++--------- drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 1 + 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 8722f5effacd44..ad6c6feeeb4b2a 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -616,24 +616,25 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) return -EINVAL; } - vdev = devm_kzalloc(dev, sizeof(*vdev), GFP_KERNEL); + vdev = kzalloc(sizeof(*vdev), GFP_KERNEL); if (!vdev) { ret = -ENOMEM; goto out_group_put; } + vfio_init_group_dev(&vdev->vdev, dev, &vfio_fsl_mc_ops, vdev); vdev->mc_dev = mc_dev; mutex_init(&vdev->igate); ret = vfio_fsl_mc_reflck_attach(vdev); if (ret) - goto out_group_put; + goto out_kfree; ret = vfio_fsl_mc_init_device(vdev); if (ret) goto out_reflck; - ret = vfio_add_group_dev(dev, &vfio_fsl_mc_ops, vdev); + ret = vfio_register_group_dev(&vdev->vdev); if (ret) { dev_err(dev, "VFIO_FSL_MC: Failed to add to vfio group\n"); goto out_device; @@ -648,14 +649,17 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) ret = vfio_fsl_mc_scan_container(mc_dev); if (ret) goto out_group_dev; + dev_set_drvdata(dev, vdev); return 0; out_group_dev: - vfio_del_group_dev(dev); + vfio_unregister_group_dev(&vdev->vdev); out_device: vfio_fsl_uninit_device(vdev); out_reflck: vfio_fsl_mc_reflck_put(vdev->reflck); +out_kfree: + kfree(vdev); out_group_put: vfio_iommu_group_put(group, dev); return ret; @@ -663,19 +667,17 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) static int vfio_fsl_mc_remove(struct fsl_mc_device *mc_dev) { - struct vfio_fsl_mc_device *vdev; struct device *dev = &mc_dev->dev; + struct vfio_fsl_mc_device *vdev = dev_get_drvdata(dev); - vdev = vfio_del_group_dev(dev); - if (!vdev) - return -EINVAL; - + vfio_unregister_group_dev(&vdev->vdev); mutex_destroy(&vdev->igate); dprc_remove_devices(mc_dev, NULL, 0); vfio_fsl_uninit_device(vdev); vfio_fsl_mc_reflck_put(vdev->reflck); + kfree(vdev); vfio_iommu_group_put(mc_dev->dev.iommu_group, dev); return 0; diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h index a97ee691ed47ec..89700e00e77d10 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc_private.h @@ -36,6 +36,7 @@ struct vfio_fsl_mc_region { }; struct vfio_fsl_mc_device { + struct vfio_device vdev; struct fsl_mc_device *mc_dev; struct notifier_block nb; int refcnt; From patchwork Tue Mar 23 16:14:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD159C433EC for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A5723619C3 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233190AbhCWQPk (ORCPT ); Tue, 23 Mar 2021 12:15:40 -0400 Received: from mail-mw2nam10on2048.outbound.protection.outlook.com ([40.107.94.48]:9569 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233158AbhCWQPT (ORCPT ); Tue, 23 Mar 2021 12:15:19 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gj0rZKm00egNYOdzhCj6W6Zx34lDyvgCQVN9OwwRtSYMT4b+b35ITT7wLdQxyc77IbqIy3w2AqtcbPWg5LsuSwSLaTPGTB9o6PSjzUVLfWZJxeMAL4mu5k2MfIsj0lnOdLNeVgOOxWxXVaW5eeBvssRS5vbGXKlwwHRrffyu/K3+ELj/Ke9jV0N1qvidJEC5Czv1K2z8mLFEZub2KGFK+6NV0LC3c597Jp+LNhE1SSR6DrFVxSCFDKW39u1YG0t4xwnN9LWFar2wAeF7K1VOvbaJI9/KdQXXyZEMtXy7B9a9qsjN2NpyHbkO07TYMI1+lVIPmP9mVw4nwgL7nAISNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XFl0KQ7jEkK//3KiO4casjYqXGqjMU/0R4gw7ZdqNVg=; b=TY4iHsMzUs/26NTyWwvemYxjjI0BbbhsFlR7s3WMs494Da0QYX0ECpEFrF8w6mYnuVtSy78SUxjJsDRcyczkNXDLjgooZkmN/GYCC/FsK+6E4ISIfVp3nccydA4UdolRzvd0PR0xhh3dmjMOBE5smSOVoKkdKsgzjA6UTfnpYorwNtEvVucp8Kh2HVldQyqIhUFXhGObuim0WgdGQO40zbMAJ5oDaFxv8kUWkEBx+SkKOGOW/nAPKLEsNOANgw+KMbGZLbj/q5Cg8zYjOFQP28dnCySHaDGczOS2xSo4QyNjHuEVd8kHnTSP8Ci/kKjSBUp+dUmDq/g1P3asctTQXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XFl0KQ7jEkK//3KiO4casjYqXGqjMU/0R4gw7ZdqNVg=; b=Txmykzpf8+lUdwycxEAnD4W+LOCQJHzcub4hD2+e1DjeNlEF+oPfRhB4oaTbvwkRTFNJGY8q7spVbdgJBVNTznrBudUFvoidtkj4KJAX7DkmDuiXP3UfXqqrdTRZ3z+YNtTR5lHwQUkX+CzK3kEvz1/Th9+AeBhzxswyMzV/wF9U0ibaJ+NXBwC47uJVCJH2TFMziZ+6vXDBCE7Mb5APGKuk/MfGEcMN4yTP5XBKqzuOTX9BdEzG9eviThFU/s9j2wG5PmADM8DB76G6B5kd5xi6qAaWEx7pesMkcNs8aWTAJ/uQYTq2+pCHvVibDqHmnkL9H34+2laHEuyDPiGMRQ== Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM5PR12MB2440.namprd12.prod.outlook.com (2603:10b6:4:b6::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Tue, 23 Mar 2021 16:15:15 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:15 +0000 From: Jason Gunthorpe To: kvm@vger.kernel.org Cc: Alex Williamson , "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 07/14] vfio/pci: Move VGA and VF initialization to functions Date: Tue, 23 Mar 2021 13:14:59 -0300 Message-Id: <7-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0193.namprd13.prod.outlook.com (2603:10b6:208:2be::18) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0193.namprd13.prod.outlook.com (2603:10b6:208:2be::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10 via Frontend Transport; Tue, 23 Mar 2021 16:15:11 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCo-Jw; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a993f366-0b02-4602-a9e9-08d8ee16d5bb X-MS-TrafficTypeDiagnostic: DM5PR12MB2440: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jCXdS1G1uEXmUSgSBZFux0iUzIvnVQV1zZXieC6cj7Kx+ysqmK5gEXXyq2Hod4go1O2RdY0x8Wrqklc6/7CG7ZAPECdyPmz555CU3zAAvlp0aMBpkOD/aROkHAolr3VYzTvu6rQVTxsjlcmM6auzczu76pvQ6mVm2wL1a9FxaSxacNzpj9EjeBasLxLZzuUY5YIKWEvlDR5b0FNR/TbjxPFP3Q16Gmq2375fNmqYZdQsIvKsWc9ns4WzZ6tMjnuU9IjPgmRLBi0SngEu3dlGh/o/+QpdukCFYOyENedeZmzDDHGALFNXxZQ/RjwGHv/YXzrdWyfW9aH2OOE2tyoW6llocUcUAZKJnwCDqTk81Qhlkk1CCK/XKYedyjGJx/RhQw8j2OZGav7OnArZyBAyezIwjlDy+I5y+c/jnzVj2J9xIJhXjjmCCFFsCvFptcpIi8W540KVYUxmNVuqX1GRtdzJ+eylhDhDO4Ng2bWHLy9/jTmgjLGkEeOXA7+wk3SzbtosmnmFj56lkrNEMZLLu9bmX70Ms07s9pdGQcDYuI0XAt+Fe+VVmsnX5mO/GWwLk3hF3Y1tr6YJGg5a249leiC8HdqC4xRE49gH7QtuX5tj3ca1zFNmO0w8+jXBcbCmpMWeQWraqwH0+SNitz+Q1Bvdch64js8lbTApORQHflbsYmm8YkWZN/jhcYColsqS X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(396003)(376002)(39860400002)(136003)(66476007)(66556008)(186003)(6916009)(36756003)(6666004)(86362001)(38100700001)(66946007)(4326008)(107886003)(8936002)(54906003)(9746002)(26005)(478600001)(426003)(8676002)(9786002)(83380400001)(2906002)(316002)(5660300002)(2616005)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: zY7uHECWtf4joSMewvhczAxLJ0ztTDm0BhbYzzGTgVkG/8Av603E3ZEey/0UUD3gcdo5PKXdxg7w3kiGruux5sOBC3t3Pui+C0vxvXvNWuHcgmxu+mCbZ6SvNoKCB6E6RFXjzdrF6qcGwg7POnMAAXSW8Oh4G1NfJDJ3YXqkzS0G/dB3TE0sN+WjP+M4nPJSwWwKOmHss0pwjZ6M0FSin7B6vna6+rDpmhdLdOtAOPlZusbzY4MejQ5v6Fd3rHqPl+oM0KvZWWQRlKC5+3quj1kgfbzkrydOe9Db7PdK0VQ87vFn/5R4aZL+19l8PohaKmjS3tJPNhfyE2Iw/slNS1ePsKr67Gc3LaHuOQqKytUhZ7pQi0JiZ6AkK+soXwh6K2Zl+HieqPOW9fp0tOXcx2tqCdQm1Ibjxt5KR+0RVdRJXV7Vfud2KgaRid5utrryQxnPW8YLAsb2Y0GA6bpuXy1nCJlNI5Se7vA4Y7ni9iQGeMIINaDn+eTdKV6+vj5uIaHgDwLdpuiNQms5mUDTU+YqRr6AWonlnqw1tEca8z4TFCRzOtv5R9DGoA0VN7+VqAC2aEAXEH4soASqA+Is1+ejnTqp8TQQ+QGqjxF/W+3MlBhmt8nQsX9Dg240nrSk/59klBD0Iu+gr09Vy8t7EbhzLq1YSwDLzxkkbfifBzyTxIXsHDG1uugiUsKPCAgkmpChjCNTrQ6U1Mc+MeXl8zCPiAIm9HskWQPIMOGgDn7TNanPp7BqtGyfHgkA4ob5ef8a1Zv++E6fAeL4bZfYtgkwQHgk3ztcfoEV3LXYNHwocio1U/c/6ImBqymuE1+rZnH27YzKCQhtO90sPFV4qQnaFEm6vIfCNfnWB4OVcSETBlAHHrTQiIHSq4QnwjHkTHrQ/99MEC6pdFZKf6G8uGKeIIfSlK62YsybF4PbxkHQW+wZs3RlnDIufu5jC3hD+XLT891wnDBUyDqx+iZsk8cb1+GuGMv3NnDY6wEbmSrxYeQz8CZ5oMVZBvgEeCG0l3awaKut6tNVNc67ArJTf24tLniAyQpXleEftwPUPqjfXh6jG7OufBV23Lm1X0aJI4TG0ZLveyiApvnvzVITw+81G0YUrsYTd8bbU7RNXRRw949WFvDWIs2XdwbVSh5ySvgdon65CvuvwSJ9Jb50ZfSnszsTeFVqgNNERuAOkPsJsDdf/RBnHqgeQRilyNBHm9SX4pz/L/fT/ZKTUR7BE0klEbwqWnOMicDFD+ffu2lX+9GzxAZcF4bWDwnYMxFNQs7fybufJhdaUkEwNK8b786XZclYn1FXHMcMa0Ic4cTEUDKQY4/pGKPm5gi1HsSU X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a993f366-0b02-4602-a9e9-08d8ee16d5bb X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:11.7580 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ObBQMx3wnXiuCw552JJJx5DuN/FyZ5DydA+fGp4ebR3W8sIRodV6X89qHD2ktNQh X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2440 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio_pci_probe() is quite complicated, with optional VF and VGA sub components. Move these into clear init/uninit functions and have a linear flow in probe/remove. This fixes a few little buglets: - vfio_pci_remove() is in the wrong order, vga_client_register() removes a notifier and is after kfree(vdev), but the notifier refers to vdev, so it can use after free in a race. - vga_client_register() can fail but was ignored Organize things so destruction order is the reverse of creation order. Fixes: ecaa1f6a0154 ("vfio-pci: Add VGA arbiter client") Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Reviewed-by: Max Gurtovoy Reviewed-by: Cornelia Huck Reviewed-by: Eric Auger Signed-off-by: Jason Gunthorpe --- drivers/vfio/pci/vfio_pci.c | 116 +++++++++++++++++++++++------------- 1 file changed, 74 insertions(+), 42 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 65e7e6b44578c2..f95b58376156a0 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -1922,6 +1922,68 @@ static int vfio_pci_bus_notifier(struct notifier_block *nb, return 0; } +static int vfio_pci_vf_init(struct vfio_pci_device *vdev) +{ + struct pci_dev *pdev = vdev->pdev; + int ret; + + if (!pdev->is_physfn) + return 0; + + vdev->vf_token = kzalloc(sizeof(*vdev->vf_token), GFP_KERNEL); + if (!vdev->vf_token) + return -ENOMEM; + + mutex_init(&vdev->vf_token->lock); + uuid_gen(&vdev->vf_token->uuid); + + vdev->nb.notifier_call = vfio_pci_bus_notifier; + ret = bus_register_notifier(&pci_bus_type, &vdev->nb); + if (ret) { + kfree(vdev->vf_token); + return ret; + } + return 0; +} + +static void vfio_pci_vf_uninit(struct vfio_pci_device *vdev) +{ + if (!vdev->vf_token) + return; + + bus_unregister_notifier(&pci_bus_type, &vdev->nb); + WARN_ON(vdev->vf_token->users); + mutex_destroy(&vdev->vf_token->lock); + kfree(vdev->vf_token); +} + +static int vfio_pci_vga_init(struct vfio_pci_device *vdev) +{ + struct pci_dev *pdev = vdev->pdev; + int ret; + + if (!vfio_pci_is_vga(pdev)) + return 0; + + ret = vga_client_register(pdev, vdev, NULL, vfio_pci_set_vga_decode); + if (ret) + return ret; + vga_set_legacy_decoding(pdev, vfio_pci_set_vga_decode(vdev, false)); + return 0; +} + +static void vfio_pci_vga_uninit(struct vfio_pci_device *vdev) +{ + struct pci_dev *pdev = vdev->pdev; + + if (!vfio_pci_is_vga(pdev)) + return; + vga_client_register(pdev, NULL, NULL, NULL); + vga_set_legacy_decoding(pdev, VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM | + VGA_RSRC_LEGACY_IO | + VGA_RSRC_LEGACY_MEM); +} + static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct vfio_pci_device *vdev; @@ -1975,28 +2037,12 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) ret = vfio_pci_reflck_attach(vdev); if (ret) goto out_del_group_dev; - - if (pdev->is_physfn) { - vdev->vf_token = kzalloc(sizeof(*vdev->vf_token), GFP_KERNEL); - if (!vdev->vf_token) { - ret = -ENOMEM; - goto out_reflck; - } - - mutex_init(&vdev->vf_token->lock); - uuid_gen(&vdev->vf_token->uuid); - - vdev->nb.notifier_call = vfio_pci_bus_notifier; - ret = bus_register_notifier(&pci_bus_type, &vdev->nb); - if (ret) - goto out_vf_token; - } - - if (vfio_pci_is_vga(pdev)) { - vga_client_register(pdev, vdev, NULL, vfio_pci_set_vga_decode); - vga_set_legacy_decoding(pdev, - vfio_pci_set_vga_decode(vdev, false)); - } + ret = vfio_pci_vf_init(vdev); + if (ret) + goto out_reflck; + ret = vfio_pci_vga_init(vdev); + if (ret) + goto out_vf; vfio_pci_probe_power_state(vdev); @@ -2016,8 +2062,8 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) return ret; -out_vf_token: - kfree(vdev->vf_token); +out_vf: + vfio_pci_vf_uninit(vdev); out_reflck: vfio_pci_reflck_put(vdev->reflck); out_del_group_dev: @@ -2039,33 +2085,19 @@ static void vfio_pci_remove(struct pci_dev *pdev) if (!vdev) return; - if (vdev->vf_token) { - WARN_ON(vdev->vf_token->users); - mutex_destroy(&vdev->vf_token->lock); - kfree(vdev->vf_token); - } - - if (vdev->nb.notifier_call) - bus_unregister_notifier(&pci_bus_type, &vdev->nb); - + vfio_pci_vf_uninit(vdev); vfio_pci_reflck_put(vdev->reflck); + vfio_pci_vga_uninit(vdev); vfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev); - kfree(vdev->region); - mutex_destroy(&vdev->ioeventfds_lock); if (!disable_idle_d3) vfio_pci_set_power_state(vdev, PCI_D0); + mutex_destroy(&vdev->ioeventfds_lock); + kfree(vdev->region); kfree(vdev->pm_save); kfree(vdev); - - if (vfio_pci_is_vga(pdev)) { - vga_client_register(pdev, NULL, NULL, NULL); - vga_set_legacy_decoding(pdev, - VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM | - VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM); - } } static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, From patchwork Tue Mar 23 16:15:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5329C433C1 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A50361990 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233183AbhCWQPi (ORCPT ); Tue, 23 Mar 2021 12:15:38 -0400 Received: from mail-bn7nam10on2089.outbound.protection.outlook.com ([40.107.92.89]:23521 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233160AbhCWQPT (ORCPT ); Tue, 23 Mar 2021 12:15:19 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z52PHsgJzG3x6+CfYWi40CRKRts017QGfl7/yN+3NX87lEUJ2CBFwylR3sBcGynuuM6MHyzU2/qkH51dJq5WVcqABBp3d2hN6rbsqJjdlkHOqgIMnkxxZ2pREmTPRlPq3/JwD1EC0QBiIEPT1bWPdBRRO/XouX/bP1fjJJ4cYn5S4V2IeBIswdh9OYbNcjLMWnMvqwToHCnvxukn2g1otB6QntFjEOYds3py7lgJo9NNNm9mDh92puhrq+seKOoCK7Sm3b+WuwY2nlFjWl6JJohR+bX92XkXjJhIDPc2qQhadwYzwM0lsAoCJVVBKzSaOpNaSJ6aKCw6DgunEO54/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z2BanRvYR4t26i+rEGhgJk449Df38w0eRIF5brcGOgw=; b=SkWuusn+T/w/9pJLDvvzn6H9eNjITpJ9fDmEGmu3MwRLKCP6KPQbrWgsiXcxoAK1G+QvWTD+oIKPgAqxe0a2EarRYxYD8O/Ny0unTy2UqdUCKyQB8QqnI9MMev8YCDptrO8Ntlj6QVfvDtI1wIzwXnITaOgQjvDLKwrfaSl3I70/ijaY1LRQLyZLR2KNiPz1IDx1qFryXZF4JutIXfwvmMQ9Q4Gyyh/ou3IVKIk53IwAkAVF3zUUejP0EfUUKMMcelCaoAi+GmWIM3ZOHF5pljHENsN9m7V8VL+lhIKnA21DfVk8HzOJG94tqzOjtC4LCQV35yj7BM8SQ2rfrgXo2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z2BanRvYR4t26i+rEGhgJk449Df38w0eRIF5brcGOgw=; b=bLrzS4J/SbSkoTkZcwNSdPBT7eU6TAoPg2S0/zdh8ssg1M1cjJ1e8xXWMmXgqX6wmDmrnaFPHFYFrKKY8KRP9NIGLVQaeExY9HBQyXAamOBoR4DzbjAhYsiQGuUq2FIcIRQRPbaPJGnhcdHofpCvlY+SpBmzHrm6iQpl+8lb52oHcdTtkCBtBT93i7TcojbulRK1y8mzS/o2T/ZTnztlc6Bjmq4beSx8SrlTmfznY/1H9spFOnZwr80PNK7SDyyAzbM1B60Cx8WupdHfM8d2GLrNZXxFGYrMzPV0N0z2z/0k2mytXLnVxdSPkN/WLDaKl8tqGi1LTl99nGBdLd2xYg== Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:14 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:14 +0000 From: Jason Gunthorpe To: kvm@vger.kernel.org Cc: Alex Williamson , "Raj, Ashok" , Christian Ehrhardt , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 08/14] vfio/pci: Re-order vfio_pci_probe() Date: Tue, 23 Mar 2021 13:15:00 -0300 Message-Id: <8-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0263.namprd13.prod.outlook.com (2603:10b6:208:2ba::28) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0263.namprd13.prod.outlook.com (2603:10b6:208:2ba::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10 via Frontend Transport; Tue, 23 Mar 2021 16:15:10 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCs-Lm; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e6e89361-8dc2-43f9-4c2d-08d8ee16d4e8 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IPMnzQstqp889iR2OJEMWugmOzfARrDC2ne565ivS6/Sf/+4OTjYCjM0GYM96Og0KE/dScFAH1Mdp0QRDThZhELLl82o77V8sE/tfAtgAr7BS4A7Hq87yr+m1xrCkMN4DjJrmHwdSXiFmvWf+iG2Djxz70kse4gLt26MgFz0h+BQAOQObABUxXeYdgNRZuCiYx2oVrNmG2Jw0jyGCeYvVYs9CpSL1PHMi6DF0M80M8HlPTCALQpwTUpw4NxTTBce0SZBzwHcct66TUIcXrZguUnBisI6QqCCPg+bhWQiBBTe6ir/AGJmRY8FtkM8clJ4MRxNlqeJZjjUvZVNdczh1pI2lAO3lkLf4gmdj5JjBZWk2/BrTvHK16f+i9OjAW0DYXc7o1hB7T8eBqHvuuXpcAvibiv2NgId97KOaDYmYIyMVNdV6GKGeQ522HRozbvXzG3PgTdYi6Kz0sL8VDdb8HcPijQyO9xj2HU9nTJ6K5jOXbhe1MAeU6eLTPd0qFIaVRXXhwwa/PUT3m2NkfNo9hdeRdF3RRP78u4QdL0KhJfWLOZkz1AmxMsrJou5WunyypsDunXY8thpVjgNG4elWAmWfQnjtXl9KUvanKO62sv7cusqd1j6nveM9YXrPYAH+G9ibO2VfOBrcRM+gwXZ6QvpUt/AZhcDrdaLSzUDGZO/W+cZ2d372UGMApaMGnNGFDQw9tqtOH+iCxldLE1nNg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(6916009)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(7416002)(6666004)(83380400001)(426003)(54906003)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001)(4216001)(169823001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 6mYNiA8AVrP7bHnHkbdrOj1XMOHXicBmd4klqs9TmBJJ9DETzI6x4JiIou5wfqKnhUjS5Qt2IbiRmBKKX4mRD3mIl31GluEGWwED2vXgk16tPucsWxOlp+jGG76GRtkNA7x9NO1iTqksbQNGjFH9mV0drsYWRjp+7JvLT1wJCA1whS4n+DgbnyedGSUQp1ZE6hXEWdH7CXLBnXbeDfKHZ3GgMSXGB+yLpmliD3QLBWs1zj83fV9pLoAYwf/qDBiRCNsJtrn6zR/MLKSkMFz11KgNGt+NzTU5SO/28bQ4uQA2ObacHTIZzpj/xO7Mz+O0JPeZwG0Khk5s+VBKx3FDPPqU4s48b2BHJWbivCJ23O+Q4Xz9gHLc3Kpukm9zWcbUAmKR+rDnyD9xBeAGars3Dkfj93vaLPMZE+l8Cd0zR1qoAGrdTT/P/aJNywkiZB8jbWc0q8nMxuVbXrHWgT97Qle0qJOL+1wG5MYa2R5QKwT/CJDmIdmNgFXlGF1kuraIYgSKAM2JLT8tSvvv2RF5r4qJHsBP1c2cUC8XSChRbT9hONfT1xwSiZKFxgTG+1MHZQORC3yWuxtpitLSADE9rJbJ66F3JRcqy+G9pGmgxXK0G0DPUL5Uo4Z1imdggoOOZZt74dxXawh58iB6bfxDYUvuWuffUyavsu8Fg1NMcv2xO8xfaqsr10uK4DUtKOOu99ioiAlIGBUyDh2z1Z2Ek7c4wkjNOHNkgiZJi8r9MJ/Htq4D2Im+WIqxEhPArb+oZR2q+l0i2NguYRXblN1UOS3wFPrF3p5/2+PGeOWSrbb/xWfoYyDe8nPxivr/VPyAgg30rN1SOQoiUCSkr9900LEWWAxW0k9jn3jrSvtQX518RObKuvv1lvSj5jzTnaIBBJsImQq3S9jOptel2xtptnFdwF2vJ7XL8OykrGBC3FVgr8WsetxllyNmAUl9Sstad73YH6rL6SKAJWwf32U6vYzXwAfP1of+WACa32+Kmus1HSQPCp0M1MNS6fUFFvmtmkrk66H+hPhdpr41rZt0B8nNiCNDouLYt+ZUTMZFnIoGKes/q0K76KwEI8VALFj76GmAVK/KcK4gWDe8RJuABcfAD5wIH07j4JGU9pMl06vRmZmCuHcPB1f90JrZ2p9LfllWpJuT+FURm2+wX9hTb6htxqVaIWwRQus6nuhVL+SJ1b5kTHzACUsYgUCkoQ1TVUyyoEFdof+pRUzsFA+63O0VeoxMKXTz0VMdi0RudVhlT3azI4cUJE84dBPLHhUIpmDDt/oJTL5tN1JsHrjLwt2Q54Bz0cyZiI5600onAgYjtuVmDJN0fyFNId30jQep X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e6e89361-8dc2-43f9-4c2d-08d8ee16d4e8 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:10.2889 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PbaOCZqCezSYxY904xzsPR/sH2X6FYs65tAXbE9Dqpl+PvAXTfoq+HzPKB/1BcQN X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio_add_group_dev() must be called only after all of the private data in vdev is fully setup and ready, otherwise there could be races with user space instantiating a device file descriptor and starting to call ops. For instance vfio_pci_reflck_attach() sets vdev->reflck and vfio_pci_open(), called by fops open, unconditionally derefs it, which will crash if things get out of order. Fixes: cc20d7999000 ("vfio/pci: Introduce VF token") Fixes: e309df5b0c9e ("vfio/pci: Parallelize device open and release") Fixes: 6eb7018705de ("vfio-pci: Move idle devices to D3hot power state") Fixes: ecaa1f6a0154 ("vfio-pci: Add VGA arbiter client") Reviewed-by: Christoph Hellwig Reviewed-by: Max Gurtovoy Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Reviewed-by: Eric Auger Signed-off-by: Jason Gunthorpe --- drivers/vfio/pci/vfio_pci.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index f95b58376156a0..0e7682e7a0b478 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -2030,13 +2030,9 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) INIT_LIST_HEAD(&vdev->vma_list); init_rwsem(&vdev->memory_lock); - ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); - if (ret) - goto out_free; - ret = vfio_pci_reflck_attach(vdev); if (ret) - goto out_del_group_dev; + goto out_free; ret = vfio_pci_vf_init(vdev); if (ret) goto out_reflck; @@ -2060,15 +2056,20 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) vfio_pci_set_power_state(vdev, PCI_D3hot); } - return ret; + ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); + if (ret) + goto out_power; + return 0; +out_power: + if (!disable_idle_d3) + vfio_pci_set_power_state(vdev, PCI_D0); out_vf: vfio_pci_vf_uninit(vdev); out_reflck: vfio_pci_reflck_put(vdev->reflck); -out_del_group_dev: - vfio_del_group_dev(&pdev->dev); out_free: + kfree(vdev->pm_save); kfree(vdev); out_group_put: vfio_iommu_group_put(group, &pdev->dev); From patchwork Tue Mar 23 16:15:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35A05C433E8 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F0535619BD for ; Tue, 23 Mar 2021 16:16:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233175AbhCWQPh (ORCPT ); Tue, 23 Mar 2021 12:15:37 -0400 Received: from mail-co1nam11on2068.outbound.protection.outlook.com ([40.107.220.68]:19553 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233146AbhCWQPR (ORCPT ); Tue, 23 Mar 2021 12:15:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JZMbxaLfeSJ+z4oTsSsBDOlDiTWCThC06IxQzydGrxeDU8Cv0N1r+5V8CKa3PVbpU2B7+4HHOe8xzKw3bV1jk3ml7CWfUNmofoXlCcZIhdFzxC18woEQgTMp9/MhSCup7WxocqxZSzSj12hgPjuGpFsLE5ZmuDdPz92/KH/jQePTXX6og2aG3aIFeuIYdt2fjbRLmsaaOpMkSIKOhtqzjfLXcuEoop2ttuZVBGQAZyj+I09qndoNS/Dc+UVtLl7uX7HDPfgJObPtfovcxLhdQ4vjhN/bML6r55dJHlG4E3VzV2DScRP25slizVyS86fA7mI5hgWTTRDl8iAjLOgOqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4i/pnK20oCPGCb7cw03L+LZNTRDUDKni4LwtedBrAdQ=; b=hCPWMN755MoVX6MDaHmeCPTSDlUAiqsWA6sZYBQQNk2Ax5QBs+FF9sj0d4s7zjndVnVBSjGLjzXlOFLhoytB5rwko6XHMlnfqdZMkysY3OKMgNqqqikf5Ohm50xBwbbE5TGzgbE1bYn70xhYCsMHtRPnR9yyfhiyfgSkAlGOxNbDgVzRONwXE5EFJ9DEC60ePEWglMT/HC4RONT2b2LffjEm11M9ybQm9c0b/ZFoAjEpe0dTPfkt2K/QanH7XmaLVTFY2+r1mDGrx+HrVlzIRdU5OPoIHk3ONOUUSXK7g6VaCaMhEpVQOAKKAMHd8OJNhfM1/49F+XeglfvJon1Buw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4i/pnK20oCPGCb7cw03L+LZNTRDUDKni4LwtedBrAdQ=; b=JlElBcJPoYVppV1n7RLrHYYAO3EXCvCK1t4KCsFVdVHsHlK+QFantj1ZoX7neok+VftKudXG+wsR8CSwUS2Em7MLcj7yAhaslv1QN6us+7dVrebPddM6LXZN8QR0OBlCffJQbmCWTAqxyv05CJQz0rY1501JYZOaaKRY+AyzbzG2kguzGl/owqTx5elSZiZCQbjgU9flmJC+vr/fnetktrCrlJL+5NwVAP6rleJYyHarfr367OYl93MzGY5R0D4U0EiSLLhoJ2Rw9N1FkQ3bm41ijAY4ceAe+0a0ldOF0AA0nyhdEjUOOlgqULejk83veqvPttwOo8wWQRnkoJ/mcQ== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:12 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:12 +0000 From: Jason Gunthorpe To: Alex Williamson , kvm@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Eric Auger , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta , Liu Yi L Subject: [PATCH v3 09/14] vfio/pci: Use vfio_init/register/unregister_group_dev Date: Tue, 23 Mar 2021 13:15:01 -0300 Message-Id: <9-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: MN2PR19CA0017.namprd19.prod.outlook.com (2603:10b6:208:178::30) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by MN2PR19CA0017.namprd19.prod.outlook.com (2603:10b6:208:178::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Tue, 23 Mar 2021 16:15:09 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aCv-NX; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7a89e928-08a0-44a5-e5a3-08d8ee16d492 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wX9Bk5pURbMpd02E9UCMX7Lxq77npajuxgM0bXgJz0dQGA2/WtBJTh7cWIKZZottTFyuUCoXQpANwYH6HA1DXC8fxY0xqefQeR6+UnLO6IyAfTXSQDYhmW+vIRW4pXE4AfS4Rvoy3dQ+22Njzye8KV617giwRSLTskWQXRuKfxPVhRDzVQrsvmTByJKpHObY1xFpYeuLilvCkzbxLx/24LUBKE6qbVZX1kGy332f4vDGPUeVfXcdyMDFxeHXZ5+tW10DP9Hrl7Jy4fbTLsBBrjg1q47p9SfDJ9HNwo2oYUsCwaaJDbvTJfTg8hN2v7fpIabDyy8ydjsJ9SQXFt/jnsV1CLtwg6IHKTisFYCPfv7PWh4HYfVn+CqM1XKZW5ojI0/ZbLomdwKFlLFxKAI30vxDin2RsadXtspvZ32J2ty09zJgiicGVYANcKwluh7vTQnStNPKqGfQLxwNEgbVwq0tW7VzvKLG6MPjG+yJalyYio3tBz6fRb5nO2Yqvmdvo1mVNpMlSNff3MHPCa5r9WHMAbY4qd9uwiq/snPWqe0xHeV0uRnwP4p2xI4quzGr/bNeqVo6o8AykZAo1IYsjbRuQZX5G0W8/DQ9aEJY6oZHa8fSr/Fu+3sjvD/xYe6UZbhfTIcF7j51697Ef4JSfw/u/30C9UJp2VNIVR+UryVKT3YQyxwIyVD9mFktmF5k X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(7416002)(6666004)(83380400001)(426003)(54906003)(316002)(8936002)(478600001)(8676002)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: VmvFKfZCr09fIVtLfM+hEseEwDO0A7J+SQMkAz2O9FUsv1hn41ZvpblfdW8mZlcxdTyzD4lnsjy/Gv8eklEK7vO9XtjwvyGdnzsTV4v9OuguflR31qQ1bsqwrOohP24bTd/ztPxFZdpyfOgrq+998Ajn6KTb+83nyN7GB1LQclNNnyGmADl05lxk96K2FdJO16DslO/upXebWkE5pMaY78k0pOVBWwKQnfrJ17tniZzhBEe1Nm4bxXarBQEGmE7W8ow2mUy3OD+KmqVfeQZ6n7puaGQDo1htwYL/n65nZ0wNNOY6pBjTM+IhlBmN6d3VM4w8ZbErDgINpMVkJaIq5idC1FldQ9gXN73AVPqVYD8krs6gI6wkpXxoM1265G/u1qcD3yzZGLXldH1cRLnJXCawGnmPwEdnfIVd4XhM56U+3Rg647CRAUt2nAd1jJRvTRFCZUrzKzkcn0j3RYuDx1+23cgMGO+LYv2KxahCMBSyzY9dnMCE0g3XSIq6qf8ZmzLNIzBIl4pAF/j0Mkwn2k7ZEVhT0d6otMO8FEJcainD+WVX6AspKDZ3HNZaxawJ1f5w6IFGRE4QPeE9KhA5Dh8yPhf/NcXLpsgxj7G/G5HF+suWaLE3uT2i0HtgJvw5Lu/e3iQpYwpwKKI3wSGiQJgUENC7dOw6jVrjNvNiS3ic+s/6TLrOLk9817MOmA4LUOioUy16c01j5DGAWOR6inAJlBMUnkdyKTErPXb33dH1hhShcAlRgpyzEtdmp5eI1b6o/OSVu7PMvd4W/9KBhOjd0vL7fLS1fjUYcYlORV+UKE0E2CQT4EmCbPjHOrzJEuScagO3WL6UJiYVRk54Rf6NeSiZ35l7Dy9IKrUPypYIcD4rsmQcg9AWhDqhTar4mMBlv0zMLnOwukZfYNLbTI1OnR54o/MpdCskdwKU6/S1DGsFaTaea59nTvQ8EBAHIniDiOthUnCeI8Ghf2kqSJ3QK7ywFsNSV1DZXvwS1vm64DadVLfaycq1HU5+ybHud5LW2Bbb1Yvd8m/WWtgzlriBdDaNjNMtcYELcWxtjK534Ye/+ohGk45MATuCt3bW/6VCT6Zli8pGw++CiH3syITNoyJQuHJDy6tYnc7XuHyQq7DlC3+Zi6P/awX4sTQOby0usgqZTHAyvqdsRx8BPdjzmz0b73Dp/lMwJW7ozNDzyZbD4Hwy8OhLScS+rAB3KvY8C+f0W/BQoY31ik9YgCVBjX+e7LY4BvrbsnnUOn+W3aGtnNVc5f7rAEl8YWhIOB+Wa4izQaOMOVUBD9/KkJpleg/6A94jYp1y2YzZHDkQ3Ubz/K+dcYtW7hWP+CeU X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7a89e928-08a0-44a5-e5a3-08d8ee16d492 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:09.7442 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: a6HirdCHsSI9eDBrKzXvVP07W5FC8haooRVYcgUbIxLXV4XHZ6m8/4rdTwu5hGkA X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org pci already allocates a struct vfio_pci_device with exactly the same lifetime as vfio_device, switch to the new API and embed vfio_device in vfio_pci_device. Reviewed-by: Christoph Hellwig Reviewed-by: Liu Yi L Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Reviewed-by: Eric Auger Signed-off-by: Jason Gunthorpe --- drivers/vfio/pci/vfio_pci.c | 10 +++++----- drivers/vfio/pci/vfio_pci_private.h | 1 + 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 0e7682e7a0b478..a0ac20a499cf6c 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -2019,6 +2019,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto out_group_put; } + vfio_init_group_dev(&vdev->vdev, &pdev->dev, &vfio_pci_ops, vdev); vdev->pdev = pdev; vdev->irq_type = VFIO_PCI_NUM_IRQS; mutex_init(&vdev->igate); @@ -2056,9 +2057,10 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) vfio_pci_set_power_state(vdev, PCI_D3hot); } - ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); + ret = vfio_register_group_dev(&vdev->vdev); if (ret) goto out_power; + dev_set_drvdata(&pdev->dev, vdev); return 0; out_power: @@ -2078,13 +2080,11 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) static void vfio_pci_remove(struct pci_dev *pdev) { - struct vfio_pci_device *vdev; + struct vfio_pci_device *vdev = dev_get_drvdata(&pdev->dev); pci_disable_sriov(pdev); - vdev = vfio_del_group_dev(&pdev->dev); - if (!vdev) - return; + vfio_unregister_group_dev(&vdev->vdev); vfio_pci_vf_uninit(vdev); vfio_pci_reflck_put(vdev->reflck); diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h index 9cd1882a05af69..8755a0febd054a 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -100,6 +100,7 @@ struct vfio_pci_mmap_vma { }; struct vfio_pci_device { + struct vfio_device vdev; struct pci_dev *pdev; void __iomem *barmap[PCI_STD_NUM_BARS]; bool bar_mmap_supported[PCI_STD_NUM_BARS]; From patchwork Tue Mar 23 16:15:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D998C433F1 for ; Tue, 23 Mar 2021 16:16:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EDB1D619B7 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233201AbhCWQPo (ORCPT ); Tue, 23 Mar 2021 12:15:44 -0400 Received: from mail-co1nam11on2068.outbound.protection.outlook.com ([40.107.220.68]:19553 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233163AbhCWQPT (ORCPT ); Tue, 23 Mar 2021 12:15:19 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XTtAnMXWAQ3cHX33qR/iEWo+nTI9wOnP2Mq7pYz65eQ+cgDtfcMoq2mw7XOKWj4CnKflzXNzD/PQnzozd8U8RXJ5r06lRkB1fwcTmie+QqFi4ZHmwpOvy2RDr6LOIW+rQK5JITMFGvihjYrTEZAEW0ODFEZmA8eOw/FR0Z+2WsgMa4+9JXr+nJiuUP9dqtGHSB0lUxArXAFE5vB9oD99WL1Ahf3EngkpLNQOie6mZmOHN8+mxOdWTCVsGVhcZ9NEJuLWozxMdWjME/Q4P6Bl6d8qQip3yAO2Tn3O0lTnOHnywt+mGfMPylAjJYHLADIsDpiKVCGzrWigKvyruq/puQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L6ysqxGZrhdOpZxb0cVTWjHkX9ItpBvek1siVut3oiA=; b=Au/KMbSRm5P0dDAm3BxM+gBcreoGjWNbyh0aUlYKaCh7g8Bx66s5wt9PUS7xunfOam1LcX1QlHFdJKVeIaJCVvAUSSPF7OsryZlnxIvLOvUokzbPBvx5jNo0NXaPmz8dn8O32aSVBjIxPRbDn4E7pcd4kOVdEEcwlztxwIz7YsPvut5H9aTbpg5q/zdu9UC0+5bcaEGmwx6nVJ2KajxkwsMj5T65pxCNIyZQuPXUw4wOEg6ZQbeWfmxMnuOX7owm8/27j5maHazZi9DnYUBIFi/A1L3Z+NYIKWaDggRt6ysFaCDFuXIk7HhbICiHXrxXMhXswJk/xbVxGiZowfrBOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L6ysqxGZrhdOpZxb0cVTWjHkX9ItpBvek1siVut3oiA=; b=UsioVh3c/f0rJTCUGpWWc32jQIt+MgxoEJp2XQ3WCf3+jiDmknGsIyA1GiOyhIkZMDNF9L35OnHxQ0PfkVVzaAq7i66n2V5/uBWnhP05CCi+v1oeaeL5xdfRQdTDsGbhHEn2HObU8h4gAErzveySiKCfmCGftDzbmceApEo45P2unmm+OHTbj3Xjlpr5hZX3Y8KDRssi9cObm5+DpgCKVaZSa8GHr2lfvTcAEhTvC4xrLHYSIsYK8EVUX3cuVCkF+XoDIxlJ+BYQ3bZgtV4O49r126SdBxvPBESxYI7vFM5UzgPO6FVuIOIshBGMGpH6dD+UoqQ/SsuAXFPZullBSw== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:14 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:14 +0000 From: Jason Gunthorpe To: Alex Williamson , kvm@vger.kernel.org, Kirti Wankhede Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta , Liu Yi L Subject: [PATCH v3 10/14] vfio/mdev: Use vfio_init/register/unregister_group_dev Date: Tue, 23 Mar 2021 13:15:02 -0300 Message-Id: <10-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0194.namprd13.prod.outlook.com (2603:10b6:208:2be::19) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0194.namprd13.prod.outlook.com (2603:10b6:208:2be::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10 via Frontend Transport; Tue, 23 Mar 2021 16:15:10 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aD0-Oz; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 940d608a-8416-48d1-8353-08d8ee16d538 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y8McTMbF0zF6/zjRieDrxKRw83jKc8XmqWYvawgpwmghs+AyBqs9iYX6Qirl4V/l/jlNyuoPu51/3btM+bvv2UF02o7UIlB37sPT0OSf1RnTJfSHzCdnfpIgCpBLkyq6UDyUcTICyvjEnRJSDF8vx47edg09hdWRuD+or4TV3j2Rpj+501hdbDP2UNN8ruvosvVMC+GWURxxpLQonFZqraNgdxwyAuGW/pZg4Fiw3xe3ox/yFecIRoxQD9uKBEihhTh/L4Q6GxKHy6LDW/tElQ2v6m3O/Ag3XWywwqT7sNfH5/goBHczyTgmSKs5bu+WCGHuy68uobzJofdxzCcTKNrGDAgBRW91XLSmu8vwm7a+X8O+ILXUL24LdoNCwsbJk0EqPko1ChnLPEwZH2A3BMy+VXZIFnxWV3WeZhWb6dUMJtqf6RSjDTukqeMRbOOWo9u4H5upbJxuVhwOYxQtbS5NtInPZOyPRw4AAcHxzGYECX7X1NiDiYa5+jaNs/2jVZiTyAfyAshS8IX/YzCq+wjhDk5tl2aBHIt5YVgdqzFrmRfPmEND5mWkAl/0wW0zDwW/Pam+OGupiONxexZbiGfPpikyjOdzgWOyCc/Qbbx97MwfxGRMRJzCugm+ppHomB/wExTGEsrwJIV9/mirwiYEI9BpDTHU9Z3gYWrixus= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(6636002)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(6666004)(83380400001)(426003)(54906003)(110136005)(316002)(8936002)(478600001)(8676002)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 2enOmlo9N6h0mTAAk20SQ7OKtTGnry0g+uNTxyfCIecHIB/hl8pReYbzmcVNfOwHaiW+PYFhMvO5LsajX/QVrSukelmzi0y/dJOyHNtuDsq5c4VUCBa1lMh+Y90eOzRIdFv5RP+vnBGKt7tR/nDwMQO9MP+zoZPqcL4KdFn13pMInyMQNLoiDrD6Jb+C2Cwdy+ynzr8uO7qKe14KqvCRvhctXKHuHK23K1LKT8Mh82oM4zblkiJ0qjov8v5xw9Jnvt1YScWlZkqTESa1YlpGRCU376EuadInI3n4gOIvR+Kx9zxGBQZ0E9/c/uLA5hdDdK9osYrb5eNMH63aySb7JjRS3nSwkKxAW+UxUE3sgW9QRRQlZDBSKV0YR4y5yW4hIHGkxEeeUnoV4PF0HlGxuE3Ah7473xL3Wq5VDLP+bKTPXBEF0XsR5h00LwMoDmJEtvPeBD+O/Yx5l+R4Q4yEZDvAEnYJrOeeUjW01tBnTFXGHp+u4m7dPLUA+B8cIbIJpk35Z7hdMoyM0tHZmShgBmrICP0tby1xtMzDsixtIjn5pGdbD2YIpvrQvJ7Fhygj1obf/OGkWdBszDvmXESoiTLp4H6IioSY5b8i8oxqZNpWsPV8QvA/ZUmHS0tVpcGzL36jvnyYBAR/XlxgoM3XeohO+ykv0zqFW3//budQhLQHTi4xEFUSTClgpGsYasreE8d7mHZKFEEC5qg3TfSP8Q+4ns9J3M91X1tSgjWrPjNoaZ+N0InWQpDdjJGIS5c9TDrKuEimP/0Dnt56x6nJpzbWbFGjtvBZs4CztDvd1aflPp8eEo7IFQKwJTPXFVfog9XV7TgMsSguTn4fY7AbwYHywx3sYGfZxQ1Mn7qSBPt5cR+ptTxYLB/r0TMz/drCmdgFUTAL1zcq/NYEPSzrUUoGK9XVcj3jn6pT7fQES4JWduUK0UGCn0QpYAbllUuMUkZqnJWeAn8fmc/o6ZrrulW/oU8y3Aoi7hOLLo/Kxrsqh7wkGkPyr02yDX/xsIkY1FS6NhjPmy105xWVITNWiM83QxVhEV20vVQyYKHgc85XXCjwaRVDupZBK+aRGkIHi1UjdAN/vt9PvQ+CHLN++zqVSZPpfdr/FL74QcSS1+kEt+UDNpDkvTHYdfMtuu7ZijkZRzt13BryR0K8eZx1OrP7tD4orL+XFFZ9QjSOjYBSVws2Q8LEo/1OhG6MwLGfXEGTXyaZVPJnc/Pz1TNLm83DjHblZAeLtSACqlcmloJ5/oqp0FmNkqkRWRuM6iiljv53hC/x8xbtdTxglJfC9O/i0VjYww0nGoaCYV3+5M+WbPUIpNgwDXkvgomjssg2 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 940d608a-8416-48d1-8353-08d8ee16d538 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:10.9095 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JLip+pvxl1Hvm90w3k4hBCeYUYY61AA7L3QZs4wsQ2IlM84w/ZS0p7N9SOIxaum2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org mdev gets little benefit because it doesn't actually do anything, however it is the last user, so move the vfio_init/register/unregister_group_dev() code here for now. Reviewed-by: Christoph Hellwig Reviewed-by: Liu Yi L Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Signed-off-by: Jason Gunthorpe --- drivers/vfio/mdev/vfio_mdev.c | 20 ++++++++++++++++-- drivers/vfio/vfio.c | 39 ++--------------------------------- include/linux/vfio.h | 5 ----- 3 files changed, 20 insertions(+), 44 deletions(-) diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c index b52eea128549ee..4043cc91f9524e 100644 --- a/drivers/vfio/mdev/vfio_mdev.c +++ b/drivers/vfio/mdev/vfio_mdev.c @@ -124,13 +124,29 @@ static const struct vfio_device_ops vfio_mdev_dev_ops = { static int vfio_mdev_probe(struct device *dev) { struct mdev_device *mdev = to_mdev_device(dev); + struct vfio_device *vdev; + int ret; - return vfio_add_group_dev(dev, &vfio_mdev_dev_ops, mdev); + vdev = kzalloc(sizeof(*vdev), GFP_KERNEL); + if (!vdev) + return -ENOMEM; + + vfio_init_group_dev(vdev, &mdev->dev, &vfio_mdev_dev_ops, mdev); + ret = vfio_register_group_dev(vdev); + if (ret) { + kfree(vdev); + return ret; + } + dev_set_drvdata(&mdev->dev, vdev); + return 0; } static void vfio_mdev_remove(struct device *dev) { - vfio_del_group_dev(dev); + struct vfio_device *vdev = dev_get_drvdata(dev); + + vfio_unregister_group_dev(vdev); + kfree(vdev); } static struct mdev_driver vfio_mdev_driver = { diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 2ea430de505b3b..180b4ab02d115a 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -99,8 +99,8 @@ MODULE_PARM_DESC(enable_unsafe_noiommu_mode, "Enable UNSAFE, no-IOMMU mode. Thi /* * vfio_iommu_group_{get,put} are only intended for VFIO bus driver probe * and remove functions, any use cases other than acquiring the first - * reference for the purpose of calling vfio_add_group_dev() or removing - * that symmetric reference after vfio_del_group_dev() should use the raw + * reference for the purpose of calling vfio_register_group_dev() or removing + * that symmetric reference after vfio_unregister_group_dev() should use the raw * iommu_group_{get,put} functions. In particular, vfio_iommu_group_put() * removes the device from the dummy group and cannot be nested. */ @@ -799,29 +799,6 @@ int vfio_register_group_dev(struct vfio_device *device) } EXPORT_SYMBOL_GPL(vfio_register_group_dev); -int vfio_add_group_dev(struct device *dev, const struct vfio_device_ops *ops, - void *device_data) -{ - struct vfio_device *device; - int ret; - - device = kzalloc(sizeof(*device), GFP_KERNEL); - if (!device) - return -ENOMEM; - - vfio_init_group_dev(device, dev, ops, device_data); - ret = vfio_register_group_dev(device); - if (ret) - goto err_kfree; - dev_set_drvdata(dev, device); - return 0; - -err_kfree: - kfree(device); - return ret; -} -EXPORT_SYMBOL_GPL(vfio_add_group_dev); - /** * Get a reference to the vfio_device for a device. Even if the * caller thinks they own the device, they could be racing with a @@ -962,18 +939,6 @@ void vfio_unregister_group_dev(struct vfio_device *device) } EXPORT_SYMBOL_GPL(vfio_unregister_group_dev); -void *vfio_del_group_dev(struct device *dev) -{ - struct vfio_device *device = dev_get_drvdata(dev); - void *device_data = device->device_data; - - vfio_unregister_group_dev(device); - dev_set_drvdata(dev, NULL); - kfree(device); - return device_data; -} -EXPORT_SYMBOL_GPL(vfio_del_group_dev); - /** * VFIO base fd, /dev/vfio/vfio */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index ad8b579d67d34a..4995faf51efeae 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -63,11 +63,6 @@ extern void vfio_iommu_group_put(struct iommu_group *group, struct device *dev); void vfio_init_group_dev(struct vfio_device *device, struct device *dev, const struct vfio_device_ops *ops, void *device_data); int vfio_register_group_dev(struct vfio_device *device); -extern int vfio_add_group_dev(struct device *dev, - const struct vfio_device_ops *ops, - void *device_data); - -extern void *vfio_del_group_dev(struct device *dev); void vfio_unregister_group_dev(struct vfio_device *device); extern struct vfio_device *vfio_device_get_from_dev(struct device *dev); extern void vfio_device_put(struct vfio_device *device); From patchwork Tue Mar 23 16:15:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1ADC0C433E5 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CF25E619BA for ; Tue, 23 Mar 2021 16:16:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233135AbhCWQPg (ORCPT ); Tue, 23 Mar 2021 12:15:36 -0400 Received: from mail-mw2nam10on2048.outbound.protection.outlook.com ([40.107.94.48]:9569 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233150AbhCWQPS (ORCPT ); Tue, 23 Mar 2021 12:15:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SG9/x9hYoOJ0/xNupuWdfabBoefOG0pEMzNBrCsJaU22KflYkhbz80QDKdKdD1J+ozTtK/X5q4ghrSDPGkyikmkIbSKcJ+39EhDKvdPOK69pklq3vdRm2toTpm9laHF/mwD3v2vpmczhFLJGaYdz2yHrP6gMZmjqrCZoGAv81BUImfGLHMpdUXLze9zN0UUbj9mdK6cq+m3KBj9mX66G8ixvt0L4GQOiqds4fVWCjf869b9s8u7fEqhoadLpOjJr3PnTdd5q2U3TfaSeC6wUu2M/id7Nr681dvEQt++1Jawym+63CJVwsvjbSjRF0TdqtpPbBIBlCwvtRIRimbN8jQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g9rRaFUnfrzInfeeGjSlCXiYHZN1FE0H55cc18pQRL0=; b=bvA2BKPs46RrPt37oLTG/3hDnekrxKRvmLJdOtTmxJXCUm4BLQIQ0VQM3K9K4XFcPdsZvcZsrTu4dsCcqy/fcUYcvkxEQTYOhPeP7CBO0vFZZVmoQ5UAYJiE+bR1DdSwbCUBMKoVJIvD5HWvnQmJPeAsFRXoP8prA6URPCRa62C58uU6hUhLYUgu7LVCyBrfYLHgvQ4+ePdEd2vA8DrCQXLaDU5HKHKfVAiDVLVJcqvJYIlsLeiG+rvyGEJGEZV1b82Va4ihlHk5Y60N3ZGMLsyHnQb1GNYdviFzzf4ME8wy87LiRWSg34s/8hQ8YI/nfx3uVGzQ70XtQEAiUzbi0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g9rRaFUnfrzInfeeGjSlCXiYHZN1FE0H55cc18pQRL0=; b=AECHw4kUDLRxsR/EWTHN/sbAc2ejYPg+GZhujHAjfL/9HZWUshjLx/17HRcF5e0GTgLzg3zH2Se0ZgViLjmac61mbq92wUE97YLgmDpigwThx1dVhfbPdq1r2hWXaJztIAqORT4pdDsLubLmXu80prlS9iE9eNjvItTGCekK5136NAaUdeNcb339O/l+6K1W0JeK9AhVjFKpA9NZfI1R7Y6ftNboHKfdjDshjo91t4/ZpxJD1/SGs/mCHU8Wzjw9pw1KcHwW9KdwahjDgJdejVGP5cjBxYBdR5SrgDAlbmYQTGkR/UpEZCBMPHG3KRaJu9gFGrGAo/GUlXjJF15jgw== Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM5PR12MB2440.namprd12.prod.outlook.com (2603:10b6:4:b6::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Tue, 23 Mar 2021 16:15:09 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:09 +0000 From: Jason Gunthorpe To: kvm@vger.kernel.org, Kirti Wankhede Cc: Alex Williamson , "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Christoph Hellwig , Jike Song , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta , Zhenyu Wang , Zhi Wang Subject: [PATCH v3 11/14] vfio/mdev: Make to_mdev_device() into a static inline Date: Tue, 23 Mar 2021 13:15:03 -0300 Message-Id: <11-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0188.namprd13.prod.outlook.com (2603:10b6:208:2be::13) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0188.namprd13.prod.outlook.com (2603:10b6:208:2be::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.10 via Frontend Transport; Tue, 23 Mar 2021 16:15:08 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aD4-Q6; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d8dc0427-e4d3-4465-7c7a-08d8ee16d3e7 X-MS-TrafficTypeDiagnostic: DM5PR12MB2440: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4125; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: amvSfpd+PPCy8fu9bBWLyM/offMPY59bk3EUFHOcdouBoFHhhLN4b5PgPSHKS91NOMHWUi/CHXIjpCXc4fjKyw7gDp1syUtXNG557F6K4wQM04u9KLAyn9cQtJaV5gbWS+bfqv48zyIGk1k0XTX9JdEAiSFomyg7gzjy/2ascZiSrgBcvDW04/ohiI25YPqYesAGpXhNMGf2vaMUPRqCV/vPCvifKQ2dzh6CxyUZxrXQjkpAWvz066J7oer8I7csez+tDlezpzfQX3cc95D28yIEV5dA1fNNzFl7THP1x6l+TPm5hVzG6g0po50ZYrpFUhOIkSbMWEg2+wK9jeHggwfuodOf6BgM8amWqqUSEAYP6n2GnxD5VXrrVCnXRF0EC6OhWIbuD02glgCStBAWf7Voe1L+EdypLxHqaopNL6xro2Pl3qHCzfWSKuNq1IbobGhSa3qtxbVp9R+vWGO3bZuSeyL5ec6biimTFfoT2F4t1WG1jVdndGZgU+VEYCZlhiUs3b10HxO4Y99sjjZJkPmy8QRqlVrw3HrnD5b5EGFprSW/ysgJk7HYnS0kLvzk/jyVTUf7p5L0aZb/v5VQa3Dv4XxQl0Xyv4YHb6zyYcW848QMC1pZmZK6lS68iJ9ZmtOEAIL6j9kEQ+9YJJXZLvPv+tsqx9G9oXapjA2f3uk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(396003)(376002)(39860400002)(136003)(66476007)(66556008)(186003)(37006003)(36756003)(6636002)(6666004)(86362001)(38100700001)(66946007)(4326008)(7416002)(8936002)(54906003)(9746002)(26005)(478600001)(426003)(8676002)(9786002)(83380400001)(2906002)(316002)(6862004)(5660300002)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: DBhsbxzpLc41nvUu0zQcVQbGdWDZR4jHEg3+OXxdLqA+n6qHYFAHlf1hNmfbJrFm0J5dZtiY30Uzk/4J50Ljw5QJYeUFLvD7SbkrFVEDOa+qzKWbH08yDfXX8P1PbbLlme8mhHb2jhJjaSwV71yS41vNlazOafjQCGJIXyhCLF6ux2QZTaSHW/xMqBcEJVAb9/GafChFyrdn5+AoJK2FcIpW+8kDSVb7Z8Gb+2+EjC/qV8zxcBgfUgAtwxYOLonobMAjIU+/0D3ZOtP+1tuNEtKFqxauEev9Y/D+gsrpXwxBi1hAsQOvMy5yMAMyUtLOcRiUW/uC0/qvXOLD995FAs9TOwQ6H/4s2/OUWIQPIYcNlVqLLPEgvdl/krTakBAb31gtXP8LNgSin2VJe3cbRW6KRmy8ue2csPQSg5avR8TA2BCA3fyduuz2SvwwNNsbcVgz+ERG92+6GIFdrQkpplXjh+f5p45cPxfjOPitE50iLWLNuEX5BHnKPoml0ycwhQ4rd4NdnZCtAnabhKrbyMtqAhz4AXeOBWCLt5HX7nGOfamx+U0kh86KNox9DNaHO1wTsly033pOBlJ3YTadPCYTrT7eAJkFdk+DtJ0p1diLT6aK+oMi/fqA7Gd94dhi5r7xkOAH3pmo3T75hy4M3OmsvvWxrIrHzSsrRCi5GZczikgmna+uFvMdSH3U/qiaG7/JzNEUHe/i5FTXEu+YisdmNR6t0vugzJwlYpttE118o0EoFW+Yy5e1jx3pZTD5D9IBTbIT+6lAo97OXvHFz0f1mx3SpkqdKVnIBfZHpTkFIgOcDGu1/CP+Zsq02p57lhd7fKTW0LMXRq3W64Q8cEyjw8ulgft3jwPINn8RYq58PM4QeIskwBWFCC2qu5reiC8XrTv0LXg5oQlYoN8nnflBzFcOMxmQHWqODyZT5jh9AKptQbb3dDzZ9Csrgor5rCGYaYq0ssTV33OgESYq9SnTh4amsVbg34R8zm1sw5WPjzini3zvF093wiH40FmVuGcVif0CB3LFkT/PO8gvquIBeAUuSKc1+AiO/gqWQ6ASUCZLwqdjrjY7+yYMSPoIHnB94lYwKy5RWSvBcFzYTpG1LD+ZYaN8hUHHk8aVzPRGRgEMBclEB0QU6gnsBAn61nLXwVWjHxcJ8Nf1x+0PmHZI9KUGb7lqtULzPBHV5EMhTkR7kabilFfFeXVogylsyckVjd5dbAcJ7AruK14B3ggTCpKd24MlzHKRHJ63PS5RItmr/IirlMMieGnZyErPGALvp23gk5lFRQ0AVVOQfSfTqTo9+IwUBkp6lC/o/dGKoXRO3wYE4aWSHHy6faFf X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d8dc0427-e4d3-4465-7c7a-08d8ee16d3e7 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:08.7218 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lFA1VsnPBhIWtE3mGEhxBYyyo3SCW0Ml+H/5n1s4Xx/vkMYAmCLMjeunFedQWVdt X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2440 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The macro wrongly uses 'dev' as both the macro argument and the member name, which means it fails compilation if any caller uses a word other than 'dev' as the single argument. Fix this defect by making it into proper static inline, which is more clear and typesafe anyhow. Fixes: 99e3123e3d72 ("vfio-mdev: Make mdev_device private and abstract interfaces") Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Signed-off-by: Jason Gunthorpe --- drivers/vfio/mdev/mdev_private.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h index 7d922950caaf3c..74c2e541146999 100644 --- a/drivers/vfio/mdev/mdev_private.h +++ b/drivers/vfio/mdev/mdev_private.h @@ -35,7 +35,10 @@ struct mdev_device { bool active; }; -#define to_mdev_device(dev) container_of(dev, struct mdev_device, dev) +static inline struct mdev_device *to_mdev_device(struct device *dev) +{ + return container_of(dev, struct mdev_device, dev); +} #define dev_is_mdev(d) ((d)->bus == &mdev_bus_type) struct mdev_type { From patchwork Tue Mar 23 16:15:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56C2CC433E6 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1CBAE619C0 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233173AbhCWQPg (ORCPT ); Tue, 23 Mar 2021 12:15:36 -0400 Received: from mail-bn7nam10on2089.outbound.protection.outlook.com ([40.107.92.89]:23521 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233141AbhCWQPR (ORCPT ); Tue, 23 Mar 2021 12:15:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fMgMbbwZyMOxYXdjPR+nTQiCqmxG3UOHpWjra6eSYBFELMZ/xOaaPqOUuRYOuIZZthbLQI+ZJKdIrRzHSrA51NatRc2P3aIamaoOrrcB50T2acoaiZQgLt3co3Gvqhlua2LVxf0hkr8ei9Ve/8eQkgPLDRccJlVjg+xPvS/Dl8TAOBe9YnG0JagORBhbwiI4hMTL5fGxpRd/dJzAXe+zti0/QXDLPYBSztYrkT+5j5G1QYg5VJ/YXom+Sf5AtdbJaKGtCG/QLcF/VCPxoBwiJhcr2w4E/okvgn+xAX18Eg2M0Mwd2t5RSlLo0jAK6sND+FEQej92Zw9W6ZKg6+kJbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tt2LG1l8OPQLTRf/lg3gHpfhqmd0UWli+Sw2vTVs5dE=; b=cTKzoLyUHDPoUMtjFdnUi4FeopyW+N3mG5AXREXAECC0TUgrijFsjUOXeQfLBu8xNL7o2xyeolPLai7rdw1hctxbdsw5/k/kLQghEZBrCB85efeDZAqun+QOCzSLGo9B69iClK23Tp8t1v8+kNfnc2zIW8D7n6vtK8QdobkZUmkmbVnwJWhuUWyJ5zR84lrkx07hvJs72IAl3flDD0dtErVXeQJOFNTl0u/a2AeFcv2ubl0eT8FcWmhyTvVdtoIIFkuPHr97ytuttxerl+MXX0dOQ+pxuKPezMsSUx9bgwxaIHOw8vLwq9m4cswo8jimlBUAVA2vlbo0wzl5Z1FlWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tt2LG1l8OPQLTRf/lg3gHpfhqmd0UWli+Sw2vTVs5dE=; b=Xdyq4i3IjnmZFdrbXqj237pTYhoLztrz42aExEWGg2HlsalRbyWLTfFy1jbTwVdMjDrVpuuzeLmSwXViOmnyk0fRTtwKchtpJTKnJO+k7d4Bq3+seIhIoe+2ydJpMbflmOa43Qbpc4R10JfFd9Rws6jLHmttutDG0cWbDiO5UZdwID8F3b2wewpXjYmoaQKg49MopY5x4LVECPnO9nOy5TJqfyCFCMusczTnm7z43gfKslA87sXoEO7oNTj5hqgyd2rDbHrqHBSPFW/QnepKEFdhaN6LWiUqJCC7hC01Rlv4W2ZKYRae+Ihtl9WsB6bKXqq5qe4DG3cswvjqjLqj+A== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:10 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:10 +0000 From: Jason Gunthorpe To: Alex Williamson , Jonathan Corbet , Diana Craciun , Eric Auger , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Christoph Hellwig , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 12/14] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *' Date: Tue, 23 Mar 2021 13:15:04 -0300 Message-Id: <12-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0162.namprd13.prod.outlook.com (2603:10b6:208:2bd::17) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0162.namprd13.prod.outlook.com (2603:10b6:208:2bd::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.9 via Frontend Transport; Tue, 23 Mar 2021 16:15:08 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aD8-S2; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 65d06126-a500-402d-3bd6-08d8ee16d3e1 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XgtK52l8c8g6qHUYjS5+ikwfG7fB6NlgzZgUVwQfXNzXaqfFzisnU+NeWluDx3TNiz5fl3+Ovynvuxn1d1Fia0bgDJD6aYJLQECsADB0PelZ/XrgvNMWNC7H7BA0TLkmECBZhs2TngUN2x+gHmILG7Zm5hWIDuH6/D2bADLATFoARQoS0yK8+1PdstbrHS+GvP+LuAJdC5TwM1xOBp+e4lfvxvTL5voCPfP93wR8nM8BqFi5YmwAGzgB97cYwo9+iJg5fMqFLrqF7w7Q1xi7piO0f7yKH8SP8fglSsDiuFq6xwxnIydV5t6RCf+3pzguun0qWaRT9I2InhdJmrKUhdlAuhuUuBCOyLDI9rIizCTamjGwVOgSqSdNYxNCU7kLb3Qyqm1QU+AvG0OevdeOHLfl0DlmqrHcMC7D/+EI2VNyt4q/zIDLvRBCVcFLSzU6oMHYIDLD/AKzcesM/SnRLp7IFyq65QzMt3tcUBCBe1amifNCuGc2jcQLvAcwltpraMrLzO8UD8aNjPmOwEHnzIqo0w/JuE0z5n3oVYT7UREE6DZaBlzxoMOOi2MjziSkHOedueNcdtTjU1BIX3KO8NRQ4c90YwfaELeLaCkCAMjHajcew5ia2gfDccOvMB6RkNFGVaJuREIghjhNhF0wr6P4Pr+l63o0ABek8AzpcJA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(30864003)(2616005)(66946007)(66556008)(2906002)(186003)(7416002)(83380400001)(426003)(54906003)(110136005)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: igIPDc3bKg98ng0x4NpCSUrMvKLRn/yUjPcW+Wg1ZtzNlR3ZurVvv/9yrvxWdLpVxMMga33Te4Hck3unDk3kfbECxWQ8nTW6JAbq35zf89/5Sn7N3kbhQT5RUmMmzoQCmLHfDAXGru/fgTdJZPdapgTt/MG+LB3cRzg41ZKK58hZn6I5YDlmb4/hUFcdjC94/d2/m1XTDhd0v1XTKB5ADcO36itZzVAsAG0fpldyo3pFxIoiRcKtnZlz7pctOPojvH1hQkMKhO2dqANsTcLyiICQeWxWdXHCJKbbtbQOOF/KZ0rIKwoyJKszARHNFodr1jOemy9gm0077m8sv/zJfdvMWncE0fkAYBWvbpYPKUrvxMCH1oWLq14vSTr0ghCyv4ByjFSsrK2F8k+FMFgA128CczLfC1N/wlUZZabR4bp9H01ILCvikLZB7au1xQiDveJvhnpBbxBzdME3q9MzF0beds4WG42OusnEzwafHsHonxQ2tC1CNcUqXRDuCKtalfGlGcvbylsOLEnShdjnyF5vBZdNE7sbmgmkXqXIFbV7UDFZjoP6UFAf3WD+03GLOt3HVZ3WLT5n+lDaChNlVG0MaUXtb9ukQFVVmzKzq3I1DJcq7L6BFJxlE/D+7Ad4GUylF9NHRXpKVzFF/73K+CpULCc4m07y2tF/W6dgScut3iKV/sKYo6m/ykcyE1wqGo7iukVJsQxyxZyArncnvWLmkba9dJb9K3s56YOKMKt1Tbf95UnDBPSqPnKyzXJrjTP+RMiKM5UzLqBj4SiVpsCi6l0+R/wTLw7zuYULkJt+gS3z0pzXe/CySPD09Lzv4OZCD9teGl8o6IYJziycRW0aIbNaowkYE5oqYHGsNZQHRLh/Hiv+/Fut0wuB0jWYd6tmegX1I/3Chb4nPizlmb/rdN5rhb1cx70zoOG6Uy32zo74PNUeypcEU+AYtoCS+hjjsRDGsMu+cHmCuyhIab2afp55Tt6/Ild3Jq7AnYq1gJrXFiXt93UpQyTLFJ3VlqbgGXDFbGy/dhpHz0X4L4NOzL6xF6hcEgl+pwlrM2CR0qOK6FDaEsgoeEj5DLCcGBkwCEmIQCPAKXOSekLbTJOqSKIFweNDrrPPZb98p9GAeOqN+Mml7z/i2AyJECgXgxYWG5q18h0cOLBtCeCMZW+c/Y3EegIpSU3rkTbjl7ECSzeH8PlyHbuAJMY7RDTPtP3pKbN+1wyKIzOs6SGctehaWSRuAcMxtoUctnYJSUb4lMiM+MHmXRvu18W5S/Tfo+fw5rLINs83TbxLG3O28aZyDosLfM7n57yNtL+gIU4VGXHmyG/vaaMpABp/c9e1 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65d06126-a500-402d-3bd6-08d8ee16d3e1 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:08.7308 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4iz6bnto2/xd70YE63StOOJ6sE2d9/g0tZdstdV/jRTIFUN2pIOjJ15wr8A5QI9B X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This is the standard kernel pattern, the ops associated with a struct get the struct pointer in for typesafety. The expected design is to use container_of to cleanly go from the subsystem level type to the driver level type without having any type erasure in a void *. Reviewed-by: Dan Williams Reviewed-by: Christoph Hellwig Reviewed-by: Cornelia Huck Signed-off-by: Jason Gunthorpe --- Documentation/driver-api/vfio.rst | 18 ++++---- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 36 +++++++++------ drivers/vfio/mdev/vfio_mdev.c | 33 +++++++------- drivers/vfio/pci/vfio_pci.c | 47 ++++++++++++-------- drivers/vfio/platform/vfio_platform_common.c | 33 ++++++++------ drivers/vfio/vfio.c | 20 ++++----- include/linux/vfio.h | 16 +++---- 7 files changed, 117 insertions(+), 86 deletions(-) diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst index d3a02300913a7f..3337f337293a32 100644 --- a/Documentation/driver-api/vfio.rst +++ b/Documentation/driver-api/vfio.rst @@ -269,20 +269,22 @@ ready before calling it. The driver provides an ops structure for callbacks similar to a file operations structure:: struct vfio_device_ops { - int (*open)(void *device_data); - void (*release)(void *device_data); - ssize_t (*read)(void *device_data, char __user *buf, + int (*open)(struct vfio_device *vdev); + void (*release)(struct vfio_device *vdev); + ssize_t (*read)(struct vfio_device *vdev, char __user *buf, size_t count, loff_t *ppos); - ssize_t (*write)(void *device_data, const char __user *buf, + ssize_t (*write)(struct vfio_device *vdev, + const char __user *buf, size_t size, loff_t *ppos); - long (*ioctl)(void *device_data, unsigned int cmd, + long (*ioctl)(struct vfio_device *vdev, unsigned int cmd, unsigned long arg); - int (*mmap)(void *device_data, struct vm_area_struct *vma); + int (*mmap)(struct vfio_device *vdev, + struct vm_area_struct *vma); }; -Each function is passed the device_data that was originally registered +Each function is passed the vdev that was originally registered in the vfio_register_group_dev() call above. This allows the bus driver -an easy place to store its opaque, private data. The open/release +to obtain its private data using container_of(). The open/release callbacks are issued when a new file descriptor is created for a device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides a direct pass through for VFIO_DEVICE_* ioctls. The read/write/mmap diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index ad6c6feeeb4b2a..45f397c04a8959 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -135,9 +135,10 @@ static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev) kfree(vdev->regions); } -static int vfio_fsl_mc_open(void *device_data) +static int vfio_fsl_mc_open(struct vfio_device *core_vdev) { - struct vfio_fsl_mc_device *vdev = device_data; + struct vfio_fsl_mc_device *vdev = + container_of(core_vdev, struct vfio_fsl_mc_device, vdev); int ret; if (!try_module_get(THIS_MODULE)) @@ -161,9 +162,10 @@ static int vfio_fsl_mc_open(void *device_data) return ret; } -static void vfio_fsl_mc_release(void *device_data) +static void vfio_fsl_mc_release(struct vfio_device *core_vdev) { - struct vfio_fsl_mc_device *vdev = device_data; + struct vfio_fsl_mc_device *vdev = + container_of(core_vdev, struct vfio_fsl_mc_device, vdev); int ret; mutex_lock(&vdev->reflck->lock); @@ -197,11 +199,12 @@ static void vfio_fsl_mc_release(void *device_data) module_put(THIS_MODULE); } -static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, - unsigned long arg) +static long vfio_fsl_mc_ioctl(struct vfio_device *core_vdev, + unsigned int cmd, unsigned long arg) { unsigned long minsz; - struct vfio_fsl_mc_device *vdev = device_data; + struct vfio_fsl_mc_device *vdev = + container_of(core_vdev, struct vfio_fsl_mc_device, vdev); struct fsl_mc_device *mc_dev = vdev->mc_dev; switch (cmd) { @@ -327,10 +330,11 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd, } } -static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf, +static ssize_t vfio_fsl_mc_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos) { - struct vfio_fsl_mc_device *vdev = device_data; + struct vfio_fsl_mc_device *vdev = + container_of(core_vdev, struct vfio_fsl_mc_device, vdev); unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos); loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK; struct fsl_mc_device *mc_dev = vdev->mc_dev; @@ -404,10 +408,12 @@ static int vfio_fsl_mc_send_command(void __iomem *ioaddr, uint64_t *cmd_data) return 0; } -static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf, - size_t count, loff_t *ppos) +static ssize_t vfio_fsl_mc_write(struct vfio_device *core_vdev, + const char __user *buf, size_t count, + loff_t *ppos) { - struct vfio_fsl_mc_device *vdev = device_data; + struct vfio_fsl_mc_device *vdev = + container_of(core_vdev, struct vfio_fsl_mc_device, vdev); unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos); loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK; struct fsl_mc_device *mc_dev = vdev->mc_dev; @@ -468,9 +474,11 @@ static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region, size, vma->vm_page_prot); } -static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma) +static int vfio_fsl_mc_mmap(struct vfio_device *core_vdev, + struct vm_area_struct *vma) { - struct vfio_fsl_mc_device *vdev = device_data; + struct vfio_fsl_mc_device *vdev = + container_of(core_vdev, struct vfio_fsl_mc_device, vdev); struct fsl_mc_device *mc_dev = vdev->mc_dev; unsigned int index; diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c index 4043cc91f9524e..11b3e15403ba4f 100644 --- a/drivers/vfio/mdev/vfio_mdev.c +++ b/drivers/vfio/mdev/vfio_mdev.c @@ -21,10 +21,11 @@ #define DRIVER_AUTHOR "NVIDIA Corporation" #define DRIVER_DESC "VFIO based driver for Mediated device" -static int vfio_mdev_open(void *device_data) +static int vfio_mdev_open(struct vfio_device *core_vdev) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; + int ret; if (unlikely(!parent->ops->open)) @@ -40,9 +41,9 @@ static int vfio_mdev_open(void *device_data) return ret; } -static void vfio_mdev_release(void *device_data) +static void vfio_mdev_release(struct vfio_device *core_vdev) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; if (likely(parent->ops->release)) @@ -51,10 +52,10 @@ static void vfio_mdev_release(void *device_data) module_put(THIS_MODULE); } -static long vfio_mdev_unlocked_ioctl(void *device_data, +static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev, unsigned int cmd, unsigned long arg) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; if (unlikely(!parent->ops->ioctl)) @@ -63,10 +64,10 @@ static long vfio_mdev_unlocked_ioctl(void *device_data, return parent->ops->ioctl(mdev, cmd, arg); } -static ssize_t vfio_mdev_read(void *device_data, char __user *buf, +static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; if (unlikely(!parent->ops->read)) @@ -75,10 +76,11 @@ static ssize_t vfio_mdev_read(void *device_data, char __user *buf, return parent->ops->read(mdev, buf, count, ppos); } -static ssize_t vfio_mdev_write(void *device_data, const char __user *buf, - size_t count, loff_t *ppos) +static ssize_t vfio_mdev_write(struct vfio_device *core_vdev, + const char __user *buf, size_t count, + loff_t *ppos) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; if (unlikely(!parent->ops->write)) @@ -87,9 +89,10 @@ static ssize_t vfio_mdev_write(void *device_data, const char __user *buf, return parent->ops->write(mdev, buf, count, ppos); } -static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma) +static int vfio_mdev_mmap(struct vfio_device *core_vdev, + struct vm_area_struct *vma) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; if (unlikely(!parent->ops->mmap)) @@ -98,9 +101,9 @@ static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma) return parent->ops->mmap(mdev, vma); } -static void vfio_mdev_request(void *device_data, unsigned int count) +static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count) { - struct mdev_device *mdev = device_data; + struct mdev_device *mdev = to_mdev_device(core_vdev->dev); struct mdev_parent *parent = mdev->parent; if (parent->ops->request) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index a0ac20a499cf6c..5f1a782d1c65ae 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -553,9 +553,10 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val) vfio_device_put(pf_dev); } -static void vfio_pci_release(void *device_data) +static void vfio_pci_release(struct vfio_device *core_vdev) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); mutex_lock(&vdev->reflck->lock); @@ -581,9 +582,10 @@ static void vfio_pci_release(void *device_data) module_put(THIS_MODULE); } -static int vfio_pci_open(void *device_data) +static int vfio_pci_open(struct vfio_device *core_vdev) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); int ret = 0; if (!try_module_get(THIS_MODULE)) @@ -797,10 +799,11 @@ struct vfio_devices { int max_index; }; -static long vfio_pci_ioctl(void *device_data, +static long vfio_pci_ioctl(struct vfio_device *core_vdev, unsigned int cmd, unsigned long arg) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); unsigned long minsz; if (cmd == VFIO_DEVICE_GET_INFO) { @@ -1402,11 +1405,10 @@ static long vfio_pci_ioctl(void *device_data, return -ENOTTY; } -static ssize_t vfio_pci_rw(void *device_data, char __user *buf, +static ssize_t vfio_pci_rw(struct vfio_pci_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos); - struct vfio_pci_device *vdev = device_data; if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions) return -EINVAL; @@ -1434,22 +1436,28 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf, return -EINVAL; } -static ssize_t vfio_pci_read(void *device_data, char __user *buf, +static ssize_t vfio_pci_read(struct vfio_device *core_vdev, char __user *buf, size_t count, loff_t *ppos) { + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); + if (!count) return 0; - return vfio_pci_rw(device_data, buf, count, ppos, false); + return vfio_pci_rw(vdev, buf, count, ppos, false); } -static ssize_t vfio_pci_write(void *device_data, const char __user *buf, +static ssize_t vfio_pci_write(struct vfio_device *core_vdev, const char __user *buf, size_t count, loff_t *ppos) { + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); + if (!count) return 0; - return vfio_pci_rw(device_data, (char __user *)buf, count, ppos, true); + return vfio_pci_rw(vdev, (char __user *)buf, count, ppos, true); } /* Return 1 on zap and vma_lock acquired, 0 on contention (only with @try) */ @@ -1646,9 +1654,10 @@ static const struct vm_operations_struct vfio_pci_mmap_ops = { .fault = vfio_pci_mmap_fault, }; -static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) +static int vfio_pci_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); struct pci_dev *pdev = vdev->pdev; unsigned int index; u64 phys_len, req_len, pgoff, req_start; @@ -1714,9 +1723,10 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) return 0; } -static void vfio_pci_request(void *device_data, unsigned int count) +static void vfio_pci_request(struct vfio_device *core_vdev, unsigned int count) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); struct pci_dev *pdev = vdev->pdev; mutex_lock(&vdev->igate); @@ -1830,9 +1840,10 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, #define VF_TOKEN_ARG "vf_token=" -static int vfio_pci_match(void *device_data, char *buf) +static int vfio_pci_match(struct vfio_device *core_vdev, char *buf) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_device *vdev = + container_of(core_vdev, struct vfio_pci_device, vdev); bool vf_token = false; uuid_t uuid; int ret; diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c index 6eb749250ee41c..f5f6b537084a67 100644 --- a/drivers/vfio/platform/vfio_platform_common.c +++ b/drivers/vfio/platform/vfio_platform_common.c @@ -218,9 +218,10 @@ static int vfio_platform_call_reset(struct vfio_platform_device *vdev, return -EINVAL; } -static void vfio_platform_release(void *device_data) +static void vfio_platform_release(struct vfio_device *core_vdev) { - struct vfio_platform_device *vdev = device_data; + struct vfio_platform_device *vdev = + container_of(core_vdev, struct vfio_platform_device, vdev); mutex_lock(&driver_lock); @@ -244,9 +245,10 @@ static void vfio_platform_release(void *device_data) module_put(vdev->parent_module); } -static int vfio_platform_open(void *device_data) +static int vfio_platform_open(struct vfio_device *core_vdev) { - struct vfio_platform_device *vdev = device_data; + struct vfio_platform_device *vdev = + container_of(core_vdev, struct vfio_platform_device, vdev); int ret; if (!try_module_get(vdev->parent_module)) @@ -293,10 +295,12 @@ static int vfio_platform_open(void *device_data) return ret; } -static long vfio_platform_ioctl(void *device_data, +static long vfio_platform_ioctl(struct vfio_device *core_vdev, unsigned int cmd, unsigned long arg) { - struct vfio_platform_device *vdev = device_data; + struct vfio_platform_device *vdev = + container_of(core_vdev, struct vfio_platform_device, vdev); + unsigned long minsz; if (cmd == VFIO_DEVICE_GET_INFO) { @@ -455,10 +459,11 @@ static ssize_t vfio_platform_read_mmio(struct vfio_platform_region *reg, return -EFAULT; } -static ssize_t vfio_platform_read(void *device_data, char __user *buf, - size_t count, loff_t *ppos) +static ssize_t vfio_platform_read(struct vfio_device *core_vdev, + char __user *buf, size_t count, loff_t *ppos) { - struct vfio_platform_device *vdev = device_data; + struct vfio_platform_device *vdev = + container_of(core_vdev, struct vfio_platform_device, vdev); unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos); loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK; @@ -531,10 +536,11 @@ static ssize_t vfio_platform_write_mmio(struct vfio_platform_region *reg, return -EFAULT; } -static ssize_t vfio_platform_write(void *device_data, const char __user *buf, +static ssize_t vfio_platform_write(struct vfio_device *core_vdev, const char __user *buf, size_t count, loff_t *ppos) { - struct vfio_platform_device *vdev = device_data; + struct vfio_platform_device *vdev = + container_of(core_vdev, struct vfio_platform_device, vdev); unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos); loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK; @@ -573,9 +579,10 @@ static int vfio_platform_mmap_mmio(struct vfio_platform_region region, req_len, vma->vm_page_prot); } -static int vfio_platform_mmap(void *device_data, struct vm_area_struct *vma) +static int vfio_platform_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma) { - struct vfio_platform_device *vdev = device_data; + struct vfio_platform_device *vdev = + container_of(core_vdev, struct vfio_platform_device, vdev); unsigned int index; index = vma->vm_pgoff >> (VFIO_PLATFORM_OFFSET_SHIFT - PAGE_SHIFT); diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 180b4ab02d115a..e6f5109fba4858 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -832,7 +832,7 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group, int ret; if (it->ops->match) { - ret = it->ops->match(it->device_data, buf); + ret = it->ops->match(it, buf); if (ret < 0) { device = ERR_PTR(ret); break; @@ -893,7 +893,7 @@ void vfio_unregister_group_dev(struct vfio_device *device) rc = try_wait_for_completion(&device->comp); while (rc <= 0) { if (device->ops->request) - device->ops->request(device->device_data, i++); + device->ops->request(device, i++); if (interrupted) { rc = wait_for_completion_timeout(&device->comp, @@ -1379,7 +1379,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) if (IS_ERR(device)) return PTR_ERR(device); - ret = device->ops->open(device->device_data); + ret = device->ops->open(device); if (ret) { vfio_device_put(device); return ret; @@ -1391,7 +1391,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) */ ret = get_unused_fd_flags(O_CLOEXEC); if (ret < 0) { - device->ops->release(device->device_data); + device->ops->release(device); vfio_device_put(device); return ret; } @@ -1401,7 +1401,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) if (IS_ERR(filep)) { put_unused_fd(ret); ret = PTR_ERR(filep); - device->ops->release(device->device_data); + device->ops->release(device); vfio_device_put(device); return ret; } @@ -1558,7 +1558,7 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep) { struct vfio_device *device = filep->private_data; - device->ops->release(device->device_data); + device->ops->release(device); vfio_group_try_dissolve_container(device->group); @@ -1575,7 +1575,7 @@ static long vfio_device_fops_unl_ioctl(struct file *filep, if (unlikely(!device->ops->ioctl)) return -EINVAL; - return device->ops->ioctl(device->device_data, cmd, arg); + return device->ops->ioctl(device, cmd, arg); } static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf, @@ -1586,7 +1586,7 @@ static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf, if (unlikely(!device->ops->read)) return -EINVAL; - return device->ops->read(device->device_data, buf, count, ppos); + return device->ops->read(device, buf, count, ppos); } static ssize_t vfio_device_fops_write(struct file *filep, @@ -1598,7 +1598,7 @@ static ssize_t vfio_device_fops_write(struct file *filep, if (unlikely(!device->ops->write)) return -EINVAL; - return device->ops->write(device->device_data, buf, count, ppos); + return device->ops->write(device, buf, count, ppos); } static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma) @@ -1608,7 +1608,7 @@ static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma) if (unlikely(!device->ops->mmap)) return -EINVAL; - return device->ops->mmap(device->device_data, vma); + return device->ops->mmap(device, vma); } static const struct file_operations vfio_device_fops = { diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 4995faf51efeae..784c34c0a28763 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -44,17 +44,17 @@ struct vfio_device { */ struct vfio_device_ops { char *name; - int (*open)(void *device_data); - void (*release)(void *device_data); - ssize_t (*read)(void *device_data, char __user *buf, + int (*open)(struct vfio_device *vdev); + void (*release)(struct vfio_device *vdev); + ssize_t (*read)(struct vfio_device *vdev, char __user *buf, size_t count, loff_t *ppos); - ssize_t (*write)(void *device_data, const char __user *buf, + ssize_t (*write)(struct vfio_device *vdev, const char __user *buf, size_t count, loff_t *size); - long (*ioctl)(void *device_data, unsigned int cmd, + long (*ioctl)(struct vfio_device *vdev, unsigned int cmd, unsigned long arg); - int (*mmap)(void *device_data, struct vm_area_struct *vma); - void (*request)(void *device_data, unsigned int count); - int (*match)(void *device_data, char *buf); + int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma); + void (*request)(struct vfio_device *vdev, unsigned int count); + int (*match)(struct vfio_device *vdev, char *buf); }; extern struct iommu_group *vfio_iommu_group_get(struct device *dev); From patchwork Tue Mar 23 16:15:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CB42C433F4 for ; Tue, 23 Mar 2021 16:16:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E548619C8 for ; Tue, 23 Mar 2021 16:16:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233199AbhCWQPn (ORCPT ); Tue, 23 Mar 2021 12:15:43 -0400 Received: from mail-bn7nam10on2089.outbound.protection.outlook.com ([40.107.92.89]:23521 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233165AbhCWQPU (ORCPT ); Tue, 23 Mar 2021 12:15:20 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SUtGrwfa6s4Wq3MH8uQg0dL76zTD8RnvmM39Nmwu2bXDKqbgkU/7oHIZYcJOMysNRjzl6CusUy3NOzp9ojIKWwDmaxt0o5KoiTu0DiYDF+H/OxdChxvkFymG7tmzNrJcEOkz6UbjUKSpFO2sp+3cXYXd5CFvJ2ZeVaEZM6lWl2MD9eQHJlnmvDsh0/jcS4AB4pekPXLS+tBZhqWLKfiWZzUnAahNYS1US1pCJyV/JMFAu6Xq4Gj//3uJhCErYnr0Hfj+v+YUq9C4io1nJb6wVYnCOmeoeX7Tj+l5ywmOK9zF8LcQcwKXeuREX0qEXk3KAahG4zt9t6I4ClUknRlMbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YLo/UiP+k06LNuQAbMuufjeTq/JGtQ4B8qDrtHi8/+0=; b=CcYe279uep/g1tz8CJQxfv97AtgXdEUlHflaxJ0YT9wBtvhBiUN+kUTleNkm5a+hEbTWCLjHI1auk4N/NdWgmRpXAM5be8e8mdjkNQtuek2L84WMjY+NUYHlH8IRTIpLVQpQk+EVJKkt4Xz+fSdf1EtdNy73YOneSQ/RvMrhg2s3ODL8A2bVXX/kVwDBmIQ+bgMMe9Uzjzkz+14kQQ91TO71jNphsl59JBDvagyuzJVpwJOWfDtOO0ZKD3e6qHn86ayqKkfoQiGvjbqzr+wCTwDhM8qFB48usIPJcZEIvka5lwlGrf94f7gzKJ4BrOjmGsEw8wPsJ5HCTElZSv2bkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YLo/UiP+k06LNuQAbMuufjeTq/JGtQ4B8qDrtHi8/+0=; b=Mk3wnjdLALi5/ch+Fo1jT2mswsJuJmRyTGnSeEjL5CAH2A5bgw/Gy3EfB5FKWAQ4/KpdEbJAtB+vK9ocTQnWPKF2G971wdmSM1AtVl+Co9RCSeZi/0ZmsggETbCmAl+2TRqA621UH/SqgFJPUPka9aYvf9ENhHhyPerP/oC+tfAYf9rtufZTmTOVnf+LtEt+X/v3zWRCmdRUzLLRSo/Uq7ebyqyZfWHkxLYIQCmBy9tKGl1ueNWgSBKP0t4g4eP1GxZcy6O1pzQK87879noJ51LfcfZ5OnAeCW3ercQi2m1zxQ1UdvCDMzzEEL04UO4DzKFwLK9fYqxMHViXj+yAHw== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:14 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:14 +0000 From: Jason Gunthorpe To: Alex Williamson , kvm@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 13/14] vfio/pci: Replace uses of vfio_device_data() with container_of Date: Tue, 23 Mar 2021 13:15:05 -0300 Message-Id: <13-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: MN2PR19CA0009.namprd19.prod.outlook.com (2603:10b6:208:178::22) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by MN2PR19CA0009.namprd19.prod.outlook.com (2603:10b6:208:178::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24 via Frontend Transport; Tue, 23 Mar 2021 16:15:11 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgQ-001aDB-Uv; Tue, 23 Mar 2021 13:15:06 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aaf17599-4898-480f-69c3-08d8ee16d56f X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KEu+ongjVEupw00NCXJVYPkfrB8gktpl51rBagg7uZIlrXQGqsmRK1l6iGXqN8+rwKhjX5InpjwwQfiOtTDQ+rPCJ+fSebvmf7FAKXcOhqcQLr6kBFTxlcHiy5y457shujtvoldqp8R1wLnWfFaYjPo5Fnqp3JiWC1n0deH0YCCIWmTdnyiIcEXmW2mr175K0zxGUNF5os8Rf995RjSKcvMKWSyZlgilyCJ7Rfs1D7xs83IE2MBqO/IqOf0qUeItP2rpK5viRwNIZHaESD132x1HJB0syWpsXwaF1RG60C4+EShmWhK3k82kMexLVRfvkmtCjPgT8RiSP7n7Z0Hhc5QZxn8ORwNzc2KjbpPe84dj04v0/eEzU1TLZBQ7tlrP64PWeLs8y1pLatNgw41RRMxJGCcjedO4pXUCh4R4lxJQWFQZk91Ieos2LQpVDP12ZkpnhQGr10iApVptdi22ZgopfcGp/j7sE8KjCbK1dE4CK5Xv4/wT5hYi0TRMMG6TI0FrmkVwdt2DFWztPtXhT8Y3KEB+fVLssGmLCK7TkmiD2W/17uNLgOKXFFpzIgaEMHe8ct5ZDKDSWnB0P+mzUWIOr8xUHZrdMl5jOKbPL/5PIoYo4G+F+7EHVYGOnfF9VowxdwwIdzORDCLKchdIRRiHFWjxSUZ+QVegHqocrao= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(83380400001)(426003)(54906003)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: EsMsdfT1L8wwqSlt8pdRZeCpLI/1wfaWjcaD0Tr+h4gjcTiZeQLN85GFtIFYqefZSBQHBNU+WqbxfMr2zdXfe5UxXmM7pKWJZ9Al/xUbgkw1eHGhMbLbD9Iz3y3aN7ncWsbv54K7LXfLlys3jd+Wug2PBiOk0nC4qWypCqOX0SuAp08vQY19Djc0+LUouLn76Nxi/bqIl5RKxd5ovQaaSMqQCmZtsYmu8qIQbpNMtiRy5Q+jWTfwAz2UiHcBg9nGFmoPyYpmE/exesLJvZG61nuKhFyxoMm/vT4mEAGstid0mnG2oYfMzq+cao3C/PRl+OQDGEnQvQWeQ+uLNdry4BkfSCYHt7WAGcNWKYTwrzvb1GjqKhDlSPUWU+6xqkq4VuRleVk4YBcRBe+mkWHuDdOJxh2dmAhXzUcBElx5ksfMeh2xk5r4sWRGUgBwzGuvy0XN+c9CejzWxX6dzHnIr0xsFzRhhoDk4Ok2PQpwZHEMARmZl/1vopLgWQ9BoNF6xK7NF/z+Ceum1fIw5bnBKTag2fZtxqjhNgyMtWYy9q7Aml16O2Hb+aZg/vVipi02L99z7r3H5bpe+ACsLToNV1ae7rJPvVrZVnonSeaMQvI6m/VRvCohHQF2eH1AFFaVJTAxjrql9p3fVoFPHi6udlFD+yJhg1SzkTHyXL3tCqIx6PopyR46+N3QkFWcZkEhcj6NKTDN7L8Q++XNpCV1wnaKdbq/aKxb1AAyI8uHXqv2DK0X9LP9LSQAHA7O3+U1arRdwLSaMU4utoJzPrZRDMKRDjr0Uk6jbn+g72zmW1Dur3xIToTffd5cdjqPtqRQ+IY5EysCZ/Sev4LzrWLvEcJ9xvOdnyvcgkI0EzvRidQyrxjTOiYv5UvHGyzWEzQkMEAigkeN8VNSHc7IJzlb/CRcwC22s7wqC0S45+FABLz+zyTYrBMu50rJA7kM423qmCwa7Gs8BQwNfZCjtoOI6osZt4pZuWQw6TWqU9N5a2Zp//bkrxazB6uJ6VgfjXXN2JabKFuOEu97XxTBEeR+0lNngK1LYHnUdKsJlQBHz91lV7KPCBwqgOiMNDtDY7yjuyrtYpTCYnMjHf5BdEGBEvYAAPdv2b+sjvTiW/OlqzPJZDHDkT9CDZgtJNoXKyBVSyrCuS9QgekoSBKqGmekjV0QahHVs3yq262fbo53rS/Z97qbu8XWcUpWmfe1fBKqw5NNz3Kn+FJ3Y3r5r8wfYicWYoyBuRDCw0EbHVqIOcnqYeVtfobPlMpjSLHBhmHRDXVlETGCDSLrh9v49OJJXJNOS08yxSWRX4Ljrl0SNo11HJfiSHKTo3QwGMbh282v X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: aaf17599-4898-480f-69c3-08d8ee16d56f X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:11.2493 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BtsvSmRLv/oT9nukS14XzroFRx+31oF4aUNU1aGseJSiddLWzzYwHBnDAcZr2jWT X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This tidies a few confused places that think they can have a refcount on the vfio_device but the device_data could be NULL, that isn't possible by design. Most of the change falls out when struct vfio_devices is updated to just store the struct vfio_pci_device itself. This wasn't possible before because there was no easy way to get from the 'struct vfio_pci_device' to the 'struct vfio_device' to put back the refcount. Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Signed-off-by: Jason Gunthorpe --- drivers/vfio/pci/vfio_pci.c | 67 +++++++++++++------------------------ 1 file changed, 24 insertions(+), 43 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 5f1a782d1c65ae..1f70387c8afe37 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -517,30 +517,29 @@ static void vfio_pci_disable(struct vfio_pci_device *vdev) static struct pci_driver vfio_pci_driver; -static struct vfio_pci_device *get_pf_vdev(struct vfio_pci_device *vdev, - struct vfio_device **pf_dev) +static struct vfio_pci_device *get_pf_vdev(struct vfio_pci_device *vdev) { struct pci_dev *physfn = pci_physfn(vdev->pdev); + struct vfio_device *pf_dev; if (!vdev->pdev->is_virtfn) return NULL; - *pf_dev = vfio_device_get_from_dev(&physfn->dev); - if (!*pf_dev) + pf_dev = vfio_device_get_from_dev(&physfn->dev); + if (!pf_dev) return NULL; if (pci_dev_driver(physfn) != &vfio_pci_driver) { - vfio_device_put(*pf_dev); + vfio_device_put(pf_dev); return NULL; } - return vfio_device_data(*pf_dev); + return container_of(pf_dev, struct vfio_pci_device, vdev); } static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val) { - struct vfio_device *pf_dev; - struct vfio_pci_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); + struct vfio_pci_device *pf_vdev = get_pf_vdev(vdev); if (!pf_vdev) return; @@ -550,7 +549,7 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val) WARN_ON(pf_vdev->vf_token->users < 0); mutex_unlock(&pf_vdev->vf_token->lock); - vfio_device_put(pf_dev); + vfio_device_put(&pf_vdev->vdev); } static void vfio_pci_release(struct vfio_device *core_vdev) @@ -794,7 +793,7 @@ int vfio_pci_register_dev_region(struct vfio_pci_device *vdev, } struct vfio_devices { - struct vfio_device **devices; + struct vfio_pci_device **devices; int cur_index; int max_index; }; @@ -1283,9 +1282,7 @@ static long vfio_pci_ioctl(struct vfio_device *core_vdev, goto hot_reset_release; for (; mem_idx < devs.cur_index; mem_idx++) { - struct vfio_pci_device *tmp; - - tmp = vfio_device_data(devs.devices[mem_idx]); + struct vfio_pci_device *tmp = devs.devices[mem_idx]; ret = down_write_trylock(&tmp->memory_lock); if (!ret) { @@ -1300,17 +1297,13 @@ static long vfio_pci_ioctl(struct vfio_device *core_vdev, hot_reset_release: for (i = 0; i < devs.cur_index; i++) { - struct vfio_device *device; - struct vfio_pci_device *tmp; - - device = devs.devices[i]; - tmp = vfio_device_data(device); + struct vfio_pci_device *tmp = devs.devices[i]; if (i < mem_idx) up_write(&tmp->memory_lock); else mutex_unlock(&tmp->vma_lock); - vfio_device_put(device); + vfio_device_put(&tmp->vdev); } kfree(devs.devices); @@ -1777,8 +1770,7 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, return 0; /* No VF token provided or required */ if (vdev->pdev->is_virtfn) { - struct vfio_device *pf_dev; - struct vfio_pci_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); + struct vfio_pci_device *pf_vdev = get_pf_vdev(vdev); bool match; if (!pf_vdev) { @@ -1791,7 +1783,7 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, } if (!vf_token) { - vfio_device_put(pf_dev); + vfio_device_put(&pf_vdev->vdev); pci_info_ratelimited(vdev->pdev, "VF token required to access device\n"); return -EACCES; @@ -1801,7 +1793,7 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, match = uuid_equal(uuid, &pf_vdev->vf_token->uuid); mutex_unlock(&pf_vdev->vf_token->lock); - vfio_device_put(pf_dev); + vfio_device_put(&pf_vdev->vdev); if (!match) { pci_info_ratelimited(vdev->pdev, @@ -2122,11 +2114,7 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, if (device == NULL) return PCI_ERS_RESULT_DISCONNECT; - vdev = vfio_device_data(device); - if (vdev == NULL) { - vfio_device_put(device); - return PCI_ERS_RESULT_DISCONNECT; - } + vdev = container_of(device, struct vfio_pci_device, vdev); mutex_lock(&vdev->igate); @@ -2142,7 +2130,6 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) { - struct vfio_pci_device *vdev; struct vfio_device *device; int ret = 0; @@ -2155,12 +2142,6 @@ static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) if (!device) return -ENODEV; - vdev = vfio_device_data(device); - if (!vdev) { - vfio_device_put(device); - return -ENODEV; - } - if (nr_virtfn == 0) pci_disable_sriov(pdev); else @@ -2220,7 +2201,7 @@ static int vfio_pci_reflck_find(struct pci_dev *pdev, void *data) return 0; } - vdev = vfio_device_data(device); + vdev = container_of(device, struct vfio_pci_device, vdev); if (vdev->reflck) { vfio_pci_reflck_get(vdev->reflck); @@ -2282,7 +2263,7 @@ static int vfio_pci_get_unused_devs(struct pci_dev *pdev, void *data) return -EBUSY; } - vdev = vfio_device_data(device); + vdev = container_of(device, struct vfio_pci_device, vdev); /* Fault if the device is not unused */ if (vdev->refcnt) { @@ -2290,7 +2271,7 @@ static int vfio_pci_get_unused_devs(struct pci_dev *pdev, void *data) return -EBUSY; } - devs->devices[devs->cur_index++] = device; + devs->devices[devs->cur_index++] = vdev; return 0; } @@ -2312,7 +2293,7 @@ static int vfio_pci_try_zap_and_vma_lock_cb(struct pci_dev *pdev, void *data) return -EBUSY; } - vdev = vfio_device_data(device); + vdev = container_of(device, struct vfio_pci_device, vdev); /* * Locking multiple devices is prone to deadlock, runaway and @@ -2323,7 +2304,7 @@ static int vfio_pci_try_zap_and_vma_lock_cb(struct pci_dev *pdev, void *data) return -EBUSY; } - devs->devices[devs->cur_index++] = device; + devs->devices[devs->cur_index++] = vdev; return 0; } @@ -2371,7 +2352,7 @@ static void vfio_pci_try_bus_reset(struct vfio_pci_device *vdev) /* Does at least one need a reset? */ for (i = 0; i < devs.cur_index; i++) { - tmp = vfio_device_data(devs.devices[i]); + tmp = devs.devices[i]; if (tmp->needs_reset) { ret = pci_reset_bus(vdev->pdev); break; @@ -2380,7 +2361,7 @@ static void vfio_pci_try_bus_reset(struct vfio_pci_device *vdev) put_devs: for (i = 0; i < devs.cur_index; i++) { - tmp = vfio_device_data(devs.devices[i]); + tmp = devs.devices[i]; /* * If reset was successful, affected devices no longer need @@ -2396,7 +2377,7 @@ static void vfio_pci_try_bus_reset(struct vfio_pci_device *vdev) vfio_pci_set_power_state(tmp, PCI_D3hot); } - vfio_device_put(devs.devices[i]); + vfio_device_put(&tmp->vdev); } kfree(devs.devices); From patchwork Tue Mar 23 16:15:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12158701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0D24C433EA for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CBB6161990 for ; Tue, 23 Mar 2021 16:16:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233195AbhCWQPl (ORCPT ); Tue, 23 Mar 2021 12:15:41 -0400 Received: from mail-co1nam11on2068.outbound.protection.outlook.com ([40.107.220.68]:19553 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S233152AbhCWQPS (ORCPT ); Tue, 23 Mar 2021 12:15:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IfLf9IUqHSx3RkShku651zfQiUSTGjC9aSBCXoA3Vq0Y5q2pST1UyhxLbgJiXNIli+orkyNkl4lAI9RHqrjHZGOmhCulhezHDcULiKqARonuFWdkLTQANWt6jVwSx05XBtNNzNxXYlKxV7RuHY60/OObPhJmbky8v/Axf8bta7wiLe73noPt2dvOSp+0ONTtQYq6t7A/IskfXZxTZIaDExyQmx/YIHT9MoxiEYk6om+uJ9slnjH+LeiAB04U66csYKco0ft6VrsA5rKiFerOim4OVsmClCtmgThsPpNfPSZMM7F8hYQKlUCWMT5T8vTjyJiu1HNRrW8o0qRob2NKcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AEak6oZ5oGrh/36XE8vfMtDr01GFf3MViduAKRXlsog=; b=NOMdI/xG6x0uoauOizQDtXBQ8KFkoTOcy78SOuObN8Cf4zl51v0jak6lwrcUWClZPUHzrn3NpiApkLdKHzumgqGsAb29TH81ilANqZbgge8ueS7rBU6pOdlKJMxwDnVP7aRXS4flMlcepl0nO7F0p9eiQ2t6f2I9HEKxXztEF+f9A2pVuA7+mXJWnlaz/M45gfNYv4MVsj9J820kjLvXiK+1zl9IHEFuk3oXBSbOWyQq+f94hSBB8FzIcLytqLyPf58JrjG6uY7BHLjLXzUeS8EgKinl7wUzZr4t1Q6OfK/CaZ4iJHeUlRPW0LRHMD9Ia/PvbQnH7IaVfxtG0ZFStw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AEak6oZ5oGrh/36XE8vfMtDr01GFf3MViduAKRXlsog=; b=sRypVxesv7lrsM6ThVuhES0Z4o9JRARqU1vP1O+4LTjhm/18650cmhamZ5aDQgh5SIgr8IFfxiM8G3OiJOovKdtBDHNwXW7hPzOBmz6yIclFyvSG8lAeO9EPBnGfAZl2oWnjd7Y26oJ6NRnXP09ddB1nRhSLLySZoDCx1+74Zaz7kcQgomCcyyKQoiz6B6macIq0T3yjILAim7akWhS07dSTKx4Y5aEAD/sHQLYKN0yVX+2oFS7/RSGXlF/B4M7YorWuqWqBEwtcIVCoHh/MhIRYYdYIgwgkU6RcmAQAu1AIpPEZkG56lxeaI3n714MT466dw0kZR3E5r7xoaiqFVA== Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) by DM6PR12MB4267.namprd12.prod.outlook.com (2603:10b6:5:21e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.24; Tue, 23 Mar 2021 16:15:12 +0000 Received: from DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87]) by DM6PR12MB3834.namprd12.prod.outlook.com ([fe80::1c62:7fa3:617b:ab87%6]) with mapi id 15.20.3955.027; Tue, 23 Mar 2021 16:15:12 +0000 From: Jason Gunthorpe To: Alex Williamson , Jonathan Corbet , Diana Craciun , Eric Auger , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org Cc: "Raj, Ashok" , Cornelia Huck , Dan Williams , Daniel Vetter , Christoph Hellwig , Kevin Tian , Leon Romanovsky , Max Gurtovoy , Tarun Gupta Subject: [PATCH v3 14/14] vfio: Remove device_data from the vfio bus driver API Date: Tue, 23 Mar 2021 13:15:06 -0300 Message-Id: <14-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> In-Reply-To: <0-v3-225de1400dfc+4e074-vfio1_jgg@nvidia.com> References: X-Originating-IP: [206.223.160.26] X-ClientProxiedBy: BL1PR13CA0253.namprd13.prod.outlook.com (2603:10b6:208:2ba::18) To DM6PR12MB3834.namprd12.prod.outlook.com (2603:10b6:5:14a::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mlx.ziepe.ca (206.223.160.26) by BL1PR13CA0253.namprd13.prod.outlook.com (2603:10b6:208:2ba::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.9 via Frontend Transport; Tue, 23 Mar 2021 16:15:09 +0000 Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1lOjgR-001aDG-1e; Tue, 23 Mar 2021 13:15:07 -0300 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8455ccc3-5a4d-42c8-a45d-08d8ee16d495 X-MS-TrafficTypeDiagnostic: DM6PR12MB4267: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4125; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9NQmoEkx6tvWkYkATEzgPChb4f7J0S5uTcW3z7PTw3ziweJHmFZW2m0pk2IpXmL8redXfwQcd5fkfWXe5tFuvAogh2gDdlwaK2CrDPOs7E0Ym7Mjr/0HGhf4+7YtHeAbHlZcCVmC5ZxH5e01Aea6mfPw0xQ1NqLiY2ojVgZFL1nktja/pBd9N45JeqHiWiDWnnvZR9GdoNN12WUbTCa/yUnC2ym0j/nWNSOfNtmKRMQTHYWFjAwr96fKfLBnSZQneW3IuWtXh8g5Jg0vQAlrVEeCsVurx6RLOqKQydcdJ2qOPuN/LXRV4VqgpsjNXY2f6m71OUUd7c9YeXiuVjQK0eDbATlkA0bKYWtELd/f2mP67iVjMFHdnCUGdt9pK5FKeDo9RYspfxFTvFI3yXElY4ZKzA/2E+vZLO8YXlGWPFEkX/kw6VqmB2d8iwd4AxtVyk7Rlbp0kBIorcVXFyF/xpPykpYIptX5z0vArYLpFsWMkId/ClZq0Zl66voxcCR71haveq8ozk4v/CtriOHyY4jmMWQG5133cfclLK+9OEMqGvXUODbAiNPdCHet9ECIxsEJ9BKxDH76G5HvFQHydXjqMcI8Fvr4ltOzu/WVRqB8VFQHXFjh4lm36TIy5KXW09BmQWL91j9D5oju/T3BxOVQVXcTXrR2HRWSQV2boyw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(376002)(136003)(396003)(366004)(5660300002)(66476007)(2616005)(66946007)(66556008)(2906002)(186003)(7416002)(83380400001)(426003)(54906003)(110136005)(316002)(8936002)(478600001)(8676002)(107886003)(4326008)(26005)(9746002)(9786002)(86362001)(36756003)(38100700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ux4RG0L/ltl6Wyr3a/FWa15GTYgdwJINtxkz+cFzfkOcEUxv2TY5wn24WVrFkEcITFEPpYz+JSQVroCAJVoyiVqgmPrB69JDonPh1c/WBpxZl3HQgyQNp1WGIx35IeMtLL6a9q/1jl8HLrAFi9eS6twRCZ8DB9i42CgkkDt7WZ/rwR7/Vpo6as5bUBgvRLpWJRe1ZEHmOYOITsVopsCcj3WzJCE+3Y7fmLPLVc3UuEqNkSW28OjxA5LS7xn7bbg94hhA5pjuVZ5JAU0bJC7BPQX2Yvp74ZNG9Xndfg5L8PU1ntl2jlAaxzRPXY40COiogPh5oE3mnZOTxwhjE+m35WODzdpjF0Yqn8HrLBLjXe0Lt3GfjstUzeBQmdGnYlbetriw7rlSP/3c9ycYHVpOWGksMwna2I/rgedNiVjSF+tmTVi0Z6e9kgExvu7bSW2kinq/MW5AG8k4z4oGYAZ2oE/uEIn0DHBhrzbz/Zf5EE5rjesCUUgzN41DM4FBHhO/1gp1e7oZmZ6bcwlzzzdTajZ1AMKN5XbKUufJ78gxDpCT7SZWWsFj/tLe7/TA6h8J4NSvbGM/a4KgrLTZwgIkKsiZEfac1d1vbNKqZUaI/Wnsk0ACNMOFitQFg4q180lwydNOza/jC7NwcUQ3tnDlC/KhXWTqZm4ZnVG8imQKbxF/RUcNNVqxLOSsQmqV3AiszGVEawbbiC66/hRfW4FrRQfmPEDNUXdmHGCU3cANQ8nKrv58NFmmAKDu+bnj8kt2KlehzN7FlVZxdTJDyeAX2i9dcSmNUFfM1n5l3ITuMcxEUWDWlZ5SuMaEuxhT/5YI2cBkBmzv8ICSLbtnklNX8sgbrz7d9maNcRhQgwnLVaw5c/9zjcWPIvrdalps/aGQp5/NKmJ8DSoIh12TcszVoOThFh8P23UIzLgBnOYpcXq4Pb56ZUg5vHqbHR/JXj2K5MuUwktZjDjKMOhTTxCa6FKN0YPPjRDRZj7yXlFdu7mUpLpODMu3wQcpAPsYAR5HF37QAiex5uWCElT+W/gS8WN0JclzZcz4mUXd3Lrhr8V91sBp2YMEn/IDleg+4iKvIY35LZuehZtnwKoJnH4OWj/XvzQkA1PgfrhHdJobfs0/hz+X3UDI9HAIRW2JXjpCi8IIJ5kbz+8NpLQUqLBU6b3MxOkqtiaTFjjAtIGeSigpNaTnp0uDrnfwKufGfvcdSNdvc1gnbNokx5GPrEondeXrdbIkV7hBCE4sSFgylUevJK2vMHjFWcuiKS7hSIgAiZWzwCaEJ+iCuuV1l/ueGSfUnCcmpR8G1YsNrBIVyYbN/gI8lWbc0Uab0GqNLyVb X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8455ccc3-5a4d-42c8-a45d-08d8ee16d495 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 16:15:09.7812 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y6TOZ3PTYOhuplm9EpH66nA1eK9BSJm7zyoyxV5hY9JqGLvq77fya5AigdRYSBa5 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4267 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org There are no longer any users, so it can go away. Everything is using container_of now. Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Reviewed-by: Cornelia Huck Reviewed-by: Max Gurtovoy Signed-off-by: Jason Gunthorpe --- Documentation/driver-api/vfio.rst | 3 +-- drivers/vfio/fsl-mc/vfio_fsl_mc.c | 5 +++-- drivers/vfio/mdev/vfio_mdev.c | 2 +- drivers/vfio/pci/vfio_pci.c | 2 +- drivers/vfio/platform/vfio_platform_common.c | 2 +- drivers/vfio/vfio.c | 12 +----------- include/linux/vfio.h | 4 +--- 7 files changed, 9 insertions(+), 21 deletions(-) diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst index 3337f337293a32..decc68cb8114ac 100644 --- a/Documentation/driver-api/vfio.rst +++ b/Documentation/driver-api/vfio.rst @@ -254,8 +254,7 @@ vfio_unregister_group_dev() respectively:: void vfio_init_group_dev(struct vfio_device *device, struct device *dev, - const struct vfio_device_ops *ops, - void *device_data); + const struct vfio_device_ops *ops); int vfio_register_group_dev(struct vfio_device *device); void vfio_unregister_group_dev(struct vfio_device *device); diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c index 45f397c04a8959..980e5955130197 100644 --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c @@ -75,7 +75,8 @@ static int vfio_fsl_mc_reflck_attach(struct vfio_fsl_mc_device *vdev) goto unlock; } - cont_vdev = vfio_device_data(device); + cont_vdev = + container_of(device, struct vfio_fsl_mc_device, vdev); if (!cont_vdev || !cont_vdev->reflck) { vfio_device_put(device); ret = -ENODEV; @@ -630,7 +631,7 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev) goto out_group_put; } - vfio_init_group_dev(&vdev->vdev, dev, &vfio_fsl_mc_ops, vdev); + vfio_init_group_dev(&vdev->vdev, dev, &vfio_fsl_mc_ops); vdev->mc_dev = mc_dev; mutex_init(&vdev->igate); diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c index 11b3e15403ba4f..ae7e322fbe3c26 100644 --- a/drivers/vfio/mdev/vfio_mdev.c +++ b/drivers/vfio/mdev/vfio_mdev.c @@ -134,7 +134,7 @@ static int vfio_mdev_probe(struct device *dev) if (!vdev) return -ENOMEM; - vfio_init_group_dev(vdev, &mdev->dev, &vfio_mdev_dev_ops, mdev); + vfio_init_group_dev(vdev, &mdev->dev, &vfio_mdev_dev_ops); ret = vfio_register_group_dev(vdev); if (ret) { kfree(vdev); diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 1f70387c8afe37..55ef27a15d4d3f 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -2022,7 +2022,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto out_group_put; } - vfio_init_group_dev(&vdev->vdev, &pdev->dev, &vfio_pci_ops, vdev); + vfio_init_group_dev(&vdev->vdev, &pdev->dev, &vfio_pci_ops); vdev->pdev = pdev; vdev->irq_type = VFIO_PCI_NUM_IRQS; mutex_init(&vdev->igate); diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c index f5f6b537084a67..361e5b57e36932 100644 --- a/drivers/vfio/platform/vfio_platform_common.c +++ b/drivers/vfio/platform/vfio_platform_common.c @@ -666,7 +666,7 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev, struct iommu_group *group; int ret; - vfio_init_group_dev(&vdev->vdev, dev, &vfio_platform_ops, vdev); + vfio_init_group_dev(&vdev->vdev, dev, &vfio_platform_ops); ret = vfio_platform_acpi_probe(vdev, dev); if (ret) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index e6f5109fba4858..5e631c359ef23c 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -741,12 +741,11 @@ static int vfio_iommu_group_notifier(struct notifier_block *nb, * VFIO driver API */ void vfio_init_group_dev(struct vfio_device *device, struct device *dev, - const struct vfio_device_ops *ops, void *device_data) + const struct vfio_device_ops *ops) { init_completion(&device->comp); device->dev = dev; device->ops = ops; - device->device_data = device_data; } EXPORT_SYMBOL_GPL(vfio_init_group_dev); @@ -851,15 +850,6 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group, return device; } -/* - * Caller must hold a reference to the vfio_device - */ -void *vfio_device_data(struct vfio_device *device) -{ - return device->device_data; -} -EXPORT_SYMBOL_GPL(vfio_device_data); - /* * Decrement the device reference count and wait for the device to be * removed. Open file descriptors for the device... */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 784c34c0a28763..a2c5b30e1763ba 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -24,7 +24,6 @@ struct vfio_device { refcount_t refcount; struct completion comp; struct list_head group_next; - void *device_data; }; /** @@ -61,12 +60,11 @@ extern struct iommu_group *vfio_iommu_group_get(struct device *dev); extern void vfio_iommu_group_put(struct iommu_group *group, struct device *dev); void vfio_init_group_dev(struct vfio_device *device, struct device *dev, - const struct vfio_device_ops *ops, void *device_data); + const struct vfio_device_ops *ops); int vfio_register_group_dev(struct vfio_device *device); void vfio_unregister_group_dev(struct vfio_device *device); extern struct vfio_device *vfio_device_get_from_dev(struct device *dev); extern void vfio_device_put(struct vfio_device *device); -extern void *vfio_device_data(struct vfio_device *device); /* events for the backend driver notify callback */ enum vfio_iommu_notify_type {