From patchwork Tue Feb 11 23:05:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377097 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44C1A924 for ; Tue, 11 Feb 2020 23:05:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2452D20848 for ; Tue, 11 Feb 2020 23:05:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AUHJ/TVI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727821AbgBKXFe (ORCPT ); Tue, 11 Feb 2020 18:05:34 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:43921 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727799AbgBKXFd (ORCPT ); Tue, 11 Feb 2020 18:05:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462333; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QeJkTOJSqucr7TuGuD5+LQV+rMV+lFuwF5Rz1IX0QZ4=; b=AUHJ/TVItGI7dWtbt+QURvfIUHetC0IhReML0kP2YgwYqKu00KH/dZ/kqFEcaP8S5ohycJ LdzIxLRgq2penhH0n1+xRP1IT3Oz9GDw0icetmwbO5CsJKF9ZIbvzdkXHR2RQoQmWAu9nr 3LdY/57Lo9GQLN0Y3WrUmZxZZVqekOk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-383-HcVH2L5WMN-9KW4I4nN_ww-1; Tue, 11 Feb 2020 18:05:31 -0500 X-MC-Unique: HcVH2L5WMN-9KW4I4nN_ww-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 288951005502; Tue, 11 Feb 2020 23:05:29 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id E2CCF26FAA; Tue, 11 Feb 2020 23:05:25 +0000 (UTC) Subject: [PATCH 1/7] vfio: Include optional device match in vfio_device_ops callbacks From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:05:25 -0700 Message-ID: <158146232551.16827.14170770732904274160.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Allow bus drivers to provide their own callback to match a device to the user provided string. Signed-off-by: Alex Williamson Reviewed-by: Cornelia Huck --- drivers/vfio/vfio.c | 20 ++++++++++++++++---- include/linux/vfio.h | 4 ++++ 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index c8482624ca34..0bd77d6ea691 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -875,11 +875,23 @@ EXPORT_SYMBOL_GPL(vfio_device_get_from_dev); static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group, char *buf) { - struct vfio_device *it, *device = NULL; + struct vfio_device *it, *device = ERR_PTR(-ENODEV); mutex_lock(&group->device_lock); list_for_each_entry(it, &group->device_list, group_next) { - if (!strcmp(dev_name(it->dev), buf)) { + int ret; + + if (it->ops->match) { + ret = it->ops->match(it->device_data, buf); + if (ret < 0) { + device = ERR_PTR(ret); + break; + } + } else { + ret = !strcmp(dev_name(it->dev), buf); + } + + if (ret) { device = it; vfio_device_get(device); break; @@ -1430,8 +1442,8 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) return -EPERM; device = vfio_device_get_from_name(group, buf); - if (!device) - return -ENODEV; + if (IS_ERR(device)) + return PTR_ERR(device); ret = device->ops->open(device->device_data); if (ret) { diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e42a711a2800..029694b977f2 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -26,6 +26,9 @@ * operations documented below * @mmap: Perform mmap(2) on a region of the device file descriptor * @request: Request for the bus driver to release the device + * @match: Optional device name match callback (return: 0 for no-match, >0 for + * match, -errno for abort (ex. match with insufficient or incorrect + * additional args) */ struct vfio_device_ops { char *name; @@ -39,6 +42,7 @@ struct vfio_device_ops { unsigned long arg); int (*mmap)(void *device_data, struct vm_area_struct *vma); void (*request)(void *device_data, unsigned int count); + int (*match)(void *device_data, char *buf); }; extern struct iommu_group *vfio_iommu_group_get(struct device *dev); From patchwork Tue Feb 11 23:05:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377103 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 36BC81820 for ; Tue, 11 Feb 2020 23:05:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0C49820842 for ; Tue, 11 Feb 2020 23:05:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SC4VRpq+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727902AbgBKXFp (ORCPT ); Tue, 11 Feb 2020 18:05:45 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:59403 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727888AbgBKXFo (ORCPT ); Tue, 11 Feb 2020 18:05:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462343; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=egySVQn95d75jOeqVctft40gVEWO2D/Q7wNGOIZKyvo=; b=SC4VRpq+jbSyokbJtns/gZcAGpHxJiq4RjkjMt0hN0EBincum/11dxtjaFz4gddt0sdoGI 0TAYbIHpXWK3vPAr2FHfKzw5yYtcLo8rUDQX9ZsmYaylfSXbM4JaCb9HKuyT3pEJy0Dwb/ UtsHV+oajzos2yZVCag+NwdB6r04zTI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-12-JMS2qA9-Nkez43ezurFWCA-1; Tue, 11 Feb 2020 18:05:39 -0500 X-MC-Unique: JMS2qA9-Nkez43ezurFWCA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A559B107ACC4; Tue, 11 Feb 2020 23:05:37 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9DDCD5C1B2; Tue, 11 Feb 2020 23:05:34 +0000 (UTC) Subject: [PATCH 2/7] vfio/pci: Implement match ops From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:05:34 -0700 Message-ID: <158146233422.16827.5520548241096752615.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org This currently serves the same purpose as the default implementation but will be expanded for additional functionality. Signed-off-by: Alex Williamson Reviewed-by: Cornelia Huck --- drivers/vfio/pci/vfio_pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 379a02c36e37..2ec6c31d0ab0 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -1278,6 +1278,13 @@ static void vfio_pci_request(void *device_data, unsigned int count) mutex_unlock(&vdev->igate); } +static int vfio_pci_match(void *device_data, char *buf) +{ + struct vfio_pci_device *vdev = device_data; + + return !strcmp(pci_name(vdev->pdev), buf); +} + static const struct vfio_device_ops vfio_pci_ops = { .name = "vfio-pci", .open = vfio_pci_open, @@ -1287,6 +1294,7 @@ static const struct vfio_device_ops vfio_pci_ops = { .write = vfio_pci_write, .mmap = vfio_pci_mmap, .request = vfio_pci_request, + .match = vfio_pci_match, }; static int vfio_pci_reflck_attach(struct vfio_pci_device *vdev); From patchwork Tue Feb 11 23:05:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377107 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3A2E921 for ; Tue, 11 Feb 2020 23:05:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8B37520842 for ; Tue, 11 Feb 2020 23:05:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AAydHoxo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727946AbgBKXFx (ORCPT ); Tue, 11 Feb 2020 18:05:53 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:57542 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727799AbgBKXFw (ORCPT ); Tue, 11 Feb 2020 18:05:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ecu+bO1iPm9vxlclR86oXw1EnTFQAryRn0Pe9nOuf94=; b=AAydHoxoybS2IZbopchN1+PDOLrWYtcWQKSgk2YGIJf9GvrI7aIqp5aAq+Diczr3JtIFh+ P5ENCAUtkRPpAl3jM5TqLnljnb4cfKBbIyn7/T0EG9fjjRP3X0ww5fbwJ+EZ/uEO/9EZIA p2fbTn68hMhvQ0uuK6vMq13EyL2gBNY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-NjE8S7QZOXmYlHkzKyQCPw-1; Tue, 11 Feb 2020 18:05:47 -0500 X-MC-Unique: NjE8S7QZOXmYlHkzKyQCPw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4338E13E6; Tue, 11 Feb 2020 23:05:46 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29FD560BF1; Tue, 11 Feb 2020 23:05:43 +0000 (UTC) Subject: [PATCH 3/7] vfio/pci: Introduce VF token From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:05:42 -0700 Message-ID: <158146234273.16827.10591457733122265139.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org If we enable SR-IOV on a vfio-pci owned PF, the resulting VFs are not fully isolated from the PF. The PF can always cause a denial of service to the VF, if not access data passed through the VF directly. This is why vfio-pci currently does not bind to PFs with SR-IOV enabled and does not provide access itself to enabling SR-IOV on a PF. The IOMMU grouping mechanism might allow us a solution to this lack of isolation, however the deficiency isn't actually in the DMA path, so much as the potential cooperation between PF and VF devices. Also, if we were to force VFs into the same IOMMU group as the PF, we severely limit the utility of having independent drivers managing PFs and VFs with vfio. Therefore we introduce the concept of a VF token. The token is implemented as a UUID and represents a shared secret which must be set by the PF driver and used by the VF drivers in order to access a vfio device file descriptor for the VF. The ioctl to set the VF token will be provided in a later commit, this commit implements the underlying infrastructure. The concept here is to augment the string the user passes to match a device within a group in order to retrieve access to the device descriptor. For example, rather than passing only the PCI device name (ex. "0000:03:00.0") the user would also pass a vf_token UUID (ex. "2ab74924-c335-45f4-9b16-8569e5b08258"). The device match string therefore becomes: "0000:03:00.0 vf_token=2ab74924-c335-45f4-9b16-8569e5b08258" This syntax is expected to be extensible to future options as well, with the standard being: "$DEVICE_NAME $OPTION1=$VALUE1 $OPTION2=$VALUE2" The device name must be first and option=value pairs are separated by spaces. The vf_token option is only required for VFs where the PF device is bound to vfio-pci. There is no change for PFs using existing host drivers. Note that in order to protect existing VF users, not only is it required to set a vf_token on the PF before VFs devices can be accessed, but also if there are existing VF users, (re)opening the PF device must also provide the current vf_token as authentication. This is intended to prevent a VF driver starting with a trusted PF driver and later being replaced by an unknown driver. A vf_token is not required to open the PF device when none of the VF devices are in use by vfio-pci drivers. Signed-off-by: Alex Williamson --- drivers/vfio/pci/vfio_pci.c | 127 +++++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_private.h | 8 ++ 2 files changed, 134 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 2ec6c31d0ab0..26aea9ac4863 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -466,6 +466,35 @@ static void vfio_pci_disable(struct vfio_pci_device *vdev) vfio_pci_set_power_state(vdev, PCI_D3hot); } +static struct pci_driver vfio_pci_driver; + +static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val) +{ + struct pci_dev *physfn = pci_physfn(vdev->pdev); + struct vfio_device *pf_dev; + struct vfio_pci_device *pf_vdev; + + if (!vdev->pdev->is_virtfn) + return; + + pf_dev = vfio_device_get_from_dev(&physfn->dev); + if (!pf_dev) + return; + + if (pci_dev_driver(physfn) != &vfio_pci_driver) { + vfio_device_put(pf_dev); + return; + } + + pf_vdev = vfio_device_data(pf_dev); + + mutex_lock(&pf_vdev->vf_token->lock); + pf_vdev->vf_token->users += val; + mutex_unlock(&pf_vdev->vf_token->lock); + + vfio_device_put(pf_dev); +} + static void vfio_pci_release(void *device_data) { struct vfio_pci_device *vdev = device_data; @@ -475,6 +504,7 @@ static void vfio_pci_release(void *device_data) if (!(--vdev->refcnt)) { vfio_spapr_pci_eeh_release(vdev->pdev); vfio_pci_disable(vdev); + vfio_pci_vf_token_user_add(vdev, -1); } mutex_unlock(&vdev->reflck->lock); @@ -493,6 +523,7 @@ static int vfio_pci_open(void *device_data) mutex_lock(&vdev->reflck->lock); if (!vdev->refcnt) { + vfio_pci_vf_token_user_add(vdev, 1); ret = vfio_pci_enable(vdev); if (ret) goto error; @@ -1278,11 +1309,86 @@ static void vfio_pci_request(void *device_data, unsigned int count) mutex_unlock(&vdev->igate); } +#define VF_TOKEN_ARG "vf_token=" + +/* Called with vdev->vf_token->lock */ +static int vfio_pci_vf_token_match(struct vfio_pci_device *vdev, char *opts) +{ + char *token; + uuid_t uuid; + int ret; + + if (!opts) + return -EINVAL; + + token = strstr(opts, VF_TOKEN_ARG); + if (!token) + return -EINVAL; + + token += strlen(VF_TOKEN_ARG); + + ret = uuid_parse(token, &uuid); + if (ret) + return ret; + + if (!uuid_equal(&uuid, &vdev->vf_token->uuid)) + return -EACCES; + + return 0; +} + static int vfio_pci_match(void *device_data, char *buf) { struct vfio_pci_device *vdev = device_data; + char *opts; + + opts = strchr(buf, ' '); + if (opts) { + *opts = 0; + opts++; + } + + if (strcmp(pci_name(vdev->pdev), buf)) + return 0; /* No match */ + + if (vdev->pdev->is_virtfn) { + struct pci_dev *physfn = pci_physfn(vdev->pdev); + struct vfio_device *pf_dev; + int ret = 0; + + pf_dev = vfio_device_get_from_dev(&physfn->dev); + if (pf_dev) { + if (pci_dev_driver(physfn) == &vfio_pci_driver) { + struct vfio_pci_device *pf_vdev = + vfio_device_data(pf_dev); + + mutex_lock(&pf_vdev->vf_token->lock); + ret = vfio_pci_vf_token_match(pf_vdev, opts); + mutex_unlock(&pf_vdev->vf_token->lock); + } + + vfio_device_put(pf_dev); + + if (ret) + return -EACCES; + } + } - return !strcmp(pci_name(vdev->pdev), buf); + if (vdev->vf_token) { + int ret = 0; + + mutex_lock(&vdev->vf_token->lock); + + if (vdev->vf_token->users) + ret = vfio_pci_vf_token_match(vdev, opts); + + mutex_unlock(&vdev->vf_token->lock); + + if (ret) + return -EACCES; + } + + return 1; /* Match */ } static const struct vfio_device_ops vfio_pci_ops = { @@ -1354,6 +1460,19 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) return ret; } + if (pdev->is_physfn) { + vdev->vf_token = kzalloc(sizeof(*vdev->vf_token), GFP_KERNEL); + if (!vdev->vf_token) { + vfio_pci_reflck_put(vdev->reflck); + vfio_del_group_dev(&pdev->dev); + vfio_iommu_group_put(group, &pdev->dev); + kfree(vdev); + return -ENOMEM; + } + mutex_init(&vdev->vf_token->lock); + uuid_gen(&vdev->vf_token->uuid); + } + if (vfio_pci_is_vga(pdev)) { vga_client_register(pdev, vdev, NULL, vfio_pci_set_vga_decode); vga_set_legacy_decoding(pdev, @@ -1387,6 +1506,12 @@ static void vfio_pci_remove(struct pci_dev *pdev) if (!vdev) return; + if (vdev->vf_token) { + WARN_ON(vdev->vf_token->users); + mutex_destroy(&vdev->vf_token->lock); + kfree(vdev->vf_token); + } + vfio_pci_reflck_put(vdev->reflck); vfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev); diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h index 8a2c7607d513..6da2b4748abf 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -12,6 +12,7 @@ #include #include #include +#include #ifndef VFIO_PCI_PRIVATE_H #define VFIO_PCI_PRIVATE_H @@ -84,6 +85,12 @@ struct vfio_pci_reflck { struct mutex lock; }; +struct vfio_pci_vf_token { + struct mutex lock; + uuid_t uuid; + u32 users; +}; + struct vfio_pci_device { struct pci_dev *pdev; void __iomem *barmap[PCI_STD_NUM_BARS]; @@ -122,6 +129,7 @@ struct vfio_pci_device { struct list_head dummy_resources_list; struct mutex ioeventfds_lock; struct list_head ioeventfds_list; + struct vfio_pci_vf_token *vf_token; }; #define is_intx(vdev) (vdev->irq_type == VFIO_PCI_INTX_IRQ_INDEX) From patchwork Tue Feb 11 23:05:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 04B66924 for ; Tue, 11 Feb 2020 23:06:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D84482086A for ; Tue, 11 Feb 2020 23:06:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ciJBLk8V" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727970AbgBKXGF (ORCPT ); Tue, 11 Feb 2020 18:06:05 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:33591 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727941AbgBKXGF (ORCPT ); Tue, 11 Feb 2020 18:06:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=euk0LJLXtslM2hWqkPSQesMItzrYYYlSLZ2gYZ8UrSA=; b=ciJBLk8Vn0WjxLNS3s2XpgrSwXYxwVA5DL4eRLhJtudR7O+kVe5zLjdOXRMNmnwxxivHbH vfectgfIIae7svgYRLPeym2ed+BMnxM57CzanMOa9BZo4Kw1+F5QE0bSXTJpBvUtJ6BmC0 zZtux5m4CjHeYWow6QsQistWEJE2W0U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-160-d2Gkq4rFN-yYNZBVxKuSkQ-1; Tue, 11 Feb 2020 18:05:56 -0500 X-MC-Unique: d2Gkq4rFN-yYNZBVxKuSkQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DF411800EB2; Tue, 11 Feb 2020 23:05:54 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id B902426FAA; Tue, 11 Feb 2020 23:05:51 +0000 (UTC) Subject: [PATCH 4/7] vfio: Introduce VFIO_DEVICE_FEATURE ioctl and first user From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:05:51 -0700 Message-ID: <158146235133.16827.7215789038918853214.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The VFIO_DEVICE_FEATURE ioctl is meant to be a general purpose, device agnostic ioctl for setting, retrieving, and probing device features. This implementation provides a 16-bit field for specifying a feature index, where the data porition of the ioctl is determined by the semantics for the given feature. Additional flag bits indicate the direction and nature of the operation; SET indicates user data is provided into the device feature, GET indicates the device feature is written out into user data. The PROBE flag augments determining whether the given feature is supported, and if provided, whether the given operation on the feature is supported. The first user of this ioctl is for setting the vfio-pci VF token, where the user provides a shared secret key (UUID) on a SR-IOV PF device, which users must provide when opening associated VF devices. Signed-off-by: Alex Williamson --- drivers/vfio/pci/vfio_pci.c | 52 +++++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 37 +++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 26aea9ac4863..5414744a3ead 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -1171,6 +1171,58 @@ static long vfio_pci_ioctl(void *device_data, return vfio_pci_ioeventfd(vdev, ioeventfd.offset, ioeventfd.data, count, ioeventfd.fd); + } else if (cmd == VFIO_DEVICE_FEATURE) { + struct vfio_device_feature feature; + uuid_t uuid; + + minsz = offsetofend(struct vfio_device_feature, flags); + + if (copy_from_user(&feature, (void __user *)arg, minsz)) + return -EFAULT; + + if (feature.argsz < minsz) + return -EINVAL; + + if (feature.flags & ~(VFIO_DEVICE_FEATURE_MASK | + VFIO_DEVICE_FEATURE_SET | + VFIO_DEVICE_FEATURE_GET | + VFIO_DEVICE_FEATURE_PROBE)) + return -EINVAL; + + switch (feature.flags & VFIO_DEVICE_FEATURE_MASK) { + case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN: + if (!vdev->vf_token) + return -ENOTTY; + + /* + * We do not support GET of the VF Token UUID as this + * could expose the token of the previous device user. + */ + if (feature.flags & VFIO_DEVICE_FEATURE_GET) + return -EINVAL; + + if (feature.flags & VFIO_DEVICE_FEATURE_PROBE) + return 0; + + /* Don't SET unless told to do so */ + if (!(feature.flags & VFIO_DEVICE_FEATURE_SET)) + return -EINVAL; + + if (feature.argsz < minsz + sizeof(uuid)) + return -EINVAL; + + if (copy_from_user(&uuid, (void __user *)(arg + minsz), + sizeof(uuid))) + return -EFAULT; + + mutex_lock(&vdev->vf_token->lock); + uuid_copy(&vdev->vf_token->uuid, &uuid); + mutex_unlock(&vdev->vf_token->lock); + + return 0; + default: + return -ENOTTY; + } } return -ENOTTY; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 9e843a147ead..c5cbf04ce5a7 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -707,6 +707,43 @@ struct vfio_device_ioeventfd { #define VFIO_DEVICE_IOEVENTFD _IO(VFIO_TYPE, VFIO_BASE + 16) +/** + * VFIO_DEVICE_FEATURE - _IORW(VFIO_TYPE, VFIO_BASE + 17, + * struct vfio_device_feature + * + * Get, set, or probe feature data of the device. The feature is selected + * using the FEATURE_MASK portion of the flags field. Support for a feature + * can be probed by setting both the FEATURE_MASK and PROBE bits. A probe + * may optionally include the GET and/or SET bits to determine read vs write + * access of the feature respectively. Probing a feature will return success + * if the feature is supported and all of the optionally indicated GET/SET + * methods are supported. The format of the data portion of the structure is + * specific to the given feature. The data portion is not required for + * probing. + * + * Return 0 on success, -errno on failure. + */ +struct vfio_device_feature { + __u32 argsz; + __u32 flags; +#define VFIO_DEVICE_FEATURE_MASK (0xffff) /* 16-bit feature index */ +#define VFIO_DEVICE_FEATURE_GET (1 << 16) /* Get feature into data[] */ +#define VFIO_DEVICE_FEATURE_SET (1 << 17) /* Set feature from data[] */ +#define VFIO_DEVICE_FEATURE_PROBE (1 << 18) /* Probe feature support */ + __u8 data[]; +}; + +#define VFIO_DEVICE_FEATURE _IO(VFIO_TYPE, VFIO_BASE + 17) + +/* + * Provide support for setting a PCI VF Token, which is used as a shared + * secret between PF and VF drivers. This feature may only be set on a + * PCI SR-IOV PF when SR-IOV is enabled on the PF and there are no existing + * open VFs. Data provided when setting this feature is a 16-byte array + * (__u8 b[16]), representing a UUID. + */ +#define VFIO_DEVICE_FEATURE_PCI_VF_TOKEN (0) + /* -------- API for Type1 VFIO IOMMU -------- */ /** From patchwork Tue Feb 11 23:06:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15FBB921 for ; Tue, 11 Feb 2020 23:06:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E139C2073C for ; Tue, 11 Feb 2020 23:06:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZuNfO/XK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727976AbgBKXGK (ORCPT ); Tue, 11 Feb 2020 18:06:10 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:22978 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727967AbgBKXGJ (ORCPT ); Tue, 11 Feb 2020 18:06:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6+YhMCUhJfXfX82EVb/m7gU3cgSgyfnQKFYJKPPSIiQ=; b=ZuNfO/XK0Hs3lQyad+PuACJhU8SoZoCu2wukhARgZ60TIyCz8O92agEUUSSbive2cKrfdl ThMfTtV7o8+1xk03I+mDprlozThTMz2EbWuKdXu8FRg6rw2WHeQrJ4l7PP+atmIFgcX226 TY7C0qzfdbR7gmle+2FPoVMq6Mp+2g8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-TwNvrJ1cOWGTS-GqWjSn7w-1; Tue, 11 Feb 2020 18:06:05 -0500 X-MC-Unique: TwNvrJ1cOWGTS-GqWjSn7w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7C54B13E6; Tue, 11 Feb 2020 23:06:03 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id 629B45C109; Tue, 11 Feb 2020 23:06:00 +0000 (UTC) Subject: [PATCH 5/7] vfio/pci: Add sriov_configure support From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:06:00 -0700 Message-ID: <158146235998.16827.12675446739944814621.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org With the VF Token interface we can now expect that a vfio userspace driver must be in collaboration with the PF driver, an unwitting userspace driver will not be able to get past the GET_DEVICE_FD step in accessing the device. We can now move on to actually allowing SR-IOV to be enabled by vfio-pci on the PF. Support for this is not enabled by default in this commit, but it does provide a module option for this to be enabled (enable_sriov=1). Enabling VFs is rather straightforward, except we don't want to risk that a VF might get autoprobed and bound to other drivers, so a bus notifier is used to "capture" VFs to vfio-pci using the driver_override support. We assume any later action to bind the device to other drivers is condoned by the system admin and allow it with a log warning. vfio-pci will disable SR-IOV on a PF before releasing the device, allowing a VF driver to be assured other drivers cannot take over the PF and that any other userspace driver must know the shared VF token. This support also does not provide a mechanism for the PF userspace driver itself to manipulate SR-IOV. With this patch SR-IOV can only be enabled via the host sysfs interface and the PF driver user cannot create and remove VF. Signed-off-by: Alex Williamson --- drivers/vfio/pci/vfio_pci.c | 106 +++++++++++++++++++++++++++++++---- drivers/vfio/pci/vfio_pci_private.h | 2 + 2 files changed, 97 insertions(+), 11 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 5414744a3ead..cce6127a9c56 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -54,6 +54,12 @@ module_param(disable_idle_d3, bool, S_IRUGO | S_IWUSR); MODULE_PARM_DESC(disable_idle_d3, "Disable using the PCI D3 low power state for idle, unused devices"); +static bool enable_sriov; +#ifdef CONFIG_PCI_IOV +module_param(enable_sriov, bool, 0644); +MODULE_PARM_DESC(enable_sriov, "Enable support for SR-IOV configuration"); +#endif + static inline bool vfio_vga_disabled(void) { #ifdef CONFIG_VFIO_PCI_VGA @@ -1457,6 +1463,35 @@ static const struct vfio_device_ops vfio_pci_ops = { static int vfio_pci_reflck_attach(struct vfio_pci_device *vdev); static void vfio_pci_reflck_put(struct vfio_pci_reflck *reflck); +static struct pci_driver vfio_pci_driver; + +static int vfio_pci_bus_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct vfio_pci_device *vdev = container_of(nb, + struct vfio_pci_device, nb); + struct device *dev = data; + struct pci_dev *pdev = to_pci_dev(dev); + struct pci_dev *physfn = pci_physfn(pdev); + + if (action == BUS_NOTIFY_ADD_DEVICE && + pdev->is_virtfn && physfn == vdev->pdev) { + pci_info(vdev->pdev, "Captured SR-IOV VF %s driver_override\n", + pci_name(pdev)); + pdev->driver_override = kasprintf(GFP_KERNEL, "%s", + vfio_pci_ops.name); + } else if (action == BUS_NOTIFY_BOUND_DRIVER && + pdev->is_virtfn && physfn == vdev->pdev) { + struct pci_driver *drv = pci_dev_driver(pdev); + + if (drv && drv != &vfio_pci_driver) + pci_warn(vdev->pdev, + "VF %s bound to driver %s while PF bound to vfio-pci\n", + pci_name(pdev), drv->name); + } + + return 0; +} static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { @@ -1468,12 +1503,12 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) return -EINVAL; /* - * Prevent binding to PFs with VFs enabled, this too easily allows - * userspace instance with VFs and PFs from the same device, which - * cannot work. Disabling SR-IOV here would initiate removing the - * VFs, which would unbind the driver, which is prone to blocking - * if that VF is also in use by vfio-pci. Just reject these PFs - * and let the user sort it out. + * Prevent binding to PFs with VFs enabled, the VFs might be in use + * by the host or other users. We cannot capture the VFs if they + * already exist, nor can we track VF users. Disabling SR-IOV here + * would initiate removing the VFs, which would unbind the driver, + * which is prone to blocking if that VF is also in use by vfio-pci. + * Just reject these PFs and let the user sort it out. */ if (pci_num_vf(pdev)) { pci_warn(pdev, "Cannot bind to PF with SR-IOV enabled\n"); @@ -1521,6 +1556,18 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) kfree(vdev); return -ENOMEM; } + + vdev->nb.notifier_call = vfio_pci_bus_notifier; + ret = bus_register_notifier(&pci_bus_type, &vdev->nb); + if (ret) { + kfree(vdev->vf_token); + vfio_pci_reflck_put(vdev->reflck); + vfio_del_group_dev(&pdev->dev); + vfio_iommu_group_put(group, &pdev->dev); + kfree(vdev); + return ret; + } + mutex_init(&vdev->vf_token->lock); uuid_gen(&vdev->vf_token->uuid); } @@ -1554,6 +1601,8 @@ static void vfio_pci_remove(struct pci_dev *pdev) { struct vfio_pci_device *vdev; + pci_disable_sriov(pdev); + vdev = vfio_del_group_dev(&pdev->dev); if (!vdev) return; @@ -1564,6 +1613,9 @@ static void vfio_pci_remove(struct pci_dev *pdev) kfree(vdev->vf_token); } + if (vdev->nb.notifier_call) + bus_unregister_notifier(&pci_bus_type, &vdev->nb); + vfio_pci_reflck_put(vdev->reflck); vfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev); @@ -1612,16 +1664,48 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, return PCI_ERS_RESULT_CAN_RECOVER; } +static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) +{ + struct vfio_pci_device *vdev; + struct vfio_device *device; + int ret = 0; + + might_sleep(); + + if (!enable_sriov) + return -ENOENT; + + device = vfio_device_get_from_dev(&pdev->dev); + if (!device) + return -ENODEV; + + vdev = vfio_device_data(device); + if (!vdev) { + vfio_device_put(device); + return -ENODEV; + } + + if (nr_virtfn == 0) + pci_disable_sriov(pdev); + else + ret = pci_enable_sriov(pdev, nr_virtfn); + + vfio_device_put(device); + + return ret < 0 ? ret : nr_virtfn; +} + static const struct pci_error_handlers vfio_err_handlers = { .error_detected = vfio_pci_aer_err_detected, }; static struct pci_driver vfio_pci_driver = { - .name = "vfio-pci", - .id_table = NULL, /* only dynamic ids */ - .probe = vfio_pci_probe, - .remove = vfio_pci_remove, - .err_handler = &vfio_err_handlers, + .name = "vfio-pci", + .id_table = NULL, /* only dynamic ids */ + .probe = vfio_pci_probe, + .remove = vfio_pci_remove, + .sriov_configure = vfio_pci_sriov_configure, + .err_handler = &vfio_err_handlers, }; static DEFINE_MUTEX(reflck_lock); diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h index 6da2b4748abf..b651a007334d 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -13,6 +13,7 @@ #include #include #include +#include #ifndef VFIO_PCI_PRIVATE_H #define VFIO_PCI_PRIVATE_H @@ -130,6 +131,7 @@ struct vfio_pci_device { struct mutex ioeventfds_lock; struct list_head ioeventfds_list; struct vfio_pci_vf_token *vf_token; + struct notifier_block nb; }; #define is_intx(vdev) (vdev->irq_type == VFIO_PCI_INTX_IRQ_INDEX) From patchwork Tue Feb 11 23:06:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EB4C924 for ; Tue, 11 Feb 2020 23:06:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 74EF62082F for ; Tue, 11 Feb 2020 23:06:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="f83juu7L" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727996AbgBKXGT (ORCPT ); Tue, 11 Feb 2020 18:06:19 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:43628 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727949AbgBKXGT (ORCPT ); Tue, 11 Feb 2020 18:06:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0+ldBvY6E5lfxm3sCVRYefAuOGmBMo0z2Mi+OdowHIs=; b=f83juu7L/KzR2cnr0tVQtryXEjGBbEcLgZS16hAs6X6onHE8TmaX45SZauZvyrtmkvbiF3 AqNsEkgeXbg5WxweAM+NVisOodwhKYjdO8TXYfRp2eH0ds2sDdUMQF1ajx310SNejbLSWT E/ejNjK2C9PL/dFyJ0DsSx/AwH4lKf0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-197-LwroSNPcNq2DppkJu5W4dQ-1; Tue, 11 Feb 2020 18:06:13 -0500 X-MC-Unique: LwroSNPcNq2DppkJu5W4dQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EF5E1800EB2; Tue, 11 Feb 2020 23:06:11 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id F12DD5D9CA; Tue, 11 Feb 2020 23:06:08 +0000 (UTC) Subject: [PATCH 6/7] vfio/pci: Remove dev_fmt definition From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:06:08 -0700 Message-ID: <158146236856.16827.12230312293359315520.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org It currently results in messages like: "vfio-pci 0000:03:00.0: vfio_pci: ..." Which is quite a bit redundant. Reviewed-by: Cornelia Huck Signed-off-by: Alex Williamson --- drivers/vfio/pci/vfio_pci.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index cce6127a9c56..a88b45ce1cc7 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -9,7 +9,6 @@ */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt -#define dev_fmt pr_fmt #include #include From patchwork Tue Feb 11 23:06:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 11377123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33BC2924 for ; Tue, 11 Feb 2020 23:06:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 132FE20714 for ; Tue, 11 Feb 2020 23:06:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y8t01xnv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728010AbgBKXG1 (ORCPT ); Tue, 11 Feb 2020 18:06:27 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:28472 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728003AbgBKXG0 (ORCPT ); Tue, 11 Feb 2020 18:06:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581462386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zl42YFae7shP7qDZsLmjotiywH2QivsM2n43EHp8HXI=; b=Y8t01xnvEEK4qO2oFYdqYX9AH2GbFaaW9rGGLkUhwwiE4iqJ72yoMbHQN1k1uNzRPwmpnq GTRqTCcXUrxVdy9qE7JvojFc9P+0Z2jCiZG3+QlBKTYjD/pzHwAa550Y2b8hN58Cpb63bH kVe2reOhQsQAx3e7xOIh9+Cp03ye60g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-386-63tPqKG0PkeYD-iktDgy0w-1; Tue, 11 Feb 2020 18:06:22 -0500 X-MC-Unique: 63tPqKG0PkeYD-iktDgy0w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9D812800D4C; Tue, 11 Feb 2020 23:06:20 +0000 (UTC) Received: from gimli.home (ovpn-116-28.phx2.redhat.com [10.3.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6ED146E40A; Tue, 11 Feb 2020 23:06:17 +0000 (UTC) Subject: [PATCH 7/7] vfio/pci: Cleanup .probe() exit paths From: Alex Williamson To: kvm@vger.kernel.org Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, dev@dpdk.org, mtosatti@redhat.com, thomas@monjalon.net, bluca@debian.org, jerinjacobk@gmail.com, bruce.richardson@intel.com, cohuck@redhat.com Date: Tue, 11 Feb 2020 16:06:17 -0700 Message-ID: <158146237704.16827.13577826727613048764.stgit@gimli.home> In-Reply-To: <158145472604.16827.15751375540102298130.stgit@gimli.home> References: <158145472604.16827.15751375540102298130.stgit@gimli.home> User-Agent: StGit/0.19-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The cleanup is getting a tad long. Signed-off-by: Alex Williamson --- drivers/vfio/pci/vfio_pci.c | 54 ++++++++++++++++++++----------------------- 1 file changed, 25 insertions(+), 29 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index a88b45ce1cc7..fff49dfc742a 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -1520,8 +1520,8 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) vdev = kzalloc(sizeof(*vdev), GFP_KERNEL); if (!vdev) { - vfio_iommu_group_put(group, &pdev->dev); - return -ENOMEM; + ret = -ENOMEM; + goto out_group_put; } vdev->pdev = pdev; @@ -1532,43 +1532,27 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) INIT_LIST_HEAD(&vdev->ioeventfds_list); ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); - if (ret) { - vfio_iommu_group_put(group, &pdev->dev); - kfree(vdev); - return ret; - } + if (ret) + goto out_free; ret = vfio_pci_reflck_attach(vdev); - if (ret) { - vfio_del_group_dev(&pdev->dev); - vfio_iommu_group_put(group, &pdev->dev); - kfree(vdev); - return ret; - } + if (ret) + goto out_del_group_dev; if (pdev->is_physfn) { vdev->vf_token = kzalloc(sizeof(*vdev->vf_token), GFP_KERNEL); if (!vdev->vf_token) { - vfio_pci_reflck_put(vdev->reflck); - vfio_del_group_dev(&pdev->dev); - vfio_iommu_group_put(group, &pdev->dev); - kfree(vdev); - return -ENOMEM; - } - - vdev->nb.notifier_call = vfio_pci_bus_notifier; - ret = bus_register_notifier(&pci_bus_type, &vdev->nb); - if (ret) { - kfree(vdev->vf_token); - vfio_pci_reflck_put(vdev->reflck); - vfio_del_group_dev(&pdev->dev); - vfio_iommu_group_put(group, &pdev->dev); - kfree(vdev); - return ret; + ret = -ENOMEM; + goto out_reflck; } mutex_init(&vdev->vf_token->lock); uuid_gen(&vdev->vf_token->uuid); + + vdev->nb.notifier_call = vfio_pci_bus_notifier; + ret = bus_register_notifier(&pci_bus_type, &vdev->nb); + if (ret) + goto out_vf_token; } if (vfio_pci_is_vga(pdev)) { @@ -1594,6 +1578,18 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) } return ret; + +out_vf_token: + kfree(vdev->vf_token); +out_reflck: + vfio_pci_reflck_put(vdev->reflck); +out_del_group_dev: + vfio_del_group_dev(&pdev->dev); +out_free: + kfree(vdev); +out_group_put: + vfio_iommu_group_put(group, &pdev->dev); + return ret; } static void vfio_pci_remove(struct pci_dev *pdev)