From patchwork Fri Mar 2 23:44:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10255861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 838E960211 for ; Fri, 2 Mar 2018 23:44:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74C76285E4 for ; Fri, 2 Mar 2018 23:44:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 69841285F2; Fri, 2 Mar 2018 23:44:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6D1D285E4 for ; Fri, 2 Mar 2018 23:44:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934956AbeCBXoc (ORCPT ); Fri, 2 Mar 2018 18:44:32 -0500 Received: from mail-it0-f65.google.com ([209.85.214.65]:50435 "EHLO mail-it0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934783AbeCBXo2 (ORCPT ); Fri, 2 Mar 2018 18:44:28 -0500 Received: by mail-it0-f65.google.com with SMTP id d13so3514910itf.0; Fri, 02 Mar 2018 15:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=J7WDKdx0lEV7Lg9lSGwPGakFlBzIRxTXbjT30/D18yc=; b=nqN9vngDBbJ4kqt44vNOLTTl57jB58iToYNXVSM2v15WJe/JoXHegQP4Hl7lcGAAjD Bpimi6kub2Ia54XvvJ0XEIeic4ry1HahQKebcr7rhcAzIyUQCUOrK/K9t50Lu76kfS9h lJCcPmiz8iPc96j0dRHdOE0xqIazh8E+Sf1BNimb9CIwi6EzjTl5oXwelrKrUEVA4k3Q 79XDcMcUda99pSXUnT5G5CBDcdK6XGcnrK3jX+jNu9mQutUHv1aqrPJLEi1YNqC1psEV 9sZFpOIDSbzPika1F4K0UCkILLy4P8Q8JMyf8t0UhPwpkjDhXWhedoj2IEOEW5pX7i4q N1zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=J7WDKdx0lEV7Lg9lSGwPGakFlBzIRxTXbjT30/D18yc=; b=UdERv93nCLYwDo6nlXZ1/MtjQMRxXbWBYkyPl98+Hm/W3ya3+Qfe8orb3kQOF7/xk5 yUgIzaeD6fr/qRDP7DFV+sDIpJ1SUPx8j8d+81tr8xS3+0YCWBfD8jk/DR7n+RjniPfc qsSxzhD4Nb8lc3vJjYZdDmAu29g8V0j1s0jUIXI959P9jPZmUJMTMoPQlCSV+bLrBI29 vAEJHZX4AddYUdaxuOUqVvUV5CMILbxTr8abzcdEdajJuqtSXwbsQA8hGiRK5C89eqMI rUt1Xh3zk8gMS4X0WpWYageAPeHXWedsZsR6kXJwaZ3uZRafEZlPM4k9Lfc4/Bs1aMCf IXBQ== X-Gm-Message-State: AElRT7EraNA+jbIafAh4PQX1vomBzlMe4PjD68b794SQ/xDjaW861JtT aEyvZyZfFhDs5rQqw0Jeor0= X-Google-Smtp-Source: AG47ELtJJpylwuqsiQBUhuXymIT3AVbqzOtjSzeapfabEkwoqMwe9Rqc+6dylAZndYRhjH9UDjDl7w== X-Received: by 10.36.189.129 with SMTP id x123mr5053212ite.13.1520034267593; Fri, 02 Mar 2018 15:44:27 -0800 (PST) Received: from localhost.localdomain ([2001:470:b:9c3:9e5c:8eff:fe4f:f2d0]) by smtp.gmail.com with ESMTPSA id w132sm5127218iow.51.2018.03.02.15.44.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Mar 2018 15:44:27 -0800 (PST) Subject: [PATCH 1/3] pci-iov: Add support for unmanaged SR-IOV From: Alexander Duyck To: bhelgaas@google.com, alexander.h.duyck@intel.com, linux-pci@vger.kernel.org Cc: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, netdev@vger.kernel.org, dan.daly@intel.com, linux-kernel@vger.kernel.org, mheyne@amazon.de, liang-min.wang@intel.com, mark.d.rustad@intel.com, dwmw2@infradead.org, dwmw@amazon.co.uk Date: Fri, 02 Mar 2018 15:44:25 -0800 Message-ID: <20180302234412.3337.71321.stgit@localhost.localdomain> In-Reply-To: <20180302234218.3337.16486.stgit@localhost.localdomain> References: <20180302234218.3337.16486.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Alexander Duyck This patch is meant to add some basic functionality to support for SR-IOV on devices when the VFs are not managed by the kernel. The functions provided here can be used by drivers such as vfio-pci and virtio to enable SR-IOV on devices that are either managed by userspace, or by some sort of firmware entity respectively. A new sysfs value called sriov_unmanaged_autoprobe has been added. This value is used as the drivers_autoprobe setting of the VFs when they are being managed by an external entity such as userspace or device firmware instead of being managed by the kernel. One side effect of this change is that the sriov_drivers_autoprobe and sriov_unmanaged_autoprobe will only apply their updates when SR-IOV is disabled. Attempts to update them when SR-IOV is in use will only update the local value and will not update sriov->autoprobe. Signed-off-by: Alexander Duyck --- Documentation/ABI/testing/sysfs-bus-pci | 17 ++++++++++++++ drivers/pci/iov.c | 37 +++++++++++++++++++++++++++++++ drivers/pci/pci-driver.c | 2 +- drivers/pci/pci-sysfs.c | 29 ++++++++++++++++++++++++ drivers/pci/pci.h | 4 +++ include/linux/pci.h | 1 + 6 files changed, 88 insertions(+), 2 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci index 44d4b2be92fd..ff0b6c19cb1a 100644 --- a/Documentation/ABI/testing/sysfs-bus-pci +++ b/Documentation/ABI/testing/sysfs-bus-pci @@ -323,3 +323,20 @@ Description: This is similar to /sys/bus/pci/drivers_autoprobe, but affects only the VFs associated with a specific PF. + +What: /sys/bus/pci/devices/.../sriov_unmanaged_autoprobe +Date: March 2018 +Contact: Alexander Duyck +Description: + This file is associated with the PF of a device that + supports SR-IOV. It determines whether newly-enabled VFs + are immediately bound to a driver when the PF driver does + not manage the VFs itself. It initially contains 0, which + means the kernel will not automatically bind VFs to a driver. + If an application writes 1 to the file before enabling VFs, + the kernel will bind VFs to a compatible driver immediately + after they are enabled. + + This overrides /sys/bus/pci/devices/.../sriov_drivers_autoprobe + when a PF driver is not present to manage a device, or the PF + driver does not provide functionality to support SR-IOV. diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 677924ae0350..3dcec1fa86bd 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -446,6 +446,7 @@ static int sriov_init(struct pci_dev *dev, int pos) pci_read_config_word(dev, pos + PCI_SRIOV_VF_DID, &iov->vf_device); iov->pgsz = pgsz; iov->self = dev; + iov->autoprobe = true; iov->drivers_autoprobe = true; pci_read_config_dword(dev, pos + PCI_SRIOV_CAP, &iov->cap); pci_read_config_byte(dev, pos + PCI_SRIOV_FUNC_LINK, &iov->link); @@ -683,6 +684,9 @@ int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn) if (!dev->is_physfn) return -ENOSYS; + /* Update autoprobe setting to reflect managed device */ + dev->sriov->autoprobe = dev->sriov->drivers_autoprobe; + return sriov_enable(dev, nr_virtfn); } EXPORT_SYMBOL_GPL(pci_enable_sriov); @@ -807,3 +811,36 @@ int pci_sriov_get_totalvfs(struct pci_dev *dev) return dev->sriov->total_VFs; } EXPORT_SYMBOL_GPL(pci_sriov_get_totalvfs); + +/** + * pci_sriov_configure_unmanaged - helper to configure unmanaged SR-IOV + * @dev: the PCI device + * @nr_virtfn: number of virtual functions to enable, 0 to disable + * + * Used to provide generic enable/disable SR-IOV option for devices + * that do not manage the VFs generated by their driver, or have no + * driver present. + */ +int pci_sriov_configure_unmanaged(struct pci_dev *dev, int nr_virtfn) +{ + int err; + + might_sleep(); + + if (!dev->is_physfn) + return -ENODEV; + + if (!nr_virtfn) { + sriov_disable(dev); + + return 0; + } + + /* Update autoprobe setting to reflect unmanaged device */ + dev->sriov->autoprobe = dev->sriov->unmanaged_autoprobe; + + err = sriov_enable(dev, nr_virtfn); + + return err ? err : nr_virtfn; +} +EXPORT_SYMBOL_GPL(pci_sriov_configure_unmanaged); diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 3bed6beda051..2cc68dff6130 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -398,7 +398,7 @@ void __weak pcibios_free_irq(struct pci_dev *dev) #ifdef CONFIG_PCI_IOV static inline bool pci_device_can_probe(struct pci_dev *pdev) { - return (!pdev->is_virtfn || pdev->physfn->sriov->drivers_autoprobe); + return (!pdev->is_virtfn || pdev->physfn->sriov->autoprobe); } #else static inline bool pci_device_can_probe(struct pci_dev *pdev) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index eb6bee8724cc..6f78fa73e317 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -710,6 +710,30 @@ static ssize_t sriov_drivers_autoprobe_store(struct device *dev, return count; } +static ssize_t sriov_unmanaged_autoprobe_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct pci_dev *pdev = to_pci_dev(dev); + + return sprintf(buf, "%u\n", pdev->sriov->unmanaged_autoprobe); +} + +static ssize_t sriov_unmanaged_autoprobe_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev = to_pci_dev(dev); + bool unmanaged_autoprobe; + + if (kstrtobool(buf, &unmanaged_autoprobe) < 0) + return -EINVAL; + + pdev->sriov->unmanaged_autoprobe = unmanaged_autoprobe; + + return count; +} + static struct device_attribute sriov_totalvfs_attr = __ATTR_RO(sriov_totalvfs); static struct device_attribute sriov_numvfs_attr = __ATTR(sriov_numvfs, (S_IRUGO|S_IWUSR|S_IWGRP), @@ -720,6 +744,10 @@ static ssize_t sriov_drivers_autoprobe_store(struct device *dev, static struct device_attribute sriov_drivers_autoprobe_attr = __ATTR(sriov_drivers_autoprobe, (S_IRUGO|S_IWUSR|S_IWGRP), sriov_drivers_autoprobe_show, sriov_drivers_autoprobe_store); +static struct device_attribute sriov_unmanaged_autoprobe_attr = + __ATTR(sriov_unmanaged_autoprobe, (S_IRUGO|S_IWUSR|S_IWGRP), + sriov_unmanaged_autoprobe_show, + sriov_unmanaged_autoprobe_store); #endif /* CONFIG_PCI_IOV */ static ssize_t driver_override_store(struct device *dev, @@ -1789,6 +1817,7 @@ static umode_t pcie_dev_attrs_are_visible(struct kobject *kobj, &sriov_stride_attr.attr, &sriov_vf_device_attr.attr, &sriov_drivers_autoprobe_attr.attr, + &sriov_unmanaged_autoprobe_attr.attr, NULL, }; diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index fcd81911b127..b5f8b034f02d 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -272,7 +272,9 @@ struct pci_sriov { struct pci_dev *dev; /* Lowest numbered PF */ struct pci_dev *self; /* This PF */ resource_size_t barsz[PCI_SRIOV_NUM_BARS]; /* VF BAR size */ - bool drivers_autoprobe; /* Auto probing of VFs by driver */ + bool autoprobe; /* Auto probing of VFs by VF driver */ + bool drivers_autoprobe; /* "" managed by PF driver */ + bool unmanaged_autoprobe; /* "" unmanaged by kernel */ }; /* pci_dev priv_flags */ diff --git a/include/linux/pci.h b/include/linux/pci.h index 024a1beda008..553860a08131 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1953,6 +1953,7 @@ static inline void pci_mmcfg_late_init(void) { } int pci_vfs_assigned(struct pci_dev *dev); int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs); int pci_sriov_get_totalvfs(struct pci_dev *dev); +int pci_sriov_configure_unmanaged(struct pci_dev *dev, int nr_virtfn); resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno); void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe); #else