From patchwork Fri May 8 03:53:27 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Yang X-Patchwork-Id: 6362221 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 09A0C9F1C2 for ; Fri, 8 May 2015 03:58:17 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0D67E203C2 for ; Fri, 8 May 2015 03:58:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A9CE9203C1 for ; Fri, 8 May 2015 03:58:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751071AbbEHD6M (ORCPT ); Thu, 7 May 2015 23:58:12 -0400 Received: from e23smtp01.au.ibm.com ([202.81.31.143]:42687 "EHLO e23smtp01.au.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750955AbbEHD6L (ORCPT ); Thu, 7 May 2015 23:58:11 -0400 Received: from /spool/local by e23smtp01.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 8 May 2015 13:58:09 +1000 Received: from d23relay06.au.ibm.com (202.81.31.225) by e23smtp01.au.ibm.com (202.81.31.207) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 8 May 2015 13:58:08 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay06.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t483sL8113566140 for ; Fri, 8 May 2015 13:54:29 +1000 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t483ruwh006343 for ; Fri, 8 May 2015 13:53:57 +1000 Received: from localhost ([9.123.251.135]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t483rum3006118; Fri, 8 May 2015 13:53:56 +1000 From: Wei Yang To: bhelgaas@google.com, gwshan@linux.vnet.ibm.com Cc: linux-pci@vger.kernel.org, Wei Yang Subject: [PATCH V2] pci/iov: fix resource leak on destroying VF Date: Fri, 8 May 2015 11:53:27 +0800 Message-Id: <1431057207-30775-1-git-send-email-weiyang@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1428655984-26903-1-git-send-email-weiyang@linux.vnet.ibm.com> References: <1428655984-26903-1-git-send-email-weiyang@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 15050803-1618-0000-0000-0000020CF050 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Beside pci_dev resources, VF has other resources between its PF, like refcount to PF, sysfs link between them and the virtual bus. When a VF is released in virtfn_remove(), those resources are released properly. But in the hotplug case, they are not. In hotplug case, VFs are removed by pci_stop_and_remove_bus_device() instead of virtfn_remove(). This leads to some leak for resources. This patch moves the resource release to pci_destroy_dev() to make sure those resources are released when VF is destroyed. Signed-off-by: Wei Yang --- drivers/pci/iov.c | 42 +++++++++++++++++++++++------------------- drivers/pci/pci.h | 10 ++++++++++ drivers/pci/remove.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 64 insertions(+), 19 deletions(-) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index ee0ebff..47daf2f 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -17,8 +17,6 @@ #include #include "pci.h" -#define VIRTFN_ID_LEN 16 - int pci_iov_virtfn_bus(struct pci_dev *dev, int vf_id) { if (!dev->is_physfn) @@ -94,7 +92,7 @@ static struct pci_bus *virtfn_add_bus(struct pci_bus *bus, int busnr) return child; } -static void virtfn_remove_bus(struct pci_bus *physbus, struct pci_bus *virtbus) +void virtfn_remove_bus(struct pci_bus *physbus, struct pci_bus *virtbus) { if (physbus != virtbus && list_empty(&virtbus->devices)) pci_remove_bus(virtbus); @@ -185,9 +183,7 @@ failed: static void virtfn_remove(struct pci_dev *dev, int id, int reset) { - char buf[VIRTFN_ID_LEN]; struct pci_dev *virtfn; - struct pci_sriov *iov = dev->sriov; virtfn = pci_get_domain_bus_and_slot(pci_domain_nr(dev->bus), pci_iov_virtfn_bus(dev, id), @@ -200,24 +196,10 @@ static void virtfn_remove(struct pci_dev *dev, int id, int reset) __pci_reset_function(virtfn); } - sprintf(buf, "virtfn%u", id); - sysfs_remove_link(&dev->dev.kobj, buf); - /* - * pci_stop_dev() could have been called for this virtfn already, - * so the directory for the virtfn may have been removed before. - * Double check to avoid spurious sysfs warnings. - */ - if (virtfn->dev.kobj.sd) - sysfs_remove_link(&virtfn->dev.kobj, "physfn"); - - mutex_lock(&iov->dev->sriov->lock); pci_stop_and_remove_bus_device(virtfn); - virtfn_remove_bus(dev->bus, virtfn->bus); - mutex_unlock(&iov->dev->sriov->lock); /* balance pci_get_domain_bus_and_slot() */ pci_dev_put(virtfn); - pci_dev_put(dev); } int __weak pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs) @@ -760,3 +742,25 @@ int pci_sriov_get_totalvfs(struct pci_dev *dev) return dev->sriov->total_VFs; } EXPORT_SYMBOL_GPL(pci_sriov_get_totalvfs); + +void pci_sriov_lock(struct pci_dev *dev) +{ + struct pci_sriov *iov; + + if (!dev->is_physfn) + return; + + iov = dev->sriov; + mutex_lock(&iov->dev->sriov->lock); +} + +void pci_sriov_unlock(struct pci_dev *dev) +{ + struct pci_sriov *iov; + + if (!dev->is_physfn) + return; + + iov = dev->sriov; + mutex_unlock(&iov->dev->sriov->lock); +} diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 9bd762c2..a0323a2 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -260,6 +260,12 @@ static inline void pci_restore_ats_state(struct pci_dev *dev) #endif /* CONFIG_PCI_ATS */ #ifdef CONFIG_PCI_IOV + +#define VIRTFN_ID_LEN 16 +void pci_sriov_lock(struct pci_dev *dev); +void pci_sriov_unlock(struct pci_dev *dev); +void virtfn_remove_bus(struct pci_bus *physbus, struct pci_bus *virtbus); + int pci_iov_init(struct pci_dev *dev); void pci_iov_release(struct pci_dev *dev); int pci_iov_resource_bar(struct pci_dev *dev, int resno); @@ -268,6 +274,10 @@ void pci_restore_iov_state(struct pci_dev *dev); int pci_iov_bus_range(struct pci_bus *bus); #else +static inline void pci_sriov_lock(struct pci_dev *dev) { } +static inline void pci_sriov_unlock(struct pci_dev *dev) { } +static inline void virtfn_remove_bus(struct pci_bus *physbus, + struct pci_bus *virtbus) { } static inline int pci_iov_init(struct pci_dev *dev) { return -ENODEV; diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c index 8a280e9..f2a07bf 100644 --- a/drivers/pci/remove.c +++ b/drivers/pci/remove.c @@ -41,6 +41,37 @@ static void pci_destroy_dev(struct pci_dev *dev) list_del(&dev->bus_list); up_write(&pci_bus_sem); +#ifdef CONFIG_PCI_IOV + if (dev->is_virtfn) { + char buf[VIRTFN_ID_LEN]; + int id; + struct pci_dev *pdev = dev->physfn; + + for (id = 0; id < pci_sriov_get_totalvfs(pdev); id++) { + if ((dev->bus->number == pci_iov_virtfn_bus(pdev, id)) + && (dev->devfn == pci_iov_virtfn_devfn(pdev, id))) { + sprintf(buf, "virtfn%u", id); + sysfs_remove_link(&pdev->dev.kobj, buf); + break; + } + } + /* + * pci_stop_dev() could have been called for this virtfn + * already, so the directory for the virtfn may have been + * removed before. Double check to avoid spurious sysfs + * warnings. + */ + if (dev->dev.kobj.sd) + sysfs_remove_link(&dev->dev.kobj, "physfn"); + + pci_sriov_lock(pdev); + virtfn_remove_bus(pdev->bus, dev->bus); + pci_sriov_unlock(pdev); + + pci_dev_put(dev->physfn); + } +#endif + pci_free_resources(dev); put_device(&dev->dev); }