From patchwork Fri Nov 21 22:17:51 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konrad Rzeszutek Wilk X-Patchwork-Id: 5358401 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 173D49F387 for ; Fri, 21 Nov 2014 22:19:28 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 410742021B for ; Fri, 21 Nov 2014 22:19:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3CF10201E4 for ; Fri, 21 Nov 2014 22:19:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752577AbaKUWSi (ORCPT ); Fri, 21 Nov 2014 17:18:38 -0500 Received: from aserp1040.oracle.com ([141.146.126.69]:34047 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752437AbaKUWSD (ORCPT ); Fri, 21 Nov 2014 17:18:03 -0500 Received: from acsinet22.oracle.com (acsinet22.oracle.com [141.146.126.238]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id sALMHvp2004393 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 21 Nov 2014 22:17:57 GMT Received: from aserz7021.oracle.com (aserz7021.oracle.com [141.146.126.230]) by acsinet22.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id sALMHuw2023119 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Fri, 21 Nov 2014 22:17:57 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by aserz7021.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id sALMHuQ1019423; Fri, 21 Nov 2014 22:17:56 GMT Received: from laptop.dumpdata.com (/50.195.21.189) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 21 Nov 2014 14:17:56 -0800 Received: by laptop.dumpdata.com (Postfix, from userid 1000) id CC3F8118C03; Fri, 21 Nov 2014 17:17:52 -0500 (EST) From: Konrad Rzeszutek Wilk To: xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, bhelgaas@google.com, linux@eikelenboom.it Cc: Konrad Rzeszutek Wilk Subject: [PATCH v4 7/7] xen/pciback: Restore configuration space when detaching from a guest. Date: Fri, 21 Nov 2014 17:17:51 -0500 Message-Id: <1416608271-18931-8-git-send-email-konrad.wilk@oracle.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1416608271-18931-1-git-send-email-konrad.wilk@oracle.com> References: <1416608271-18931-1-git-send-email-konrad.wilk@oracle.com> X-Source-IP: acsinet22.oracle.com [141.146.126.238] Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The commit "xen/pciback: Don't deadlock when unbinding." was using the version of pci_reset_function which would lock the device lock. That is no good as we can dead-lock. As such we swapped to using the lock-less version and requiring that the callers of 'pcistub_put_pci_dev' take the device lock. And as such this bug got exposed. Using the lock-less version is OK, except that we tried to use 'pci_restore_state' after the lock-less version of __pci_reset_function_locked - which won't work as 'state_saved' is set to false. Said 'state_saved' is a toggle boolean that is to be used by the sequence of a) pci_save_state/pci_restore_state or b) pci_load_and_free_saved_state/pci_restore_state. We don't want to use a) as the guest might have messed up the PCI configuration space and we want it to revert to the state when the PCI device was binded to us. Therefore we pick b) to restore the configuration space. We restore from our 'golden' version of PCI configuration space, when an: - Device is unbinded from pciback - Device is detached from a guest. Reported-by: Sander Eikelenboom Signed-off-by: Konrad Rzeszutek Wilk --- drivers/xen/xen-pciback/pci_stub.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c index 843a2ba..eb8b58e 100644 --- a/drivers/xen/xen-pciback/pci_stub.c +++ b/drivers/xen/xen-pciback/pci_stub.c @@ -105,7 +105,7 @@ static void pcistub_device_release(struct kref *kref) */ __pci_reset_function_locked(dev); if (pci_load_and_free_saved_state(dev, &dev_data->pci_saved_state)) - dev_dbg(&dev->dev, "Could not reload PCI state\n"); + dev_info(&dev->dev, "Could not reload PCI state\n"); else pci_restore_state(dev); @@ -257,6 +257,8 @@ void pcistub_put_pci_dev(struct pci_dev *dev) { struct pcistub_device *psdev, *found_psdev = NULL; unsigned long flags; + struct xen_pcibk_dev_data *dev_data; + int ret; spin_lock_irqsave(&pcistub_devices_lock, flags); @@ -279,9 +281,19 @@ void pcistub_put_pci_dev(struct pci_dev *dev) * (so it's ready for the next domain) */ device_lock_assert(&dev->dev); - __pci_reset_function_locked(dev); - pci_restore_state(dev); - + dev_data = pci_get_drvdata(dev); + ret = pci_load_saved_state(dev, dev_data->pci_saved_state); + if (ret < 0) + dev_warn(&dev->dev, "Could not reload PCI state\n"); + else { + __pci_reset_function_locked(dev); + /* + * The usual sequence is pci_save_state & pci_restore_state + * but the guest might have messed the configuration space up. + * Use the initial version (when device was bound to us). + */ + pci_restore_state(dev); + } /* This disables the device. */ xen_pcibk_reset_device(dev);