From patchwork Mon Mar 11 13:31:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847451 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2510018A6 for ; Mon, 11 Mar 2019 13:32:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0EC42941C for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF2DB2945F; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31CC42941C for ; Mon, 11 Mar 2019 13:31:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727581AbfCKNbe (ORCPT ); Mon, 11 Mar 2019 09:31:34 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50916 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727388AbfCKNbe (ORCPT ); Mon, 11 Mar 2019 09:31:34 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9D7CF419DA; Mon, 11 Mar 2019 13:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311091; x=1554125492; bh=A5ubc+8NetLWy7BN9IG3hEbaDHSG7hHtYX/ dxkrS/A4=; b=IB/CsufXEAgKUwkEXbUyAL/Nu4TAlf7imHHVfmEvHMYhlf6lc+G 1Utqi53yvyxgidRUw8atJ7X1rgRPSJ7DKtz3JlGklB5D8FMXIxXjgSN3t8Ljq/8m IcDZZo0dCFKqGkHWzInuprYRriTrHwH8Gqcpnfh2s8BasB0dJECipcko= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LYMd0zxOCJVY; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 850994193F; Mon, 11 Mar 2019 16:31:30 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:29 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 01/21] PCI: Fix writing invalid BARs during pci_restore_state() Date: Mon, 11 Mar 2019 16:31:02 +0300 Message-ID: <20190311133122.11417-2-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If BAR movement has happened (due to PCIe hotplug) after pci_save_state(), the saved addresses will become outdated. Restore them the most recently calculated values, not the ones stored in an arbitrary moment. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 7c1b362f599a..f006068be209 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1376,7 +1376,7 @@ static void pci_restore_config_space(struct pci_dev *pdev) if (pdev->hdr_type == PCI_HEADER_TYPE_NORMAL) { pci_restore_config_space_range(pdev, 10, 15, 0, false); /* Restore BARs before the command register. */ - pci_restore_config_space_range(pdev, 4, 9, 10, false); + pci_restore_bars(pdev); pci_restore_config_space_range(pdev, 0, 3, 0, false); } else if (pdev->hdr_type == PCI_HEADER_TYPE_BRIDGE) { pci_restore_config_space_range(pdev, 12, 15, 0, false); From patchwork Mon Mar 11 13:31:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847457 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A90C21823 for ; Mon, 11 Mar 2019 13:32:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8533929430 for ; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 833E729459; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A4B529430 for ; Mon, 11 Mar 2019 13:31:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727590AbfCKNbf (ORCPT ); Mon, 11 Mar 2019 09:31:35 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50926 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727498AbfCKNbe (ORCPT ); Mon, 11 Mar 2019 09:31:34 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id D9CDC419E1; Mon, 11 Mar 2019 13:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311091; x=1554125492; bh=66s2kCOqq88zcimSppWGg7PR6QLy3zAR7m1 vSMIMRF0=; b=HY76VbSeRpu59ojIX6F1rUTcaYZLq8lluzVmKUqdiLeZKrmvW4c igxcC1BMPFxdIYHHXxv0mPDtyppM8yiebzIJdiFODlyzCrIF6v3L/5ZhTA+hA3lI BzhEqKDDRjluDIV7LAgetuU4bgECK+a6GNi43Q2h+6waUk/m/xPw5D24= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z-zh5dnGxCVe; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id B3FD341954; Mon, 11 Mar 2019 16:31:30 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:30 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 02/21] PCI: Fix race condition in pci_enable/disable_device() Date: Mon, 11 Mar 2019 16:31:03 +0300 Message-ID: <20190311133122.11417-3-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP CPU0 CPU1 pci_enable_device_mem() pci_enable_device_mem() pci_enable_bridge() pci_enable_bridge() pci_is_enabled() return false; atomic_inc_return(enable_cnt) Start actual enabling the bridge ... pci_is_enabled() ... return true; ... Start memory requests <-- FAIL ... Set the PCI_COMMAND_MEMORY bit <-- Must wait for this This patch protects the pci_enable/disable_device() and pci_enable_bridge() with mutexes. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.c | 26 ++++++++++++++++++++++---- drivers/pci/probe.c | 1 + include/linux/pci.h | 1 + 3 files changed, 24 insertions(+), 4 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index f006068be209..895201d4c9e6 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1615,6 +1615,8 @@ static void pci_enable_bridge(struct pci_dev *dev) struct pci_dev *bridge; int retval; + mutex_lock(&dev->enable_mutex); + bridge = pci_upstream_bridge(dev); if (bridge) pci_enable_bridge(bridge); @@ -1622,6 +1624,7 @@ static void pci_enable_bridge(struct pci_dev *dev) if (pci_is_enabled(dev)) { if (!dev->is_busmaster) pci_set_master(dev); + mutex_unlock(&dev->enable_mutex); return; } @@ -1630,11 +1633,14 @@ static void pci_enable_bridge(struct pci_dev *dev) pci_err(dev, "Error enabling bridge (%d), continuing\n", retval); pci_set_master(dev); + mutex_unlock(&dev->enable_mutex); } static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags) { struct pci_dev *bridge; + /* Enable-locking of bridges is performed within the pci_enable_bridge() */ + bool need_lock = !dev->subordinate; int err; int i, bars = 0; @@ -1650,8 +1656,13 @@ static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags) dev->current_state = (pmcsr & PCI_PM_CTRL_STATE_MASK); } - if (atomic_inc_return(&dev->enable_cnt) > 1) + if (need_lock) + mutex_lock(&dev->enable_mutex); + if (pci_is_enabled(dev)) { + if (need_lock) + mutex_unlock(&dev->enable_mutex); return 0; /* already enabled */ + } bridge = pci_upstream_bridge(dev); if (bridge) @@ -1666,8 +1677,10 @@ static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags) bars |= (1 << i); err = do_pci_enable_device(dev, bars); - if (err < 0) - atomic_dec(&dev->enable_cnt); + if (err >= 0) + atomic_inc(&dev->enable_cnt); + if (need_lock) + mutex_unlock(&dev->enable_mutex); return err; } @@ -1910,15 +1923,20 @@ void pci_disable_device(struct pci_dev *dev) if (dr) dr->enabled = 0; + mutex_lock(&dev->enable_mutex); dev_WARN_ONCE(&dev->dev, atomic_read(&dev->enable_cnt) <= 0, "disabling already-disabled device"); - if (atomic_dec_return(&dev->enable_cnt) != 0) + if (atomic_dec_return(&dev->enable_cnt) != 0) { + mutex_unlock(&dev->enable_mutex); return; + } do_pci_disable_device(dev); dev->is_busmaster = 0; + + mutex_unlock(&dev->enable_mutex); } EXPORT_SYMBOL(pci_disable_device); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 2ec0df04e0dc..977a127ce791 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2267,6 +2267,7 @@ struct pci_dev *pci_alloc_dev(struct pci_bus *bus) INIT_LIST_HEAD(&dev->bus_list); dev->dev.type = &pci_dev_type; dev->bus = pci_bus_get(bus); + mutex_init(&dev->enable_mutex); return dev; } diff --git a/include/linux/pci.h b/include/linux/pci.h index 77448215ef5b..cb2760a31fe2 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -419,6 +419,7 @@ struct pci_dev { unsigned int no_vf_scan:1; /* Don't scan for VFs after IOV enablement */ pci_dev_flags_t dev_flags; atomic_t enable_cnt; /* pci_enable_device has been called */ + struct mutex enable_mutex; u32 saved_config_space[16]; /* Config space saved at suspend time */ struct hlist_head saved_cap_space; From patchwork Mon Mar 11 13:31:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847443 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D1A71823 for ; Mon, 11 Mar 2019 13:32:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC3FB2943C for ; Mon, 11 Mar 2019 13:31:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EA5652944B; Mon, 11 Mar 2019 13:31:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4FE7F29436 for ; Mon, 11 Mar 2019 13:31:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727498AbfCKNbf (ORCPT ); Mon, 11 Mar 2019 09:31:35 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50940 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727320AbfCKNbf (ORCPT ); Mon, 11 Mar 2019 09:31:35 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 88A3F419E5; Mon, 11 Mar 2019 13:31:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311092; x=1554125493; bh=vliVFv7PA6/r2aUHNWMkyssw6gL7ytIVfqT QWut6tdQ=; b=OzLhL0vGVud5aBPqVINc5xslo9Tp3VrN/FSTSl5W0UOaoa1ajIX Mgrdqr08600WVLxTaA3caCI10hTSsRQIBLMehSLR2SUgKyhsAOyt5D7Rw2SblVcn kfopcUTUbU1YLvJoXERj5+RV3rnL9+L2ARJQzg/QkgTM1lEyMcVbKvnM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jmF_aYm7xpBa; Mon, 11 Mar 2019 16:31:32 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id E6FAF41980; Mon, 11 Mar 2019 16:31:30 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:30 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 03/21] PCI: Enable bridge's I/O and MEM access for hotplugged devices Date: Mon, 11 Mar 2019 16:31:04 +0300 Message-ID: <20190311133122.11417-4-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After updating the bridge window resources, the PCI_COMMAND_IO and PCI_COMMAND_MEMORY bits of the bridge must be addressed as well. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 895201d4c9e6..69898fe5255e 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1622,6 +1622,14 @@ static void pci_enable_bridge(struct pci_dev *dev) pci_enable_bridge(bridge); if (pci_is_enabled(dev)) { + int i, bars = 0; + + for (i = PCI_BRIDGE_RESOURCES; i < DEVICE_COUNT_RESOURCE; i++) { + if (dev->resource[i].flags & (IORESOURCE_MEM | IORESOURCE_IO)) + bars |= (1 << i); + } + do_pci_enable_device(dev, bars); + if (!dev->is_busmaster) pci_set_master(dev); mutex_unlock(&dev->enable_mutex); From patchwork Mon Mar 11 13:31:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847453 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5228318FD for ; Mon, 11 Mar 2019 13:32:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1103D292F2 for ; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0EF6F2944D; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB6C129437 for ; Mon, 11 Mar 2019 13:31:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727599AbfCKNbg (ORCPT ); Mon, 11 Mar 2019 09:31:36 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50946 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727388AbfCKNbf (ORCPT ); Mon, 11 Mar 2019 09:31:35 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id D040C419E8; Mon, 11 Mar 2019 13:31:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311092; x=1554125493; bh=D+5t+XYJy2whiChmoUpciLcCd/RVrE0+9uo ha9o9Ma8=; b=n2mFSY5GApvd3DVC4xRrJJKCx8udjGs2eELsWla2kVeSoyNjMk1 MN7MuXfiGJOBuIuXE4w6Aoaq0OQJpukJ4LsQnqhoRiI76y23CmIXey62WeLkTqeW aeF6F+t/4QrKbuSZJ3qK/c0rKZW++ZFANk53L8/FhaIFe29CBbZEO3NE= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vN0iJy3olqB8; Mon, 11 Mar 2019 16:31:32 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 35CCB41987; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:30 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 04/21] PCI: Define PCI-specific version of the release_child_resources() Date: Mon, 11 Mar 2019 16:31:05 +0300 Message-ID: <20190311133122.11417-5-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the released resources of a bridge valid for later re-assignment: clear the STARTALIGN flag. Resources marked with PCI_FIXED must preserve their offset and size. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 47 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index ec44a0f3a7ac..3644feb13179 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1483,6 +1483,51 @@ static void __pci_bridge_assign_resources(const struct pci_dev *bridge, (IORESOURCE_IO | IORESOURCE_MEM | IORESOURCE_PREFETCH |\ IORESOURCE_MEM_64) +/* + * Similar to release_child_resources(), but aware of PCI_FIXED and STARTALIGN flags + */ +static void pci_release_child_resources(struct resource *r) +{ + struct resource *tmp, *p; + + if (!r) + return; + + if (r->flags & IORESOURCE_PCI_FIXED) + return; + + p = r->child; + r->child = NULL; + while (p) { + resource_size_t size = resource_size(p); + + tmp = p; + p = p->sibling; + + tmp->parent = NULL; + tmp->sibling = NULL; + pci_release_child_resources(tmp); + + if (!tmp->flags) + continue; + + if (tmp->flags & IORESOURCE_PCI_FIXED) { + pr_debug("PCI: release fixed %pR (%s), keep its flags, base and size\n", + tmp, tmp->name); + continue; + } + + pr_debug("PCI: release %pR (%s)\n", tmp, tmp->name); + + /* need to restore size, and keep all the flags but STARTALIGN */ + tmp->start = 0; + tmp->end = size - 1; + + tmp->flags &= ~IORESOURCE_STARTALIGN; + tmp->flags |= IORESOURCE_SIZEALIGN; + } +} + static void pci_bridge_release_resources(struct pci_bus *bus, unsigned long type) { @@ -1528,7 +1573,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus, * if there are children under that, we should release them * all */ - release_child_resources(r); + pci_release_child_resources(r); if (!release_resource(r)) { type = old_flags = r->flags & PCI_RES_TYPE_MASK; pci_printk(KERN_DEBUG, dev, "resource %d %pR released\n", From patchwork Mon Mar 11 13:31:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847447 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED2BD14DE for ; Mon, 11 Mar 2019 13:32:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 797B0293A8 for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 744F829451; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 07791293A8 for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727650AbfCKNbh (ORCPT ); Mon, 11 Mar 2019 09:31:37 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50954 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727579AbfCKNbg (ORCPT ); Mon, 11 Mar 2019 09:31:36 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9F6AB419ED; Mon, 11 Mar 2019 13:31:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311093; x=1554125494; bh=2YSqpqAp7RereidNxrz9EwHm+jdyZfSmbhy QTmFWZz4=; b=pBtieKAH1ePQi9Lvjq2H4QrxcTyHuS1RjjXzNZPbBSO8CLTE7BE NUiVfpiuAQiRnrGRCzAzVHwhI63qXb4aVI4CaJz3GQ8TyJQTJqiB9tEgpkHvVZ81 v9FAav89QEgCjAkXBvO58abT5PtTpDJzBOXeDGp8eNE2Nw1WYPX7D+qk= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z03ACO5LPSsC; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 61CA441988; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:30 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 05/21] PCI: hotplug: Add a flag for the movable BARs feature Date: Mon, 11 Mar 2019 16:31:06 +0300 Message-ID: <20190311133122.11417-6-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a new PCIe device has been hot-plugged between the two active ones without big enough gap between their BARs, these BARs should be moved if their drivers support this feature. The drivers should be notified and paused during the procedure: 1) dev 8 (new) | v .. | dev 3 | dev 3 | dev 5 | dev 7 | .. | BAR 0 | BAR 1 | BAR 0 | BAR 0 | 2) dev 8 | v .. | dev 3 | dev 3 | --> --> | dev 5 | dev 7 | .. | BAR 0 | BAR 1 | --> --> | BAR 0 | BAR 0 | 3) .. | dev 3 | dev 3 | dev 8 | dev 8 | dev 5 | dev 7 | .. | BAR 0 | BAR 1 | BAR 0 | BAR 1 | BAR 0 | BAR 0 | Thus, prior reservation of memory regions by BIOS/bootloader/firmware is not required anymore for the PCIe hotplug. The PCI_MOVABLE_BARS flag is set by the platform is this feature is supported and tested, but can be overridden by the following command line option: pcie_movable_bars={ off | force } Signed-off-by: Sergey Miroshnichenko --- .../admin-guide/kernel-parameters.txt | 7 ++++++ drivers/pci/pci.c | 24 +++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 33 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 2b8ee90bb644..d40eaf993f80 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3417,6 +3417,13 @@ nomsi Do not use MSI for native PCIe PME signaling (this makes all PCIe root ports use INTx for all services). + pcie_movable_bars=[PCIE] + Override the movable BARs support detection: + off + Disable even if supported by the platform + force + Enable even if not explicitly declared as supported + pcmv= [HW,PCMCIA] BadgePAD 4 pd_ignore_unused diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 69898fe5255e..4dac49a887ec 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -139,6 +139,30 @@ static int __init pcie_port_pm_setup(char *str) } __setup("pcie_port_pm=", pcie_port_pm_setup); +static bool pcie_movable_bars_off; +static bool pcie_movable_bars_force; +static int __init pcie_movable_bars_setup(char *str) +{ + if (!strcmp(str, "off")) + pcie_movable_bars_off = true; + else if (!strcmp(str, "force")) + pcie_movable_bars_force = true; + return 1; +} +__setup("pcie_movable_bars=", pcie_movable_bars_setup); + +bool pci_movable_bars_enabled(void) +{ + if (pcie_movable_bars_off) + return false; + + if (pcie_movable_bars_force) + return true; + + return pci_has_flag(PCI_MOVABLE_BARS); +} +EXPORT_SYMBOL(pci_movable_bars_enabled); + /* Time to wait after a reset for device to become responsive */ #define PCIE_RESET_READY_POLL_MS 60000 diff --git a/include/linux/pci.h b/include/linux/pci.h index cb2760a31fe2..cbe661aff9f5 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -866,6 +866,7 @@ enum { PCI_ENABLE_PROC_DOMAINS = 0x00000010, /* Enable domains in /proc */ PCI_COMPAT_DOMAIN_0 = 0x00000020, /* ... except domain 0 */ PCI_SCAN_ALL_PCIE_DEVS = 0x00000040, /* Scan all, not just dev 0 */ + PCI_MOVABLE_BARS = 0x00000080, /* Runtime BAR reassign after hotplug */ }; /* These external functions are only available when PCI support is enabled */ @@ -1345,6 +1346,7 @@ unsigned char pci_bus_max_busnr(struct pci_bus *bus); void pci_setup_bridge(struct pci_bus *bus); resource_size_t pcibios_window_alignment(struct pci_bus *bus, unsigned long type); +bool pci_movable_bars_enabled(void); #define PCI_VGA_STATE_CHANGE_BRIDGE (1 << 0) #define PCI_VGA_STATE_CHANGE_DECODES (1 << 1) From patchwork Mon Mar 11 13:31:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847445 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD21E1874 for ; Mon, 11 Mar 2019 13:32:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D2042944B for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4AE7529439; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB1792944D for ; Mon, 11 Mar 2019 13:31:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727579AbfCKNbh (ORCPT ); Mon, 11 Mar 2019 09:31:37 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50956 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727595AbfCKNbg (ORCPT ); Mon, 11 Mar 2019 09:31:36 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id D0640419F0; Mon, 11 Mar 2019 13:31:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311093; x=1554125494; bh=JVyoC34eGmx5ip94yGCGHtFiL2Yb7i7wQdT eL3g7f0E=; b=Vji5435/EYsHto3qxulaR3Fsu5a2VZenvqnsNfPgWrYntsRj4yP 7bwlGxaXAdWPuNCa5Zl9ZmC8jZCiv2tcmgpKgVIBMecz4ZGZWdUoZ75c6QMJT5nf rVeRwQMg+FdmWBmtOuQ0KNoKYWs3Rhz/wTzaMSIs6EfBVs1xX0oOd9oM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ng4E5I6HCMU9; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 8F9674198C; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:31 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 06/21] PCI: Pause the devices with movable BARs during rescan Date: Mon, 11 Mar 2019 16:31:07 +0300 Message-ID: <20190311133122.11417-7-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Drivers indicate their support of movable BARs by implementing the new rescan_prepare() and rescan_done() hooks in the struct pci_driver. All device's activity must be stopped during a rescan, and iounmap() +ioremap() must be applied to every used BAR. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/probe.c | 51 +++++++++++++++++++++++++++++++++++++++++++-- include/linux/pci.h | 2 ++ 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 977a127ce791..88350dd56344 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3248,6 +3248,38 @@ unsigned int pci_rescan_bus_bridge_resize(struct pci_dev *bridge) return max; } +static void pci_bus_rescan_prepare(struct pci_bus *bus) +{ + struct pci_dev *dev; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child = dev->subordinate; + + if (child) { + pci_bus_rescan_prepare(child); + } else if (dev->driver && + dev->driver->rescan_prepare) { + dev->driver->rescan_prepare(dev); + } + } +} + +static void pci_bus_rescan_done(struct pci_bus *bus) +{ + struct pci_dev *dev; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child = dev->subordinate; + + if (child) { + pci_bus_rescan_done(child); + } else if (dev->driver && + dev->driver->rescan_done) { + dev->driver->rescan_done(dev); + } + } +} + /** * pci_rescan_bus - Scan a PCI bus for devices * @bus: PCI bus to scan @@ -3261,8 +3293,23 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) { unsigned int max; - max = pci_scan_child_bus(bus); - pci_assign_unassigned_bus_resources(bus); + if (pci_movable_bars_enabled()) { + struct pci_bus *root = bus; + + while (!pci_is_root_bus(root)) + root = root->parent; + + pci_bus_rescan_prepare(root); + + max = pci_scan_child_bus(root); + pci_assign_unassigned_root_bus_resources(root); + + pci_bus_rescan_done(root); + } else { + max = pci_scan_child_bus(bus); + pci_assign_unassigned_bus_resources(bus); + } + pci_bus_add_devices(bus); return max; diff --git a/include/linux/pci.h b/include/linux/pci.h index cbe661aff9f5..3d52f5538282 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -780,6 +780,8 @@ struct pci_driver { int (*resume)(struct pci_dev *dev); /* Device woken up */ void (*shutdown)(struct pci_dev *dev); int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ + void (*rescan_prepare)(struct pci_dev *dev); + void (*rescan_done)(struct pci_dev *dev); const struct pci_error_handlers *err_handler; const struct attribute_group **groups; struct device_driver driver; From patchwork Mon Mar 11 13:31:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847455 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58D9046E2 for ; Mon, 11 Mar 2019 13:32:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 283F729437 for ; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2696A2945C; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C713B29459 for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727626AbfCKNbi (ORCPT ); Mon, 11 Mar 2019 09:31:38 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50964 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727320AbfCKNbi (ORCPT ); Mon, 11 Mar 2019 09:31:38 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 83F5F419F2; Mon, 11 Mar 2019 13:31:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311094; x=1554125495; bh=gIXHQd009lpgxVuce3TuU6iKAvHLcp7oViQ HqJBpuOY=; b=oM4pJG36orAJjiug6BYUzXbId0D5PFADFlr/4h8ZfTx38dKbx8l EMvV+bQtqy/ucfBuD0pSq2BC9JUJ7HFTfJ++k373X8nyTvOjEDRBSZaiita/zvu+ 1uvac+IgDPNRR/OiYOFLb0zySjWKjRQ2VKIkYOqkzdqnPEHQCnMrFIh0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5w5el8JkOmmg; Mon, 11 Mar 2019 16:31:34 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id BAC9D41992; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:31 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 07/21] PCI: Wake up bridges during rescan when movable BARs enabled Date: Mon, 11 Mar 2019 16:31:08 +0300 Message-ID: <20190311133122.11417-8-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Use the PM runtime methods to wake up the bridges before accessing their config space. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/probe.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 88350dd56344..dc935f82a595 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3252,6 +3252,8 @@ static void pci_bus_rescan_prepare(struct pci_bus *bus) { struct pci_dev *dev; + pm_runtime_get_sync(&bus->dev); + list_for_each_entry(dev, &bus->devices, bus_list) { struct pci_bus *child = dev->subordinate; @@ -3278,6 +3280,8 @@ static void pci_bus_rescan_done(struct pci_bus *bus) dev->driver->rescan_done(dev); } } + + pm_runtime_put(&bus->dev); } /** From patchwork Mon Mar 11 13:31:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847449 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19D871669 for ; Mon, 11 Mar 2019 13:32:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D89D729449 for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D12962925D; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E1F929453 for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727654AbfCKNbh (ORCPT ); Mon, 11 Mar 2019 09:31:37 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50966 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727388AbfCKNbh (ORCPT ); Mon, 11 Mar 2019 09:31:37 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id D941441992; Mon, 11 Mar 2019 13:31:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311094; x=1554125495; bh=9BP3Tbv9+ubeBXLOIcXMl63xSSLjPLFozXx oo+2YPJw=; b=LPmcZXttUt0GGAXWzlUfEKdo0PFqn+bU9zEx13/PIhxYZU7kLnQ IgEb1dOYJ6mSNfXKqPZS0/3Nqv3cjaZQuw4MlAby7sCSHK20PIUF72BJ2VpK5Myr Ms3vmQ1FbYWv3NjzxbxAg4/9CMLKeznpNZBX4oZIXuwRWtrNXQtwF/y8= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fNIflPFjFzos; Mon, 11 Mar 2019 16:31:34 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id EF6EF419CA; Mon, 11 Mar 2019 16:31:31 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:31 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 08/21] nvme-pci: Handle movable BARs Date: Mon, 11 Mar 2019 16:31:09 +0300 Message-ID: <20190311133122.11417-9-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hotplugged devices can affect the existing ones by moving their BARs. PCI subsystem will inform the NVME driver about this by invoking reset_prepare()+reset_done(), then iounmap()+ioremap() must be called. Signed-off-by: Sergey Miroshnichenko --- drivers/nvme/host/pci.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 92bad1c810ac..ccea3033a67a 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -106,6 +106,7 @@ struct nvme_dev { unsigned int num_vecs; int q_depth; u32 db_stride; + resource_size_t current_phys_bar; void __iomem *bar; unsigned long bar_mapped_size; struct work_struct remove_work; @@ -1672,13 +1673,16 @@ static int nvme_remap_bar(struct nvme_dev *dev, unsigned long size) { struct pci_dev *pdev = to_pci_dev(dev->dev); - if (size <= dev->bar_mapped_size) + if (dev->bar && + dev->current_phys_bar == pci_resource_start(pdev, 0) && + size <= dev->bar_mapped_size) return 0; if (size > pci_resource_len(pdev, 0)) return -ENOMEM; if (dev->bar) iounmap(dev->bar); - dev->bar = ioremap(pci_resource_start(pdev, 0), size); + dev->current_phys_bar = pci_resource_start(pdev, 0); + dev->bar = ioremap(dev->current_phys_bar, size); if (!dev->bar) { dev->bar_mapped_size = 0; return -ENOMEM; @@ -2504,6 +2508,8 @@ static void nvme_reset_work(struct work_struct *work) if (WARN_ON(dev->ctrl.state != NVME_CTRL_RESETTING)) goto out; + nvme_remap_bar(dev, db_bar_size(dev, 0)); + /* * If we're called to reset a live controller first shut it down before * moving on. @@ -2910,6 +2916,23 @@ static void nvme_error_resume(struct pci_dev *pdev) flush_work(&dev->ctrl.reset_work); } +void nvme_rescan_prepare(struct pci_dev *pdev) +{ + struct nvme_dev *dev = pci_get_drvdata(pdev); + + nvme_dev_disable(dev, false); + nvme_dev_unmap(dev); + dev->bar = NULL; +} + +void nvme_rescan_done(struct pci_dev *pdev) +{ + struct nvme_dev *dev = pci_get_drvdata(pdev); + + nvme_dev_map(dev); + nvme_reset_ctrl_sync(&dev->ctrl); +} + static const struct pci_error_handlers nvme_err_handler = { .error_detected = nvme_error_detected, .slot_reset = nvme_slot_reset, @@ -2974,6 +2997,8 @@ static struct pci_driver nvme_driver = { }, .sriov_configure = pci_sriov_configure_simple, .err_handler = &nvme_err_handler, + .rescan_prepare = nvme_rescan_prepare, + .rescan_done = nvme_rescan_done, }; static int __init nvme_init(void) From patchwork Mon Mar 11 13:31:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847461 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE53C1669 for ; Mon, 11 Mar 2019 13:32:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46E66292E0 for ; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 44F97292DE; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4EAC29454 for ; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727388AbfCKNbi (ORCPT ); Mon, 11 Mar 2019 09:31:38 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50972 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727626AbfCKNbh (ORCPT ); Mon, 11 Mar 2019 09:31:37 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 68201419CA; Mon, 11 Mar 2019 13:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311095; x=1554125496; bh=Lng2D1PYFils89ewEVSxOowNkJZhOaTQIft ZeaalTUk=; b=AZIZynvfwl9Kd/Ro4/+KfNyEJ5btwb9dDcIDHKTvX5E6uz/vekO xhVYBZN1Q0rPZrkJzstkr6+d7rGvQFaDhLz41plo3gYQh2kJO9f0Oqj2CpMoM7EJ gfZ13u14eGOBmzRlo4XQ7mdI5Hyr/k9hSTyUuYsiTXbqTPX4B2ZS+bEY= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ud4tR18LosMb; Mon, 11 Mar 2019 16:31:35 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 38462419CF; Mon, 11 Mar 2019 16:31:32 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:31 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 09/21] PCI: Mark immovable BARs with PCI_FIXED Date: Mon, 11 Mar 2019 16:31:10 +0300 Message-ID: <20190311133122.11417-10-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a PCIe device driver doesn't yet have support for movable BARs, mark device's BARs with IORESOURCE_PCI_FIXED. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/probe.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index dc935f82a595..1cf6ec960236 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3262,6 +3262,21 @@ static void pci_bus_rescan_prepare(struct pci_bus *bus) } else if (dev->driver && dev->driver->rescan_prepare) { dev->driver->rescan_prepare(dev); + } else if (dev->driver || ((dev->class >> 8) == PCI_CLASS_DISPLAY_VGA)) { + int i; + + for (i = 0; i < PCI_NUM_RESOURCES; i++) { + struct resource *r = &dev->resource[i]; + + if (!r->flags || !r->parent || + (r->flags & IORESOURCE_UNSET) || + (r->flags & IORESOURCE_PCI_FIXED)) + continue; + + r->flags |= IORESOURCE_PCI_FIXED; + pci_warn(dev, "%s: no support for movable BARs, mark BAR %d (%pR) as fixed\n", + __func__, i, r); + } } } } From patchwork Mon Mar 11 13:31:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847459 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D7A01669 for ; Mon, 11 Mar 2019 13:32:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 898DE29458 for ; Mon, 11 Mar 2019 13:31:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 87E2829472; Mon, 11 Mar 2019 13:31:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82FA929458 for ; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727661AbfCKNbi (ORCPT ); Mon, 11 Mar 2019 09:31:38 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50974 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727595AbfCKNbi (ORCPT ); Mon, 11 Mar 2019 09:31:38 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id BE34D419CF; Mon, 11 Mar 2019 13:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311095; x=1554125496; bh=VjC5ICoQdeuXnQ8WzrZDzFu/nzYS7FudbeH OOK8UNoE=; b=EhNpmvyvQSRYsi40wRDqF81FeAm6ZnN4e0wduO2SsG4T4p5U91m fr3nfejAIoo8lNIlA5K+n0E7sjiGDkPBise63czH0UTnmEmacb0k1299eASaTnaC dNBCeNjawLXsCxBO6CHCmKJmek3W/Z5SyYGKvhev1lqHLYfWfMu4rWWY= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0Ai3lLbZUm70; Mon, 11 Mar 2019 16:31:35 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 659DF419D8; Mon, 11 Mar 2019 16:31:32 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:31 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 10/21] PCI: Fix assigning of fixed prefetchable resources Date: Mon, 11 Mar 2019 16:31:11 +0300 Message-ID: <20190311133122.11417-11-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow matching them to non-prefetchable windows, as it is done for movable resources. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 3644feb13179..be7d4e6d7b65 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1301,15 +1301,20 @@ static void assign_fixed_resource_on_bus(struct pci_bus *b, struct resource *r) { int i; struct resource *parent_r; - unsigned long mask = IORESOURCE_IO | IORESOURCE_MEM | - IORESOURCE_PREFETCH; + unsigned long mask = IORESOURCE_TYPE_BITS; pci_bus_for_each_resource(b, parent_r, i) { if (!parent_r) continue; - if ((r->flags & mask) == (parent_r->flags & mask) && - resource_contains(parent_r, r)) + if ((r->flags & mask) != (parent_r->flags & mask)) + continue; + + if (parent_r->flags & IORESOURCE_PREFETCH && + !(r->flags & IORESOURCE_PREFETCH)) + continue; + + if (resource_contains(parent_r, r)) request_resource(parent_r, r); } } From patchwork Mon Mar 11 13:31:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847471 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64FFB1669 for ; Mon, 11 Mar 2019 13:32:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6C62290D1 for ; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C52752948E; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54043290D1 for ; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727553AbfCKNbo (ORCPT ); Mon, 11 Mar 2019 09:31:44 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50964 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727673AbfCKNbo (ORCPT ); Mon, 11 Mar 2019 09:31:44 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 380A841980; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311096; x=1554125497; bh=XcwuVqHempHAfwnox4/GHtbL1ImHCqBJfQJ RZzsuR4Q=; b=NsikD/q4CliFskFoKcpmrSW5mL9XkZLJh8IM9qo+0MEhCtiEPn0 W/zxAuOcVM38/yo8eCHAV9fXX5Vq9PIOBBB5T/OsfcDfc+9AZjaY+CYagc5atDoH azVKdWWQ9VqkwXYHuRMrjcCrGN6Cq08smKk0BPpMK+Se3jnA9TOOrNBc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NRQ7k-twhriP; Mon, 11 Mar 2019 16:31:36 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id A13C7419DD; Mon, 11 Mar 2019 16:31:32 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:32 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 11/21] PCI: Release and reassign the root bridge resources during rescan Date: Mon, 11 Mar 2019 16:31:12 +0300 Message-ID: <20190311133122.11417-12-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When the movable BARs feature is enabled, don't rely on the memory gaps reserved by the BIOS/bootloader/firmware, but instead rearrange the BARs and bridge windows starting from the root. Endpoint device's BARs, after being released, are resorted and written back by the pci_assign_unassigned_root_bus_resources(). The last step of writing the recalculated windows to the bridges is done by the new pci_setup_bridges() function. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.h | 1 + drivers/pci/probe.c | 22 ++++++++++++++++++++++ drivers/pci/setup-bus.c | 11 ++++++++++- 3 files changed, 33 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 224d88634115..e06e8692a7b1 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -248,6 +248,7 @@ void __pci_bus_assign_resources(const struct pci_bus *bus, struct list_head *realloc_head, struct list_head *fail_head); bool pci_bus_clip_resource(struct pci_dev *dev, int idx); +void pci_bus_release_root_bridge_resources(struct pci_bus *bus); void pci_reassigndev_resource_alignment(struct pci_dev *dev); void pci_disable_bridge_window(struct pci_dev *dev); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 1cf6ec960236..692752c71f71 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3299,6 +3299,25 @@ static void pci_bus_rescan_done(struct pci_bus *bus) pm_runtime_put(&bus->dev); } +static void pci_setup_bridges(struct pci_bus *bus) +{ + struct pci_dev *dev; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child; + + if (!pci_dev_is_added(dev) || pci_dev_is_ignored(dev)) + continue; + + child = dev->subordinate; + if (child) + pci_setup_bridges(child); + } + + if (bus->self) + pci_setup_bridge(bus); +} + /** * pci_rescan_bus - Scan a PCI bus for devices * @bus: PCI bus to scan @@ -3321,8 +3340,11 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) pci_bus_rescan_prepare(root); max = pci_scan_child_bus(root); + + pci_bus_release_root_bridge_resources(root); pci_assign_unassigned_root_bus_resources(root); + pci_setup_bridges(root); pci_bus_rescan_done(root); } else { max = pci_scan_child_bus(bus); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index be7d4e6d7b65..36a1907d9509 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1584,7 +1584,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus, pci_printk(KERN_DEBUG, dev, "resource %d %pR released\n", PCI_BRIDGE_RESOURCES + idx, r); /* keep the old size */ - r->end = resource_size(r) - 1; + r->end = pci_movable_bars_enabled() ? 0 : (resource_size(r) - 1); r->start = 0; r->flags = 0; @@ -1637,6 +1637,15 @@ static void pci_bus_release_bridge_resources(struct pci_bus *bus, pci_bridge_release_resources(bus, type); } +void pci_bus_release_root_bridge_resources(struct pci_bus *root_bus) +{ + pci_bus_release_bridge_resources(root_bus, IORESOURCE_IO, whole_subtree); + pci_bus_release_bridge_resources(root_bus, IORESOURCE_MEM, whole_subtree); + pci_bus_release_bridge_resources(root_bus, + IORESOURCE_MEM_64 | IORESOURCE_PREFETCH, + whole_subtree); +} + static void pci_bus_dump_res(struct pci_bus *bus) { struct resource *res; From patchwork Mon Mar 11 13:31:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847467 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E485518A6 for ; Mon, 11 Mar 2019 13:32:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 303C0292FC for ; Mon, 11 Mar 2019 13:31:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2DB6529489; Mon, 11 Mar 2019 13:31:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17985292FC for ; Mon, 11 Mar 2019 13:31:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727667AbfCKNbk (ORCPT ); Mon, 11 Mar 2019 09:31:40 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50956 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727320AbfCKNbk (ORCPT ); Mon, 11 Mar 2019 09:31:40 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 1F285419D8; Mon, 11 Mar 2019 13:31:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311096; x=1554125497; bh=/9KIMo8tGFGsSavddHdVGWehEBkzqeWWLDq mnnymnno=; b=CIecb3sld/QncDrstpQzACVN4VFypvS9//JFArcQVUpxByywJ6S 1hym/Xaifynl5kTbB7pYQLLGAk21UGJhc2U0jrW/dcdyefr+TTkrxLLJdl5pGkEi 3Ws/zX/W4IEjasbjvNab/XFLdhupE89uPDNd3B1VVufU7b/yIa0La09Q= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d1xed_7j1Bn8; Mon, 11 Mar 2019 16:31:36 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id CF4D14193F; Mon, 11 Mar 2019 16:31:32 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:32 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 12/21] PCI: Don't allow hotplugged devices to steal resources Date: Mon, 11 Mar 2019 16:31:13 +0300 Message-ID: <20190311133122.11417-13-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When movable BARs are enabled, the PCI subsystem at first releases all the bridge windows and then performs an attempt to assign new requested resources and re-assign the existing ones. If a hotplugged device gets its resources first, there could be no space left to re-assign resources of already working devices, which is unacceptable. If this happens, this patch marks one of the new devices with the new introduced flag PCI_DEV_IGNORE and retries the resource assignment. This patch adds a new res_mask bitmask to the struct pci_dev for storing the indices of assigned resources. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/bus.c | 5 ++ drivers/pci/pci.h | 11 +++++ drivers/pci/probe.c | 100 +++++++++++++++++++++++++++++++++++++++- drivers/pci/setup-bus.c | 15 ++++++ include/linux/pci.h | 1 + 5 files changed, 130 insertions(+), 2 deletions(-) diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index 5cb40b2518f9..a9784144d6f2 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -311,6 +311,11 @@ void pci_bus_add_device(struct pci_dev *dev) { int retval; + if (pci_dev_is_ignored(dev)) { + pci_warn(dev, "%s: don't enable the ignored device\n", __func__); + return; + } + /* * Can not put in pci_device_add yet because resources * are not assigned yet for some devices. diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index e06e8692a7b1..56b905068ac5 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -366,6 +366,7 @@ static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) /* pci_dev priv_flags */ #define PCI_DEV_ADDED 0 +#define PCI_DEV_IGNORE 1 static inline void pci_dev_assign_added(struct pci_dev *dev, bool added) { @@ -377,6 +378,16 @@ static inline bool pci_dev_is_added(const struct pci_dev *dev) return test_bit(PCI_DEV_ADDED, &dev->priv_flags); } +static inline void pci_dev_ignore(struct pci_dev *dev, bool ignore) +{ + assign_bit(PCI_DEV_IGNORE, &dev->priv_flags, ignore); +} + +static inline bool pci_dev_is_ignored(const struct pci_dev *dev) +{ + return test_bit(PCI_DEV_IGNORE, &dev->priv_flags); +} + #ifdef CONFIG_PCIEAER #include diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 692752c71f71..62f4058a001f 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3248,6 +3248,23 @@ unsigned int pci_rescan_bus_bridge_resize(struct pci_dev *bridge) return max; } +static unsigned int pci_dev_res_mask(struct pci_dev *dev) +{ + unsigned int res_mask = 0; + int i; + + for (i = 0; i < PCI_BRIDGE_RESOURCES; i++) { + struct resource *r = &dev->resource[i]; + + if (!r->flags || (r->flags & IORESOURCE_UNSET) || !r->parent) + continue; + + res_mask |= (1 << i); + } + + return res_mask; +} + static void pci_bus_rescan_prepare(struct pci_bus *bus) { struct pci_dev *dev; @@ -3257,6 +3274,8 @@ static void pci_bus_rescan_prepare(struct pci_bus *bus) list_for_each_entry(dev, &bus->devices, bus_list) { struct pci_bus *child = dev->subordinate; + dev->res_mask = pci_dev_res_mask(dev); + if (child) { pci_bus_rescan_prepare(child); } else if (dev->driver && @@ -3318,6 +3337,84 @@ static void pci_setup_bridges(struct pci_bus *bus) pci_setup_bridge(bus); } +static struct pci_dev *pci_find_next_new_device(struct pci_bus *bus) +{ + struct pci_dev *dev; + + if (!bus) + return NULL; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child_bus = dev->subordinate; + + if (!pci_dev_is_added(dev) && !pci_dev_is_ignored(dev)) + return dev; + + if (child_bus) { + struct pci_dev *next_new_dev; + + next_new_dev = pci_find_next_new_device(child_bus); + if (next_new_dev) + return next_new_dev; + } + } + + return NULL; +} + +static bool pci_bus_validate_resources(struct pci_bus *bus) +{ + struct pci_dev *dev; + bool ret = true; + + if (!bus) + return false; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child = dev->subordinate; + unsigned int res_mask = pci_dev_res_mask(dev); + + if (pci_dev_is_ignored(dev)) + continue; + + if (dev->res_mask & ~res_mask) { + pci_err(dev, "%s: Non-re-enabled resources found: 0x%x -> 0x%x\n", + __func__, dev->res_mask, res_mask); + ret = false; + } + + if (child && !pci_bus_validate_resources(child)) + ret = false; + } + + return ret; +} + +static void pci_reassign_root_bus_resources(struct pci_bus *root) +{ + do { + struct pci_dev *next_new_dev; + + pci_bus_release_root_bridge_resources(root); + pci_assign_unassigned_root_bus_resources(root); + + if (pci_bus_validate_resources(root)) + break; + + next_new_dev = pci_find_next_new_device(root); + if (!next_new_dev) { + dev_err(&root->dev, "%s: failed to re-assign resources even after ignoring all the hotplugged devices\n", + __func__); + break; + } + + dev_warn(&root->dev, "%s: failed to re-assign resources, disable the next hotplugged device %s and retry\n", + __func__, dev_name(&next_new_dev->dev)); + + pci_dev_ignore(next_new_dev, true); + } while (true); +} + /** * pci_rescan_bus - Scan a PCI bus for devices * @bus: PCI bus to scan @@ -3341,8 +3438,7 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) max = pci_scan_child_bus(root); - pci_bus_release_root_bridge_resources(root); - pci_assign_unassigned_root_bus_resources(root); + pci_reassign_root_bus_resources(root); pci_setup_bridges(root); pci_bus_rescan_done(root); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 36a1907d9509..551108f48df7 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -131,6 +131,9 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head) { int i; + if (pci_dev_is_ignored(dev)) + return; + for (i = 0; i < PCI_NUM_RESOURCES; i++) { struct resource *r; struct pci_dev_resource *dev_res, *tmp; @@ -181,6 +184,9 @@ static void __dev_sort_resources(struct pci_dev *dev, { u16 class = dev->class >> 8; + if (pci_dev_is_ignored(dev)) + return; + /* Don't touch classless devices or host bridges or ioapics. */ if (class == PCI_CLASS_NOT_DEFINED || class == PCI_CLASS_BRIDGE_HOST) return; @@ -284,6 +290,9 @@ static void assign_requested_resources_sorted(struct list_head *head, int idx; list_for_each_entry(dev_res, head, list) { + if (pci_dev_is_ignored(dev_res->dev)) + continue; + res = dev_res->res; idx = res - &dev_res->dev->resource[0]; if (resource_size(res) && @@ -991,6 +1000,9 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, list_for_each_entry(dev, &bus->devices, bus_list) { int i; + if (pci_dev_is_ignored(dev)) + continue; + for (i = 0; i < PCI_NUM_RESOURCES; i++) { struct resource *r = &dev->resource[i]; resource_size_t r_size; @@ -1353,6 +1365,9 @@ void __pci_bus_assign_resources(const struct pci_bus *bus, pbus_assign_resources_sorted(bus, realloc_head, fail_head); list_for_each_entry(dev, &bus->devices, bus_list) { + if (pci_dev_is_ignored(dev)) + continue; + pdev_assign_fixed_resources(dev); b = dev->subordinate; diff --git a/include/linux/pci.h b/include/linux/pci.h index 3d52f5538282..26aa59cb6220 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -369,6 +369,7 @@ struct pci_dev { */ unsigned int irq; struct resource resource[DEVICE_COUNT_RESOURCE]; /* I/O and memory regions + expansion ROMs */ + unsigned int res_mask; /* Bitmask of assigned resources */ bool match_driver; /* Skip attaching driver */ From patchwork Mon Mar 11 13:31:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847465 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A45D814DE for ; Mon, 11 Mar 2019 13:32:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34B1A291E4 for ; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 32FA929460; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3484292E0 for ; Mon, 11 Mar 2019 13:31:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727320AbfCKNbl (ORCPT ); Mon, 11 Mar 2019 09:31:41 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50974 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727595AbfCKNbk (ORCPT ); Mon, 11 Mar 2019 09:31:40 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 142C04193F; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311098; x=1554125499; bh=iZ/t91iYGvx7w3wRPGcE7pZIHKXtq/jzwIf KcW2LmTs=; b=QG/ttT/7m+YwdGOMjVOu4DicVRjyitOeGUjZqitIaUqJu7WTzjC ymf/5q15obtzDrEiPYURgEz5vHhyvPm67l8CMTew+iPmVnlA2ug7aixx5X/c5q8u 3fCxdShDSBdbiKUa2e9YxJp1s5RJrYXSVbY2FIg2+Rjt4m3UuPE4qnJM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uGB8F9fIjggi; Mon, 11 Mar 2019 16:31:38 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 06B1241954; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:32 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 13/21] PCI: Include fixed BARs into the bus size calculating Date: Mon, 11 Mar 2019 16:31:14 +0300 Message-ID: <20190311133122.11417-14-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The only difference between the fixed and movable BARs is an offset preservation during the release+reassign procedure on PCIe rescan. When fixed BARs are included into the result of pbus_size_mem(), these BARs can be restricted: assign them to direct parents only. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 551108f48df7..9d93f2b32bf1 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1007,12 +1007,20 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, struct resource *r = &dev->resource[i]; resource_size_t r_size; - if (r->parent || (r->flags & IORESOURCE_PCI_FIXED) || + if (r->parent || ((r->flags & mask) != type && (r->flags & mask) != type2 && (r->flags & mask) != type3)) continue; r_size = resource_size(r); + + if (r->flags & IORESOURCE_PCI_FIXED) { + if (pci_movable_bars_enabled()) + size += r_size; + + continue; + } + #ifdef CONFIG_PCI_IOV /* put SRIOV requested res to the optional list */ if (realloc_head && i >= PCI_IOV_RESOURCES && @@ -1351,6 +1359,8 @@ static void pdev_assign_fixed_resources(struct pci_dev *dev) while (b && !r->parent) { assign_fixed_resource_on_bus(b, r); b = b->parent; + if (!r->parent && pci_movable_bars_enabled()) + break; } } } From patchwork Mon Mar 11 13:31:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847463 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B24971823 for ; Mon, 11 Mar 2019 13:32:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8787529478 for ; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8574E29488; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2639D29478 for ; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727595AbfCKNbl (ORCPT ); Mon, 11 Mar 2019 09:31:41 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50964 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727553AbfCKNbl (ORCPT ); Mon, 11 Mar 2019 09:31:41 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id E953841954; Mon, 11 Mar 2019 13:31:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311099; x=1554125500; bh=9umfpOrcBX+EqDyUX+SwxQGZbwDX1rCGnNX /j2sIf7I=; b=lGD8ses8RVm1Vg7RT6PCUACJ6DpftAd4xZ9HYKkso+rEpyY693I y88L9oL8qKSubyspUSdTpyLUzMa2q8G31sElrOKh/TESpBfhVB7OdkXvqSA45iLj 7W0n/nE02E0a/L/JDMPrGUdg88OCOgc9t+XZxicdhjQXOrFF3kB6AuMA= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fCvI2kAHhsql; Mon, 11 Mar 2019 16:31:39 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 3EC77419E3; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:32 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 14/21] PCI: Don't reserve memory for hotplug when enabled movable BARs Date: Mon, 11 Mar 2019 16:31:15 +0300 Message-ID: <20190311133122.11417-15-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP pbus_size_mem() returns a precise amount of memory required to fit all the requested BARs and windows of children bridges. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 9d93f2b32bf1..f9d605cd1725 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1229,7 +1229,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) case PCI_HEADER_TYPE_BRIDGE: pci_bridge_check_ranges(bus); - if (bus->self->is_hotplug_bridge) { + if (bus->self->is_hotplug_bridge && !pci_movable_bars_enabled()) { additional_io_size = pci_hotplug_io_size; additional_mem_size = pci_hotplug_mem_size; } From patchwork Mon Mar 11 13:31:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847469 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38F8218FD for ; Mon, 11 Mar 2019 13:32:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0ACD629485 for ; Mon, 11 Mar 2019 13:31:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 090752948E; Mon, 11 Mar 2019 13:31:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9EFB29485 for ; Mon, 11 Mar 2019 13:31:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727676AbfCKNbn (ORCPT ); Mon, 11 Mar 2019 09:31:43 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50956 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727669AbfCKNbm (ORCPT ); Mon, 11 Mar 2019 09:31:42 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id E41D3419E3; Mon, 11 Mar 2019 13:31:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311099; x=1554125500; bh=6wkpGdtzO/glOaSma5i0/AP+howVXtM9FAV V3bCUcjI=; b=ffWiV6Zl2kFzQitU39/RxXbDBgSmy2dZEjwK7Laz189NuugOYP1 i4d/FzIPnNpdjw2vyBtYfsMKXQhHHWL58sIDvrOyhbmnEa+QYguGwZtJf5svCNok +8eYE2+sBLZWV36zFn4DDcQyza6mRTANaGQ6bFVwGbwTBEwjHEXnKANM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RHwZd4YlmNcX; Mon, 11 Mar 2019 16:31:39 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 76AD4419E4; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:33 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 15/21] PCI: Allow the failed resources to be reassigned later Date: Mon, 11 Mar 2019 16:31:16 +0300 Message-ID: <20190311133122.11417-16-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Don't lose the size of the requested EP's BAR if it can't be fit in a current trial, so this can be retried. But a failed bridge window must be dropped and recalculated in the next trial. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 3 ++- drivers/pci/setup-res.c | 12 ++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index f9d605cd1725..c1559a4a8564 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -309,7 +309,8 @@ static void assign_requested_resources_sorted(struct list_head *head, 0 /* don't care */, 0 /* don't care */); } - reset_resource(res); + if (!pci_movable_bars_enabled()) + reset_resource(res); } } } diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index d8ca40a97693..732d18f60f1b 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -298,6 +298,18 @@ static int _pci_assign_resource(struct pci_dev *dev, int resno, bus = dev->bus; while ((ret = __pci_assign_resource(bus, dev, resno, size, min_align))) { + if (pci_movable_bars_enabled()) { + if (resno >= PCI_BRIDGE_RESOURCES && + resno <= PCI_BRIDGE_RESOURCE_END) { + struct resource *res = dev->resource + resno; + + res->start = 0; + res->end = 0; + res->flags = 0; + } + break; + } + if (!bus->parent || !bus->self->transparent) break; bus = bus->parent; From patchwork Mon Mar 11 13:31:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847473 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB5E31823 for ; Mon, 11 Mar 2019 13:32:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFEA02925C for ; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD4962948E; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28A932948F for ; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727741AbfCKNbo (ORCPT ); Mon, 11 Mar 2019 09:31:44 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50974 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727553AbfCKNbo (ORCPT ); Mon, 11 Mar 2019 09:31:44 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 33F86419E4; Mon, 11 Mar 2019 13:31:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311100; x=1554125501; bh=R7IbRdTRKbIoc00RoB3Bk7kPe21b7o79Tav On/8ZP3w=; b=ZCJpYZ4yq7nZg/pplJqpxKb++yYp2HFPD1QEfNM40a5U9WRfntA htn0K5+zwZtKFyk3Kn1LHfap9GQQ8U+bJ6tMHGQ/umK83+m0k75vo9ogClTNgIO/ 3ZYFUTNodGWC4esrOQAowron+oOWXA7nZLtVJPLR0KMvBKTmwdHFZbAk= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bj_zk5tmWxGs; Mon, 11 Mar 2019 16:31:40 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id A95E341980; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:33 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 16/21] PCI: Calculate fixed areas of bridge windows based on fixed BARs Date: Mon, 11 Mar 2019 16:31:17 +0300 Message-ID: <20190311133122.11417-17-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For every (IO, MEM, MEM64) bridge window, count the fixed resources of its children endpoints and children bridge windows: | <- BAR -> | | <- child bus fixed_range_hard -> | | <- fixed BAR -> | | <- bus's fixed_range_hard -> | | <- bus's bridge window -> | These ranges will be later used to arrange bridge windows in a way which covers every immovable BAR as well as the movable ones during hotplug. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.h | 14 +++++++ drivers/pci/probe.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/pci/setup-bus.c | 17 +++++++++ include/linux/pci.h | 6 +++ 4 files changed, 119 insertions(+) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 56b905068ac5..14e3ebe68010 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -364,6 +364,20 @@ static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) return dev->error_state == pci_channel_io_perm_failure; } +static inline int pci_get_bridge_resource_idx(struct resource *r) +{ + int idx = 1; + + if (r->flags & IORESOURCE_IO) + idx = 0; + else if (!(r->flags & IORESOURCE_PREFETCH)) + idx = 1; + else if (r->flags & IORESOURCE_MEM_64) + idx = 2; + + return idx; +} + /* pci_dev priv_flags */ #define PCI_DEV_ADDED 0 #define PCI_DEV_IGNORE 1 diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 62f4058a001f..70b15654f253 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -551,6 +551,7 @@ void pci_read_bridge_bases(struct pci_bus *child) static struct pci_bus *pci_alloc_bus(struct pci_bus *parent) { struct pci_bus *b; + int idx; b = kzalloc(sizeof(*b), GFP_KERNEL); if (!b) @@ -567,6 +568,11 @@ static struct pci_bus *pci_alloc_bus(struct pci_bus *parent) if (parent) b->domain_nr = parent->domain_nr; #endif + for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) { + b->fixed_range_hard[idx].start = (resource_size_t)-1; + b->fixed_range_hard[idx].end = 0; + } + return b; } @@ -3337,6 +3343,81 @@ static void pci_setup_bridges(struct pci_bus *bus) pci_setup_bridge(bus); } +static void pci_bus_update_fixed_range_hard(struct pci_bus *bus) +{ + struct pci_dev *dev; + int idx; + resource_size_t start, end; + + for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) { + bus->fixed_range_hard[idx].start = (resource_size_t)-1; + bus->fixed_range_hard[idx].end = 0; + } + + list_for_each_entry(dev, &bus->devices, bus_list) + if (dev->subordinate) + pci_bus_update_fixed_range_hard(dev->subordinate); + + list_for_each_entry(dev, &bus->devices, bus_list) { + int i; + + for (i = 0; i < PCI_BRIDGE_RESOURCES; ++i) { + struct resource *r = &dev->resource[i]; + + if (!r->flags || (r->flags & IORESOURCE_UNSET) || !r->parent) + continue; + + if (r->flags & IORESOURCE_PCI_FIXED) { + idx = pci_get_bridge_resource_idx(r); + start = bus->fixed_range_hard[idx].start; + end = bus->fixed_range_hard[idx].end; + + if (start > r->start) + start = r->start; + if (end < r->end) + end = r->end; + + if (bus->fixed_range_hard[idx].start != start || + bus->fixed_range_hard[idx].end != end) { + dev_dbg(&bus->dev, "%s: Found fixed 0x%llx-0x%llx in %s, expand the fixed bridge window %d to 0x%llx-0x%llx\n", + __func__, + (unsigned long long)r->start, + (unsigned long long)r->end, + dev_name(&dev->dev), idx, + (unsigned long long)start, + (unsigned long long)end); + bus->fixed_range_hard[idx].start = start; + bus->fixed_range_hard[idx].end = end; + } + } + } + + if (dev->subordinate) { + struct pci_bus *child = dev->subordinate; + + for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) { + start = bus->fixed_range_hard[idx].start; + end = bus->fixed_range_hard[idx].end; + + if (start > child->fixed_range_hard[idx].start) + start = child->fixed_range_hard[idx].start; + if (end < child->fixed_range_hard[idx].end) + end = child->fixed_range_hard[idx].end; + + if (start < bus->fixed_range_hard[idx].start || + end > bus->fixed_range_hard[idx].end) { + dev_dbg(&bus->dev, "%s: Expand the fixed bridge window %d from %s to 0x%llx-0x%llx\n", + __func__, idx, dev_name(&child->dev), + (unsigned long long)start, + (unsigned long long)end); + bus->fixed_range_hard[idx].start = start; + bus->fixed_range_hard[idx].end = end; + } + } + } + } +} + static struct pci_dev *pci_find_next_new_device(struct pci_bus *bus) { struct pci_dev *dev; @@ -3437,6 +3518,7 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) pci_bus_rescan_prepare(root); max = pci_scan_child_bus(root); + pci_bus_update_fixed_range_hard(root); pci_reassign_root_bus_resources(root); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index c1559a4a8564..a1fd7f3c5ea8 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -879,9 +879,17 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size, resource_size_t children_add_size = 0; resource_size_t min_align, align; + resource_size_t fixed_start = bus->fixed_range_hard[0].start; + resource_size_t fixed_end = bus->fixed_range_hard[0].end; + resource_size_t fixed_size = (fixed_start < fixed_end) ? + (fixed_end - fixed_start + 1) : 0; + if (!b_res) return; + if (min_size < fixed_size) + min_size = fixed_size; + min_align = window_alignment(bus, IORESOURCE_IO); list_for_each_entry(dev, &bus->devices, bus_list) { int i; @@ -990,6 +998,15 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, resource_size_t children_add_size = 0; resource_size_t children_add_align = 0; resource_size_t add_align = 0; + bool is_mem64 = (mask & IORESOURCE_MEM_64); + + resource_size_t fixed_start = bus->fixed_range_hard[is_mem64 ? 2 : 1].start; + resource_size_t fixed_end = bus->fixed_range_hard[is_mem64 ? 2 : 1].end; + resource_size_t fixed_size = (fixed_start < fixed_end) ? + (fixed_end - fixed_start + 1) : 0; + + if (min_size < fixed_size) + min_size = fixed_size; if (!b_res) return -ENOSPC; diff --git a/include/linux/pci.h b/include/linux/pci.h index 26aa59cb6220..7a4d62d84bc1 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -572,6 +572,12 @@ struct pci_bus { struct list_head resources; /* Address space routed to this bus */ struct resource busn_res; /* Bus numbers routed to this bus */ + /* + * If there are fixed resources in the bridge window, the hard range + * contains the lowest and the highest addresses of them. + */ + struct resource fixed_range_hard[PCI_BRIDGE_RESOURCE_NUM]; + struct pci_ops *ops; /* Configuration access functions */ struct msi_controller *msi; /* MSI controller */ void *sysdata; /* Hook for sys-specific extension */ From patchwork Mon Mar 11 13:31:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847477 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4359C14DE for ; Mon, 11 Mar 2019 13:32:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B3A502948F for ; Mon, 11 Mar 2019 13:31:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1CAC291EB; Mon, 11 Mar 2019 13:31:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 171DB29499 for ; Mon, 11 Mar 2019 13:31:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727669AbfCKNbp (ORCPT ); Mon, 11 Mar 2019 09:31:45 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:51006 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727728AbfCKNbo (ORCPT ); Mon, 11 Mar 2019 09:31:44 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 483DC419DD; Mon, 11 Mar 2019 13:31:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311102; x=1554125503; bh=v4RqgUFHH6GplQ7nmnx9ZtEMNEIUG8asLgr rg3uwImo=; b=Scguby7+Eur4COPC1ALrnHWBX0JboLuSkw5xeQ51qAEhT7QC8hZ N0KyURfjcWznw1O65QSJF07DOnAnY9A/2NIjjOVjMu6KC4feU7Lu0lcoXCpbe6iY mXErXuM7br3MUbPk34pj4UpE4hVaF6QXJnOg7TvG1duT5t2Q0gclXY60= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v3ap44qg5gAG; Mon, 11 Mar 2019 16:31:42 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id D4E7741987; Mon, 11 Mar 2019 16:31:33 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:33 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 17/21] PCI: Calculate boundaries for bridge windows Date: Mon, 11 Mar 2019 16:31:18 +0300 Message-ID: <20190311133122.11417-18-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a bridge window contains fixed areas (there are PCIe devices with immovable BARs located on this bus), this window must be allocated within the bound memory area, limited by windows size and by address range of fixed resources, calculated as follows: | <-- bus's fixed_range_hard --> | | <-- fixed_range_hard.end - window size --> | | <-- fixed_range_hard.start + window size --> | | <-- bus's fixed_range_soft --> | Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 56 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 4 ++- 2 files changed, 59 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index a1fd7f3c5ea8..f4737339d5ec 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1809,6 +1809,61 @@ static enum enable_type pci_realloc_detect(struct pci_bus *bus, } #endif +static void pci_bus_update_fixed_range_soft(struct pci_bus *bus) +{ + struct pci_dev *dev; + struct pci_bus *parent = bus->parent; + int idx; + + list_for_each_entry(dev, &bus->devices, bus_list) + if (dev->subordinate) + pci_bus_update_fixed_range_soft(dev->subordinate); + + if (!parent || !bus->self) + return; + + for (idx = 0; idx < ARRAY_SIZE(bus->fixed_range_hard); ++idx) { + struct resource *r; + resource_size_t soft_start, soft_end; + resource_size_t hard_start = bus->fixed_range_hard[idx].start; + resource_size_t hard_end = bus->fixed_range_hard[idx].end; + + if (hard_start > hard_end) + continue; + + r = bus->resource[idx]; + + soft_start = hard_end - resource_size(r) + 1; + soft_end = hard_start + resource_size(r) - 1; + + if (soft_start > hard_start) + soft_start = hard_start; + + if (soft_end < hard_end) + soft_end = hard_end; + + list_for_each_entry(dev, &parent->devices, bus_list) { + struct pci_bus *sibling = dev->subordinate; + resource_size_t s_start, s_end; + + if (!sibling || sibling == bus) + continue; + + s_start = sibling->fixed_range_hard[idx].start; + s_end = sibling->fixed_range_hard[idx].end; + + if (s_start > s_end) + continue; + + if (s_end < hard_start && s_end > soft_start) + soft_start = s_end; + } + + bus->fixed_range_soft[idx].start = soft_start; + bus->fixed_range_soft[idx].end = soft_end; + } +} + /* * first try will not touch pci bridge res * second and later try will clear small leaf bridge res @@ -1847,6 +1902,7 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus) /* Depth first, calculate sizes and alignments of all subordinate buses. */ __pci_bus_size_bridges(bus, add_list); + pci_bus_update_fixed_range_soft(bus); /* Depth last, allocate resources and update the hardware. */ __pci_bus_assign_resources(bus, add_list, &fail_head); diff --git a/include/linux/pci.h b/include/linux/pci.h index 7a4d62d84bc1..75a56db73ad4 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -574,9 +574,11 @@ struct pci_bus { /* * If there are fixed resources in the bridge window, the hard range - * contains the lowest and the highest addresses of them. + * contains the lowest and the highest addresses of them, and this + * bridge window must reside within the soft range. */ struct resource fixed_range_hard[PCI_BRIDGE_RESOURCE_NUM]; + struct resource fixed_range_soft[PCI_BRIDGE_RESOURCE_NUM]; struct pci_ops *ops; /* Configuration access functions */ struct msi_controller *msi; /* MSI controller */ From patchwork Mon Mar 11 13:31:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847475 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 135BB1874 for ; Mon, 11 Mar 2019 13:32:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 634132948E for ; Mon, 11 Mar 2019 13:31:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6167F29491; Mon, 11 Mar 2019 13:31:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E817B29493 for ; Mon, 11 Mar 2019 13:31:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727708AbfCKNbp (ORCPT ); Mon, 11 Mar 2019 09:31:45 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:50956 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727669AbfCKNbp (ORCPT ); Mon, 11 Mar 2019 09:31:45 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 4013C41860; Mon, 11 Mar 2019 13:31:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311102; x=1554125503; bh=ES4IHYHawY+rGNAmCsSDSgYU25ke2e/UjSw 9HcjfDXE=; b=C/jBGkrTep+X72mxk3fx8zgYxE9ykTcnb9p1pCoQRm5n117jRAB PvE9q5+g09D3y3qIg8MQUHUjKJPIA0LlyaAsHcH44w5J5pPUxL6908ORCOvel86T yur4iLHhFKDFwzHr2Qx9/SafCGFpH70CfkWqdgq/SgzqtG5P0QUz1l04= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yF24H1U3X0Y8; Mon, 11 Mar 2019 16:31:42 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 0873E419E9; Mon, 11 Mar 2019 16:31:34 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:33 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 18/21] PCI: Make sure bridge windows include their fixed BARs Date: Mon, 11 Mar 2019 16:31:19 +0300 Message-ID: <20190311133122.11417-19-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Consider previously calculated boundaries when allocating a bridge window, setting the lowest allowed address and checking the result. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/bus.c | 2 +- drivers/pci/setup-res.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index a9784144d6f2..ce2d2aeedbd3 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -192,7 +192,7 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res, * this is an already-configured bridge window, its start * overrides "min". */ - if (avail.start) + if (min_used < avail.start) min_used = avail.start; max = avail.end; diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 732d18f60f1b..04442339548d 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -248,9 +248,22 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, struct resource *res = dev->resource + resno; resource_size_t min; int ret; + resource_size_t start = (resource_size_t)-1; + resource_size_t end = 0; min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM; + if (dev->subordinate && resno >= PCI_BRIDGE_RESOURCES) { + struct pci_bus *child_bus = dev->subordinate; + int b_resno = resno - PCI_BRIDGE_RESOURCES; + resource_size_t soft_start = child_bus->fixed_range_soft[b_resno].start; + + start = child_bus->fixed_range_hard[b_resno].start; + end = child_bus->fixed_range_hard[b_resno].end; + if (start < end) + min = soft_start; + } + /* * First, try exact prefetching match. Even if a 64-bit * prefetchable bridge window is below 4GB, we can't put a 32-bit @@ -262,7 +275,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, IORESOURCE_PREFETCH | IORESOURCE_MEM_64, pcibios_align_resource, dev); if (ret == 0) - return 0; + goto check_fixed; /* * If the prefetchable window is only 32 bits wide, we can put @@ -274,7 +287,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, IORESOURCE_PREFETCH, pcibios_align_resource, dev); if (ret == 0) - return 0; + goto check_fixed; } /* @@ -287,6 +300,20 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, ret = pci_bus_alloc_resource(bus, res, size, align, min, 0, pcibios_align_resource, dev); +check_fixed: + if (ret == 0 && start < end) { + if (res->start > start || res->end < end) { + dev_err(&bus->dev, "%s: fixed area 0x%llx-0x%llx for %s doesn't fit in the allocated %pR (0x%llx-0x%llx)", + __func__, + (unsigned long long)start, (unsigned long long)end, + dev_name(&dev->dev), + res, (unsigned long long)res->start, + (unsigned long long)res->end); + release_resource(res); + return -1; + } + } + return ret; } From patchwork Mon Mar 11 13:31:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847481 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 981A51823 for ; Mon, 11 Mar 2019 13:32:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 382262930C for ; Mon, 11 Mar 2019 13:31:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 35F4C292DC; Mon, 11 Mar 2019 13:31:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5DB929495 for ; Mon, 11 Mar 2019 13:31:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727630AbfCKNbr (ORCPT ); Mon, 11 Mar 2019 09:31:47 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:51026 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727742AbfCKNbq (ORCPT ); Mon, 11 Mar 2019 09:31:46 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 5D71041988; Mon, 11 Mar 2019 13:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311103; x=1554125504; bh=MjIvT+kNNwGlIGwMxKj5uZhAYpA426wYUCt RLWth4v4=; b=pc1kZ9xXkA/eLuPWaDBEph52JVTsWQ1sp+hDa9pa92G44joqlqj Gg/4Rh8kMMk6s8ovNf0AMJ2OlRu1jeA2ipb1yIJpqg1PZI1qnzjCJ5jCPKDUqrR/ 0hGOQKv0orw9XiJ/Xq/2KV1JDwNcl7fUShywHwobYGQqdpzaL9nO4puU= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n-xtV4EdcOCb; Mon, 11 Mar 2019 16:31:43 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 3AE57419EB; Mon, 11 Mar 2019 16:31:34 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:33 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 19/21] PCI: Prioritize fixed BAR assigning over the movable ones Date: Mon, 11 Mar 2019 16:31:20 +0300 Message-ID: <20190311133122.11417-20-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The allocated bridge windows are big enough to house all the children bridges and BARs, but the fixed resources must be assigned first, so the movable ones later divide the rest of the window. That's the assignment order: 1. Bridge windows with fixed areas; 2. Fixed BARs; 3. The rest of BARs and bridge windows. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 69 ++++++++++++++++++++++++++++++++--------- 1 file changed, 55 insertions(+), 14 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index f4737339d5ec..932a6c020d10 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -272,31 +272,54 @@ static void reassign_resources_sorted(struct list_head *realloc_head, } } -/** - * assign_requested_resources_sorted() - satisfy resource requests - * - * @head : head of the list tracking requests for resources - * @fail_head : head of the list tracking requests that could - * not be allocated - * - * Satisfy resource requests of each element in the list. Add - * requests that could not satisfied to the failed_list. - */ -static void assign_requested_resources_sorted(struct list_head *head, - struct list_head *fail_head) +enum assign_step { + assign_fixed_bridge_windows, + assign_fixed_resources, + assign_float_resources, +}; + +static void _assign_requested_resources_sorted(struct list_head *head, + struct list_head *fail_head, + enum assign_step step) { struct resource *res; struct pci_dev_resource *dev_res; int idx; list_for_each_entry(dev_res, head, list) { + bool is_fixed; + bool is_fixed_bridge; + bool is_bridge; + if (pci_dev_is_ignored(dev_res->dev)) continue; res = dev_res->res; + if (!resource_size(res)) + continue; + idx = res - &dev_res->dev->resource[0]; - if (resource_size(res) && - pci_assign_resource(dev_res->dev, idx)) { + is_fixed = res->flags & IORESOURCE_PCI_FIXED; + is_bridge = dev_res->dev->subordinate && idx >= PCI_BRIDGE_RESOURCES; + + if (is_bridge) { + struct pci_bus *child = dev_res->dev->subordinate; + int b_res_idx = pci_get_bridge_resource_idx(res); + struct resource *fixed_res = &child->fixed_range_hard[b_res_idx]; + + is_fixed_bridge = fixed_res->start < fixed_res->end; + } else { + is_fixed_bridge = false; + } + + if (assign_fixed_bridge_windows == step && !is_fixed_bridge) + continue; + else if (assign_fixed_resources == step && (!is_fixed || is_bridge)) + continue; + else if (assign_float_resources == step && (is_fixed || is_fixed_bridge)) + continue; + + if (pci_assign_resource(dev_res->dev, idx)) { if (fail_head) { /* * if the failed res is for ROM BAR, and it will @@ -315,6 +338,24 @@ static void assign_requested_resources_sorted(struct list_head *head, } } +/** + * assign_requested_resources_sorted() - satisfy resource requests + * + * @head : head of the list tracking requests for resources + * @fail_head : head of the list tracking requests that could + * not be allocated + * + * Satisfy resource requests of each element in the list. Add + * requests that could not satisfied to the failed_list. + */ +static void assign_requested_resources_sorted(struct list_head *head, + struct list_head *fail_head) +{ + _assign_requested_resources_sorted(head, fail_head, assign_fixed_bridge_windows); + _assign_requested_resources_sorted(head, fail_head, assign_fixed_resources); + _assign_requested_resources_sorted(head, fail_head, assign_float_resources); +} + static unsigned long pci_fail_res_type_mask(struct list_head *fail_head) { struct pci_dev_resource *fail_res; From patchwork Mon Mar 11 13:31:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847479 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7005D1669 for ; Mon, 11 Mar 2019 13:32:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F052329090 for ; Mon, 11 Mar 2019 13:31:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EDB8429300; Mon, 11 Mar 2019 13:31:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93AF7291E0 for ; Mon, 11 Mar 2019 13:31:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727594AbfCKNbq (ORCPT ); Mon, 11 Mar 2019 09:31:46 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:51024 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727673AbfCKNbq (ORCPT ); Mon, 11 Mar 2019 09:31:46 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 30462419E9; Mon, 11 Mar 2019 13:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311103; x=1554125504; bh=OgLo7/CRt0OwDQd8jzrs/7oiC/sWradzhSM CAbKv7SA=; b=MfryYkYgoL0ESvNPca5BbnafmrLlVRzCJFl32zCyjwNHzYgjMjG VFwmi2pgHk5SADoAab0Jp6Q8TZAlZNTFFE5TxYC0P+6sN00oxtrv+PzMv51B0r0S Pfwi5q3S1ov0Snx6Vz8e4S8kZ0sqW2vpS0bSrLG1eZPnzXjw6eN5rjAo= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mV7G5ccgtkaw; Mon, 11 Mar 2019 16:31:43 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id BB5DB41988; Mon, 11 Mar 2019 16:31:34 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:34 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 20/21] PCI: pciehp: Add support for the movable BARs feature Date: Mon, 11 Mar 2019 16:31:21 +0300 Message-ID: <20190311133122.11417-21-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With movable BARs, adding a hotplugged device may affect all the PCIe domain starting from the root, so use a pci_rescan_bus() function which handles the rearrangement of existing BARs and bridge windows. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/hotplug/pciehp_pci.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/drivers/pci/hotplug/pciehp_pci.c b/drivers/pci/hotplug/pciehp_pci.c index b9c1396db6fe..7c0871db5bae 100644 --- a/drivers/pci/hotplug/pciehp_pci.c +++ b/drivers/pci/hotplug/pciehp_pci.c @@ -56,12 +56,16 @@ int pciehp_configure_device(struct controller *ctrl) goto out; } - for_each_pci_bridge(dev, parent) - pci_hp_add_bridge(dev); + if (pci_movable_bars_enabled()) { + pci_rescan_bus(parent); + } else { + for_each_pci_bridge(dev, parent) + pci_hp_add_bridge(dev); - pci_assign_unassigned_bridge_resources(bridge); - pcie_bus_configure_settings(parent); - pci_bus_add_devices(parent); + pci_assign_unassigned_bridge_resources(bridge); + pcie_bus_configure_settings(parent); + pci_bus_add_devices(parent); + } out: pci_unlock_rescan_remove(); From patchwork Mon Mar 11 13:31:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10847483 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CE0E14DE for ; Mon, 11 Mar 2019 13:32:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC37629298 for ; Mon, 11 Mar 2019 13:31:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AA16029351; Mon, 11 Mar 2019 13:31:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CC82292A7 for ; Mon, 11 Mar 2019 13:31:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727989AbfCKNbw (ORCPT ); Mon, 11 Mar 2019 09:31:52 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:51076 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727636AbfCKNbw (ORCPT ); Mon, 11 Mar 2019 09:31:52 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id C8A0E41987; Mon, 11 Mar 2019 13:31:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1552311104; x=1554125505; bh=bm2osjHKk/ccyYGLhVDVztUr8WmMcO+SQF+ o6YpHL+k=; b=b2Dts59HJ0JQ0nA2uwE3UNTvgLjDHjuNqy0s0L+Necqh6HQg+lD CUhuPsxobDCppKMAzePp7BgRiY3jYVtjWhe3K2lxptqh7aB7Gt08eiL+qIim6lGU C4ycYTPQW7AWWbkTJYjyetDTTMk+GKP9gJbKRgg6SYMzZLOTEcTP56eY= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vZCoBgRh6kbV; Mon, 11 Mar 2019 16:31:44 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id F18EE4198C; Mon, 11 Mar 2019 16:31:34 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 11 Mar 2019 16:31:34 +0300 From: Sergey Miroshnichenko To: , CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC v4 21/21] powerpc/pci: Fix crash with enabled movable BARs Date: Mon, 11 Mar 2019 16:31:22 +0300 Message-ID: <20190311133122.11417-22-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190311133122.11417-1-s.miroshnichenko@yadro.com> References: <20190311133122.11417-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Check a resource for the UNSET flags. Signed-off-by: Sergey Miroshnichenko --- arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index fa6af52b5219..353b36727f6a 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -2977,7 +2977,8 @@ static void pnv_ioda_setup_pe_res(struct pnv_ioda_pe *pe, int index; int64_t rc; - if (!res || !res->flags || res->start > res->end) + if (!res || !res->flags || res->start > res->end || + (res->flags & IORESOURCE_UNSET)) return; if (res->flags & IORESOURCE_IO) {