From patchwork Fri Sep 14 16:14:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10601017 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D65931750 for ; Fri, 14 Sep 2018 16:21:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B634F2AD92 for ; Fri, 14 Sep 2018 16:21:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9C1D2BBE0; Fri, 14 Sep 2018 16:21:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 514D52AD92 for ; Fri, 14 Sep 2018 16:21:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726845AbeINVgo (ORCPT ); Fri, 14 Sep 2018 17:36:44 -0400 Received: from mta-01.yadro.com ([89.207.88.251]:60240 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728012AbeINVgo (ORCPT ); Fri, 14 Sep 2018 17:36:44 -0400 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 8F0F3418F7; Fri, 14 Sep 2018 16:14:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:mime-version:x-mailer:message-id:date :date:subject:subject:from:from:received:received:received; s= mta-01; t=1536941650; x=1538756051; bh=nQwMhNB4kBjF8lz5AE2gl+WJI jBmuNP9Q+NhsTQOUo4=; b=b08nVOxclu7kb0uCi7E3MnEwjaZ0axGXdNMG5Gcwe vaBHO5qoonZu0W/SnCYUovhxeOLzakq/gnuVQG69X7WCfjWbHSao+PAMt+5FtWS1 RMUnOLrqfPSUz9VMknDQm2qHkx/mKm+kFQEIFEtxgsS6DMlm5wr99CkrnLGfixsD jY= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vKtZ7Ibk5jvG; Fri, 14 Sep 2018 19:14:10 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id ADF6440311; Fri, 14 Sep 2018 19:14:10 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Fri, 14 Sep 2018 19:14:10 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , , Sergey Miroshnichenko Subject: [PATCH RFC 0/4] PCI: Allow BAR movement during hotplug Date: Fri, 14 Sep 2018 19:14:00 +0300 Message-ID: <20180914161404.4685-1-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the firmware or kernel has arranged memory for PCIe devices in a way that doesn't provide enough space for BARs of a new hotplugged device, the kernel can pause the drivers of the "obstructing" devices and move their BARs, so new BARs can fit into the created hole. When a driver is un-paused by the kernel after the PCIe rescan, it should check if its BARs had changed and ioremap() them if needed. BARs are moved not directly, but by releasing bridge resources, then sorting and assigning them back, similarly to the initial PCIe topology scan during system boot (when pci=realloc is passed). Pausing drivers is performed via reset_prepare() and reset_done() callbacks of struct pci_error_handlers. Drivers should pause during rescan not only because of potential movement of their BARs, but also because of possible updating of the bridge windows. This patchset is a part of our work on adding support for hotplugging bridges full of NVME devices (without special requirement such as Hot-Plug Controller, reservation of bus numbers and memory regions by firmware, etc.), should I also add here the patch that adds support of moving BARs to the NVME driver? Sergey Miroshnichenko (4): PCI: hotplug: Add parameter to put devices to reset during rescan PCI: Release and reassign resources from the root during rescan PCI: Invalidate the released BAR resources PCI: Fix writing invalid BARs during pci_restore_state() .../admin-guide/kernel-parameters.txt | 6 ++ drivers/pci/pci.c | 4 +- drivers/pci/pci.h | 8 ++ drivers/pci/probe.c | 78 ++++++++++++++++++- drivers/pci/setup-bus.c | 33 +++++--- include/linux/pci.h | 1 + 6 files changed, 119 insertions(+), 11 deletions(-)