From patchwork Thu Oct 11 18:34:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10637117 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58E2369B1 for ; Thu, 11 Oct 2018 18:37:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 129DE2BC00 for ; Thu, 11 Oct 2018 18:37:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0669D2BCC2; Thu, 11 Oct 2018 18:37:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9E4CD2BD0C for ; Thu, 11 Oct 2018 18:37:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728092AbeJLCFk (ORCPT ); Thu, 11 Oct 2018 22:05:40 -0400 Received: from mga17.intel.com ([192.55.52.151]:50095 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728581AbeJLCFk (ORCPT ); Thu, 11 Oct 2018 22:05:40 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Oct 2018 11:37:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,369,1534834800"; d="scan'208";a="80468695" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga007.jf.intel.com with ESMTP; 11 Oct 2018 11:37:15 -0700 From: Keith Busch To: linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Keith Busch Subject: [PATCHv2 1/4] PCI/AER: Reuse existing pcie_port_find_device() interface Date: Thu, 11 Oct 2018 12:34:10 -0600 Message-Id: <20181011183413.13183-2-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181011183413.13183-1-keith.busch@intel.com> References: <20181011183413.13183-1-keith.busch@intel.com> Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The port services driver already provides a method to find the pcie_device for a service. Export that function, use it from the aer_inject module, and remove the duplicate functionality. Signed-off-by: Keith Busch --- drivers/pci/pcie/aer_inject.c | 25 ++++--------------------- drivers/pci/pcie/portdrv_core.c | 1 + 2 files changed, 5 insertions(+), 21 deletions(-) diff --git a/drivers/pci/pcie/aer_inject.c b/drivers/pci/pcie/aer_inject.c index 0eb24346cad3..f40ed5867c89 100644 --- a/drivers/pci/pcie/aer_inject.c +++ b/drivers/pci/pcie/aer_inject.c @@ -303,32 +303,13 @@ static int pci_bus_set_aer_ops(struct pci_bus *bus) return 0; } -static int find_aer_device_iter(struct device *device, void *data) -{ - struct pcie_device **result = data; - struct pcie_device *pcie_dev; - - if (device->bus == &pcie_port_bus_type) { - pcie_dev = to_pcie_device(device); - if (pcie_dev->service & PCIE_PORT_SERVICE_AER) { - *result = pcie_dev; - return 1; - } - } - return 0; -} - -static int find_aer_device(struct pci_dev *dev, struct pcie_device **result) -{ - return device_for_each_child(&dev->dev, result, find_aer_device_iter); -} - static int aer_inject(struct aer_error_inj *einj) { struct aer_error *err, *rperr; struct aer_error *err_alloc = NULL, *rperr_alloc = NULL; struct pci_dev *dev, *rpdev; struct pcie_device *edev; + struct device *device; unsigned long flags; unsigned int devfn = PCI_DEVFN(einj->dev, einj->fn); int pos_cap_err, rp_pos_cap_err; @@ -464,7 +445,9 @@ static int aer_inject(struct aer_error_inj *einj) if (ret) goto out_put; - if (find_aer_device(rpdev, &edev)) { + device = pcie_port_find_device(rpdev, PCIE_PORT_SERVICE_AER); + if (device) { + edev = to_pcie_device(device); if (!get_service_data(edev)) { dev_warn(&edev->device, "aer_inject: AER service is not initialized\n"); diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c index 6542c48c7f59..f458ac9cb70c 100644 --- a/drivers/pci/pcie/portdrv_core.c +++ b/drivers/pci/pcie/portdrv_core.c @@ -486,6 +486,7 @@ struct device *pcie_port_find_device(struct pci_dev *dev, device = pdrvs.dev; return device; } +EXPORT_SYMBOL_GPL(pcie_port_find_device); /** * pcie_port_device_remove - unregister PCI Express port service devices From patchwork Thu Oct 11 18:34:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10637121 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DA6E69B1 for ; Thu, 11 Oct 2018 18:37:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 311A62BD0C for ; Thu, 11 Oct 2018 18:37:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25A892BE33; Thu, 11 Oct 2018 18:37:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B66772BD0C for ; Thu, 11 Oct 2018 18:37:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728849AbeJLCFl (ORCPT ); Thu, 11 Oct 2018 22:05:41 -0400 Received: from mga17.intel.com ([192.55.52.151]:50095 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728581AbeJLCFl (ORCPT ); Thu, 11 Oct 2018 22:05:41 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Oct 2018 11:37:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,369,1534834800"; d="scan'208";a="80468700" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga007.jf.intel.com with ESMTP; 11 Oct 2018 11:37:15 -0700 From: Keith Busch To: linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Keith Busch Subject: [PATCHv2 2/4] PCI/AER: Abstract AER interrupt handling Date: Thu, 11 Oct 2018 12:34:11 -0600 Message-Id: <20181011183413.13183-3-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181011183413.13183-1-keith.busch@intel.com> References: <20181011183413.13183-1-keith.busch@intel.com> Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The aer_inject module was directly calling aer_irq(). This required the AER driver export its private IRQ handler for no other reason than to support error injection. A driver should not have to expose its private interfaces, so use the IRQ subsystem to route injection to the AER driver, and make aer_irq() a private interface. This provides additional benefits: First, directly calling the IRQ handler bypassed the IRQ subsytem so the injection wasn't really synthesizing what happens if a shared AER interrupt occurs. The error injection had to provide the callback data directly, which may be racing with a removal that is freeing that structure. The IRQ subsystem can handle that race. Finally, using the IRQ subsystem automatically reacts to threaded IRQs, keeping the error injection abstracted from that implementation detail. Signed-off-by: Keith Busch --- drivers/pci/pcie/aer.c | 3 +-- drivers/pci/pcie/aer_inject.c | 5 ++++- drivers/pci/pcie/portdrv.h | 4 ---- 3 files changed, 5 insertions(+), 7 deletions(-) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 90b53abf621d..a90a9194ac4a 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1229,7 +1229,7 @@ static irqreturn_t aer_isr(int irq, void *context) * * Invoked when Root Port detects AER messages. */ -irqreturn_t aer_irq(int irq, void *context) +static irqreturn_t aer_irq(int irq, void *context) { struct pcie_device *pdev = (struct pcie_device *)context; struct aer_rpc *rpc = get_service_data(pdev); @@ -1249,7 +1249,6 @@ irqreturn_t aer_irq(int irq, void *context) return IRQ_WAKE_THREAD; } -EXPORT_SYMBOL_GPL(aer_irq); static int set_device_error_reporting(struct pci_dev *dev, void *data) { diff --git a/drivers/pci/pcie/aer_inject.c b/drivers/pci/pcie/aer_inject.c index f40ed5867c89..726987f8d53c 100644 --- a/drivers/pci/pcie/aer_inject.c +++ b/drivers/pci/pcie/aer_inject.c @@ -14,6 +14,7 @@ #include #include +#include #include #include #include @@ -457,7 +458,9 @@ static int aer_inject(struct aer_error_inj *einj) dev_info(&edev->device, "aer_inject: Injecting errors %08x/%08x into device %s\n", einj->cor_status, einj->uncor_status, pci_name(dev)); - aer_irq(-1, edev); + local_irq_disable(); + generic_handle_irq(edev->irq); + local_irq_enable(); } else { pci_err(rpdev, "aer_inject: AER device not found\n"); ret = -ENODEV; diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h index abfdc2ae7979..e495f04394d0 100644 --- a/drivers/pci/pcie/portdrv.h +++ b/drivers/pci/pcie/portdrv.h @@ -151,10 +151,6 @@ static inline int pcie_aer_get_firmware_first(struct pci_dev *pci_dev) } #endif -#ifdef CONFIG_PCIEAER -irqreturn_t aer_irq(int irq, void *context); -#endif - struct pcie_port_service_driver *pcie_port_find_service(struct pci_dev *dev, u32 service); struct device *pcie_port_find_device(struct pci_dev *dev, u32 service); From patchwork Thu Oct 11 18:34:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10637119 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CF7017E3 for ; Thu, 11 Oct 2018 18:37:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 106BD2BDB6 for ; Thu, 11 Oct 2018 18:37:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 052D02BE33; Thu, 11 Oct 2018 18:37:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 955E92BECB for ; Thu, 11 Oct 2018 18:37:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728581AbeJLCFl (ORCPT ); Thu, 11 Oct 2018 22:05:41 -0400 Received: from mga17.intel.com ([192.55.52.151]:50095 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728761AbeJLCFl (ORCPT ); Thu, 11 Oct 2018 22:05:41 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Oct 2018 11:37:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,369,1534834800"; d="scan'208";a="80468705" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga007.jf.intel.com with ESMTP; 11 Oct 2018 11:37:16 -0700 From: Keith Busch To: linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Keith Busch Subject: [PATCHv2 3/4] PCI/AER: Refactor error injection fallbacks Date: Thu, 11 Oct 2018 12:34:12 -0600 Message-Id: <20181011183413.13183-4-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181011183413.13183-1-keith.busch@intel.com> References: <20181011183413.13183-1-keith.busch@intel.com> Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch just moves the bus ops fallback into separate functions. No functional change here. Signed-off-by: Keith Busch --- drivers/pci/pcie/aer_inject.c | 66 +++++++++++++++++++++++++------------------ 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/drivers/pci/pcie/aer_inject.c b/drivers/pci/pcie/aer_inject.c index 726987f8d53c..95d4759664b3 100644 --- a/drivers/pci/pcie/aer_inject.c +++ b/drivers/pci/pcie/aer_inject.c @@ -176,14 +176,48 @@ static u32 *find_pci_config_dword(struct aer_error *err, int where, return target; } +static int aer_inj_read(struct pci_bus *bus, unsigned int devfn, int where, + int size, u32 *val) +{ + struct pci_ops *ops, *my_ops; + int rv; + + ops = __find_pci_bus_ops(bus); + if (!ops) + return -1; + + my_ops = bus->ops; + bus->ops = ops; + rv = ops->read(bus, devfn, where, size, val); + bus->ops = my_ops; + + return rv; +} + +static int aer_inj_write(struct pci_bus *bus, unsigned int devfn, int where, + int size, u32 val) +{ + struct pci_ops *ops, *my_ops; + int rv; + + ops = __find_pci_bus_ops(bus); + if (!ops) + return -1; + + my_ops = bus->ops; + bus->ops = ops; + rv = ops->write(bus, devfn, where, size, val); + bus->ops = my_ops; + + return rv; +} + static int aer_inj_read_config(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val) { u32 *sim; struct aer_error *err; unsigned long flags; - struct pci_ops *ops; - struct pci_ops *my_ops; int domain; int rv; @@ -204,18 +238,7 @@ static int aer_inj_read_config(struct pci_bus *bus, unsigned int devfn, return 0; } out: - ops = __find_pci_bus_ops(bus); - /* - * pci_lock must already be held, so we can directly - * manipulate bus->ops. Many config access functions, - * including pci_generic_config_read() require the original - * bus->ops be installed to function, so temporarily put them - * back. - */ - my_ops = bus->ops; - bus->ops = ops; - rv = ops->read(bus, devfn, where, size, val); - bus->ops = my_ops; + rv = aer_inj_read(bus, devfn, where, size, val); spin_unlock_irqrestore(&inject_lock, flags); return rv; } @@ -227,8 +250,6 @@ static int aer_inj_write_config(struct pci_bus *bus, unsigned int devfn, struct aer_error *err; unsigned long flags; int rw1cs; - struct pci_ops *ops; - struct pci_ops *my_ops; int domain; int rv; @@ -252,18 +273,7 @@ static int aer_inj_write_config(struct pci_bus *bus, unsigned int devfn, return 0; } out: - ops = __find_pci_bus_ops(bus); - /* - * pci_lock must already be held, so we can directly - * manipulate bus->ops. Many config access functions, - * including pci_generic_config_write() require the original - * bus->ops be installed to function, so temporarily put them - * back. - */ - my_ops = bus->ops; - bus->ops = ops; - rv = ops->write(bus, devfn, where, size, val); - bus->ops = my_ops; + rv = aer_inj_write(bus, devfn, where, size, val); spin_unlock_irqrestore(&inject_lock, flags); return rv; } From patchwork Thu Oct 11 18:34:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10637123 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B072C13AD for ; Thu, 11 Oct 2018 18:37:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A32252BC00 for ; Thu, 11 Oct 2018 18:37:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 977302BE33; Thu, 11 Oct 2018 18:37:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11DF12BC00 for ; Thu, 11 Oct 2018 18:37:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728761AbeJLCGD (ORCPT ); Thu, 11 Oct 2018 22:06:03 -0400 Received: from mga14.intel.com ([192.55.52.115]:32882 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728164AbeJLCGD (ORCPT ); Thu, 11 Oct 2018 22:06:03 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Oct 2018 11:37:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,369,1534834800"; d="scan'208";a="80468709" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga007.jf.intel.com with ESMTP; 11 Oct 2018 11:37:16 -0700 From: Keith Busch To: linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Keith Busch Subject: [PATCHv2 4/4] PCI/AER: Covertly inject errors with ftrace hooks Date: Thu, 11 Oct 2018 12:34:13 -0600 Message-Id: <20181011183413.13183-5-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181011183413.13183-1-keith.busch@intel.com> References: <20181011183413.13183-1-keith.busch@intel.com> Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The aer_inject module had been intercepting config requests by overwriting the config accessor operations in the pci_bus ops. This has several issues. First, the module was tracking kernel objects unbeknownst to the drivers that own them. The kernel may free those devices, leaving the AER inject module holding stale references and no way to know that happened. Second, the PCI enumeration has child devices inherit pci_bus ops from the parent bus. Since errors may lead to link resets that trigger re-enumeration, the child devices would inherit operations that don't know about the devices using them, causing kernel crashes. Finally, CONFIG_PCI_LOCKLESS_CONFIG doesn't block accessing the pci_bus ops, so it's racing with potential in-flight config requests. This patch uses a different error injection approach leveraging ftrace to thunk the config space functions. If the kernel and architecture are capable, the ftrace hook will overwrite the processor's function call address with the error injection function. This discreet error injection doesn't modify or track driver structures, fixing the issues with the current method. If either the kernel config or platform arch do not support the necessary ftrace capabilities, the aer_inject module will fallback to the older way so that it may continue to be used as before. Signed-off-by: Keith Busch --- drivers/pci/pcie/aer_inject.c | 152 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 151 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/aer_inject.c b/drivers/pci/pcie/aer_inject.c index 95d4759664b3..f08bd20e8907 100644 --- a/drivers/pci/pcie/aer_inject.c +++ b/drivers/pci/pcie/aer_inject.c @@ -15,6 +15,10 @@ #include #include #include +#include +#include +#include +#include #include #include #include @@ -65,6 +69,8 @@ struct pci_bus_ops { struct pci_ops *ops; }; +static bool legacy; + static LIST_HEAD(einjected); static LIST_HEAD(pci_bus_ops_list); @@ -298,6 +304,9 @@ static int pci_bus_set_aer_ops(struct pci_bus *bus) struct pci_bus_ops *bus_ops; unsigned long flags; + if (!legacy) + return 0; + bus_ops = kmalloc(sizeof(*bus_ops), GFP_KERNEL); if (!bus_ops) return -ENOMEM; @@ -514,9 +523,148 @@ static struct miscdevice aer_inject_device = { .fops = &aer_inject_fops, }; +static asmlinkage int (*read_config_dword)(struct pci_bus *bus, + unsigned int devfn, + int where, u32 *val); +static asmlinkage int (*write_config_dword)(struct pci_bus *bus, + unsigned int devfn, + int where, u32 val); +struct aer_hook { + struct ftrace_ops ops; + const char *name; + void *function; + void *original; + unsigned long address; +}; + +static int asmlinkage ainj_read_config_dword(struct pci_bus *bus, + unsigned int devfn, int where, u32 *val) +{ + if (!aer_inj_read_config(bus, devfn, where, sizeof(u32), (u32 *)val)) + return 0; + return read_config_dword(bus, devfn, where, val); +} + +static int asmlinkage ainj_write_config_dword(struct pci_bus *bus, + unsigned int devfn, int where, u32 val) +{ + if (!aer_inj_write_config(bus, devfn, where, sizeof(u32), val)) + return 0; + return write_config_dword(bus, devfn, where, val); +} + +static int aer_inject_resolve_hook_address(struct aer_hook *hook) +{ + hook->address = kallsyms_lookup_name(hook->name); + + if (!hook->address) { + pr_warn("unresolved symbol: %s\n", hook->name); + return -ENOENT; + } + *((unsigned long*) hook->original) = hook->address + MCOUNT_INSN_SIZE; + + return 0; +} + +static void notrace aer_inject_ftrace_thunk(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *ops, struct pt_regs *regs) +{ + struct aer_hook *hook = ops->private; + instruction_pointer_set(regs, (unsigned long)hook->function); +} + +static int aer_inject_install_hook(struct aer_hook *hook) +{ + int err; + + err = aer_inject_resolve_hook_address(hook); + if (err) + return err; + + hook->ops.private = hook; + hook->ops.func = aer_inject_ftrace_thunk; + hook->ops.flags = FTRACE_OPS_FL_SAVE_REGS | + FTRACE_OPS_FL_RECURSION_SAFE | + FTRACE_OPS_FL_IPMODIFY; + err = ftrace_set_filter_ip(&hook->ops, hook->address, 0, 0); + if (err) { + pr_warn("ftrace_set_filter_ip() failed: %d\n", err); + return err; + } + + err = register_ftrace_function(&hook->ops); + if (err) { + pr_warn("register_ftrace_function() failed: %d\n", err); + ftrace_set_filter_ip(&hook->ops, hook->address, 1, 0); + return err; + } + return 0; +} + +static void aer_inject_remove_hook(struct aer_hook *hook) +{ + int err; + + err = unregister_ftrace_function(&hook->ops); + if (err) + pr_warn("unregister_ftrace_function() failed: %d\n", err); + + err = ftrace_set_filter_ip(&hook->ops, hook->address, 1, 0); + if (err) + pr_warn("ftrace_set_filter_ip() failed: %d\n", err); +} + +static int aer_inject_install_hooks(struct aer_hook *hooks, size_t count) +{ + int err, i; + + for (i = 0; i < count; i++) { + err = aer_inject_install_hook(&hooks[i]); + if (err) + goto error; + } + return 0; +error: + for (i--; i >= 0; i--) + aer_inject_remove_hook(&hooks[i]); + return err; +} + +static void aer_inject_remove_hooks(struct aer_hook *hooks, size_t count) +{ + int i; + + for (i = 0; i < count; i++) + aer_inject_remove_hook(&hooks[i]); +} + +static struct aer_hook aer_hooks[] = { + { + .name = "pci_bus_read_config_dword", + .function = ainj_read_config_dword, + .original = &read_config_dword, + }, + { + .name = "pci_bus_write_config_dword", + .function = ainj_write_config_dword, + .original = &write_config_dword, + }, +}; + static int __init aer_inject_init(void) { - return misc_register(&aer_inject_device); + int err; + + err = misc_register(&aer_inject_device); + if (err) + return err; + + err = aer_inject_install_hooks(aer_hooks, ARRAY_SIZE(aer_hooks)); + if (err) { + pr_info("aer_inject: using legacy error inject method\n"); + legacy = true; + } + return 0; } static void __exit aer_inject_exit(void) @@ -525,6 +673,8 @@ static void __exit aer_inject_exit(void) unsigned long flags; struct pci_bus_ops *bus_ops; + if (!legacy) + aer_inject_remove_hooks(aer_hooks, ARRAY_SIZE(aer_hooks)); misc_deregister(&aer_inject_device); while ((bus_ops = pci_bus_ops_pop())) {