From patchwork Fri Jun 8 20:05:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 10455139 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 55F90601D4 for ; Fri, 8 Jun 2018 20:07:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 535FB294C8 for ; Fri, 8 Jun 2018 20:07:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4813E295ED; Fri, 8 Jun 2018 20:07:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A47F0294C8 for ; Fri, 8 Jun 2018 20:07:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752974AbeFHUFI (ORCPT ); Fri, 8 Jun 2018 16:05:08 -0400 Received: from mail.kernel.org ([198.145.29.99]:48824 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752910AbeFHUFF (ORCPT ); Fri, 8 Jun 2018 16:05:05 -0400 Received: from localhost (unknown [150.199.191.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0313E20874; Fri, 8 Jun 2018 20:05:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1528488305; bh=/NPlGUMxMcKfOC8MTQsb3nshP9Atq37zD357qMN/0vU=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=pvmn6SCIdLHsd5+XgQKbmVeXAmuM2fkMnshxlSWP7B8HfEc0xqUBxvfNtoMoCy+z+ CyL8XJfvCJlyFuExEcjmTnqLGfPikjVUyiTQK7vN4Qsj+8/o7ODWToQItzt9IZC7Xq AYFHlDUoJts9agTJFlodThpzPdHBLj4feA5ihxiI= Subject: [PATCH v1 03/13] PCI/AER: Squash aerdrv_core.c into aerdrv.c From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: Keith Busch , Borislav Petkov , Oza Pawandeep , linux-kernel@vger.kernel.org Date: Fri, 08 Jun 2018 15:05:04 -0500 Message-ID: <152848830414.11888.9793521055800870273.stgit@bhelgaas-glaptop.roam.corp.google.com> In-Reply-To: <152848785553.11888.12243539903985770441.stgit@bhelgaas-glaptop.roam.corp.google.com> References: <152848785553.11888.12243539903985770441.stgit@bhelgaas-glaptop.roam.corp.google.com> User-Agent: StGit/0.18 MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bjorn Helgaas Squash aerdrv_core.c into aerdrv.c. No functional change intended. Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/aer/Makefile | 2 drivers/pci/pcie/aer/aerdrv.c | 474 ++++++++++++++++++++++++++++++++++ drivers/pci/pcie/aer/aerdrv.h | 1 drivers/pci/pcie/aer/aerdrv_core.c | 496 ------------------------------------ 4 files changed, 475 insertions(+), 498 deletions(-) delete mode 100644 drivers/pci/pcie/aer/aerdrv_core.c diff --git a/drivers/pci/pcie/aer/Makefile b/drivers/pci/pcie/aer/Makefile index 09bd890875a3..c85bd0851903 100644 --- a/drivers/pci/pcie/aer/Makefile +++ b/drivers/pci/pcie/aer/Makefile @@ -7,7 +7,7 @@ obj-$(CONFIG_PCIEAER) += aerdriver.o obj-$(CONFIG_PCIE_ECRC) += ecrc.o -aerdriver-objs := aerdrv_errprint.o aerdrv_core.o aerdrv.o +aerdriver-objs := aerdrv_errprint.o aerdrv.o aerdriver-$(CONFIG_ACPI) += aerdrv_acpi.o obj-$(CONFIG_PCIEAER_INJECT) += aer_inject.o diff --git a/drivers/pci/pcie/aer/aerdrv.c b/drivers/pci/pcie/aer/aerdrv.c index 8d3bc28821ba..c185069ef880 100644 --- a/drivers/pci/pcie/aer/aerdrv.c +++ b/drivers/pci/pcie/aer/aerdrv.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "aerdrv.h" @@ -35,6 +36,479 @@ bool pci_aer_available(void) return !pcie_aer_disable && pci_msi_enabled(); } +#define PCI_EXP_AER_FLAGS (PCI_EXP_DEVCTL_CERE | PCI_EXP_DEVCTL_NFERE | \ + PCI_EXP_DEVCTL_FERE | PCI_EXP_DEVCTL_URRE) + +int pci_enable_pcie_error_reporting(struct pci_dev *dev) +{ + if (pcie_aer_get_firmware_first(dev)) + return -EIO; + + if (!dev->aer_cap) + return -EIO; + + return pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_AER_FLAGS); +} +EXPORT_SYMBOL_GPL(pci_enable_pcie_error_reporting); + +int pci_disable_pcie_error_reporting(struct pci_dev *dev) +{ + if (pcie_aer_get_firmware_first(dev)) + return -EIO; + + return pcie_capability_clear_word(dev, PCI_EXP_DEVCTL, + PCI_EXP_AER_FLAGS); +} +EXPORT_SYMBOL_GPL(pci_disable_pcie_error_reporting); + +int pci_cleanup_aer_uncorrect_error_status(struct pci_dev *dev) +{ + int pos; + u32 status; + + pos = dev->aer_cap; + if (!pos) + return -EIO; + + pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, &status); + if (status) + pci_write_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, status); + + return 0; +} +EXPORT_SYMBOL_GPL(pci_cleanup_aer_uncorrect_error_status); + +int pci_cleanup_aer_error_status_regs(struct pci_dev *dev) +{ + int pos; + u32 status; + int port_type; + + if (!pci_is_pcie(dev)) + return -ENODEV; + + pos = dev->aer_cap; + if (!pos) + return -EIO; + + port_type = pci_pcie_type(dev); + if (port_type == PCI_EXP_TYPE_ROOT_PORT) { + pci_read_config_dword(dev, pos + PCI_ERR_ROOT_STATUS, &status); + pci_write_config_dword(dev, pos + PCI_ERR_ROOT_STATUS, status); + } + + pci_read_config_dword(dev, pos + PCI_ERR_COR_STATUS, &status); + pci_write_config_dword(dev, pos + PCI_ERR_COR_STATUS, status); + + pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, &status); + pci_write_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, status); + + return 0; +} + +int pci_aer_init(struct pci_dev *dev) +{ + dev->aer_cap = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR); + return pci_cleanup_aer_error_status_regs(dev); +} + +/** + * add_error_device - list device to be handled + * @e_info: pointer to error info + * @dev: pointer to pci_dev to be added + */ +static int add_error_device(struct aer_err_info *e_info, struct pci_dev *dev) +{ + if (e_info->error_dev_num < AER_MAX_MULTI_ERR_DEVICES) { + e_info->dev[e_info->error_dev_num] = dev; + e_info->error_dev_num++; + return 0; + } + return -ENOSPC; +} + +/** + * is_error_source - check whether the device is source of reported error + * @dev: pointer to pci_dev to be checked + * @e_info: pointer to reported error info + */ +static bool is_error_source(struct pci_dev *dev, struct aer_err_info *e_info) +{ + int pos; + u32 status, mask; + u16 reg16; + + /* + * When bus id is equal to 0, it might be a bad id + * reported by root port. + */ + if ((PCI_BUS_NUM(e_info->id) != 0) && + !(dev->bus->bus_flags & PCI_BUS_FLAGS_NO_AERSID)) { + /* Device ID match? */ + if (e_info->id == ((dev->bus->number << 8) | dev->devfn)) + return true; + + /* Continue id comparing if there is no multiple error */ + if (!e_info->multi_error_valid) + return false; + } + + /* + * When either + * 1) bus id is equal to 0. Some ports might lose the bus + * id of error source id; + * 2) bus flag PCI_BUS_FLAGS_NO_AERSID is set + * 3) There are multiple errors and prior ID comparing fails; + * We check AER status registers to find possible reporter. + */ + if (atomic_read(&dev->enable_cnt) == 0) + return false; + + /* Check if AER is enabled */ + pcie_capability_read_word(dev, PCI_EXP_DEVCTL, ®16); + if (!(reg16 & PCI_EXP_AER_FLAGS)) + return false; + + pos = dev->aer_cap; + if (!pos) + return false; + + /* Check if error is recorded */ + if (e_info->severity == AER_CORRECTABLE) { + pci_read_config_dword(dev, pos + PCI_ERR_COR_STATUS, &status); + pci_read_config_dword(dev, pos + PCI_ERR_COR_MASK, &mask); + } else { + pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, &status); + pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_MASK, &mask); + } + if (status & ~mask) + return true; + + return false; +} + +static int find_device_iter(struct pci_dev *dev, void *data) +{ + struct aer_err_info *e_info = (struct aer_err_info *)data; + + if (is_error_source(dev, e_info)) { + /* List this device */ + if (add_error_device(e_info, dev)) { + /* We cannot handle more... Stop iteration */ + /* TODO: Should print error message here? */ + return 1; + } + + /* If there is only a single error, stop iteration */ + if (!e_info->multi_error_valid) + return 1; + } + return 0; +} + +/** + * find_source_device - search through device hierarchy for source device + * @parent: pointer to Root Port pci_dev data structure + * @e_info: including detailed error information such like id + * + * Return true if found. + * + * Invoked by DPC when error is detected at the Root Port. + * Caller of this function must set id, severity, and multi_error_valid of + * struct aer_err_info pointed by @e_info properly. This function must fill + * e_info->error_dev_num and e_info->dev[], based on the given information. + */ +static bool find_source_device(struct pci_dev *parent, + struct aer_err_info *e_info) +{ + struct pci_dev *dev = parent; + int result; + + /* Must reset in this function */ + e_info->error_dev_num = 0; + + /* Is Root Port an agent that sends error message? */ + result = find_device_iter(dev, e_info); + if (result) + return true; + + pci_walk_bus(parent->subordinate, find_device_iter, e_info); + + if (!e_info->error_dev_num) { + pci_printk(KERN_DEBUG, parent, "can't find device of ID%04x\n", + e_info->id); + return false; + } + return true; +} + +/** + * handle_error_source - handle logging error into an event log + * @dev: pointer to pci_dev data structure of error source device + * @info: comprehensive error information + * + * Invoked when an error being detected by Root Port. + */ +static void handle_error_source(struct pci_dev *dev, struct aer_err_info *info) +{ + int pos; + + if (info->severity == AER_CORRECTABLE) { + /* + * Correctable error does not need software intervention. + * No need to go through error recovery process. + */ + pos = dev->aer_cap; + if (pos) + pci_write_config_dword(dev, pos + PCI_ERR_COR_STATUS, + info->status); + } else if (info->severity == AER_NONFATAL) + pcie_do_nonfatal_recovery(dev); + else if (info->severity == AER_FATAL) + pcie_do_fatal_recovery(dev, PCIE_PORT_SERVICE_AER); +} + +#ifdef CONFIG_ACPI_APEI_PCIEAER + +#define AER_RECOVER_RING_ORDER 4 +#define AER_RECOVER_RING_SIZE (1 << AER_RECOVER_RING_ORDER) + +struct aer_recover_entry { + u8 bus; + u8 devfn; + u16 domain; + int severity; + struct aer_capability_regs *regs; +}; + +static DEFINE_KFIFO(aer_recover_ring, struct aer_recover_entry, + AER_RECOVER_RING_SIZE); + +static void aer_recover_work_func(struct work_struct *work) +{ + struct aer_recover_entry entry; + struct pci_dev *pdev; + + while (kfifo_get(&aer_recover_ring, &entry)) { + pdev = pci_get_domain_bus_and_slot(entry.domain, entry.bus, + entry.devfn); + if (!pdev) { + pr_err("AER recover: Can not find pci_dev for %04x:%02x:%02x:%x\n", + entry.domain, entry.bus, + PCI_SLOT(entry.devfn), PCI_FUNC(entry.devfn)); + continue; + } + cper_print_aer(pdev, entry.severity, entry.regs); + if (entry.severity == AER_NONFATAL) + pcie_do_nonfatal_recovery(pdev); + else if (entry.severity == AER_FATAL) + pcie_do_fatal_recovery(pdev, PCIE_PORT_SERVICE_AER); + pci_dev_put(pdev); + } +} + +/* + * Mutual exclusion for writers of aer_recover_ring, reader side don't + * need lock, because there is only one reader and lock is not needed + * between reader and writer. + */ +static DEFINE_SPINLOCK(aer_recover_ring_lock); +static DECLARE_WORK(aer_recover_work, aer_recover_work_func); + +void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, + int severity, struct aer_capability_regs *aer_regs) +{ + unsigned long flags; + struct aer_recover_entry entry = { + .bus = bus, + .devfn = devfn, + .domain = domain, + .severity = severity, + .regs = aer_regs, + }; + + spin_lock_irqsave(&aer_recover_ring_lock, flags); + if (kfifo_put(&aer_recover_ring, entry)) + schedule_work(&aer_recover_work); + else + pr_err("AER recover: Buffer overflow when recovering AER for %04x:%02x:%02x:%x\n", + domain, bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); + spin_unlock_irqrestore(&aer_recover_ring_lock, flags); +} +EXPORT_SYMBOL_GPL(aer_recover_queue); +#endif + +/** + * get_device_error_info - read error status from dev and store it to info + * @dev: pointer to the device expected to have a error record + * @info: pointer to structure to store the error record + * + * Return 1 on success, 0 on error. + * + * Note that @info is reused among all error devices. Clear fields properly. + */ +static int get_device_error_info(struct pci_dev *dev, struct aer_err_info *info) +{ + int pos, temp; + + /* Must reset in this function */ + info->status = 0; + info->tlp_header_valid = 0; + + pos = dev->aer_cap; + + /* The device might not support AER */ + if (!pos) + return 0; + + if (info->severity == AER_CORRECTABLE) { + pci_read_config_dword(dev, pos + PCI_ERR_COR_STATUS, + &info->status); + pci_read_config_dword(dev, pos + PCI_ERR_COR_MASK, + &info->mask); + if (!(info->status & ~info->mask)) + return 0; + } else if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE || + info->severity == AER_NONFATAL) { + + /* Link is still healthy for IO reads */ + pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, + &info->status); + pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_MASK, + &info->mask); + if (!(info->status & ~info->mask)) + return 0; + + /* Get First Error Pointer */ + pci_read_config_dword(dev, pos + PCI_ERR_CAP, &temp); + info->first_error = PCI_ERR_CAP_FEP(temp); + + if (info->status & AER_LOG_TLP_MASKS) { + info->tlp_header_valid = 1; + pci_read_config_dword(dev, + pos + PCI_ERR_HEADER_LOG, &info->tlp.dw0); + pci_read_config_dword(dev, + pos + PCI_ERR_HEADER_LOG + 4, &info->tlp.dw1); + pci_read_config_dword(dev, + pos + PCI_ERR_HEADER_LOG + 8, &info->tlp.dw2); + pci_read_config_dword(dev, + pos + PCI_ERR_HEADER_LOG + 12, &info->tlp.dw3); + } + } + + return 1; +} + +static inline void aer_process_err_devices(struct aer_err_info *e_info) +{ + int i; + + /* Report all before handle them, not to lost records by reset etc. */ + for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) { + if (get_device_error_info(e_info->dev[i], e_info)) + aer_print_error(e_info->dev[i], e_info); + } + for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) { + if (get_device_error_info(e_info->dev[i], e_info)) + handle_error_source(e_info->dev[i], e_info); + } +} + +/** + * aer_isr_one_error - consume an error detected by root port + * @rpc: pointer to the root port which holds an error + * @e_src: pointer to an error source + */ +static void aer_isr_one_error(struct aer_rpc *rpc, + struct aer_err_source *e_src) +{ + struct pci_dev *pdev = rpc->rpd; + struct aer_err_info *e_info = &rpc->e_info; + + /* + * There is a possibility that both correctable error and + * uncorrectable error being logged. Report correctable error first. + */ + if (e_src->status & PCI_ERR_ROOT_COR_RCV) { + e_info->id = ERR_COR_ID(e_src->id); + e_info->severity = AER_CORRECTABLE; + + if (e_src->status & PCI_ERR_ROOT_MULTI_COR_RCV) + e_info->multi_error_valid = 1; + else + e_info->multi_error_valid = 0; + aer_print_port_info(pdev, e_info); + + if (find_source_device(pdev, e_info)) + aer_process_err_devices(e_info); + } + + if (e_src->status & PCI_ERR_ROOT_UNCOR_RCV) { + e_info->id = ERR_UNCOR_ID(e_src->id); + + if (e_src->status & PCI_ERR_ROOT_FATAL_RCV) + e_info->severity = AER_FATAL; + else + e_info->severity = AER_NONFATAL; + + if (e_src->status & PCI_ERR_ROOT_MULTI_UNCOR_RCV) + e_info->multi_error_valid = 1; + else + e_info->multi_error_valid = 0; + + aer_print_port_info(pdev, e_info); + + if (find_source_device(pdev, e_info)) + aer_process_err_devices(e_info); + } +} + +/** + * get_e_source - retrieve an error source + * @rpc: pointer to the root port which holds an error + * @e_src: pointer to store retrieved error source + * + * Return 1 if an error source is retrieved, otherwise 0. + * + * Invoked by DPC handler to consume an error. + */ +static int get_e_source(struct aer_rpc *rpc, struct aer_err_source *e_src) +{ + unsigned long flags; + + /* Lock access to Root error producer/consumer index */ + spin_lock_irqsave(&rpc->e_lock, flags); + if (rpc->prod_idx == rpc->cons_idx) { + spin_unlock_irqrestore(&rpc->e_lock, flags); + return 0; + } + + *e_src = rpc->e_sources[rpc->cons_idx]; + rpc->cons_idx++; + if (rpc->cons_idx == AER_ERROR_SOURCES_MAX) + rpc->cons_idx = 0; + spin_unlock_irqrestore(&rpc->e_lock, flags); + + return 1; +} + +/** + * aer_isr - consume errors detected by root port + * @work: definition of this work item + * + * Invoked, as DPC, when root port records new detected error + */ +static void aer_isr(struct work_struct *work) +{ + struct aer_rpc *rpc = container_of(work, struct aer_rpc, dpc_handler); + struct aer_err_source uninitialized_var(e_src); + + mutex_lock(&rpc->rpc_mutex); + while (get_e_source(rpc, &e_src)) + aer_isr_one_error(rpc, &e_src); + mutex_unlock(&rpc->rpc_mutex); +} + /** * aer_irq - Root Port's ISR * @irq: IRQ assigned to Root Port diff --git a/drivers/pci/pcie/aer/aerdrv.h b/drivers/pci/pcie/aer/aerdrv.h index 6e0ad9a68fd9..059b89441acb 100644 --- a/drivers/pci/pcie/aer/aerdrv.h +++ b/drivers/pci/pcie/aer/aerdrv.h @@ -77,7 +77,6 @@ struct aer_rpc { }; extern struct bus_type pcie_port_bus_type; -void aer_isr(struct work_struct *work); void aer_print_error(struct pci_dev *dev, struct aer_err_info *info); void aer_print_port_info(struct pci_dev *dev, struct aer_err_info *info); irqreturn_t aer_irq(int irq, void *context); diff --git a/drivers/pci/pcie/aer/aerdrv_core.c b/drivers/pci/pcie/aer/aerdrv_core.c deleted file mode 100644 index 42d4f3f32282..000000000000 --- a/drivers/pci/pcie/aer/aerdrv_core.c +++ /dev/null @@ -1,496 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Implement the core part of PCIe AER. When a PCIe error is delivered, an - * error message will be collected and printed to console, then an error - * recovery procedure will be executed by following the PCI error recovery - * rules. - * - * Copyright (C) 2006 Intel Corp. - * Tom Long Nguyen (tom.l.nguyen@intel.com) - * Zhang Yanmin (yanmin.zhang@intel.com) - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include "aerdrv.h" -#include "../../pci.h" - -#define PCI_EXP_AER_FLAGS (PCI_EXP_DEVCTL_CERE | PCI_EXP_DEVCTL_NFERE | \ - PCI_EXP_DEVCTL_FERE | PCI_EXP_DEVCTL_URRE) - -int pci_enable_pcie_error_reporting(struct pci_dev *dev) -{ - if (pcie_aer_get_firmware_first(dev)) - return -EIO; - - if (!dev->aer_cap) - return -EIO; - - return pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_AER_FLAGS); -} -EXPORT_SYMBOL_GPL(pci_enable_pcie_error_reporting); - -int pci_disable_pcie_error_reporting(struct pci_dev *dev) -{ - if (pcie_aer_get_firmware_first(dev)) - return -EIO; - - return pcie_capability_clear_word(dev, PCI_EXP_DEVCTL, - PCI_EXP_AER_FLAGS); -} -EXPORT_SYMBOL_GPL(pci_disable_pcie_error_reporting); - -int pci_cleanup_aer_uncorrect_error_status(struct pci_dev *dev) -{ - int pos; - u32 status; - - pos = dev->aer_cap; - if (!pos) - return -EIO; - - pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, &status); - if (status) - pci_write_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, status); - - return 0; -} -EXPORT_SYMBOL_GPL(pci_cleanup_aer_uncorrect_error_status); - -int pci_cleanup_aer_error_status_regs(struct pci_dev *dev) -{ - int pos; - u32 status; - int port_type; - - if (!pci_is_pcie(dev)) - return -ENODEV; - - pos = dev->aer_cap; - if (!pos) - return -EIO; - - port_type = pci_pcie_type(dev); - if (port_type == PCI_EXP_TYPE_ROOT_PORT) { - pci_read_config_dword(dev, pos + PCI_ERR_ROOT_STATUS, &status); - pci_write_config_dword(dev, pos + PCI_ERR_ROOT_STATUS, status); - } - - pci_read_config_dword(dev, pos + PCI_ERR_COR_STATUS, &status); - pci_write_config_dword(dev, pos + PCI_ERR_COR_STATUS, status); - - pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, &status); - pci_write_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, status); - - return 0; -} - -int pci_aer_init(struct pci_dev *dev) -{ - dev->aer_cap = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR); - return pci_cleanup_aer_error_status_regs(dev); -} - -/** - * add_error_device - list device to be handled - * @e_info: pointer to error info - * @dev: pointer to pci_dev to be added - */ -static int add_error_device(struct aer_err_info *e_info, struct pci_dev *dev) -{ - if (e_info->error_dev_num < AER_MAX_MULTI_ERR_DEVICES) { - e_info->dev[e_info->error_dev_num] = dev; - e_info->error_dev_num++; - return 0; - } - return -ENOSPC; -} - -/** - * is_error_source - check whether the device is source of reported error - * @dev: pointer to pci_dev to be checked - * @e_info: pointer to reported error info - */ -static bool is_error_source(struct pci_dev *dev, struct aer_err_info *e_info) -{ - int pos; - u32 status, mask; - u16 reg16; - - /* - * When bus id is equal to 0, it might be a bad id - * reported by root port. - */ - if ((PCI_BUS_NUM(e_info->id) != 0) && - !(dev->bus->bus_flags & PCI_BUS_FLAGS_NO_AERSID)) { - /* Device ID match? */ - if (e_info->id == ((dev->bus->number << 8) | dev->devfn)) - return true; - - /* Continue id comparing if there is no multiple error */ - if (!e_info->multi_error_valid) - return false; - } - - /* - * When either - * 1) bus id is equal to 0. Some ports might lose the bus - * id of error source id; - * 2) bus flag PCI_BUS_FLAGS_NO_AERSID is set - * 3) There are multiple errors and prior ID comparing fails; - * We check AER status registers to find possible reporter. - */ - if (atomic_read(&dev->enable_cnt) == 0) - return false; - - /* Check if AER is enabled */ - pcie_capability_read_word(dev, PCI_EXP_DEVCTL, ®16); - if (!(reg16 & PCI_EXP_AER_FLAGS)) - return false; - - pos = dev->aer_cap; - if (!pos) - return false; - - /* Check if error is recorded */ - if (e_info->severity == AER_CORRECTABLE) { - pci_read_config_dword(dev, pos + PCI_ERR_COR_STATUS, &status); - pci_read_config_dword(dev, pos + PCI_ERR_COR_MASK, &mask); - } else { - pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, &status); - pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_MASK, &mask); - } - if (status & ~mask) - return true; - - return false; -} - -static int find_device_iter(struct pci_dev *dev, void *data) -{ - struct aer_err_info *e_info = (struct aer_err_info *)data; - - if (is_error_source(dev, e_info)) { - /* List this device */ - if (add_error_device(e_info, dev)) { - /* We cannot handle more... Stop iteration */ - /* TODO: Should print error message here? */ - return 1; - } - - /* If there is only a single error, stop iteration */ - if (!e_info->multi_error_valid) - return 1; - } - return 0; -} - -/** - * find_source_device - search through device hierarchy for source device - * @parent: pointer to Root Port pci_dev data structure - * @e_info: including detailed error information such like id - * - * Return true if found. - * - * Invoked by DPC when error is detected at the Root Port. - * Caller of this function must set id, severity, and multi_error_valid of - * struct aer_err_info pointed by @e_info properly. This function must fill - * e_info->error_dev_num and e_info->dev[], based on the given information. - */ -static bool find_source_device(struct pci_dev *parent, - struct aer_err_info *e_info) -{ - struct pci_dev *dev = parent; - int result; - - /* Must reset in this function */ - e_info->error_dev_num = 0; - - /* Is Root Port an agent that sends error message? */ - result = find_device_iter(dev, e_info); - if (result) - return true; - - pci_walk_bus(parent->subordinate, find_device_iter, e_info); - - if (!e_info->error_dev_num) { - pci_printk(KERN_DEBUG, parent, "can't find device of ID%04x\n", - e_info->id); - return false; - } - return true; -} - -/** - * handle_error_source - handle logging error into an event log - * @dev: pointer to pci_dev data structure of error source device - * @info: comprehensive error information - * - * Invoked when an error being detected by Root Port. - */ -static void handle_error_source(struct pci_dev *dev, struct aer_err_info *info) -{ - int pos; - - if (info->severity == AER_CORRECTABLE) { - /* - * Correctable error does not need software intervention. - * No need to go through error recovery process. - */ - pos = dev->aer_cap; - if (pos) - pci_write_config_dword(dev, pos + PCI_ERR_COR_STATUS, - info->status); - } else if (info->severity == AER_NONFATAL) - pcie_do_nonfatal_recovery(dev); - else if (info->severity == AER_FATAL) - pcie_do_fatal_recovery(dev, PCIE_PORT_SERVICE_AER); -} - -#ifdef CONFIG_ACPI_APEI_PCIEAER - -#define AER_RECOVER_RING_ORDER 4 -#define AER_RECOVER_RING_SIZE (1 << AER_RECOVER_RING_ORDER) - -struct aer_recover_entry { - u8 bus; - u8 devfn; - u16 domain; - int severity; - struct aer_capability_regs *regs; -}; - -static DEFINE_KFIFO(aer_recover_ring, struct aer_recover_entry, - AER_RECOVER_RING_SIZE); - -static void aer_recover_work_func(struct work_struct *work) -{ - struct aer_recover_entry entry; - struct pci_dev *pdev; - - while (kfifo_get(&aer_recover_ring, &entry)) { - pdev = pci_get_domain_bus_and_slot(entry.domain, entry.bus, - entry.devfn); - if (!pdev) { - pr_err("AER recover: Can not find pci_dev for %04x:%02x:%02x:%x\n", - entry.domain, entry.bus, - PCI_SLOT(entry.devfn), PCI_FUNC(entry.devfn)); - continue; - } - cper_print_aer(pdev, entry.severity, entry.regs); - if (entry.severity == AER_NONFATAL) - pcie_do_nonfatal_recovery(pdev); - else if (entry.severity == AER_FATAL) - pcie_do_fatal_recovery(pdev, PCIE_PORT_SERVICE_AER); - pci_dev_put(pdev); - } -} - -/* - * Mutual exclusion for writers of aer_recover_ring, reader side don't - * need lock, because there is only one reader and lock is not needed - * between reader and writer. - */ -static DEFINE_SPINLOCK(aer_recover_ring_lock); -static DECLARE_WORK(aer_recover_work, aer_recover_work_func); - -void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn, - int severity, struct aer_capability_regs *aer_regs) -{ - unsigned long flags; - struct aer_recover_entry entry = { - .bus = bus, - .devfn = devfn, - .domain = domain, - .severity = severity, - .regs = aer_regs, - }; - - spin_lock_irqsave(&aer_recover_ring_lock, flags); - if (kfifo_put(&aer_recover_ring, entry)) - schedule_work(&aer_recover_work); - else - pr_err("AER recover: Buffer overflow when recovering AER for %04x:%02x:%02x:%x\n", - domain, bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); - spin_unlock_irqrestore(&aer_recover_ring_lock, flags); -} -EXPORT_SYMBOL_GPL(aer_recover_queue); -#endif - -/** - * get_device_error_info - read error status from dev and store it to info - * @dev: pointer to the device expected to have a error record - * @info: pointer to structure to store the error record - * - * Return 1 on success, 0 on error. - * - * Note that @info is reused among all error devices. Clear fields properly. - */ -static int get_device_error_info(struct pci_dev *dev, struct aer_err_info *info) -{ - int pos, temp; - - /* Must reset in this function */ - info->status = 0; - info->tlp_header_valid = 0; - - pos = dev->aer_cap; - - /* The device might not support AER */ - if (!pos) - return 0; - - if (info->severity == AER_CORRECTABLE) { - pci_read_config_dword(dev, pos + PCI_ERR_COR_STATUS, - &info->status); - pci_read_config_dword(dev, pos + PCI_ERR_COR_MASK, - &info->mask); - if (!(info->status & ~info->mask)) - return 0; - } else if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE || - info->severity == AER_NONFATAL) { - - /* Link is still healthy for IO reads */ - pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_STATUS, - &info->status); - pci_read_config_dword(dev, pos + PCI_ERR_UNCOR_MASK, - &info->mask); - if (!(info->status & ~info->mask)) - return 0; - - /* Get First Error Pointer */ - pci_read_config_dword(dev, pos + PCI_ERR_CAP, &temp); - info->first_error = PCI_ERR_CAP_FEP(temp); - - if (info->status & AER_LOG_TLP_MASKS) { - info->tlp_header_valid = 1; - pci_read_config_dword(dev, - pos + PCI_ERR_HEADER_LOG, &info->tlp.dw0); - pci_read_config_dword(dev, - pos + PCI_ERR_HEADER_LOG + 4, &info->tlp.dw1); - pci_read_config_dword(dev, - pos + PCI_ERR_HEADER_LOG + 8, &info->tlp.dw2); - pci_read_config_dword(dev, - pos + PCI_ERR_HEADER_LOG + 12, &info->tlp.dw3); - } - } - - return 1; -} - -static inline void aer_process_err_devices(struct aer_err_info *e_info) -{ - int i; - - /* Report all before handle them, not to lost records by reset etc. */ - for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) { - if (get_device_error_info(e_info->dev[i], e_info)) - aer_print_error(e_info->dev[i], e_info); - } - for (i = 0; i < e_info->error_dev_num && e_info->dev[i]; i++) { - if (get_device_error_info(e_info->dev[i], e_info)) - handle_error_source(e_info->dev[i], e_info); - } -} - -/** - * aer_isr_one_error - consume an error detected by root port - * @rpc: pointer to the root port which holds an error - * @e_src: pointer to an error source - */ -static void aer_isr_one_error(struct aer_rpc *rpc, - struct aer_err_source *e_src) -{ - struct pci_dev *pdev = rpc->rpd; - struct aer_err_info *e_info = &rpc->e_info; - - /* - * There is a possibility that both correctable error and - * uncorrectable error being logged. Report correctable error first. - */ - if (e_src->status & PCI_ERR_ROOT_COR_RCV) { - e_info->id = ERR_COR_ID(e_src->id); - e_info->severity = AER_CORRECTABLE; - - if (e_src->status & PCI_ERR_ROOT_MULTI_COR_RCV) - e_info->multi_error_valid = 1; - else - e_info->multi_error_valid = 0; - aer_print_port_info(pdev, e_info); - - if (find_source_device(pdev, e_info)) - aer_process_err_devices(e_info); - } - - if (e_src->status & PCI_ERR_ROOT_UNCOR_RCV) { - e_info->id = ERR_UNCOR_ID(e_src->id); - - if (e_src->status & PCI_ERR_ROOT_FATAL_RCV) - e_info->severity = AER_FATAL; - else - e_info->severity = AER_NONFATAL; - - if (e_src->status & PCI_ERR_ROOT_MULTI_UNCOR_RCV) - e_info->multi_error_valid = 1; - else - e_info->multi_error_valid = 0; - - aer_print_port_info(pdev, e_info); - - if (find_source_device(pdev, e_info)) - aer_process_err_devices(e_info); - } -} - -/** - * get_e_source - retrieve an error source - * @rpc: pointer to the root port which holds an error - * @e_src: pointer to store retrieved error source - * - * Return 1 if an error source is retrieved, otherwise 0. - * - * Invoked by DPC handler to consume an error. - */ -static int get_e_source(struct aer_rpc *rpc, struct aer_err_source *e_src) -{ - unsigned long flags; - - /* Lock access to Root error producer/consumer index */ - spin_lock_irqsave(&rpc->e_lock, flags); - if (rpc->prod_idx == rpc->cons_idx) { - spin_unlock_irqrestore(&rpc->e_lock, flags); - return 0; - } - - *e_src = rpc->e_sources[rpc->cons_idx]; - rpc->cons_idx++; - if (rpc->cons_idx == AER_ERROR_SOURCES_MAX) - rpc->cons_idx = 0; - spin_unlock_irqrestore(&rpc->e_lock, flags); - - return 1; -} - -/** - * aer_isr - consume errors detected by root port - * @work: definition of this work item - * - * Invoked, as DPC, when root port records new detected error - */ -void aer_isr(struct work_struct *work) -{ - struct aer_rpc *rpc = container_of(work, struct aer_rpc, dpc_handler); - struct aer_err_source uninitialized_var(e_src); - - mutex_lock(&rpc->rpc_mutex); - while (get_e_source(rpc, &e_src)) - aer_isr_one_error(rpc, &e_src); - mutex_unlock(&rpc->rpc_mutex); -}