From patchwork Tue Apr 27 17:51:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 12227095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E389C433ED for ; Tue, 27 Apr 2021 17:54:12 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F3218610FB for ; Tue, 27 Apr 2021 17:54:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3218610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:Cc:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wnaC85O4ak8HG4rGzSctAvS1Ab5FY4TvZa5BmU/NkYY=; b=pltEMqu4e9iWEoO8jLRmlv5NgF NGp0DuL/fKat9r5NJotWMqhe98/lXx+sy9VhOP9/AgxaO0+U+1C2D4Mi7ZXbz7z28vPYQ2Lwcu00G 3+6oFr7SNpWtwvcjJbgMjUfdZX71713vsaicro9bHZcbcuZNMLrR2s1T3k+AWBzKwrFeowpJpLkWG B8g/704alLRdrCo++3tShJRpG8KdaUn92C3RraqT7ptRNCJq1UtKJyb9P8MpVg05SNDVJ6YUqsE7w O7rqFjAoXX5rASH8XAICKvjPxq6eCQzeuNA9Z6W5nayEHa1VehJRDPW3RvO5SPfOKLouEv5vKV384 eMsCsMoQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lbRsw-001zUX-Mc; Tue, 27 Apr 2021 17:52:35 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbRsM-001zQj-Py; Tue, 27 Apr 2021 17:51:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description; bh=iB4Oot/0lSTvUQU4iqLoWuoZs5LYAMg1vqOt+d8WS9Y=; b=qrBUVov10xQh93OQrbVh1OewZ6 eMRlei7xM4EcqWedoxczYwwxCjmvcmcRYn1tm7GgB4z+MyNf1dZy4A7X2l0a1JA0iekGa02Ul7Bix AuAyiDrC3lWJ7Yt5XN+1XnDjInJ++Z/kecYhTaa8S9elWAmaWvvNuQBOOQODw/sfhL9KcXFUKk0fs zm2ZQMxEVS+OJs0FkC8W5uba7+auTk2Am2ZnDsbx86OFmA/ZjL2kDwxNc/d+s1w8/GukTNMNRb4q3 j1T5etS8Q5Fug6Sm+r8vxidbhJzO9xtzh5fPAaEiqjFhqTh2AFRqaDwXv/XCQ9crUJPv7x33YD9kP d0/AbvWA==; Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbRsJ-00Gtm2-Tr; Tue, 27 Apr 2021 17:51:57 +0000 Received: by mail-pl1-x630.google.com with SMTP id s15so2897223plg.6; Tue, 27 Apr 2021 10:51:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iB4Oot/0lSTvUQU4iqLoWuoZs5LYAMg1vqOt+d8WS9Y=; b=bZqqTgnJlQGzbai3jyFB3ehCinYQgPhAhwo1ZLmVVUr6SD0z7iakzmaZMB1JJdYyo6 weePY+ZuTn5ZcU01JuNl677/XDFojGvffrR/qrhvSGoW2QqtCD+PcOrXxn8w/24Dq7a+ jNFD4ch0i836/Ug6if0d4dsHPQ4pd4rsCHijf88kLCT2N/tRYuFOWPGNET3C5op3kEKf ZlzP996/DaMphxK2SfFuAyPsOnGNZZk1//PK7siHo2NHz22m3gMirCpVUFRcRHgAZo0G k/4NxBg7xb8/qaJzxQ/Vp9EDdq4fNMuPq/nHZRT/rS3/0J3nfXp0fbgppCZYrwzH70Sf EwXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iB4Oot/0lSTvUQU4iqLoWuoZs5LYAMg1vqOt+d8WS9Y=; b=ucDDLcFQJp4EOrqCS0SA3kOva+0C8tyipbXqU3RCsD3361LPy5SO/QblCL0hCRY6ar T+BTKm8tEr33XMuJdgnhyKDZ4V8aRfDN3kHNoWizOgvj0eykGqtInqjzg3vQFJBCZd0Z M5tXovjA4lmCmFZVWV+cBgAr3z9WDL1LA1k+bY5FTkpE0KbAB38CzWojvLq1BHbSfqcq pKqerw6V9kJUjxX0O3hEGcDYLQgM6TkCqvlJF1Zdqt7wmH1aqxqaTRVgN9K6haVMfy+6 JCWidPVSdRZTGbVM3Xrh2UECkJTBglTr8R60q5wPpQommyo/uHa7Yp25wUZ54PG8OLSb ENQg== X-Gm-Message-State: AOAM530biH8Nf9zs4z/rWOv78sISxXqs2xh6ZjiSB1vpudgWXsK2bXU7 qd+dR3WDOKsNjr1JN23p0D4T2DaULLU= X-Google-Smtp-Source: ABdhPJzHGHFcKV8nvFEefnummSuloMSBCvVu6bgx287p5NtPcZB/6U7cngbAXxMcDWS3vRqJ1BqAeA== X-Received: by 2002:a17:90b:1b03:: with SMTP id nu3mr6130980pjb.62.1619545914772; Tue, 27 Apr 2021 10:51:54 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.231.250]) by smtp.gmail.com with ESMTPSA id h21sm2456833pfo.211.2021.04.27.10.51.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Apr 2021 10:51:54 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Bjorn Helgaas , Nicolas Saenz Julienne , bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com Cc: Jim Quinlan , Nicolas Saenz Julienne , Lorenzo Pieralisi , Rob Herring , Florian Fainelli , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v1 3/4] PCI: brcmstb: Add panic/die handler to RC driver Date: Tue, 27 Apr 2021 13:51:38 -0400 Message-Id: <20210427175140.17800-4-jim2101024@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210427175140.17800-1-jim2101024@gmail.com> References: <20210427175140.17800-1-jim2101024@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210427_105155_987462_34AEBF2E X-CRM114-Status: GOOD ( 19.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Whereas most PCIe HW returns 0xffffffff on illegal accesses and the like, by default Broadcom's STB PCIe controller effects an abort. This simple handler determines if the PCIe controller was the cause of the abort and if so, prints out diagnostic info. Example output: brcm-pcie 8b20000.pcie: Error: Mem Acc: 32bit, Read, @0x38000000 brcm-pcie 8b20000.pcie: Type: TO=0 Abt=0 UnspReq=1 AccDsble=0 BadAddr=0 Signed-off-by: Jim Quinlan Acked-by: Florian Fainelli --- drivers/pci/controller/pcie-brcmstb.c | 122 ++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index 3b6a62dd2e72..d3af8d84f0d6 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -12,11 +12,13 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include @@ -184,6 +186,39 @@ #define PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK 0x1 #define PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT 0x0 +/* Error report regiseters */ +#define PCIE_OUTB_ERR_TREAT 0x6000 +#define PCIE_OUTB_ERR_TREAT_CONFIG_MASK 0x1 +#define PCIE_OUTB_ERR_TREAT_MEM_MASK 0x2 +#define PCIE_OUTB_ERR_VALID 0x6004 +#define PCIE_OUTB_ERR_CLEAR 0x6008 +#define PCIE_OUTB_ERR_ACC_INFO 0x600c +#define PCIE_OUTB_ERR_ACC_INFO_CFG_ERR_MASK 0x01 +#define PCIE_OUTB_ERR_ACC_INFO_MEM_ERR_MASK 0x02 +#define PCIE_OUTB_ERR_ACC_INFO_TYPE_64_MASK 0x04 +#define PCIE_OUTB_ERR_ACC_INFO_DIR_WRITE_MASK 0x10 +#define PCIE_OUTB_ERR_ACC_INFO_BYTE_LANES_MASK 0xff00 +#define PCIE_OUTB_ERR_ACC_ADDR 0x6010 +#define PCIE_OUTB_ERR_ACC_ADDR_BUS_MASK 0xff00000 +#define PCIE_OUTB_ERR_ACC_ADDR_DEV_MASK 0xf8000 +#define PCIE_OUTB_ERR_ACC_ADDR_FUNC_MASK 0x7000 +#define PCIE_OUTB_ERR_ACC_ADDR_REG_MASK 0xfff +#define PCIE_OUTB_ERR_CFG_CAUSE 0x6014 +#define PCIE_OUTB_ERR_CFG_CAUSE_TIMEOUT_MASK 0x40 +#define PCIE_OUTB_ERR_CFG_CAUSE_ABORT_MASK 0x20 +#define PCIE_OUTB_ERR_CFG_CAUSE_UNSUPP_REQ_MASK 0x10 +#define PCIE_OUTB_ERR_CFG_CAUSE_ACC_TIMEOUT_MASK 0x4 +#define PCIE_OUTB_ERR_CFG_CAUSE_ACC_DISABLED_MASK 0x2 +#define PCIE_OUTB_ERR_CFG_CAUSE_ACC_64BIT__MASK 0x1 +#define PCIE_OUTB_ERR_MEM_ADDR_LO 0x6018 +#define PCIE_OUTB_ERR_MEM_ADDR_HI 0x601c +#define PCIE_OUTB_ERR_MEM_CAUSE 0x6020 +#define PCIE_OUTB_ERR_MEM_CAUSE_TIMEOUT_MASK 0x40 +#define PCIE_OUTB_ERR_MEM_CAUSE_ABORT_MASK 0x20 +#define PCIE_OUTB_ERR_MEM_CAUSE_UNSUPP_REQ_MASK 0x10 +#define PCIE_OUTB_ERR_MEM_CAUSE_ACC_DISABLED_MASK 0x2 +#define PCIE_OUTB_ERR_MEM_CAUSE_BAD_ADDR_MASK 0x1 + /* Forward declarations */ struct brcm_pcie; static inline void brcm_pcie_bridge_sw_init_set_7278(struct brcm_pcie *pcie, u32 val); @@ -215,6 +250,7 @@ struct pcie_cfg_data { const enum pcie_type type; void (*perst_set)(struct brcm_pcie *pcie, u32 val); void (*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val); + const bool has_err_report; }; static const int pcie_offsets[] = { @@ -262,6 +298,7 @@ static const struct pcie_cfg_data bcm7216_cfg = { .type = BCM7278, .perst_set = brcm_pcie_perst_set_7278, .bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_7278, + .has_err_report = true, }; struct brcm_msi { @@ -302,8 +339,87 @@ struct brcm_pcie { u32 hw_rev; void (*perst_set)(struct brcm_pcie *pcie, u32 val); void (*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val); + bool has_err_report; + struct notifier_block die_notifier; }; +/* Dump out PCIe errors on die or panic */ +static int dump_pcie_error(struct notifier_block *self, unsigned long v, void *p) +{ + const struct brcm_pcie *pcie = container_of(self, struct brcm_pcie, die_notifier); + void __iomem *base = pcie->base; + int i, is_cfg_err, is_mem_err, lanes; + char *width_str, *direction_str, lanes_str[9]; + u32 info; + + if (readl(base + PCIE_OUTB_ERR_VALID) == 0) + return NOTIFY_DONE; + info = readl(base + PCIE_OUTB_ERR_ACC_INFO); + + + is_cfg_err = !!(info & PCIE_OUTB_ERR_ACC_INFO_CFG_ERR_MASK); + is_mem_err = !!(info & PCIE_OUTB_ERR_ACC_INFO_MEM_ERR_MASK); + width_str = (info & PCIE_OUTB_ERR_ACC_INFO_TYPE_64_MASK) ? "64bit" : "32bit"; + direction_str = (info & PCIE_OUTB_ERR_ACC_INFO_DIR_WRITE_MASK) ? "Write" : "Read"; + lanes = FIELD_GET(PCIE_OUTB_ERR_ACC_INFO_BYTE_LANES_MASK, info); + for (i = 0, lanes_str[8] = 0; i < 8; i++) + lanes_str[i] = (lanes & (1 << i)) ? '1' : '0'; + + if (is_cfg_err) { + u32 cfg_addr = readl(base + PCIE_OUTB_ERR_ACC_ADDR); + u32 cause = readl(base + PCIE_OUTB_ERR_CFG_CAUSE); + int bus = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_BUS_MASK, cfg_addr); + int dev = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_DEV_MASK, cfg_addr); + int func = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_FUNC_MASK, cfg_addr); + int reg = FIELD_GET(PCIE_OUTB_ERR_ACC_ADDR_REG_MASK, cfg_addr); + + dev_err(pcie->dev, "Error: CFG Acc, %s, %s, Bus=%d, Dev=%d, Fun=%d, Reg=0x%x, lanes=%s\n", + width_str, direction_str, bus, dev, func, reg, lanes_str); + dev_err(pcie->dev, " Type: TO=%d Abt=%d UnsupReq=%d AccTO=%d AccDsbld=%d Acc64bit=%d\n", + !!(cause & PCIE_OUTB_ERR_CFG_CAUSE_TIMEOUT_MASK), + !!(cause & PCIE_OUTB_ERR_CFG_CAUSE_ABORT_MASK), + !!(cause & PCIE_OUTB_ERR_CFG_CAUSE_UNSUPP_REQ_MASK), + !!(cause & PCIE_OUTB_ERR_CFG_CAUSE_ACC_TIMEOUT_MASK), + !!(cause & PCIE_OUTB_ERR_CFG_CAUSE_ACC_DISABLED_MASK), + !!(cause & PCIE_OUTB_ERR_CFG_CAUSE_ACC_64BIT__MASK)); + } + + if (is_mem_err) { + u32 cause = readl(base + PCIE_OUTB_ERR_MEM_CAUSE); + u32 lo = readl(base + PCIE_OUTB_ERR_MEM_ADDR_LO); + u32 hi = readl(base + PCIE_OUTB_ERR_MEM_ADDR_HI); + u64 addr = ((u64)hi << 32) | (u64)lo; + + dev_err(pcie->dev, "Error: Mem Acc, %s, %s, @0x%llx, lanes=%s\n", + width_str, direction_str, addr, lanes_str); + dev_err(pcie->dev, " Type: TO=%d Abt=%d UnsupReq=%d AccDsble=%d BadAddr=%d\n", + !!(cause & PCIE_OUTB_ERR_MEM_CAUSE_TIMEOUT_MASK), + !!(cause & PCIE_OUTB_ERR_MEM_CAUSE_ABORT_MASK), + !!(cause & PCIE_OUTB_ERR_MEM_CAUSE_UNSUPP_REQ_MASK), + !!(cause & PCIE_OUTB_ERR_MEM_CAUSE_ACC_DISABLED_MASK), + !!(cause & PCIE_OUTB_ERR_MEM_CAUSE_BAD_ADDR_MASK)); + } + + /* Clear the error */ + writel(1, base + PCIE_OUTB_ERR_CLEAR); + + return NOTIFY_DONE; +} + +static void brcm_register_die_notifiers(struct brcm_pcie *pcie) +{ + pcie->die_notifier.notifier_call = dump_pcie_error; + register_die_notifier(&pcie->die_notifier); + atomic_notifier_chain_register(&panic_notifier_list, &pcie->die_notifier); +} + +static void brcm_unregister_die_notifiers(struct brcm_pcie *pcie) +{ + unregister_die_notifier(&pcie->die_notifier); + atomic_notifier_chain_unregister(&panic_notifier_list, &pcie->die_notifier); + pcie->die_notifier.notifier_call = NULL; +} + /* * This is to convert the size of the inbound "BAR" region to the * non-linear values of PCIE_X_MISC_RC_BAR[123]_CONFIG_LO.SIZE @@ -1216,6 +1332,8 @@ static int brcm_pcie_remove(struct platform_device *pdev) struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie); pci_stop_root_bus(bridge->bus); + if (pcie->has_err_report) + brcm_unregister_die_notifiers(pcie); pci_remove_root_bus(bridge->bus); __brcm_pcie_remove(pcie); @@ -1255,6 +1373,7 @@ static int brcm_pcie_probe(struct platform_device *pdev) pcie->np = np; pcie->reg_offsets = data->offsets; pcie->type = data->type; + pcie->has_err_report = data->has_err_report; pcie->perst_set = data->perst_set; pcie->bridge_sw_init_set = data->bridge_sw_init_set; @@ -1322,6 +1441,9 @@ static int brcm_pcie_probe(struct platform_device *pdev) platform_set_drvdata(pdev, pcie); + if (pcie->has_err_report) + brcm_register_die_notifiers(pcie); + return pci_host_probe(bridge); fail: __brcm_pcie_remove(pcie);