From patchwork Tue Aug 27 13:08:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 13779540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 129AEC5472F for ; Tue, 27 Aug 2024 13:10:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zVI5R+WdttBIprX1m04d9bNMFDDS+NINGmtqeG2h9Pk=; b=BnEWxutWQ/cmLHXlaMSUEhVmTC 6fL5eFyqF9UOjRF34cSJshAstavhRF7qjwpRl6IChBwU+q6pQYRS7LEhkxVPN2b2hQ69VcUTsr+AM IHKNhtJu5KyH9jk+cVoWFvG/aJIKATCX83YQUEAh9lKXfm27t/GX4H0DQDBlcWXko9lmqOl/whJub dRpjDIDOgCc7NW1XFA727/RaEcx4IzovOoHaNxtkmjDaBwU0zocG+tvux/pCSCkTLUVU5ZgYD2OiF FCWK9Eb+tNQxvDMvUFTFdM2nQxRJ/irmkQ9uj88+5BfX9usKbv/CBGFIWoC9LoaHRwNUrHuikTKx6 tQ7KnFNw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sivxc-0000000BO7i-1ujn; Tue, 27 Aug 2024 13:10:12 +0000 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sivw2-0000000BNiI-1A7e for linux-arm-kernel@lists.infradead.org; Tue, 27 Aug 2024 13:08:36 +0000 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-4280c55e488so29906615e9.0 for ; Tue, 27 Aug 2024 06:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1724764112; x=1725368912; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zVI5R+WdttBIprX1m04d9bNMFDDS+NINGmtqeG2h9Pk=; b=oxQNfVDgSk1xBkJCnKeyxE52GLCjIUsZ6GWpFk6dzRDeF/5OpkCVhaNcAH2ljS6ZOd xng0T/1Se7+WN6jX33SYW9Q+HQnY91+TFfnO4zCHjNej/CcG1Co2tppc7BicgPgXQlgc CWjLN9CIoXJ2KCQtkg99LZ8ZuQ0C2kSvqgukvCpIgQSCmiQJVWO7YGej5+hSk99LSulo mqhyqo/gwUu1c4wB8b+2GK1heY78q/4rLXi2pef7zq/JyVdIuI9URMZJ8Rlljwc27NfE uTDU+mJ1MW+XOLZif0IJYJETmbvs44OHH8IYI8ZCoeghEoCf/JLrjlIe4qNAj5rmDptm lDoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724764112; x=1725368912; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zVI5R+WdttBIprX1m04d9bNMFDDS+NINGmtqeG2h9Pk=; b=BQHZeee/EuHxSbtSTrg0rdwJNFoPhHRPJiB95XkWhc2OOtZZdpceyX4iWG8/AM/13T QCrBW8C6o64H25fXoWPpqH/yWaEl3YhDEwwb2oZCUZvM1pqOE8nJMlGjARjPynzRWSWM Zmp7AZOl4veWHYXLLWbES3DwQQ5nQ1M//eErvfDY9c9M1gZ3SPRzsZ3Y3rpjrI+WJdmJ pfVwOapTMkCq7lkcCiAaOLL7aAMg19DlpCmhlhfwGHtCuuRZVstqN33nk2Nbd+UY6GDx FtArcppSLtzt9H97MNc5RP1/6EkzVgy51IxIRYPB5IL+dCwf3N0VwRZ5Na3ztRSIgrUS tiQw== X-Forwarded-Encrypted: i=1; AJvYcCVmskPYBilPhDE4zRrH/2lACSc2waxKEyoXAzbmOHpwsT4VAUSOzCBCei2IH12FjziHbznRTErt4s1wdon5I4Zs@lists.infradead.org X-Gm-Message-State: AOJu0YwE3ARhLQv3NE0Btlmx5CDeBubU7L7v1AFRIWEkeE2e1uHD6w8D vaWRcdwdcPmgi4/aLXL23nADdJXZ9cTBZz+P03rMQToXcB+tgiluo292vnlC1eQ= X-Google-Smtp-Source: AGHT+IE8YdSXKOIj7u+2bEQxKE7kkLyia2JoNSH9WemPGkONFgqu20dBwFvHn7im4jBKQqBOOuvFvA== X-Received: by 2002:a05:600c:19c7:b0:426:5b19:d2b3 with SMTP id 5b1f17b1804b1-42b9a682241mr16687955e9.14.1724764111836; Tue, 27 Aug 2024 06:08:31 -0700 (PDT) Received: from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42abefcb674sm221517205e9.32.2024.08.27.06.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 06:08:30 -0700 (PDT) Received: from draig.lan (localhost [IPv6:::1]) by draig.lan (Postfix) with ESMTP id 29A415F9F9; Tue, 27 Aug 2024 14:08:30 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, maz@kernel.org, arnd@linaro.org, D Scott Phillips , =?utf-8?q?Alex_Benn=C3=A9e?= Subject: [PATCH 2/3] ampere/arm64: Work around Ampere Altra erratum #82288 PCIE_65 Date: Tue, 27 Aug 2024 14:08:28 +0100 Message-Id: <20240827130829.43632-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240827130829.43632-1-alex.bennee@linaro.org> References: <20240827130829.43632-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240827_060834_350277_8460BE05 X-CRM114-Status: GOOD ( 20.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: D Scott Phillips Altra's PCIe controller may generate incorrect addresses when receiving writes from the CPU with a discontiguous set of byte enables. Attempt to work around this by handing out Device-nGnRE maps instead of Normal Non-cacheable maps for PCIe memory areas. Upstream-Status: Pending Signed-off-by: D Scott Phillips Signed-off-by: Alex Bennée --- arch/arm64/Kconfig | 22 +++++++++++++++++++++- arch/arm64/include/asm/io.h | 3 +++ arch/arm64/include/asm/pgtable.h | 27 ++++++++++++++++++++++----- arch/arm64/mm/ioremap.c | 27 +++++++++++++++++++++++++++ drivers/pci/quirks.c | 9 +++++++++ include/asm-generic/io.h | 4 ++++ mm/ioremap.c | 2 +- 7 files changed, 87 insertions(+), 7 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b3fc891f15442..01adb50df214e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -440,6 +440,27 @@ config AMPERE_ERRATUM_AC03_CPU_38 config ARM64_WORKAROUND_CLEAN_CACHE bool +config ALTRA_ERRATUM_82288 + bool "Ampere Altra: 82288: PCIE_65: PCIe Root Port outbound write combining issue" + default y + help + This option adds an alternative code sequence to work around + Ampere Altra erratum 82288. + + PCIe device drivers may map MMIO space as Normal, non-cacheable + memory attribute (e.g. Linux kernel drivers mapping MMIO + using ioremap_wc). This may be for the purpose of enabling write + combining or unaligned accesses. This can result in data corruption + on the PCIe interface’s outbound MMIO writes due to issues with the + write-combining operation. + + The workaround modifies software that maps PCIe MMIO space as Normal, + non-cacheable memory (e.g. ioremap_wc) to instead Device, + non-gatheringmemory (e.g. ioremap). And all memory operations on PCIe + MMIO space must be strictly aligned. + + If unsure, say Y. + config ARM64_ERRATUM_826319 bool "Cortex-A53: 826319: System might deadlock if a write cannot complete until read data is accepted" default y @@ -2388,4 +2409,3 @@ endmenu # "CPU Power Management" source "drivers/acpi/Kconfig" source "arch/arm64/kvm/Kconfig" - diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h index 41fd90895dfc3..403b65f2f44de 100644 --- a/arch/arm64/include/asm/io.h +++ b/arch/arm64/include/asm/io.h @@ -273,6 +273,9 @@ __iowrite64_copy(void __iomem *to, const void *from, size_t count) #define ioremap_prot ioremap_prot +pgprot_t ioremap_map_prot(phys_addr_t phys_addr, size_t size, unsigned long prot); +#define ioremap_map_prot ioremap_map_prot + #define _PAGE_IOREMAP PROT_DEVICE_nGnRE #define ioremap_wc(addr, size) \ diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7a4f5604be3f7..f4603924390eb 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -236,11 +236,6 @@ static inline pte_t pte_mkyoung(pte_t pte) return set_pte_bit(pte, __pgprot(PTE_AF)); } -static inline pte_t pte_mkspecial(pte_t pte) -{ - return set_pte_bit(pte, __pgprot(PTE_SPECIAL)); -} - static inline pte_t pte_mkcont(pte_t pte) { pte = set_pte_bit(pte, __pgprot(PTE_CONT)); @@ -682,6 +677,28 @@ static inline bool pud_table(pud_t pud) { return true; } PUD_TYPE_TABLE) #endif +#ifdef CONFIG_ALTRA_ERRATUM_82288 +extern bool __read_mostly have_altra_erratum_82288; +#endif + +static inline pte_t pte_mkspecial(pte_t pte) +{ +#ifdef CONFIG_ALTRA_ERRATUM_82288 + phys_addr_t phys = __pte_to_phys(pte); + pgprot_t prot = __pgprot(pte_val(pte) & ~PTE_ADDR_LOW); + + if (unlikely(have_altra_erratum_82288) && + (phys < 0x80000000 || + (phys >= 0x200000000000 && phys < 0x400000000000) || + (phys >= 0x600000000000 && phys < 0x800000000000))) { + pte = __pte(__phys_to_pte_val(phys) | pgprot_val(pgprot_device(prot))); + } +#endif + + return set_pte_bit(pte, __pgprot(PTE_SPECIAL)); +} + + extern pgd_t init_pg_dir[]; extern pgd_t init_pg_end[]; extern pgd_t swapper_pg_dir[]; diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c index 269f2f63ab7dc..8965766181359 100644 --- a/arch/arm64/mm/ioremap.c +++ b/arch/arm64/mm/ioremap.c @@ -3,6 +3,33 @@ #include #include +#ifdef CONFIG_ALTRA_ERRATUM_82288 + +bool have_altra_erratum_82288 __read_mostly; +EXPORT_SYMBOL(have_altra_erratum_82288); + +static bool is_altra_pci(phys_addr_t phys_addr, size_t size) +{ + phys_addr_t end = phys_addr + size; + + return (phys_addr < 0x80000000 || + (end > 0x200000000000 && phys_addr < 0x400000000000) || + (end > 0x600000000000 && phys_addr < 0x800000000000)); +} +#endif + +pgprot_t ioremap_map_prot(phys_addr_t phys_addr, size_t size, + unsigned long prot_val) +{ + pgprot_t prot = __pgprot(prot_val); +#ifdef CONFIG_ALTRA_ERRATUM_82288 + if (unlikely(have_altra_erratum_82288 && is_altra_pci(phys_addr, size))) { + prot = pgprot_device(prot); + } +#endif + return prot; +} + void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, unsigned long prot) { diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index a2ce4e08edf5a..8baf90ee3357c 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -6234,6 +6234,15 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0xa73f, dpc_log_size); DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0xa76e, dpc_log_size); #endif +#ifdef CONFIG_ALTRA_ERRATUM_82288 +static void quirk_altra_erratum_82288(struct pci_dev *dev) +{ + pr_info_once("Write combining PCI maps disabled due to hardware erratum\n"); + have_altra_erratum_82288 = true; +} +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMPERE, 0xe100, quirk_altra_erratum_82288); +#endif + /* * For a PCI device with multiple downstream devices, its driver may use * a flattened device tree to describe the downstream devices. diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index 80de699bf6af4..75670d7094537 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -1047,6 +1047,10 @@ static inline void iounmap(volatile void __iomem *addr) #elif defined(CONFIG_GENERIC_IOREMAP) #include +#ifndef ioremap_map_prot +#define ioremap_map_prot(phys_addr, size, prot) __pgprot(prot) +#endif + void __iomem *generic_ioremap_prot(phys_addr_t phys_addr, size_t size, pgprot_t prot); diff --git a/mm/ioremap.c b/mm/ioremap.c index 3e049dfb28bd0..a4e6950682f33 100644 --- a/mm/ioremap.c +++ b/mm/ioremap.c @@ -52,7 +52,7 @@ void __iomem *generic_ioremap_prot(phys_addr_t phys_addr, size_t size, void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, unsigned long prot) { - return generic_ioremap_prot(phys_addr, size, __pgprot(prot)); + return generic_ioremap_prot(phys_addr, size, ioremap_map_prot(phys_addr, size, prot)); } EXPORT_SYMBOL(ioremap_prot); #endif