From patchwork Wed May 11 21:41:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12846631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 509CDC433EF for ; Wed, 11 May 2022 21:41:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=miSWrYMBY0fblCfI7nDTaIysLCYWpfkBODeDNS42rIg=; b=GOh3jHnkvdlmPJ B/Gi3UDKu8kKU207D3AaZNqnC0uEoraivBI1JSj6LJycrN4EHcsOdMQYzb/ErnVEXWliHuvbnbW6v n0sUFXICDoqktu6ajsONtDgTYxyOZBUcHv+dR8B9JRqi9uH36syQxgdKoJmQHFqDD8n4XeERRA+mc AaGWrz0sXq7TDbmMKiBLUVdhXax9amUBH5Jm4cbtmWAiU7NJcy5otqaCfi1SBI6FFLbIU9nZD6iAn zP2n3EbuhPukSvb7omjwlHPiqvWUENVb1NsP7Fx+XKEI8bxGLGbDnzlDG4TqW04CulWqr/DGM/om7 EXg98B64fRL6Ap10kzDg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nou5T-008p4s-4s; Wed, 11 May 2022 21:41:39 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nou5Q-008p36-77 for linux-riscv@lists.infradead.org; Wed, 11 May 2022 21:41:37 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nou5O-0005NU-Ka; Wed, 11 May 2022 23:41:34 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, samuel@sholland.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, Heiko Stuebner Subject: [PATCH v2 1/3] dt-bindings: riscv: document cbom-block-size Date: Wed, 11 May 2022 23:41:30 +0200 Message-Id: <20220511214132.2281431-2-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220511214132.2281431-1-heiko@sntech.de> References: <20220511214132.2281431-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220511_144136_289489_97BD44FD X-CRM114-Status: UNSURE ( 9.66 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The Zicbom operates on a block-size defined for the cpu-core, which does not necessarily match other cache-sizes used. So add the necessary property for the system to know the core's block-size. Signed-off-by: Heiko Stuebner Reviewed-by: Anup Patel --- Documentation/devicetree/bindings/riscv/cpus.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml index d632ac76532e..b179bfd155a3 100644 --- a/Documentation/devicetree/bindings/riscv/cpus.yaml +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -63,6 +63,13 @@ properties: - riscv,sv48 - riscv,none + riscv,cbom-block-size: + $ref: /schemas/types.yaml#/definitions/uint32 + description: + Blocksize in bytes for the Zicbom cache operations. The block + size is a property of the core itself and does not necessarily + match other software defined cache sizes. + riscv,isa: description: Identifies the specific RISC-V instruction set architecture From patchwork Wed May 11 21:41:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12846632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A6B58C433F5 for ; Wed, 11 May 2022 21:41:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rn6OLVmgxS8T0W7iabOyXTerhPGtNVBlLUrHSVdZfY8=; b=Z8rWMS4b3M2iGl 4yKeUncXdJbFjFRfMWMcUB/C2ton1HuCWef6uZbGLatOf4OuXt+Qy6k7BFo7tbF50n5xqa6fCNzYg VKORyXUpJUDo0MqyeAUCPlhOdl/mtBEPGGnmIuAb834Suyx2PeHY/njWve0JbrJjkHtf7GvkIrS3E Z8ExJn1sLeu7e7REHEvkIrcf+6L3yyDsFrbvFohaUaZ+/nvwum23XAtC4xd8ItBcW1nk/elaBoJq+ CON2Zi8YeGgDB/M4/LNvpAAg5Oy6HzYQebei8fFgho0y5jrzsXSVzPu4EDN64F7BGaHi+oRCtGme9 Kt5b07kxNreNhVNyjQfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nou5W-008p6C-VT; Wed, 11 May 2022 21:41:42 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nou5R-008p3m-GQ for linux-riscv@lists.infradead.org; Wed, 11 May 2022 21:41:39 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nou5P-0005NU-7M; Wed, 11 May 2022 23:41:35 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, samuel@sholland.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, Heiko Stuebner , Christoph Hellwig , Atish Patra Subject: [PATCH v2 2/3] riscv: Implement Zicbom-based cache management operations Date: Wed, 11 May 2022 23:41:31 +0200 Message-Id: <20220511214132.2281431-3-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220511214132.2281431-1-heiko@sntech.de> References: <20220511214132.2281431-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220511_144137_742643_B58B42F4 X-CRM114-Status: GOOD ( 26.02 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The Zicbom ISA-extension was ratified in november 2021 and introduces instructions for dcache invalidate, clean and flush operations. Implement cache management operations based on them. Of course not all cores will support this, so implement an alternative-based mechanism that replaces empty instructions with ones done around Zicbom instructions. We're using prebuild instructions for the Zicbom instructions for now, to not require a bleeding-edge compiler (gcc-12) for these somewhat simple instructions. Signed-off-by: Heiko Stuebner Cc: Christoph Hellwig Cc: Atish Patra Cc: Guo Ren Reviewed-by: Anup Patel Reviewed-by: Guo Ren --- arch/riscv/Kconfig | 15 +++++ arch/riscv/include/asm/cacheflush.h | 6 ++ arch/riscv/include/asm/errata_list.h | 39 +++++++++++- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 17 +++++ arch/riscv/kernel/setup.c | 2 + arch/riscv/mm/Makefile | 1 + arch/riscv/mm/dma-noncoherent.c | 92 ++++++++++++++++++++++++++++ 9 files changed, 173 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/mm/dma-noncoherent.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 65285b980134..532db45367a7 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -363,6 +363,21 @@ config RISCV_ISA_SVPBMT If you don't know what to do here, say Y. +config RISCV_ISA_ZICBOM + bool "Zicbom extension support for non-coherent dma operation" + select ARCH_HAS_DMA_PREP_COHERENT + select ARCH_HAS_SYNC_DMA_FOR_DEVICE + select ARCH_HAS_SYNC_DMA_FOR_CPU + select ARCH_HAS_SETUP_DMA_OPS + select DMA_DIRECT_REMAP + select RISCV_ALTERNATIVE + default y + help + Adds support to dynamically detect the presence of the ZICBOM extension + (Cache Block Management Operations) and enable its usage. + + If you don't know what to do here, say Y. + config FPU bool "FPU support" default y diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index 23ff70350992..eb12d014b158 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -42,6 +42,12 @@ void flush_icache_mm(struct mm_struct *mm, bool local); #endif /* CONFIG_SMP */ +#ifdef CONFIG_RISCV_ISA_ZICBOM +void riscv_init_cbom_blocksize(void); +#else +static inline void riscv_init_cbom_blocksize(void) { } +#endif + /* * Bits in sys_riscv_flush_icache()'s flags argument. */ diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 9e2888dbb5b1..eebcd4415049 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -20,7 +20,8 @@ #endif #define CPUFEATURE_SVPBMT 0 -#define CPUFEATURE_NUMBER 1 +#define CPUFEATURE_CMO 1 +#define CPUFEATURE_NUMBER 2 #ifdef __ASSEMBLY__ @@ -93,6 +94,42 @@ asm volatile(ALTERNATIVE( \ #define ALT_THEAD_PMA(_val) #endif +/* + * cbo.clean rs1 + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0...01 rs1 010 00000 0001111 + * + * cbo.flush rs1 + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0...10 rs1 010 00000 0001111 + * + * cbo.inval rs1 + * | 31 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0...00 rs1 010 00000 0001111 + */ +#define CBO_INVAL_A0 ".long 0x15200F" +#define CBO_CLEAN_A0 ".long 0x25200F" +#define CBO_FLUSH_A0 ".long 0x05200F" + +#define ALT_CMO_OP(_op, _start, _size, _cachesize) \ +asm volatile(ALTERNATIVE( \ + "nop\n\t" \ + "nop\n\t" \ + "nop\n\t" \ + "nop\n\t" \ + "nop", \ + "mv a0, %1\n\t" \ + "j 2f\n\t" \ + "3:\n\t" \ + CBO_##_op##_A0 "\n\t" \ + "add a0, a0, %0\n\t" \ + "2:\n\t" \ + "bltu a0, %2, 3b\n\t", 0, \ + CPUFEATURE_CMO, CONFIG_RISCV_ISA_ZICBOM) \ + : : "r"(_cachesize), \ + "r"(ALIGN((_start), (_cachesize))), \ + "r"(ALIGN((_start) + (_size), (_cachesize)))) + #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 4e2486881840..6044e402003d 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -53,6 +53,7 @@ extern unsigned long elf_hwcap; enum riscv_isa_ext_id { RISCV_ISA_EXT_SSCOFPMF = RISCV_ISA_EXT_BASE, RISCV_ISA_EXT_SVPBMT, + RISCV_ISA_EXT_ZICBOM, RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX, }; diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index 40c8776aec12..8f6fc15baa8e 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -89,6 +89,7 @@ int riscv_of_parent_hartid(struct device_node *node) static struct riscv_isa_ext_data isa_ext_arr[] = { __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), + __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), }; diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index dea3ea19deee..db3c02409a4a 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -199,6 +199,7 @@ void __init riscv_fill_hwcap(void) } else { SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF); SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); + SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); } #undef SET_ISA_EXT_MAP } @@ -265,12 +266,28 @@ static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage) return false; } +static bool __init_or_module cpufeature_cmo_check_func(unsigned int stage) +{ + switch (stage) { + case RISCV_ALTERNATIVES_EARLY_BOOT: + return false; + default: + return riscv_isa_extension_available(NULL, ZICBOM); + } + + return false; +} + static const struct cpufeature_info __initdata_or_module cpufeature_list[CPUFEATURE_NUMBER] = { { .name = "svpbmt", .check_func = cpufeature_svpbmt_check_func }, + { + .name = "cmo", + .check_func = cpufeature_cmo_check_func + }, }; static u32 __init_or_module cpufeature_probe(unsigned int stage) diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index 9162e9a824d2..cc5bfeba499a 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -296,6 +297,7 @@ void __init setup_arch(char **cmdline_p) #endif riscv_fill_hwcap(); + riscv_init_cbom_blocksize(); apply_boot_alternatives(); } diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index ac7a25298a04..548f2f3c00e9 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -30,3 +30,4 @@ endif endif obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o +obj-$(CONFIG_RISCV_ISA_ZICBOM) += dma-noncoherent.o diff --git a/arch/riscv/mm/dma-noncoherent.c b/arch/riscv/mm/dma-noncoherent.c new file mode 100644 index 000000000000..99decaa25324 --- /dev/null +++ b/arch/riscv/mm/dma-noncoherent.c @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RISC-V specific functions to support DMA for non-coherent devices + * + * Copyright (c) 2021 Western Digital Corporation or its affiliates. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static unsigned int riscv_cbom_block_size = L1_CACHE_BYTES; + +void arch_sync_dma_for_device(phys_addr_t paddr, size_t size, enum dma_data_direction dir) +{ + switch (dir) { + case DMA_TO_DEVICE: + ALT_CMO_OP(CLEAN, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); + break; + case DMA_FROM_DEVICE: + ALT_CMO_OP(INVAL, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); + break; + case DMA_BIDIRECTIONAL: + ALT_CMO_OP(FLUSH, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); + break; + default: + break; + } +} + +void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, enum dma_data_direction dir) +{ + switch (dir) { + case DMA_TO_DEVICE: + break; + case DMA_FROM_DEVICE: + case DMA_BIDIRECTIONAL: + ALT_CMO_OP(INVAL, (unsigned long)phys_to_virt(paddr), size, riscv_cbom_block_size); + break; + default: + break; + } +} + +void arch_dma_prep_coherent(struct page *page, size_t size) +{ + void *flush_addr = page_address(page); + + memset(flush_addr, 0, size); + ALT_CMO_OP(FLUSH, (unsigned long)flush_addr, size, riscv_cbom_block_size); +} + +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, + const struct iommu_ops *iommu, bool coherent) +{ + /* If a specific device is dma-coherent, set it here */ + dev->dma_coherent = coherent; +} + +void riscv_init_cbom_blocksize(void) +{ + struct device_node *node; + int ret; + u32 val; + + for_each_of_cpu_node(node) { + int hartid = riscv_of_processor_hartid(node); + int cbom_hartid; + + if (hartid < 0) + continue; + + /* set block-size for cbom extension if available */ + ret = of_property_read_u32(node, "riscv,cbom-block-size", &val); + if (ret) + continue; + + if (!riscv_cbom_block_size) { + riscv_cbom_block_size = val; + cbom_hartid = hartid; + } else { + if (riscv_cbom_block_size != val) + pr_warn("cbom-block-size mismatched between harts %d and %d\n", + cbom_hartid, hartid); + } + } +} From patchwork Wed May 11 21:41:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12846633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1568C4332F for ; Wed, 11 May 2022 21:41:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=c1ZEZ35Y4/IIi+RgB/OY1a1wvIX5it57z62g/GkrlmE=; b=Mmi/Yx2ifbLKx7 VQcB8VXknKR0IvR4XZ7YaU5vLKHFZKB1kRTFoBAJ5jOP2FkeNeeP6AJxKS8Nj+N+KJixRmHBhSuwF yivaT69LHipQVm7sxMylA91dF6QTX98UHSVsPoB469Nt8AtTiacQQ59S4wXi0h9U5nUEz7uYMwq60 m4LpxD285o2VoQAO/So8+WHE8Qc5n336fdgUB/pfCWLxlJU+DgP6Ea68dDxe8oaW8/wGm4B8BfIDm Cgf6FEwyLdGrWycsVY2qV8n21/COAfjJFKoQ0CyAkhP3cDn5eUfBblyAVoP6PG50glE5si/GukkSe G+MtR2aqic6zIrPC07Hg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nou5V-008p5Y-I9; Wed, 11 May 2022 21:41:41 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nou5R-008p3o-P5 for linux-riscv@lists.infradead.org; Wed, 11 May 2022 21:41:39 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nou5P-0005NU-OT; Wed, 11 May 2022 23:41:35 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, samuel@sholland.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, Heiko Stuebner Subject: [PATCH v2 3/3] riscv: implement cache-management errata for T-Head SoCs Date: Wed, 11 May 2022 23:41:32 +0200 Message-Id: <20220511214132.2281431-4-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220511214132.2281431-1-heiko@sntech.de> References: <20220511214132.2281431-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220511_144137_862879_DC1ADF8D X-CRM114-Status: GOOD ( 14.19 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The T-Head C906 and C910 implement a scheme for handling cache operations different from the generic Zicbom extension. Add an errata for it next to the generic dma coherency ops. Tested-by: Samuel Holland Signed-off-by: Heiko Stuebner Reviewed-by: Anup Patel Reviewed-by: Guo Ren --- arch/riscv/Kconfig.erratas | 10 ++++++ arch/riscv/errata/thead/errata.c | 5 +++ arch/riscv/include/asm/errata_list.h | 47 +++++++++++++++++++++++++--- 3 files changed, 58 insertions(+), 4 deletions(-) diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas index ebfcd5cc6eaf..213629bac5d7 100644 --- a/arch/riscv/Kconfig.erratas +++ b/arch/riscv/Kconfig.erratas @@ -54,4 +54,14 @@ config ERRATA_THEAD_PBMT If you don't know what to do here, say "Y". +config ERRATA_THEAD_CMO + bool "Apply T-Head cache management errata" + depends on ERRATA_THEAD && RISCV_ISA_ZICBOM + default y + help + This will apply the cache management errata to handle the + non-standard handling on non-coherent operations on T-Head SoCs. + + If you don't know what to do here, say "Y". + endmenu diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c index e5d75270b99c..9545f43d3504 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -33,6 +33,11 @@ static const struct errata_info errata_list[ERRATA_THEAD_NUMBER] = { .stage = RISCV_ALTERNATIVES_EARLY_BOOT, .check_func = errata_mt_check_func }, + { + .name = "cache-management", + .stage = RISCV_ALTERNATIVES_BOOT, + .check_func = errata_mt_check_func + }, }; static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index eebcd4415049..1da311fc5126 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -16,7 +16,8 @@ #ifdef CONFIG_ERRATA_THEAD #define ERRATA_THEAD_PBMT 0 -#define ERRATA_THEAD_NUMBER 1 +#define ERRATA_THEAD_CMO 1 +#define ERRATA_THEAD_NUMBER 2 #endif #define CPUFEATURE_SVPBMT 0 @@ -111,8 +112,37 @@ asm volatile(ALTERNATIVE( \ #define CBO_CLEAN_A0 ".long 0x25200F" #define CBO_FLUSH_A0 ".long 0x05200F" +/* + * dcache.ipa rs1 (invalidate, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01010 rs1 000 00000 0001011 + * dache.iva rs1 (invalida, virtual address) + * 0000001 00110 rs1 000 00000 0001011 + * + * dcache.cpa rs1 (clean, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01001 rs1 000 00000 0001011 + * dcache.cva rs1 (clean, virtual address) + * 0000001 00100 rs1 000 00000 0001011 + * + * dcache.cipa rs1 (clean then invalidate, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01011 rs1 000 00000 0001011 + * dcache.civa rs1 (... virtual address) + * 0000001 00111 rs1 000 00000 0001011 + * + * sync.s (make sure all cache operations finished) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000000 11001 00000 000 00000 0001011 + */ +#define THEAD_INVAL_A0 ".long 0x0265000b" +#define THEAD_CLEAN_A0 ".long 0x0245000b" +#define THEAD_FLUSH_A0 ".long 0x0275000b" +#define THEAD_SYNC_S ".long 0x0190000b" + #define ALT_CMO_OP(_op, _start, _size, _cachesize) \ -asm volatile(ALTERNATIVE( \ +asm volatile(ALTERNATIVE_2( \ + "nop\n\t" \ "nop\n\t" \ "nop\n\t" \ "nop\n\t" \ @@ -124,8 +154,17 @@ asm volatile(ALTERNATIVE( \ CBO_##_op##_A0 "\n\t" \ "add a0, a0, %0\n\t" \ "2:\n\t" \ - "bltu a0, %2, 3b\n\t", 0, \ - CPUFEATURE_CMO, CONFIG_RISCV_ISA_ZICBOM) \ + "bltu a0, %2, 3b\n\t" \ + "nop", 0, CPUFEATURE_CMO, CONFIG_RISCV_ISA_ZICBOM, \ + "mv a0, %1\n\t" \ + "j 2f\n\t" \ + "3:\n\t" \ + THEAD_##_op##_A0 "\n\t" \ + "add a0, a0, %0\n\t" \ + "2:\n\t" \ + "bltu a0, %2, 3b\n\t" \ + THEAD_SYNC_S, THEAD_VENDOR_ID, \ + ERRATA_THEAD_CMO, CONFIG_ERRATA_THEAD_CMO) \ : : "r"(_cachesize), \ "r"(ALIGN((_start), (_cachesize))), \ "r"(ALIGN((_start) + (_size), (_cachesize))))