From patchwork Fri Feb 28 18:24:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Miko=C5=82aj_Lenczewski?= X-Patchwork-Id: 13996923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2DB1C282C6 for ; Fri, 28 Feb 2025 18:28:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=h1re+Y7X/2gcEbWaAwtp1zcYWAIt/E/bdei6T+vOaWI=; b=WTmEFgfOIb8KFZ10I9AzbCz6YE HPj1lj9tnbBt+ZV53JyrmLMv9LBaxe6K5Ny8dryPavS9Aqfy/C0mic5D8Kvz3txWkOqKGpOkzNEz7 xdWTcszAzZbX2Ts6J8P7npz2YR5nJ/l0pd+Caj5zT0/RYevOdpI3MrVJliNNSROan6+oyP8TgF3NF rrr1HS7LFtMj8Lt9lDlLN5bFj9WC19RDkqYQGuZQAn5s/ySSyAI4gcpUFx8Aifm8++vKv/otSxAEZ 9kT+5x/wQiqnfd0A4JChfpl4+MsI56Elp1QIcjIA7Bus/3jqiYjXF7rqsjCduJIVOnPPTPoJABQdS Zm8WxvRQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1to56A-0000000C8hV-3vqC; Fri, 28 Feb 2025 18:28:34 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1to52t-0000000C7dQ-49xS for linux-arm-kernel@lists.infradead.org; Fri, 28 Feb 2025 18:25:13 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9F5781515; Fri, 28 Feb 2025 10:25:26 -0800 (PST) Received: from mazurka.cambridge.arm.com (mazurka.cambridge.arm.com [10.2.80.18]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 413A03F6A8; Fri, 28 Feb 2025 10:25:08 -0800 (PST) From: =?utf-8?q?Miko=C5=82aj_Lenczewski?= To: ryan.roberts@arm.com, suzuki.poulose@arm.com, yang@os.amperecomputing.com, catalin.marinas@arm.com, will@kernel.org, joro@8bytes.org, jean-philippe@linaro.org, mark.rutland@arm.com, joey.gouly@arm.com, oliver.upton@linux.dev, james.morse@arm.com, broonie@kernel.org, maz@kernel.org, david@redhat.com, akpm@linux-foundation.org, jgg@ziepe.ca, nicolinc@nvidia.com, mshavit@google.com, jsnitsel@redhat.com, smostafa@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: =?utf-8?q?Miko=C5=82aj_Lenczewski?= Subject: [PATCH v2 1/4] arm64: Add BBM Level 2 cpu feature Date: Fri, 28 Feb 2025 18:24:01 +0000 Message-ID: <20250228182403.6269-3-miko.lenczewski@arm.com> X-Mailer: git-send-email 2.45.3 In-Reply-To: <20250228182403.6269-2-miko.lenczewski@arm.com> References: <20250228182403.6269-2-miko.lenczewski@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250228_102512_120429_D4C086F9 X-CRM114-Status: GOOD ( 21.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Break-Before-Make cpu feature supports multiple levels (levels 0-2), and this commit adds a dedicated BBML2 cpufeature to test against support for. This is a system feature as we might have a big.LITTLE architecture where some cores support BBML2 and some don't, but we want all cores to be available and BBM to default to level 0 (as opposed to having cores without BBML2 not coming online). To support BBML2 in as wide a range of contexts as we can, we want not only the architectural guarantees that BBML2 makes, but additionally want BBML2 to not create TLB conflict aborts. Not causing aborts avoids us having to prove that no recursive faults can be induced in any path that uses BBML2, allowing its use for arbitrary kernel mappings. Support detection of such CPUs. Signed-off-by: MikoĊ‚aj Lenczewski --- arch/arm64/Kconfig | 11 +++++ arch/arm64/include/asm/cpucaps.h | 2 + arch/arm64/include/asm/cpufeature.h | 5 +++ arch/arm64/kernel/cpufeature.c | 69 +++++++++++++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + 5 files changed, 88 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..baae6d458996 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2057,6 +2057,17 @@ config ARM64_TLB_RANGE The feature introduces new assembly instructions, and they were support when binutils >= 2.30. +config ARM64_ENABLE_BBML2_NOABORT + bool "Enable support for Break-Before-Make Level 2 detection and usage" + default y + help + FEAT_BBM provides detection of support levels for break-before-make + sequences. If BBM level 2 is supported, some TLB maintenance requirements + can be relaxed to improve performance. We additonally require the + property that the implementation cannot ever raise TLB Conflict Aborts. + Selecting N causes the kernel to fallback to BBM level 0 behaviour + even if the system supports BBM level 2. + endmenu # "ARMv8.4 architectural features" menu "ARMv8.5 architectural features" diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 0b5ca6e0eb09..2d6db33d4e45 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -23,6 +23,8 @@ cpucap_is_possible(const unsigned int cap) return IS_ENABLED(CONFIG_ARM64_PAN); case ARM64_HAS_EPAN: return IS_ENABLED(CONFIG_ARM64_EPAN); + case ARM64_HAS_BBML2_NOABORT: + return IS_ENABLED(CONFIG_ARM64_BBML2_NOABORT); case ARM64_SVE: return IS_ENABLED(CONFIG_ARM64_SVE); case ARM64_SME: diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index e0e4478f5fb5..108ef3fbbc00 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -866,6 +866,11 @@ static __always_inline bool system_supports_mpam_hcr(void) return alternative_has_cap_unlikely(ARM64_MPAM_HCR); } +static inline bool system_supports_bbml2_noabort(void) +{ + return alternative_has_cap_unlikely(ARM64_HAS_BBML2_NOABORT); +} + int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt); bool try_emulate_mrs(struct pt_regs *regs, u32 isn); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index d561cf3b8ac7..63f6d356dc77 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2176,6 +2176,68 @@ static bool hvhe_possible(const struct arm64_cpu_capabilities *entry, return arm64_test_sw_feature_override(ARM64_SW_FEATURE_OVERRIDE_HVHE); } +static bool cpu_has_bbml2_noabort(unsigned int cpu_midr) +{ + /* We want to allow usage of bbml2 in as wide a range of kernel contexts + * as possible. This list is therefore an allow-list of known-good + * implementations that both support bbml2 and additionally, fulfill the + * extra constraint of never generating TLB conflict aborts when using + * the relaxed bbml2 semantics (such aborts make use of bbml2 in certain + * kernel contexts difficult to prove safe against recursive aborts). + * + * Note that implementations can only be considered "known-good" if their + * implementors attest to the fact that the implementation never raises + * TLBI conflict aborts for bbml2 mapping granularity changes. + */ + static const struct midr_range supports_bbml2_noabort_list[] = { + MIDR_REV_RANGE(MIDR_CORTEX_X4, 0, 3, 0xf), + MIDR_REV_RANGE(MIDR_NEOVERSE_V3, 0, 2, 0xf), + {} + }; + + return is_midr_in_range_list(cpu_midr, supports_bbml2_noabort_list); +} + +static inline unsigned int __cpu_read_midr(int cpu) +{ + WARN_ON_ONCE(!cpu_online(cpu)); + + return per_cpu(cpu_data, cpu).reg_midr; +} + +static bool has_bbml2_noabort(const struct arm64_cpu_capabilities *caps, int scope) +{ + if (!IS_ENABLED(CONFIG_ARM64_ENABLE_BBML2_NOABORT)) + return false; + + if (scope & SCOPE_SYSTEM) { + int cpu; + + /* We are a boot CPU, and must verify that all enumerated boot + * CPUs have MIDR values within our allowlist. Otherwise, we do + * not allow the BBML2 feature to avoid potential faults when + * the insufficient CPUs access memory regions using BBML2 + * semantics. + */ + for_each_online_cpu(cpu) { + if (!cpu_has_bbml2_noabort(__cpu_read_midr(cpu))) + return false; + } + + return true; + } else if (scope & SCOPE_LOCAL_CPU) { + /* We are a hot-plugged CPU, so only need to check our MIDR. + * If we have the correct MIDR, but the kernel booted on an + * insufficient CPU, we will not use BBML2 (this is safe). If + * we have an incorrect MIDR, but the kernel booted on a + * sufficient CPU, we will not bring up this CPU. + */ + return cpu_has_bbml2_noabort(read_cpuid_id()); + } + + return false; +} + #ifdef CONFIG_ARM64_PAN static void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused) { @@ -2926,6 +2988,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature, ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, EVT, IMP) }, + { + .desc = "BBM Level 2 without conflict abort", + .capability = ARM64_HAS_BBML2_NOABORT, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_bbml2_noabort, + ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, BBM, 2) + }, { .desc = "52-bit Virtual Addressing for KVM (LPA2)", .capability = ARM64_HAS_LPA2, diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index 1e65f2fb45bd..b03a375e5507 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -14,6 +14,7 @@ HAS_ADDRESS_AUTH_ARCH_QARMA5 HAS_ADDRESS_AUTH_IMP_DEF HAS_AMU_EXTN HAS_ARMv8_4_TTL +HAS_BBML2_NOABORT HAS_CACHE_DIC HAS_CACHE_IDC HAS_CNP