From patchwork Fri Feb 21 14:50:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 11396749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0867114BC for ; Fri, 21 Feb 2020 14:50:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9CA1020656 for ; Fri, 21 Feb 2020 14:50:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="SEffTNco" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9CA1020656 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=04ON5wLWoMvKn1urH2370mltOIYIYIv152t2DoEbbMU=; b=SEffTNcotf88IiX0EGtDkalO14 wCTHZHfHK+LU8EwYVrY5UcIeezAef/3y0WSlPIDae+NM1FamCNbISVbBch/7SPHKuWonlbRPGgCFb tIP35rwt7mXjzn9m69Zav7Kxj8AQoXrLZzA64qhzdqunnINxJN3qFP/d6884Fk5CHp1KOzXe8yCG0 vvB1upZQYu479yOYmM2N3FRdO9S/TeEIF3d43nKVuBNxjRsTXjtCuLhrvh82ZbrfX0nSaR9KAGaFi HTfaydVCMn7Ekl6+COL2qnrNLdg6c+0ww3EaYrOookRGAU2+ChSgKueMDnmX7UW/wdxhsl0Tk5fBE SKh7NiIA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j59da-0002TJ-Eu; Fri, 21 Feb 2020 14:50:42 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j59dR-0002Kf-Bc for linux-arm-kernel@lists.infradead.org; Fri, 21 Feb 2020 14:50:35 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8193030E; Fri, 21 Feb 2020 06:50:32 -0800 (PST) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4D6A93F703; Fri, 21 Feb 2020 06:50:31 -0800 (PST) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Subject: [PATCHv2 1/2] arm64: cpufeature: add cpus_have_final_cap() Date: Fri, 21 Feb 2020 14:50:21 +0000 Message-Id: <20200221145022.27217-2-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20200221145022.27217-1-mark.rutland@arm.com> References: <20200221145022.27217-1-mark.rutland@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200221_065033_481716_CC95CFEF X-CRM114-Status: GOOD ( 15.83 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.3 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [217.140.110.172 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, james.morse@arm.com, maz@kernel.org, will@kernel.org, kvmarm@lists.cs.columbia.edu, julien.thierry.kdev@gmail.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org When cpus_have_const_cap() was originally introduced it was intended to be safe in hyp context, where it is not safe to access the cpu_hwcaps array as cpus_have_cap() did. For more details see commit: a4023f682739439b ("arm64: Add hypervisor safe helper for checking constant capabilities") We then made use of cpus_have_const_cap() throughout the kernel. Subsequently, we had to defer updating the static_key associated with each capability in order to avoid lockdep complaints. To avoid breaking kernel-wide usage of cpus_have_const_cap(), this was updated to fall back to the cpu_hwcaps array if called before the static_keys were updated. As the kvm hyp code was only called later than this, the fallback is redundant but not functionally harmful. For more details, see commit: 63a1e1c95e60e798 ("arm64/cpufeature: don't use mutex in bringup path") Today we have more users of cpus_have_const_cap() which are only called once the relevant static keys are initialized, and it would be beneficial to avoid the redundant code. To that end, this patch adds a new cpus_have_final_cap(), helper which is intend to be used in code which is only run once capabilities have been finalized, and will never check the cpus_hwcap array. This helps the compiler to generate better code as it no longer needs to generate code to address and test the cpus_hwcap array. To help catch misuse, cpus_have_final_cap() will BUG() if called before capabilities are finalized. In hyp context, BUG() will result in a hyp panic, but the specific BUG() instance will not be identified in the usual way. Comments are added to the various cpus_have_*_cap() helpers to describe the constraints on when they can be used. For clarity cpus_have_cap() is moved above the other helpers. Similarly the helpers are updated to use system_capabilities_finalized() consistently, and this is made __always_inline as required by its new callers. Signed-off-by: Mark Rutland Reviewed-by: Marc Zyngier Reviewed-by: Suzuki K Poulose Cc: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/cpufeature.h | 58 ++++++++++++++++++++++++++++++------- 1 file changed, 47 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 92ef9539874a..940b2b67b428 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -390,14 +390,16 @@ unsigned long cpu_get_elf_hwcap2(void); #define cpu_set_named_feature(name) cpu_set_feature(cpu_feature(name)) #define cpu_have_named_feature(name) cpu_have_feature(cpu_feature(name)) -/* System capability check for constant caps */ -static __always_inline bool __cpus_have_const_cap(int num) +static __always_inline bool system_capabilities_finalized(void) { - if (num >= ARM64_NCAPS) - return false; - return static_branch_unlikely(&cpu_hwcap_keys[num]); + return static_branch_likely(&arm64_const_caps_ready); } +/* + * Test for a capability with a runtime check. + * + * Before the capability is detected, this returns false. + */ static inline bool cpus_have_cap(unsigned int num) { if (num >= ARM64_NCAPS) @@ -405,14 +407,53 @@ static inline bool cpus_have_cap(unsigned int num) return test_bit(num, cpu_hwcaps); } +/* + * Test for a capability without a runtime check. + * + * Before capabilities are finalized, this returns false. + * After capabilities are finalized, this is patched to avoid a runtime check. + * + * @num must be a compile-time constant. + */ +static __always_inline bool __cpus_have_const_cap(int num) +{ + if (num >= ARM64_NCAPS) + return false; + return static_branch_unlikely(&cpu_hwcap_keys[num]); +} + +/* + * Test for a capability, possibly with a runtime check. + * + * Before capabilities are finalized, this behaves as cpus_have_cap(). + * After capabilities are finalized, this is patched to avoid a runtime check. + * + * @num must be a compile-time constant. + */ static __always_inline bool cpus_have_const_cap(int num) { - if (static_branch_likely(&arm64_const_caps_ready)) + if (system_capabilities_finalized()) return __cpus_have_const_cap(num); else return cpus_have_cap(num); } +/* + * Test for a capability without a runtime check. + * + * Before capabilities are finalized, this will BUG(). + * After capabilities are finalized, this is patched to avoid a runtime check. + * + * @num must be a compile-time constant. + */ +static __always_inline bool cpus_have_final_cap(int num) +{ + if (system_capabilities_finalized()) + return __cpus_have_const_cap(num); + else + BUG(); +} + static inline void cpus_set_cap(unsigned int num) { if (num >= ARM64_NCAPS) { @@ -613,11 +654,6 @@ static inline bool system_has_prio_mask_debugging(void) system_uses_irq_prio_masking(); } -static inline bool system_capabilities_finalized(void) -{ - return static_branch_likely(&arm64_const_caps_ready); -} - #define ARM64_BP_HARDEN_UNKNOWN -1 #define ARM64_BP_HARDEN_WA_NEEDED 0 #define ARM64_BP_HARDEN_NOT_REQUIRED 1