From patchwork Thu Apr 8 14:37:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12191653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 534CFC433B4 for ; Thu, 8 Apr 2021 14:39:17 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E5437610E8 for ; Thu, 8 Apr 2021 14:39:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5437610E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=vBszo41bjMf1TPFPH4WYWL/G2I/iURGOwrHvtAKrPWE=; b=CU+1UrmPrzF5E0ZFjiS/9frPNT WguiNNgf0kuc9M9XFC0r2MnjhlZtwJeOTdUIhC+Gl1I4XpsmQDU0oXEvnISGcwG+3AA2jtUaxkb/n RHvLJvn20TsXIL+yvdVm+XD5MFWKbt7zPrSflKqlOHPJsz9tgPLBRRGJpxiuQX5shb0J50ONWcuir 4ACcKGt+zhdj1N9AbqnV2C0LueoYqXrOwHbsZtha57cRbCE+ixMVoFtca3YLdclS4W818tQGxpU6V VswmaNQ7bHUZtLmL1QTMFEFGnAYRaFYbFe0nLsb+zhWDxX7F6+2swCq8EMqO04FEvS3RYhxVsydtf CSQ72OPg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUVmw-008GaA-Ap; Thu, 08 Apr 2021 14:37:42 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUVmq-008GZC-1A for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 14:37:38 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 08229D6E; Thu, 8 Apr 2021 07:37:33 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1F32A3F694; Thu, 8 Apr 2021 07:37:32 -0700 (PDT) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Catalin Marinas , Will Deacon Subject: [PATCH] arm64: mte: Move MTE TCF0 check in entry-common Date: Thu, 8 Apr 2021 15:37:23 +0100 Message-Id: <20210408143723.13024-1-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_153736_420917_D239CA50 X-CRM114-Status: GOOD ( 14.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The check_mte_async_tcf macro sets the TIF flag non-atomically. This can race with another CPU doing a set_tsk_thread_flag() and the flag can be lost in the process. Move the tcf0 check to enter_from_user_mode() and clear tcf0 in exit_to_user_mode() to address the problem. Note: Moving the check in entry-common allows to use set_thread_flag() which is safe. Fixes: 637ec831ea4f ("arm64: mte: Handle synchronous and asynchronous tag check faults") Cc: Catalin Marinas Cc: Will Deacon Reported-by: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/mte.h | 8 ++++++++ arch/arm64/kernel/entry-common.c | 6 ++++++ arch/arm64/kernel/entry.S | 30 ------------------------------ arch/arm64/kernel/mte.c | 25 +++++++++++++++++++++++-- 4 files changed, 37 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 9b557a457f24..188f778c6f7b 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -31,6 +31,8 @@ void mte_invalidate_tags(int type, pgoff_t offset); void mte_invalidate_tags_area(int type); void *mte_allocate_tag_storage(void); void mte_free_tag_storage(char *storage); +void check_mte_async_tcf0(void); +void clear_mte_async_tcf0(void); #ifdef CONFIG_ARM64_MTE @@ -83,6 +85,12 @@ static inline int mte_ptrace_copy_tags(struct task_struct *child, { return -EIO; } +void check_mte_async_tcf0(void) +{ +} +void clear_mte_async_tcf0(void) +{ +} static inline void mte_assign_mem_tag_range(void *addr, size_t size) { diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 9d3588450473..837d3624a1d5 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -289,10 +289,16 @@ asmlinkage void noinstr enter_from_user_mode(void) CT_WARN_ON(ct_state() != CONTEXT_USER); user_exit_irqoff(); trace_hardirqs_off_finish(); + + /* Check for asynchronous tag check faults in user space */ + check_mte_async_tcf0(); } asmlinkage void noinstr exit_to_user_mode(void) { + /* Ignore asynchronous tag check faults in the uaccess routines */ + clear_mte_async_tcf0(); + trace_hardirqs_on_prepare(); lockdep_hardirqs_on_prepare(CALLER_ADDR0); user_enter_irqoff(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a31a0a713c85..fafd74ae5021 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -147,32 +147,6 @@ alternative_cb_end .L__asm_ssbd_skip\@: .endm - /* Check for MTE asynchronous tag check faults */ - .macro check_mte_async_tcf, flgs, tmp -#ifdef CONFIG_ARM64_MTE -alternative_if_not ARM64_MTE - b 1f -alternative_else_nop_endif - mrs_s \tmp, SYS_TFSRE0_EL1 - tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f - /* Asynchronous TCF occurred for TTBR0 access, set the TI flag */ - orr \flgs, \flgs, #_TIF_MTE_ASYNC_FAULT - str \flgs, [tsk, #TSK_TI_FLAGS] - msr_s SYS_TFSRE0_EL1, xzr -1: -#endif - .endm - - /* Clear the MTE asynchronous tag check faults */ - .macro clear_mte_async_tcf -#ifdef CONFIG_ARM64_MTE -alternative_if ARM64_MTE - dsb ish - msr_s SYS_TFSRE0_EL1, xzr -alternative_else_nop_endif -#endif - .endm - .macro mte_set_gcr, tmp, tmp2 #ifdef CONFIG_ARM64_MTE /* @@ -243,8 +217,6 @@ alternative_else_nop_endif ldr x19, [tsk, #TSK_TI_FLAGS] disable_step_tsk x19, x20 - /* Check for asynchronous tag check faults in user space */ - check_mte_async_tcf x19, x22 apply_ssbd 1, x22, x23 ptrauth_keys_install_kernel tsk, x20, x22, x23 @@ -775,8 +747,6 @@ SYM_CODE_START_LOCAL(ret_to_user) cbnz x2, work_pending finish_ret_to_user: user_enter_irqoff - /* Ignore asynchronous tag check faults in the uaccess routines */ - clear_mte_async_tcf enable_step_tsk x19, x2 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index b3c70a612c7a..e759b0eca47e 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -166,14 +166,35 @@ static void set_gcr_el1_excl(u64 excl) */ } +void check_mte_async_tcf0(void) +{ + /* + * dsb(ish) is not required before the register read + * because the TFSRE0_EL1 is automatically synchronized + * by the hardware on exception entry as SCTLR_EL1.ITFSB + * is set. + */ + u64 tcf0 = read_sysreg_s(SYS_TFSRE0_EL1); + + if (tcf0 & SYS_TFSR_EL1_TF0) + set_thread_flag(TIF_MTE_ASYNC_FAULT); + + write_sysreg_s(0, SYS_TFSRE0_EL1); +} + +void clear_mte_async_tcf0(void) +{ + dsb(ish); + write_sysreg_s(0, SYS_TFSRE0_EL1); +} + void flush_mte_state(void) { if (!system_supports_mte()) return; /* clear any pending asynchronous tag fault */ - dsb(ish); - write_sysreg_s(0, SYS_TFSRE0_EL1); + clear_mte_async_tcf0(); clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking */ set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE);