From patchwork Fri Apr 9 13:24:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12194159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93B84C433B4 for ; Fri, 9 Apr 2021 13:26:33 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F20B610FB for ; Fri, 9 Apr 2021 13:26:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F20B610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=5JLROLG7s4eykpXUfWJBiZSM/yqDANHHN3+ho0fRn+0=; b=NsdHYpcmPQ6Gp9w6oyFJRHHoHV SVBTka9mu8w2OEh4hG9QpTqTDTRJX0XGWIloJV5HgGpGrjSOcuvewxb9rY6vWVZvqxvf9+uW3gAUU cPDU5dxXzGQeR+pBKiJEpxy0idjDPSUdPWxFy8ClSgs3zu4d9A/Orua5wAtXVe3L1qssf/+YT23DH yOblwVauRli+Io6E85ISpj5z7NHGSlGSeRykjasAEjP0og64thauIWS8HBQYEZf25R9lDt/s5T4TQ Kq1BnUvGEboincT7Po4hdPZodonA4ClpU3mXrneZHFjDeQvpsu5HeBoMlna23rA8ZjFQWk9phVA3/ RyAUU59Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUr7k-000pCb-9E; Fri, 09 Apr 2021 13:24:36 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lUr7i-000pCR-4L for linux-arm-kernel@desiato.infradead.org; Fri, 09 Apr 2021 13:24:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=NluZiwv49NXqZER+0T6Haj9/RQ114oeVnbMR0z5Jpis=; b=Z7WoM0K0i0jZme2sw+PoG4k2jD RDEhZrboynOat/jfNXnU/EcQuOTN300vcRRDRx+jwch+z3rTFhykMZgqr65ogHkWhzf72MjZ7C/D5 IOmm1s71ZFecVzf7zMu6kiIJnT7LiFZn2eyf0wR08R1BRDKJAqFc3prjVoFV/gvIJxtt62mK1hI4N pGcqManckfPEe7k6rjLIZIQK1N+sSSeUd3dPFlO5Nv+T74L+OlsEyOxrz3nKpVLYBf4HLEDh45jYf P2jQ8PSCwbXTNDJF81eksZHuecHXio3iV2TOyVG/KS/KnvFmIzxk54MdM/SDY43JyHUZypUsJN3cD 0slm8RMw==; Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUr7f-004Xv3-0L for linux-arm-kernel@lists.infradead.org; Fri, 09 Apr 2021 13:24:32 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 31B231FB; Fri, 9 Apr 2021 06:24:27 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2F5283F694; Fri, 9 Apr 2021 06:24:26 -0700 (PDT) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Catalin Marinas , Will Deacon , stable@vger.kernel.org Subject: [PATCH v3] arm64: mte: Move MTE TCF0 check in entry-common Date: Fri, 9 Apr 2021 14:24:19 +0100 Message-Id: <20210409132419.29965-1-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210409_062431_167036_BD0A032E X-CRM114-Status: GOOD ( 15.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The check_mte_async_tcf macro sets the TIF flag non-atomically. This can race with another CPU doing a set_tsk_thread_flag() and all the other flags can be lost in the process. Move the tcf0 check to enter_from_user_mode() and clear tcf0 in exit_to_user_mode() to address the problem. Note: Moving the check in entry-common allows to use set_thread_flag() which is safe. Fixes: 637ec831ea4f ("arm64: mte: Handle synchronous and asynchronous tag check faults") Cc: Catalin Marinas Cc: Will Deacon Cc: stable@vger.kernel.org Reported-by: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/mte.h | 9 +++++++++ arch/arm64/kernel/entry-common.c | 6 ++++++ arch/arm64/kernel/entry.S | 34 -------------------------------- arch/arm64/kernel/mte.c | 33 +++++++++++++++++++++++++++++-- 4 files changed, 46 insertions(+), 36 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 9b557a457f24..c7ab681a95c3 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -49,6 +49,9 @@ int mte_ptrace_copy_tags(struct task_struct *child, long request, void mte_assign_mem_tag_range(void *addr, size_t size); +void noinstr check_mte_async_tcf0(void); +void noinstr clear_mte_async_tcf0(void); + #else /* CONFIG_ARM64_MTE */ /* unused if !CONFIG_ARM64_MTE, silence the compiler */ @@ -83,6 +86,12 @@ static inline int mte_ptrace_copy_tags(struct task_struct *child, { return -EIO; } +static inline void check_mte_async_tcf0(void) +{ +} +static inline void clear_mte_async_tcf0(void) +{ +} static inline void mte_assign_mem_tag_range(void *addr, size_t size) { diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 9d3588450473..837d3624a1d5 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -289,10 +289,16 @@ asmlinkage void noinstr enter_from_user_mode(void) CT_WARN_ON(ct_state() != CONTEXT_USER); user_exit_irqoff(); trace_hardirqs_off_finish(); + + /* Check for asynchronous tag check faults in user space */ + check_mte_async_tcf0(); } asmlinkage void noinstr exit_to_user_mode(void) { + /* Ignore asynchronous tag check faults in the uaccess routines */ + clear_mte_async_tcf0(); + trace_hardirqs_on_prepare(); lockdep_hardirqs_on_prepare(CALLER_ADDR0); user_enter_irqoff(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a31a0a713c85..fb57df0d453f 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -34,15 +34,11 @@ * user and kernel mode. */ .macro user_exit_irqoff -#if defined(CONFIG_CONTEXT_TRACKING) || defined(CONFIG_TRACE_IRQFLAGS) bl enter_from_user_mode -#endif .endm .macro user_enter_irqoff -#if defined(CONFIG_CONTEXT_TRACKING) || defined(CONFIG_TRACE_IRQFLAGS) bl exit_to_user_mode -#endif .endm .macro clear_gp_regs @@ -147,32 +143,6 @@ alternative_cb_end .L__asm_ssbd_skip\@: .endm - /* Check for MTE asynchronous tag check faults */ - .macro check_mte_async_tcf, flgs, tmp -#ifdef CONFIG_ARM64_MTE -alternative_if_not ARM64_MTE - b 1f -alternative_else_nop_endif - mrs_s \tmp, SYS_TFSRE0_EL1 - tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f - /* Asynchronous TCF occurred for TTBR0 access, set the TI flag */ - orr \flgs, \flgs, #_TIF_MTE_ASYNC_FAULT - str \flgs, [tsk, #TSK_TI_FLAGS] - msr_s SYS_TFSRE0_EL1, xzr -1: -#endif - .endm - - /* Clear the MTE asynchronous tag check faults */ - .macro clear_mte_async_tcf -#ifdef CONFIG_ARM64_MTE -alternative_if ARM64_MTE - dsb ish - msr_s SYS_TFSRE0_EL1, xzr -alternative_else_nop_endif -#endif - .endm - .macro mte_set_gcr, tmp, tmp2 #ifdef CONFIG_ARM64_MTE /* @@ -243,8 +213,6 @@ alternative_else_nop_endif ldr x19, [tsk, #TSK_TI_FLAGS] disable_step_tsk x19, x20 - /* Check for asynchronous tag check faults in user space */ - check_mte_async_tcf x19, x22 apply_ssbd 1, x22, x23 ptrauth_keys_install_kernel tsk, x20, x22, x23 @@ -775,8 +743,6 @@ SYM_CODE_START_LOCAL(ret_to_user) cbnz x2, work_pending finish_ret_to_user: user_enter_irqoff - /* Ignore asynchronous tag check faults in the uaccess routines */ - clear_mte_async_tcf enable_step_tsk x19, x2 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index b3c70a612c7a..84a942c25870 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -166,14 +166,43 @@ static void set_gcr_el1_excl(u64 excl) */ } -void flush_mte_state(void) +void noinstr check_mte_async_tcf0(void) +{ + u64 tcf0; + + if (!system_supports_mte()) + return; + + /* + * dsb(ish) is not required before the register read + * because the TFSRE0_EL1 is automatically synchronized + * by the hardware on exception entry as SCTLR_EL1.ITFSB + * is set. + */ + tcf0 = read_sysreg_s(SYS_TFSRE0_EL1); + + if (tcf0 & SYS_TFSR_EL1_TF0) + set_thread_flag(TIF_MTE_ASYNC_FAULT); + + write_sysreg_s(0, SYS_TFSRE0_EL1); +} + +void noinstr clear_mte_async_tcf0(void) { if (!system_supports_mte()) return; - /* clear any pending asynchronous tag fault */ dsb(ish); write_sysreg_s(0, SYS_TFSRE0_EL1); +} + +void flush_mte_state(void) +{ + if (!system_supports_mte()) + return; + + /* clear any pending asynchronous tag fault */ + clear_mte_async_tcf0(); clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking */ set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE);