From patchwork Wed Nov 10 22:07:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12692201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8180AC433EF for ; Wed, 10 Nov 2021 22:09:24 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 53549619E0 for ; Wed, 10 Nov 2021 22:09:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 53549619E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Mime-Version: Message-Id:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=0WkL7Lqo+UxO5YzyO3w8PjgI34R6bisTOqAvVOs/ALY=; b=Y4h NH2ERi8J+deDkBnKNQSu0oAh38tEo3XskPYw3pGBLbLG6inPeoKbTnIgntoOgWl+yN1cVshg6g/H6 K9TdWqcW6UowUn2Kgk+erRbQnZ6TY/qd7Pb/+7N8jnpk9PomBJU6qEHkyC49l2xjbJJt2xHAt3FwN b1rCc12HN5FOZvv6b5tPW8xskfow+V8Svy5cHIPZiz3tmiWf+cQ81PaRMJPqWBu/nPH3mzv4eOwL9 D+blJ37hb25ABMo7EqqI/V/XEx/R+RJ6y0HNGCr4Hd4/E4QIq0BYWR0rieCsCiHGx8H+Nlqvvn6ei aHMdpeFZBdkvvAtrNDPBFhUR4LPlhFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mkvl2-006RJH-JU; Wed, 10 Nov 2021 22:07:52 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mkvky-006RIx-44 for linux-arm-kernel@lists.infradead.org; Wed, 10 Nov 2021 22:07:49 +0000 Received: by mail-pj1-x104a.google.com with SMTP id d7-20020a17090a7bc700b001a6ebe3f9cbso1372317pjl.0 for ; Wed, 10 Nov 2021 14:07:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=phpnnPSsRykNDDyQ9B5ySBAjVckLLPgvJMRYQ5VDfKo=; b=GcwdwtfNaDdISOY8z/kRJlJ6DbY8+fNFYp4WQJB8wsX8mPBCKApL8P/NMdt4fuBFXk mn/FE7ZsOVLNLseqfFJVJhn2fO4fMs3Wqawff+ujUKL3gOXW4MVtWqUr82TgIguNd9Dn Qrtqn1EPyFst9lcmCIY+z93CklqjtZI1oQhfHhfi1EeNl6kiuu51JtYpzYOaMo7HtTq5 pUnvLP6B0qE33g/PyLXo/r5dxPw/0Ts3fbapjpAe8TQznhWzgBVznOiigrEEo1zyS2w+ dHWiL8IHsyGKy2pN4YmMEMNwmpP09nGBWUKbmTnUtVvAAT/1jeTdYmr4HGlrWCwVXK8j LfCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=phpnnPSsRykNDDyQ9B5ySBAjVckLLPgvJMRYQ5VDfKo=; b=nYyPpSKKMneVgtxEM3KsawmQT9qPbD6DSstnTUPhUZuDV02J5Zc/tNA8uu7hs44W4J 1+dYisxDWssk0SqmTyhFX3l+KgOGD1QXl6QqbOm7MXbdJH/JxGOL3EXv3x56xxux6bFt pIoFxdXg/hy61lerz8mFZcG1HhT08UwliEM7y+IflPIwJ/z6ctZJtfYu4iaw2lO2Rf4F JJoak9xtoah40swrPPfDrNUXuuBp6DoQVvl26MCptxx2bg5lJbydNUcUWIt6Bui0Op6x TLZ/etTNJI/f0yX2+OoRtaRcudaEzGIT2qRgEQMF9haevXuHdN7SSGkC/tKL4hc2JRnY M95Q== X-Gm-Message-State: AOAM532MF9T1cJZfkqLHaBFNHIlPhSeqj4mPvrYIyROCDpEZbDJ8UqQd mK9bEDRICMqlHadGgOjPDSY4alw= X-Google-Smtp-Source: ABdhPJzwYXFuJii46rltzalTjrE6b+cYrY4huvvZ09X1VYOsQD3uSHi7k8gsYN5kSTdRHIB3zsP0yf0= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:ed7d:ed66:964d:8986]) (user=pcc job=sendgmr) by 2002:a17:90a:284f:: with SMTP id p15mr39223pjf.1.1636582062777; Wed, 10 Nov 2021 14:07:42 -0800 (PST) Date: Wed, 10 Nov 2021 14:07:35 -0800 Message-Id: <20211110220735.3937127-1-pcc@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.34.0.rc0.344.g81b53c2807-goog Subject: [PATCH v2] arm64: mte: avoid clearing PSTATE.TCO on entry unless necessary From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon , Andrey Konovalov , Mark Rutland Cc: Peter Collingbourne , Evgenii Stepanov , linux-arm-kernel@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211110_140748_212530_38C41549 X-CRM114-Status: GOOD ( 15.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On some microarchitectures, clearing PSTATE.TCO is expensive. Clearing TCO is only necessary if in-kernel MTE is enabled, or if MTE is enabled in the userspace process in synchronous (or, soon, asymmetric) mode, because we do not report uaccess faults to userspace in none or asynchronous modes. Therefore, adjust the kernel entry code to clear TCO only if necessary. Because it is now possible to switch to a task in which TCO needs to be clear from a task in which TCO is set, we also need to do the same thing on task switch. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/I52d82a580bd0500d420be501af2c35fa8c90729e --- v2: - do the same thing in cpu_switch_to() arch/arm64/kernel/entry.S | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 2f69ae43941d..a78ec15f5bbc 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -189,6 +189,27 @@ alternative_cb_end #endif .endm + .macro mte_clear_tco, sctlr + /* + * Re-enable tag checking (TCO set on exception entry). This is only + * necessary if MTE is enabled in either the kernel or the userspace + * task in synchronous mode. With MTE disabled in the kernel and + * disabled or asynchronous in userspace, tag check faults (including in + * uaccesses) are not reported, therefore there is no need to re-enable + * checking. This is beneficial on microarchitectures where re-enabling + * TCO is expensive. + */ +#ifdef CONFIG_ARM64_MTE +alternative_cb kasan_hw_tags_enable + tbz \sctlr, #SCTLR_EL1_TCF0_SHIFT, 1f +alternative_cb_end +alternative_if ARM64_MTE + SET_PSTATE_TCO(0) +alternative_else_nop_endif +1: +#endif + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -269,7 +290,11 @@ alternative_else_nop_endif .else add x21, sp, #PT_REGS_SIZE get_current_task tsk + ldr x0, [tsk, THREAD_SCTLR_USER] .endif /* \el == 0 */ + + mte_clear_tco x0 + mrs x22, elr_el1 mrs x23, spsr_el1 stp lr, x21, [sp, #S_LR] @@ -308,13 +333,6 @@ alternative_if ARM64_HAS_IRQ_PRIO_MASKING msr_s SYS_ICC_PMR_EL1, x20 alternative_else_nop_endif - /* Re-enable tag checking (TCO set on exception entry) */ -#ifdef CONFIG_ARM64_MTE -alternative_if ARM64_MTE - SET_PSTATE_TCO(0) -alternative_else_nop_endif -#endif - /* * Registers that may be useful after this macro is invoked: * @@ -742,6 +760,8 @@ SYM_FUNC_START(cpu_switch_to) ptrauth_keys_install_kernel x1, x8, x9, x10 scs_save x0 scs_load x1 + ldr x8, [x1, THREAD_SCTLR_USER] + mte_clear_tco x8 ret SYM_FUNC_END(cpu_switch_to) NOKPROBE(cpu_switch_to)