From patchwork Fri Mar 12 14:22:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AE39C433DB for ; Fri, 12 Mar 2021 14:24:09 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E73D64FD6 for ; Fri, 12 Mar 2021 14:24:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E73D64FD6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8KjRPcLnUQMBXMIQTVAyARHzh/1dx0B45eSLmEApWgM=; b=B3/zIreikO63xf1TU1x+aARnK UlNeV9mEyQcFBmDZi2YCtxhiukW1e5C+2hCqWiLeJmnn1Gbrt/+Cxl736yNJfYakPZU13kPnLc5Bv E+6NGCMpYvFxdV8sEg2rE+Lh/yEb9plrQk/dRPyu9QaVprM0LnkKwRn0obzDZqaY8HLKFySM80LG5 RFDwAcK50AzsHOgX/TUtNPa3azIDYAY+XbGVyP+Rjv3BxWhNYIaY2Y+KaA1t8Si882Px/DG6g1DWN wSWefIKIypqGOxRIOEVCmO4XJ3JCuoxON01Cgn8n8NFLQbc2HbKnZ1XFEKaFU18iBAa2qdcv2zIE8 3Gh8UPqFw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigS-00Bccn-6O; Fri, 12 Mar 2021 14:22:32 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigN-00BcaB-QE for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E06441063; Fri, 12 Mar 2021 06:22:24 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0672C3F793; Fri, 12 Mar 2021 06:22:22 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 1/8] arm64: mte: Add asynchronous mode support Date: Fri, 12 Mar 2021 14:22:03 +0000 Message-Id: <20210312142210.21326-2-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142228_238260_87582133 X-CRM114-Status: GOOD ( 12.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MTE provides an asynchronous mode for detecting tag exceptions. In particular instead of triggering a fault the arm64 core updates a register which is checked by the kernel after the asynchronous tag check fault has occurred. Add support for MTE asynchronous mode. The exception handling mechanism will be added with a future patch. Note: KASAN HW activates async mode via kasan.mode kernel parameter. The default mode is set to synchronous. The code that verifies the status of TFSR_EL1 will be added with a future patch. Cc: Catalin Marinas Cc: Will Deacon Reviewed-by: Catalin Marinas Reviewed-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/memory.h | 4 +++- arch/arm64/include/asm/mte-kasan.h | 9 +++++++-- arch/arm64/kernel/mte.c | 16 ++++++++++++++-- 3 files changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 0aabc3be9a75..8fbc8dab044f 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -243,7 +243,9 @@ static inline const void *__tag_set(const void *addr, u8 tag) } #ifdef CONFIG_KASAN_HW_TAGS -#define arch_enable_tagging() mte_enable_kernel() +#define arch_enable_tagging_sync() mte_enable_kernel_sync() +#define arch_enable_tagging_async() mte_enable_kernel_async() +#define arch_enable_tagging() arch_enable_tagging_sync() #define arch_set_tagging_report_once(state) mte_set_report_once(state) #define arch_init_tags(max_tag) mte_init_tags(max_tag) #define arch_get_random_tag() mte_get_random_tag() diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 7ab500e2ad17..4acf8bf41cad 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -77,7 +77,8 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag) } while (curr != end); } -void mte_enable_kernel(void); +void mte_enable_kernel_sync(void); +void mte_enable_kernel_async(void); void mte_init_tags(u64 max_tag); void mte_set_report_once(bool state); @@ -104,7 +105,11 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag) { } -static inline void mte_enable_kernel(void) +static inline void mte_enable_kernel_sync(void) +{ +} + +static inline void mte_enable_kernel_async(void) { } diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index b3c70a612c7a..fa755cf94e01 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -107,11 +107,23 @@ void mte_init_tags(u64 max_tag) write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1); } -void mte_enable_kernel(void) +static inline void __mte_enable_kernel(const char *mode, unsigned long tcf) { /* Enable MTE Sync Mode for EL1. */ - sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, tcf); isb(); + + pr_info_once("MTE: enabled in %s mode at EL1\n", mode); +} + +void mte_enable_kernel_sync(void) +{ + __mte_enable_kernel("synchronous", SCTLR_ELx_TCF_SYNC); +} + +void mte_enable_kernel_async(void) +{ + __mte_enable_kernel("asynchronous", SCTLR_ELx_TCF_ASYNC); } void mte_set_report_once(bool state) From patchwork Fri Mar 12 14:22:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A9CEC433E6 for ; Fri, 12 Mar 2021 14:24:10 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8FC564FAD for ; Fri, 12 Mar 2021 14:24:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8FC564FAD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=4PLYvTP1dZYphBYo2A/gYI6cFsAFbe5roI1KsSvd60Y=; b=AkDcOacbDElWurwBtN4Mk7jz2 mwsKe4kTiaw9xqUxiprHX+9s49ZayCPD53ieTRDPQp9/VNSgawIBBx+p8A+LYtZ4LCf7qGLDFn2Ml bGr9QAPjK1zOjeQYfzsX6RUtmAmT2nhSIUZFP8MkHK8+TlKIU/K2621W79mP6LEe2fUa2Gq6YU9VL bbiO6UMTOPHg0GxXA+FJ5q1GaotAV8AMoZZpt/1KdODlCxNsaCBOE/i4vKmGF8bYOhSmO1a6TZpke 5tfXjJBgZPzhdD3I6xnh8hKStknUAj1cpDFbfB31nRdUydYHYaeDyS8Sa54HMwLjwXoD6bYWrwA4Y A6OZgwDaw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigb-00Bcha-6a; Fri, 12 Mar 2021 14:22:41 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigO-00BcaR-RI for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:31 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1F63011B3; Fri, 12 Mar 2021 06:22:27 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 20DEC3F793; Fri, 12 Mar 2021 06:22:25 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 2/8] kasan: Add KASAN mode kernel parameter Date: Fri, 12 Mar 2021 14:22:04 +0000 Message-Id: <20210312142210.21326-3-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142229_191636_C4AB4335 X-CRM114-Status: GOOD ( 24.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Architectures supported by KASAN_HW_TAGS can provide a sync or async mode of execution. On an MTE enabled arm64 hw for example this can be identified with the synchronous or asynchronous tagging mode of execution. In synchronous mode, an exception is triggered if a tag check fault occurs. In asynchronous mode, if a tag check fault occurs, the TFSR_EL1 register is updated asynchronously. The kernel checks the corresponding bits periodically. KASAN requires a specific kernel command line parameter to make use of this hw features. Add KASAN HW execution mode kernel command line parameter. Note: This patch adds the kasan.mode kernel parameter and the sync/async kernel command line options to enable the described features. Cc: Dmitry Vyukov Cc: Andrey Ryabinin Cc: Alexander Potapenko Cc: Andrey Konovalov Reviewed-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino [ Add a new var instead of exposing kasan_arg_mode to be consistent with flags for other command line arguments. ] Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 9 +++++ lib/test_kasan.c | 2 +- mm/kasan/hw_tags.c | 60 ++++++++++++++++++++++++++++--- mm/kasan/kasan.h | 18 ++++++---- 4 files changed, 78 insertions(+), 11 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index ddf4239a5890..6f6ab3ed7b79 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -161,6 +161,15 @@ particular KASAN features. - ``kasan=off`` or ``=on`` controls whether KASAN is enabled (default: ``on``). +- ``kasan.mode=sync`` or ``=async`` controls whether KASAN is configured in + synchronous or asynchronous mode of execution (default: ``sync``). + Synchronous mode: a bad access is detected immediately when a tag + check fault occurs. + Asynchronous mode: a bad access detection is delayed. When a tag check + fault occurs, the information is stored in hardware (in the TFSR_EL1 + register for arm64). The kernel periodically checks the hardware and + only reports tag faults during these checks. + - ``kasan.stacktrace=off`` or ``=on`` disables or enables alloc and free stack traces collection (default: ``on``). diff --git a/lib/test_kasan.c b/lib/test_kasan.c index e5647d147b35..479c31a5dc21 100644 --- a/lib/test_kasan.c +++ b/lib/test_kasan.c @@ -97,7 +97,7 @@ static void kasan_test_exit(struct kunit *test) READ_ONCE(fail_data.report_found)); \ if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) { \ if (READ_ONCE(fail_data.report_found)) \ - kasan_enable_tagging(); \ + kasan_enable_tagging_sync(); \ migrate_enable(); \ } \ } while (0) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 2aad21fda156..1df4ce803861 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -25,6 +25,12 @@ enum kasan_arg { KASAN_ARG_ON, }; +enum kasan_arg_mode { + KASAN_ARG_MODE_DEFAULT, + KASAN_ARG_MODE_SYNC, + KASAN_ARG_MODE_ASYNC, +}; + enum kasan_arg_stacktrace { KASAN_ARG_STACKTRACE_DEFAULT, KASAN_ARG_STACKTRACE_OFF, @@ -38,6 +44,7 @@ enum kasan_arg_fault { }; static enum kasan_arg kasan_arg __ro_after_init; +static enum kasan_arg_mode kasan_arg_mode __ro_after_init; static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init; static enum kasan_arg_fault kasan_arg_fault __ro_after_init; @@ -45,6 +52,10 @@ static enum kasan_arg_fault kasan_arg_fault __ro_after_init; DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled); EXPORT_SYMBOL(kasan_flag_enabled); +/* Whether the asynchronous mode is enabled. */ +bool kasan_flag_async __ro_after_init; +EXPORT_SYMBOL_GPL(kasan_flag_async); + /* Whether to collect alloc/free stack traces. */ DEFINE_STATIC_KEY_FALSE(kasan_flag_stacktrace); @@ -68,6 +79,23 @@ static int __init early_kasan_flag(char *arg) } early_param("kasan", early_kasan_flag); +/* kasan.mode=sync/async */ +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "sync")) + kasan_arg_mode = KASAN_ARG_MODE_SYNC; + else if (!strcmp(arg, "async")) + kasan_arg_mode = KASAN_ARG_MODE_ASYNC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.mode", early_kasan_mode); + /* kasan.stacktrace=off/on */ static int __init early_kasan_flag_stacktrace(char *arg) { @@ -115,7 +143,15 @@ void kasan_init_hw_tags_cpu(void) return; hw_init_tags(KASAN_TAG_MAX); - hw_enable_tagging(); + + /* + * Enable async mode only when explicitly requested through + * the command line. + */ + if (kasan_arg_mode == KASAN_ARG_MODE_ASYNC) + hw_enable_tagging_async(); + else + hw_enable_tagging_sync(); } /* kasan_init_hw_tags() is called once on boot CPU. */ @@ -132,6 +168,22 @@ void __init kasan_init_hw_tags(void) /* Enable KASAN. */ static_branch_enable(&kasan_flag_enabled); + switch (kasan_arg_mode) { + case KASAN_ARG_MODE_DEFAULT: + /* + * Default to sync mode. + * Do nothing, kasan_flag_async keeps its default value. + */ + break; + case KASAN_ARG_MODE_SYNC: + /* Do nothing, kasan_flag_async keeps its default value. */ + break; + case KASAN_ARG_MODE_ASYNC: + /* Async mode enabled. */ + kasan_flag_async = true; + break; + } + switch (kasan_arg_stacktrace) { case KASAN_ARG_STACKTRACE_DEFAULT: /* Default to enabling stack trace collection. */ @@ -194,10 +246,10 @@ void kasan_set_tagging_report_once(bool state) } EXPORT_SYMBOL_GPL(kasan_set_tagging_report_once); -void kasan_enable_tagging(void) +void kasan_enable_tagging_sync(void) { - hw_enable_tagging(); + hw_enable_tagging_sync(); } -EXPORT_SYMBOL_GPL(kasan_enable_tagging); +EXPORT_SYMBOL_GPL(kasan_enable_tagging_sync); #endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 3436c6bf7c0c..265ad35a04ad 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -21,6 +21,7 @@ static inline bool kasan_stack_collection_enabled(void) #endif extern bool kasan_flag_panic __ro_after_init; +extern bool kasan_flag_async __ro_after_init; #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) @@ -275,8 +276,11 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #ifdef CONFIG_KASAN_HW_TAGS -#ifndef arch_enable_tagging -#define arch_enable_tagging() +#ifndef arch_enable_tagging_sync +#define arch_enable_tagging_sync() +#endif +#ifndef arch_enable_tagging_async +#define arch_enable_tagging_async() #endif #ifndef arch_init_tags #define arch_init_tags(max_tag) @@ -294,7 +298,8 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #define arch_set_mem_tag_range(addr, size, tag) ((void *)(addr)) #endif -#define hw_enable_tagging() arch_enable_tagging() +#define hw_enable_tagging_sync() arch_enable_tagging_sync() +#define hw_enable_tagging_async() arch_enable_tagging_async() #define hw_init_tags(max_tag) arch_init_tags(max_tag) #define hw_set_tagging_report_once(state) arch_set_tagging_report_once(state) #define hw_get_random_tag() arch_get_random_tag() @@ -303,7 +308,8 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #else /* CONFIG_KASAN_HW_TAGS */ -#define hw_enable_tagging() +#define hw_enable_tagging_sync() +#define hw_enable_tagging_async() #define hw_set_tagging_report_once(state) #endif /* CONFIG_KASAN_HW_TAGS */ @@ -311,12 +317,12 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #if defined(CONFIG_KASAN_HW_TAGS) && IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) void kasan_set_tagging_report_once(bool state); -void kasan_enable_tagging(void); +void kasan_enable_tagging_sync(void); #else /* CONFIG_KASAN_HW_TAGS || CONFIG_KASAN_KUNIT_TEST */ static inline void kasan_set_tagging_report_once(bool state) { } -static inline void kasan_enable_tagging(void) { } +static inline void kasan_enable_tagging_sync(void) { } #endif /* CONFIG_KASAN_HW_TAGS || CONFIG_KASAN_KUNIT_TEST */ From patchwork Fri Mar 12 14:22:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E936C433DB for ; Fri, 12 Mar 2021 14:24:26 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7B9D64FB2 for ; Fri, 12 Mar 2021 14:24:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7B9D64FB2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=I6VJuglO4EZNShjl5VGz3cueLmOhkGYwDSIrKEOGL8s=; b=iQVwq5TrT4rs05FnM4x/oVmWS JGojNz882c3gTgSAdclvC0918Am281zYqH8xRn5LfXCJpX3JZj6IbKYUAx+7sayf17AV2UylvhUns sBrFiB0W5ANlxz3MEwQaV+EPC573k4LuEZSj1OkDnoAqHHCPBDpuFrqM+IW2GscSpv6zs01dDPyoJ FvZAuWp86k7xN9AeT56fFnx4fsmZ65yaM5UJAOVwsiS1qSaP9NeioxlcbXByiEipl5PSD6/tFRkFW oTeQbwRg9uNVlZAUcPpks2GS6NabiiPHmH4rdx0Rp2K2u2JyJw/EAuPeQ6vcRMAIueHZnys5zH09u 2ljXg+bgA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigt-00BclC-JJ; Fri, 12 Mar 2021 14:22:59 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigQ-00Bcat-P0 for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:33 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3A5E311FB; Fri, 12 Mar 2021 06:22:29 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 537C33F793; Fri, 12 Mar 2021 06:22:27 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 3/8] arm64: mte: Drop arch_enable_tagging() Date: Fri, 12 Mar 2021 14:22:05 +0000 Message-Id: <20210312142210.21326-4-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142231_117652_EA513BCD X-CRM114-Status: UNSURE ( 8.22 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org arch_enable_tagging() was left in memory.h after the introduction of async mode to not break the bysectability of the KASAN KUNIT tests. Remove the function now that KASAN has been fully converted. Cc: Catalin Marinas Cc: Will Deacon Acked-by: Catalin Marinas Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/memory.h | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 8fbc8dab044f..8b0beaedbe1f 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -245,7 +245,6 @@ static inline const void *__tag_set(const void *addr, u8 tag) #ifdef CONFIG_KASAN_HW_TAGS #define arch_enable_tagging_sync() mte_enable_kernel_sync() #define arch_enable_tagging_async() mte_enable_kernel_async() -#define arch_enable_tagging() arch_enable_tagging_sync() #define arch_set_tagging_report_once(state) mte_set_report_once(state) #define arch_init_tags(max_tag) mte_init_tags(max_tag) #define arch_get_random_tag() mte_get_random_tag() From patchwork Fri Mar 12 14:22:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD29CC433E0 for ; Fri, 12 Mar 2021 14:24:39 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7B75664FF9 for ; Fri, 12 Mar 2021 14:24:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B75664FF9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hY9vHes3odVbTHF9rXHqiQXGRLHvYLTa8DaFqXHx0qA=; b=guCLNics06cvMGoPgPRsugT6s fQ+LgDGxAA4EfXPhfmE34l2TiUP0IogmLbummzQsuSEBhcwtnVCt/BSTvvPvWj9XNUaAsJ8zoGqbE jymiHgCABmKGBnpVXJ7qfatFtDNC0GJOHKqwzGVZJPiMmNl8DmWyP3wkl2Qt8gniUwx++MK0JJ6r7 sWWlCPCdzmyuKCkY9cz0THU6hkl/3FMLLXXv++DEelyNdHKklroCKi/36WwbdrkVpTNmVTwo9B5Uc tB5dOPDUCYvv7MLenBbw7GGMt+Igx06FUryiYunO0xrI3MC9DbwiBE1osPTVIKtOT401TwQswVTrG 1X/+6CA8g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKih1-00Bcmm-33; Fri, 12 Mar 2021 14:23:08 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigS-00BceI-0p for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:34 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4FBBA1063; Fri, 12 Mar 2021 06:22:31 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6D9F13F793; Fri, 12 Mar 2021 06:22:29 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 4/8] kasan: Add report for async mode Date: Fri, 12 Mar 2021 14:22:06 +0000 Message-Id: <20210312142210.21326-5-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142232_436664_3ABD9097 X-CRM114-Status: GOOD ( 10.47 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org KASAN provides an asynchronous mode of execution. Add reporting functionality for this mode. Cc: Dmitry Vyukov Cc: Andrey Ryabinin Cc: Alexander Potapenko Cc: Andrey Konovalov Reviewed-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 6 ++++++ mm/kasan/kasan.h | 16 ++++++++++++++++ mm/kasan/report.c | 17 ++++++++++++++++- 3 files changed, 38 insertions(+), 1 deletion(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 44c147dae7e3..9f5faefd1744 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -377,6 +377,12 @@ static inline void *kasan_reset_tag(const void *addr) #endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS*/ +#ifdef CONFIG_KASAN_HW_TAGS + +void kasan_report_async(void); + +#endif /* CONFIG_KASAN_HW_TAGS */ + #ifdef CONFIG_KASAN_SW_TAGS void __init kasan_init_sw_tags(void); #else diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 265ad35a04ad..02957cec1a61 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -7,17 +7,33 @@ #include #ifdef CONFIG_KASAN_HW_TAGS + #include + DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace); +extern bool kasan_flag_async __ro_after_init; + static inline bool kasan_stack_collection_enabled(void) { return static_branch_unlikely(&kasan_flag_stacktrace); } + +static inline bool kasan_async_mode_enabled(void) +{ + return kasan_flag_async; +} #else + static inline bool kasan_stack_collection_enabled(void) { return true; } + +static inline bool kasan_async_mode_enabled(void) +{ + return false; +} + #endif extern bool kasan_flag_panic __ro_after_init; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 87b271206163..8b0843a2cdd7 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -87,7 +87,8 @@ static void start_report(unsigned long *flags) static void end_report(unsigned long *flags, unsigned long addr) { - trace_error_report_end(ERROR_DETECTOR_KASAN, addr); + if (!kasan_async_mode_enabled()) + trace_error_report_end(ERROR_DETECTOR_KASAN, addr); pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); spin_unlock_irqrestore(&report_lock, *flags); @@ -360,6 +361,20 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags, (unsigned long)object); } +#ifdef CONFIG_KASAN_HW_TAGS +void kasan_report_async(void) +{ + unsigned long flags; + + start_report(&flags); + pr_err("BUG: KASAN: invalid-access\n"); + pr_err("Asynchronous mode enabled: no access details available\n"); + pr_err("\n"); + dump_stack(); + end_report(&flags, 0); +} +#endif /* CONFIG_KASAN_HW_TAGS */ + static void __kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { From patchwork Fri Mar 12 14:22:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2132DC433DB for ; Fri, 12 Mar 2021 14:25:09 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC39664FB2 for ; Fri, 12 Mar 2021 14:25:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC39664FB2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/f+K6WrJzl7zki9r1XIOu7SMeSoUOMeWiGXcv/Hkuek=; b=CBy6AIHzQJexxZbeEl0GenMJs jthUb6SyORP9wT/q4y2zW0ipsx4bH+9aAicBgTecU3GEVDulAjh6ZVQ/s2D0AY6iZVeoCZAMsKA8G aaQ3gAUb0BXW62E79m/P59+v10HGRnC3FnBOyCBOOHqcs3rhQdUnfBiMsJ39i9oanXIWaYvPFS8od 6rXlnfNX3qOQJCdOtxKWUgysfIHnOr08H2SsZty9+jowOB4JNpKIX0/vh1hcv8eUpLs83EUGxfLhQ EczlY/4tvNewIEvPd/twGTJACWJK3K3eVwRUTdYUJlzUOKoQR51UKeJUyQpKDSNacy6rkpyK49OGi NftxSDOcg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKihB-00Bcp6-FA; Fri, 12 Mar 2021 14:23:18 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigU-00Bcfr-Tu for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:37 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 693151FB; Fri, 12 Mar 2021 06:22:33 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 85EB03F793; Fri, 12 Mar 2021 06:22:31 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 5/8] arm64: mte: Enable TCO in functions that can read beyond buffer limits Date: Fri, 12 Mar 2021 14:22:07 +0000 Message-Id: <20210312142210.21326-6-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142235_691179_40B6C3C0 X-CRM114-Status: GOOD ( 18.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org load_unaligned_zeropad() and __get/put_kernel_nofault() functions can read past some buffer limits which may include some MTE granule with a different tag. When MTE async mode is enabled, the load operation crosses the boundaries and the next granule has a different tag the PE sets the TFSR_EL1.TF1 bit as if an asynchronous tag fault is happened. Enable Tag Check Override (TCO) in these functions before the load and disable it afterwards to prevent this to happen. Note: The same condition can be hit in MTE sync mode but we deal with it through the exception handling. In the current implementation, mte_async_mode flag is set only at boot time but in future kasan might acquire some runtime features that that change the mode dynamically, hence we disable it when sync mode is selected for future proof. Cc: Catalin Marinas Cc: Will Deacon Reported-by: Branislav Rankov Tested-by: Branislav Rankov Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/mte.h | 15 +++++++++++++++ arch/arm64/include/asm/uaccess.h | 22 ++++++++++++++++++++++ arch/arm64/include/asm/word-at-a-time.h | 4 ++++ arch/arm64/kernel/mte.c | 22 ++++++++++++++++++++++ 4 files changed, 63 insertions(+) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 9b557a457f24..8603c6636a7d 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -90,5 +90,20 @@ static inline void mte_assign_mem_tag_range(void *addr, size_t size) #endif /* CONFIG_ARM64_MTE */ +#ifdef CONFIG_KASAN_HW_TAGS +/* Whether the MTE asynchronous mode is enabled. */ +DECLARE_STATIC_KEY_FALSE(mte_async_mode); + +static inline bool system_uses_mte_async_mode(void) +{ + return static_branch_unlikely(&mte_async_mode); +} +#else +static inline bool system_uses_mte_async_mode(void) +{ + return false; +} +#endif /* CONFIG_KASAN_HW_TAGS */ + #endif /* __ASSEMBLY__ */ #endif /* __ASM_MTE_H */ diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 0deb88467111..b5f08621fa29 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -188,6 +189,23 @@ static inline void __uaccess_enable_tco(void) ARM64_MTE, CONFIG_KASAN_HW_TAGS)); } +/* + * These functions disable tag checking only if in MTE async mode + * since the sync mode generates exceptions synchronously and the + * nofault or load_unaligned_zeropad can handle them. + */ +static inline void __uaccess_disable_tco_async(void) +{ + if (system_uses_mte_async_mode()) + __uaccess_disable_tco(); +} + +static inline void __uaccess_enable_tco_async(void) +{ + if (system_uses_mte_async_mode()) + __uaccess_enable_tco(); +} + static inline void uaccess_disable_privileged(void) { __uaccess_disable_tco(); @@ -307,8 +325,10 @@ do { \ do { \ int __gkn_err = 0; \ \ + __uaccess_enable_tco_async(); \ __raw_get_mem("ldr", *((type *)(dst)), \ (__force type *)(src), __gkn_err); \ + __uaccess_disable_tco_async(); \ if (unlikely(__gkn_err)) \ goto err_label; \ } while (0) @@ -380,8 +400,10 @@ do { \ do { \ int __pkn_err = 0; \ \ + __uaccess_enable_tco_async(); \ __raw_put_mem("str", *((type *)(src)), \ (__force type *)(dst), __pkn_err); \ + __uaccess_disable_tco_async(); \ if (unlikely(__pkn_err)) \ goto err_label; \ } while(0) diff --git a/arch/arm64/include/asm/word-at-a-time.h b/arch/arm64/include/asm/word-at-a-time.h index 3333950b5909..c62d9fa791aa 100644 --- a/arch/arm64/include/asm/word-at-a-time.h +++ b/arch/arm64/include/asm/word-at-a-time.h @@ -55,6 +55,8 @@ static inline unsigned long load_unaligned_zeropad(const void *addr) { unsigned long ret, offset; + __uaccess_enable_tco_async(); + /* Load word from unaligned pointer addr */ asm( "1: ldr %0, %3\n" @@ -76,6 +78,8 @@ static inline unsigned long load_unaligned_zeropad(const void *addr) : "=&r" (ret), "=&r" (offset) : "r" (addr), "Q" (*(unsigned long *)addr)); + __uaccess_disable_tco_async(); + return ret; } diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index fa755cf94e01..9362928ba0d5 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -26,6 +26,10 @@ u64 gcr_kernel_excl __ro_after_init; static bool report_fault_once = true; +/* Whether the MTE asynchronous mode is enabled. */ +DEFINE_STATIC_KEY_FALSE(mte_async_mode); +EXPORT_SYMBOL_GPL(mte_async_mode); + static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_swap) { pte_t old_pte = READ_ONCE(*ptep); @@ -118,12 +122,30 @@ static inline void __mte_enable_kernel(const char *mode, unsigned long tcf) void mte_enable_kernel_sync(void) { + /* + * Make sure we enter this function when no PE has set + * async mode previously. + */ + WARN_ONCE(system_uses_mte_async_mode(), + "MTE async mode enabled system wide!"); + __mte_enable_kernel("synchronous", SCTLR_ELx_TCF_SYNC); } void mte_enable_kernel_async(void) { __mte_enable_kernel("asynchronous", SCTLR_ELx_TCF_ASYNC); + + /* + * MTE async mode is set system wide by the first PE that + * executes this function. + * + * Note: If in future KASAN acquires a runtime switching + * mode in between sync and async, this strategy needs + * to be reviewed. + */ + if (!system_uses_mte_async_mode()) + static_branch_enable(&mte_async_mode); } void mte_set_report_once(bool state) From patchwork Fri Mar 12 14:22:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D97EC433E6 for ; Fri, 12 Mar 2021 14:25:10 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D106D64FC3 for ; Fri, 12 Mar 2021 14:25:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D106D64FC3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3RgYlAhn5HiVNPD591IOH22f8W3i71MLAhn3JRQUbWc=; b=Hpj71MpbMrw5FSHbau3DohYJM 1EqjGKzFWHeV2RBLG3ZSO5K+bl9WVQB7L7Clms/Erq87hEJ2EVoo3XsHmmHhisJLhZKY6mbIEEpIs EvitXtftyw+rR3IPFeU1TQD+IE50GEvFjrpR8SSe7BVK0HrKdjFt8XwMR003e+YUDBONrYOFT3cGL tmGKkdERx8DxwjpRbjgZ8qG+C0C6eDfU8M9dLRT/ndIQiHSq5V7RBKLhpoHdQgK6awG8HH5yAt9yA dniG3uaE1eUoGpRCMW2cAOyspcxS0dtBWnoYodCWhqgZLYco6sBrBIhqVawixzGqRZMqoDEqJXJoc YHNuAu8fw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKihL-00BctM-MN; Fri, 12 Mar 2021 14:23:27 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigW-00BceI-0C for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:38 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8537011B3; Fri, 12 Mar 2021 06:22:35 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9D5253F793; Fri, 12 Mar 2021 06:22:33 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 6/8] arm64: mte: Enable async tag check fault Date: Fri, 12 Mar 2021 14:22:08 +0000 Message-Id: <20210312142210.21326-7-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142236_558811_CE421813 X-CRM114-Status: GOOD ( 17.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MTE provides a mode that asynchronously updates the TFSR_EL1 register when a tag check exception is detected. To take advantage of this mode the kernel has to verify the status of the register at: 1. Context switching 2. Return to user/EL0 (Not required in entry from EL0 since the kernel did not run) 3. Kernel entry from EL1 4. Kernel exit to EL1 If the register is non-zero a trace is reported. Add the required features for EL1 detection and reporting. Note: ITFSB bit is set in the SCTLR_EL1 register hence it guaranties that the indirect writes to TFSR_EL1 are synchronized at exception entry to EL1. On the context switch path the synchronization is guarantied by the dsb() in __switch_to(). The dsb(nsh) in mte_check_tfsr_exit() is provisional pending confirmation by the architects. Cc: Catalin Marinas Cc: Will Deacon Reviewed-by: Catalin Marinas Acked-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/mte.h | 29 +++++++++++++++++++++++++ arch/arm64/kernel/entry-common.c | 6 ++++++ arch/arm64/kernel/mte.c | 36 ++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 8603c6636a7d..9a929620ca5d 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -98,11 +98,40 @@ static inline bool system_uses_mte_async_mode(void) { return static_branch_unlikely(&mte_async_mode); } + +void mte_check_tfsr_el1(void); + +static inline void mte_check_tfsr_entry(void) +{ + mte_check_tfsr_el1(); +} + +static inline void mte_check_tfsr_exit(void) +{ + /* + * The asynchronous faults are sync'ed automatically with + * TFSR_EL1 on kernel entry but for exit an explicit dsb() + * is required. + */ + dsb(nsh); + isb(); + + mte_check_tfsr_el1(); +} #else static inline bool system_uses_mte_async_mode(void) { return false; } +static inline void mte_check_tfsr_el1(void) +{ +} +static inline void mte_check_tfsr_entry(void) +{ +} +static inline void mte_check_tfsr_exit(void) +{ +} #endif /* CONFIG_KASAN_HW_TAGS */ #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 9d3588450473..a1ec351c36bd 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -37,6 +37,8 @@ static void noinstr enter_from_kernel_mode(struct pt_regs *regs) lockdep_hardirqs_off(CALLER_ADDR0); rcu_irq_enter_check_tick(); trace_hardirqs_off_finish(); + + mte_check_tfsr_entry(); } /* @@ -47,6 +49,8 @@ static void noinstr exit_to_kernel_mode(struct pt_regs *regs) { lockdep_assert_irqs_disabled(); + mte_check_tfsr_exit(); + if (interrupts_enabled(regs)) { if (regs->exit_rcu) { trace_hardirqs_on_prepare(); @@ -293,6 +297,8 @@ asmlinkage void noinstr enter_from_user_mode(void) asmlinkage void noinstr exit_to_user_mode(void) { + mte_check_tfsr_exit(); + trace_hardirqs_on_prepare(); lockdep_hardirqs_on_prepare(CALLER_ADDR0); user_enter_irqoff(); diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 9362928ba0d5..ef6664979533 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -158,6 +158,29 @@ bool mte_report_once(void) return READ_ONCE(report_fault_once); } +#ifdef CONFIG_KASAN_HW_TAGS +void mte_check_tfsr_el1(void) +{ + u64 tfsr_el1; + + if (!system_supports_mte()) + return; + + tfsr_el1 = read_sysreg_s(SYS_TFSR_EL1); + + if (unlikely(tfsr_el1 & SYS_TFSR_EL1_TF1)) { + /* + * Note: isb() is not required after this direct write + * because there is no indirect read subsequent to it + * (per ARM DDI 0487F.c table D13-1). + */ + write_sysreg_s(0, SYS_TFSR_EL1); + + kasan_report_async(); + } +} +#endif + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ @@ -223,6 +246,19 @@ void mte_thread_switch(struct task_struct *next) /* avoid expensive SCTLR_EL1 accesses if no change */ if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); + else + isb(); + + /* + * Check if an async tag exception occurred at EL1. + * + * Note: On the context switch path we rely on the dsb() present + * in __switch_to() to guarantee that the indirect writes to TFSR_EL1 + * are synchronized before this point. + * isb() above is required for the same reason. + * + */ + mte_check_tfsr_el1(); } void mte_suspend_exit(void) From patchwork Fri Mar 12 14:22:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93CAAC433E0 for ; Fri, 12 Mar 2021 14:25:17 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 26AE764FD6 for ; Fri, 12 Mar 2021 14:25:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26AE764FD6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OV+GySxUYJjofmJZ5/BMEMYKpLbkryzN/KHMzcUggvw=; b=XgXx0UykOpU9Be+zo0P5Y8DCK FjMUkCbws8Gaxx5nOWo2KgRBOc2fG0eAfbgU0VmrsCuYDAGoemF4EbP2vFE4QEOVEq/nMZXRp0Tmg PbYwSJN6gEDZg9D7JOk7fTnpuoEjHA3//k9DmhgSBMkjYCuh6nEPbCF8GououArBs5nkBWvZe10d8 Xm3Ex+j1oMnQQHjJoYoMZ255ZESqpsAU3+FvPrpEhOEHCUQl3vkO8DqSowPf7V0ZJrM7bM6qn1phA cgL4ov5HihEzQCXkj6YCxi80vdLfXa/nCNwyl+Y5saKWGD3AXiPCeeXeVSmsfjAAu9Vq3I+lapPNc fFC66cCKg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKihV-00Bcws-OI; Fri, 12 Mar 2021 14:23:37 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKigZ-00BceI-CU for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:41 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D8C51063; Fri, 12 Mar 2021 06:22:37 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B9D9B3F793; Fri, 12 Mar 2021 06:22:35 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 7/8] arm64: mte: Report async tag faults before suspend Date: Fri, 12 Mar 2021 14:22:09 +0000 Message-Id: <20210312142210.21326-8-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142239_814230_A1553DB6 X-CRM114-Status: GOOD ( 14.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When MTE async mode is enabled TFSR_EL1 contains the accumulative asynchronous tag check faults for EL1 and EL0. During the suspend/resume operations the firmware might perform some operations that could change the state of the register resulting in a spurious tag check fault report. Report asynchronous tag faults before suspend and clear the TFSR_EL1 register after resume to prevent this to happen. Cc: Catalin Marinas Cc: Will Deacon Cc: Lorenzo Pieralisi Reviewed-by: Catalin Marinas Reviewed-by: Lorenzo Pieralisi Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/mte.h | 4 ++++ arch/arm64/kernel/mte.c | 16 ++++++++++++++++ arch/arm64/kernel/suspend.c | 3 +++ 3 files changed, 23 insertions(+) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 9a929620ca5d..a38abc15186c 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -41,6 +41,7 @@ void mte_sync_tags(pte_t *ptep, pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void flush_mte_state(void); void mte_thread_switch(struct task_struct *next); +void mte_suspend_enter(void); void mte_suspend_exit(void); long set_mte_ctrl(struct task_struct *task, unsigned long arg); long get_mte_ctrl(struct task_struct *task); @@ -66,6 +67,9 @@ static inline void flush_mte_state(void) static inline void mte_thread_switch(struct task_struct *next) { } +static inline void mte_suspend_enter(void) +{ +} static inline void mte_suspend_exit(void) { } diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index ef6664979533..5d3205878236 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -261,6 +261,22 @@ void mte_thread_switch(struct task_struct *next) mte_check_tfsr_el1(); } +void mte_suspend_enter(void) +{ + if (!system_supports_mte()) + return; + + /* + * The barriers are required to guarantee that the indirect writes + * to TFSR_EL1 are synchronized before we report the state. + */ + dsb(nsh); + isb(); + + /* Report SYS_TFSR_EL1 before suspend entry */ + mte_check_tfsr_el1(); +} + void mte_suspend_exit(void) { if (!system_supports_mte()) diff --git a/arch/arm64/kernel/suspend.c b/arch/arm64/kernel/suspend.c index d7564891ffe1..6fdc8292b4f5 100644 --- a/arch/arm64/kernel/suspend.c +++ b/arch/arm64/kernel/suspend.c @@ -91,6 +91,9 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long)) unsigned long flags; struct sleep_stack_data state; + /* Report any MTE async fault before going to suspend */ + mte_suspend_enter(); + /* * From this point debug exceptions are disabled to prevent * updates to mdscr register (saved and restored along with From patchwork Fri Mar 12 14:22:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12134781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 933C3C433E0 for ; Fri, 12 Mar 2021 14:25:31 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D58964FAD for ; Fri, 12 Mar 2021 14:25:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D58964FAD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uOTqZsGIlQ4W1jTN8hBBJPBNbCJH3upFXq8zW1shYpY=; b=FvXkkgei5NZxejsY9tpVbRqjm BHoiXGuzmjLUznWPoKPAsmKOnCyDw6S366ifs5bIyd7iyJzawUhFyIX0ReDtb9H10IJVXwpZYJAOF VQ8WU8WsVenbKcPoNc6eB4BbKcAuXe5Oz1uwyrHGoQg2drvrQpgNhJR4Z8SHaM2XBfE5e9Y+b7vR4 79GhdWIo6ogcmskFRUYtjBC+nZT6xy7pkOuR8iLMxrZyQOWYctLoD+nTMo4Y98g7IREr88+MA/LnD bATO/LH5nu6SlYYHl2iu8qDOChHgCKvFZHimk9opulKAVfRo5sj1403qNt9WAnsS5QFGTyOL/I0zT fN6AAPaTw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKihj-00Bd1U-He; Fri, 12 Mar 2021 14:23:52 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKiga-00BchU-BV for linux-arm-kernel@lists.infradead.org; Fri, 12 Mar 2021 14:22:42 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B3A8C11B3; Fri, 12 Mar 2021 06:22:39 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D16D13F793; Fri, 12 Mar 2021 06:22:37 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Cc: Vincenzo Frascino , Andrew Morton , Catalin Marinas , Will Deacon , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Andrey Konovalov , Lorenzo Pieralisi Subject: [PATCH v15 8/8] kasan, arm64: tests supports for HW_TAGS async mode Date: Fri, 12 Mar 2021 14:22:10 +0000 Message-Id: <20210312142210.21326-9-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210312142210.21326-1-vincenzo.frascino@arm.com> References: <20210312142210.21326-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_142240_718853_3FD46FD9 X-CRM114-Status: GOOD ( 18.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This change adds KASAN-KUnit tests support for the async HW_TAGS mode. In async mode, tag fault aren't being generated synchronously when a bad access happens, but are instead explicitly checked for by the kernel. As each KASAN-KUnit test expect a fault to happen before the test is over, check for faults as a part of the test handler. Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/memory.h | 1 + lib/test_kasan.c | 17 +++++++++++------ mm/kasan/hw_tags.c | 6 ++++++ mm/kasan/kasan.h | 6 ++++++ mm/kasan/report.c | 5 +++++ 5 files changed, 29 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 8b0beaedbe1f..b943879c1c24 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -246,6 +246,7 @@ static inline const void *__tag_set(const void *addr, u8 tag) #define arch_enable_tagging_sync() mte_enable_kernel_sync() #define arch_enable_tagging_async() mte_enable_kernel_async() #define arch_set_tagging_report_once(state) mte_set_report_once(state) +#define arch_force_async_tag_fault() mte_check_tfsr_exit() #define arch_init_tags(max_tag) mte_init_tags(max_tag) #define arch_get_random_tag() mte_get_random_tag() #define arch_get_mem_tag(addr) mte_get_mem_tag(addr) diff --git a/lib/test_kasan.c b/lib/test_kasan.c index 479c31a5dc21..785e724ce0d8 100644 --- a/lib/test_kasan.c +++ b/lib/test_kasan.c @@ -69,10 +69,10 @@ static void kasan_test_exit(struct kunit *test) * resource named "kasan_data". Do not use this name for KUnit resources * outside of KASAN tests. * - * For hardware tag-based KASAN, when a tag fault happens, tag checking is - * normally auto-disabled. When this happens, this test handler reenables - * tag checking. As tag checking can be only disabled or enabled per CPU, this - * handler disables migration (preemption). + * For hardware tag-based KASAN in sync mode, when a tag fault happens, tag + * checking is auto-disabled. When this happens, this test handler reenables + * tag checking. As tag checking can be only disabled or enabled per CPU, + * this handler disables migration (preemption). * * Since the compiler doesn't see that the expression can change the fail_data * fields, it can reorder or optimize away the accesses to those fields. @@ -80,7 +80,8 @@ static void kasan_test_exit(struct kunit *test) * expression to prevent that. */ #define KUNIT_EXPECT_KASAN_FAIL(test, expression) do { \ - if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) \ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS) && \ + !kasan_async_mode_enabled()) \ migrate_disable(); \ WRITE_ONCE(fail_data.report_expected, true); \ WRITE_ONCE(fail_data.report_found, false); \ @@ -92,10 +93,14 @@ static void kasan_test_exit(struct kunit *test) barrier(); \ expression; \ barrier(); \ + if (kasan_async_mode_enabled()) \ + kasan_force_async_fault(); \ + barrier(); \ KUNIT_EXPECT_EQ(test, \ READ_ONCE(fail_data.report_expected), \ READ_ONCE(fail_data.report_found)); \ - if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) { \ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS) && \ + !kasan_async_mode_enabled()) { \ if (READ_ONCE(fail_data.report_found)) \ kasan_enable_tagging_sync(); \ migrate_enable(); \ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 1df4ce803861..4004388b4e4b 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -252,4 +252,10 @@ void kasan_enable_tagging_sync(void) } EXPORT_SYMBOL_GPL(kasan_enable_tagging_sync); +void kasan_force_async_fault(void) +{ + hw_force_async_tag_fault(); +} +EXPORT_SYMBOL_GPL(kasan_force_async_fault); + #endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 02957cec1a61..c1581e8a9b8e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -304,6 +304,9 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #ifndef arch_set_tagging_report_once #define arch_set_tagging_report_once(state) #endif +#ifndef arch_force_async_tag_fault +#define arch_force_async_tag_fault() +#endif #ifndef arch_get_random_tag #define arch_get_random_tag() (0xFF) #endif @@ -318,6 +321,7 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #define hw_enable_tagging_async() arch_enable_tagging_async() #define hw_init_tags(max_tag) arch_init_tags(max_tag) #define hw_set_tagging_report_once(state) arch_set_tagging_report_once(state) +#define hw_force_async_tag_fault() arch_force_async_tag_fault() #define hw_get_random_tag() arch_get_random_tag() #define hw_get_mem_tag(addr) arch_get_mem_tag(addr) #define hw_set_mem_tag_range(addr, size, tag) arch_set_mem_tag_range((addr), (size), (tag)) @@ -334,11 +338,13 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) void kasan_set_tagging_report_once(bool state); void kasan_enable_tagging_sync(void); +void kasan_force_async_fault(void); #else /* CONFIG_KASAN_HW_TAGS || CONFIG_KASAN_KUNIT_TEST */ static inline void kasan_set_tagging_report_once(bool state) { } static inline void kasan_enable_tagging_sync(void) { } +static inline void kasan_force_async_fault(void) { } #endif /* CONFIG_KASAN_HW_TAGS || CONFIG_KASAN_KUNIT_TEST */ diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 8b0843a2cdd7..14bd51ea2348 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -366,6 +366,11 @@ void kasan_report_async(void) { unsigned long flags; +#if IS_ENABLED(CONFIG_KUNIT) + if (current->kunit_test) + kasan_update_kunit_status(current->kunit_test); +#endif /* IS_ENABLED(CONFIG_KUNIT) */ + start_report(&flags); pr_err("BUG: KASAN: invalid-access\n"); pr_err("Asynchronous mode enabled: no access details available\n");