From patchwork Mon Nov 23 20:07:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11926609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8141EC2D0E4 for ; Mon, 23 Nov 2020 20:21:34 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F08D7206E5 for ; Mon, 23 Nov 2020 20:21:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="R9OkzAJl"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Zjw7Bjb0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F08D7206E5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zjw41SdMmxTLI2CkApCYrj19CLxLjqA7FVnSXIiQU2o=; b=R9OkzAJl13hk/3VNvw4GuF7TN 4Mv4p4nYPHC9oBsyEQDmxFKgvkt1ArtzGjIxGu5/kV3ZTEA2ZH63DHT3dtuJ14/2cL5z7sau9jNyQ OetKkCdfC4AWjxCmWCrBOJh6yWp//A575w0xfaNp0xO38uuMpjMmMrWXmh5qo1zXmZoSdaPBsF43q Sf5e2gSPGxhIFihMht30m/pCLzGgrQy/oZSBIsMqB7SZ8m1wOCYDRf9aPiJ9gngoLP0Zd7yyt2aRH 8v705RoKqjH/AWpINy03RtIlEgcgY7oR65S+59HAWMqEP0QjQmGyHgLYTDS42buvLhy/jjWmmVbce A4lEby0Fg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1khIKL-0004a9-5w; Mon, 23 Nov 2020 20:20:45 +0000 Received: from mail-qk1-f201.google.com ([209.85.222.201]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1khI9b-0008KG-Lt for linux-arm-kernel@lists.infradead.org; Mon, 23 Nov 2020 20:09:54 +0000 Received: by mail-qk1-f201.google.com with SMTP id z129so2825616qkb.13 for ; Mon, 23 Nov 2020 12:09:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=rVMBHBVJQeUCZLewjiSRiYOL9IjQshxMwG3NdWet2ZQ=; b=Zjw7Bjb0EieYeLR3i+7toAO5bnDeRvutOgtkBZRShwjuTUuJ870vZm1vdPcskK+Khe 6Ci+kghVW52zYO64/WdaIvWrFqE2VyquVaX7KJMlSEy7/lclemhxncoZVKlgFfyG2GeE vHa1pt7MxhnAdjscAzzFH3RLfwFSBLq3pcjV2DaqXDxSrwlrPdubBqHZ5Pzku9wK+Jig SjNxp+jRVJ7b+lfMlnBOpt5InCabQZWAiqEby8RK6xQsPoGexbw6FRS8e/YMNY+w+m/C MDs+Mq9NLbv04CN2K+z5RM63cwRXmM7RQ0jRYNVctQQcCIzWXa8VsSIPmeyW1PJkJaHE 3g7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rVMBHBVJQeUCZLewjiSRiYOL9IjQshxMwG3NdWet2ZQ=; b=joFmSoXXcxxGZd32Q6lFvdh3vJjcVOKmN4QL0lV+saDaW/AwzWyxCzvJJLLqP4SdSj GVKi4XM9sBOcJNK149j15SaJI8GJMjINq5yrAdrEyWAcgemfknhaaHxoF58d6fYcFp+k m9xxENH/4LCqbO1I+Ui8A1BlheWEqwKqvEXxyFzvJrDuDTWSTUvxFkaG09UNITctV2ce jOupH0Ge6xJ72yPZ6REkQASw1bAkAM1z0j8ETPkAA+dHmJlItiNo2EdCB0VKhcTd0bqH C3rtbwhy4vttURLGZYGR+6IPke30KkUqLHfT5K4HjWDnrx65rrCeETLNAOzrA4i7jRjA 2rIg== X-Gm-Message-State: AOAM531xcUlfEBjc0KyWbW8P+1P5oyoz1+xbWdo/5V4fFrRQRGsPXTQ2 uh4SjQKBq4XgADZCAzBOha4hbEN3czBb1SFq X-Google-Smtp-Source: ABdhPJzHMl1mbdwup/jzKm/e31Xz7AdnHxGJhgVnDJJTQN6EFH67dsLT2eiWkAdM3Q4HjJUvj4UmozVfjlzKoJly X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:40d1:: with SMTP id x17mr1213864qvp.21.1606162172985; Mon, 23 Nov 2020 12:09:32 -0800 (PST) Date: Mon, 23 Nov 2020 21:07:52 +0100 In-Reply-To: Message-Id: <7352b0a0899af65c2785416c8ca6bf3845b66fa1.1606161801.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.454.gaff20da3a2-goog Subject: [PATCH mm v11 28/42] arm64: kasan: Allow enabling in-kernel MTE From: Andrey Konovalov To: Andrew Morton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201123_150939_996367_2BACD478 X-CRM114-Status: GOOD ( 19.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-kernel@lists.infradead.org, Marco Elver , Catalin Marinas , Kevin Brodsky , Will Deacon , Branislav Rankov , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Vincenzo Frascino Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) feature and requires it to be enabled. MTE supports This patch adds a new mte_enable_kernel() helper, that enables MTE in Synchronous mode in EL1 and is intended to be called from KASAN runtime during initialization. The Tag Checking operation causes a synchronous data abort as a consequence of a tag check fault when MTE is configured in synchronous mode. As part of this change enable match-all tag for EL1 to allow the kernel to access user pages without faulting. This is required because the kernel does not have knowledge of the tags set by the user in a page. Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a similar way as TCF0 affects EL0. MTE that is built on top of the Top Byte Ignore (TBI) feature hence we enable it as part of this patch as well. Signed-off-by: Vincenzo Frascino Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas --- Change-Id: I4d67497268bb7f0c2fc5dcacefa1e273df4af71d --- arch/arm64/include/asm/mte-kasan.h | 6 ++++++ arch/arm64/kernel/mte.c | 7 +++++++ arch/arm64/mm/proc.S | 23 ++++++++++++++++++++--- 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 3a70fb1807fd..71ff6c6786ac 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -29,6 +29,8 @@ u8 mte_get_mem_tag(void *addr); u8 mte_get_random_tag(void); void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag); +void mte_enable_kernel(void); + #else /* CONFIG_ARM64_MTE */ static inline u8 mte_get_ptr_tag(void *ptr) @@ -49,6 +51,10 @@ static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) return addr; } +static inline void mte_enable_kernel(void) +{ +} + #endif /* CONFIG_ARM64_MTE */ #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 86d554ce98b6..7899e165f30a 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -129,6 +129,13 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) return ptr; } +void mte_enable_kernel(void) +{ + /* Enable MTE Sync Mode for EL1. */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + isb(); +} + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 0eaf16b0442a..0d85e6df42bc 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -40,9 +40,15 @@ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA #ifdef CONFIG_KASAN_SW_TAGS -#define TCR_KASAN_FLAGS TCR_TBI1 +#define TCR_KASAN_SW_FLAGS TCR_TBI1 #else -#define TCR_KASAN_FLAGS 0 +#define TCR_KASAN_SW_FLAGS 0 +#endif + +#ifdef CONFIG_KASAN_HW_TAGS +#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1 +#else +#define TCR_KASAN_HW_FLAGS 0 #endif /* @@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x5, MAIR_EL1_SET #ifdef CONFIG_ARM64_MTE + mte_tcr .req x20 + + mov mte_tcr, #0 + /* * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported * (ID_AA64PFR1_EL1[11:8] > 1). @@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup) /* clear any pending tag check faults in TFSR*_EL1 */ msr_s SYS_TFSR_EL1, xzr msr_s SYS_TFSRE0_EL1, xzr + + /* set the TCR_EL1 bits */ + mov_q mte_tcr, TCR_KASAN_HW_FLAGS 1: #endif msr mair_el1, x5 @@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS + TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS +#ifdef CONFIG_ARM64_MTE + orr x10, x10, mte_tcr + .unreq mte_tcr +#endif tcr_clear_errata_bits x10, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52