From patchwork Wed Nov 4 23:18:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11883091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF586C388F9 for ; Thu, 5 Nov 2020 04:09:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 32CAC20732 for ; Thu, 5 Nov 2020 04:09:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="enUXn85G"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="dQd5whzc"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="RkNt0KlM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 32CAC20732 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SdmoOluRf/V9sC6bEgk6XMevVAc6y0w/5OCrAhTn4Zk=; b=enUXn85G38A4R+JvU2YKGZu+A zI4Khnal9oLcOx+4IUJbJ9Pcje1GOxmZwAJRENU6YWUEYxoBtLaM5A0Q6cDGOebfElJgPMG1l/9bH U0TWnaKva/SYzWyAXyOjUGTwapdoNNWCzs/TfyfzIKCZsoUw0VJE+eiNg3zAZcrIZstIoBtPg5+vQ FUlX0jALsxU3v7rQBFbVYRCGYe7zm1g8U9ckDwTKoBrHrA1Vm8yoMW1Cc6JK/BUg2fTKWTn9qt+mY t869PeTLwVAEnyUOUxo+epVacn2VttLBa3JAgNOv/4lMQ81nJEKuAWKLsAJrlDz5knRGMt4dr83La kV7hUtbUA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaWZ9-0003aV-Qe; Thu, 05 Nov 2020 04:08:03 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaW59-000145-Lr for linux-arm-kernel@merlin.infradead.org; Thu, 05 Nov 2020 03:37:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=MtscXzxM87FBM4U9jZNGOq87ComGejjIMcvAPHhplUo=; b=dQd5whzcdPdhp4AdvFmdGjxwKv Yr4QBaAsCZEnAlBd6p1kpHaA9YkTi85QEfWcJ010VnkJrqgBoqjb4oCYnwKpqIqMjlPR54x/RZBfu BgvZndaD/o5L7AfXQtc76u65y3LAkEhH6dVVBQ/Q7Bk9Kve+qhD9nvWsk8Bf1ys7EhpFrPgnfdWSp LXu61t+cpNInWbV1FP4e/DBjxkVuzo2iQ9b8AjmyA9vhdiN9Pk6p9aVedVMpkpsZgrtm+EOoogTIM eU0LZTxDLuto7pl22ffT85cCTaEx+/qHeJjPhtU3CHoOn5LdE4JnwWKbcdyMA3fgiWQU8c98ga4CL b5vsFP5A==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaS5f-00046k-Jp for linux-arm-kernel@lists.infradead.org; Wed, 04 Nov 2020 23:21:24 +0000 Received: by mail-wr1-x44a.google.com with SMTP id u1so47181wri.6 for ; Wed, 04 Nov 2020 15:21:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=MtscXzxM87FBM4U9jZNGOq87ComGejjIMcvAPHhplUo=; b=RkNt0KlMPVKqql5aFoLFHi6rGlddyN9w/V3DFG2q81cETVgCkQ8okQXMMbjNzMQ96I mwU7ZSEhTaXsiUWNNku5hGipjZDcCjWjUI1Ef1y1n+qOALlDfqSVTjUOgAPAXNzcEIjN GXIaoPCgj3a0abb+SjL/2vx6SooAFdwwNdqfwvXbIINa3zBfFHMTTtZVE2+cD2+AGR9j EMc4/uPUzHhKLoZy0ZJsaMlzC4K5IbkfQ24I3WSOdW/ZbJLlMW0ZGpgkqGt4Wc0s4P3i pzQT5WwpOU/al93W15vT/E+oWdj4PuJPwZJDOt5hYzh7XMLfaEc8CJ8Rb4SG0hFV95bI QhoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MtscXzxM87FBM4U9jZNGOq87ComGejjIMcvAPHhplUo=; b=GC1JdTPc+MGaZQOn5PNYRZgMbyn8eh+ZEWvWmAnmaGiWutvsuUOH8bG4GtmjVNfkmj sERLTdCb31Tgv+u2pZ5m3vA6IbU2zru7XFBqqgz5H5rgorQTltwPkj5HN47fDDRrIjyq 9L+7ghT8ghZ8FDgu693K8kYgbUlyN0lmpQYiqFLTDgBkxdaH4zK6bRLjiLhEadeUqJ15 V0zOQcpRR7VZ6YxzkK/huDyHhhoFW+Zg+Tz1ZaKQO0nrONOv4higCL3+l345ygsBEuCx 972sfquiG6Npla7guLEQKzXnCiHB+P0r4v2lQ5bVNN2M0p4sCQL+MOHX08cDhrizBSvq mUDQ== X-Gm-Message-State: AOAM530iYJ9ZWUmkYWAOz2wjZOopVvrmQoDc+T/VfWYqzH30szmQJ6kA 62O1R0FVYhsnF9EyT2EzeL4InCbYcfi6Z/ft X-Google-Smtp-Source: ABdhPJy5wD9hF4sxdUdmxkMLqPnuuSZnyFIVK724uWXL41j9mgrpkfI3oMqKXHk2SY6X+2USrkMRcnknjIfOPhfW X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:2803:: with SMTP id o3mr59728wmo.97.1604532018890; Wed, 04 Nov 2020 15:20:18 -0800 (PST) Date: Thu, 5 Nov 2020 00:18:45 +0100 In-Reply-To: Message-Id: <5e3c76cac4b161fe39e3fc8ace614400bc2fb5b1.1604531793.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH v8 30/43] arm64: kasan: Allow enabling in-kernel MTE From: Andrey Konovalov To: Catalin Marinas X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201104_232119_843911_2BC5A30B X-CRM114-Status: GOOD ( 19.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arm-kernel@lists.infradead.org, Marco Elver , Andrey Konovalov , Kevin Brodsky , Will Deacon , Branislav Rankov , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko , Evgenii Stepanov , Andrey Ryabinin , Andrew Morton , Vincenzo Frascino , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) feature and requires it to be enabled. MTE supports This patch adds a new mte_init_tags() helper, that enables MTE in Synchronous mode in EL1 and is intended to be called from KASAN runtime during initialization. The Tag Checking operation causes a synchronous data abort as a consequence of a tag check fault when MTE is configured in synchronous mode. As part of this change enable match-all tag for EL1 to allow the kernel to access user pages without faulting. This is required because the kernel does not have knowledge of the tags set by the user in a page. Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a similar way as TCF0 affects EL0. MTE that is built on top of the Top Byte Ignore (TBI) feature hence we enable it as part of this patch as well. Signed-off-by: Vincenzo Frascino Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov --- Change-Id: I4d67497268bb7f0c2fc5dcacefa1e273df4af71d --- arch/arm64/include/asm/mte-kasan.h | 6 ++++++ arch/arm64/kernel/mte.c | 7 +++++++ arch/arm64/mm/proc.S | 23 ++++++++++++++++++++--- 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 3a70fb1807fd..ae75feaea2d4 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -29,6 +29,8 @@ u8 mte_get_mem_tag(void *addr); u8 mte_get_random_tag(void); void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag); +void __init mte_init_tags(u64 max_tag); + #else /* CONFIG_ARM64_MTE */ static inline u8 mte_get_ptr_tag(void *ptr) @@ -49,6 +51,10 @@ static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) return addr; } +static inline void mte_init_tags(u64 max_tag) +{ +} + #endif /* CONFIG_ARM64_MTE */ #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 06ba6c923ab7..fcfbefcc3174 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -121,6 +121,13 @@ void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) return ptr; } +void __init mte_init_tags(u64 max_tag) +{ + /* Enable MTE Sync Mode for EL1. */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + isb(); +} + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 23c326a06b2d..7c3304fb15d9 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -40,9 +40,15 @@ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA #ifdef CONFIG_KASAN_SW_TAGS -#define TCR_KASAN_FLAGS TCR_TBI1 +#define TCR_KASAN_SW_FLAGS TCR_TBI1 #else -#define TCR_KASAN_FLAGS 0 +#define TCR_KASAN_SW_FLAGS 0 +#endif + +#ifdef CONFIG_KASAN_HW_TAGS +#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1 +#else +#define TCR_KASAN_HW_FLAGS 0 #endif /* @@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x5, MAIR_EL1_SET #ifdef CONFIG_ARM64_MTE + mte_tcr .req x20 + + mov mte_tcr, #0 + /* * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported * (ID_AA64PFR1_EL1[11:8] > 1). @@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup) /* clear any pending tag check faults in TFSR*_EL1 */ msr_s SYS_TFSR_EL1, xzr msr_s SYS_TFSRE0_EL1, xzr + + /* set the TCR_EL1 bits */ + mov_q mte_tcr, TCR_KASAN_HW_FLAGS 1: #endif msr mair_el1, x5 @@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS + TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS +#ifdef CONFIG_ARM64_MTE + orr x10, x10, mte_tcr + .unreq mte_tcr +#endif tcr_clear_errata_bits x10, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52