From patchwork Thu Oct 29 19:25:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11867185 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C4CDD6A2 for ; Thu, 29 Oct 2020 19:26:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70B6020825 for ; Thu, 29 Oct 2020 19:26:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="aMgPSNzn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70B6020825 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD7A86B007B; Thu, 29 Oct 2020 15:26:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C0F1A6B007D; Thu, 29 Oct 2020 15:26:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8E816B007E; Thu, 29 Oct 2020 15:26:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id 6D46D6B007B for ; Thu, 29 Oct 2020 15:26:24 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 043AE181AEF15 for ; Thu, 29 Oct 2020 19:26:24 +0000 (UTC) X-FDA: 77425944288.04.step50_5a158392728f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id DC766800B4E3 for ; Thu, 29 Oct 2020 19:26:23 +0000 (UTC) X-Spam-Summary: 1,0,0,996ef5b189634519,d41d8cd98f00b204,3xrebxwokcpocpftg0mpxniqqing.eqonkpwz-oomxcem.qti@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2393:2559:2562:2895:2899:2901:3138:3139:3140:3141:3142:3152:3355:3865:3866:3867:3868:3870:3871:3872:3874:4117:4250:4321:4605:5007:6117:6119:6261:6653:6742:7576:7903:9969:10004:10400:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12679:12895:12986:13138:13141:13161:13180:13229:13230:13231:13868:14181:14394:14659:14721:21080:21365:21444:21451:21627:21795:30051:30054:30070,0,RBL:209.85.221.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygop8ji3kbt55g6ocie5nwkkfsbock3n79ejjthyj3jrggozru9dy9umzirkb.4moduoupyd5ioqfw93janfza3jbbkkms4pg8n7ydezeee7hu4e31psadpwa4pjz.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Ne tcheck:n X-HE-Tag: step50_5a158392728f X-Filterd-Recvd-Size: 6821 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Thu, 29 Oct 2020 19:26:23 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id 33so1659964wrf.22 for ; Thu, 29 Oct 2020 12:26:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=VsN8c2WI1TMox3v/PYtKI3bTw78pIVnf1hcHZhXvdoM=; b=aMgPSNznb8KDmgeX4zCY3JQ6iMPVq+V0tG/nxtgplXyyzTIfPIxT3ZW9nTi/4RLXwr nlmWaYbkcVbt+27PlF1jPwDD9n16AYTVAS0GJ0hQWgWmMb5IIjk8JCtqvB3DXiTdH7da L9mPsHL0zvpCzgCy33zMT/aEqaCDMwOFNHtb7mt9JU6HNZBMEBG9xjm1ra8ivKqwEOEt iZ+h9wSj/S8BKGv37kwJ/GxB5WJnMYfFVPtz+Yi0IZd+Cd5VQTsh9qJpGuRRH+lmo1F3 Qx5Wh4setkAbsxAJ0N2Ex8YLzrBuM7mXvFM2mJYma9H3ZtR3/Xz8zRJswxYI6a/G2KbD RktA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VsN8c2WI1TMox3v/PYtKI3bTw78pIVnf1hcHZhXvdoM=; b=dEF3zw9HbjLdEgrUuQl0OnkMlUTYjjyLemPZHvKZB4S1uK6FjCg8XGWZ0wwxookAqp xIEawDfAOeiNaekNQ/obwO+QV/bPmyovTZGWXKy6PTVLLoM1/LJceakZ9G/5/H82z6kJ mFK8UDXV3y9wXh7wk3A5gV8AKBuoFajI6h7P7QGKaj3YQYhGxb8Ihejo4j+t4cAH+sRX wo5/lE1G4ukBNXjGtTvatIA/fSIRr27r+yc3NCE0l+Yb5bbK0prBK41W/WtKl4KZUg4r dODZbVBnq7PJRdo4eK/4Nn+krhjcXN+VC1baWSpaAW/7+2mOf0u8k2ke1un2FweARfmh F80w== X-Gm-Message-State: AOAM530g4yp6HxQFgBrDB4X0qKTaID1hC4awWc47pRVml7DoAffC9Epi u9pw9KWNckcAjSybJp3XKHtD+0gQuXZRd3qO X-Google-Smtp-Source: ABdhPJxXtIAvCMcgFOm+6WtylaUymV//WX1xOOvyLrlVkJpCK5lsLsad8dWrc4o+phSqbHAU2GsI4kMlj0LZq0yu X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a7b:c957:: with SMTP id i23mr405696wml.155.1603999581892; Thu, 29 Oct 2020 12:26:21 -0700 (PDT) Date: Thu, 29 Oct 2020 20:25:27 +0100 In-Reply-To: Message-Id: <9b5881af53f0206f423117d400167c96e8584024.1603999489.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH v6 06/40] arm64: kasan: Enable in-kernel MTE From: Andrey Konovalov To: Catalin Marinas , Will Deacon Cc: Vincenzo Frascino , kasan-dev@googlegroups.com, Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Elena Petrova , Branislav Rankov , Kevin Brodsky , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) feature and requires it to be enabled. The Tag Checking operation causes a synchronous data abort as a consequence of a tag check fault when MTE is configured in synchronous mode. Enable MTE in Synchronous mode in EL1 to provide a more immediate way of tag check failure detection in the kernel. As part of this change enable match-all tag for EL1 to allow the kernel to access user pages without faulting. This is required because the kernel does not have knowledge of the tags set by the user in a page. Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a similar way as TCF0 affects EL0. MTE that is built on top of the Top Byte Ignore (TBI) feature hence we enable it as part of this patch as well. Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas --- Change-Id: I4d67497268bb7f0c2fc5dcacefa1e273df4af71d --- arch/arm64/kernel/cpufeature.c | 7 +++++++ arch/arm64/mm/proc.S | 23 ++++++++++++++++++++--- 2 files changed, 27 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index dcc165b3fc04..c61f201042b2 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1704,6 +1704,13 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) cleared_zero_page = true; mte_clear_page_tags(lm_alias(empty_zero_page)); } + + /* Enable in-kernel MTE only if KASAN_HW_TAGS is enabled */ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) { + /* Enable MTE Sync Mode for EL1 */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + isb(); + } } #endif /* CONFIG_ARM64_MTE */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 23c326a06b2d..7c3304fb15d9 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -40,9 +40,15 @@ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA #ifdef CONFIG_KASAN_SW_TAGS -#define TCR_KASAN_FLAGS TCR_TBI1 +#define TCR_KASAN_SW_FLAGS TCR_TBI1 #else -#define TCR_KASAN_FLAGS 0 +#define TCR_KASAN_SW_FLAGS 0 +#endif + +#ifdef CONFIG_KASAN_HW_TAGS +#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1 +#else +#define TCR_KASAN_HW_FLAGS 0 #endif /* @@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x5, MAIR_EL1_SET #ifdef CONFIG_ARM64_MTE + mte_tcr .req x20 + + mov mte_tcr, #0 + /* * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported * (ID_AA64PFR1_EL1[11:8] > 1). @@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup) /* clear any pending tag check faults in TFSR*_EL1 */ msr_s SYS_TFSR_EL1, xzr msr_s SYS_TFSRE0_EL1, xzr + + /* set the TCR_EL1 bits */ + mov_q mte_tcr, TCR_KASAN_HW_FLAGS 1: #endif msr mair_el1, x5 @@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS + TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS +#ifdef CONFIG_ARM64_MTE + orr x10, x10, mte_tcr + .unreq mte_tcr +#endif tcr_clear_errata_bits x10, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52