From patchwork Tue Apr 21 14:25:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501559 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C37BA913 for ; Tue, 21 Apr 2020 14:26:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9AFD5206F4 for ; Tue, 21 Apr 2020 14:26:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9AFD5206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2FE8B8E0006; Tue, 21 Apr 2020 10:26:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2BA9C8E0003; Tue, 21 Apr 2020 10:26:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EB828E0006; Tue, 21 Apr 2020 10:26:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 07E9C8E0003 for ; Tue, 21 Apr 2020 10:26:15 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B79095DE1 for ; Tue, 21 Apr 2020 14:26:14 +0000 (UTC) X-FDA: 76732087068.02.night13_194f45e09602e X-Spam-Summary: 2,0,0,152b20a06de4b8cc,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1500:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3867:3870:4321:5007:6261:8634:10004:11026:11657:11658:11914:12043:12296:12297:12438:12555:12663:12986:13069:13161:13229:13255:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: night13_194f45e09602e X-Filterd-Recvd-Size: 2837 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:13 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E7CFB101E; Tue, 21 Apr 2020 07:26:12 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 66CE73F68F; Tue, 21 Apr 2020 07:26:11 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 01/23] arm64: alternative: Allow alternative_insn to always issue the first instruction Date: Tue, 21 Apr 2020 15:25:41 +0100 Message-Id: <20200421142603.3894-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are situations where we do not want to disable the whole block based on a config option, only the alternative part while keeping the first instruction. Improve the alternative_insn assembler macro to take a 'first_insn' argument, default 0, to preserve the current behaviour. Signed-off-by: Catalin Marinas Cc: Will Deacon Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/alternative.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index 5e5dc05d63a0..67d7cc608336 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -111,7 +111,11 @@ static inline void apply_alternatives_module(void *start, size_t length) { } .byte \alt_len .endm -.macro alternative_insn insn1, insn2, cap, enable = 1 +/* + * Disable the whole block if enable == 0, unless first_insn == 1 in which + * case insn1 will always be issued but without an alternative insn2. + */ +.macro alternative_insn insn1, insn2, cap, enable = 1, first_insn = 0 .if \enable 661: \insn1 662: .pushsection .altinstructions, "a" @@ -122,6 +126,8 @@ static inline void apply_alternatives_module(void *start, size_t length) { } 664: .popsection .org . - (664b-663b) + (662b-661b) .org . - (662b-661b) + (664b-663b) + .elseif \first_insn + \insn1 .endif .endm From patchwork Tue Apr 21 14:25:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501561 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 851F4913 for ; Tue, 21 Apr 2020 14:26:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 260C1206F4 for ; Tue, 21 Apr 2020 14:26:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 260C1206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 806C48E0007; Tue, 21 Apr 2020 10:26:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 794318E0003; Tue, 21 Apr 2020 10:26:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E4858E0007; Tue, 21 Apr 2020 10:26:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id 47D058E0003 for ; Tue, 21 Apr 2020 10:26:16 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 04F82180AD81F for ; Tue, 21 Apr 2020 14:26:16 +0000 (UTC) X-FDA: 76732087152.13.anger65_198e8d8d25725 X-Spam-Summary: 2,0,0,763920191f7ad0ae,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:2901:3138:3139:3140:3141:3142:3865:3871:4050:4119:4250:4321:4605:4648:5007:6119:6261:7576:7875:8634:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12663:12679:12986:13138:13161:13180:13229:13231:13894:13972:14394:21080:21220:21230:21451:21627:21990:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: anger65_198e8d8d25725 X-Filterd-Recvd-Size: 8919 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:15 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F40D3C14; Tue, 21 Apr 2020 07:26:14 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 27F203F68F; Tue, 21 Apr 2020 07:26:13 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 02/23] arm64: mte: system register definitions Date: Tue, 21 Apr 2020 15:25:42 +0100 Message-Id: <20200421142603.3894-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add Memory Tagging Extension system register definitions together with the relevant bitfields. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v2: - Added SET_PSTATE_TCO() macro. arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/include/asm/sysreg.h | 54 ++++++++++++++++++++++++++++ arch/arm64/include/uapi/asm/ptrace.h | 1 + arch/arm64/kernel/ptrace.c | 2 +- 4 files changed, 57 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 51c1d9918999..8a1cbfd544d6 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -12,6 +12,7 @@ #include /* Hyp Configuration Register (HCR) bits */ +#define HCR_ATA (UL(1) << 56) #define HCR_FWB (UL(1) << 46) #define HCR_API (UL(1) << 41) #define HCR_APK (UL(1) << 40) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index c4ac0ac25a00..e823e93b7429 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -91,10 +91,12 @@ #define PSTATE_PAN pstate_field(0, 4) #define PSTATE_UAO pstate_field(0, 3) #define PSTATE_SSBS pstate_field(3, 1) +#define PSTATE_TCO pstate_field(3, 4) #define SET_PSTATE_PAN(x) __emit_inst(0xd500401f | PSTATE_PAN | ((!!x) << PSTATE_Imm_shift)) #define SET_PSTATE_UAO(x) __emit_inst(0xd500401f | PSTATE_UAO | ((!!x) << PSTATE_Imm_shift)) #define SET_PSTATE_SSBS(x) __emit_inst(0xd500401f | PSTATE_SSBS | ((!!x) << PSTATE_Imm_shift)) +#define SET_PSTATE_TCO(x) __emit_inst(0xd500401f | PSTATE_TCO | ((!!x) << PSTATE_Imm_shift)) #define __SYS_BARRIER_INSN(CRm, op2, Rt) \ __emit_inst(0xd5000000 | sys_insn(0, 3, 3, (CRm), (op2)) | ((Rt) & 0x1f)) @@ -174,6 +176,8 @@ #define SYS_SCTLR_EL1 sys_reg(3, 0, 1, 0, 0) #define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1) #define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2) +#define SYS_RGSR_EL1 sys_reg(3, 0, 1, 0, 5) +#define SYS_GCR_EL1 sys_reg(3, 0, 1, 0, 6) #define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0) @@ -211,6 +215,8 @@ #define SYS_ERXADDR_EL1 sys_reg(3, 0, 5, 4, 3) #define SYS_ERXMISC0_EL1 sys_reg(3, 0, 5, 5, 0) #define SYS_ERXMISC1_EL1 sys_reg(3, 0, 5, 5, 1) +#define SYS_TFSR_EL1 sys_reg(3, 0, 5, 6, 0) +#define SYS_TFSRE0_EL1 sys_reg(3, 0, 5, 6, 1) #define SYS_FAR_EL1 sys_reg(3, 0, 6, 0, 0) #define SYS_PAR_EL1 sys_reg(3, 0, 7, 4, 0) @@ -361,6 +367,7 @@ #define SYS_CCSIDR_EL1 sys_reg(3, 1, 0, 0, 0) #define SYS_CLIDR_EL1 sys_reg(3, 1, 0, 0, 1) +#define SYS_GMID_EL1 sys_reg(3, 1, 0, 0, 4) #define SYS_AIDR_EL1 sys_reg(3, 1, 0, 0, 7) #define SYS_CSSELR_EL1 sys_reg(3, 2, 0, 0, 0) @@ -453,6 +460,7 @@ #define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0) #define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3) #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0) +#define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0) #define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0) #define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1) @@ -509,6 +517,7 @@ #define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0) #define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1) #define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0) +#define SYS_TFSR_EL12 sys_reg(3, 5, 5, 6, 0) #define SYS_FAR_EL12 sys_reg(3, 5, 6, 0, 0) #define SYS_MAIR_EL12 sys_reg(3, 5, 10, 2, 0) #define SYS_AMAIR_EL12 sys_reg(3, 5, 10, 3, 0) @@ -524,6 +533,15 @@ /* Common SCTLR_ELx flags. */ #define SCTLR_ELx_DSSBS (BIT(44)) +#define SCTLR_ELx_ATA (BIT(43)) + +#define SCTLR_ELx_TCF_SHIFT 40 +#define SCTLR_ELx_TCF_NONE (UL(0x0) << SCTLR_ELx_TCF_SHIFT) +#define SCTLR_ELx_TCF_SYNC (UL(0x1) << SCTLR_ELx_TCF_SHIFT) +#define SCTLR_ELx_TCF_ASYNC (UL(0x2) << SCTLR_ELx_TCF_SHIFT) +#define SCTLR_ELx_TCF_MASK (UL(0x3) << SCTLR_ELx_TCF_SHIFT) + +#define SCTLR_ELx_ITFSB (BIT(37)) #define SCTLR_ELx_ENIA (BIT(31)) #define SCTLR_ELx_ENIB (BIT(30)) #define SCTLR_ELx_ENDA (BIT(27)) @@ -552,6 +570,14 @@ #endif /* SCTLR_EL1 specific flags. */ +#define SCTLR_EL1_ATA0 (BIT(42)) + +#define SCTLR_EL1_TCF0_SHIFT 38 +#define SCTLR_EL1_TCF0_NONE (UL(0x0) << SCTLR_EL1_TCF0_SHIFT) +#define SCTLR_EL1_TCF0_SYNC (UL(0x1) << SCTLR_EL1_TCF0_SHIFT) +#define SCTLR_EL1_TCF0_ASYNC (UL(0x2) << SCTLR_EL1_TCF0_SHIFT) +#define SCTLR_EL1_TCF0_MASK (UL(0x3) << SCTLR_EL1_TCF0_SHIFT) + #define SCTLR_EL1_UCI (BIT(26)) #define SCTLR_EL1_E0E (BIT(24)) #define SCTLR_EL1_SPAN (BIT(23)) @@ -586,6 +612,7 @@ #define MAIR_ATTR_DEVICE_GRE UL(0x0c) #define MAIR_ATTR_NORMAL_NC UL(0x44) #define MAIR_ATTR_NORMAL_WT UL(0xbb) +#define MAIR_ATTR_NORMAL_TAGGED UL(0xf0) #define MAIR_ATTR_NORMAL UL(0xff) #define MAIR_ATTR_MASK UL(0xff) @@ -660,11 +687,16 @@ /* id_aa64pfr1 */ #define ID_AA64PFR1_SSBS_SHIFT 4 +#define ID_AA64PFR1_MTE_SHIFT 8 #define ID_AA64PFR1_SSBS_PSTATE_NI 0 #define ID_AA64PFR1_SSBS_PSTATE_ONLY 1 #define ID_AA64PFR1_SSBS_PSTATE_INSNS 2 +#define ID_AA64PFR1_MTE_NI 0x0 +#define ID_AA64PFR1_MTE_EL0 0x1 +#define ID_AA64PFR1_MTE 0x2 + /* id_aa64zfr0 */ #define ID_AA64ZFR0_F64MM_SHIFT 56 #define ID_AA64ZFR0_F32MM_SHIFT 52 @@ -822,6 +854,28 @@ #define CPACR_EL1_ZEN_EL0EN (BIT(17)) /* enable EL0 access, if EL1EN set */ #define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN) +/* TCR EL1 Bit Definitions */ +#define SYS_TCR_EL1_TCMA1 (BIT(58)) +#define SYS_TCR_EL1_TCMA0 (BIT(57)) + +/* GCR_EL1 Definitions */ +#define SYS_GCR_EL1_RRND (BIT(16)) +#define SYS_GCR_EL1_EXCL_MASK 0xffffUL + +/* RGSR_EL1 Definitions */ +#define SYS_RGSR_EL1_TAG_MASK 0xfUL +#define SYS_RGSR_EL1_SEED_SHIFT 8 +#define SYS_RGSR_EL1_SEED_MASK 0xffffUL + +/* GMID_EL1 field definitions */ +#define SYS_GMID_EL1_BS_SHIFT 0 +#define SYS_GMID_EL1_BS_SIZE 4 + +/* TFSR{,E0}_EL1 bit definitions */ +#define SYS_TFSR_EL1_TF0_SHIFT 0 +#define SYS_TFSR_EL1_TF1_SHIFT 1 +#define SYS_TFSR_EL1_TF0 (UL(1) << SYS_TFSR_EL1_TF0_SHIFT) +#define SYS_TFSR_EL1_TF1 (UK(2) << SYS_TFSR_EL1_TF1_SHIFT) /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */ #define SYS_MPIDR_SAFE_VAL (BIT(31)) diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index d1bb5b69f1ce..1daf6dda8af0 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -50,6 +50,7 @@ #define PSR_PAN_BIT 0x00400000 #define PSR_UAO_BIT 0x00800000 #define PSR_DIT_BIT 0x01000000 +#define PSR_TCO_BIT 0x02000000 #define PSR_V_BIT 0x10000000 #define PSR_C_BIT 0x20000000 #define PSR_Z_BIT 0x40000000 diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index b3d3005d9515..077e352495eb 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -1873,7 +1873,7 @@ void syscall_trace_exit(struct pt_regs *regs) * We also reserve IL for the kernel; SS is handled dynamically. */ #define SPSR_EL1_AARCH64_RES0_BITS \ - (GENMASK_ULL(63, 32) | GENMASK_ULL(27, 25) | GENMASK_ULL(23, 22) | \ + (GENMASK_ULL(63, 32) | GENMASK_ULL(27, 26) | GENMASK_ULL(23, 22) | \ GENMASK_ULL(20, 13) | GENMASK_ULL(11, 10) | GENMASK_ULL(5, 5)) #define SPSR_EL1_AARCH32_RES0_BITS \ (GENMASK_ULL(63, 32) | GENMASK_ULL(22, 22) | GENMASK_ULL(20, 20)) From patchwork Tue Apr 21 14:25:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501565 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 05A0E1667 for ; Tue, 21 Apr 2020 14:26:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C42CB206D9 for ; Tue, 21 Apr 2020 14:26:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C42CB206D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7457F8E0008; Tue, 21 Apr 2020 10:26:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6CE228E0003; Tue, 21 Apr 2020 10:26:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C0748E0008; Tue, 21 Apr 2020 10:26:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0231.hostedemail.com [216.40.44.231]) by kanga.kvack.org (Postfix) with ESMTP id 350F18E0003 for ; Tue, 21 Apr 2020 10:26:18 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DA8C0181AEF1F for ; Tue, 21 Apr 2020 14:26:17 +0000 (UTC) X-FDA: 76732087194.19.pie14_19d55cd71f425 X-Spam-Summary: 2,0,0,7495324b9bde395a,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3874:4049:4119:4250:4321:4605:5007:6261:7576:7903:7904:8634:10004:11026:11473:11657:11658:11914:12043:12291:12297:12438:12555:12679:12986:13161:13229:13255:13894:13972:14093:14394:21080:21230:21451:21627:21795:21990:30051:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: pie14_19d55cd71f425 X-Filterd-Recvd-Size: 8305 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:17 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D5E4A101E; Tue, 21 Apr 2020 07:26:16 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3457F3F68F; Tue, 21 Apr 2020 07:26:15 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Suzuki K Poulose Subject: [PATCH v3 03/23] arm64: mte: CPU feature detection and initial sysreg configuration Date: Tue, 21 Apr 2020 15:25:43 +0100 Message-Id: <20200421142603.3894-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add the cpufeature and hwcap entries to detect the presence of MTE on the boot CPUs (primary and secondary). Any late secondary CPU not supporting the feature, if detected during boot, will be parked. In addition, add the minimum SCTLR_EL1 and HCR_EL2 bits for enabling MTE. Without subsequent setting of MAIR, these bits do not have an effect on tag checking. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon Cc: Suzuki K Poulose --- arch/arm64/include/asm/cpucaps.h | 4 +++- arch/arm64/include/asm/cpufeature.h | 6 ++++++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/asm/kvm_arm.h | 2 +- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 2 ++ arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++++++++++++++++ arch/arm64/kernel/cpuinfo.c | 2 ++ 8 files changed, 46 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 8eb5a088ae65..4731ebacff54 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -61,7 +61,9 @@ #define ARM64_HAS_AMU_EXTN 51 #define ARM64_HAS_ADDRESS_AUTH 52 #define ARM64_HAS_GENERIC_AUTH 53 +/* 54 reserved for ARM64_BTI */ +#define ARM64_MTE 55 -#define ARM64_NCAPS 54 +#define ARM64_NCAPS 56 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index afe08251ff95..afc315814563 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -674,6 +674,12 @@ static inline bool system_uses_irq_prio_masking(void) cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING); } +static inline bool system_supports_mte(void) +{ + return IS_ENABLED(CONFIG_ARM64_MTE) && + cpus_have_const_cap(ARM64_MTE); +} + static inline bool system_has_prio_mask_debugging(void) { return IS_ENABLED(CONFIG_ARM64_DEBUG_PRIORITY_MASKING) && diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h index 0f00265248b5..8b302c88cfeb 100644 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -94,6 +94,7 @@ #define KERNEL_HWCAP_BF16 __khwcap2_feature(BF16) #define KERNEL_HWCAP_DGH __khwcap2_feature(DGH) #define KERNEL_HWCAP_RNG __khwcap2_feature(RNG) +#define KERNEL_HWCAP_MTE __khwcap2_feature(MTE) /* * This yields a mask that user programs can use to figure out what diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 8a1cbfd544d6..6c3b2fc922bb 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -78,7 +78,7 @@ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) -#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK) +#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) /* TCR_EL2 Registers bits */ diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index e823e93b7429..86236ae6c4e7 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -604,6 +604,7 @@ SCTLR_EL1_SA0 | SCTLR_EL1_SED | SCTLR_ELx_I |\ SCTLR_EL1_DZE | SCTLR_EL1_UCT |\ SCTLR_EL1_NTWE | SCTLR_ELx_IESB | SCTLR_EL1_SPAN |\ + SCTLR_ELx_ITFSB| SCTLR_ELx_ATA | SCTLR_EL1_ATA0 |\ ENDIAN_SET_EL1 | SCTLR_EL1_UCI | SCTLR_EL1_RES1) /* MAIR_ELx memory attributes (used by Linux) */ diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h index 7752d93bb50f..73ac5aede18c 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -73,5 +73,7 @@ #define HWCAP2_BF16 (1 << 14) #define HWCAP2_DGH (1 << 15) #define HWCAP2_RNG (1 << 16) +/* bit 17 reserved for HWCAP2_BTI */ +#define HWCAP2_MTE (1 << 18) #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 9fac745aa7bb..512a8b24c5df 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -182,6 +182,8 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = { ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI), + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE), + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI), ARM64_FTR_END, }; @@ -1409,6 +1411,18 @@ static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry, } #endif +#ifdef CONFIG_ARM64_MTE +static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) +{ + /* all non-zero tags excluded by default */ + write_sysreg_s(SYS_GCR_EL1_RRND | SYS_GCR_EL1_EXCL_MASK, SYS_GCR_EL1); + write_sysreg_s(0, SYS_TFSR_EL1); + write_sysreg_s(0, SYS_TFSRE0_EL1); + + isb(); +} +#endif /* CONFIG_ARM64_MTE */ + /* Internal helper functions to match cpu capability type */ static bool cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap) @@ -1779,6 +1793,19 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .min_field_value = 1, }, #endif +#ifdef CONFIG_ARM64_MTE + { + .desc = "Memory Tagging Extension", + .capability = ARM64_MTE, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, + .sys_reg = SYS_ID_AA64PFR1_EL1, + .field_pos = ID_AA64PFR1_MTE_SHIFT, + .min_field_value = ID_AA64PFR1_MTE, + .sign = FTR_UNSIGNED, + .cpu_enable = cpu_enable_mte, + }, +#endif /* CONFIG_ARM64_MTE */ {}, }; @@ -1892,6 +1919,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { HWCAP_MULTI_CAP(ptr_auth_hwcap_addr_matches, CAP_HWCAP, KERNEL_HWCAP_PACA), HWCAP_MULTI_CAP(ptr_auth_hwcap_gen_matches, CAP_HWCAP, KERNEL_HWCAP_PACG), #endif +#ifdef CONFIG_ARM64_MTE + HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_MTE_SHIFT, FTR_UNSIGNED, ID_AA64PFR1_MTE, CAP_HWCAP, KERNEL_HWCAP_MTE), +#endif /* CONFIG_ARM64_MTE */ {}, }; diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index 86136075ae41..d14b29de2c73 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -92,6 +92,8 @@ static const char *const hwcap_str[] = { "bf16", "dgh", "rng", + "", /* reserved for BTI */ + "mte", NULL }; From patchwork Tue Apr 21 14:25:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501567 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AFE79913 for ; Tue, 21 Apr 2020 14:26:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 796C92070B for ; Tue, 21 Apr 2020 14:26:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 796C92070B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 486A78E0009; Tue, 21 Apr 2020 10:26:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 412258E0003; Tue, 21 Apr 2020 10:26:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D92E8E0009; Tue, 21 Apr 2020 10:26:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 11B538E0003 for ; Tue, 21 Apr 2020 10:26:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C66684DB6 for ; Tue, 21 Apr 2020 14:26:19 +0000 (UTC) X-FDA: 76732087278.22.star95_1a1d98b342e49 X-Spam-Summary: 2,0,0,e7a08a70d43ad374,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:2:41:355:379:541:800:960:973:981:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2553:2559:2562:2693:2895:2898:2901:2903:2924:2926:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4050:4120:4250:4321:4605:5007:6119:6261:7875:7903:7904:8634:8957:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12986:13161:13180:13229:13894:14394:21063:21080:21230:21451:21611:21627:21795:21966:21990:30051:30054:30055:30069:30070:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: star95_1a1d98b342e49 X-Filterd-Recvd-Size: 9187 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:19 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B320CC14; Tue, 21 Apr 2020 07:26:18 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 15C1C3F68F; Tue, 21 Apr 2020 07:26:16 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Suzuki K Poulose Subject: [PATCH v3 04/23] arm64: mte: Use Normal Tagged attributes for the linear map Date: Tue, 21 Apr 2020 15:25:44 +0100 Message-Id: <20200421142603.3894-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Once user space is given access to tagged memory, the kernel must be able to clear/save/restore tags visible to the user. This is done via the linear mapping, therefore map it as such. The new MT_NORMAL_TAGGED index for MAIR_EL1 is initially mapped as Normal memory and later changed to Normal Tagged via the cpufeature infrastructure. From a mismatched attribute aliases perspective, the Tagged memory is considered a permission and it won't lead to undefined behaviour. The empty_zero_page is cleared to ensure that the tags it contains are already zeroed. The actual tags-aware clear_page() implementation is part of a subsequent patch. Signed-off-by: Catalin Marinas Cc: Will Deacon Cc: Suzuki K Poulose --- Notes: v3: - Restrict the safe attribute change in pgattr_change_is_safe() only to Normal to/from Normal-Tagged (old version allow any other type as long as old or new was Normal(-Tagged)). arch/arm64/include/asm/memory.h | 1 + arch/arm64/include/asm/pgtable-prot.h | 2 ++ arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++++++++++++++ arch/arm64/mm/dump.c | 4 ++++ arch/arm64/mm/mmu.c | 22 ++++++++++++++++++-- arch/arm64/mm/proc.S | 8 +++++-- 6 files changed, 63 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index a1871bb32bb1..472c77a68225 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -136,6 +136,7 @@ #define MT_NORMAL_NC 3 #define MT_NORMAL 4 #define MT_NORMAL_WT 5 +#define MT_NORMAL_TAGGED 6 /* * Memory types for Stage-2 translation diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 1305e28225fc..9c924b09d5c8 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -39,6 +39,7 @@ extern bool arm64_use_ng_mappings; #define PROT_NORMAL_NC (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_NC)) #define PROT_NORMAL_WT (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_WT)) #define PROT_NORMAL (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL)) +#define PROT_NORMAL_TAGGED (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_TAGGED)) #define PROT_SECT_DEVICE_nGnRE (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_DEVICE_nGnRE)) #define PROT_SECT_NORMAL (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL)) @@ -48,6 +49,7 @@ extern bool arm64_use_ng_mappings; #define _HYP_PAGE_DEFAULT _PAGE_DEFAULT #define PAGE_KERNEL __pgprot(PROT_NORMAL) +#define PAGE_KERNEL_TAGGED __pgprot(PROT_NORMAL_TAGGED) #define PAGE_KERNEL_RO __pgprot((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY) #define PAGE_KERNEL_ROX __pgprot((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 512a8b24c5df..d2fe8ff72324 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1414,13 +1414,43 @@ static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry, #ifdef CONFIG_ARM64_MTE static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) { + u64 mair; + /* all non-zero tags excluded by default */ write_sysreg_s(SYS_GCR_EL1_RRND | SYS_GCR_EL1_EXCL_MASK, SYS_GCR_EL1); write_sysreg_s(0, SYS_TFSR_EL1); write_sysreg_s(0, SYS_TFSRE0_EL1); + /* + * Update the MT_NORMAL_TAGGED index in MAIR_EL1. Tag checking is + * disabled for the kernel, so there won't be any observable effect + * other than allowing the kernel to read and write tags. + */ + mair = read_sysreg_s(SYS_MAIR_EL1); + mair &= ~MAIR_ATTRIDX(MAIR_ATTR_MASK, MT_NORMAL_TAGGED); + mair |= MAIR_ATTRIDX(MAIR_ATTR_NORMAL_TAGGED, MT_NORMAL_TAGGED); + write_sysreg_s(mair, SYS_MAIR_EL1); + isb(); } + +static int __init system_enable_mte(void) +{ + if (!system_supports_mte()) + return 0; + + /* Ensure the TLB does not have stale MAIR attributes */ + flush_tlb_all(); + + /* + * Clear the zero page (again) so that tags are reset. This needs to + * be done via the linear map which has the Tagged attribute. + */ + clear_page(lm_alias(empty_zero_page)); + + return 0; +} +core_initcall(system_enable_mte); #endif /* CONFIG_ARM64_MTE */ /* Internal helper functions to match cpu capability type */ diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c index 860c00ec8bd3..416a2404ac83 100644 --- a/arch/arm64/mm/dump.c +++ b/arch/arm64/mm/dump.c @@ -165,6 +165,10 @@ static const struct prot_bits pte_bits[] = { .mask = PTE_ATTRINDX_MASK, .val = PTE_ATTRINDX(MT_NORMAL), .set = "MEM/NORMAL", + }, { + .mask = PTE_ATTRINDX_MASK, + .val = PTE_ATTRINDX(MT_NORMAL_TAGGED), + .set = "MEM/NORMAL-TAGGED", } }; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a374e4f51a62..37bb5b19bdf4 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -121,7 +121,7 @@ static bool pgattr_change_is_safe(u64 old, u64 new) * The following mapping attributes may be updated in live * kernel mappings without the need for break-before-make. */ - static const pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG; + pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG; /* creating or taking down mappings is always safe */ if (old == 0 || new == 0) @@ -135,6 +135,19 @@ static bool pgattr_change_is_safe(u64 old, u64 new) if (old & ~new & PTE_NG) return false; + if (system_supports_mte()) { + /* + * Changing the memory type between Normal and Normal-Tagged + * is safe since Tagged is considered a permission attribute + * from the mismatched attribute aliases perspective. + */ + if (((old & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL) || + (old & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL_TAGGED)) && + ((new & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL) || + (new & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL_TAGGED))) + mask |= PTE_ATTRINDX_MASK; + } + return ((old ^ new) & ~mask) == 0; } @@ -489,7 +502,12 @@ static void __init map_mem(pgd_t *pgdp) if (memblock_is_nomap(reg)) continue; - __map_memblock(pgdp, start, end, PAGE_KERNEL, flags); + /* + * The linear map must allow allocation tags reading/writing + * if MTE is present. Otherwise, it has the same attributes as + * PAGE_KERNEL. + */ + __map_memblock(pgdp, start, end, PAGE_KERNEL_TAGGED, flags); } /* diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 197a9ba2d5ea..48891095eaed 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -44,14 +44,18 @@ #define TCR_KASAN_FLAGS 0 #endif -/* Default MAIR_EL1 */ +/* + * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory and + * changed later to Normal Tagged if the system supports MTE. + */ #define MAIR_EL1_SET \ (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRE, MT_DEVICE_nGnRE) | \ MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ - MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT)) + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) #ifdef CONFIG_CPU_PM /** From patchwork Tue Apr 21 14:25:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501569 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E93D1575 for ; Tue, 21 Apr 2020 14:26:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6C8A92074B for ; Tue, 21 Apr 2020 14:26:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C8A92074B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 39AD08E000A; Tue, 21 Apr 2020 10:26:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 325CF8E0003; Tue, 21 Apr 2020 10:26:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19EF28E000A; Tue, 21 Apr 2020 10:26:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id E9DAF8E0003 for ; Tue, 21 Apr 2020 10:26:21 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A1CD9181AEF1F for ; Tue, 21 Apr 2020 14:26:21 +0000 (UTC) X-FDA: 76732087362.12.cake30_1a5dcd3580e3b X-Spam-Summary: 2,0,0,b334469e8cc46684,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2553:2559:2562:2693:2918:3138:3139:3140:3141:3142:3352:3865:3867:3871:3874:5007:6117:6119:6261:7875:7903:8603:8634:10004:11026:11232:11473:11657:11658:11914:12043:12297:12438:12555:13069:13141:13180:13229:13230:13255:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:21795:30051:30054:30070:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: cake30_1a5dcd3580e3b X-Filterd-Recvd-Size: 2893 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:21 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 874FA101E; Tue, 21 Apr 2020 07:26:20 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E8D423F68F; Tue, 21 Apr 2020 07:26:18 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 05/23] arm64: mte: Assembler macros and default architecture for .S files Date: Tue, 21 Apr 2020 15:25:45 +0100 Message-Id: <20200421142603.3894-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the multitag_transfer_size macro to the arm64 assembler.h, together with '.arch armv8.5-a+memtag' when CONFIG_ARM64_MTE is enabled. Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: This patch may change as ".arch armv8.5-a" will be required for other features like BTI. v2: - Separate .arch armv8.5-a from .arch_extension memtag. arch/arm64/include/asm/assembler.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0bff325117b4..e7338e129dfd 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -21,8 +21,14 @@ #include #include #include +#include #include +#ifdef CONFIG_ARM64_MTE + .arch armv8.5-a + .arch_extension memtag +#endif + .macro save_and_disable_daif, flags mrs \flags, daif msr daifset, #0xf @@ -736,4 +742,15 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU .Lyield_out_\@ : .endm +/* + * multitag_transfer_size - set \reg to the block size that is accessed by the + * LDGM/STGM instructions. + */ + .macro multitag_transfer_size, reg, tmp + mrs_s \reg, SYS_GMID_EL1 + ubfx \reg, \reg, #SYS_GMID_EL1_BS_SHIFT, #SYS_GMID_EL1_BS_SIZE + mov \tmp, #4 + lsl \reg, \tmp, \reg + .endm + #endif /* __ASM_ASSEMBLER_H */ From patchwork Tue Apr 21 14:25:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501571 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F4001575 for ; Tue, 21 Apr 2020 14:26:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9A55206B9 for ; Tue, 21 Apr 2020 14:26:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9A55206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D116B8E000B; Tue, 21 Apr 2020 10:26:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C99A98E0003; Tue, 21 Apr 2020 10:26:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3A4C8E000B; Tue, 21 Apr 2020 10:26:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 929E88E0003 for ; Tue, 21 Apr 2020 10:26:23 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5E2215DE1 for ; Tue, 21 Apr 2020 14:26:23 +0000 (UTC) X-FDA: 76732087446.11.meat50_1aa5054213b43 X-Spam-Summary: 2,0,0,d7ea99e0dcd0e223,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:4321:5007:6119:6261:7576:8634:9592:10004:11026:11473:11658:11914:12297:12555:12679:12986:13069:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:21795:30051:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: meat50_1aa5054213b43 X-Filterd-Recvd-Size: 2612 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:22 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 53DF3C14; Tue, 21 Apr 2020 07:26:22 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BBE6A3F68F; Tue, 21 Apr 2020 07:26:20 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 06/23] arm64: mte: Tags-aware clear_page() implementation Date: Tue, 21 Apr 2020 15:25:46 +0100 Message-Id: <20200421142603.3894-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be set to zero a page is cleared as they are visible to the user. Introduce an MTE-aware clear_page() which clears the tags in addition to data. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/lib/clear_page.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S index 073acbf02a7c..9f85a4cf9568 100644 --- a/arch/arm64/lib/clear_page.S +++ b/arch/arm64/lib/clear_page.S @@ -5,7 +5,9 @@ #include #include +#include #include +#include #include /* @@ -19,8 +21,9 @@ SYM_FUNC_START(clear_page) and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - -1: dc zva, x0 +1: +alternative_insn "dc zva, x0", "stzgm xzr, [x0]", \ + ARM64_MTE, IS_ENABLED(CONFIG_ARM64_MTE), 1 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b From patchwork Tue Apr 21 14:25:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501573 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D277B913 for ; Tue, 21 Apr 2020 14:26:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AAB45206F4 for ; Tue, 21 Apr 2020 14:26:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAB45206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7B8AF8E000C; Tue, 21 Apr 2020 10:26:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 76A6C8E0003; Tue, 21 Apr 2020 10:26:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E3CB8E000C; Tue, 21 Apr 2020 10:26:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 4291F8E0003 for ; Tue, 21 Apr 2020 10:26:25 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F10C682499A8 for ; Tue, 21 Apr 2020 14:26:24 +0000 (UTC) X-FDA: 76732087488.13.brick29_1ae2cfe9bb507 X-Spam-Summary: 2,0,0,3188803638e23887,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3867:3871:4250:4321:5007:6117:6119:6261:7576:8634:10004:11026:11473:11658:11914:12291:12297:12438:12555:12679:12986:13069:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:30054:30079,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: brick29_1ae2cfe9bb507 X-Filterd-Recvd-Size: 2655 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:24 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2F53731B; Tue, 21 Apr 2020 07:26:24 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 885653F68F; Tue, 21 Apr 2020 07:26:22 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 07/23] arm64: mte: Tags-aware copy_page() implementation Date: Tue, 21 Apr 2020 15:25:47 +0100 Message-Id: <20200421142603.3894-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be preserved across page copy (e.g. for copy-on-write). Introduce MTE-aware copy_page() which preserves the tags across page copy. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/lib/copy_page.S | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S index e7a793961408..c3234175efe0 100644 --- a/arch/arm64/lib/copy_page.S +++ b/arch/arm64/lib/copy_page.S @@ -25,6 +25,29 @@ alternative_if ARM64_HAS_NO_HW_PREFETCH prfm pldl1strm, [x1, #384] alternative_else_nop_endif +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 2f +alternative_else_nop_endif + /* + * Copy tags if MTE has been enabled. + */ + mov x2, x0 + mov x3, x1 + + multitag_transfer_size x7, x5 +1: + ldgm x4, [x3] + stgm x4, [x2] + + add x2, x2, x7 + add x3, x3, x7 + + tst x2, #(PAGE_SIZE - 1) + b.ne 1b +2: +#endif + ldp x2, x3, [x1] ldp x4, x5, [x1, #16] ldp x6, x7, [x1, #32] From patchwork Tue Apr 21 14:25:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501577 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6BB91575 for ; Tue, 21 Apr 2020 14:26:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6E497206D9 for ; Tue, 21 Apr 2020 14:26:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E497206D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 93F978E000D; Tue, 21 Apr 2020 10:26:27 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8CD658E0003; Tue, 21 Apr 2020 10:26:27 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6596B8E000D; Tue, 21 Apr 2020 10:26:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 4DCE08E0003 for ; Tue, 21 Apr 2020 10:26:27 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 01337181AEF07 for ; Tue, 21 Apr 2020 14:26:27 +0000 (UTC) X-FDA: 76732087614.26.bean29_1b2a9343cf85a X-Spam-Summary: 2,0,0,1b8027e719b13282,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:1801:1981:2194:2199:2393:2538:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:4321:4605:5007:6117:6119:6261:7875:7903:8634:10004:11026:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12683:12986:13894:14096:14181:14394:14721:21080:21230:21451:21627:21990:30003:30054:30067:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: bean29_1b2a9343cf85a X-Filterd-Recvd-Size: 5946 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:26 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E9115C14; Tue, 21 Apr 2020 07:26:25 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6383D3F68F; Tue, 21 Apr 2020 07:26:24 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 08/23] arm64: Tags-aware memcmp_pages() implementation Date: Tue, 21 Apr 2020 15:25:48 +0100 Message-Id: <20200421142603.3894-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When the Memory Tagging Extension is enabled, two pages are identical only if both their data and tags are identical. Make the generic memcmp_pages() a __weak function and add an arm64-specific implementation which takes care of the tags comparison. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/mte.h | 11 +++++++++ arch/arm64/lib/Makefile | 2 ++ arch/arm64/lib/mte.S | 46 ++++++++++++++++++++++++++++++++++++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/cmppages.c | 26 ++++++++++++++++++++ mm/util.c | 2 +- 6 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/mte.h create mode 100644 arch/arm64/lib/mte.S create mode 100644 arch/arm64/mm/cmppages.c diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h new file mode 100644 index 000000000000..64e814273659 --- /dev/null +++ b/arch/arm64/include/asm/mte.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_MTE_H +#define __ASM_MTE_H + +#ifndef __ASSEMBLY__ + +/* Memory Tagging API */ +int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_MTE_H */ diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index 2fc253466dbf..d31e1169d9b8 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -16,3 +16,5 @@ lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o obj-$(CONFIG_CRC32) += crc32.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o + +obj-$(CONFIG_ARM64_MTE) += mte.o diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S new file mode 100644 index 000000000000..bd51ea7e2fcb --- /dev/null +++ b/arch/arm64/lib/mte.S @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2019 ARM Ltd. + */ +#include + +#include + +/* + * Compare tags of two pages + * x0 - page1 address + * x1 - page2 address + * Returns: + * w0 - negative, zero or positive value if the tag in the first page is + * less than, equal to or greater than the tag in the second page + */ +SYM_FUNC_START(mte_memcmp_pages) + multitag_transfer_size x7, x5 +1: + ldgm x2, [x0] + ldgm x3, [x1] + + eor x4, x2, x3 + cbnz x4, 2f + + add x0, x0, x7 + add x1, x1, x7 + + tst x0, #(PAGE_SIZE - 1) + b.ne 1b + + mov w0, #0 + ret +2: + rbit x4, x4 + clz x4, x4 // count the least significant equal bits + and x4, x4, #~3 // round down to a multiple of 4 (bits per tag) + + lsr x2, x2, x4 // remove equal tags + lsr x3, x3, x4 + + lsl w2, w2, #28 // compare the differing tags + sub w0, w2, w3, lsl #28 + + ret +SYM_FUNC_END(mte_memcmp_pages) diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index d91030f0ffee..e93d696295d0 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_PTDUMP_CORE) += dump.o obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o +obj-$(CONFIG_ARM64_MTE) += cmppages.o KASAN_SANITIZE_physaddr.o += n obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/arm64/mm/cmppages.c b/arch/arm64/mm/cmppages.c new file mode 100644 index 000000000000..943c1877e014 --- /dev/null +++ b/arch/arm64/mm/cmppages.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2019 ARM Ltd. + */ + +#include +#include + +#include +#include + +int memcmp_pages(struct page *page1, struct page *page2) +{ + char *addr1, *addr2; + int ret; + + addr1 = page_address(page1); + addr2 = page_address(page2); + + ret = memcmp(addr1, addr2, PAGE_SIZE); + /* if page content identical, check the tags */ + if (ret == 0 && system_supports_mte()) + ret = mte_memcmp_pages(addr1, addr2); + + return ret; +} diff --git a/mm/util.c b/mm/util.c index 988d11e6c17c..662fb3da6d01 100644 --- a/mm/util.c +++ b/mm/util.c @@ -899,7 +899,7 @@ int get_cmdline(struct task_struct *task, char *buffer, int buflen) return res; } -int memcmp_pages(struct page *page1, struct page *page2) +int __weak memcmp_pages(struct page *page1, struct page *page2) { char *addr1, *addr2; int ret; From patchwork Tue Apr 21 14:25:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501579 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A3A4F913 for ; Tue, 21 Apr 2020 14:26:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B1F1206D9 for ; Tue, 21 Apr 2020 14:26:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B1F1206D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 39BC18E000E; Tue, 21 Apr 2020 10:26:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3269F8E0003; Tue, 21 Apr 2020 10:26:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C8788E000E; Tue, 21 Apr 2020 10:26:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0113.hostedemail.com [216.40.44.113]) by kanga.kvack.org (Postfix) with ESMTP id E97838E0003 for ; Tue, 21 Apr 2020 10:26:28 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A339182499A8 for ; Tue, 21 Apr 2020 14:26:28 +0000 (UTC) X-FDA: 76732087656.06.dirt61_1b6a05dd65400 X-Spam-Summary: 2,0,0,7f4515a737ffec79,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1542:1711:1730:1747:1777:1792:1981:2194:2199:2393:2559:2562:2693:2895:3138:3139:3140:3141:3142:3353:3865:3867:3868:4250:4321:5007:6119:6261:7576:7875:8634:9121:10004:11026:11232:11473:11658:11914:12043:12296:12297:12438:12555:12679:12986:13180:13229:13894:14096:14181:14394:14721:21080:21220:21230:21451:21627:30012:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: dirt61_1b6a05dd65400 X-Filterd-Recvd-Size: 3418 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:28 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CC20331B; Tue, 21 Apr 2020 07:26:27 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2E3BB3F68F; Tue, 21 Apr 2020 07:26:26 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Arnd Bergmann Subject: [PATCH v3 09/23] arm64: mte: Add specific SIGSEGV codes Date: Tue, 21 Apr 2020 15:25:49 +0100 Message-Id: <20200421142603.3894-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add MTE-specific SIGSEGV codes to siginfo.h and update the x86 BUILD_BUG_ON(NSIGSEGV != 7) compile check. Signed-off-by: Vincenzo Frascino [catalin.marinas@arm.com: renamed precise/imprecise to sync/async] [catalin.marinas@arm.com: dropped #ifdef __aarch64__, renumbered] Signed-off-by: Catalin Marinas Acked-by: "Eric W. Biederman" Cc: Arnd Bergmann Cc: Will Deacon --- Notes: v3: - Fixed the BUILD_BUG_ON(NSIGSEGV != 7) on x86 - Updated the commit log v2: - Dropped the #ifdef __aarch64__. - Renumbered the SEGV_MTE* values to avoid clash with ADI. arch/x86/kernel/signal_compat.c | 2 +- include/uapi/asm-generic/siginfo.h | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/signal_compat.c b/arch/x86/kernel/signal_compat.c index 9ccbf0576cd0..a7f3e12cfbdb 100644 --- a/arch/x86/kernel/signal_compat.c +++ b/arch/x86/kernel/signal_compat.c @@ -27,7 +27,7 @@ static inline void signal_compat_build_tests(void) */ BUILD_BUG_ON(NSIGILL != 11); BUILD_BUG_ON(NSIGFPE != 15); - BUILD_BUG_ON(NSIGSEGV != 7); + BUILD_BUG_ON(NSIGSEGV != 9); BUILD_BUG_ON(NSIGBUS != 5); BUILD_BUG_ON(NSIGTRAP != 5); BUILD_BUG_ON(NSIGCHLD != 6); diff --git a/include/uapi/asm-generic/siginfo.h b/include/uapi/asm-generic/siginfo.h index cb3d6c267181..7aacf9389010 100644 --- a/include/uapi/asm-generic/siginfo.h +++ b/include/uapi/asm-generic/siginfo.h @@ -229,7 +229,9 @@ typedef struct siginfo { #define SEGV_ACCADI 5 /* ADI not enabled for mapped object */ #define SEGV_ADIDERR 6 /* Disrupting MCD error */ #define SEGV_ADIPERR 7 /* Precise MCD exception */ -#define NSIGSEGV 7 +#define SEGV_MTEAERR 8 /* Asynchronous ARM MTE error */ +#define SEGV_MTESERR 9 /* Synchronous ARM MTE exception */ +#define NSIGSEGV 9 /* * SIGBUS si_codes From patchwork Tue Apr 21 14:25:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501581 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50B5A913 for ; Tue, 21 Apr 2020 14:26:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4898C206F4 for ; Tue, 21 Apr 2020 14:26:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4898C206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B01B8E000F; Tue, 21 Apr 2020 10:26:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 239D18E0003; Tue, 21 Apr 2020 10:26:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DB748E000F; Tue, 21 Apr 2020 10:26:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0059.hostedemail.com [216.40.44.59]) by kanga.kvack.org (Postfix) with ESMTP id E5B5D8E0003 for ; Tue, 21 Apr 2020 10:26:30 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A6ACA282D for ; Tue, 21 Apr 2020 14:26:30 +0000 (UTC) X-FDA: 76732087740.07.wood47_1bacf9e1a2043 X-Spam-Summary: 2,0,0,c978e761def71b15,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:1:41:355:379:541:800:960:973:981:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1605:1730:1747:1777:1792:2393:2538:2553:2559:2562:2637:2689:2693:2895:2899:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4605:5007:6117:6119:6261:7576:7875:7903:8603:8634:8660:9389:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12679:12986:13053:13138:13148:13153:13161:13180:13228:13229:13230:13231:13255:13868:13894:13972:14394:21080:21230:21433:21451:21611:21627:21740:21795:21987:21990:30003:30045:30051:30054:30067:30070:30083:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: wood47_1bacf9e1a2043 X-Filterd-Recvd-Size: 12755 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:30 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A0964C14; Tue, 21 Apr 2020 07:26:29 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0C6A53F68F; Tue, 21 Apr 2020 07:26:27 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 10/23] arm64: mte: Handle synchronous and asynchronous tag check faults Date: Tue, 21 Apr 2020 15:25:50 +0100 Message-Id: <20200421142603.3894-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino The Memory Tagging Extension has two modes of notifying a tag check fault at EL0, configurable through the SCTLR_EL1.TCF0 field: 1. Synchronous raising of a Data Abort exception with DFSC 17. 2. Asynchronous setting of a cumulative bit in TFSRE0_EL1. Add the exception handler for the synchronous exception and handling of the asynchronous TFSRE0_EL1.TF0 bit setting via a new TIF flag in do_notify_resume(). On a tag check failure in user-space, whether synchronous or asynchronous, a SIGSEGV will be raised on the faulting thread. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v3: - Asynchronous tag check faults during the uaccess routines in the kernel are ignored. - Fix check_mte_async_tcf calling site as it expects the first argument to be the thread flags. - Move the mte_thread_switch() definition and call to a later patch as this became empty with the removal of async uaccess checking. - Add dsb() and clearing of TFSRE0_EL1 in flush_mte_state(), in case execve() triggered a asynchronous tag check fault. - Clear TIF_MTE_ASYNC_FAULT in arch_dup_task_struct() so that the child does not inherit any pending tag fault in the parent. v2: - Clear PSTATE.TCO on exception entry (automatically set by the hardware). - On syscall entry, for asynchronous tag check faults from user space, generate the signal early via syscall restarting. - Before context switch, save any potential async tag check fault generated by the kernel to the TIF flag (this follows an architecture update where the uaccess routines use the TCF0 mode). - Moved the flush_mte_state() and mte_thread_switch() function to a new mte.c file. arch/arm64/include/asm/mte.h | 10 ++++++++ arch/arm64/include/asm/thread_info.h | 4 +++- arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/entry.S | 36 ++++++++++++++++++++++++++++ arch/arm64/kernel/mte.c | 21 ++++++++++++++++ arch/arm64/kernel/process.c | 5 ++++ arch/arm64/kernel/signal.c | 8 +++++++ arch/arm64/kernel/syscall.c | 10 ++++++++ arch/arm64/mm/fault.c | 9 ++++++- 9 files changed, 102 insertions(+), 2 deletions(-) create mode 100644 arch/arm64/kernel/mte.c diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 64e814273659..e9711ea51eb5 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -4,8 +4,18 @@ #ifndef __ASSEMBLY__ +#include + /* Memory Tagging API */ int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); +#ifdef CONFIG_ARM64_MTE +void flush_mte_state(void); +#else +static inline void flush_mte_state(void) +{ +} +#endif + #endif /* __ASSEMBLY__ */ #endif /* __ASM_MTE_H */ diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 512174a8e789..0c6e5523b932 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -63,6 +63,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ +#define TIF_MTE_ASYNC_FAULT 6 /* MTE Asynchronous Tag Check Fault */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -91,10 +92,11 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_FSCHECK (1 << TIF_FSCHECK) #define _TIF_32BIT (1 << TIF_32BIT) #define _TIF_SVE (1 << TIF_SVE) +#define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK) + _TIF_UPROBE | _TIF_FSCHECK | _TIF_MTE_ASYNC_FAULT) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 4e5b8ee31442..dbede7a4c5fb 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -63,6 +63,7 @@ obj-$(CONFIG_CRASH_CORE) += crash_core.o obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o obj-$(CONFIG_ARM64_SSBD) += ssbd.o obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o +obj-$(CONFIG_ARM64_MTE) += mte.o obj-y += vdso/ probes/ obj-$(CONFIG_COMPAT_VDSO) += vdso32/ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index ddcde093c433..3650a0a77ed0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -145,6 +145,31 @@ alternative_cb_end #endif .endm + /* Check for MTE asynchronous tag check faults */ + .macro check_mte_async_tcf, flgs, tmp +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 1f +alternative_else_nop_endif + mrs_s \tmp, SYS_TFSRE0_EL1 + tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f + /* Asynchronous TCF occurred for TTBR0 access, set the TI flag */ + orr \flgs, \flgs, #_TIF_MTE_ASYNC_FAULT + str \flgs, [tsk, #TSK_TI_FLAGS] + msr_s SYS_TFSRE0_EL1, xzr +1: +#endif + .endm + + /* Clear the MTE asynchronous tag check faults */ + .macro clear_mte_async_tcf +#ifdef CONFIG_ARM64_MTE +alternative_if ARM64_MTE + msr_s SYS_TFSRE0_EL1, xzr +alternative_else_nop_endif +#endif + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -176,6 +201,8 @@ alternative_cb_end ldr x19, [tsk, #TSK_TI_FLAGS] disable_step_tsk x19, x20 + /* Check for asynchronous tag check faults in user space */ + check_mte_async_tcf x19, x22 apply_ssbd 1, x22, x23 ptrauth_keys_install_kernel tsk, 1, x20, x22, x23 @@ -244,6 +271,13 @@ alternative_if ARM64_HAS_IRQ_PRIO_MASKING str x20, [sp, #S_PMR_SAVE] alternative_else_nop_endif + /* Re-enable tag checking (TCO set on exception entry) */ +#ifdef CONFIG_ARM64_MTE +alternative_if ARM64_MTE + SET_PSTATE_TCO(0) +alternative_else_nop_endif +#endif + /* * Registers that may be useful after this macro is invoked: * @@ -744,6 +778,8 @@ work_pending: ret_to_user: disable_daif gic_prio_kentry_setup tmp=x3 + /* Ignore asynchronous tag check faults in the uaccess routines */ + clear_mte_async_tcf ldr x1, [tsk, #TSK_TI_FLAGS] and x2, x1, #_TIF_WORK_MASK cbnz x2, work_pending diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c new file mode 100644 index 000000000000..032016823957 --- /dev/null +++ b/arch/arm64/kernel/mte.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 ARM Ltd. + */ + +#include + +#include +#include +#include + +void flush_mte_state(void) +{ + if (!system_supports_mte()) + return; + + /* clear any pending asynchronous tag fault */ + dsb(ish); + write_sysreg_s(0, SYS_TFSRE0_EL1); + clear_thread_flag(TIF_MTE_ASYNC_FAULT); +} diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 56be4cbf771f..740047c9cd13 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -50,6 +50,7 @@ #include #include #include +#include #include #include #include @@ -323,6 +324,7 @@ void flush_thread(void) tls_thread_flush(); flush_ptrace_hw_breakpoint(current); flush_tagged_addr_state(); + flush_mte_state(); } void release_thread(struct task_struct *dead_task) @@ -355,6 +357,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) dst->thread.sve_state = NULL; clear_tsk_thread_flag(dst, TIF_SVE); + /* clear any pending asynchronous tag fault raised by the parent */ + clear_tsk_thread_flag(dst, TIF_MTE_ASYNC_FAULT); + return 0; } diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 339882db5a91..e377d77c065e 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -732,6 +732,9 @@ static void setup_return(struct pt_regs *regs, struct k_sigaction *ka, regs->regs[29] = (unsigned long)&user->next_frame->fp; regs->pc = (unsigned long)ka->sa.sa_handler; + /* TCO (Tag Check Override) always cleared for signal handlers */ + regs->pstate &= ~PSR_TCO_BIT; + if (ka->sa.sa_flags & SA_RESTORER) sigtramp = ka->sa.sa_restorer; else @@ -923,6 +926,11 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, if (thread_flags & _TIF_UPROBE) uprobe_notify_resume(regs); + if (thread_flags & _TIF_MTE_ASYNC_FAULT) { + clear_thread_flag(TIF_MTE_ASYNC_FAULT); + force_signal_inject(SIGSEGV, SEGV_MTEAERR, 0); + } + if (thread_flags & _TIF_SIGPENDING) do_signal(regs); diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index a12c0c88d345..db25f5d6a07c 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -102,6 +102,16 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, local_daif_restore(DAIF_PROCCTX); user_exit(); + if (system_supports_mte() && (flags & _TIF_MTE_ASYNC_FAULT)) { + /* + * Process the asynchronous tag check fault before the actual + * syscall. do_notify_resume() will send a signal to userspace + * before the syscall is restarted. + */ + regs->regs[0] = -ERESTARTNOINTR; + return; + } + if (has_syscall_work(flags)) { /* set default errno for user-issued syscall(-1) */ if (scno == NO_SYSCALL) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index c9cedc0432d2..38b59cace3e3 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -650,6 +650,13 @@ static int do_sea(unsigned long addr, unsigned int esr, struct pt_regs *regs) return 0; } +static int do_tag_check_fault(unsigned long addr, unsigned int esr, + struct pt_regs *regs) +{ + do_bad_area(addr, esr, regs); + return 0; +} + static const struct fault_info fault_info[] = { { do_bad, SIGKILL, SI_KERNEL, "ttbr address size fault" }, { do_bad, SIGKILL, SI_KERNEL, "level 1 address size fault" }, @@ -668,7 +675,7 @@ static const struct fault_info fault_info[] = { { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" }, { do_sea, SIGBUS, BUS_OBJERR, "synchronous external abort" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 17" }, + { do_tag_check_fault, SIGSEGV, SEGV_MTESERR, "synchronous tag check fault" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 18" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 19" }, { do_sea, SIGKILL, SI_KERNEL, "level 0 (translation table walk)" }, From patchwork Tue Apr 21 14:25:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501583 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E86961667 for ; Tue, 21 Apr 2020 14:26:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C00A7206D9 for ; Tue, 21 Apr 2020 14:26:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C00A7206D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E98A38E0010; Tue, 21 Apr 2020 10:26:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E21A18E0003; Tue, 21 Apr 2020 10:26:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE9D98E0010; Tue, 21 Apr 2020 10:26:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id B14DF8E0003 for ; Tue, 21 Apr 2020 10:26:32 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6D5CD282D for ; Tue, 21 Apr 2020 14:26:32 +0000 (UTC) X-FDA: 76732087824.19.coach81_1bf55394f214b X-Spam-Summary: 2,0,0,10e4b275c2ca3c4f,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:2895:2897:3138:3139:3140:3141:3142:3352:3865:3867:3868:4321:4605:5007:6261:7576:7875:7903:8634:10004:11026:11232:11658:11914:12043:12114:12297:12438:12555:12679:13069:13180:13229:13311:13357:13894:14096:14181:14384:14394:14721:21080:21230:21451:21627:21990:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: coach81_1bf55394f214b X-Filterd-Recvd-Size: 3195 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:31 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7F1DE31B; Tue, 21 Apr 2020 07:26:31 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D4DEE3F68F; Tue, 21 Apr 2020 07:26:29 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Kevin Brodsky Subject: [PATCH v3 11/23] mm: Introduce arch_calc_vm_flag_bits() Date: Tue, 21 Apr 2020 15:25:51 +0100 Message-Id: <20200421142603.3894-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Kevin Brodsky Similarly to arch_calc_vm_prot_bits(), introduce a dummy arch_calc_vm_flag_bits() invoked from calc_vm_flag_bits(). This macro can be overridden by architectures to insert specific VM_* flags derived from the mmap() MAP_* flags. Signed-off-by: Kevin Brodsky Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v2: - Updated the comment above arch_calc_vm_prot_bits(). - Changed author since this patch had already been posted (internally). include/linux/mman.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/mman.h b/include/linux/mman.h index 4b08e9c9c538..15c1162b9d65 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -74,13 +74,17 @@ static inline void vm_unacct_memory(long pages) } /* - * Allow architectures to handle additional protection bits + * Allow architectures to handle additional protection and flag bits */ #ifndef arch_calc_vm_prot_bits #define arch_calc_vm_prot_bits(prot, pkey) 0 #endif +#ifndef arch_calc_vm_flag_bits +#define arch_calc_vm_flag_bits(flags) 0 +#endif + #ifndef arch_vm_get_page_prot #define arch_vm_get_page_prot(vm_flags) __pgprot(0) #endif @@ -131,7 +135,8 @@ calc_vm_flag_bits(unsigned long flags) return _calc_vm_trans(flags, MAP_GROWSDOWN, VM_GROWSDOWN ) | _calc_vm_trans(flags, MAP_DENYWRITE, VM_DENYWRITE ) | _calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED ) | - _calc_vm_trans(flags, MAP_SYNC, VM_SYNC ); + _calc_vm_trans(flags, MAP_SYNC, VM_SYNC ) | + arch_calc_vm_flag_bits(flags); } unsigned long vm_commit_limit(void); From patchwork Tue Apr 21 14:25:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501585 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E15A81575 for ; Tue, 21 Apr 2020 14:26:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ABCA3206B9 for ; Tue, 21 Apr 2020 14:26:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABCA3206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C27C58E0011; Tue, 21 Apr 2020 10:26:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BB0588E0003; Tue, 21 Apr 2020 10:26:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A50238E0011; Tue, 21 Apr 2020 10:26:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id 8E7478E0003 for ; Tue, 21 Apr 2020 10:26:34 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4CB81180AD83A for ; Tue, 21 Apr 2020 14:26:34 +0000 (UTC) X-FDA: 76732087908.11.table87_1c37a30839712 X-Spam-Summary: 2,0,0,d7aac1adf3607f11,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1605:1730:1747:1777:1792:1981:2194:2198:2199:2200:2393:2553:2559:2562:2693:2892:2895:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4051:4120:4250:4321:4605:4641:5007:6119:6261:7903:8634:8660:8957:10004:11026:11232:11473:11657:11658:11914:12043:12291:12295:12296:12297:12438:12555:12683:12986:13138:13141:13148:13161:13180:13184:13229:13230:13231:13894:13972:14394:21080:21220:21230:21433:21451:21627:21795:21990:30051:30054:30055:30056:30070:30075:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: table87_1c37a30839712 X-Filterd-Recvd-Size: 9607 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:33 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 46306C14; Tue, 21 Apr 2020 07:26:33 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B3E163F68F; Tue, 21 Apr 2020 07:26:31 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 12/23] arm64: mte: Add PROT_MTE support to mmap() and mprotect() Date: Tue, 21 Apr 2020 15:25:52 +0100 Message-Id: <20200421142603.3894-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To enable tagging on a memory range, the user must explicitly opt in via a new PROT_MTE flag passed to mmap() or mprotect(). Since this is a new memory type in the AttrIndx field of a pte, simplify the or'ing of these bits over the protection_map[] attributes by making MT_NORMAL index 0. There are two conditions for arch_vm_get_page_prot() to return the MT_NORMAL_TAGGED memory type: (1) the user requested it via PROT_MTE, registered as VM_MTE in the vm_flags, and (2) the vma supports MTE, decided during the mmap() call (only) and registered as VM_MTE_ALLOWED. arch_calc_vm_prot_bits() is responsible for registering the user request as VM_MTE. The newly introduced arch_calc_vm_flag_bits() sets VM_MTE_ALLOWED if the mapping is MAP_ANONYMOUS. An MTE-capable filesystem (RAM-based) may be able to set VM_MTE_ALLOWED during its mmap() file ops call. In addition, update VM_DATA_DEFAULT_FLAGS to allow mprotect(PROT_MTE) on stack or brk area. The Linux mmap() syscall currently ignores unknown PROT_* flags. In the presence of MTE, an mmap(PROT_MTE) on a file which does not support MTE will not report an error and the memory will not be mapped as Normal Tagged. For consistency, mprotect(PROT_MTE) will not report an error either if the memory range does not support MTE. Two subsequent patches in the series will propose tightening of this behaviour. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v2: - Add VM_MTE_ALLOWED to show_smap_vma_flags(). arch/arm64/include/asm/memory.h | 18 +++++---- arch/arm64/include/asm/mman.h | 64 ++++++++++++++++++++++++++++++ arch/arm64/include/asm/page.h | 2 +- arch/arm64/include/asm/pgtable.h | 7 +++- arch/arm64/include/uapi/asm/mman.h | 14 +++++++ fs/proc/task_mmu.c | 4 ++ include/linux/mm.h | 8 ++++ 7 files changed, 108 insertions(+), 9 deletions(-) create mode 100644 arch/arm64/include/asm/mman.h create mode 100644 arch/arm64/include/uapi/asm/mman.h diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 472c77a68225..770535b7ca35 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -129,14 +129,18 @@ /* * Memory types available. + * + * IMPORTANT: MT_NORMAL must be index 0 since vm_get_page_prot() may 'or' in + * the MT_NORMAL_TAGGED memory type for PROT_MTE mappings. Note + * that protection_map[] only contains MT_NORMAL attributes. */ -#define MT_DEVICE_nGnRnE 0 -#define MT_DEVICE_nGnRE 1 -#define MT_DEVICE_GRE 2 -#define MT_NORMAL_NC 3 -#define MT_NORMAL 4 -#define MT_NORMAL_WT 5 -#define MT_NORMAL_TAGGED 6 +#define MT_NORMAL 0 +#define MT_NORMAL_TAGGED 1 +#define MT_NORMAL_NC 2 +#define MT_NORMAL_WT 3 +#define MT_DEVICE_nGnRnE 4 +#define MT_DEVICE_nGnRE 5 +#define MT_DEVICE_GRE 6 /* * Memory types for Stage-2 translation diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h new file mode 100644 index 000000000000..c77a23869223 --- /dev/null +++ b/arch/arm64/include/asm/mman.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_MMAN_H__ +#define __ASM_MMAN_H__ + +#include + +/* + * There are two conditions required for returning a Normal Tagged memory type + * in arch_vm_get_page_prot(): (1) the user requested it via PROT_MTE passed + * to mmap() or mprotect() and (2) the corresponding vma supports MTE. We + * register (1) as VM_MTE in the vma->vm_flags and (2) as VM_MTE_ALLOWED. Note + * that the latter can only be set during the mmap() call since mprotect() + * does not accept MAP_* flags. + */ +static inline unsigned long arch_calc_vm_prot_bits(unsigned long prot, + unsigned long pkey) +{ + if (!system_supports_mte()) + return 0; + + if (prot & PROT_MTE) + return VM_MTE; + + return 0; +} +#define arch_calc_vm_prot_bits arch_calc_vm_prot_bits + +static inline unsigned long arch_calc_vm_flag_bits(unsigned long flags) +{ + if (!system_supports_mte()) + return 0; + + /* + * Only allow MTE on anonymous mappings as these are guaranteed to be + * backed by tags-capable memory. The vm_flags may be overridden by a + * filesystem supporting MTE (RAM-based). + */ + if (flags & MAP_ANONYMOUS) + return VM_MTE_ALLOWED; + + return 0; +} +#define arch_calc_vm_flag_bits arch_calc_vm_flag_bits + +static inline pgprot_t arch_vm_get_page_prot(unsigned long vm_flags) +{ + return (vm_flags & VM_MTE) && (vm_flags & VM_MTE_ALLOWED) ? + __pgprot(PTE_ATTRINDX(MT_NORMAL_TAGGED)) : + __pgprot(0); +} +#define arch_vm_get_page_prot arch_vm_get_page_prot + +static inline bool arch_validate_prot(unsigned long prot, unsigned long addr) +{ + unsigned long supported = PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM; + + if (system_supports_mte()) + supported |= PROT_MTE; + + return (prot & ~supported) == 0; +} +#define arch_validate_prot arch_validate_prot + +#endif /* !__ASM_MMAN_H__ */ diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h index c01b52add377..673033e0393b 100644 --- a/arch/arm64/include/asm/page.h +++ b/arch/arm64/include/asm/page.h @@ -36,7 +36,7 @@ extern int pfn_valid(unsigned long); #endif /* !__ASSEMBLY__ */ -#define VM_DATA_DEFAULT_FLAGS VM_DATA_FLAGS_TSK_EXEC +#define VM_DATA_DEFAULT_FLAGS (VM_DATA_FLAGS_TSK_EXEC | VM_MTE_ALLOWED) #include diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 538c85e62f86..39a372bf8afc 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -659,8 +659,13 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd) static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) { + /* + * Normal and Normal-Tagged are two different memory types and indices + * in MAIR_EL1. The mask below has to include PTE_ATTRINDX_MASK. + */ const pteval_t mask = PTE_USER | PTE_PXN | PTE_UXN | PTE_RDONLY | - PTE_PROT_NONE | PTE_VALID | PTE_WRITE; + PTE_PROT_NONE | PTE_VALID | PTE_WRITE | + PTE_ATTRINDX_MASK; /* preserve the hardware dirty information */ if (pte_hw_dirty(pte)) pte = pte_mkdirty(pte); diff --git a/arch/arm64/include/uapi/asm/mman.h b/arch/arm64/include/uapi/asm/mman.h new file mode 100644 index 000000000000..d7677ee84878 --- /dev/null +++ b/arch/arm64/include/uapi/asm/mman.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _UAPI__ASM_MMAN_H +#define _UAPI__ASM_MMAN_H + +#include + +/* + * The generic mman.h file reserves 0x10 and 0x20 for arch-specific PROT_* + * flags. + */ +/* 0x10 reserved for PROT_BTI */ +#define PROT_MTE 0x20 /* Normal Tagged mapping */ + +#endif /* !_UAPI__ASM_MMAN_H */ diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 8d382d4ec067..2f26112ebb77 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -647,6 +647,10 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_MERGEABLE)] = "mg", [ilog2(VM_UFFD_MISSING)]= "um", [ilog2(VM_UFFD_WP)] = "uw", +#ifdef CONFIG_ARM64_MTE + [ilog2(VM_MTE)] = "mt", + [ilog2(VM_MTE_ALLOWED)] = "", +#endif #ifdef CONFIG_ARCH_HAS_PKEYS /* These come out via ProtectionKey: */ [ilog2(VM_PKEY_BIT0)] = "", diff --git a/include/linux/mm.h b/include/linux/mm.h index 5a323422d783..132ca88e407d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -336,6 +336,14 @@ extern unsigned int kobjsize(const void *objp); # define VM_MPX VM_NONE #endif +#if defined(CONFIG_ARM64_MTE) +# define VM_MTE VM_HIGH_ARCH_0 /* Use Tagged memory for access control */ +# define VM_MTE_ALLOWED VM_HIGH_ARCH_1 /* Tagged memory permitted */ +#else +# define VM_MTE VM_NONE +# define VM_MTE_ALLOWED VM_NONE +#endif + #ifndef VM_GROWSUP # define VM_GROWSUP VM_NONE #endif From patchwork Tue Apr 21 14:25:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F4228913 for ; Tue, 21 Apr 2020 14:26:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CB552206F4 for ; Tue, 21 Apr 2020 14:26:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB552206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4D2CF8E0012; Tue, 21 Apr 2020 10:26:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 482C98E0003; Tue, 21 Apr 2020 10:26:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34DE78E0012; Tue, 21 Apr 2020 10:26:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 1C0D68E0003 for ; Tue, 21 Apr 2020 10:26:36 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C6DF3181AEF07 for ; Tue, 21 Apr 2020 14:26:35 +0000 (UTC) X-FDA: 76732087950.03.list73_1c720ef15000c X-Spam-Summary: 2,0,0,ded20cb4ab4b2cc5,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:2693:2895:2914:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3872:4321:4385:4605:5007:6261:7875:8634:10004:11026:11232:11658:11914:12043:12114:12296:12297:12438:12555:13180:13229:13894:14096:14181:14394:14721:21080:21230:21451:21627:21990:30012:30054:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:13,LUA_SUMMARY:none X-HE-Tag: list73_1c720ef15000c X-Filterd-Recvd-Size: 3769 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:35 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 097C131B; Tue, 21 Apr 2020 07:26:35 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7A5283F68F; Tue, 21 Apr 2020 07:26:33 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 13/23] mm: Introduce arch_validate_flags() Date: Tue, 21 Apr 2020 15:25:53 +0100 Message-Id: <20200421142603.3894-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to arch_validate_prot() called from do_mprotect_pkey(), an architecture may need to sanity-check the new vm_flags. Define a dummy function always returning true. In addition to do_mprotect_pkey(), also invoke it from mmap_region() prior to updating vma->vm_page_prot to allow the architecture code to veto potentially inconsistent vm_flags. Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v2: - Some comments updated. include/linux/mman.h | 13 +++++++++++++ mm/mmap.c | 9 +++++++++ mm/mprotect.c | 6 ++++++ 3 files changed, 28 insertions(+) diff --git a/include/linux/mman.h b/include/linux/mman.h index 15c1162b9d65..09dd414b81b6 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -103,6 +103,19 @@ static inline bool arch_validate_prot(unsigned long prot, unsigned long addr) #define arch_validate_prot arch_validate_prot #endif +#ifndef arch_validate_flags +/* + * This is called from mmap() and mprotect() with the updated vma->vm_flags. + * + * Returns true if the VM_* flags are valid. + */ +static inline bool arch_validate_flags(unsigned long flags) +{ + return true; +} +#define arch_validate_flags arch_validate_flags +#endif + /* * Optimisation macro. It is equivalent to: * (x & bit1) ? bit2 : 0 diff --git a/mm/mmap.c b/mm/mmap.c index f609e9ec4a25..d5fc93c2072e 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1792,6 +1792,15 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vma_set_anonymous(vma); } + /* Allow architectures to sanity-check the vm_flags */ + if (!arch_validate_flags(vma->vm_flags)) { + error = -EINVAL; + if (file) + goto unmap_and_free_vma; + else + goto free_vma; + } + vma_link(mm, vma, prev, rb_link, rb_parent); /* Once vma denies write, undo our temporary denial count */ if (file) { diff --git a/mm/mprotect.c b/mm/mprotect.c index 494192ca954b..04b1d2cf0e74 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -603,6 +603,12 @@ static int do_mprotect_pkey(unsigned long start, size_t len, goto out; } + /* Allow architectures to sanity-check the new flags */ + if (!arch_validate_flags(newflags)) { + error = -EINVAL; + goto out; + } + error = security_file_mprotect(vma, reqprot, prot); if (error) goto out; From patchwork Tue Apr 21 14:25:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BED11575 for ; Tue, 21 Apr 2020 14:26:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A01B1206D9 for ; Tue, 21 Apr 2020 14:26:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A01B1206D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3ECA28E0013; Tue, 21 Apr 2020 10:26:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2872E8E0003; Tue, 21 Apr 2020 10:26:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 177468E0013; Tue, 21 Apr 2020 10:26:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0213.hostedemail.com [216.40.44.213]) by kanga.kvack.org (Postfix) with ESMTP id EE6028E0003 for ; Tue, 21 Apr 2020 10:26:37 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 8C37E52C7 for ; Tue, 21 Apr 2020 14:26:37 +0000 (UTC) X-FDA: 76732088034.14.quilt39_1cb6473a67e5b X-Spam-Summary: 2,0,0,c35bf76fd7777ad3,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3871:4321:5007:6261:8634:10004:11026:11473:11657:11658:11914:12043:12114:12296:12297:12438:12555:12986:13069:13161:13229:13255:13311:13357:13894:14096:14181:14384:14394:14721:21080:21212:21230:21451:21627:21990:30054:30069:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: quilt39_1cb6473a67e5b X-Filterd-Recvd-Size: 2939 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:37 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C73A811B3; Tue, 21 Apr 2020 07:26:36 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 416A33F68F; Tue, 21 Apr 2020 07:26:35 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 14/23] arm64: mte: Validate the PROT_MTE request via arch_validate_flags() Date: Tue, 21 Apr 2020 15:25:54 +0100 Message-Id: <20200421142603.3894-15-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make use of the newly introduced arch_validate_flags() hook to sanity-check the PROT_MTE request passed to mmap() and mprotect(). If the mapping does not support MTE, these syscalls will return -EINVAL. Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/mman.h | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h index c77a23869223..5c356d1ca266 100644 --- a/arch/arm64/include/asm/mman.h +++ b/arch/arm64/include/asm/mman.h @@ -44,7 +44,11 @@ static inline unsigned long arch_calc_vm_flag_bits(unsigned long flags) static inline pgprot_t arch_vm_get_page_prot(unsigned long vm_flags) { - return (vm_flags & VM_MTE) && (vm_flags & VM_MTE_ALLOWED) ? + /* + * Checking for VM_MTE only is sufficient since arch_validate_flags() + * does not permit (VM_MTE & !VM_MTE_ALLOWED). + */ + return (vm_flags & VM_MTE) ? __pgprot(PTE_ATTRINDX(MT_NORMAL_TAGGED)) : __pgprot(0); } @@ -61,4 +65,14 @@ static inline bool arch_validate_prot(unsigned long prot, unsigned long addr) } #define arch_validate_prot arch_validate_prot +static inline bool arch_validate_flags(unsigned long flags) +{ + if (!system_supports_mte()) + return true; + + /* only allow VM_MTE if VM_MTE_ALLOWED has been set previously */ + return !(flags & VM_MTE) || (flags & VM_MTE_ALLOWED); +} +#define arch_validate_flags arch_validate_flags + #endif /* !__ASM_MMAN_H__ */ From patchwork Tue Apr 21 14:25:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6FE21575 for ; Tue, 21 Apr 2020 14:26:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E549206B9 for ; Tue, 21 Apr 2020 14:26:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E549206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 19E548E0014; Tue, 21 Apr 2020 10:26:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 12AE88E0003; Tue, 21 Apr 2020 10:26:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0AC58E0014; Tue, 21 Apr 2020 10:26:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id D00AE8E0003 for ; Tue, 21 Apr 2020 10:26:39 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8EF43181AEF07 for ; Tue, 21 Apr 2020 14:26:39 +0000 (UTC) X-FDA: 76732088118.14.gold06_1cf94ca8c4a0d X-Spam-Summary: 2,0,0,f73e80515afd6910,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1540:1711:1730:1747:1777:1792:2393:2559:2562:2693:2895:3138:3139:3140:3141:3142:3352:3865:3870:3871:4321:5007:6261:7875:7903:8634:10004:11026:11473:11658:11914:12043:12296:12297:12438:12555:12986:13069:13184:13229:13311:13357:13894:14181:14384:14394:14664:14721:21080:21230:21451:21627:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: gold06_1cf94ca8c4a0d X-Filterd-Recvd-Size: 2242 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:38 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 96AEC101E; Tue, 21 Apr 2020 07:26:38 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 06D3D3F68F; Tue, 21 Apr 2020 07:26:36 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 15/23] mm: Allow arm64 mmap(PROT_MTE) on RAM-based files Date: Tue, 21 Apr 2020 15:25:55 +0100 Message-Id: <20200421142603.3894-16-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since arm64 memory (allocation) tags can only be stored in RAM, mapping files with PROT_MTE is not allowed by default. RAM-based files like those in a tmpfs mount or memfd_create() can support memory tagging, so update the vm_flags accordingly in shmem_mmap(). Signed-off-by: Catalin Marinas Cc: Will Deacon --- mm/shmem.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index d722eb830317..73754ed7af69 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2221,6 +2221,9 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma) vma->vm_flags &= ~(VM_MAYWRITE); } + /* arm64 - allow memory tagging on RAM-based files */ + vma->vm_flags |= VM_MTE_ALLOWED; + file_accessed(file); vma->vm_ops = &shmem_vm_ops; if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && From patchwork Tue Apr 21 14:25:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501593 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A62DB913 for ; Tue, 21 Apr 2020 14:27:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6FDC9206B9 for ; Tue, 21 Apr 2020 14:27:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6FDC9206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 08CF68E0015; Tue, 21 Apr 2020 10:26:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 03EFF8E0003; Tue, 21 Apr 2020 10:26:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E47D18E0015; Tue, 21 Apr 2020 10:26:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id C338C8E0003 for ; Tue, 21 Apr 2020 10:26:41 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7D055584D for ; Tue, 21 Apr 2020 14:26:41 +0000 (UTC) X-FDA: 76732088202.22.cars20_1d44917a4e23d X-Spam-Summary: 2,0,0,d83adced2e1e7d38,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1605:1730:1747:1777:1792:2198:2199:2393:2538:2559:2562:2693:2895:2903:3138:3139:3140:3141:3142:3622:3865:3866:3867:3868:3870:3871:3872:3874:4050:4119:4250:4321:4605:5007:6119:6261:7875:7903:8634:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12683:12986:13138:13141:13161:13180:13229:13230:13231:13255:13894:13972:14096:14394:21080:21220:21230:21433:21451:21524:21525:21627:21795:21966:21990:30051:30054:30062:30067:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: cars20_1d44917a4e23d X-Filterd-Recvd-Size: 8961 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:40 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 88C3A31B; Tue, 21 Apr 2020 07:26:40 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CEDBD3F68F; Tue, 21 Apr 2020 07:26:38 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 16/23] arm64: mte: Allow user control of the tag check mode via prctl() Date: Tue, 21 Apr 2020 15:25:56 +0100 Message-Id: <20200421142603.3894-17-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: By default, even if PROT_MTE is set on a memory range, there is no tag check fault reporting (SIGSEGV). Introduce a set of option to the exiting prctl(PR_SET_TAGGED_ADDR_CTRL) to allow user control of the tag check fault mode: PR_MTE_TCF_NONE - no reporting (default) PR_MTE_TCF_SYNC - synchronous tag check fault reporting PR_MTE_TCF_ASYNC - asynchronous tag check fault reporting These options translate into the corresponding SCTLR_EL1.TCF0 bitfield, context-switched by the kernel. Note that uaccess done by the kernel is not checked and cannot be configured by the user. Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v3: - Use SCTLR_EL1_TCF0_NONE instead of 0 for consistency. - Move mte_thread_switch() in this patch from an earlier one. In addition, it is called after the dsb() in __switch_to() so that any asynchronous tag check faults have been registered in the TFSR_EL1 registers (to be added with the in-kernel MTE support. v2: - Handle SCTLR_EL1_TCF0_NONE explicitly for consistency with PR_MTE_TCF_NONE. - Fix SCTLR_EL1 register setting in flush_mte_state() (thanks to Peter Collingbourne). - Added ISB to update_sctlr_el1_tcf0() since, with the latest architecture update/fix, the TCF0 field is used by the uaccess routines. arch/arm64/include/asm/mte.h | 14 ++++++ arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/kernel/mte.c | 77 ++++++++++++++++++++++++++++++ arch/arm64/kernel/process.c | 26 ++++++++-- include/uapi/linux/prctl.h | 6 +++ 5 files changed, 123 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index e9711ea51eb5..3dc0a7977124 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -11,10 +11,24 @@ int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); #ifdef CONFIG_ARM64_MTE void flush_mte_state(void); +void mte_thread_switch(struct task_struct *next); +long set_mte_ctrl(unsigned long arg); +long get_mte_ctrl(void); #else static inline void flush_mte_state(void) { } +static inline void mte_thread_switch(struct task_struct *next) +{ +} +static inline long set_mte_ctrl(unsigned long arg) +{ + return 0; +} +static inline long get_mte_ctrl(void) +{ + return 0; +} #endif #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 240fe5e5b720..80e7f0573309 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -151,6 +151,9 @@ struct thread_struct { struct ptrauth_keys_user keys_user; struct ptrauth_keys_kernel keys_kernel; #endif +#ifdef CONFIG_ARM64_MTE + u64 sctlr_tcf0; +#endif }; static inline void arch_thread_struct_whitelist(unsigned long *offset, diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 032016823957..e62d02890d12 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -3,12 +3,34 @@ * Copyright (C) 2020 ARM Ltd. */ +#include +#include #include #include #include #include +static void update_sctlr_el1_tcf0(u64 tcf0) +{ + /* ISB required for the kernel uaccess routines */ + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); + isb(); +} + +static void set_sctlr_el1_tcf0(u64 tcf0) +{ + /* + * mte_thread_switch() checks current->thread.sctlr_tcf0 as an + * optimisation. Disable preemption so that it does not see + * the variable update before the SCTLR_EL1.TCF0 one. + */ + preempt_disable(); + current->thread.sctlr_tcf0 = tcf0; + update_sctlr_el1_tcf0(tcf0); + preempt_enable(); +} + void flush_mte_state(void) { if (!system_supports_mte()) @@ -18,4 +40,59 @@ void flush_mte_state(void) dsb(ish); write_sysreg_s(0, SYS_TFSRE0_EL1); clear_thread_flag(TIF_MTE_ASYNC_FAULT); + /* disable tag checking */ + set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE); +} + +void mte_thread_switch(struct task_struct *next) +{ + if (!system_supports_mte()) + return; + + /* avoid expensive SCTLR_EL1 accesses if no change */ + if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) + update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); +} + +long set_mte_ctrl(unsigned long arg) +{ + u64 tcf0; + + if (!system_supports_mte()) + return 0; + + switch (arg & PR_MTE_TCF_MASK) { + case PR_MTE_TCF_NONE: + tcf0 = SCTLR_EL1_TCF0_NONE; + break; + case PR_MTE_TCF_SYNC: + tcf0 = SCTLR_EL1_TCF0_SYNC; + break; + case PR_MTE_TCF_ASYNC: + tcf0 = SCTLR_EL1_TCF0_ASYNC; + break; + default: + return -EINVAL; + } + + set_sctlr_el1_tcf0(tcf0); + + return 0; +} + +long get_mte_ctrl(void) +{ + if (!system_supports_mte()) + return 0; + + switch (current->thread.sctlr_tcf0) { + case SCTLR_EL1_TCF0_NONE: + return PR_MTE_TCF_NONE; + case SCTLR_EL1_TCF0_SYNC: + return PR_MTE_TCF_SYNC; + case SCTLR_EL1_TCF0_ASYNC: + return PR_MTE_TCF_ASYNC; + } + + return 0; } diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 740047c9cd13..ff6031a398d0 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -529,6 +529,13 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, */ dsb(ish); + /* + * MTE thread switching must happen after the DSB above to ensure that + * any asynchronous tag check faults have been logged in the TFSR*_EL1 + * registers. + */ + mte_thread_switch(next); + /* the actual thread switch */ last = cpu_switch_to(prev, next); @@ -588,9 +595,15 @@ static unsigned int tagged_addr_disabled; long set_tagged_addr_ctrl(unsigned long arg) { + unsigned long valid_mask = PR_TAGGED_ADDR_ENABLE; + if (is_compat_task()) return -EINVAL; - if (arg & ~PR_TAGGED_ADDR_ENABLE) + + if (system_supports_mte()) + valid_mask |= PR_MTE_TCF_MASK; + + if (arg & ~valid_mask) return -EINVAL; /* @@ -600,6 +613,9 @@ long set_tagged_addr_ctrl(unsigned long arg) if (arg & PR_TAGGED_ADDR_ENABLE && tagged_addr_disabled) return -EINVAL; + if (set_mte_ctrl(arg) != 0) + return -EINVAL; + update_thread_flag(TIF_TAGGED_ADDR, arg & PR_TAGGED_ADDR_ENABLE); return 0; @@ -607,13 +623,17 @@ long set_tagged_addr_ctrl(unsigned long arg) long get_tagged_addr_ctrl(void) { + long ret = 0; + if (is_compat_task()) return -EINVAL; if (test_thread_flag(TIF_TAGGED_ADDR)) - return PR_TAGGED_ADDR_ENABLE; + ret = PR_TAGGED_ADDR_ENABLE; - return 0; + ret |= get_mte_ctrl(); + + return ret; } /* diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 07b4f8131e36..2390ab324afa 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -233,6 +233,12 @@ struct prctl_mm_map { #define PR_SET_TAGGED_ADDR_CTRL 55 #define PR_GET_TAGGED_ADDR_CTRL 56 # define PR_TAGGED_ADDR_ENABLE (1UL << 0) +/* MTE tag check fault modes */ +# define PR_MTE_TCF_SHIFT 1 +# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) /* Control reclaim behavior when allocating memory */ #define PR_SET_IO_FLUSHER 57 From patchwork Tue Apr 21 14:25:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501595 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52C91913 for ; Tue, 21 Apr 2020 14:27:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 12CAA206B9 for ; Tue, 21 Apr 2020 14:27:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12CAA206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 324AF8E0016; Tue, 21 Apr 2020 10:26:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2D6768E0003; Tue, 21 Apr 2020 10:26:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 153A88E0016; Tue, 21 Apr 2020 10:26:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id E75908E0003 for ; Tue, 21 Apr 2020 10:26:43 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A8E432C14 for ; Tue, 21 Apr 2020 14:26:43 +0000 (UTC) X-FDA: 76732088286.18.scale75_1d95c12152f01 X-Spam-Summary: 2,0,0,54f0512f9ff95a8c,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:2:41:355:379:541:800:960:966:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1605:1606:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3622:3865:3867:3868:3870:3871:3874:4117:4250:4321:4385:4605:5007:6261:7875:7903:8634:9036:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12683:12986:13138:13161:13180:13229:13231:13255:13894:13972:14394:21080:21230:21451:21627:21795:21990:30003:30051:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:49,LUA_SUMMARY:none X-HE-Tag: scale75_1d95c12152f01 X-Filterd-Recvd-Size: 6610 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:43 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B783FC14; Tue, 21 Apr 2020 07:26:42 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C1FB63F68F; Tue, 21 Apr 2020 07:26:40 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 17/23] arm64: mte: Allow user control of the generated random tags via prctl() Date: Tue, 21 Apr 2020 15:25:57 +0100 Message-Id: <20200421142603.3894-18-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The IRG, ADDG and SUBG instructions insert a random tag in the resulting address. Certain tags can be excluded via the GCR_EL1.Exclude bitmap when, for example, the user wants a certain colour for freed buffers. Since the GCR_EL1 register is not accessible at EL0, extend the prctl(PR_SET_TAGGED_ADDR_CTRL) interface to include a 16-bit field in the first argument for controlling which tags can be generated by the above instruction (an include rather than exclude mask). Note that by default all non-zero tags are excluded. This setting is per-thread. Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v2: - Switch from an exclude mask to an include one for the prctl() interface. - Reset the allowed tags mask during flush_thread(). arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/sysreg.h | 7 ++++++ arch/arm64/kernel/mte.c | 35 +++++++++++++++++++++++++++--- arch/arm64/kernel/process.c | 2 +- include/uapi/linux/prctl.h | 3 +++ 5 files changed, 44 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 80e7f0573309..996b882a32d9 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -153,6 +153,7 @@ struct thread_struct { #endif #ifdef CONFIG_ARM64_MTE u64 sctlr_tcf0; + u64 gcr_incl; #endif }; diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 86236ae6c4e7..cb247f2f75ca 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -981,6 +981,13 @@ write_sysreg(__scs_new, sysreg); \ } while (0) +#define sysreg_clear_set_s(sysreg, clear, set) do { \ + u64 __scs_val = read_sysreg_s(sysreg); \ + u64 __scs_new = (__scs_val & ~(u64)(clear)) | (set); \ + if (__scs_new != __scs_val) \ + write_sysreg_s(__scs_new, sysreg); \ +} while (0) + #endif #endif /* __ASM_SYSREG_H */ diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index e62d02890d12..212b9fac294d 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -31,6 +31,25 @@ static void set_sctlr_el1_tcf0(u64 tcf0) preempt_enable(); } +static void update_gcr_el1_excl(u64 incl) +{ + u64 excl = ~incl & SYS_GCR_EL1_EXCL_MASK; + + /* + * Note that 'incl' is an include mask (controlled by the user via + * prctl()) while GCR_EL1 accepts an exclude mask. + * No need for ISB since this only affects EL0 currently, implicit + * with ERET. + */ + sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl); +} + +static void set_gcr_el1_excl(u64 incl) +{ + current->thread.gcr_incl = incl; + update_gcr_el1_excl(incl); +} + void flush_mte_state(void) { if (!system_supports_mte()) @@ -42,6 +61,8 @@ void flush_mte_state(void) clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking */ set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE); + /* reset tag generation mask */ + set_gcr_el1_excl(0); } void mte_thread_switch(struct task_struct *next) @@ -52,6 +73,7 @@ void mte_thread_switch(struct task_struct *next) /* avoid expensive SCTLR_EL1 accesses if no change */ if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); + update_gcr_el1_excl(next->thread.gcr_incl); } long set_mte_ctrl(unsigned long arg) @@ -76,23 +98,30 @@ long set_mte_ctrl(unsigned long arg) } set_sctlr_el1_tcf0(tcf0); + set_gcr_el1_excl((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT); return 0; } long get_mte_ctrl(void) { + unsigned long ret; + if (!system_supports_mte()) return 0; + ret = current->thread.gcr_incl << PR_MTE_TAG_SHIFT; + switch (current->thread.sctlr_tcf0) { case SCTLR_EL1_TCF0_NONE: return PR_MTE_TCF_NONE; case SCTLR_EL1_TCF0_SYNC: - return PR_MTE_TCF_SYNC; + ret |= PR_MTE_TCF_SYNC; + break; case SCTLR_EL1_TCF0_ASYNC: - return PR_MTE_TCF_ASYNC; + ret |= PR_MTE_TCF_ASYNC; + break; } - return 0; + return ret; } diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index ff6031a398d0..697571be259b 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -601,7 +601,7 @@ long set_tagged_addr_ctrl(unsigned long arg) return -EINVAL; if (system_supports_mte()) - valid_mask |= PR_MTE_TCF_MASK; + valid_mask |= PR_MTE_TCF_MASK | PR_MTE_TAG_MASK; if (arg & ~valid_mask) return -EINVAL; diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 2390ab324afa..7f0827705c9a 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -239,6 +239,9 @@ struct prctl_mm_map { # define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) # define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) # define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) +/* MTE tag inclusion mask */ +# define PR_MTE_TAG_SHIFT 3 +# define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT) /* Control reclaim behavior when allocating memory */ #define PR_SET_IO_FLUSHER 57 From patchwork Tue Apr 21 14:25:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4B351575 for ; Tue, 21 Apr 2020 14:27:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AB82E206B9 for ; Tue, 21 Apr 2020 14:27:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB82E206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 08CFF8E0017; Tue, 21 Apr 2020 10:26:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 03E998E0003; Tue, 21 Apr 2020 10:26:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DAD578E0017; Tue, 21 Apr 2020 10:26:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id B98AD8E0003 for ; Tue, 21 Apr 2020 10:26:45 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7EC6E82499A8 for ; Tue, 21 Apr 2020 14:26:45 +0000 (UTC) X-FDA: 76732088370.11.error11_1ddc5e3f32827 X-Spam-Summary: 2,0,0,c33272038776cca4,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2559:2562:2693:2896:3138:3139:3140:3141:3142:3353:3865:3866:3870:4250:4321:4605:5007:6261:8634:10004:11026:11232:11473:11657:11658:11914:12043:12296:12297:12438:12555:12986:13161:13180:13229:13894:13972:14096:14130:14181:14394:14721:21080:21230:21433:21451:21627:21990:30054:30069,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:5,LUA_SUMMARY:none X-HE-Tag: error11_1ddc5e3f32827 X-Filterd-Recvd-Size: 3786 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:45 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A1C15C14; Tue, 21 Apr 2020 07:26:44 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id EFEF93F68F; Tue, 21 Apr 2020 07:26:42 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Lorenzo Pieralisi Subject: [PATCH v3 18/23] arm64: mte: Restore the GCR_EL1 register after a suspend Date: Tue, 21 Apr 2020 15:25:58 +0100 Message-Id: <20200421142603.3894-19-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The CPU resume/suspend routines only take care of the common system registers. Restore GCR_EL1 in addition via the __cpu_suspend_exit() function. Signed-off-by: Catalin Marinas Cc: Will Deacon Cc: Lorenzo Pieralisi Reviewed-by: Lorenzo Pieralisi --- Notes: New in v3. arch/arm64/include/asm/mte.h | 4 ++++ arch/arm64/kernel/mte.c | 8 ++++++++ arch/arm64/kernel/suspend.c | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 3dc0a7977124..22eb3e06f311 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -12,6 +12,7 @@ int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); #ifdef CONFIG_ARM64_MTE void flush_mte_state(void); void mte_thread_switch(struct task_struct *next); +void mte_suspend_exit(void); long set_mte_ctrl(unsigned long arg); long get_mte_ctrl(void); #else @@ -21,6 +22,9 @@ static inline void flush_mte_state(void) static inline void mte_thread_switch(struct task_struct *next) { } +static inline void mte_suspend_exit(void) +{ +} static inline long set_mte_ctrl(unsigned long arg) { return 0; diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 212b9fac294d..fa4a4196b248 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -76,6 +76,14 @@ void mte_thread_switch(struct task_struct *next) update_gcr_el1_excl(next->thread.gcr_incl); } +void mte_suspend_exit(void) +{ + if (!system_supports_mte()) + return; + + update_gcr_el1_excl(current->thread.gcr_incl); +} + long set_mte_ctrl(unsigned long arg) { u64 tcf0; diff --git a/arch/arm64/kernel/suspend.c b/arch/arm64/kernel/suspend.c index 9405d1b7f4b0..1d405b73d009 100644 --- a/arch/arm64/kernel/suspend.c +++ b/arch/arm64/kernel/suspend.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -74,6 +75,9 @@ void notrace __cpu_suspend_exit(void) */ if (arm64_get_ssbd_state() == ARM64_SSBD_FORCE_DISABLE) arm64_set_ssbd_mitigation(false); + + /* Restore additional MTE-specific configuration */ + mte_suspend_exit(); } /* From patchwork Tue Apr 21 14:25:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E725913 for ; Tue, 21 Apr 2020 14:27:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3D462206B9 for ; Tue, 21 Apr 2020 14:27:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D462206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3C05A8E0018; Tue, 21 Apr 2020 10:26:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 34E5A8E0003; Tue, 21 Apr 2020 10:26:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 129368E0018; Tue, 21 Apr 2020 10:26:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id E8ABB8E0003 for ; Tue, 21 Apr 2020 10:26:47 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A8E28181AEF07 for ; Tue, 21 Apr 2020 14:26:47 +0000 (UTC) X-FDA: 76732088454.29.mass28_1e291cf9f111d X-Spam-Summary: 2,0,0,7cb082da0b98289c,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:1:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1605:1730:1747:1777:1792:2393:2538:2559:2562:2693:2895:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3874:4052:4128:4250:4321:4605:5007:6119:6121:6261:6742:7875:7903:7974:8634:10004:11026:11232:11233:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12986:13161:13180:13229:13894:14394:21080:21094:21230:21323:21433:21451:21627:21795:21939:21990:30003:30034:30051:30054:30067:30070:30079,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: mass28_1e291cf9f111d X-Filterd-Recvd-Size: 11167 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:47 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C377F101E; Tue, 21 Apr 2020 07:26:46 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D7B4B3F68F; Tue, 21 Apr 2020 07:26:44 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Alan Hayward , Luis Machado , Omair Javaid Subject: [PATCH v3 19/23] arm64: mte: Add PTRACE_{PEEK,POKE}MTETAGS support Date: Tue, 21 Apr 2020 15:25:59 +0100 Message-Id: <20200421142603.3894-20-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add support for bulk setting/getting of the MTE tags in a tracee's address space at 'addr' in the ptrace() syscall prototype. 'data' points to a struct iovec in the tracer's address space with iov_base representing the address of a tracer's buffer of length iov_len. The tags to be copied to/from the tracer's buffer are stored as one tag per byte. On successfully copying at least one tag, ptrace() returns 0 and updates the tracer's iov_len with the number of tags copied. In case of error, either -EIO or -EFAULT is returned, trying to follow the ptrace() man page. Note that the tag copying functions are not performance critical, therefore they lack optimisations found in typical memory copy routines. Signed-off-by: Catalin Marinas Cc: Will Deacon Cc: Alan Hayward Cc: Luis Machado Cc: Omair Javaid --- Notes: New in v3. arch/arm64/include/asm/mte.h | 17 ++++ arch/arm64/include/uapi/asm/ptrace.h | 3 + arch/arm64/kernel/mte.c | 127 +++++++++++++++++++++++++++ arch/arm64/kernel/ptrace.c | 15 +++- arch/arm64/lib/mte.S | 50 +++++++++++ 5 files changed, 211 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 22eb3e06f311..0ca2aaff07a1 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -2,12 +2,21 @@ #ifndef __ASM_MTE_H #define __ASM_MTE_H +#define MTE_ALLOC_SIZE UL(16) +#define MTE_ALLOC_MASK (~(MTE_ALLOC_SIZE - 1)) +#define MTE_TAG_SHIFT (56) +#define MTE_TAG_SIZE (4) + #ifndef __ASSEMBLY__ #include /* Memory Tagging API */ int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); +unsigned long mte_copy_tags_from_user(void *to, const void __user *from, + unsigned long n); +unsigned long mte_copy_tags_to_user(void __user *to, void *from, + unsigned long n); #ifdef CONFIG_ARM64_MTE void flush_mte_state(void); @@ -15,6 +24,8 @@ void mte_thread_switch(struct task_struct *next); void mte_suspend_exit(void); long set_mte_ctrl(unsigned long arg); long get_mte_ctrl(void); +int mte_ptrace_copy_tags(struct task_struct *child, long request, + unsigned long addr, unsigned long data); #else static inline void flush_mte_state(void) { @@ -33,6 +44,12 @@ static inline long get_mte_ctrl(void) { return 0; } +static inline int mte_ptrace_copy_tags(struct task_struct *child, + long request, unsigned long addr, + unsigned long data) +{ + return -EIO; +} #endif #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index 1daf6dda8af0..cd2a4a164de3 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -67,6 +67,9 @@ /* syscall emulation path in ptrace */ #define PTRACE_SYSEMU 31 #define PTRACE_SYSEMU_SINGLESTEP 32 +/* MTE allocation tag access */ +#define PTRACE_PEEKMTETAGS 33 +#define PTRACE_POKEMTETAGS 34 #ifndef __ASSEMBLY__ diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index fa4a4196b248..0cb496ed9bf9 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -3,12 +3,17 @@ * Copyright (C) 2020 ARM Ltd. */ +#include +#include #include #include +#include #include +#include #include #include +#include #include static void update_sctlr_el1_tcf0(u64 tcf0) @@ -133,3 +138,125 @@ long get_mte_ctrl(void) return ret; } + +/* + * Access MTE tags in another process' address space as given in mm. Update + * the number of tags copied. Return 0 if any tags copied, error otherwise. + * Inspired by __access_remote_vm(). + */ +static int __access_remote_tags(struct task_struct *tsk, struct mm_struct *mm, + unsigned long addr, struct iovec *kiov, + unsigned int gup_flags) +{ + struct vm_area_struct *vma; + void __user *buf = kiov->iov_base; + size_t len = kiov->iov_len; + int ret; + int write = gup_flags & FOLL_WRITE; + + if (down_read_killable(&mm->mmap_sem)) + return -EIO; + + if (!access_ok(buf, len)) + return -EFAULT; + + while (len) { + unsigned long tags, offset; + void *maddr; + struct page *page = NULL; + + ret = get_user_pages_remote(tsk, mm, addr, 1, gup_flags, + &page, &vma, NULL); + if (ret <= 0) + break; + + /* limit access to the end of the page */ + offset = offset_in_page(addr); + tags = min(len, (PAGE_SIZE - offset) / MTE_ALLOC_SIZE); + + maddr = page_address(page); + if (write) { + tags = mte_copy_tags_from_user(maddr + offset, buf, tags); + set_page_dirty_lock(page); + } else { + tags = mte_copy_tags_to_user(buf, maddr + offset, tags); + } + put_page(page); + + /* error accessing the tracer's buffer */ + if (!tags) + break; + + len -= tags; + buf += tags; + addr += tags * MTE_ALLOC_SIZE; + } + up_read(&mm->mmap_sem); + + /* return an error if no tags copied */ + kiov->iov_len = buf - kiov->iov_base; + if (!kiov->iov_len) { + /* check for error accessing the tracee's address space */ + if (ret <= 0) + return -EIO; + else + return -EFAULT; + } + + return 0; +} + +/* + * Copy MTE tags in another process' address space at 'addr' to/from tracer's + * iovec buffer. Return 0 on success. Inspired by ptrace_access_vm(). + */ +static int access_remote_tags(struct task_struct *tsk, unsigned long addr, + struct iovec *kiov, unsigned int gup_flags) +{ + struct mm_struct *mm; + int ret; + + mm = get_task_mm(tsk); + if (!mm) + return -EPERM; + + if (!tsk->ptrace || (current != tsk->parent) || + ((get_dumpable(mm) != SUID_DUMP_USER) && + !ptracer_capable(tsk, mm->user_ns))) { + mmput(mm); + return -EPERM; + } + + ret = __access_remote_tags(tsk, mm, addr, kiov, gup_flags); + mmput(mm); + + return ret; +} + +int mte_ptrace_copy_tags(struct task_struct *child, long request, + unsigned long addr, unsigned long data) +{ + int ret; + struct iovec kiov; + struct iovec __user *uiov = (void __user *)data; + unsigned int gup_flags = FOLL_FORCE; + + if (!system_supports_mte()) + return -EIO; + + if (get_user(kiov.iov_base, &uiov->iov_base) || + get_user(kiov.iov_len, &uiov->iov_len)) + return -EFAULT; + + if (request == PTRACE_POKEMTETAGS) + gup_flags |= FOLL_WRITE; + + /* align addr to the MTE tag granule */ + addr &= MTE_ALLOC_MASK; + + ret = access_remote_tags(child, addr, &kiov, gup_flags); + if (!ret) + ret = __put_user(kiov.iov_len, &uiov->iov_len); + + return ret; +} diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 077e352495eb..1fdb841ad536 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -1797,7 +1798,19 @@ const struct user_regset_view *task_user_regset_view(struct task_struct *task) long arch_ptrace(struct task_struct *child, long request, unsigned long addr, unsigned long data) { - return ptrace_request(child, request, addr, data); + int ret; + + switch (request) { + case PTRACE_PEEKMTETAGS: + case PTRACE_POKEMTETAGS: + ret = mte_ptrace_copy_tags(child, request, addr, data); + break; + default: + ret = ptrace_request(child, request, addr, data); + break; + } + + return ret; } enum ptrace_syscall_dir { diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index bd51ea7e2fcb..45be04a8c73c 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -5,6 +5,7 @@ #include #include +#include /* * Compare tags of two pages @@ -44,3 +45,52 @@ SYM_FUNC_START(mte_memcmp_pages) ret SYM_FUNC_END(mte_memcmp_pages) + +/* + * Read tags from a user buffer (one tag per byte) and set the corresponding + * tags at the given kernel address. Used by PTRACE_POKEMTETAGS. + * x0 - kernel address (to) + * x1 - user buffer (from) + * x2 - number of tags/bytes (n) + * Returns: + * x0 - number of tags read/set + */ +SYM_FUNC_START(mte_copy_tags_from_user) + mov x3, x1 +1: +USER(2f, ldtrb w4, [x1]) + lsl x4, x4, #MTE_TAG_SHIFT + stg x4, [x0], #MTE_ALLOC_SIZE + add x1, x1, #1 + subs x2, x2, #1 + b.ne 1b + + // exception handling and function return +2: sub x0, x1, x3 // update the number of tags set + ret +SYM_FUNC_END(mte_copy_tags_from_user) + +/* + * Get the tags from a kernel address range and write the tag values to the + * given user buffer (one tag per byte). Used by PTRACE_PEEKMTETAGS. + * x0 - user buffer (to) + * x1 - kernel address (from) + * x2 - number of tags/bytes (n) + * Returns: + * x0 - number of tags read/set + */ +SYM_FUNC_START(mte_copy_tags_to_user) + mov x3, x0 +1: + ldg x4, [x1] + ubfx x4, x4, #MTE_TAG_SHIFT, #MTE_TAG_SIZE +USER(2f, sttrb w4, [x0]) + add x0, x0, #1 + add x1, x1, #MTE_ALLOC_SIZE + subs x2, x2, #1 + b.ne 1b + + // exception handling and function return +2: sub x0, x0, x3 // update the number of tags copied + ret +SYM_FUNC_END(mte_copy_tags_from_user) From patchwork Tue Apr 21 14:26:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBAB01575 for ; Tue, 21 Apr 2020 14:27:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A3329206B9 for ; Tue, 21 Apr 2020 14:27:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3329206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E47CA8E0019; Tue, 21 Apr 2020 10:26:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DFAC48E0003; Tue, 21 Apr 2020 10:26:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC0F28E0019; Tue, 21 Apr 2020 10:26:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0070.hostedemail.com [216.40.44.70]) by kanga.kvack.org (Postfix) with ESMTP id B4D158E0003 for ; Tue, 21 Apr 2020 10:26:49 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 77527181AEF07 for ; Tue, 21 Apr 2020 14:26:49 +0000 (UTC) X-FDA: 76732088538.09.club71_1e6fc95c75933 X-Spam-Summary: 2,0,0,312af78e3fa3fa66,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:421:541:800:960:966:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1543:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:2901:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:4321:4385:4605:5007:6261:7875:7904:8603:8634:10004:11026:11232:11473:11657:11658:11914:12043:12114:12296:12297:12438:12555:13141:13161:13180:13229:13230:13255:13894:13972:14096:14181:14394:14721:21080:21230:21451:21627:21740:21795:21796:21990:30036:30051:30054:30079,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: club71_1e6fc95c75933 X-Filterd-Recvd-Size: 5090 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:49 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A20A231B; Tue, 21 Apr 2020 07:26:48 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 038733F68F; Tue, 21 Apr 2020 07:26:46 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Alexander Viro Subject: [PATCH v3 20/23] fs: Allow copy_mount_options() to access user-space in a single pass Date: Tue, 21 Apr 2020 15:26:00 +0100 Message-Id: <20200421142603.3894-21-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The copy_mount_options() function takes a user pointer argument but not a size. It tries to read up to a PAGE_SIZE. However, copy_from_user() is not guaranteed to return all the accessible bytes if, for example, the access crosses a page boundary and gets a fault on the second page. To work around this, the current copy_mount_options() implementations performs to copy_from_user() passes, first to the end of the current page and the second to what's left in the subsequent page. Some architectures like arm64 can guarantee an exact copy_from_user() depending on the size (since the arch function performs some alignment on the source register). Introduce an arch_has_exact_copy_from_user() function and allow copy_mount_options() to perform the user access in a single pass. While this function is not on a critical path, the single-pass behaviour is required for arm64 MTE (memory tagging) support where a uaccess can trigger intra-page faults (tag not matching). With the current implementation, if this happens during the first page, the function will return -EFAULT. Signed-off-by: Catalin Marinas Cc: Alexander Viro Cc: Will Deacon Signed-off-by: Catalin Marinas --- Notes: New in v3. arch/arm64/include/asm/uaccess.h | 11 +++++++++++ fs/namespace.c | 7 +++++-- include/linux/uaccess.h | 8 ++++++++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 32fc8061aa76..566da441eba2 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -416,6 +416,17 @@ extern unsigned long __must_check __arch_copy_in_user(void __user *to, const voi #define INLINE_COPY_TO_USER #define INLINE_COPY_FROM_USER +static inline bool arch_has_exact_copy_from_user(unsigned long n) +{ + /* + * copy_from_user() aligns the source pointer if the size is greater + * than 15. Since all the loads are naturally aligned, they can only + * fail on the first byte. + */ + return n > 15; +} +#define arch_has_exact_copy_from_user + extern unsigned long __must_check __arch_clear_user(void __user *to, unsigned long n); static inline unsigned long __must_check __clear_user(void __user *to, unsigned long n) { diff --git a/fs/namespace.c b/fs/namespace.c index a28e4db075ed..8febc50dfc5d 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -3025,13 +3025,16 @@ void *copy_mount_options(const void __user * data) if (!copy) return ERR_PTR(-ENOMEM); - size = PAGE_SIZE - offset_in_page(data); + size = PAGE_SIZE; + if (!arch_has_exact_copy_from_user(size)) + size -= offset_in_page(data); - if (copy_from_user(copy, data, size)) { + if (copy_from_user(copy, data, size) == size) { kfree(copy); return ERR_PTR(-EFAULT); } if (size != PAGE_SIZE) { + WARN_ON(1); if (copy_from_user(copy + size, data + size, PAGE_SIZE - size)) memset(copy + size, 0, PAGE_SIZE - size); } diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 67f016010aad..00e097a9e8d6 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -152,6 +152,14 @@ copy_to_user(void __user *to, const void *from, unsigned long n) n = _copy_to_user(to, from, n); return n; } + +#ifndef arch_has_exact_copy_from_user +static inline bool arch_has_exact_copy_from_user(unsigned long n) +{ + return false; +} +#endif + #ifdef CONFIG_COMPAT static __always_inline unsigned long __must_check copy_in_user(void __user *to, const void __user *from, unsigned long n) From patchwork Tue Apr 21 14:26:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 540281575 for ; Tue, 21 Apr 2020 14:27:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1B5A3206B9 for ; Tue, 21 Apr 2020 14:27:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1B5A3206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 151C18E001A; Tue, 21 Apr 2020 10:26:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0DAA28E0003; Tue, 21 Apr 2020 10:26:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6E488E001A; Tue, 21 Apr 2020 10:26:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id CDF718E0003 for ; Tue, 21 Apr 2020 10:26:51 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 99F1C1F06 for ; Tue, 21 Apr 2020 14:26:51 +0000 (UTC) X-FDA: 76732088622.23.ring44_1ebdc29f9b762 X-Spam-Summary: 30,2,0,bd5d2ca645883178,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:982:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1544:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3165:3355:3865:3866:3867:3868:3870:3871:3872:3874:4117:4250:4321:4605:5007:6119:6261:6742:7901:7904:8634:10010:11026:11232:11233:11473:11657:11658:11914:12043:12291:12294:12296:12297:12438:12555:12895:12986:13180:13229:13894:13972:14181:14394:14721:21063:21080:21230:21451:21627:21966:21990:30054:30056:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: ring44_1ebdc29f9b762 X-Filterd-Recvd-Size: 6143 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C096A11B3; Tue, 21 Apr 2020 07:26:50 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D69753F68F; Tue, 21 Apr 2020 07:26:48 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org, Rob Herring , Mark Rutland , Suzuki K Poulose Subject: [PATCH v3 21/23] arm64: mte: Check the DT memory nodes for MTE support Date: Tue, 21 Apr 2020 15:26:01 +0100 Message-Id: <20200421142603.3894-22-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Even if the ID_AA64PFR1_EL1 register advertises the presence of MTE, it is not guaranteed that the memory system on the SoC supports the feature. In the absence of system-wide MTE support, the behaviour is undefined and the kernel should not enable the MTE memory type in MAIR_EL1. For FDT, add an 'arm,armv8.5-memtag' property to the /memory nodes and check for its presence during MTE probing. For example: memory@80000000 { device_type = "memory"; arm,armv8.5-memtag; reg = <0x00000000 0x80000000 0 0x80000000>, <0x00000008 0x80000000 0 0x80000000>; }; If the /memory nodes are not present in DT or if at least one node does not support MTE, the feature will be disabled. On EFI systems, it is assumed that the memory description matches the EFI memory map (if not, it is considered a firmware bug). MTE is not currently supported on ACPI systems. Signed-off-by: Catalin Marinas Cc: Rob Herring Cc: Mark Rutland Cc: Will Deacon Cc: Suzuki K Poulose Reviewed-by: Suzuki K Poulose --- Notes: New in v3. Ongoing (internal) discussions on whether this is the right approach. The issue needs to be solved similarly for ACPI systems. arch/arm64/boot/dts/arm/fvp-base-revc.dts | 1 + arch/arm64/kernel/cpufeature.c | 51 ++++++++++++++++++++++- 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/arch/arm64/boot/dts/arm/fvp-base-revc.dts b/arch/arm64/boot/dts/arm/fvp-base-revc.dts index 66381d89c1ce..c620a289f15e 100644 --- a/arch/arm64/boot/dts/arm/fvp-base-revc.dts +++ b/arch/arm64/boot/dts/arm/fvp-base-revc.dts @@ -94,6 +94,7 @@ memory@80000000 { device_type = "memory"; + arm,armv8.5-memtag; reg = <0x00000000 0x80000000 0 0x80000000>, <0x00000008 0x80000000 0 0x80000000>; }; diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index d2fe8ff72324..a32aad1d5b57 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -7,6 +7,7 @@ #define pr_fmt(fmt) "CPU features: " fmt +#include #include #include #include @@ -14,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -1412,6 +1414,51 @@ static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry, #endif #ifdef CONFIG_ARM64_MTE +static bool has_usable_mte(const struct arm64_cpu_capabilities *entry, + int scope) +{ + struct device_node *np; + bool memory_checked = false; + bool mte_capable = true; + + if (!has_cpuid_feature(entry, scope)) + return false; + + /* + * If !SCOPE_SYSTEM, return true as per the above CPUID check (late + * CPU bring-up/hotplug). Otherwise, perform addtional checks on the + * system memory MTE support. + */ + if (scope != SCOPE_SYSTEM) + return true; + + if (!acpi_disabled) { + pr_warn("MTE not supported on ACPI systems\n"); + return false; + } + + /* check the "memory" nodes for MTE support */ + for_each_node_by_type(np, "memory") { + memory_checked = true; + mte_capable &= of_property_read_bool(np, "arm,armv8.5-memtag"); + } + + if (!memory_checked || !mte_capable) { + pr_warn("System memory is not MTE-capable\n"); + return false; + } + + return true; +} + +static bool has_hwcap_mte(const struct arm64_cpu_capabilities *entry, + int scope) +{ + if (scope == SCOPE_SYSTEM) + return system_supports_mte(); + return this_cpu_has_cap(ARM64_MTE); +} + static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) { u64 mair; @@ -1828,7 +1875,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .desc = "Memory Tagging Extension", .capability = ARM64_MTE, .type = ARM64_CPUCAP_SYSTEM_FEATURE, - .matches = has_cpuid_feature, + .matches = has_usable_mte, .sys_reg = SYS_ID_AA64PFR1_EL1, .field_pos = ID_AA64PFR1_MTE_SHIFT, .min_field_value = ID_AA64PFR1_MTE, @@ -1950,7 +1997,7 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { HWCAP_MULTI_CAP(ptr_auth_hwcap_gen_matches, CAP_HWCAP, KERNEL_HWCAP_PACG), #endif #ifdef CONFIG_ARM64_MTE - HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_MTE_SHIFT, FTR_UNSIGNED, ID_AA64PFR1_MTE, CAP_HWCAP, KERNEL_HWCAP_MTE), + HWCAP_CAP_MATCH(has_hwcap_mte, CAP_HWCAP, KERNEL_HWCAP_MTE), #endif /* CONFIG_ARM64_MTE */ {}, }; From patchwork Tue Apr 21 14:26:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DCE56913 for ; Tue, 21 Apr 2020 14:27:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B490C206B9 for ; Tue, 21 Apr 2020 14:27:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B490C206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C2B6C8E001B; Tue, 21 Apr 2020 10:26:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BB2B58E0003; Tue, 21 Apr 2020 10:26:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DACA8E001B; Tue, 21 Apr 2020 10:26:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 853648E0003 for ; Tue, 21 Apr 2020 10:26:53 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4F81D2C14 for ; Tue, 21 Apr 2020 14:26:53 +0000 (UTC) X-FDA: 76732088706.25.bears26_1efcf39c35127 X-Spam-Summary: 2,0,0,662d1a4f957934fe,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1042:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3874:4250:5007:6261:7576:7808:8634:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12679:12683:13069:13142:13230:13311:13357:13894:14093:14096:14110:14181:14384:14394:14721:21080:21230:21451:21627:21772:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-64.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: bears26_1efcf39c35127 X-Filterd-Recvd-Size: 3124 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 82A8EC14; Tue, 21 Apr 2020 07:26:52 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 00BE13F68F; Tue, 21 Apr 2020 07:26:50 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 22/23] arm64: mte: Kconfig entry Date: Tue, 21 Apr 2020 15:26:02 +0100 Message-Id: <20200421142603.3894-23-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add Memory Tagging Extension support to the arm64 kbuild. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 40fb05d96c60..af2e6e5dae1b 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1606,6 +1606,38 @@ config ARCH_RANDOM endmenu +menu "ARMv8.5 architectural features" + +config ARM64_AS_HAS_MTE + def_bool $(as-instr,.arch armv8.5-a+memtag) + +config ARM64_MTE + bool "Memory Tagging Extension support" + depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI + select ARCH_USES_HIGH_VMA_FLAGS + select ARCH_NO_SWAP + help + Memory Tagging (part of the ARMv8.5 Extensions) provides + architectural support for run-time, always-on detection of + various classes of memory error to aid with software debugging + to eliminate vulnerabilities arising from memory-unsafe + languages. + + This option enables the support for the Memory Tagging + Extension at EL0 (i.e. for userspace). + + Selecting this option allows the feature to be detected at + runtime. Any secondary CPU not implementing this feature will + not be allowed a late bring-up. + + Userspace binaries that want to use this feature must + explicitly opt in. The mechanism for the userspace is + described in: + + Documentation/arm64/memory-tagging-extension.rst. + +endmenu + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y From patchwork Tue Apr 21 14:26:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11501609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B94A1575 for ; Tue, 21 Apr 2020 14:27:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4BC8C206B9 for ; Tue, 21 Apr 2020 14:27:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BC8C206B9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9DFF98E001C; Tue, 21 Apr 2020 10:26:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 966CC8E0003; Tue, 21 Apr 2020 10:26:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 855578E001C; Tue, 21 Apr 2020 10:26:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 6C5208E0003 for ; Tue, 21 Apr 2020 10:26:55 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 30D81180AD81F for ; Tue, 21 Apr 2020 14:26:55 +0000 (UTC) X-FDA: 76732088790.23.story06_1f3e700f5bd53 X-Spam-Summary: 2,0,0,612a23a9839fccab,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:1:41:69:355:379:541:800:960:966:968:973:982:988:989:1042:1260:1261:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:1801:2196:2199:2393:2553:2559:2562:2639:2693:2895:2901:2904:2918:2924:2926:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4385:4605:5007:6117:6119:6261:7576:7875:7903:7904:8634:8660:8784:9121:11026:11232:11233:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12679:12683:12740:12895:12986:13138:13148:13180:13229:13230:13231:13894:13972:14394:14827:21063:21080:21230:21433:21451:21611:21627:21772:21795:21966:21990:30003:30012:30029:30034:30051:30054:30055:30056:30070:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: story06_1f3e700f5bd53 X-Filterd-Recvd-Size: 14881 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Apr 2020 14:26:54 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4629511D4; Tue, 21 Apr 2020 07:26:54 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B80453F68F; Tue, 21 Apr 2020 07:26:52 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH v3 23/23] arm64: mte: Add Memory Tagging Extension documentation Date: Tue, 21 Apr 2020 15:26:03 +0100 Message-Id: <20200421142603.3894-24-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200421142603.3894-1-catalin.marinas@arm.com> References: <20200421142603.3894-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Memory Tagging Extension (part of the ARMv8.5 Extensions) provides a mechanism to detect the sources of memory related errors which may be vulnerable to exploitation, including bounds violations, use-after-free, use-after-return, use-out-of-scope and use before initialization errors. Add Memory Tagging Extension documentation for the arm64 linux kernel support. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v3: - Modify the uaccess checking conditions: only when the sync mode is selected by the user. In async mode, the kernel uaccesses are not checked. - Clarify that an include mask of 0 (exclude mask 0xffff) results in always generating tag 0. - Document the ptrace() interface. v2: - Documented the uaccess kernel tag checking mode. - Removed the BTI definitions from cpu-feature-registers.rst. - Removed the paragraph stating that MTE depends on the tagged address ABI (while the Kconfig entry does, there is no requirement for the user to enable both). - Changed the GCR_EL1.Exclude handling description following the change in the prctl() interface (include vs exclude mask). - Updated the example code. Documentation/arm64/cpu-feature-registers.rst | 2 + Documentation/arm64/elf_hwcaps.rst | 5 + Documentation/arm64/index.rst | 1 + .../arm64/memory-tagging-extension.rst | 260 ++++++++++++++++++ 4 files changed, 268 insertions(+) create mode 100644 Documentation/arm64/memory-tagging-extension.rst diff --git a/Documentation/arm64/cpu-feature-registers.rst b/Documentation/arm64/cpu-feature-registers.rst index 41937a8091aa..b5679fa85ad9 100644 --- a/Documentation/arm64/cpu-feature-registers.rst +++ b/Documentation/arm64/cpu-feature-registers.rst @@ -174,6 +174,8 @@ infrastructure: +------------------------------+---------+---------+ | Name | bits | visible | +------------------------------+---------+---------+ + | MTE | [11-8] | y | + +------------------------------+---------+---------+ | SSBS | [7-4] | y | +------------------------------+---------+---------+ diff --git a/Documentation/arm64/elf_hwcaps.rst b/Documentation/arm64/elf_hwcaps.rst index 7dfb97dfe416..ca7f90e99e3a 100644 --- a/Documentation/arm64/elf_hwcaps.rst +++ b/Documentation/arm64/elf_hwcaps.rst @@ -236,6 +236,11 @@ HWCAP2_RNG Functionality implied by ID_AA64ISAR0_EL1.RNDR == 0b0001. +HWCAP2_MTE + + Functionality implied by ID_AA64PFR1_EL1.MTE == 0b0010, as described + by Documentation/arm64/memory-tagging-extension.rst. + 4. Unused AT_HWCAP bits ----------------------- diff --git a/Documentation/arm64/index.rst b/Documentation/arm64/index.rst index 09cbb4ed2237..4cd0e696f064 100644 --- a/Documentation/arm64/index.rst +++ b/Documentation/arm64/index.rst @@ -14,6 +14,7 @@ ARM64 Architecture hugetlbpage legacy_instructions memory + memory-tagging-extension pointer-authentication silicon-errata sve diff --git a/Documentation/arm64/memory-tagging-extension.rst b/Documentation/arm64/memory-tagging-extension.rst new file mode 100644 index 000000000000..f82dfbd70061 --- /dev/null +++ b/Documentation/arm64/memory-tagging-extension.rst @@ -0,0 +1,260 @@ +=============================================== +Memory Tagging Extension (MTE) in AArch64 Linux +=============================================== + +Authors: Vincenzo Frascino + Catalin Marinas + +Date: 2020-02-25 + +This document describes the provision of the Memory Tagging Extension +functionality in AArch64 Linux. + +Introduction +============ + +ARMv8.5 based processors introduce the Memory Tagging Extension (MTE) +feature. MTE is built on top of the ARMv8.0 virtual address tagging TBI +(Top Byte Ignore) feature and allows software to access a 4-bit +allocation tag for each 16-byte granule in the physical address space. +Such memory range must be mapped with the Normal-Tagged memory +attribute. A logical tag is derived from bits 59-56 of the virtual +address used for the memory access. A CPU with MTE enabled will compare +the logical tag against the allocation tag and potentially raise an +exception on mismatch, subject to system registers configuration. + +Userspace Support +================= + +When ``CONFIG_ARM64_MTE`` is selected and Memory Tagging Extension is +supported by the hardware, the kernel advertises the feature to +userspace via ``HWCAP2_MTE``. + +PROT_MTE +-------- + +To access the allocation tags, a user process must enable the Tagged +memory attribute on an address range using a new ``prot`` flag for +``mmap()`` and ``mprotect()``: + +``PROT_MTE`` - Pages allow access to the MTE allocation tags. + +The allocation tag is set to 0 when such pages are first mapped in the +user address space and preserved on copy-on-write. ``MAP_SHARED`` is +supported and the allocation tags can be shared between processes. + +**Note**: ``PROT_MTE`` is only supported on ``MAP_ANONYMOUS`` and +RAM-based file mappings (``tmpfs``, ``memfd``). Passing it to other +types of mapping will result in ``-EINVAL`` returned by these system +calls. + +**Note**: The ``PROT_MTE`` flag (and corresponding memory type) cannot +be cleared by ``mprotect()``. + +Tag Check Faults +---------------- + +When ``PROT_MTE`` is enabled on an address range and a mismatch between +the logical and allocation tags occurs on access, there are three +configurable behaviours: + +- *Ignore* - This is the default mode. The CPU (and kernel) ignores the + tag check fault. + +- *Synchronous* - The kernel raises a ``SIGSEGV`` synchronously, with + ``.si_code = SEGV_MTESERR`` and ``.si_addr = ``. The + memory access is not performed. + +- *Asynchronous* - The kernel raises a ``SIGSEGV``, in the current + thread, asynchronously following one or multiple tag check faults, + with ``.si_code = SEGV_MTEAERR`` and ``.si_addr = 0``. + +**Note**: There are no *match-all* logical tags available for user +applications. + +The user can select the above modes, per thread, using the +``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where +``flags`` contain one of the following values in the ``PR_MTE_TCF_MASK`` +bit-field: + +- ``PR_MTE_TCF_NONE`` - *Ignore* tag check faults +- ``PR_MTE_TCF_SYNC`` - *Synchronous* tag check fault mode +- ``PR_MTE_TCF_ASYNC`` - *Asynchronous* tag check fault mode + +Tag checking can also be disabled for a user thread by setting the +``PSTATE.TCO`` bit with ``MSR TCO, #1``. + +**Note**: Signal handlers are always invoked with ``PSTATE.TCO = 0``, +irrespective of the interrupted context. + +**Note**: Kernel accesses to user memory (e.g. ``read()`` system call) +are only checked if the current thread tag checking mode is +PR_MTE_TCF_SYNC. + +Excluding Tags in the ``IRG``, ``ADDG`` and ``SUBG`` instructions +----------------------------------------------------------------- + +The architecture allows excluding certain tags to be randomly generated +via the ``GCR_EL1.Exclude`` register bit-field. By default, Linux +excludes all tags other than 0. A user thread can enable specific tags +in the randomly generated set using the ``prctl(PR_SET_TAGGED_ADDR_CTRL, +flags, 0, 0, 0)`` system call where ``flags`` contains the tags bitmap +in the ``PR_MTE_TAG_MASK`` bit-field. + +**Note**: The hardware uses an exclude mask but the ``prctl()`` +interface provides an include mask. An include mask of ``0`` (exclusion +mask ``0xffff``) results in the CPU always generating tag ``0``. + +The ``ptrace()`` interface +-------------------------- + +``PTRACE_PEEKMTETAGS`` and ``PTRACE_POKEMTETAGS`` allow a tracer to read +the tags from or set the tags to a tracee's address space. The +``ptrace()`` syscall is invoked as ``ptrace(request, pid, addr, data)`` +where: + +- ``request`` - one of ``PTRACE_PEEKMTETAGS`` or ``PTRACE_PEEKMTETAGS``. +- ``pid`` - the tracee's PID. +- ``addr`` - address in the tracee's address space. +- ``data`` - pointer to a ``struct iovec`` where ``iov_base`` points to + a buffer of ``iov_len`` length in the tracer's address space. + +The tags in the tracer's ``iov_base`` buffer are represented as one tag +per byte and correspond to a 16-byte MTE tag granule in the tracee's +address space. + +``ptrace()`` return value: + +- 0 - success, the tracer's ``iov_len`` was updated to the number of + tags copied (it may be smaller than the requested ``iov_len`` if the + requested address range in the tracee's or the tracer's space cannot + be fully accessed). +- ``-EPERM`` - the specified process cannot be traced. +- ``-EIO`` - the tracee's address range cannot be accessed (e.g. invalid + address) and no tags copied. ``iov_len`` not updated. +- ``-EFAULT`` - fault on accessing the tracer's memory (``struct iovec`` + or ``iov_base`` buffer) and no tags copied. ``iov_len`` not updated. + +Example of correct usage +======================== + +*MTE Example code* + +.. code-block:: c + + /* + * To be compiled with -march=armv8.5-a+memtag + */ + #include + #include + #include + #include + #include + #include + #include + + /* + * From arch/arm64/include/uapi/asm/hwcap.h + */ + #define HWCAP2_MTE (1 << 18) + + /* + * From arch/arm64/include/uapi/asm/mman.h + */ + #define PROT_MTE 0x20 + + /* + * From include/uapi/linux/prctl.h + */ + #define PR_SET_TAGGED_ADDR_CTRL 55 + #define PR_GET_TAGGED_ADDR_CTRL 56 + # define PR_TAGGED_ADDR_ENABLE (1UL << 0) + # define PR_MTE_TCF_SHIFT 1 + # define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TAG_SHIFT 3 + # define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT) + + /* + * Insert a random logical tag into the given pointer. + */ + #define insert_random_tag(ptr) ({ \ + __u64 __val; \ + asm("irg %0, %1" : "=r" (__val) : "r" (ptr)); \ + __val; \ + }) + + /* + * Set the allocation tag on the destination address. + */ + #define set_tag(tagged_addr) do { \ + asm volatile("stg %0, [%0]" : : "r" (tagged_addr) : "memory"); \ + } while (0) + + int main() + { + unsigned long *a; + unsigned long page_sz = getpagesize(); + unsigned long hwcap2 = getauxval(AT_HWCAP2); + + /* check if MTE is present */ + if (!(hwcap2 & HWCAP2_MTE)) + return -1; + + /* + * Enable the tagged address ABI, synchronous MTE tag check faults and + * allow all non-zero tags in the randomly generated set. + */ + if (prctl(PR_SET_TAGGED_ADDR_CTRL, + PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | (0xfffe << PR_MTE_TAG_SHIFT), + 0, 0, 0)) { + perror("prctl() failed"); + return -1; + } + + a = mmap(0, page_sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (a == MAP_FAILED) { + perror("mmap() failed"); + return -1; + } + + /* + * Enable MTE on the above anonymous mmap. The flag could be passed + * directly to mmap() and skip this step. + */ + if (mprotect(a, page_sz, PROT_READ | PROT_WRITE | PROT_MTE)) { + perror("mprotect() failed"); + return -1; + } + + /* access with the default tag (0) */ + a[0] = 1; + a[1] = 2; + + printf("a[0] = %lu a[1] = %lu\n", a[0], a[1]); + + /* set the logical and allocation tags */ + a = (unsigned long *)insert_random_tag(a); + set_tag(a); + + printf("%p\n", a); + + /* non-zero tag access */ + a[0] = 3; + printf("a[0] = %lu a[1] = %lu\n", a[0], a[1]); + + /* + * If MTE is enabled correctly the next instruction will generate an + * exception. + */ + printf("Expecting SIGSEGV...\n"); + a[2] = 0xdead; + + /* this should not be printed in the PR_MTE_TCF_SYNC mode */ + printf("...done\n"); + + return 0; + }