From patchwork Wed Dec 11 18:40:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1103B109A for ; Wed, 11 Dec 2019 18:49:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D9EB4206A5 for ; Wed, 11 Dec 2019 18:49:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9EB4206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 662526B3373; Wed, 11 Dec 2019 13:48:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 403956B337C; Wed, 11 Dec 2019 13:48:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 360B76B336C; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 2D0FF6B337A for ; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id E493D181AEF1D for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.02.team29_134c8d1cd9910 X-Spam-Summary: 2,0,0,f0ef75cdc73f4dcb,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org:dave.martin@arm.com:arnd@arndb.de,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1541:1711:1730:1747:1777:1792:1978:1981:2194:2199:2393:2553:2559:2562:2693:2918:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3874:4250:4321:5007:6261:6742:7576:7903:8634:10004:11658:11914:12043:12257:12296:12297:12555:12679:12986:13069:13149:13161:13229:13230:13311:13357:13894:14096:14181:14384:14394:14721:21080:21230:21451:21524:21627:21990:30054:30070:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0: 0:0,LFti X-HE-Tag: team29_134c8d1cd9910 X-Filterd-Recvd-Size: 2882 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 104B7328; Wed, 11 Dec 2019 10:40:39 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 662103F6CF; Wed, 11 Dec 2019 10:40:37 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org, Dave Martin , Arnd Bergmann Subject: [PATCH 01/22] mm: Reserve asm-generic prot flags 0x10 and 0x20 for arch use Date: Wed, 11 Dec 2019 18:40:06 +0000 Message-Id: <20191211184027.20130-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Martin The asm-generic/mman.h definitions are used by a few architectures that also define arch-specific PROT flags with value 0x10 and 0x20. This currently applies to sparc and powerpc for 0x10, while arm64 will soon join with 0x10 and 0x20. To help future maintainers, document the use of this flag in the asm-generic header too. Cc: Arnd Bergmann Signed-off-by: Dave Martin [catalin.marinas@arm.com: reserve 0x20 as well] Signed-off-by: Catalin Marinas Acked-by: Arnd Bergmann --- include/uapi/asm-generic/mman-common.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h index c160a5354eb6..f94f65d429be 100644 --- a/include/uapi/asm-generic/mman-common.h +++ b/include/uapi/asm-generic/mman-common.h @@ -11,6 +11,8 @@ #define PROT_WRITE 0x2 /* page can be written */ #define PROT_EXEC 0x4 /* page can be executed */ #define PROT_SEM 0x8 /* page may be used for atomic ops */ +/* 0x10 reserved for arch-specific use */ +/* 0x20 reserved for arch-specific use */ #define PROT_NONE 0x0 /* page can not be accessed */ #define PROT_GROWSDOWN 0x01000000 /* mprotect flag: extend change to start of growsdown vma */ #define PROT_GROWSUP 0x02000000 /* mprotect flag: extend change to end of growsup vma */ From patchwork Wed Dec 11 18:40:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 14016109A for ; Wed, 11 Dec 2019 18:49:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DC46C206A5 for ; Wed, 11 Dec 2019 18:49:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DC46C206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C3D8A6B3376; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B7E896B3378; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0EF486B3377; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id 84A766B336D for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 46C4482499A8 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.02.waves05_132e7a2d27462 X-Spam-Summary: 2,0,0,920eb07fbbeb6fdf,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org:yamada.masahiro@socionext.com:linux-kbuild@vger.kernel.org:vladimir.murzin@arm.com,RULES_HIT:41:355:379:541:800:960:973:982:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2899:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3872:3874:4250:4321:5007:6261:6742:7904:8634:8660:10004:11026:11473:11658:11914:12043:12297:12555:13069:13148:13230:13311:13357:13894:14181:14384:14394:14721:21080:21221:21230:21451:21627:21990:30012:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:31,LUA_SUMMARY:none X-HE-Tag: waves05_132e7a2d27462 X-Filterd-Recvd-Size: 2930 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 091551045; Wed, 11 Dec 2019 10:40:41 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4461F3F6CF; Wed, 11 Dec 2019 10:40:39 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org, Masahiro Yamada , linux-kbuild@vger.kernel.org, Vladimir Murzin Subject: [PATCH 02/22] kbuild: Add support for 'as-instr' to be used in Kconfig files Date: Wed, 11 Dec 2019 18:40:07 +0000 Message-Id: <20191211184027.20130-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similar to 'cc-option' or 'ld-option', it is occasionally necessary to check whether the assembler supports certain ISA extensions. In the arm64 code we currently do this in Makefile with an additional define: lseinstr := $(call as-instr,.arch_extension lse,-DCONFIG_AS_LSE=1) Add the 'as-instr' option so that it can be used in Kconfig directly: def_bool $(as-instr,.arch_extension lse) Cc: Masahiro Yamada Cc: linux-kbuild@vger.kernel.org Reviewed-by: Vladimir Murzin Signed-off-by: Catalin Marinas Acked-by: Masahiro Yamada --- scripts/Kconfig.include | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/Kconfig.include b/scripts/Kconfig.include index d4adfbe42690..9d07e59cbdf7 100644 --- a/scripts/Kconfig.include +++ b/scripts/Kconfig.include @@ -31,6 +31,10 @@ cc-option = $(success,$(CC) -Werror $(CLANG_FLAGS) $(1) -E -x c /dev/null -o /de # Return y if the linker supports , n otherwise ld-option = $(success,$(LD) -v $(1)) +# $(as-instr,) +# Return y if the assembler supports , n otherwise +as-instr = $(success,printf "%b\n" "$(1)" | $(CC) $(CLANG_FLAGS) -c -x assembler -o /dev/null -) + # check if $(CC) and $(LD) exist $(error-if,$(failure,command -v $(CC)),compiler '$(CC)' not found) $(error-if,$(failure,command -v $(LD)),linker '$(LD)' not found) From patchwork Wed Dec 11 18:40:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286031 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03CD1138D for ; Wed, 11 Dec 2019 18:49:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C6F6F206A5 for ; Wed, 11 Dec 2019 18:49:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C6F6F206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1920E6B3369; Wed, 11 Dec 2019 13:48:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DC5056B3375; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E0C96B336A; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0093.hostedemail.com [216.40.44.93]) by kanga.kvack.org (Postfix) with ESMTP id A4E396B336A for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 71F8082499B9 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.11.books58_13302ebd9bc0b X-Spam-Summary: 2,0,0,72982656636a6e5d,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1500:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3867:3870:4321:5007:6261:8634:10004:11026:11657:11658:11914:12043:12296:12297:12438:12555:12663:13069:13161:13229:13255:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: books58_13302ebd9bc0b X-Filterd-Recvd-Size: 2823 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A5DE5106F; Wed, 11 Dec 2019 10:40:42 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3CDB93F6CF; Wed, 11 Dec 2019 10:40:41 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 03/22] arm64: alternative: Allow alternative_insn to always issue the first instruction Date: Wed, 11 Dec 2019 18:40:08 +0000 Message-Id: <20191211184027.20130-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are situations where we do not want to disable the whole block based on a config option, only the alternative part while keeping the first instruction. Improve the alternative_insn assembler macro to take a 'first_insn' argument, default 0 to preserve the current behaviour. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/alternative.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index b9f8d787eea9..b4d3ffe16ca6 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -101,7 +101,11 @@ static inline void apply_alternatives_module(void *start, size_t length) { } .byte \alt_len .endm -.macro alternative_insn insn1, insn2, cap, enable = 1 +/* + * Disable the whole block if enable == 0, unless first_insn == 1 in which + * case insn1 will always be issued but without an alternative insn2. + */ +.macro alternative_insn insn1, insn2, cap, enable = 1, first_insn = 0 .if \enable 661: \insn1 662: .pushsection .altinstructions, "a" @@ -112,6 +116,8 @@ static inline void apply_alternatives_module(void *start, size_t length) { } 664: .popsection .org . - (664b-663b) + (662b-661b) .org . - (662b-661b) + (664b-663b) + .elseif \first_insn + \insn1 .endif .endm From patchwork Wed Dec 11 18:40:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286009 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8776109A for ; Wed, 11 Dec 2019 18:48:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0C2F206A5 for ; Wed, 11 Dec 2019 18:48:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0C2F206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0D4256B3363; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E85AE6B336B; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4E0A6B3365; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 821246B3365 for ; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 3C515824999B for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) X-FDA: 76253747262.25.bun08_13050cd98602e X-Spam-Summary: 2,0,0,dd24708aa8394b52,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2559:2562:2901:3138:3139:3140:3141:3142:3354:3622:3865:3866:3868:3870:3871:3874:4250:4321:4605:5007:6119:6261:7903:7904:8634:8957:10004:11026:11473:11657:11658:11914:12043:12297:12438:12555:12986:13894:13972:14181:14394:14721:21080:21230:21451:21627:21740:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: bun08_13050cd98602e X-Filterd-Recvd-Size: 4177 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4E5C5113E; Wed, 11 Dec 2019 10:40:44 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D9A5D3F6CF; Wed, 11 Dec 2019 10:40:42 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 04/22] arm64: Use macros instead of hard-coded constants for MAIR_EL1 Date: Wed, 11 Dec 2019 18:40:09 +0000 Message-Id: <20191211184027.20130-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, the arm64 __cpu_setup has hard-coded constants for the memory attributes that go into the MAIR_EL1 register. Define proper macros in asm/sysreg.h and make use of them in proc.S. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/sysreg.h | 12 ++++++++++++ arch/arm64/mm/proc.S | 27 ++++++++++----------------- 2 files changed, 22 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 6e919fafb43d..e21470337c5e 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -538,6 +538,18 @@ SCTLR_EL1_NTWE | SCTLR_ELx_IESB | SCTLR_EL1_SPAN |\ ENDIAN_SET_EL1 | SCTLR_EL1_UCI | SCTLR_EL1_RES1) +/* MAIR_ELx memory attributes (used by Linux) */ +#define MAIR_ATTR_DEVICE_nGnRnE UL(0x00) +#define MAIR_ATTR_DEVICE_nGnRE UL(0x04) +#define MAIR_ATTR_DEVICE_GRE UL(0x0c) +#define MAIR_ATTR_NORMAL_NC UL(0x44) +#define MAIR_ATTR_NORMAL_WT UL(0xbb) +#define MAIR_ATTR_NORMAL UL(0xff) +#define MAIR_ATTR_MASK UL(0xff) + +/* Position the attr at the correct index */ +#define MAIR_ATTRIDX(attr, idx) ((attr) << ((idx) * 8)) + /* id_aa64isar0 */ #define ID_AA64ISAR0_TS_SHIFT 52 #define ID_AA64ISAR0_FHM_SHIFT 48 diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index a1e0592d1fbc..55f715957b36 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -42,7 +42,14 @@ #define TCR_KASAN_FLAGS 0 #endif -#define MAIR(attr, mt) ((attr) << ((mt) * 8)) +/* Default MAIR_EL1 */ +#define MAIR_EL1_SET \ + (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRE, MT_DEVICE_nGnRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT)) #ifdef CONFIG_CPU_PM /** @@ -416,23 +423,9 @@ ENTRY(__cpu_setup) enable_dbg // since this is per-cpu reset_pmuserenr_el0 x0 // Disable PMU access from EL0 /* - * Memory region attributes for LPAE: - * - * n = AttrIndx[2:0] - * n MAIR - * DEVICE_nGnRnE 000 00000000 - * DEVICE_nGnRE 001 00000100 - * DEVICE_GRE 010 00001100 - * NORMAL_NC 011 01000100 - * NORMAL 100 11111111 - * NORMAL_WT 101 10111011 + * Memory region attributes */ - ldr x5, =MAIR(0x00, MT_DEVICE_nGnRnE) | \ - MAIR(0x04, MT_DEVICE_nGnRE) | \ - MAIR(0x0c, MT_DEVICE_GRE) | \ - MAIR(0x44, MT_NORMAL_NC) | \ - MAIR(0xff, MT_NORMAL) | \ - MAIR(0xbb, MT_NORMAL_WT) + mov_q x5, MAIR_EL1_SET msr mair_el1, x5 /* * Prepare SCTLR From patchwork Wed Dec 11 18:40:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286049 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B73EF138D for ; Wed, 11 Dec 2019 18:49:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 82385206A5 for ; Wed, 11 Dec 2019 18:49:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 82385206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 824506B3375; Wed, 11 Dec 2019 13:48:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5D6E06B337B; Wed, 11 Dec 2019 13:48:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6210B6B337A; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 43EB46B337D for ; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 01D7F180AD80F for ; Wed, 11 Dec 2019 18:48:53 +0000 (UTC) X-FDA: 76253747346.27.class91_134e0817a2045 X-Spam-Summary: 2,0,0,4ad56d21dadb6ee6,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2901:3138:3139:3140:3141:3142:3865:3871:4049:4119:4250:4321:4605:4648:5007:6119:6261:7576:8634:10004:11026:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12679:12986:13138:13161:13229:13231:13894:13972:14394:21080:21230:21451:21627:21990:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: class91_134e0817a2045 X-Filterd-Recvd-Size: 8017 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EB3ED11B3; Wed, 11 Dec 2019 10:40:45 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 82B0E3F6CF; Wed, 11 Dec 2019 10:40:44 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 05/22] arm64: mte: system register definitions Date: Wed, 11 Dec 2019 18:40:10 +0000 Message-Id: <20191211184027.20130-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add Memory Tagging Extension system register definitions together with the relevant bitfields. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/include/asm/sysreg.h | 50 ++++++++++++++++++++++++++++ arch/arm64/include/uapi/asm/ptrace.h | 1 + arch/arm64/kernel/ptrace.c | 2 +- 4 files changed, 53 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 6e5d839f42b5..0b25f9a81c57 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -12,6 +12,7 @@ #include /* Hyp Configuration Register (HCR) bits */ +#define HCR_ATA (UL(1) << 56) #define HCR_FWB (UL(1) << 46) #define HCR_API (UL(1) << 41) #define HCR_APK (UL(1) << 40) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index e21470337c5e..609d1b3238dd 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -171,6 +171,8 @@ #define SYS_SCTLR_EL1 sys_reg(3, 0, 1, 0, 0) #define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1) #define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2) +#define SYS_RGSR_EL1 sys_reg(3, 0, 1, 0, 5) +#define SYS_GCR_EL1 sys_reg(3, 0, 1, 0, 6) #define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0) @@ -208,6 +210,8 @@ #define SYS_ERXADDR_EL1 sys_reg(3, 0, 5, 4, 3) #define SYS_ERXMISC0_EL1 sys_reg(3, 0, 5, 5, 0) #define SYS_ERXMISC1_EL1 sys_reg(3, 0, 5, 5, 1) +#define SYS_TFSR_EL1 sys_reg(3, 0, 5, 6, 0) +#define SYS_TFSRE0_EL1 sys_reg(3, 0, 5, 6, 1) #define SYS_FAR_EL1 sys_reg(3, 0, 6, 0, 0) #define SYS_PAR_EL1 sys_reg(3, 0, 7, 4, 0) @@ -358,6 +362,7 @@ #define SYS_CCSIDR_EL1 sys_reg(3, 1, 0, 0, 0) #define SYS_CLIDR_EL1 sys_reg(3, 1, 0, 0, 1) +#define SYS_GMID_EL1 sys_reg(3, 1, 0, 0, 4) #define SYS_AIDR_EL1 sys_reg(3, 1, 0, 0, 7) #define SYS_CSSELR_EL1 sys_reg(3, 2, 0, 0, 0) @@ -411,6 +416,7 @@ #define SYS_ESR_EL2 sys_reg(3, 4, 5, 2, 0) #define SYS_VSESR_EL2 sys_reg(3, 4, 5, 2, 3) #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0) +#define SYS_TFSR_EL2 sys_reg(3, 4, 5, 6, 0) #define SYS_FAR_EL2 sys_reg(3, 4, 6, 0, 0) #define SYS_VDISR_EL2 sys_reg(3, 4, 12, 1, 1) @@ -467,6 +473,7 @@ #define SYS_AFSR0_EL12 sys_reg(3, 5, 5, 1, 0) #define SYS_AFSR1_EL12 sys_reg(3, 5, 5, 1, 1) #define SYS_ESR_EL12 sys_reg(3, 5, 5, 2, 0) +#define SYS_TFSR_EL12 sys_reg(3, 5, 5, 6, 0) #define SYS_FAR_EL12 sys_reg(3, 5, 6, 0, 0) #define SYS_MAIR_EL12 sys_reg(3, 5, 10, 2, 0) #define SYS_AMAIR_EL12 sys_reg(3, 5, 10, 3, 0) @@ -482,6 +489,14 @@ /* Common SCTLR_ELx flags. */ #define SCTLR_ELx_DSSBS (BIT(44)) +#define SCTLR_ELx_ATA (BIT(43)) + +#define SCTLR_ELx_TCF_SHIFT 40 +#define SCTLR_ELx_TCF_SYNC (UL(0x1) << SCTLR_ELx_TCF_SHIFT) +#define SCTLR_ELx_TCF_ASYNC (UL(0x2) << SCTLR_ELx_TCF_SHIFT) +#define SCTLR_ELx_TCF_MASK (UL(0x3) << SCTLR_ELx_TCF_SHIFT) + +#define SCTLR_ELx_ITFSB (BIT(37)) #define SCTLR_ELx_ENIA (BIT(31)) #define SCTLR_ELx_ENIB (BIT(30)) #define SCTLR_ELx_ENDA (BIT(27)) @@ -510,6 +525,13 @@ #endif /* SCTLR_EL1 specific flags. */ +#define SCTLR_EL1_ATA0 (BIT(42)) + +#define SCTLR_EL1_TCF0_SHIFT 38 +#define SCTLR_EL1_TCF0_SYNC (UL(0x1) << SCTLR_EL1_TCF0_SHIFT) +#define SCTLR_EL1_TCF0_ASYNC (UL(0x2) << SCTLR_EL1_TCF0_SHIFT) +#define SCTLR_EL1_TCF0_MASK (UL(0x3) << SCTLR_EL1_TCF0_SHIFT) + #define SCTLR_EL1_UCI (BIT(26)) #define SCTLR_EL1_E0E (BIT(24)) #define SCTLR_EL1_SPAN (BIT(23)) @@ -544,6 +566,7 @@ #define MAIR_ATTR_DEVICE_GRE UL(0x0c) #define MAIR_ATTR_NORMAL_NC UL(0x44) #define MAIR_ATTR_NORMAL_WT UL(0xbb) +#define MAIR_ATTR_NORMAL_TAGGED UL(0xf0) #define MAIR_ATTR_NORMAL UL(0xff) #define MAIR_ATTR_MASK UL(0xff) @@ -611,11 +634,16 @@ /* id_aa64pfr1 */ #define ID_AA64PFR1_SSBS_SHIFT 4 +#define ID_AA64PFR1_MTE_SHIFT 8 #define ID_AA64PFR1_SSBS_PSTATE_NI 0 #define ID_AA64PFR1_SSBS_PSTATE_ONLY 1 #define ID_AA64PFR1_SSBS_PSTATE_INSNS 2 +#define ID_AA64PFR1_MTE_NI 0x0 +#define ID_AA64PFR1_MTE_EL0 0x1 +#define ID_AA64PFR1_MTE 0x2 + /* id_aa64zfr0 */ #define ID_AA64ZFR0_SM4_SHIFT 40 #define ID_AA64ZFR0_SHA3_SHIFT 32 @@ -746,6 +774,28 @@ #define CPACR_EL1_ZEN_EL0EN (BIT(17)) /* enable EL0 access, if EL1EN set */ #define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN) +/* TCR EL1 Bit Definitions */ +#define SYS_TCR_EL1_TCMA1 (BIT(58)) +#define SYS_TCR_EL1_TCMA0 (BIT(57)) + +/* GCR_EL1 Definitions */ +#define SYS_GCR_EL1_RRND (BIT(16)) +#define SYS_GCR_EL1_EXCL_MASK 0xffffUL + +/* RGSR_EL1 Definitions */ +#define SYS_RGSR_EL1_TAG_MASK 0xfUL +#define SYS_RGSR_EL1_SEED_SHIFT 8 +#define SYS_RGSR_EL1_SEED_MASK 0xffffUL + +/* GMID_EL1 field definitions */ +#define SYS_GMID_EL1_BS_SHIFT 0 +#define SYS_GMID_EL1_BS_SIZE 4 + +/* TFSR{,E0}_EL1 bit definitions */ +#define SYS_TFSR_EL1_TF0_SHIFT 0 +#define SYS_TFSR_EL1_TF1_SHIFT 1 +#define SYS_TFSR_EL1_TF0 (UL(1) << SYS_TFSR_EL1_TF0_SHIFT) +#define SYS_TFSR_EL1_TF1 (UK(2) << SYS_TFSR_EL1_TF1_SHIFT) /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */ #define SYS_MPIDR_SAFE_VAL (BIT(31)) diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index 7ed9294e2004..0ae569ec7960 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -49,6 +49,7 @@ #define PSR_SSBS_BIT 0x00001000 #define PSR_PAN_BIT 0x00400000 #define PSR_UAO_BIT 0x00800000 +#define PSR_TCO_BIT 0x02000000 #define PSR_V_BIT 0x10000000 #define PSR_C_BIT 0x20000000 #define PSR_Z_BIT 0x40000000 diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 6771c399d40c..05dcb4853b8d 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -1852,7 +1852,7 @@ void syscall_trace_exit(struct pt_regs *regs) * We also reserve IL for the kernel; SS is handled dynamically. */ #define SPSR_EL1_AARCH64_RES0_BITS \ - (GENMASK_ULL(63, 32) | GENMASK_ULL(27, 25) | GENMASK_ULL(23, 22) | \ + (GENMASK_ULL(63, 32) | GENMASK_ULL(27, 26) | GENMASK_ULL(23, 22) | \ GENMASK_ULL(20, 13) | GENMASK_ULL(11, 10) | GENMASK_ULL(5, 5)) #define SPSR_EL1_AARCH32_RES0_BITS \ (GENMASK_ULL(63, 32) | GENMASK_ULL(22, 22) | GENMASK_ULL(20, 20)) From patchwork Wed Dec 11 18:40:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286011 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03F93138D for ; Wed, 11 Dec 2019 18:49:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C2E4D206A5 for ; Wed, 11 Dec 2019 18:48:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2E4D206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2FCEF6B3364; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 09FE86B3369; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD4536B3367; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 8E32D6B3368 for ; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 49FE04DC0 for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) X-FDA: 76253747262.05.maid74_130444b8e2f3f X-Spam-Summary: 2,0,0,33c5ecfe77f4aa3f,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3874:4049:4119:4250:4321:4605:5007:6261:7576:7903:8634:10004:11026:11473:11657:11658:11914:12043:12291:12297:12438:12555:12679:12986:13161:13229:13255:13894:13972:14093:14394:21080:21230:21451:21627:21740:21795:21990:30051:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: maid74_130444b8e2f3f X-Filterd-Recvd-Size: 8186 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 93EC311FB; Wed, 11 Dec 2019 10:40:47 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2B6E03F6CF; Wed, 11 Dec 2019 10:40:46 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 06/22] arm64: mte: CPU feature detection and initial sysreg configuration Date: Wed, 11 Dec 2019 18:40:11 +0000 Message-Id: <20191211184027.20130-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add the cpufeature and hwcap entries to detect the presence of MTE on the boot CPUs (primary and secondary). Any late secondary CPU not supporting the feature, if detected during boot, will be parked. In addition, add the minimum SCTLR_EL1 and HCR_EL2 bits for enabling MTE. Without subsequent setting of MAIR, these bits do not have an effect on tag checking. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/cpucaps.h | 5 ++++- arch/arm64/include/asm/cpufeature.h | 6 ++++++ arch/arm64/include/asm/hwcap.h | 1 + arch/arm64/include/asm/kvm_arm.h | 2 +- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/hwcap.h | 2 ++ arch/arm64/kernel/cpufeature.c | 29 +++++++++++++++++++++++++++++ arch/arm64/kernel/cpuinfo.c | 2 ++ 8 files changed, 46 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index b92683871119..8a56604ec04e 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -56,7 +56,10 @@ #define ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM 46 #define ARM64_WORKAROUND_1542419 47 #define ARM64_WORKAROUND_1319367 48 +/* 49 reserved for ARM64_HAS_E0PD */ +/* 50 reserved for ARM64_BTI */ +#define ARM64_MTE 51 -#define ARM64_NCAPS 49 +#define ARM64_NCAPS 52 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 4261d55e8506..7d464797eff6 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -607,6 +607,12 @@ static inline bool system_uses_irq_prio_masking(void) cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING); } +static inline bool system_supports_mte(void) +{ + return IS_ENABLED(CONFIG_ARM64_MTE) && + cpus_have_const_cap(ARM64_MTE); +} + static inline bool system_has_prio_mask_debugging(void) { return IS_ENABLED(CONFIG_ARM64_DEBUG_PRIORITY_MASKING) && diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h index 3d2f2472a36c..a98b3347c4a4 100644 --- a/arch/arm64/include/asm/hwcap.h +++ b/arch/arm64/include/asm/hwcap.h @@ -86,6 +86,7 @@ #define KERNEL_HWCAP_SVESM4 __khwcap2_feature(SVESM4) #define KERNEL_HWCAP_FLAGM2 __khwcap2_feature(FLAGM2) #define KERNEL_HWCAP_FRINT __khwcap2_feature(FRINT) +#define KERNEL_HWCAP_MTE __khwcap2_feature(MTE) /* * This yields a mask that user programs can use to figure out what diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 0b25f9a81c57..37bcb93b376c 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -78,7 +78,7 @@ HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \ HCR_FMO | HCR_IMO) #define HCR_VIRT_EXCP_MASK (HCR_VSE | HCR_VI | HCR_VF) -#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK) +#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK | HCR_ATA) #define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H) /* TCR_EL2 Registers bits */ diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 609d1b3238dd..9e5753272f4b 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -558,6 +558,7 @@ SCTLR_EL1_SA0 | SCTLR_EL1_SED | SCTLR_ELx_I |\ SCTLR_EL1_DZE | SCTLR_EL1_UCT |\ SCTLR_EL1_NTWE | SCTLR_ELx_IESB | SCTLR_EL1_SPAN |\ + SCTLR_ELx_ITFSB| SCTLR_ELx_ATA | SCTLR_EL1_ATA0 |\ ENDIAN_SET_EL1 | SCTLR_EL1_UCI | SCTLR_EL1_RES1) /* MAIR_ELx memory attributes (used by Linux) */ diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h index a1e72886b30c..57a29c9b29d9 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -65,5 +65,7 @@ #define HWCAP2_SVESM4 (1 << 6) #define HWCAP2_FLAGM2 (1 << 7) #define HWCAP2_FRINT (1 << 8) +/* bit 9 reserved for HWCAP2_BTI */ +#define HWCAP2_MTE (1 << 10) #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 04cf64e9f0c9..a3eea2cce6b0 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -172,6 +172,8 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = { ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_SSBS_SHIFT, 4, ID_AA64PFR1_SSBS_PSTATE_NI), + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_MTE), + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR1_MTE_SHIFT, 4, ID_AA64PFR1_MTE_NI), ARM64_FTR_END, }; @@ -1267,6 +1269,17 @@ static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry, } #endif +#ifdef CONFIG_ARM64_MTE +static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) +{ + write_sysreg_s(SYS_GCR_EL1_RRND, SYS_GCR_EL1); + write_sysreg_s(0, SYS_TFSR_EL1); + write_sysreg_s(0, SYS_TFSRE0_EL1); + + isb(); +} +#endif /* CONFIG_ARM64_MTE */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -1567,6 +1580,19 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .min_field_value = 1, }, #endif +#ifdef CONFIG_ARM64_MTE + { + .desc = "Memory Tagging Extension", + .capability = ARM64_MTE, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, + .sys_reg = SYS_ID_AA64PFR1_EL1, + .field_pos = ID_AA64PFR1_MTE_SHIFT, + .min_field_value = ID_AA64PFR1_MTE, + .sign = FTR_UNSIGNED, + .cpu_enable = cpu_enable_mte, + }, +#endif /* CONFIG_ARM64_MTE */ {}, }; @@ -1666,6 +1692,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { HWCAP_MULTI_CAP(ptr_auth_hwcap_addr_matches, CAP_HWCAP, KERNEL_HWCAP_PACA), HWCAP_MULTI_CAP(ptr_auth_hwcap_gen_matches, CAP_HWCAP, KERNEL_HWCAP_PACG), #endif +#ifdef CONFIG_ARM64_MTE + HWCAP_CAP(SYS_ID_AA64PFR1_EL1, ID_AA64PFR1_MTE_SHIFT, FTR_UNSIGNED, ID_AA64PFR1_MTE, CAP_HWCAP, KERNEL_HWCAP_MTE), +#endif /* CONFIG_ARM64_MTE */ {}, }; diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index 56bba746da1c..6f686e64610e 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -84,6 +84,8 @@ static const char *const hwcap_str[] = { "svesm4", "flagm2", "frint", + "", /* reserved for BTI */ + "mte", NULL }; From patchwork Wed Dec 11 18:40:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286043 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1EC1138D for ; Wed, 11 Dec 2019 18:49:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8D797206A5 for ; Wed, 11 Dec 2019 18:49:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D797206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2256B6B3377; Wed, 11 Dec 2019 13:48:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E62756B3375; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F20826B3378; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 2A9156B3369 for ; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id E742E4408 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.09.plot05_134aabdc42d24 X-Spam-Summary: 2,0,0,b46ccbd33cd3c27f,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:2:41:355:379:541:800:960:973:981:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2553:2559:2562:2895:2898:2901:2903:2924:2926:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4050:4119:4250:4321:4605:5007:6119:6261:8634:8957:10004:11026:11232:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12986:13161:13229:13894:14394:21063:21080:21230:21451:21627:21795:21966:21990:30051:30054:30055:30069:30070:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1, LUA_SUMM X-HE-Tag: plot05_134aabdc42d24 X-Filterd-Recvd-Size: 8720 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C69212FC; Wed, 11 Dec 2019 10:40:49 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C80933F6CF; Wed, 11 Dec 2019 10:40:47 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 07/22] arm64: mte: Use Normal Tagged attributes for the linear map Date: Wed, 11 Dec 2019 18:40:12 +0000 Message-Id: <20191211184027.20130-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Once user space is given access to tagged memory, the kernel must be able to clear/save/restore tags visible to the user. This is done via the linear mapping, therefore map it as such. The new MT_NORMAL_TAGGED index for MAIR_EL1 is initially mapped as Normal memory and later changed to Normal Tagged via the cpufeature infrastructure. From a mismatched attribute aliases perspective, the Tagged memory is considered a permission and it won't lead to undefined behaviour. The empty_zero_page is cleared to ensure that the tags it contains are already zeroed. The actual tags-aware clear_page() implementation is part of a subsequent patch. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/memory.h | 1 + arch/arm64/include/asm/pgtable-prot.h | 2 ++ arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++++++++++++++ arch/arm64/mm/dump.c | 4 ++++ arch/arm64/mm/mmu.c | 22 ++++++++++++++++++-- arch/arm64/mm/proc.S | 8 +++++-- 6 files changed, 63 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index a4f9ca5479b0..55994ab362ae 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -145,6 +145,7 @@ #define MT_NORMAL_NC 3 #define MT_NORMAL 4 #define MT_NORMAL_WT 5 +#define MT_NORMAL_TAGGED 6 /* * Memory types for Stage-2 translation diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 8dc6c5cdabe6..ef1e565c3a79 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -37,6 +37,7 @@ #define PROT_NORMAL_NC (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_NC)) #define PROT_NORMAL_WT (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_WT)) #define PROT_NORMAL (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL)) +#define PROT_NORMAL_TAGGED (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_TAGGED)) #define PROT_SECT_DEVICE_nGnRE (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_DEVICE_nGnRE)) #define PROT_SECT_NORMAL (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL)) @@ -46,6 +47,7 @@ #define _HYP_PAGE_DEFAULT _PAGE_DEFAULT #define PAGE_KERNEL __pgprot(PROT_NORMAL) +#define PAGE_KERNEL_TAGGED __pgprot(PROT_NORMAL_TAGGED) #define PAGE_KERNEL_RO __pgprot((PROT_NORMAL & ~PTE_WRITE) | PTE_RDONLY) #define PAGE_KERNEL_ROX __pgprot((PROT_NORMAL & ~(PTE_WRITE | PTE_PXN)) | PTE_RDONLY) #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index a3eea2cce6b0..06f3f6677284 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1272,12 +1272,42 @@ static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry, #ifdef CONFIG_ARM64_MTE static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) { + u64 mair; + write_sysreg_s(SYS_GCR_EL1_RRND, SYS_GCR_EL1); write_sysreg_s(0, SYS_TFSR_EL1); write_sysreg_s(0, SYS_TFSRE0_EL1); + /* + * Update the MT_NORMAL_TAGGED index in MAIR_EL1. Tag checking is + * disabled for the kernel, so there won't be any observable effect + * other than allowing the kernel to read and write tags. + */ + mair = read_sysreg_s(SYS_MAIR_EL1); + mair &= ~MAIR_ATTRIDX(MAIR_ATTR_MASK, MT_NORMAL_TAGGED); + mair |= MAIR_ATTRIDX(MAIR_ATTR_NORMAL_TAGGED, MT_NORMAL_TAGGED); + write_sysreg_s(mair, SYS_MAIR_EL1); + isb(); } + +static int __init system_enable_mte(void) +{ + if (!system_supports_mte()) + return 0; + + /* Ensure the TLB does not have stale MAIR attributes */ + flush_tlb_all(); + + /* + * Clear the zero page (again) so that tags are reset. This needs to + * be done via the linear map which has the Tagged attribute. + */ + clear_page(lm_alias(empty_zero_page)); + + return 0; +} +core_initcall(system_enable_mte); #endif /* CONFIG_ARM64_MTE */ static const struct arm64_cpu_capabilities arm64_features[] = { diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c index 0a920b538a89..1f75a71e63f2 100644 --- a/arch/arm64/mm/dump.c +++ b/arch/arm64/mm/dump.c @@ -163,6 +163,10 @@ static const struct prot_bits pte_bits[] = { .mask = PTE_ATTRINDX_MASK, .val = PTE_ATTRINDX(MT_NORMAL), .set = "MEM/NORMAL", + }, { + .mask = PTE_ATTRINDX_MASK, + .val = PTE_ATTRINDX(MT_NORMAL_TAGGED), + .set = "MEM/NORMAL-TAGGED", } }; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 5a3b15a14a7f..a039a5540cd1 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -120,7 +120,7 @@ static bool pgattr_change_is_safe(u64 old, u64 new) * The following mapping attributes may be updated in live * kernel mappings without the need for break-before-make. */ - static const pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG; + pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG; /* creating or taking down mappings is always safe */ if (old == 0 || new == 0) @@ -134,6 +134,19 @@ static bool pgattr_change_is_safe(u64 old, u64 new) if (old & ~new & PTE_NG) return false; + if (system_supports_mte()) { + /* + * Changing the memory type between Normal and Normal-Tagged + * is safe since Tagged is considered a permission attribute + * from the mismatched attribute aliases perspective. + */ + if ((old & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL) || + (old & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL_TAGGED) || + (new & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL) || + (new & PTE_ATTRINDX_MASK) == PTE_ATTRINDX(MT_NORMAL_TAGGED)) + mask |= PTE_ATTRINDX_MASK; + } + return ((old ^ new) & ~mask) == 0; } @@ -488,7 +501,12 @@ static void __init map_mem(pgd_t *pgdp) if (memblock_is_nomap(reg)) continue; - __map_memblock(pgdp, start, end, PAGE_KERNEL, flags); + /* + * The linear map must allow allocation tags reading/writing + * if MTE is present. Otherwise, it has the same attributes as + * PAGE_KERNEL. + */ + __map_memblock(pgdp, start, end, PAGE_KERNEL_TAGGED, flags); } /* diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 55f715957b36..a8ba4078aa84 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -42,14 +42,18 @@ #define TCR_KASAN_FLAGS 0 #endif -/* Default MAIR_EL1 */ +/* + * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory and + * changed later to Normal Tagged if the system supports MTE. + */ #define MAIR_EL1_SET \ (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRE, MT_DEVICE_nGnRE) | \ MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ - MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT)) + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) #ifdef CONFIG_CPU_PM /** From patchwork Wed Dec 11 18:40:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 75CAE109A for ; Wed, 11 Dec 2019 18:49:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3E572206A5 for ; Wed, 11 Dec 2019 18:49:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E572206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E5F666B3370; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C38DE6B336C; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64A036B337C; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0169.hostedemail.com [216.40.44.169]) by kanga.kvack.org (Postfix) with ESMTP id AD69E6B336B for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 69093180AD81F for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.03.birth00_133d130b67644 X-Spam-Summary: 2,0,0,cf5d302524fcebd6,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2918:3138:3139:3140:3141:3142:3352:3865:3867:3871:3874:4321:5007:6117:6119:6261:7903:8603:8634:10004:11026:11473:11657:11658:11914:12043:12297:12438:12555:13069:13255:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:21795:30051:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: birth00_133d130b67644 X-Filterd-Recvd-Size: 2672 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E18DD1396; Wed, 11 Dec 2019 10:40:50 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7033F3F6CF; Wed, 11 Dec 2019 10:40:49 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 08/22] arm64: mte: Assembler macros and default architecture for .S files Date: Wed, 11 Dec 2019 18:40:13 +0000 Message-Id: <20191211184027.20130-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the multitag_transfer_size macro to the arm64 assembler.h, together with '.arch armv8.5-a+memtag' when CONFIG_ARM64_MTE is enabled. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/assembler.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index b8cf7c85ffa2..74a649a0b6e6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -21,8 +21,13 @@ #include #include #include +#include #include +#ifdef CONFIG_ARM64_MTE + .arch armv8.5-a+memtag +#endif + .macro save_and_disable_daif, flags mrs \flags, daif msr daifset, #0xf @@ -756,4 +761,15 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU .Lyield_out_\@ : .endm +/* + * multitag_transfer_size - set \reg to the block size that is accessed by the + * LDGM/STGM instructions. + */ + .macro multitag_transfer_size, reg, tmp + mrs_s \reg, SYS_GMID_EL1 + ubfx \reg, \reg, #SYS_GMID_EL1_BS_SHIFT, #SYS_GMID_EL1_BS_SIZE + mov \tmp, #4 + lsl \reg, \tmp, \reg + .endm + #endif /* __ASM_ASSEMBLER_H */ From patchwork Wed Dec 11 18:40:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286013 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35D22138D for ; Wed, 11 Dec 2019 18:49:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F24E7206A5 for ; Wed, 11 Dec 2019 18:49:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F24E7206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 53BC36B3365; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 18BA76B3367; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 051E06B3365; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0183.hostedemail.com [216.40.44.183]) by kanga.kvack.org (Postfix) with ESMTP id 9CBF16B3363 for ; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 6CBB9180AD830 for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) X-FDA: 76253747262.28.scale65_12ff3b6e7e939 X-Spam-Summary: 2,0,0,59f5f8336d6e8140,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:5007:6119:6261:7576:8634:9592:10004:11473:11658:11914:12297:12555:12679:13069:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:21795:30051:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:29,LUA_SUMMARY:none X-HE-Tag: scale65_12ff3b6e7e939 X-Filterd-Recvd-Size: 2592 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9467A1435; Wed, 11 Dec 2019 10:40:52 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 218BA3F6CF; Wed, 11 Dec 2019 10:40:51 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 09/22] arm64: mte: Tags-aware clear_page() implementation Date: Wed, 11 Dec 2019 18:40:14 +0000 Message-Id: <20191211184027.20130-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be set to zero a page is cleared as they are visible to the user. Introduce an MTE-aware clear_page() which clears the tags in addition to data. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/lib/clear_page.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S index 78a9ef66288a..575cea03f68a 100644 --- a/arch/arm64/lib/clear_page.S +++ b/arch/arm64/lib/clear_page.S @@ -5,7 +5,9 @@ #include #include +#include #include +#include #include /* @@ -19,8 +21,9 @@ ENTRY(clear_page) and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - -1: dc zva, x0 +1: +alternative_insn "dc zva, x0", "stzgm xzr, [x0]", \ + ARM64_MTE, IS_ENABLED(CONFIG_ARM64_MTE), 1 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b From patchwork Wed Dec 11 18:40:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286007 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA6F1109A for ; Wed, 11 Dec 2019 18:48:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A26A8206A5 for ; Wed, 11 Dec 2019 18:48:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A26A8206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D56A16B3368; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A8FE06B3364; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92AF96B3366; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 76C606B3364 for ; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 40355180AD81F for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) X-FDA: 76253747262.24.roll48_130221ee85848 X-Spam-Summary: 2,0,0,bc14bb2b0318214e,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3871:4250:4321:5007:6117:6119:6261:7576:8634:10004:11026:11473:11658:11914:12291:12297:12438:12555:12679:13069:13311:13357:13894:14181:14384:14394:14721:21080:21230:21451:21627:30054:30079,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:29,LUA_SUMMARY:none X-HE-Tag: roll48_130221ee85848 X-Filterd-Recvd-Size: 2644 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 46349143B; Wed, 11 Dec 2019 10:40:54 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C8DE03F6CF; Wed, 11 Dec 2019 10:40:52 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 10/22] arm64: mte: Tags-aware copy_page() implementation Date: Wed, 11 Dec 2019 18:40:15 +0000 Message-Id: <20191211184027.20130-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be preserved across page copy (e.g. for copy-on-write). Introduce MTE-aware copy_page() which preserves the tags across page copy. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/lib/copy_page.S | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S index bbb8562396af..970b7a20da70 100644 --- a/arch/arm64/lib/copy_page.S +++ b/arch/arm64/lib/copy_page.S @@ -25,6 +25,29 @@ alternative_if ARM64_HAS_NO_HW_PREFETCH prfm pldl1strm, [x1, #384] alternative_else_nop_endif +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 2f +alternative_else_nop_endif + /* + * Copy tags if MTE has been enabled. + */ + mov x2, x0 + mov x3, x1 + + multitag_transfer_size x7, x5 +1: + ldgm x4, [x3] + stgm x4, [x2] + + add x2, x2, x7 + add x3, x3, x7 + + tst x2, #(PAGE_SIZE - 1) + b.ne 1b +2: +#endif + ldp x2, x3, [x1] ldp x4, x5, [x1, #16] ldp x6, x7, [x1, #32] From patchwork Wed Dec 11 18:40:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286039 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D08CD109A for ; Wed, 11 Dec 2019 18:49:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9BFD2206A5 for ; Wed, 11 Dec 2019 18:49:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9BFD2206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B99566B336A; Wed, 11 Dec 2019 13:48:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8DDF86B337C; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5F9D6B3373; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0031.hostedemail.com [216.40.44.31]) by kanga.kvack.org (Postfix) with ESMTP id EE8886B3376 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id C0F78181AC9CB for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.14.class11_1348fdcf7494a X-Spam-Summary: 2,0,0,03561b9aa970039b,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:1801:1981:2194:2199:2393:2538:2559:2562:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:4321:4605:5007:6117:6119:6261:7875:7903:8634:10004:11026:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12683:12986:13894:14096:14181:14394:14721:21080:21230:21451:21627:21990:30003:30054:30067:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: class11_1348fdcf7494a X-Filterd-Recvd-Size: 5933 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E47FD1474; Wed, 11 Dec 2019 10:40:55 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7A7C13F6CF; Wed, 11 Dec 2019 10:40:54 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 11/22] arm64: Tags-aware memcmp_pages() implementation Date: Wed, 11 Dec 2019 18:40:16 +0000 Message-Id: <20191211184027.20130-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When the Memory Tagging Extension is enabled, two pages are identical only if both their data and tags are identical. Make the generic memcmp_pages() a __weak function and add an arm64-specific implementation which takes care of the tags comparison. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/mte.h | 11 +++++++++ arch/arm64/lib/Makefile | 2 ++ arch/arm64/lib/mte.S | 46 ++++++++++++++++++++++++++++++++++++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/cmppages.c | 26 ++++++++++++++++++++ mm/util.c | 2 +- 6 files changed, 87 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/include/asm/mte.h create mode 100644 arch/arm64/lib/mte.S create mode 100644 arch/arm64/mm/cmppages.c diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h new file mode 100644 index 000000000000..64e814273659 --- /dev/null +++ b/arch/arm64/include/asm/mte.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_MTE_H +#define __ASM_MTE_H + +#ifndef __ASSEMBLY__ + +/* Memory Tagging API */ +int mte_memcmp_pages(const void *page1_addr, const void *page2_addr); + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_MTE_H */ diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index c21b936dc01d..178879a754b0 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -16,3 +16,5 @@ lib-$(CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE) += uaccess_flushcache.o obj-$(CONFIG_CRC32) += crc32.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o + +obj-$(CONFIG_ARM64_MTE) += mte.o diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S new file mode 100644 index 000000000000..d41955ab4134 --- /dev/null +++ b/arch/arm64/lib/mte.S @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2019 ARM Ltd. + */ +#include + +#include + +/* + * Compare tags of two pages + * x0 - page1 address + * x1 - page2 address + * Returns: + * w0 - negative, zero or positive value if the tag in the first page is + * less than, equal to or greater than the tag in the second page + */ +ENTRY(mte_memcmp_pages) + multitag_transfer_size x7, x5 +1: + ldgm x2, [x0] + ldgm x3, [x1] + + eor x4, x2, x3 + cbnz x4, 2f + + add x0, x0, x7 + add x1, x1, x7 + + tst x0, #(PAGE_SIZE - 1) + b.ne 1b + + mov w0, #0 + ret +2: + rbit x4, x4 + clz x4, x4 // count the least significant equal bits + and x4, x4, #~3 // round down to a multiple of 4 (bits per tag) + + lsr x2, x2, x4 // remove equal tags + lsr x3, x3, x4 + + lsl w2, w2, #28 // compare the differing tags + sub w0, w2, w3, lsl #28 + + ret +ENDPROC(mte_memcmp_pages) diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 849c1df3d214..7a556e9a19dc 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -8,6 +8,7 @@ obj-$(CONFIG_ARM64_PTDUMP_CORE) += dump.o obj-$(CONFIG_ARM64_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o +obj-$(CONFIG_ARM64_MTE) += cmppages.o KASAN_SANITIZE_physaddr.o += n obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/arm64/mm/cmppages.c b/arch/arm64/mm/cmppages.c new file mode 100644 index 000000000000..943c1877e014 --- /dev/null +++ b/arch/arm64/mm/cmppages.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2019 ARM Ltd. + */ + +#include +#include + +#include +#include + +int memcmp_pages(struct page *page1, struct page *page2) +{ + char *addr1, *addr2; + int ret; + + addr1 = page_address(page1); + addr2 = page_address(page2); + + ret = memcmp(addr1, addr2, PAGE_SIZE); + /* if page content identical, check the tags */ + if (ret == 0 && system_supports_mte()) + ret = mte_memcmp_pages(addr1, addr2); + + return ret; +} diff --git a/mm/util.c b/mm/util.c index 988d11e6c17c..662fb3da6d01 100644 --- a/mm/util.c +++ b/mm/util.c @@ -899,7 +899,7 @@ int get_cmdline(struct task_struct *task, char *buffer, int buflen) return res; } -int memcmp_pages(struct page *page1, struct page *page2) +int __weak memcmp_pages(struct page *page1, struct page *page2) { char *addr1, *addr2; int ret; From patchwork Wed Dec 11 18:40:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5EB18109A for ; Wed, 11 Dec 2019 18:49:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2ED53206A5 for ; Wed, 11 Dec 2019 18:49:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2ED53206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 53B976B336D; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 276F96B3374; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E85076B336F; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 84C766B336E for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 4271A824999B for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.29.rod36_133445d374347 X-Spam-Summary: 2,0,0,22caea5f1afc8685,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org:arnd@arndb.de,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1541:1711:1730:1747:1777:1792:1981:2194:2199:2393:2559:2562:2897:3138:3139:3140:3141:3142:3352:3865:3867:3868:3874:4250:4321:5007:6119:6261:7576:8634:10004:11658:11914:12043:12296:12297:12555:12679:13069:13161:13180:13229:13311:13357:13894:14096:14181:14384:14394:14721:21080:21220:21230:21451:21627:30012:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: rod36_133445d374347 X-Filterd-Recvd-Size: 2781 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A7D2D143D; Wed, 11 Dec 2019 10:40:57 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 241EC3F6CF; Wed, 11 Dec 2019 10:40:56 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org, Arnd Bergmann Subject: [PATCH 12/22] arm64: mte: Add specific SIGSEGV codes Date: Wed, 11 Dec 2019 18:40:17 +0000 Message-Id: <20191211184027.20130-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add MTE-specific SIGSEGV codes to siginfo.h. Note that the for MTE we are reusing the same SPARC ADI codes because the two functionalities are similar and they cannot coexist on the same system. Cc: Arnd Bergmann Signed-off-by: Vincenzo Frascino [catalin.marinas@arm.com: renamed precise/imprecise to sync/async] Signed-off-by: Catalin Marinas --- include/uapi/asm-generic/siginfo.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/include/uapi/asm-generic/siginfo.h b/include/uapi/asm-generic/siginfo.h index cb3d6c267181..a5184a5438c6 100644 --- a/include/uapi/asm-generic/siginfo.h +++ b/include/uapi/asm-generic/siginfo.h @@ -227,8 +227,13 @@ typedef struct siginfo { # define SEGV_PKUERR 4 /* failed protection key checks */ #endif #define SEGV_ACCADI 5 /* ADI not enabled for mapped object */ -#define SEGV_ADIDERR 6 /* Disrupting MCD error */ -#define SEGV_ADIPERR 7 /* Precise MCD exception */ +#ifdef __aarch64__ +# define SEGV_MTEAERR 6 /* Asynchronous MTE error */ +# define SEGV_MTESERR 7 /* Synchronous MTE exception */ +#else +# define SEGV_ADIDERR 6 /* Disrupting MCD error */ +# define SEGV_ADIPERR 7 /* Precise MCD exception */ +#endif #define NSIGSEGV 7 /* From patchwork Wed Dec 11 18:40:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286033 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48848138D for ; Wed, 11 Dec 2019 18:49:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D725206A5 for ; Wed, 11 Dec 2019 18:49:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D725206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 405046B336F; Wed, 11 Dec 2019 13:48:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 13BA56B337C; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86C0F6B337B; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id BD5406B3371 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 891AB4DB7 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.15.sheep05_1336657550832 X-Spam-Summary: 2,0,0,af5bdb8c726eb329,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3354:3865:3866:3868:3870:3871:4049:4118:4250:4321:4605:5007:6117:6119:6261:7576:8603:8634:10004:11026:11473:11657:11658:11914:12043:12296:12297:12438:12555:12679:13053:13138:13161:13229:13231:13894:14394:21080:21230:21451:21627:21740:21795:21987:21990:30003:30045:30051:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: sheep05_1336657550832 X-Filterd-Recvd-Size: 7493 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 508DC1476; Wed, 11 Dec 2019 10:40:59 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DC1E33F6CF; Wed, 11 Dec 2019 10:40:57 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 13/22] arm64: mte: Handle synchronous and asynchronous tag check faults Date: Wed, 11 Dec 2019 18:40:18 +0000 Message-Id: <20191211184027.20130-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino The Memory Tagging Extension has two modes of notifying a tag check fault at EL0, configurable through the SCTLR_EL1.TCF0 field: 1. Synchronous raising of a Data Abort exception with DFSC 17. 2. Asynchronous setting of a cumulative bit in TFSRE0_EL1. Add the exception handler for the synchronous exception and handling of the asynchronous TFSRE0_EL1.TF0 bit setting via a new TIF flag in do_notify_resume(). Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/thread_info.h | 4 +++- arch/arm64/kernel/entry.S | 17 +++++++++++++++++ arch/arm64/kernel/process.c | 7 +++++++ arch/arm64/kernel/signal.c | 8 ++++++++ arch/arm64/mm/fault.c | 9 ++++++++- 5 files changed, 43 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index f0cec4160136..f759a0215a71 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -63,6 +63,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ +#define TIF_MTE_ASYNC_FAULT 6 /* MTE Asynchronous Tag Check Fault */ #define TIF_NOHZ 7 #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ @@ -93,10 +94,11 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_FSCHECK (1 << TIF_FSCHECK) #define _TIF_32BIT (1 << TIF_32BIT) #define _TIF_SVE (1 << TIF_SVE) +#define _TIF_MTE_ASYNC_FAULT (1 << TIF_MTE_ASYNC_FAULT) #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK) + _TIF_UPROBE | _TIF_FSCHECK | _TIF_MTE_ASYNC_FAULT) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 7c6a0a41676f..c221a539e61d 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -144,6 +144,22 @@ alternative_cb_end #endif .endm + // Check for MTE asynchronous tag check faults + .macro check_mte_async_tcf, flgs, tmp +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 1f +alternative_else_nop_endif + mrs_s \tmp, SYS_TFSRE0_EL1 + tbz \tmp, #SYS_TFSR_EL1_TF0_SHIFT, 1f + // Asynchronous TCF occurred at EL0, set the TI flag + orr \flgs, \flgs, #_TIF_MTE_ASYNC_FAULT + str \flgs, [tsk, #TSK_TI_FLAGS] + msr_s SYS_TFSRE0_EL1, xzr +1: +#endif + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -171,6 +187,7 @@ alternative_cb_end ldr x19, [tsk, #TSK_TI_FLAGS] // since we can unmask debug disable_step_tsk x19, x20 // exceptions when scheduling. + check_mte_async_tcf x19, x22 apply_ssbd 1, x22, x23 .else diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 71f788cd2b18..dd98d539894e 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -317,12 +317,19 @@ static void flush_tagged_addr_state(void) clear_thread_flag(TIF_TAGGED_ADDR); } +static void flush_mte_state(void) +{ + if (system_supports_mte()) + clear_thread_flag(TIF_MTE_ASYNC_FAULT); +} + void flush_thread(void) { fpsimd_flush_thread(); tls_thread_flush(); flush_ptrace_hw_breakpoint(current); flush_tagged_addr_state(); + flush_mte_state(); } void release_thread(struct task_struct *dead_task) diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index dd2cdc0d5be2..41fae64af82a 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -730,6 +730,9 @@ static void setup_return(struct pt_regs *regs, struct k_sigaction *ka, regs->regs[29] = (unsigned long)&user->next_frame->fp; regs->pc = (unsigned long)ka->sa.sa_handler; + /* TCO (Tag Check Override) always cleared for signal handlers */ + regs->pstate &= ~PSR_TCO_BIT; + if (ka->sa.sa_flags & SA_RESTORER) sigtramp = ka->sa.sa_restorer; else @@ -921,6 +924,11 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, if (thread_flags & _TIF_UPROBE) uprobe_notify_resume(regs); + if (thread_flags & _TIF_MTE_ASYNC_FAULT) { + clear_thread_flag(TIF_MTE_ASYNC_FAULT); + force_signal_inject(SIGSEGV, SEGV_MTEAERR, 0); + } + if (thread_flags & _TIF_SIGPENDING) do_signal(regs); diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 077b02a2d4d3..ef3bfa2bf2b1 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -660,6 +660,13 @@ static int do_sea(unsigned long addr, unsigned int esr, struct pt_regs *regs) return 0; } +static int do_tag_check_fault(unsigned long addr, unsigned int esr, + struct pt_regs *regs) +{ + do_bad_area(addr, esr, regs); + return 0; +} + static const struct fault_info fault_info[] = { { do_bad, SIGKILL, SI_KERNEL, "ttbr address size fault" }, { do_bad, SIGKILL, SI_KERNEL, "level 1 address size fault" }, @@ -678,7 +685,7 @@ static const struct fault_info fault_info[] = { { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" }, { do_sea, SIGBUS, BUS_OBJERR, "synchronous external abort" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 17" }, + { do_tag_check_fault, SIGSEGV, SEGV_MTESERR, "synchronous tag check fault" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 18" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 19" }, { do_sea, SIGKILL, SI_KERNEL, "level 0 (translation table walk)" }, From patchwork Wed Dec 11 18:40:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 203D3138D for ; Wed, 11 Dec 2019 18:49:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E87ED206A5 for ; Wed, 11 Dec 2019 18:49:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E87ED206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 279C66B3379; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 162FF6B336D; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3A2E6B3372; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 667506B336C for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 314D3181AC9CB for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.02.nerve46_13329b2260149 X-Spam-Summary: 2,0,0,869cc9f245661990,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:4321:4605:5007:6261:7903:8634:10004:11026:11658:11914:12043:12114:12297:12438:12555:13069:13311:13357:13894:14096:14181:14384:14394:14721:21080:21230:21451:21627:21990:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: nerve46_13329b2260149 X-Filterd-Recvd-Size: 2693 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 02EC71477; Wed, 11 Dec 2019 10:41:01 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8475A3F6CF; Wed, 11 Dec 2019 10:40:59 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 14/22] mm: Introduce arch_calc_vm_flag_bits() Date: Wed, 11 Dec 2019 18:40:19 +0000 Message-Id: <20191211184027.20130-15-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to arch_calc_vm_prot_bits(), introduce a dummy arch_calc_vm_flag_bits() invoked from calc_vm_flag_bits(). This macro can be overridden by architectures to insert specific VM_* flags derived from the mmap() MAP_* flags. Signed-off-by: Catalin Marinas --- include/linux/mman.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/include/linux/mman.h b/include/linux/mman.h index 4b08e9c9c538..c53b194c11a1 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -81,6 +81,10 @@ static inline void vm_unacct_memory(long pages) #define arch_calc_vm_prot_bits(prot, pkey) 0 #endif +#ifndef arch_calc_vm_flag_bits +#define arch_calc_vm_flag_bits(flags) 0 +#endif + #ifndef arch_vm_get_page_prot #define arch_vm_get_page_prot(vm_flags) __pgprot(0) #endif @@ -131,7 +135,8 @@ calc_vm_flag_bits(unsigned long flags) return _calc_vm_trans(flags, MAP_GROWSDOWN, VM_GROWSDOWN ) | _calc_vm_trans(flags, MAP_DENYWRITE, VM_DENYWRITE ) | _calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED ) | - _calc_vm_trans(flags, MAP_SYNC, VM_SYNC ); + _calc_vm_trans(flags, MAP_SYNC, VM_SYNC ) | + arch_calc_vm_flag_bits(flags); } unsigned long vm_commit_limit(void); From patchwork Wed Dec 11 18:40:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D19A138D for ; Wed, 11 Dec 2019 18:49:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 03EF5206A5 for ; Wed, 11 Dec 2019 18:49:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03EF5206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9D1C46B3372; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8B6396B3377; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 422FE6B3372; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0067.hostedemail.com [216.40.44.67]) by kanga.kvack.org (Postfix) with ESMTP id 81D146B3369 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 44C10180AD80F for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.03.net66_13302a68c0a21 X-Spam-Summary: 2,0,0,51e8be02739c2f5e,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1605:1730:1747:1777:1792:1981:2194:2198:2199:2200:2393:2553:2559:2562:2693:2892:2895:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4051:4120:4250:4321:4605:4641:5007:6119:6261:7903:8634:8660:8957:10004:11026:11232:11473:11657:11658:11914:12043:12291:12295:12296:12297:12438:12555:12683:12986:13138:13141:13148:13161:13184:13229:13230:13231:13894:13972:14394:21080:21230:21433:21451:21627:21795:21990:30051:30054:30055:30056:30070:30075:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCa che:0,MS X-HE-Tag: net66_13302a68c0a21 X-Filterd-Recvd-Size: 9656 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A01661478; Wed, 11 Dec 2019 10:41:02 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 376903F6CF; Wed, 11 Dec 2019 10:41:01 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 15/22] arm64: mte: Add PROT_MTE support to mmap() and mprotect() Date: Wed, 11 Dec 2019 18:40:20 +0000 Message-Id: <20191211184027.20130-16-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To enable tagging on a memory range, the user must explicitly opt in via a new PROT_MTE flag passed to mmap() or mprotect(). Since this is a new memory type in the AttrIndx field of a pte, simplify the or'ing of these bits over the protection_map[] attributes by making MT_NORMAL index 0. There are two conditions for arch_vm_get_page_prot() to return the MT_NORMAL_TAGGED memory type: (1) the user requested it via PROT_MTE, registered as VM_MTE in the vm_flags, and (2) the vma supports MTE, decided during the mmap() call (only) and registered as VM_MTE_ALLOWED. arch_calc_vm_prot_bits() is responsible for registering the user request as VM_MTE. The newly introduced arch_calc_vm_flag_bits() sets VM_MTE_ALLOWED if the mapping is MAP_ANONYMOUS. An MTE-capable filesystem (RAM-based) may be able to set VM_MTE_ALLOWED during its mmap() file ops call. In addition, update VM_DATA_DEFAULT_FLAGS to allow mprotect(PROT_MTE) on stack or brk area. The Linux mmap() syscall currently ignores unknown PROT_* flags. In the presence of MTE, an mmap(PROT_MTE) on a file which does not support MTE will not report an error and the memory will not be mapped as Normal Tagged. For consistency, mprotect(PROT_MTE) will not report an error either if the memory range does not support MTE. Two subsequent patches in the series will propose tightening of this behaviour. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/memory.h | 18 +++++---- arch/arm64/include/asm/mman.h | 64 ++++++++++++++++++++++++++++++ arch/arm64/include/asm/page.h | 4 +- arch/arm64/include/asm/pgtable.h | 7 +++- arch/arm64/include/uapi/asm/mman.h | 14 +++++++ fs/proc/task_mmu.c | 3 ++ include/linux/mm.h | 8 ++++ 7 files changed, 109 insertions(+), 9 deletions(-) create mode 100644 arch/arm64/include/asm/mman.h create mode 100644 arch/arm64/include/uapi/asm/mman.h diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 55994ab362ae..f0e535895a78 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -138,14 +138,18 @@ /* * Memory types available. + * + * IMPORTANT: MT_NORMAL must be index 0 since vm_get_page_prot() may 'or' in + * the MT_NORMAL_TAGGED memory type for PROT_MTE mappings. Note + * that protection_map[] only contains MT_NORMAL attributes. */ -#define MT_DEVICE_nGnRnE 0 -#define MT_DEVICE_nGnRE 1 -#define MT_DEVICE_GRE 2 -#define MT_NORMAL_NC 3 -#define MT_NORMAL 4 -#define MT_NORMAL_WT 5 -#define MT_NORMAL_TAGGED 6 +#define MT_NORMAL 0 +#define MT_NORMAL_TAGGED 1 +#define MT_NORMAL_NC 2 +#define MT_NORMAL_WT 3 +#define MT_DEVICE_nGnRnE 4 +#define MT_DEVICE_nGnRE 5 +#define MT_DEVICE_GRE 6 /* * Memory types for Stage-2 translation diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h new file mode 100644 index 000000000000..c77a23869223 --- /dev/null +++ b/arch/arm64/include/asm/mman.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_MMAN_H__ +#define __ASM_MMAN_H__ + +#include + +/* + * There are two conditions required for returning a Normal Tagged memory type + * in arch_vm_get_page_prot(): (1) the user requested it via PROT_MTE passed + * to mmap() or mprotect() and (2) the corresponding vma supports MTE. We + * register (1) as VM_MTE in the vma->vm_flags and (2) as VM_MTE_ALLOWED. Note + * that the latter can only be set during the mmap() call since mprotect() + * does not accept MAP_* flags. + */ +static inline unsigned long arch_calc_vm_prot_bits(unsigned long prot, + unsigned long pkey) +{ + if (!system_supports_mte()) + return 0; + + if (prot & PROT_MTE) + return VM_MTE; + + return 0; +} +#define arch_calc_vm_prot_bits arch_calc_vm_prot_bits + +static inline unsigned long arch_calc_vm_flag_bits(unsigned long flags) +{ + if (!system_supports_mte()) + return 0; + + /* + * Only allow MTE on anonymous mappings as these are guaranteed to be + * backed by tags-capable memory. The vm_flags may be overridden by a + * filesystem supporting MTE (RAM-based). + */ + if (flags & MAP_ANONYMOUS) + return VM_MTE_ALLOWED; + + return 0; +} +#define arch_calc_vm_flag_bits arch_calc_vm_flag_bits + +static inline pgprot_t arch_vm_get_page_prot(unsigned long vm_flags) +{ + return (vm_flags & VM_MTE) && (vm_flags & VM_MTE_ALLOWED) ? + __pgprot(PTE_ATTRINDX(MT_NORMAL_TAGGED)) : + __pgprot(0); +} +#define arch_vm_get_page_prot arch_vm_get_page_prot + +static inline bool arch_validate_prot(unsigned long prot, unsigned long addr) +{ + unsigned long supported = PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM; + + if (system_supports_mte()) + supported |= PROT_MTE; + + return (prot & ~supported) == 0; +} +#define arch_validate_prot arch_validate_prot + +#endif /* !__ASM_MMAN_H__ */ diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h index d39ddb258a04..10d71f927b70 100644 --- a/arch/arm64/include/asm/page.h +++ b/arch/arm64/include/asm/page.h @@ -32,9 +32,11 @@ extern int pfn_valid(unsigned long); #endif /* !__ASSEMBLY__ */ +/* Used for stack and brk memory ranges */ #define VM_DATA_DEFAULT_FLAGS \ (((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0) | \ - VM_READ | VM_WRITE | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) + VM_READ | VM_WRITE | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC | \ + VM_MTE_ALLOWED) #include diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 5d15b4735a0e..e5e2cb6f2f3c 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -661,8 +661,13 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd) static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) { + /* + * Normal and Normal-Tagged are two different memory types and indices + * in MAIR_EL1. The mask below has to include PTE_ATTRINDX_MASK. + */ const pteval_t mask = PTE_USER | PTE_PXN | PTE_UXN | PTE_RDONLY | - PTE_PROT_NONE | PTE_VALID | PTE_WRITE; + PTE_PROT_NONE | PTE_VALID | PTE_WRITE | + PTE_ATTRINDX_MASK; /* preserve the hardware dirty information */ if (pte_hw_dirty(pte)) pte = pte_mkdirty(pte); diff --git a/arch/arm64/include/uapi/asm/mman.h b/arch/arm64/include/uapi/asm/mman.h new file mode 100644 index 000000000000..d7677ee84878 --- /dev/null +++ b/arch/arm64/include/uapi/asm/mman.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _UAPI__ASM_MMAN_H +#define _UAPI__ASM_MMAN_H + +#include + +/* + * The generic mman.h file reserves 0x10 and 0x20 for arch-specific PROT_* + * flags. + */ +/* 0x10 reserved for PROT_BTI */ +#define PROT_MTE 0x20 /* Normal Tagged mapping */ + +#endif /* !_UAPI__ASM_MMAN_H */ diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 9442631fd4af..34bc9e0b4896 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -677,6 +677,9 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_MERGEABLE)] = "mg", [ilog2(VM_UFFD_MISSING)]= "um", [ilog2(VM_UFFD_WP)] = "uw", +#ifdef CONFIG_ARM64_MTE + [ilog2(VM_MTE)] = "mt", +#endif #ifdef CONFIG_ARCH_HAS_PKEYS /* These come out via ProtectionKey: */ [ilog2(VM_PKEY_BIT0)] = "", diff --git a/include/linux/mm.h b/include/linux/mm.h index c97ea3b694e6..cf59b4558bbe 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -340,6 +340,14 @@ extern unsigned int kobjsize(const void *objp); # define VM_MPX VM_NONE #endif +#if defined(CONFIG_ARM64_MTE) +# define VM_MTE VM_HIGH_ARCH_0 /* Use Tagged memory for access control */ +# define VM_MTE_ALLOWED VM_HIGH_ARCH_1 /* Tagged memory permitted */ +#else +# define VM_MTE VM_NONE +# define VM_MTE_ALLOWED VM_NONE +#endif + #ifndef VM_GROWSUP # define VM_GROWSUP VM_NONE #endif From patchwork Wed Dec 11 18:40:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286005 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 02C0C109A for ; Wed, 11 Dec 2019 18:48:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C90FD21556 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C90FD21556 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B505E6B3366; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A16916B3369; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88F316B3367; Wed, 11 Dec 2019 13:48:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id 732B76B3363 for ; Wed, 11 Dec 2019 13:48:51 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 44B0C4DA6 for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) X-FDA: 76253747262.03.elbow41_1304887174d42 X-Spam-Summary: 2,0,0,09ce94bdb0e469e9,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:2895:2914:3138:3139:3140:3141:3142:3353:3865:3867:3870:3871:3872:4321:4385:4605:5007:6261:8634:10004:11026:11658:11914:12043:12114:12296:12297:12438:12555:13894:14096:14181:14394:14721:21080:21230:21451:21627:21990:30012:30054:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:31,LUA_SUMMARY:none X-HE-Tag: elbow41_1304887174d42 X-Filterd-Recvd-Size: 3710 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 48A84147A; Wed, 11 Dec 2019 10:41:04 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D3D583F6CF; Wed, 11 Dec 2019 10:41:02 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 16/22] mm: Introduce arch_validate_flags() Date: Wed, 11 Dec 2019 18:40:21 +0000 Message-Id: <20191211184027.20130-17-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to arch_validate_prot() called from do_mprotect_pkey(), an architecture may need to sanity-check the new vm_flags. Define a dummy function always returning true. In addition to do_mprotect_pkey(), also invoke it from mmap_region() prior to updating vma->vm_page_prot to allow the architecture code to veto potentially inconsistent vm_flags. Signed-off-by: Catalin Marinas --- include/linux/mman.h | 13 +++++++++++++ mm/mmap.c | 9 +++++++++ mm/mprotect.c | 8 ++++++++ 3 files changed, 30 insertions(+) diff --git a/include/linux/mman.h b/include/linux/mman.h index c53b194c11a1..686fa6c98ce7 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -103,6 +103,19 @@ static inline bool arch_validate_prot(unsigned long prot, unsigned long addr) #define arch_validate_prot arch_validate_prot #endif +#ifndef arch_validate_flags +/* + * This is called from mprotect() with the updated vma->vm_flags. + * + * Returns true if the VM_* flags are valid. + */ +static inline bool arch_validate_flags(unsigned long flags) +{ + return true; +} +#define arch_validate_flags arch_validate_flags +#endif + /* * Optimisation macro. It is equivalent to: * (x & bit1) ? bit2 : 0 diff --git a/mm/mmap.c b/mm/mmap.c index 9c648524e4dc..433355c5bdf1 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1804,6 +1804,15 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vma_set_anonymous(vma); } + /* Allow architectures to sanity-check the vm_flags */ + if (!arch_validate_flags(vma->vm_flags)) { + error = -EINVAL; + if (file) + goto unmap_and_free_vma; + else + goto free_vma; + } + vma_link(mm, vma, prev, rb_link, rb_parent); /* Once vma denies write, undo our temporary denial count */ if (file) { diff --git a/mm/mprotect.c b/mm/mprotect.c index 7a8e84f86831..787b071bcbfd 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -543,6 +543,14 @@ static int do_mprotect_pkey(unsigned long start, size_t len, goto out; } + /* + * Allow architectures to sanity-check the new flags. + */ + if (!arch_validate_flags(newflags)) { + error = -EINVAL; + goto out; + } + error = security_file_mprotect(vma, reqprot, prot); if (error) goto out; From patchwork Wed Dec 11 18:40:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98B59138D for ; Wed, 11 Dec 2019 18:49:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 674BF206A5 for ; Wed, 11 Dec 2019 18:49:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 674BF206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E3D36B3374; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 534326B336C; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAA1E6B336C; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id 973396B3370 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 6FCD7249D for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.22.peace50_13315933e5047 X-Spam-Summary: 2,0,0,3c21aaf40884e608,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3871:4321:5007:6261:8634:10004:11026:11473:11657:11658:11914:12043:12114:12296:12297:12438:12555:12986:13069:13161:13229:13255:13311:13357:13894:14096:14181:14384:14394:14721:21080:21212:21230:21451:21627:21990:30054:30069:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: peace50_13315933e5047 X-Filterd-Recvd-Size: 2926 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E52FC1480; Wed, 11 Dec 2019 10:41:05 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7C6503F6CF; Wed, 11 Dec 2019 10:41:04 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 17/22] arm64: mte: Validate the PROT_MTE request via arch_validate_flags() Date: Wed, 11 Dec 2019 18:40:22 +0000 Message-Id: <20191211184027.20130-18-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make use of the newly introduced arch_validate_flags() hook to sanity-check the PROT_MTE request passed to mmap() and mprotect(). If the mapping does not support MTE, these syscalls will return -EINVAL. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/mman.h | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/mman.h b/arch/arm64/include/asm/mman.h index c77a23869223..5c356d1ca266 100644 --- a/arch/arm64/include/asm/mman.h +++ b/arch/arm64/include/asm/mman.h @@ -44,7 +44,11 @@ static inline unsigned long arch_calc_vm_flag_bits(unsigned long flags) static inline pgprot_t arch_vm_get_page_prot(unsigned long vm_flags) { - return (vm_flags & VM_MTE) && (vm_flags & VM_MTE_ALLOWED) ? + /* + * Checking for VM_MTE only is sufficient since arch_validate_flags() + * does not permit (VM_MTE & !VM_MTE_ALLOWED). + */ + return (vm_flags & VM_MTE) ? __pgprot(PTE_ATTRINDX(MT_NORMAL_TAGGED)) : __pgprot(0); } @@ -61,4 +65,14 @@ static inline bool arch_validate_prot(unsigned long prot, unsigned long addr) } #define arch_validate_prot arch_validate_prot +static inline bool arch_validate_flags(unsigned long flags) +{ + if (!system_supports_mte()) + return true; + + /* only allow VM_MTE if VM_MTE_ALLOWED has been set previously */ + return !(flags & VM_MTE) || (flags & VM_MTE_ALLOWED); +} +#define arch_validate_flags arch_validate_flags + #endif /* !__ASM_MMAN_H__ */ From patchwork Wed Dec 11 18:40:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286015 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 814B1109A for ; Wed, 11 Dec 2019 18:49:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 548C5206A5 for ; Wed, 11 Dec 2019 18:49:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 548C5206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C1926B3367; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6B1526B336B; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 497806B336B; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id 320B36B3365 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 0072E4DA2 for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) X-FDA: 76253747262.08.wind69_132c4aa569b42 X-Spam-Summary: 2,0,0,92186428259b6292,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1534:1540:1711:1730:1747:1777:1792:2393:2559:2562:2895:3138:3139:3140:3141:3142:3352:3865:3870:3871:4321:5007:6261:7875:7903:8634:10004:11026:11473:11658:11914:12043:12296:12297:12438:12555:13069:13184:13229:13311:13357:13894:14181:14384:14394:14664:14721:21080:21230:21451:21627:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMARY:none X-HE-Tag: wind69_132c4aa569b42 X-Filterd-Recvd-Size: 2235 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 963BD14BF; Wed, 11 Dec 2019 10:41:07 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 24D543F6CF; Wed, 11 Dec 2019 10:41:06 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 18/22] mm: Allow arm64 mmap(PROT_MTE) on RAM-based files Date: Wed, 11 Dec 2019 18:40:23 +0000 Message-Id: <20191211184027.20130-19-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since arm64 memory (allocation) tags can only be stored in RAM, mapping files with PROT_MTE is not allowed by default. RAM-based files like those in a tmpfs mount or memfd_create() can support memory tagging, so update the vm_flags accordingly in shmem_mmap(). Signed-off-by: Catalin Marinas --- mm/shmem.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/shmem.c b/mm/shmem.c index 165fa6332993..1b1753f90e2d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2225,6 +2225,9 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma) vma->vm_flags &= ~(VM_MAYWRITE); } + /* arm64 - allow memory tagging on RAM-based files */ + vma->vm_flags |= VM_MTE_ALLOWED; + file_accessed(file); vma->vm_ops = &shmem_vm_ops; if (IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE) && From patchwork Wed Dec 11 18:40:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286041 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40357109A for ; Wed, 11 Dec 2019 18:49:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0AF7E206A5 for ; Wed, 11 Dec 2019 18:49:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0AF7E206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EFA1E6B3378; Wed, 11 Dec 2019 13:48:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B4B4B6B337B; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E5BF06B336A; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 1660D6B3370 for ; Wed, 11 Dec 2019 13:48:53 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id D3AD5181AEF1A for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.16.taste31_1348a86117414 X-Spam-Summary: 2,0,0,799460bb703b4d0f,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2393:2559:2562:2895:3138:3139:3140:3141:3142:3622:3865:3866:3867:3868:3870:3871:3872:3874:4049:4118:4250:4321:4605:5007:6119:6261:7875:7903:8634:10004:11026:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12986:13161:13229:13868:13894:14096:14394:21080:21230:21433:21451:21524:21525:21627:21795:21966:21990:30051:30054:30055:30062:30069:30070:30079,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:1,LUA_SUMMAR Y:none X-HE-Tag: taste31_1348a86117414 X-Filterd-Recvd-Size: 7725 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3EC2E14F6; Wed, 11 Dec 2019 10:41:09 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CA4783F6CF; Wed, 11 Dec 2019 10:41:07 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 19/22] arm64: mte: Allow user control of the tag check mode via prctl() Date: Wed, 11 Dec 2019 18:40:24 +0000 Message-Id: <20191211184027.20130-20-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: By default, even if PROT_MTE is set on a memory range, there is no tag check fault reporting (SIGSEGV). Introduce a set of option to the exiting prctl(PR_SET_TAGGED_ADDR_CTRL) to allow user control of the tag check fault mode: PR_MTE_TCF_NONE - no reporting (default) PR_MTE_TCF_SYNC - synchronous tag check fault reporting PR_MTE_TCF_ASYNC - asynchronous tag check fault reporting These options translate into the corresponding SCTLR_EL1.TCF0 bitfield, context-switched by the kernel. Note that uaccess done by the kernel is not checked and cannot be configured by the user. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/processor.h | 3 + arch/arm64/kernel/process.c | 119 +++++++++++++++++++++++++++-- include/uapi/linux/prctl.h | 6 ++ 3 files changed, 123 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 5ba63204d078..91aa270afc7d 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -148,6 +148,9 @@ struct thread_struct { #ifdef CONFIG_ARM64_PTR_AUTH struct ptrauth_keys keys_user; #endif +#ifdef CONFIG_ARM64_MTE + u64 sctlr_tcf0; +#endif }; static inline void arch_thread_struct_whitelist(unsigned long *offset, diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index dd98d539894e..47ce98f47253 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -317,11 +317,22 @@ static void flush_tagged_addr_state(void) clear_thread_flag(TIF_TAGGED_ADDR); } +#ifdef CONFIG_ARM64_MTE +static void flush_mte_state(void) +{ + if (!system_supports_mte()) + return; + + /* clear any pending asynchronous tag fault */ + clear_thread_flag(TIF_MTE_ASYNC_FAULT); + /* disable tag checking */ + current->thread.sctlr_tcf0 = 0; +} +#else static void flush_mte_state(void) { - if (system_supports_mte()) - clear_thread_flag(TIF_MTE_ASYNC_FAULT); } +#endif void flush_thread(void) { @@ -484,6 +495,29 @@ static void ssbs_thread_switch(struct task_struct *next) set_ssbs_bit(regs); } +#ifdef CONFIG_ARM64_MTE +static void update_sctlr_el1_tcf0(u64 tcf0) +{ + /* no need for ISB since this only affects EL0, implicit with ERET */ + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); +} + +/* Handle MTE thread switch */ +static void mte_thread_switch(struct task_struct *next) +{ + if (!system_supports_mte()) + return; + + /* avoid expensive SCTLR_EL1 accesses if no change */ + if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) + update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); +} +#else +static void mte_thread_switch(struct task_struct *next) +{ +} +#endif + /* * We store our current task in sp_el0, which is clobbered by userspace. Keep a * shadow copy so that we can restore this upon entry from userspace. @@ -514,6 +548,7 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, uao_thread_switch(next); ptrauth_thread_switch(next); ssbs_thread_switch(next); + mte_thread_switch(next); /* * Complete any pending TLB or cache maintenance on this CPU in case @@ -574,6 +609,67 @@ void arch_setup_new_exec(void) ptrauth_thread_init_user(current); } +#ifdef CONFIG_ARM64_MTE +static long set_mte_ctrl(unsigned long arg) +{ + u64 tcf0; + + if (!system_supports_mte()) + return 0; + + switch (arg & PR_MTE_TCF_MASK) { + case PR_MTE_TCF_NONE: + tcf0 = 0; + break; + case PR_MTE_TCF_SYNC: + tcf0 = SCTLR_EL1_TCF0_SYNC; + break; + case PR_MTE_TCF_ASYNC: + tcf0 = SCTLR_EL1_TCF0_ASYNC; + break; + default: + return -EINVAL; + } + + /* + * mte_thread_switch() checks current->thread.sctlr_tcf0 as an + * optimisation. Disable preemption so that it does not see + * the variable update before the SCTLR_EL1.TCF0 one. + */ + preempt_disable(); + current->thread.sctlr_tcf0 = tcf0; + update_sctlr_el1_tcf0(tcf0); + preempt_enable(); + + return 0; +} + +static long get_mte_ctrl(void) +{ + if (!system_supports_mte()) + return 0; + + switch (current->thread.sctlr_tcf0) { + case SCTLR_EL1_TCF0_SYNC: + return PR_MTE_TCF_SYNC; + case SCTLR_EL1_TCF0_ASYNC: + return PR_MTE_TCF_ASYNC; + } + + return 0; +} +#else +static long set_mte_ctrl(unsigned long arg) +{ + return 0; +} + +static long get_mte_ctrl(void) +{ + return 0; +} +#endif + #ifdef CONFIG_ARM64_TAGGED_ADDR_ABI /* * Control the relaxed ABI allowing tagged user addresses into the kernel. @@ -582,9 +678,15 @@ static unsigned int tagged_addr_disabled; long set_tagged_addr_ctrl(unsigned long arg) { + unsigned long valid_mask = PR_TAGGED_ADDR_ENABLE; + if (is_compat_task()) return -EINVAL; - if (arg & ~PR_TAGGED_ADDR_ENABLE) + + if (system_supports_mte()) + valid_mask |= PR_MTE_TCF_MASK; + + if (arg & ~valid_mask) return -EINVAL; /* @@ -594,6 +696,9 @@ long set_tagged_addr_ctrl(unsigned long arg) if (arg & PR_TAGGED_ADDR_ENABLE && tagged_addr_disabled) return -EINVAL; + if (set_mte_ctrl(arg) != 0) + return -EINVAL; + update_thread_flag(TIF_TAGGED_ADDR, arg & PR_TAGGED_ADDR_ENABLE); return 0; @@ -601,13 +706,17 @@ long set_tagged_addr_ctrl(unsigned long arg) long get_tagged_addr_ctrl(void) { + long ret = 0; + if (is_compat_task()) return -EINVAL; if (test_thread_flag(TIF_TAGGED_ADDR)) - return PR_TAGGED_ADDR_ENABLE; + ret = PR_TAGGED_ADDR_ENABLE; - return 0; + ret |= get_mte_ctrl(); + + return ret; } /* diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 7da1b37b27aa..5e9323e66a38 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -233,5 +233,11 @@ struct prctl_mm_map { #define PR_SET_TAGGED_ADDR_CTRL 55 #define PR_GET_TAGGED_ADDR_CTRL 56 # define PR_TAGGED_ADDR_ENABLE (1UL << 0) +/* MTE tag check fault modes */ +# define PR_MTE_TCF_SHIFT 1 +# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) #endif /* _LINUX_PRCTL_H */ From patchwork Wed Dec 11 18:40:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8289138D for ; Wed, 11 Dec 2019 18:49:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AF588206A5 for ; Wed, 11 Dec 2019 18:49:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF588206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C3F766B336E; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B776F6B3373; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CA5A6B3371; Wed, 11 Dec 2019 13:48:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0142.hostedemail.com [216.40.44.142]) by kanga.kvack.org (Postfix) with ESMTP id 644F66B336A for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 44EA14DA8 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.04.hole04_13338b6e8f510 X-Spam-Summary: 2,0,0,ef33e7fef088a279,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1544:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:3138:3139:3140:3141:3142:3355:3622:3865:3867:3868:3870:3871:4250:4321:4385:4605:5007:6261:8634:9036:10004:11026:11473:11657:11658:11914:12043:12291:12296:12297:12438:12555:12986:13138:13161:13229:13231:13894:13972:14181:14394:14721:21080:21230:21451:21611:21627:21795:21990:30003:30051:30054:30070,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: hole04_13338b6e8f510 X-Filterd-Recvd-Size: 5660 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:51 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DB91314FF; Wed, 11 Dec 2019 10:41:10 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 729483F6CF; Wed, 11 Dec 2019 10:41:09 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 20/22] arm64: mte: Allow user control of the excluded tags via prctl() Date: Wed, 11 Dec 2019 18:40:25 +0000 Message-Id: <20191211184027.20130-21-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The IRG, ADDG and SUBG instructions insert a random tag in the resulting address. Certain tags can be excluded via the GCR_EL1.Exclude bitmap when, for example, the user wants a certain colour for freed buffers. Since the GCR_EL1 register is not accessible at EL0, extend the prctl(PR_SET_TAGGED_ADDR_CTRL) interface to include a 16-bit field in the first argument for controlling the excluded tags. This setting is pre-thread. Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/sysreg.h | 7 +++++++ arch/arm64/kernel/process.c | 27 +++++++++++++++++++++++---- include/uapi/linux/prctl.h | 3 +++ 4 files changed, 34 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 91aa270afc7d..5b6988035334 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -150,6 +150,7 @@ struct thread_struct { #endif #ifdef CONFIG_ARM64_MTE u64 sctlr_tcf0; + u64 gcr_excl; #endif }; diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 9e5753272f4b..b6bb6d31f1cd 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -901,6 +901,13 @@ write_sysreg(__scs_new, sysreg); \ } while (0) +#define sysreg_clear_set_s(sysreg, clear, set) do { \ + u64 __scs_val = read_sysreg_s(sysreg); \ + u64 __scs_new = (__scs_val & ~(u64)(clear)) | (set); \ + if (__scs_new != __scs_val) \ + write_sysreg_s(__scs_new, sysreg); \ +} while (0) + #endif #endif /* __ASM_SYSREG_H */ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 47ce98f47253..5ec6889795fc 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -502,6 +502,15 @@ static void update_sctlr_el1_tcf0(u64 tcf0) sysreg_clear_set(sctlr_el1, SCTLR_EL1_TCF0_MASK, tcf0); } +static void update_gcr_el1_excl(u64 excl) +{ + /* + * No need for ISB since this only affects EL0 currently, implicit + * with ERET. + */ + sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl); +} + /* Handle MTE thread switch */ static void mte_thread_switch(struct task_struct *next) { @@ -511,6 +520,7 @@ static void mte_thread_switch(struct task_struct *next) /* avoid expensive SCTLR_EL1 accesses if no change */ if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); + update_gcr_el1_excl(next->thread.gcr_excl); } #else static void mte_thread_switch(struct task_struct *next) @@ -641,22 +651,31 @@ static long set_mte_ctrl(unsigned long arg) update_sctlr_el1_tcf0(tcf0); preempt_enable(); + current->thread.gcr_excl = (arg & PR_MTE_EXCL_MASK) >> PR_MTE_EXCL_SHIFT; + update_gcr_el1_excl(current->thread.gcr_excl); + return 0; } static long get_mte_ctrl(void) { + unsigned long ret; + if (!system_supports_mte()) return 0; + ret = current->thread.gcr_excl << PR_MTE_EXCL_SHIFT; + switch (current->thread.sctlr_tcf0) { case SCTLR_EL1_TCF0_SYNC: - return PR_MTE_TCF_SYNC; + ret |= PR_MTE_TCF_SYNC; + break; case SCTLR_EL1_TCF0_ASYNC: - return PR_MTE_TCF_ASYNC; + ret |= PR_MTE_TCF_ASYNC; + break; } - return 0; + return ret; } #else static long set_mte_ctrl(unsigned long arg) @@ -684,7 +703,7 @@ long set_tagged_addr_ctrl(unsigned long arg) return -EINVAL; if (system_supports_mte()) - valid_mask |= PR_MTE_TCF_MASK; + valid_mask |= PR_MTE_TCF_MASK | PR_MTE_EXCL_MASK; if (arg & ~valid_mask) return -EINVAL; diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 5e9323e66a38..749de5ab4f9f 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -239,5 +239,8 @@ struct prctl_mm_map { # define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) # define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) # define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) +/* MTE tag exclusion mask */ +# define PR_MTE_EXCL_SHIFT 3 +# define PR_MTE_EXCL_MASK (0xffffUL << PR_MTE_EXCL_SHIFT) #endif /* _LINUX_PRCTL_H */ From patchwork Wed Dec 11 18:40:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 512BF188B for ; Wed, 11 Dec 2019 18:49:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 29604206A5 for ; Wed, 11 Dec 2019 18:49:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 29604206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 949816B3371; Wed, 11 Dec 2019 13:48:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 693796B3375; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF2D86B336B; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0142.hostedemail.com [216.40.44.142]) by kanga.kvack.org (Postfix) with ESMTP id EC8BE6B3373 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id CBA52249D for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.10.cup13_1347a925d3d00 X-Spam-Summary: 2,0,0,b27304a2befe7924,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1042:1260:1261:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3871:3874:4250:5007:6261:7576:7808:8634:10004:11026:11473:11658:11914:12043:12291:12297:12438:12555:12679:12683:13069:13142:13230:13311:13357:13894:14093:14096:14110:14181:14384:14394:14721:21080:21230:21451:21627:21772:30054,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: cup13_1347a925d3d00 X-Filterd-Recvd-Size: 3115 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 847B41500; Wed, 11 Dec 2019 10:41:12 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1B8E63F6CF; Wed, 11 Dec 2019 10:41:11 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 21/22] arm64: mte: Kconfig entry Date: Wed, 11 Dec 2019 18:40:26 +0000 Message-Id: <20191211184027.20130-22-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Add Memory Tagging Extension support to the arm64 kbuild. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- arch/arm64/Kconfig | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b1b4476ddb83..9a119c402bde 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1484,6 +1484,38 @@ config ARM64_PTR_AUTH endmenu +menu "ARMv8.5 architectural features" + +config ARM64_AS_HAS_MTE + def_bool $(as-instr,.arch armv8.5-a+memtag) + +config ARM64_MTE + bool "Memory Tagging Extension support" + depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI + select ARCH_USES_HIGH_VMA_FLAGS + select ARCH_NO_SWAP + help + Memory Tagging (part of the ARMv8.5 Extensions) provides + architectural support for run-time, always-on detection of + various classes of memory error to aid with software debugging + to eliminate vulnerabilities arising from memory-unsafe + languages. + + This option enables the support for the Memory Tagging + Extension at EL0 (i.e. for userspace). + + Selecting this option allows the feature to be detected at + runtime. Any secondary CPU not implementing this feature will + not be allowed a late bring-up. + + Userspace binaries that want to use this feature must + explicitly opt in. The mechanism for the userspace is + described in: + + Documentation/arm64/memory-tagging-extension.rst. + +endmenu + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y From patchwork Wed Dec 11 18:40:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11286035 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10ED5109A for ; Wed, 11 Dec 2019 18:49:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BCDEC20836 for ; Wed, 11 Dec 2019 18:49:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BCDEC20836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 72D306B336B; Wed, 11 Dec 2019 13:48:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 33E556B337B; Wed, 11 Dec 2019 13:48:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CA286B3371; Wed, 11 Dec 2019 13:48:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0080.hostedemail.com [216.40.44.80]) by kanga.kvack.org (Postfix) with ESMTP id D40226B3375 for ; Wed, 11 Dec 2019 13:48:52 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 9A17C180AD830 for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) X-FDA: 76253747304.12.frogs10_133abe01ba50b X-Spam-Summary: 2,0,0,245658c876c2c68f,d41d8cd98f00b204,catalin.marinas@arm.com,:linux-arm-kernel@lists.infradead.org:will@kernel.org:maz@kernel.org:vincenzo.frascino@arm.com:szabolcs.nagy@arm.com:richard.earnshaw@arm.com:kevin.brodsky@arm.com:andreyknvl@google.com::linux-arch@vger.kernel.org,RULES_HIT:1:41:69:355:379:541:800:960:966:973:988:989:1042:1260:1261:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:1801:2196:2199:2393:2553:2559:2562:2637:2693:2901:2904:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4250:4321:4385:4605:5007:6117:6119:6261:6755:7576:7875:7903:7904:8634:8660:8784:9121:10004:11026:11232:11233:11473:11657:11658:11914:12043:12219:12291:12296:12297:12438:12555:12679:12683:12740:12895:12986:13138:13148:13230:13231:13894:13972:14394:14827:21063:21080:21230:21433:21451:21627:21772:21795:21966:21990:30003:30012:30029:30051:30054:30055:30056:30070:30074:30080:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,C acheIP:n X-HE-Tag: frogs10_133abe01ba50b X-Filterd-Recvd-Size: 12763 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 18:48:52 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D83E1509; Wed, 11 Dec 2019 10:41:14 -0800 (PST) Received: from arrakis.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B8FAD3F6CF; Wed, 11 Dec 2019 10:41:12 -0800 (PST) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Marc Zyngier , Vincenzo Frascino , Szabolcs Nagy , Richard Earnshaw , Kevin Brodsky , Andrey Konovalov , linux-mm@kvack.org, linux-arch@vger.kernel.org Subject: [PATCH 22/22] arm64: mte: Add Memory Tagging Extension documentation Date: Wed, 11 Dec 2019 18:40:27 +0000 Message-Id: <20191211184027.20130-23-catalin.marinas@arm.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191211184027.20130-1-catalin.marinas@arm.com> References: <20191211184027.20130-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Memory Tagging Extension (part of the ARMv8.5 Extensions) provides a mechanism to detect the sources of memory related errors which may be vulnerable to exploitation, including bounds violations, use-after-free, use-after-return, use-out-of-scope and use before initialization errors. Add Memory Tagging Extension documentation for the arm64 linux kernel support. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas --- Documentation/arm64/cpu-feature-registers.rst | 4 + Documentation/arm64/elf_hwcaps.rst | 4 + Documentation/arm64/index.rst | 1 + .../arm64/memory-tagging-extension.rst | 229 ++++++++++++++++++ 4 files changed, 238 insertions(+) create mode 100644 Documentation/arm64/memory-tagging-extension.rst diff --git a/Documentation/arm64/cpu-feature-registers.rst b/Documentation/arm64/cpu-feature-registers.rst index b6e44884e3ad..67305a5f613a 100644 --- a/Documentation/arm64/cpu-feature-registers.rst +++ b/Documentation/arm64/cpu-feature-registers.rst @@ -172,8 +172,12 @@ infrastructure: +------------------------------+---------+---------+ | Name | bits | visible | +------------------------------+---------+---------+ + | MTE | [11-8] | y | + +------------------------------+---------+---------+ | SSBS | [7-4] | y | +------------------------------+---------+---------+ + | BT | [3-0] | n | + +------------------------------+---------+---------+ 4) MIDR_EL1 - Main ID Register diff --git a/Documentation/arm64/elf_hwcaps.rst b/Documentation/arm64/elf_hwcaps.rst index 7fa3d215ae6a..0f52d22c28af 100644 --- a/Documentation/arm64/elf_hwcaps.rst +++ b/Documentation/arm64/elf_hwcaps.rst @@ -204,6 +204,10 @@ HWCAP2_FRINT Functionality implied by ID_AA64ISAR1_EL1.FRINTTS == 0b0001. +HWCAP2_MTE + + Functionality implied by ID_AA64PFR1_EL1.MTE == 0b0010. + Documentation/arm64/memory-tagging-extension.rst. 4. Unused AT_HWCAP bits ----------------------- diff --git a/Documentation/arm64/index.rst b/Documentation/arm64/index.rst index 5c0c69dc58aa..82970c6d384f 100644 --- a/Documentation/arm64/index.rst +++ b/Documentation/arm64/index.rst @@ -13,6 +13,7 @@ ARM64 Architecture hugetlbpage legacy_instructions memory + memory-tagging-extension pointer-authentication silicon-errata sve diff --git a/Documentation/arm64/memory-tagging-extension.rst b/Documentation/arm64/memory-tagging-extension.rst new file mode 100644 index 000000000000..ae02f0771971 --- /dev/null +++ b/Documentation/arm64/memory-tagging-extension.rst @@ -0,0 +1,229 @@ +=============================================== +Memory Tagging Extension (MTE) in AArch64 Linux +=============================================== + +Authors: Vincenzo Frascino + Catalin Marinas + +Date: 2019-11-29 + +This document describes the provision of the Memory Tagging Extension +functionality in AArch64 Linux. + +Introduction +============ + +ARMv8.5 based processors introduce the Memory Tagging Extension (MTE) +feature. MTE is built on top of the ARMv8.0 virtual address tagging TBI +(Top Byte Ignore) feature and allows software to access a 4-bit +allocation tag for each 16-byte granule in the physical address space. +Such memory range must be mapped with the Normal-Tagged memory +attribute. A logical tag is derived from bits 59-56 of the virtual +address used for the memory access. A CPU with MTE enabled will compare +the logical tag against the allocation tag and potentially raise an +exception on mismatch, subject to system registers configuration. + +Userspace Support +================= + +Memory Tagging Extension Linux support depends on AArch64 Tagged Address +ABI being enabled in the kernel. For more details on AArch64 Tagged +Address ABI refer to Documentation/arm64/tagged-address-abi.rst. + +When ``CONFIG_ARM64_MTE`` is selected and Memory Tagging Extension is +supported by the hardware, the kernel advertises the feature to +userspace via ``HWCAP2_MTE``. + +PROT_MTE +-------- + +To access the allocation tags, a user process must enable the Tagged +memory attribute on an address range using a new ``prot`` flag for +``mmap()`` and ``mprotect()``: + +``PROT_MTE`` - Pages allow access to the MTE allocation tags. + +The allocation tag is set to 0 when such pages are first mapped in the +user address space and preserved on copy-on-write. ``MAP_SHARED`` is +supported and the allocation tags can be shared between processes. + +**Note**: ``PROT_MTE`` is only supported on ``MAP_ANONYMOUS`` and +RAM-based file mappings (``tmpfs``, ``memfd``). Passing it to other +types of mapping will result in ``-EINVAL`` returned by these system +calls. + +**Note**: The ``PROT_MTE`` flag (and corresponding memory type) cannot +be cleared by ``mprotect()``. If this is desirable, ``munmap()`` +(followed by ``mmap()``) must be used. + +Tag Check Faults +---------------- + +When ``PROT_MTE`` is enabled on an address range and a mismatch between +the logical and allocation tags occurs on access, there are three +configurable behaviours: + +- *Ignore* - This is the default mode. The CPU (and kernel) ignores the + tag check fault. + +- *Synchronous* - The kernel raises a ``SIGSEGV`` synchronously, with + ``.si_code = SEGV_MTESERR`` and ``.si_addr = ``. The + memory access is not performed. + +- *Asynchronous* - The kernel raises a ``SIGSEGV``, in the current + thread, asynchronously following one or multiple tag check faults, + with ``.si_code = SEGV_MTEAERR`` and ``.si_addr = 0``. + +**Note**: There are no *match-all* logical tags available for user +applications. + +The user can select the above modes, per thread, using the +``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where +``flags`` contain one of the following values in the ``PR_MTE_TCF_MASK`` +bit-field: + +- ``PR_MTE_TCF_NONE`` - *Ignore* tag check faults +- ``PR_MTE_TCF_SYNC`` - *Synchronous* tag check fault mode +- ``PR_MTE_TCF_ASYNC`` - *Asynchronous* tag check fault mode + +Tag checking can also be disabled for a user thread by setting the +``PSTATE.TCO`` bit with ``MSR TCO, #1``. + +**Note**: Signal handlers are always invoked with ``PSTATE.TCO = 0``, +irrespective of the interrupted context. + +**Note**: Kernel accesses to user memory (e.g. ``read()`` system call) +do not generate a tag check fault. + +Excluding Tags in the ``IRG``, ``ADDG`` and ``SUBG`` instructions +----------------------------------------------------------------- + +The architecture allows excluding certain tags to be randomly generated +via the ``GCR_EL1.Exclude`` register bit-field. This can be configured, +per thread, using the ``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` +system call where ``flags`` contains the exclusion bitmap in the +``PR_MTE_EXCL_MASK`` bit-field. + +Example of correct usage +======================== + +*MTE Example code* + +.. code-block:: c + + /* + * To be compiled with -march=armv8.5-a+memtag + */ + #include + #include + #include + #include + #include + #include + #include + + /* + * From arch/arm64/include/uapi/asm/hwcap.h + */ + #define HWCAP2_MTE (1 << 10) + + /* + * From arch/arm64/include/uapi/asm/mman.h + */ + #define PROT_MTE 0x20 + + /* + * From include/uapi/linux/prctl.h + */ + #define PR_SET_TAGGED_ADDR_CTRL 55 + #define PR_GET_TAGGED_ADDR_CTRL 56 + # define PR_TAGGED_ADDR_ENABLE (1UL << 0) + # define PR_MTE_TCF_SHIFT 1 + # define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) + # define PR_MTE_EXCL_SHIFT 3 + # define PR_MTE_EXCL_MASK (0xffffUL << PR_MTE_EXCL_SHIFT) + + /* + * Insert a random logical tag into the given pointer. + */ + #define insert_random_tag(ptr) ({ \ + __u64 __val; \ + asm("irg %0, %1" : "=r" (__val) : "r" (ptr)); \ + __val; \ + }) + + /* + * Set the allocation tag on the destination address. + */ + #define set_tag(tag, addr) do { \ + asm volatile("stg %0, [%1]" : : "r" (tag), "r" (addr)); \ + } while (0) + + int main() + { + unsigned long *a; + unsigned long page_sz = getpagesize(); + unsigned long hwcap2 = getauxval(AT_HWCAP2); + + /* check if MTE is present */ + if (!(hwcap2 & HWCAP2_MTE)) + return -1; + + /* + * Enable the tagged address ABI, synchronous MTE tag check faults and + * exclude tag 0 from the randomly generated set. + */ + if (prctl(PR_SET_TAGGED_ADDR_CTRL, + PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | (1 << PR_MTE_EXCL_SHIFT), + 0, 0, 0)) { + perror("prctl() failed"); + return -1; + } + + a = mmap(0, page_sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (a == MAP_FAILED) { + perror("mmap() failed"); + return -1; + } + + /* + * Enable MTE on the above anonymous mmap. The flag could be passed + * directly to mmap() and skip this step. + */ + if (mprotect(a, page_sz, PROT_READ | PROT_WRITE | PROT_MTE)) { + perror("mprotect() failed"); + return -1; + } + + /* access with the default tag (0) */ + a[0] = 1; + a[1] = 2; + + printf("a[0] = %lu a[1] = %lu\n", a[0], a[1]); + + /* set the logical and allocation tags */ + a = (unsigned long *)insert_random_tag(a); + set_tag(a, a); + + printf("%p\n", a); + + /* non-zero tag access */ + a[0] = 3; + printf("a[0] = %lu a[1] = %lu\n", a[0], a[1]); + + /* + * If MTE is enabled correctly the next instruction will generate an + * exception. + */ + printf("Expecting SIGSEGV...\n"); + a[2] = 0xdead; + + /* this should not be printed in the PR_MTE_TCF_SYNC mode */ + printf("...done\n"); + + return 0; + }