From patchwork Fri May 15 17:15:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 11552685 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60ED6739 for ; Fri, 15 May 2020 17:16:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 353FD20671 for ; Fri, 15 May 2020 17:16:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 353FD20671 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B7138E000D; Fri, 15 May 2020 13:16:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 23F5D8E0001; Fri, 15 May 2020 13:16:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0979C8E000D; Fri, 15 May 2020 13:16:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0022.hostedemail.com [216.40.44.22]) by kanga.kvack.org (Postfix) with ESMTP id E22ED8E0001 for ; Fri, 15 May 2020 13:16:39 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9C4692489E for ; Fri, 15 May 2020 17:16:39 +0000 (UTC) X-FDA: 76819607718.21.flock21_9168e88559156 X-Spam-Summary: 2,0,0,177d92b8e00ec453,d41d8cd98f00b204,catalin.marinas@arm.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1534:1543:1711:1730:1747:1777:1792:2393:2538:2559:2562:2693:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:4321:4605:5007:6117:6119:6261:7576:7875:8603:8634:10004:11026:11232:11473:11657:11658:11914:12043:12297:12438:12555:12679:12986:13180:13229:13894:14096:14181:14394:14721:21080:21230:21451:21627:30003:30054:30067:30079,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:29,LUA_SUMMARY:none X-HE-Tag: flock21_9168e88559156 X-Filterd-Recvd-Size: 4609 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Fri, 15 May 2020 17:16:39 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 735701063; Fri, 15 May 2020 10:16:38 -0700 (PDT) Received: from localhost.localdomain (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D68D93F305; Fri, 15 May 2020 10:16:36 -0700 (PDT) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, Will Deacon , Dave P Martin , Vincenzo Frascino , Szabolcs Nagy , Kevin Brodsky , Andrey Konovalov , Peter Collingbourne Subject: [PATCH v4 08/26] arm64: mte: Tags-aware copy_page() implementation Date: Fri, 15 May 2020 18:15:54 +0100 Message-Id: <20200515171612.1020-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200515171612.1020-1-catalin.marinas@arm.com> References: <20200515171612.1020-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino When the Memory Tagging Extension is enabled, the tags need to be preserved across page copy (e.g. for copy-on-write). Introduce MTE-aware copy_page() which preserves the tags across page copy. Signed-off-by: Vincenzo Frascino Co-developed-by: Catalin Marinas Signed-off-by: Catalin Marinas Cc: Will Deacon --- Notes: v4: - Moved the tag copying to a separate function in mte.S and only called if the source page has the PG_mte_tagged flag set. arch/arm64/include/asm/mte.h | 4 ++++ arch/arm64/lib/mte.S | 19 +++++++++++++++++++ arch/arm64/mm/copypage.c | 14 ++++++++++++-- 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 4310a7ff10c0..c1a09499c678 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -19,6 +19,7 @@ void mte_clear_page_tags(void *addr, size_t size); #define PG_mte_tagged PG_arch_2 void mte_sync_tags(pte_t *ptep, pte_t pte); +void mte_copy_page_tags(void *kto, const void *kfrom); void flush_mte_state(void); #else @@ -29,6 +30,9 @@ void flush_mte_state(void); static inline void mte_sync_tags(pte_t *ptep, pte_t pte) { } +static inline void mte_copy_page_tags(void *kto, const void *kfrom) +{ +} static inline void flush_mte_state(void) { } diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index 130fb7047e17..a531b52fa5ba 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -5,6 +5,7 @@ #include #include +#include .arch armv8.5-a+memtag @@ -21,3 +22,21 @@ SYM_FUNC_START(mte_clear_page_tags) cbnz x1, 1b ret SYM_FUNC_END(mte_clear_page_tags) + +/* + * Copy the tags from the source page to the destination one + * x0 - address of the destination page + * x1 - address of the source page + */ +SYM_FUNC_START(mte_copy_page_tags) + mov x2, x0 + mov x3, x1 + multitag_transfer_size x5, x6 +1: ldgm x4, [x3] + stgm x4, [x2] + add x2, x2, x5 + add x3, x3, x5 + tst x2, #(PAGE_SIZE - 1) + b.ne 1b +2: +SYM_FUNC_END(mte_copy_page_tags) diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c index 2ee7b73433a5..2560ddc479ac 100644 --- a/arch/arm64/mm/copypage.c +++ b/arch/arm64/mm/copypage.c @@ -6,16 +6,26 @@ * Copyright (C) 2012 ARM Ltd. */ +#include #include #include #include +#include +#include void __cpu_copy_user_page(void *kto, const void *kfrom, unsigned long vaddr) { - struct page *page = virt_to_page(kto); + struct page *to_page = virt_to_page(kto); + struct page *from_page = virt_to_page(kfrom); + copy_page(kto, kfrom); - flush_dcache_page(page); + if (system_supports_mte() && + test_bit(PG_mte_tagged, &from_page->flags)) { + mte_copy_page_tags(kto, kfrom); + set_bit(PG_mte_tagged, &to_page->flags); + } + flush_dcache_page(to_page); } EXPORT_SYMBOL_GPL(__cpu_copy_user_page);