From patchwork Mon Dec 18 10:50:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13496573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EF43C35274 for ; Mon, 18 Dec 2023 10:51:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C38D56B00A3; Mon, 18 Dec 2023 05:51:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE89B6B00A4; Mon, 18 Dec 2023 05:51:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB0C16B00A5; Mon, 18 Dec 2023 05:51:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9C3616B00A3 for ; Mon, 18 Dec 2023 05:51:53 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 754A71A0D08 for ; Mon, 18 Dec 2023 10:51:53 +0000 (UTC) X-FDA: 81579623706.15.8DFD84F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf07.hostedemail.com (Postfix) with ESMTP id B245640018 for ; Mon, 18 Dec 2023 10:51:51 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702896711; a=rsa-sha256; cv=none; b=VNbB5q8KToskOCeV0GNd5SDRC5m7+P20u6He7Hw3WWyYZp7EdXM/n5GFXw6jWrhOvkNrEX VVmeIr+JggJOqq31tmSGd19VykuKLu3cZIyXoh00RfSH2tN0p8HQFm79Ae0CSxFqRMvqP+ 4/h0flInFQaXaW7vqJ3NwzZtxj3DKk4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702896711; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=80eWkD1e3ScDQ5SjUCye7wOJgIG+FP4dY1bE4B1pVFU=; b=fnUtxoJyn6YfHVIQ6knZYs2jx5ipEim8BEbnqvq3f66+roWPh6bgnDIeaTvUfYe9WqNU4Y tgmipr5oJaS4bmfI7XN5OHlbr27gElRx0UmraRT912wExVZ/0LWNiP+hazAmDAGJk15Q8m SeJ/t2oPf561O/nmTKP3IS0Q+Y7QVDQ= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 843712F4; Mon, 18 Dec 2023 02:52:35 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A89773F738; Mon, 18 Dec 2023 02:51:47 -0800 (PST) From: Ryan Roberts To: Catalin Marinas , Will Deacon , Ard Biesheuvel , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , Anshuman Khandual , Matthew Wilcox , Yu Zhao , Mark Rutland , David Hildenbrand , Kefeng Wang , John Hubbard , Zi Yan , Barry Song <21cnbao@gmail.com>, Alistair Popple , Yang Shi Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 09/16] arm64/mm: ptep_clear_flush_young(): New layer to manage contig bit Date: Mon, 18 Dec 2023 10:50:53 +0000 Message-Id: <20231218105100.172635-10-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231218105100.172635-1-ryan.roberts@arm.com> References: <20231218105100.172635-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B245640018 X-Stat-Signature: 3ojx1nf74sam5177hrktmc1mwpb4tpjq X-HE-Tag: 1702896711-429556 X-HE-Meta: U2FsdGVkX1+gjxteIXBm7tpB/Qq0tYeTE/Js+33YXOZJSZ4J4+ka7KvWlKa2C4Wc86Lrb/nHkHxES5DsSht3+cz0q+D0D0uMk323Imrr6M5c+WYjYrIearBtjBfWMG6cmHYBnRO1Xos+jteeXpwJBMlHfX+lHy2v3LI5USvqvnk6Kn0x7KW3vzK2nVD94eexQL3Ffyv56IXpKGcsctVTEebZQWydXb7mkJejClM0prUF2VL8Q4h6MwIljgVdofI9UWRVuNv/N89MH+7MxSw5pvlEEPNAQTbgqSNaFsaPLfjzBOWmynAfSWBxUTGf8waSu2wdNKUawvDyyPR/fcLUA+lwtUSxtLaJNEKaCHLmrygDxGqZuiJAeA7SR6j7XfKIPyA7f/vu14KfGbqM675JsUzkkhr3OjFcW/o9eZtsex7/GWEcRqqSpXdI6okjnTpXsc++3J7Nn07bGMN34lNyDw+7S62gVFXIvn3VNW2q9WmKEdOaGTgWy0QeR+K1rw1692B84JlLxgC1jx5LLc6C2SEAmln9JjfImu2S983a3M7X5U2ggWw2Am56IG7GuYOVAJz67VxO3glRIFEf/nZNgmEJOg+YdXLoz+wakKmOBYlZ9vYWJ3r6BPmxKA8B6Pt4NSvsXYJhlL8Dg5vBzdOF+FfdERZjhJbVLaL48uCyEkCx8o9oQs55nRfOlpemtfMDS5MwQ8vouM+adAQvjMFBQMlhuua4bbddBuRczZ9tIkT1VPVk6PdT/7tkt9BYTUcFybP9JBZqGqGRpS1VV56UQedDJrbjP35L6YVoRxLDSP5bi2goQyjAW8s56jAAZq/XMx07dUIpiVipGoEMvHkLG7WzFofBPfGVBSp35IeuEgkYCvxD2/5uWpwxB8hTfAZm8MpLahoOwAmOUpHDAhATy1F9Z2jg0BD+NHG8QVnMDsEQcwc20IgI29MaJ1wh+VyA4+hU+lj+e0FO1AuwxqQ 6i/7BaMy p17jylGGFBYhrraXQelWkbovg6+YCM3jNtP9p+dRvq/G/OdZOtYZyukJUWN4/gBzksUgR7H7ACV2ohHFfxhUgBxqwT5WNYJb1EPAxGsxF2Jns2cz3PObFe3+3dMCiMFETdUG5CPwYPP2e2Bl1a2w0cN8zlWJMH49bdeeMt0ILOFCAcRP/A/Wl1ZtoqwhCRxy5JvMffrqt72e9tSHnjURMTH5/bM07S5j3qDuKS27J2DSXhz9PveJLYtmt0fdpwLjm0CQU1bsQc6+KGBhnC661jYr5hJ5DeuoegnL6zmBqDCSjN/4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Create a new layer for the in-table PTE manipulation APIs. For now, The existing API is prefixed with double underscore to become the arch-private API and the public API is just a simple wrapper that calls the private API. The public API implementation will subsequently be used to transparently manipulate the contiguous bit where appropriate. But since there are already some contig-aware users (e.g. hugetlb, kernel mapper), we must first ensure those users use the private API directly so that the future contig-bit manipulations in the public API do not interfere with those existing uses. Tested-by: John Hubbard Signed-off-by: Ryan Roberts --- arch/arm64/include/asm/pgtable.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 9b4a9909fd5b..fc1005222ee4 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -138,7 +138,7 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys) * so that we don't erroneously return false for pages that have been * remapped as PROT_NONE but are yet to be flushed from the TLB. * Note that we can't make any assumptions based on the state of the access - * flag, since ptep_clear_flush_young() elides a DSB when invalidating the + * flag, since __ptep_clear_flush_young() elides a DSB when invalidating the * TLB. */ #define pte_accessible(mm, pte) \ @@ -904,8 +904,7 @@ static inline int __ptep_test_and_clear_young(struct vm_area_struct *vma, return pte_young(pte); } -#define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH -static inline int ptep_clear_flush_young(struct vm_area_struct *vma, +static inline int __ptep_clear_flush_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep) { int young = __ptep_test_and_clear_young(vma, address, ptep); @@ -1119,6 +1118,8 @@ extern void ptep_modify_prot_commit(struct vm_area_struct *vma, #define ptep_get_and_clear __ptep_get_and_clear #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG #define ptep_test_and_clear_young __ptep_test_and_clear_young +#define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH +#define ptep_clear_flush_young __ptep_clear_flush_young #endif /* !__ASSEMBLY__ */