From patchwork Mon Nov 22 05:28:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhaoyang Huang X-Patchwork-Id: 12631237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92F2FC433EF for ; Mon, 22 Nov 2021 05:29:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCF336B0071; Mon, 22 Nov 2021 00:29:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C7E946B0072; Mon, 22 Nov 2021 00:29:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B6DF26B0073; Mon, 22 Nov 2021 00:29:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id A71CD6B0071 for ; Mon, 22 Nov 2021 00:29:21 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5D243181DA0A2 for ; Mon, 22 Nov 2021 05:29:11 +0000 (UTC) X-FDA: 78835437702.21.14C9849 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf13.hostedemail.com (Postfix) with ESMTP id D91971052CBE for ; Mon, 22 Nov 2021 05:29:08 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id u11so13104998plf.3 for ; Sun, 21 Nov 2021 21:29:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id; bh=uPLgIBlHzlDQAOxeK/1rgfm4GBYhPTkogYnAxbthSW4=; b=amL2AcIf4MijIAxFlA4zp8S+pih7hjHTs+2DPv7zcdL6CSlJvsWw9D/BbIkvgnJQPP EC2dZNRKYEwtlxQSjemxgTb42pAN67c7IBWHvvmBYvc2+cDIk889jYsCtF5vD2+l5COR vHINg8Jqj4lZUuHNw7WXZ+uYI9miAmX5jhOC1SnTM+QFCypuIuCVTQ+R1ilGkt5Bv7dh 9uBiKm+SWdLG/fVCRGpeXG+PNIcxMD6jWbvh80/4PVuSlhD0cCc+3LiqEzdrj35/+j1z vnwEmhyKpRJi5lQXijSjvsCTPliOtt3ZhkL/pKZH05v3fzAHVmBAtTFw3But/sJp6mDL 3pDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id; bh=uPLgIBlHzlDQAOxeK/1rgfm4GBYhPTkogYnAxbthSW4=; b=UG8O7p5gRWaIVPJrokMHTRBQEGUrBBwSiXiqwTvSiFhJ87TzM8iQrk79d9tNj2rtaP E6Efv4LxGCiS2qnvnBqB1mLAKknRiWIegwYzOnesVGXrkwO9rqiesUTxRySoUIxcKBWi 1O6gSsGs4E22e6lIpKTUl9a2meolnolaprcNiFTSV0X+V3Rc3b0Yg80HMO/bSt94e1PR uZMlKhEiSJWQJAV8s0L8nHbvC2VQM20URSeEkgnxx8oBjZ/vrY6O8oEuQ53zHwHFBFS7 1o5DbZ2lgDRXS9MJ+0moJKRcvboZlgHgwewN+/GLGTuz9kabVnXFP+2QahGGGd8hR9/Z 5xwQ== X-Gm-Message-State: AOAM530uv4CODyDBmgn4ti+yG4w0rPLZ2o7uCGYhHf/2hPt2KTk/ENWS k2RXo4dfaUTD91M9qro9FcA= X-Google-Smtp-Source: ABdhPJwzZQByaR18uPu69Kq/3ouZzOAOpUx9v+kDWevWfW8GYWYxEoicSE+M8rNxgkhr890XjMhybQ== X-Received: by 2002:a17:902:e405:b0:141:b2fa:b00 with SMTP id m5-20020a170902e40500b00141b2fa0b00mr103683808ple.22.1637558949950; Sun, 21 Nov 2021 21:29:09 -0800 (PST) Received: from bj03382pcu.spreadtrum.com ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id g21sm7592725pfc.95.2021.11.21.21.29.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 21 Nov 2021 21:29:09 -0800 (PST) From: Huangzhaoyang To: Ard Biesheuvel , Catalin Marinas , Will Deacon , Anshuman Khandual , Andrew Morton , Nicholas Piggin , Mike Rapoport , Pavel Tatashin , Christophe Leroy , Jonathan Marek , Zhaoyang Huang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH] arch: arm64: introduce RODATA_FULL_USE_PTE_CONT Date: Mon, 22 Nov 2021 13:28:49 +0800 Message-Id: <1637558929-22971-1-git-send-email-huangzhaoyang@gmail.com> X-Mailer: git-send-email 1.7.9.5 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D91971052CBE X-Stat-Signature: yjsmyet69s14ikf34oyugnp51i438r51 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=amL2AcIf; spf=pass (imf13.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1637558948-177483 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhaoyang Huang Kernel linear mapping will be split to the smallest granularity when RODATA_FULL applied, which could lead to TLB pressure. Introduce a method to apply PTE_CONT on pte. Signed-off-by: Zhaoyang Huang --- arch/arm64/Kconfig | 9 +++++++++ arch/arm64/mm/mmu.c | 10 ++++++++-- arch/arm64/mm/pageattr.c | 9 +++++++++ 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index fee914c..3f8fbf0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1198,6 +1198,15 @@ config RODATA_FULL_DEFAULT_ENABLED This requires the linear region to be mapped down to pages, which may adversely affect performance in some cases. +config RODATA_FULL_USE_PTE_CONT + bool "Apply PTE_CONT when RODATA_FULL_DEFAULT_ENABLED enabled" + depends on RODATA_FULL_DEFAULT_ENABLED + default y + help + Apply PTE_CONT on linear mapping as much as we can when + RODATA_FULL_DEFAULT_ENABLED enabled which could decrease the + impaction on performance by small pte granularity. + config ARM64_SW_TTBR0_PAN bool "Emulate Privileged Access Never using TTBR0_EL1 switching" help diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index cfd9deb..8017b17 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -124,15 +124,21 @@ static bool pgattr_change_is_safe(u64 old, u64 new) * The following mapping attributes may be updated in live * kernel mappings without the need for break-before-make. */ +#ifndef CONFIG_RODATA_FULL_USE_PTE_CONT pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG; +#else + pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG | PTE_CONT; +#endif /* creating or taking down mappings is always safe */ if (old == 0 || new == 0) return true; /* live contiguous mappings may not be manipulated at all */ - if ((old | new) & PTE_CONT) +#ifndef CONFIG_RODATA_FULL_USE_PTE_CONT + if (old | new) & PTE_CONT) return false; +#endif /* Transitioning from Non-Global to Global is unsafe */ if (old & ~new & PTE_NG) @@ -206,7 +212,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, /* use a contiguous mapping if the range is suitably aligned */ if ((((addr | next | phys) & ~CONT_PTE_MASK) == 0) && - (flags & NO_CONT_MAPPINGS) == 0) + (IS_ENABLED(CONFIG_RODATA_FULL_USE_PTE_CONT) || (flags & NO_CONT_MAPPINGS) == 0)) __prot = __pgprot(pgprot_val(prot) | PTE_CONT); init_pte(pmdp, addr, next, phys, __prot); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index a3bacd7..88a87eb 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -99,6 +99,15 @@ static int change_memory_common(unsigned long addr, int numpages, if (rodata_full && (pgprot_val(set_mask) == PTE_RDONLY || pgprot_val(clear_mask) == PTE_RDONLY)) { for (i = 0; i < area->nr_pages; i++) { +#ifdef CONFIG_RODATA_FULL_USE_PTE_CONT + unsigned long cont_pte_low_bound; + unsigned long addr; + + addr = (u64)page_address(area->pages[i]); + cont_pte_low_bound = addr & CONT_PTE_MASK; + __change_memory_common(cont_pte_low_bound, + (~CONT_PTE_MASK + 1), __pgprot(0) , __pgprot(PTE_CONT)); +#endif __change_memory_common((u64)page_address(area->pages[i]), PAGE_SIZE, set_mask, clear_mask); }