From patchwork Mon Mar 17 14:16:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EBA4C28B30 for ; Mon, 17 Mar 2025 14:21:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1E3B280004; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA6D1280001; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2112280004; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B2FE1280001 for ; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 828AE55C61 for ; Mon, 17 Mar 2025 14:21:44 +0000 (UTC) X-FDA: 83231256528.30.D20B075 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 3094F40021 for ; Mon, 17 Mar 2025 14:21:40 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FkrrTiKic5g/SkGADpTJcYzWJvDsKPZ0Ua0I8vQ0aUw=; b=MzlcFMEwq1fyhE0yQk3klyEutCBAOdo1MJwWCP7jupp7IFZ1deLTEjCXKkbX1WhS2ms+k6 6wmblhqfZiDLOnCGKaJD/vBIE/XgfIjJ7gj2bOGvh1B+/bpMJEHoNE1tfqa+NtR1dBc7VI khVeU5ZpYXrAgLZNoLqF/gE/FODbWkE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221301; a=rsa-sha256; cv=none; b=CglAHESkp1G/ooN5yDicBAned/LWmF0ReK2yXJE2rBGzMAx+HBzNIoodIGpFDi1Ijnp1pi mRG1itMkNYrBkE8bDE+ve9EGJA8EnOSMg4ZVzEOUKtFzS8gT5CH78xFwIDFILGdIHXyI6S ejhed4pAVUUKApwGMPtDgUMCk07/G/4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CF25D22FA; Mon, 17 Mar 2025 07:21:48 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A78843F63F; Mon, 17 Mar 2025 07:21:35 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 02/11] mm: Call ctor/dtor for kernel PTEs Date: Mon, 17 Mar 2025 14:16:51 +0000 Message-ID: <20250317141700.3701581-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3094F40021 X-Stat-Signature: tnijosxkt4fzow1a9kiwokfieus9nuex X-HE-Tag: 1742221300-116014 X-HE-Meta: U2FsdGVkX18mJXJ5qHIYQqhtpizedAix/dUCwDjxnbZp6gH3icRfN77Cc0vjIU4Rr32xZ9a9q7ev7u7z0tll8NKfee0FkAXhsR9DEaBryWod1Kot4bYPSrhatLw+uIfsJn1bi5TMYiDPZ7AnWxQCBqMp1u+0ISmaZIPIiDMW7am1FzwxG6YU0eRIP27icrlIpztJABZ/Yq4DsM5UtNA9rvINEgcFi8Z5+6r0rGKTZwGUYg+vUChkSF3hIViMfpCks+7AD77x12875fo8Sjs26hUcHC1Ii3Qsn1DNKQNonetSiBP8FD080vk3o3WQ+m82fNfGJApLUCTsPYc81zVnne0UiKFizX2NtSEZCT6kcAOs7YBozkhmbM/RisZKf2I0AbWAsUPRk/h7yk8btzwTHEja2ZndjcQS23oeh4Bda6LFyFb+LBkFebGC3lJyPSBlggo4YyHqHiKFw+GHuQFfzPWoDFB8eXvoAakFQ/yHstzBqE3tPh95zpaiVU9nXJpY6X0Y1p8DMM9mFhdqWprRxK2N4AsXuMehvIoO6t9yx5by3NCMKFy35eiHAd7Gf0LOdtQsGdBMtdpUNA/7NxGysDJn9DrEILn8f6izDovOp/Q2QQZR9+kihPfet9lhZe9w+n2piVvwGgnJG1ZDXU63zwcbOq0Ciz1yN2y86ms/HCKc6CP0u4xzd4nrwgctclNONnO4U6ELWdUn0SJsZNf7YM1eJEm4XYa2y1jv+CF6opI3dc1bp6t/c4hmrnQOYNQhhKPUks0Q7O48OKY637ZhU5a/UvvnX22znNCZONV/HALKTCieTfMW+YvtdJLRI/Mhlz0TZwPmPTeVpXqdJGwP+SAqXtfoF/WNp8JiEUuFhMZmmc4o21oqwG98u66Xv+XIdiUgq9OJZrSOR5E/yxbeINivKc2AQ2Xtkok1pIuTa0mcFZbfT9fUIpfpx2F68oAbB9kUmeDi8VxLGhpyssw 6Z9TnVmz s3v02ZtX0qWvDLuYKBuBxY5SDsFC6ob357abXW0YkYuu+z2+qmudCQPZEcmsHYAXRu9iluL4NQ2rtKmOaYYCbBhaaghGvlTG/zLxxMC8zwa9QuYi/NcIPIw658iRZvAcDy2JIXIpwcmuS5J3qBPdMFE/7RajBbB6GeosqtFh8VIFjhOR3flGA6lxvaw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since [1], constructors/destructors are expected to be called for all page table pages, at all levels and for both user and kernel pgtables. There is however one glaring exception: kernel PTEs are managed via separate helpers (pte_alloc_kernel/pte_free_kernel), which do not call the [cd]tor, at least not in the generic implementation. The most obvious reason for this anomaly is that init_mm is special-cased not to use split page table locks. As a result calling ptlock_init() for PTEs associated with init_mm would be wasteful, potentially resulting in dynamic memory allocation. However, pgtable [cd]tors perform other actions - currently related to accounting/statistics, and potentially more functionally significant in the future. Now that pagetable_pte_ctor() is passed the associated mm, we can make it skip the call to ptlock_init() for init_mm; this allows us to call the ctor from pte_alloc_one_kernel() too. This is matched by a call to the pgtable destructor in pte_free_kernel(); no special-casing is needed on that path, as ptlock_free() is already called unconditionally. (ptlock_free() is a no-op unless a ptlock was allocated for the given PTP.) This patch ensures that all architectures that rely on call the [cd]tor for kernel PTEs. pte_free_kernel() cannot be overridden so changing the generic implementation is sufficient. pte_alloc_one_kernel() can be overridden using __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL, and a few architectures implement it by calling the page allocator directly. We amend those so that they call the generic __pte_alloc_one_kernel() instead, if possible, ensuring that the ctor is called. A few architectures do not use ; those will be taken care of separately. [1] https://lore.kernel.org/linux-mm/20250103184415.2744423-1-kevin.brodsky@arm.com/ Signed-off-by: Kevin Brodsky --- arch/csky/include/asm/pgalloc.h | 2 +- arch/microblaze/mm/pgtable.c | 2 +- arch/openrisc/mm/ioremap.c | 2 +- include/asm-generic/pgalloc.h | 7 ++++++- include/linux/mm.h | 2 +- 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index bf8400c28b5a..288dca0d160a 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; unsigned long i; - pte = (pte_t *) __get_free_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); if (!pte) return NULL; diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c index 9f73265aad4e..e96dd1b7aba4 100644 --- a/arch/microblaze/mm/pgtable.c +++ b/arch/microblaze/mm/pgtable.c @@ -245,7 +245,7 @@ unsigned long iopa(unsigned long addr) __ref pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { if (mem_init_done) - return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pte_alloc_one_kernel(mm); else return memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, MEMBLOCK_LOW_LIMIT, diff --git a/arch/openrisc/mm/ioremap.c b/arch/openrisc/mm/ioremap.c index 8e63e86251ca..3b352f97fecb 100644 --- a/arch/openrisc/mm/ioremap.c +++ b/arch/openrisc/mm/ioremap.c @@ -36,7 +36,7 @@ pte_t __ref *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; if (likely(mem_init_done)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); } else { pte = memblock_alloc_or_panic(PAGE_SIZE, PAGE_SIZE); } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index e164ca66f0f6..3c8ec3bfea44 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -23,6 +23,11 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm) if (!ptdesc) return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } + return ptdesc_address(ptdesc); } #define __pte_alloc_one_kernel(...) alloc_hooks(__pte_alloc_one_kernel_noprof(__VA_ARGS__)) @@ -48,7 +53,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm) */ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - pagetable_free(virt_to_ptdesc(pte)); + pagetable_dtor_free(virt_to_ptdesc(pte)); } /** diff --git a/include/linux/mm.h b/include/linux/mm.h index d92c16f6cfa2..ee31ffd7ead2 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3018,7 +3018,7 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) static inline bool pagetable_pte_ctor(struct mm_struct *mm, struct ptdesc *ptdesc) { - if (!ptlock_init(ptdesc)) + if (mm != &init_mm && !ptlock_init(ptdesc)) return false; __pagetable_ctor(ptdesc); return true;