From patchwork Wed Jul 6 08:59:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 12907613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91C36CCA47C for ; Wed, 6 Jul 2022 08:59:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 316388E0003; Wed, 6 Jul 2022 04:59:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C5FC6B0074; Wed, 6 Jul 2022 04:59:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18E978E0003; Wed, 6 Jul 2022 04:59:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 090E76B0073 for ; Wed, 6 Jul 2022 04:59:39 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id CAA51120C01 for ; Wed, 6 Jul 2022 08:59:38 +0000 (UTC) X-FDA: 79656076836.04.323C570 Received: from out30-43.freemail.mail.aliyun.com (out30-43.freemail.mail.aliyun.com [115.124.30.43]) by imf23.hostedemail.com (Postfix) with ESMTP id AB63A140014 for ; Wed, 6 Jul 2022 08:59:37 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=33;SR=0;TI=SMTPD_---0VIXd7l-_1657097967; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VIXd7l-_1657097967) by smtp.aliyun-inc.com; Wed, 06 Jul 2022 16:59:29 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: rppt@linux.ibm.com, willy@infradead.org, will@kernel.org, aneesh.kumar@linux.ibm.com, npiggin@gmail.com, peterz@infradead.org, catalin.marinas@arm.com, chenhuacai@kernel.org, kernel@xen0n.name, tsbogend@alpha.franken.de, dave.hansen@linux.intel.com, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, arnd@arndb.de, guoren@kernel.org, monstr@monstr.eu, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, shorne@gmail.com, baolin.wang@linux.alibaba.com, x86@kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-csky@vger.kernel.org, openrisc@lists.librecores.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] mm: Add kernel PTE level pagetable pages account Date: Wed, 6 Jul 2022 16:59:17 +0800 Message-Id: <398ead25695e530f766849be5edafaf62c1c864d.1657096412.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657097978; a=rsa-sha256; cv=none; b=x0KzpjFY669OIL9rkIyrfAVeEYeMXASFnHLtnsIA3+6/1C7wMa7kNeiB0zsxjlaZwTNAVL Q6OO8Lkciq3ifGmiQVuHVF/uzXvWwaOQOl+3+2BN65zBOe8hAXAAbCyD3aWzX1ImPMTZRQ bpKsjS7v/n1hUb5HwjpWUmUbnWW5Eho= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.43 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657097978; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to:in-reply-to: references:references:references; bh=V1sF18Otxk1pJCFyw2S8lFHKn6XclY/sq7caOiaZ+1E=; b=WxlRlaSfiEc1knIssypqR85beAAwxGRUhRMaAPbCVMjqqB4X9qAy/vBtWZJjZJyOB806NY h/nt5930QjFWoXdiSP3+3+heU/HZAHQ1T9RS0Vg59x3fGV8LQMOJbwONb5HLjJBQ455ljV V2+jgW7+idB27oEPTNeh8UxjeohJiNc= Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.43 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: kd7iidnrw9hef5jjmmz757fmihhq79tk X-Rspamd-Queue-Id: AB63A140014 X-HE-Tag: 1657097977-924211 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now the kernel PTE level ptes are always protected by mm->page_table_lock instead of split pagetable lock, so the kernel PTE level pagetable pages are not accounted. Especially the vmalloc()/vmap() can consume lots of kernel pagetable, so to get an accurate pagetable accounting, calling new helpers page_{set,clear}_pgtable() when allocating or freeing a kernel PTE level pagetable page. Meanwhile converting architectures to use corresponding generic PTE pagetable allocation and freeing functions. Note this patch only adds accounting to the page tables allocated after boot. Signed-off-by: Baolin Wang Reported-by: kernel test robot --- arch/csky/include/asm/pgalloc.h | 2 +- arch/microblaze/mm/pgtable.c | 2 +- arch/openrisc/mm/ioremap.c | 2 +- arch/x86/mm/pgtable.c | 2 +- include/asm-generic/pgalloc.h | 14 ++++++++++++-- 5 files changed, 16 insertions(+), 6 deletions(-) diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index 7d57e5d..56f8d25 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; unsigned long i; - pte = (pte_t *) __get_free_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); if (!pte) return NULL; diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c index 9f73265..e96dd1b 100644 --- a/arch/microblaze/mm/pgtable.c +++ b/arch/microblaze/mm/pgtable.c @@ -245,7 +245,7 @@ unsigned long iopa(unsigned long addr) __ref pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { if (mem_init_done) - return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pte_alloc_one_kernel(mm); else return memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, MEMBLOCK_LOW_LIMIT, diff --git a/arch/openrisc/mm/ioremap.c b/arch/openrisc/mm/ioremap.c index daae13a..3453acc 100644 --- a/arch/openrisc/mm/ioremap.c +++ b/arch/openrisc/mm/ioremap.c @@ -118,7 +118,7 @@ pte_t __ref *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; if (likely(mem_init_done)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); } else { pte = memblock_alloc(PAGE_SIZE, PAGE_SIZE); if (!pte) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index ea39670..20f3076 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -858,7 +858,7 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) /* INVLPG to clear all paging-structure caches */ flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); - free_page((unsigned long)pte); + pte_free_kernel(NULL, pte); return 1; } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 8ce8d7c..cd8420f 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -18,7 +18,14 @@ */ static inline pte_t *__pte_alloc_one_kernel(struct mm_struct *mm) { - return (pte_t *)__get_free_page(GFP_PGTABLE_KERNEL); + struct page *page; + gfp_t gfp = GFP_PGTABLE_KERNEL; + + page = alloc_pages(gfp, 0); + if (!page) + return NULL; + page_set_pgtable(page); + return (pte_t *)page_address(page); } #ifndef __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL @@ -41,7 +48,10 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) */ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - free_page((unsigned long)pte); + struct page *page = virt_to_page(pte); + + page_clear_pgtable(page); + __free_page(page); } /**