From patchwork Wed Jun 22 08:58:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 12890325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C25FCCA47E for ; Wed, 22 Jun 2022 08:59:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 525A88E00A1; Wed, 22 Jun 2022 04:59:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 439478E008A; Wed, 22 Jun 2022 04:59:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20B4C8E00A1; Wed, 22 Jun 2022 04:59:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 105658E008A for ; Wed, 22 Jun 2022 04:59:34 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DD00234168 for ; Wed, 22 Jun 2022 08:59:33 +0000 (UTC) X-FDA: 79605273426.02.0BFBA9B Received: from out30-56.freemail.mail.aliyun.com (out30-56.freemail.mail.aliyun.com [115.124.30.56]) by imf07.hostedemail.com (Postfix) with ESMTP id 034ED4001D for ; Wed, 22 Jun 2022 08:59:27 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R351e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VH55S7r_1655888345; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VH55S7r_1655888345) by smtp.aliyun-inc.com; Wed, 22 Jun 2022 16:59:06 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: rppt@linux.ibm.com, willy@infradead.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH v2 3/3] mm: Add kernel PTE level pagetable pages account Date: Wed, 22 Jun 2022 16:58:54 +0800 Message-Id: <7882bbf467440f9a3ebe41d96ba5b6f384081bb7.1655887440.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655888372; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to:in-reply-to: references:references:references; bh=5WS/921Au9to8/YlGn7HoKpMtcJronokX/y9cxI3cNM=; b=DiHPMKlcB2HeqYauzTOeQUQgTKL0f49nAVSU5TFZf9751emwGhoJ9OlRx1ZC+JCuMUkNrd UFUvXFHW5VP6VhnoJat0eVaUnmMymGN5Fxd4d8Ye8U8GYdUXkSS+SDehyPPqKfD6AIWnDu a0yByo3jgXf7e03k/sIc78zEZHybcLM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=temperror (imf07.hostedemail.com: error in processing during lookup of baolin.wang@linux.alibaba.com: DNS error) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=temperror reason="SPF/DKIM temp error" header.from=alibaba.com (policy=temperror) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655888372; a=rsa-sha256; cv=none; b=rTuDOLFn35P9E6r5Ex7pkmxCkxv1u//7Lx21XQqwTuPrB7VjjXAUnySUVuHqbMkyUq1gqu bz94U9UIMMX0yjXYpqGRIE0DZrr3FrVGnj5Yzw2PFxAYRmwWKT1uA+xMMJGItlLgrEkHXF t896JSgP3dI6aP53PlyLWEr4wf5cKY0= X-Rspamd-Queue-Id: 034ED4001D Authentication-Results: imf07.hostedemail.com; dkim=none; spf=temperror (imf07.hostedemail.com: error in processing during lookup of baolin.wang@linux.alibaba.com: DNS error) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=temperror reason="SPF/DKIM temp error" header.from=alibaba.com (policy=temperror) X-Rspam-User: X-Stat-Signature: 8pmykgxxnyt7brk4nb51w4y5dx58dgh8 X-Rspamd-Server: rspam04 X-HE-Tag: 1655888367-202963 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now the kernel PTE level ptes are always protected by mm->page_table_lock instead of split pagetable lock, so the kernel PTE level pagetable pages are not accounted. Especially the vmalloc()/vmap() can consume lots of kernel pagetable, so to get an accurate pagetable accounting, calling new helpers pgtable_set_and_inc()/pgtable_clear_and_dec() when allocating or freeing a kernel PTE level pagetable page. Meanwhile converting architectures to use corresponding generic PTE pagetable allocation and freeing functions. Signed-off-by: Baolin Wang Reported-by: kernel test robot Reported-by: kernel test robot --- arch/csky/include/asm/pgalloc.h | 2 +- arch/microblaze/mm/pgtable.c | 2 +- arch/openrisc/mm/ioremap.c | 2 +- arch/x86/mm/pgtable.c | 5 ++++- include/asm-generic/pgalloc.h | 14 ++++++++++++-- 5 files changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index bbbd069..2443226 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; unsigned long i; - pte = (pte_t *) __get_free_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); if (!pte) return NULL; diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c index 9f73265..e96dd1b 100644 --- a/arch/microblaze/mm/pgtable.c +++ b/arch/microblaze/mm/pgtable.c @@ -245,7 +245,7 @@ unsigned long iopa(unsigned long addr) __ref pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { if (mem_init_done) - return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pte_alloc_one_kernel(mm); else return memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, MEMBLOCK_LOW_LIMIT, diff --git a/arch/openrisc/mm/ioremap.c b/arch/openrisc/mm/ioremap.c index daae13a..3453acc 100644 --- a/arch/openrisc/mm/ioremap.c +++ b/arch/openrisc/mm/ioremap.c @@ -118,7 +118,7 @@ pte_t __ref *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; if (likely(mem_init_done)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); } else { pte = memblock_alloc(PAGE_SIZE, PAGE_SIZE); if (!pte) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index a8ab3f9..fc2b9ef 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -851,6 +851,7 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; + struct page *page; pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); @@ -858,7 +859,9 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) /* INVLPG to clear all paging-structure caches */ flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); - free_page((unsigned long)pte); + page = virt_to_page(pte); + pgtable_clear_and_dec(page); + __free_page(page); return 1; } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 328a714..2e20e9e 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -18,7 +18,14 @@ */ static inline pte_t *__pte_alloc_one_kernel(struct mm_struct *mm) { - return (pte_t *)__get_free_page(GFP_PGTABLE_KERNEL); + struct page *page; + gfp_t gfp = GFP_PGTABLE_KERNEL; + + page = alloc_pages(gfp & ~__GFP_HIGHMEM, 0); + if (!page) + return NULL; + pgtable_set_and_inc(page); + return (pte_t *)page_address(page); } #ifndef __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL @@ -41,7 +48,10 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) */ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - free_page((unsigned long)pte); + struct page *page = virt_to_page(pte); + + pgtable_clear_and_dec(page); + __free_page(page); } /**