From patchwork Mon Aug 7 23:04:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13345287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6A78C04A94 for ; Mon, 7 Aug 2023 23:05:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72E62900002; Mon, 7 Aug 2023 19:05:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DC30940008; Mon, 7 Aug 2023 19:05:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57DF4900003; Mon, 7 Aug 2023 19:05:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 455FF900002 for ; Mon, 7 Aug 2023 19:05:27 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1FA3C160598 for ; Mon, 7 Aug 2023 23:05:27 +0000 (UTC) X-FDA: 81098841894.02.C936BC2 Received: from mail-yb1-f178.google.com (mail-yb1-f178.google.com [209.85.219.178]) by imf17.hostedemail.com (Postfix) with ESMTP id 4989940014 for ; Mon, 7 Aug 2023 23:05:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=L1Go0rwX; spf=pass (imf17.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.219.178 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691449525; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CHNVeLQ4ERi2aiC2atfDR7Ra/O6uVctCEHn/pbNh+mQ=; b=rz8H1xPB2aRt43ghLUmI9j14Qd7iIw/t0TOx8EqBthrYVT7y9VHSg9CROMRN9dL+FnBmsc jD8QPstWhuI1ORk18l3j0f5ANf4PEFYuraE32GO19Tyfbz8sWvcU1IHHw2coZuUSJjff2B XWFs19f3WgA9vqf9OlBVcg4OPyEu3Bw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691449525; a=rsa-sha256; cv=none; b=5FAHG3yc+K+1YLJvt91ngjJDlvZL60+Kd0CqqZELphZGgm8wJaCEIqqd2SDCRY3W8ROpeq 4fYjhp8wxmVhLnSsMu68P4GBP3we4YJ4d73rzSg6leCFpqAQ3u6+uEJ2JUlWN0E9Zuf8qS U8jxiWfymbC86/iI6ru4GgqgD5Fmjlc= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=L1Go0rwX; spf=pass (imf17.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.219.178 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yb1-f178.google.com with SMTP id 3f1490d57ef6-d0b597e7ac1so5664492276.1 for ; Mon, 07 Aug 2023 16:05:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691449524; x=1692054324; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CHNVeLQ4ERi2aiC2atfDR7Ra/O6uVctCEHn/pbNh+mQ=; b=L1Go0rwXz95r8RKjbEpMnhh6c018jiQRjrFUv0ntz442yZnhpaeW+aHqb02Xo98603 EPNoaJuJCjZc6fdKeFl/4FxJrSloFA7bUFX05fyGjegXA7V7cuLZVOde3eglcKPAc2dh 7JAUmVzhkgUSMtLYpwSzyd8Ca3OvJH/6CC+nMDpgrmjfqgC7q9SKrYP1x1rjn8+euG1c w/F3CAIitPe7XXZVoEz/F21xQq4ZfepD4AFcrR4pjaM61cX8qLF4fMMNmTw3E/GvPxLQ CGZCqn+AGAHhqM6FL6AO+cy3AaHEnYT+L7t2jnss6vAgyAEw9i6+eoE0/wpeJRdeDqZo nMqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691449524; x=1692054324; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CHNVeLQ4ERi2aiC2atfDR7Ra/O6uVctCEHn/pbNh+mQ=; b=IG4DxhZFX2Lfg61tR08NlAPV2xXLhZIJqDz9t/ZDqx1aeUSjSsq48IaQvEgFWxhzoy LIkTE9eLo5zv2me3DZMH25lju2xfeAX6BQN7QQHbFJOW38IY+J2PpqlflpdAIuReB7sq PvsigzMiMZ6cNV+CpAkWnd0UAS7O1zZouPUFnusLM/6I1WNJj8JcjfgR2clNAF0Emy+I qLvuYiuB3j5h5grwHb0TStj2eOKn4o56++g0R8YcrmsUwAaahPf/tbt6/iN/5QtJnvLl 3ZU1X5td1w8zUPPP+ZHB9gaGsNId9muu34VbcTWr8tKmIk5TZSliC8u+lPQjz5Q6oyLb xvtg== X-Gm-Message-State: AOJu0YwvnMjxvqv/v0k1N9DPhSQ56zb2rRJLXdZa3o48+1cNAePqwGLr NoVE+1S1m+FqcMXx0YSzZtA= X-Google-Smtp-Source: AGHT+IF30NcYMYJc6ATZDEFP+FAmEVcusnv37vl/vjImNQWmcJZtPuqM/aGhVMbEV/I/t7ROvZmYcQ== X-Received: by 2002:a25:ca58:0:b0:d15:9cdc:5d0c with SMTP id a85-20020a25ca58000000b00d159cdc5d0cmr11907374ybg.42.1691449524354; Mon, 07 Aug 2023 16:05:24 -0700 (PDT) Received: from unknowna0e70b2ca394.attlocal.net ([2600:1700:2f7d:1800::16]) by smtp.googlemail.com with ESMTPSA id d190-20020a25cdc7000000b00d3596aca5bcsm2545203ybf.34.2023.08.07.16.05.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Aug 2023 16:05:24 -0700 (PDT) From: "Vishal Moola (Oracle)" To: Andrew Morton , Matthew Wilcox Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Hugh Dickins , "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v9 03/31] mm: add utility functions for ptdesc Date: Mon, 7 Aug 2023 16:04:45 -0700 Message-Id: <20230807230513.102486-4-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230807230513.102486-1-vishal.moola@gmail.com> References: <20230807230513.102486-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4989940014 X-Rspam-User: X-Stat-Signature: s58869wiffy3fy6wh79m7sncmcq9seo5 X-Rspamd-Server: rspam03 X-HE-Tag: 1691449525-382133 X-HE-Meta: U2FsdGVkX1+kyWfdWEKe5R2Z610k2uk6w8qODR3QrNt9wl/51EHPRSDP+KdaoPApNawPWm5BLW64Eb+lAXBK5+aF1Dlo2RzKVWz1/9/e/0/7tTn+ehAYQLFwOmrgS9Gh8xJDh+JfMbSPCUt4XNMXszpwL7Rp8BK6gnwkw8TIK94MNXnOUNTX+/AmhpEFwnzhgVf+fBIpwRjfEkfphzYTU17zbzG96x4HaVevtqRRKkTROfFCNaMuEc7GANM5saAEjOaZBHzqpsmbFTE/RD9jMzUfXnMMQBaedxv6wS3JUnNMPTyUmngakSqFkEh1BRNEOOc1NXLaxVteCyq2fvPH9+2yVJQchuFlciJA3aaCUn2PRKnZ2g/LO0pyP8P1WqrNa3LR5Wf6Qz4I+b7iQC2Zq4m8T4Tgvhlo/n06lUGeZYpLVOklUqX/5MTx4NLey2x43LrbndifL/1VeLNdLvvWI5iaIoi5pEIzrw+wZWnk9hAgAkM7WkyAV3wFhbbRh6Ksq8V2sFo3N0bw3JyLXYpJDsTLGvyVjq51VH35XtLNU/ykB1lCkIglnpewZeGLjAaQjjGc2ZkpWEwJHK28nY8dFlcm/zTwesktBjx3G9eUnbcwlntRctrCK7Fj3rgze5TuoHxnpfITraQPWhlCSLpb+WpfuDUt/fDKW5ngUhwgsx5sRR8fTIctRev4METHTCer0M1/QN/CQ/xQpI12rQ2jcOvn4GJDMtkrrFwYpjQvX090p3CsgMEsRqs4dt3BRBsnZpKJItJyASfPIRAUIhF4bWVngfhFWvxOcQfV8S2bYjJ+OY8daa0edFYEwGIqAPTAGNCh3zdzP0QNPkxcoxapAaVYG7ftoKv0JAXQ8iQyEcL3/S25a4uxQrflDizpTKtrhmQgEY9QqonBbVUx7HduAZ2vjBp43+Yi8i6WQIXUz6nVQXv4Iku2WXWe8zfyjhFch/KzmoiEdL9N3RGSRCl 7J5w/Ajo xoGHLCXal2XPRQ4dDIz9z+lmrIWjZJHaMoR7ZhtHqvYjZgPXQH0LLHc/w2vj1l+lXYRbgbkpvs1ptUmDM4dmkqV2qMFj68xiSli20UfkbtG3TVOUELy+K27XDet6IgABjt2hbId0DY4frFEZWOt2/HPozvikRU0dNJ0ADbrGDDiw8nSP1/8SdCOPYK19LBW34/2/ZYiRtTmEQq0RlaToAcfWAMhVtOCUlfN7I1MRV+3WFt5D6s9j5n2qDeSznOsjw1EKCmQ0oJBLCYsU44RnZw441z1ZNgsCz8ZfdqNwbEV9sy8qJ3MXZsGhFZD2YbjK5uGsh1bt68YJI5W7EjHFvXH4DoSWTvM+YGfj6v603Vmyzl5beWic3n9t9dyW/UA98abLyKIwRV8E90ENBfelHaowAaJE4Q9+rhnESjX8ntDKmAs02pvynWEuxrw6/HVrxjJzkzkakJ5WcniZgNzQNAt5Z/OTuRRRWBEGHJ1s6Q7Z378clEuG1ZYxK0GDkL3k0hqFYzv9Ab6e8XqBM9XRX93JlCA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce utility functions setting the foundation for ptdescs. These will also assist in the splitting out of ptdesc from struct page. Functions that focus on the descriptor are prefixed with ptdesc_* while functions that focus on the pagetable are prefixed with pagetable_*. pagetable_alloc() is defined to allocate new ptdesc pages as compound pages. This is to standardize ptdescs by allowing for one allocation and one free function, in contrast to 2 allocation and 2 free functions. Signed-off-by: Vishal Moola (Oracle) --- include/asm-generic/tlb.h | 11 +++++++ include/linux/mm.h | 61 +++++++++++++++++++++++++++++++++++++++ include/linux/mm_types.h | 12 ++++++++ 3 files changed, 84 insertions(+) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index bc32a2284c56..129a3a759976 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -480,6 +480,17 @@ static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page) return tlb_remove_page_size(tlb, page, PAGE_SIZE); } +static inline void tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + tlb_remove_table(tlb, pt); +} + +/* Like tlb_remove_ptdesc, but for page-like page directories. */ +static inline void tlb_remove_page_ptdesc(struct mmu_gather *tlb, struct ptdesc *pt) +{ + tlb_remove_page(tlb, ptdesc_page(pt)); +} + static inline void tlb_change_page_size(struct mmu_gather *tlb, unsigned int page_size) { diff --git a/include/linux/mm.h b/include/linux/mm.h index ec15ebc6def1..54dc176b90ea 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2806,6 +2806,57 @@ static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long a } #endif /* CONFIG_MMU */ +static inline struct ptdesc *virt_to_ptdesc(const void *x) +{ + return page_ptdesc(virt_to_page(x)); +} + +static inline void *ptdesc_to_virt(const struct ptdesc *pt) +{ + return page_to_virt(ptdesc_page(pt)); +} + +static inline void *ptdesc_address(const struct ptdesc *pt) +{ + return folio_address(ptdesc_folio(pt)); +} + +static inline bool pagetable_is_reserved(struct ptdesc *pt) +{ + return folio_test_reserved(ptdesc_folio(pt)); +} + +/** + * pagetable_alloc - Allocate pagetables + * @gfp: GFP flags + * @order: desired pagetable order + * + * pagetable_alloc allocates memory for page tables as well as a page table + * descriptor to describe that memory. + * + * Return: The ptdesc describing the allocated page tables. + */ +static inline struct ptdesc *pagetable_alloc(gfp_t gfp, unsigned int order) +{ + struct page *page = alloc_pages(gfp | __GFP_COMP, order); + + return page_ptdesc(page); +} + +/** + * pagetable_free - Free pagetables + * @pt: The page table descriptor + * + * pagetable_free frees the memory of all page tables described by a page + * table descriptor and the memory for the descriptor itself. + */ +static inline void pagetable_free(struct ptdesc *pt) +{ + struct page *page = ptdesc_page(pt); + + __free_pages(page, compound_order(page)); +} + #if USE_SPLIT_PTE_PTLOCKS #if ALLOC_SPLIT_PTLOCKS void __init ptlock_cache_init(void); @@ -2932,6 +2983,11 @@ static inline struct page *pmd_pgtable_page(pmd_t *pmd) return virt_to_page((void *)((unsigned long) pmd & mask)); } +static inline struct ptdesc *pmd_ptdesc(pmd_t *pmd) +{ + return page_ptdesc(pmd_pgtable_page(pmd)); +} + static inline spinlock_t *pmd_lockptr(struct mm_struct *mm, pmd_t *pmd) { return ptlock_ptr(pmd_pgtable_page(pmd)); @@ -3044,6 +3100,11 @@ static inline void mark_page_reserved(struct page *page) adjust_managed_page_count(page, -1); } +static inline void free_reserved_ptdesc(struct ptdesc *pt) +{ + free_reserved_page(ptdesc_page(pt)); +} + /* * Default method to free all the __init memory into the buddy system. * The freed pages will be poisoned with pattern "poison" if it's within diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index cb47438ae17f..ea34b22b4cbf 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -467,6 +467,18 @@ TABLE_MATCH(memcg_data, pt_memcg_data); #undef TABLE_MATCH static_assert(sizeof(struct ptdesc) <= sizeof(struct page)); +#define ptdesc_page(pt) (_Generic((pt), \ + const struct ptdesc *: (const struct page *)(pt), \ + struct ptdesc *: (struct page *)(pt))) + +#define ptdesc_folio(pt) (_Generic((pt), \ + const struct ptdesc *: (const struct folio *)(pt), \ + struct ptdesc *: (struct folio *)(pt))) + +#define page_ptdesc(p) (_Generic((p), \ + const struct page *: (const struct ptdesc *)(p), \ + struct page *: (struct ptdesc *)(p))) + /* * Used for sizing the vmemmap region on some architectures */