From patchwork Mon Jun 12 21:03:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13277349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5242C88CB8 for ; Mon, 12 Jun 2023 21:04:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8135094000D; Mon, 12 Jun 2023 17:04:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C22794000B; Mon, 12 Jun 2023 17:04:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63FDF94000D; Mon, 12 Jun 2023 17:04:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4E7C394000B for ; Mon, 12 Jun 2023 17:04:59 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 23D1CC033C for ; Mon, 12 Jun 2023 21:04:59 +0000 (UTC) X-FDA: 80895325518.25.C43532E Received: from mail-yw1-f180.google.com (mail-yw1-f180.google.com [209.85.128.180]) by imf10.hostedemail.com (Postfix) with ESMTP id 4F90FC0022 for ; Mon, 12 Jun 2023 21:04:57 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=XHctMnuj; spf=pass (imf10.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.128.180 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686603897; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E077vqoCqwl2/4qA0TBAbJoWDfL60BXPSkhV6wk0YUw=; b=IG4UVSfyZmQ1siHD9QfQlpkgxL9J/lD3y1T0tj9nkZ/7CkBCQAvJ0shjWRIhTaDuwYz/T0 TDcJyKxOsk5ocp3e7NGhzbF9V4wA5T1FEy0z8NflmnMUzvEFI+oSUk1w8qRi0EdC3jtyoE EWidKVNNMT6/PZX7ZxoJ1n9S8M0iYBM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686603897; a=rsa-sha256; cv=none; b=RzFMI5FFUw3rZaqU3pCcGFfMTJD3TBUnS7nGE+PsFyK04ne4T4jvtYhF0dAiLjhXVxvB5z hn3vy2pa/BvWF/HBBsizrLs/t+RWXbx1tlC+tz3/mZ3i6LqmLgG0EmujJLdKx5pevGiIcS mF73GkMYBX2D6aMYmRE5kNbO+o/euw8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=XHctMnuj; spf=pass (imf10.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.128.180 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yw1-f180.google.com with SMTP id 00721157ae682-56d0d574964so20390247b3.0 for ; Mon, 12 Jun 2023 14:04:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686603896; x=1689195896; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E077vqoCqwl2/4qA0TBAbJoWDfL60BXPSkhV6wk0YUw=; b=XHctMnujg/tVWYZlPY5bXFqva6VGPkLJitWyIF76elwmA+hNNOuZMn/71AAwR57M4B IyANMxsihrptf60QavxQFQZEOMJ5l1UtI4amrfsMrJlTcE45HruYhKHkR8dOb9vrhLyj wIzwXKD/U1Tx8l72JrhxAelagPrpqFK0zX9Nfim+mlei1ja/Gf0amJMsRM2MRf8e/2kO jriFwnl1iYZ7eeVYPFopluumvDGCM2xgx/viKkcjIMlifM5jUo+p/eIAsvrxVoA3HaI3 YFqZHN+rA25I8RXwpppjrBbB0BFU/dJTiV2alRnBnUUYr7XItuIWUT0u54vTo5/wZ++Y E5Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686603896; x=1689195896; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E077vqoCqwl2/4qA0TBAbJoWDfL60BXPSkhV6wk0YUw=; b=KPm/pSg8hteRnYw1hYyyFXVwHf4dC7AKAcr2tTNZNvrRGhj94Ljc/ItzATaYtX7vGT r8SMwZpOMAmsDr7zDK8IsCkEqG6RG947hp4EG7aVxmorhRgMzrpVLIPXuzDdcDlG74ts xdsMivtSzBZTc3InvtIAZ1ZSt3+KbyyaL/6sto9HSEK9TrPsJOAkV/hKYX2YAxlGC5JB ffvbDaLGDhDCUH1b+xjw/c4vnu7T3EHX6YWYVaNS9W0LmnTfyo/YUKDH8Iqco1RW6Tor Ha93yGNNZlWpDj1om4ImvMeqS94Lb6BJBZ93FKVrxwae1Kf+UCNvlR73+O8cNQVDp9PI dyzw== X-Gm-Message-State: AC+VfDwQFwZkRNduSLFy6ImqAmM60lC0K6a2OM5nRQ8tQJHJe/QpqUU8 SOYSAHbS9EfszP/Hsov6zQo= X-Google-Smtp-Source: ACHHUZ70i2xb+pf7p7UU1uRX9rQOpj1AFV5IFVauNZ+PMgf0TtCpACb/4dq/MMeKyVRgHZ+W6e2xbw== X-Received: by 2002:a81:5a55:0:b0:569:74f3:f3e1 with SMTP id o82-20020a815a55000000b0056974f3f3e1mr10812817ywb.0.1686603896297; Mon, 12 Jun 2023 14:04:56 -0700 (PDT) Received: from unknowna0e70b2ca394.attlocal.net ([2600:1700:2f7d:1800::16]) by smtp.googlemail.com with ESMTPSA id s125-20020a817783000000b00569eb609458sm2757115ywc.81.2023.06.12.14.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jun 2023 14:04:56 -0700 (PDT) From: "Vishal Moola (Oracle)" To: Andrew Morton , Matthew Wilcox Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Hugh Dickins , "Vishal Moola (Oracle)" Subject: [PATCH v4 05/34] mm: add utility functions for ptdesc Date: Mon, 12 Jun 2023 14:03:54 -0700 Message-Id: <20230612210423.18611-6-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230612210423.18611-1-vishal.moola@gmail.com> References: <20230612210423.18611-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Stat-Signature: gtstc5uncbspkrsd97tbcj63p9wd8aci X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 4F90FC0022 X-Rspam-User: X-HE-Tag: 1686603897-75848 X-HE-Meta: U2FsdGVkX1+HyEVMXVFnSsvE1ULcvVsEoeyJQOyC9Jc4b0/6Uio78mAfc9xA6ZtK8TljiZWqiKcBcbr+ftvlxD5AQrXxXmBmyyFM0dQ51oHJ2q4sUJxwpHLYa5WPmUKRgdGC6zConh3imxQDvJgVT8ciSNL5PvcxitZ6u2umgWO58iOI/QMfqjqR50LSZRzuWGBvSreaRdRRtYmmvy4zdVoyUTbNnMQeTsq+gRO4657IRKI1GGHcyFT4UvKTPv8DPnwy4C0mZ6lMsgXPuND2Vh9VHSEyqNTbh8pJ9QKGphOQVjQCkixleeSpclv/c1+YgRFTBr6cFa146bIkZftkkOd+g24EZjskN3XyBeFfnesq1Re3+OJlgHBeHPKKks32XuiEgjoy+5HsBecHhNK9BH3HfuX/4+5FxaJ8bcuHwYIGMY4c0LCfZUyJSmHMip/FqdFTpuyxnrsZuQhfnZNzCA1ezJQsEgtfFS/HDVS5pUz1xZQcnUddBl8lNLgjgR5JyAdmSWjWXHvnj6NjKU2YtiWO8vKNongCdwdqr7mfh+vXhM1WzC16bfYtm8xtbbP//TeuWuKFEqqAOH6UnE8BeZ2vtRP4hBF7ezmAItsot75cN6NlTAGBOQJ8IxuXZd1T2lt+274A0b6lI5bRbmBArmpoMcm3lvYOELjjS2gOzMMLeCl8bauxbZhBbS+JAjqQAc0CaKFnziOjB6506NqUVx0Cd0pE7T/bmY/cURMonz4bJ2QleeYwnsWLeUF4G/ru4mshnRtGsbjrtxaCqWCx/xHG/j0SrIBGxRVhftuxzh81smZeV7SO3zQmulO4F48RhQ59+c+6drRV/smL+FJ7Je/NO8E+A43EA9WY6VYIytREToFErA6I12hI/So79kXurAwQzi3HveIXvhXCLFGOrGtPcLvMDBFa7wi8vEwCtXduaA21Pk0pVLWZoMus6/vFhhTqj6yH1w0Q5OF68D+ VvdcHduE 7vKoi/NmOlya68yRtQJp/JhKyB8rpwmAGQ5Z7k9rkOnkLkB4dztqAApPITj6gtioVSF2JH4R1o3L5o9uTXfnrdfLyXNOvSMKo55IdRTknOUuGMMCv+mvr1PU93ToZIPWwrs+FtygnzphNHc4UXkE78QHXQc8+fMeqbbybu5Wta4ruhP0uAth/c2xB6HAJiVYZ9lLvk1TX0+WPoFvWgnwSzEj3TNLUYJ/n+UvGKy4yqvaNX+avwh98JKnawdOIi7a/jy8ITzrVplmWrmPDdLazHq5CslqTuVfSzDm03mu3sUQb//1LC+k8xO9WvgX93H+OQCB8eaIJSN5N/4SM53TqS/cIURI88svNTktMcV3lzvq7nhsaEPpzPekvLmwab3lYHkYLc/VNoJjTAud6DYDvZHQfKjvMtYPz29P4awEHVEIts97Du+pS3Gu5hppq7WO/aCuDTLll6bqutnFsUX/wnKGi5/VhJ2fiOFl73uD1yNQAp27KS+QxCxgIwyTyKy3QEXahxdNR25g7YKt7qPoLXGVjxw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce utility functions setting the foundation for ptdescs. These will also assist in the splitting out of ptdesc from struct page. Functions that focus on the descriptor are prefixed with ptdesc_* while functions that focus on the pagetable are prefixed with pagetable_*. pagetable_alloc() is defined to allocate new ptdesc pages as compound pages. This is to standardize ptdescs by allowing for one allocation and one free function, in contrast to 2 allocation and 2 free functions. Signed-off-by: Vishal Moola (Oracle) --- include/asm-generic/tlb.h | 11 +++++++ include/linux/mm.h | 61 +++++++++++++++++++++++++++++++++++++++ include/linux/pgtable.h | 12 ++++++++ 3 files changed, 84 insertions(+) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index b46617207c93..6bade9e0e799 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -481,6 +481,17 @@ static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page) return tlb_remove_page_size(tlb, page, PAGE_SIZE); } +static inline void tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + tlb_remove_table(tlb, pt); +} + +/* Like tlb_remove_ptdesc, but for page-like page directories. */ +static inline void tlb_remove_page_ptdesc(struct mmu_gather *tlb, struct ptdesc *pt) +{ + tlb_remove_page(tlb, ptdesc_page(pt)); +} + static inline void tlb_change_page_size(struct mmu_gather *tlb, unsigned int page_size) { diff --git a/include/linux/mm.h b/include/linux/mm.h index 0db09639dd2d..f184f1eba85d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2766,6 +2766,62 @@ static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long a } #endif /* CONFIG_MMU */ +static inline struct ptdesc *virt_to_ptdesc(const void *x) +{ + return page_ptdesc(virt_to_page(x)); +} + +static inline void *ptdesc_to_virt(const struct ptdesc *pt) +{ + return page_to_virt(ptdesc_page(pt)); +} + +static inline void *ptdesc_address(const struct ptdesc *pt) +{ + return folio_address(ptdesc_folio(pt)); +} + +static inline bool pagetable_is_reserved(struct ptdesc *pt) +{ + return folio_test_reserved(ptdesc_folio(pt)); +} + +/** + * pagetable_alloc - Allocate pagetables + * @gfp: GFP flags + * @order: desired pagetable order + * + * pagetable_alloc allocates a page table descriptor as well as all pages + * described by it. + * + * Return: The ptdesc describing the allocated page tables. + */ +static inline struct ptdesc *pagetable_alloc(gfp_t gfp, unsigned int order) +{ + struct page *page = alloc_pages(gfp | __GFP_COMP, order); + + return page_ptdesc(page); +} + +/** + * pagetable_free - Free pagetables + * @pt: The page table descriptor + * + * pagetable_free frees a page table descriptor as well as all page + * tables described by said ptdesc. + */ +static inline void pagetable_free(struct ptdesc *pt) +{ + struct page *page = ptdesc_page(pt); + + __free_pages(page, compound_order(page)); +} + +static inline void pagetable_clear(void *x) +{ + clear_page(x); +} + #if USE_SPLIT_PTE_PTLOCKS #if ALLOC_SPLIT_PTLOCKS void __init ptlock_cache_init(void); @@ -2992,6 +3048,11 @@ static inline void mark_page_reserved(struct page *page) adjust_managed_page_count(page, -1); } +static inline void free_reserved_ptdesc(struct ptdesc *pt) +{ + free_reserved_page(ptdesc_page(pt)); +} + /* * Default method to free all the __init memory into the buddy system. * The freed pages will be poisoned with pattern "poison" if it's within diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 330de96ebfd6..c405f74d3875 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1026,6 +1026,18 @@ TABLE_MATCH(ptl, ptl); #undef TABLE_MATCH static_assert(sizeof(struct ptdesc) <= sizeof(struct page)); +#define ptdesc_page(pt) (_Generic((pt), \ + const struct ptdesc *: (const struct page *)(pt), \ + struct ptdesc *: (struct page *)(pt))) + +#define ptdesc_folio(pt) (_Generic((pt), \ + const struct ptdesc *: (const struct folio *)(pt), \ + struct ptdesc *: (struct folio *)(pt))) + +#define page_ptdesc(p) (_Generic((p), \ + const struct page *: (const struct ptdesc *)(p), \ + struct page *: (struct ptdesc *)(p))) + /* * No-op macros that just return the current protection value. Defined here * because these macros can be used even if CONFIG_MMU is not defined.