From patchwork Wed Jul 12 04:39:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13309565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40D39EB64D9 for ; Wed, 12 Jul 2023 04:39:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5FE36B0072; Wed, 12 Jul 2023 00:39:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D0F866B0075; Wed, 12 Jul 2023 00:39:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BFEEB6B0078; Wed, 12 Jul 2023 00:39:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B2B9A6B0072 for ; Wed, 12 Jul 2023 00:39:56 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 801FB1204C0 for ; Wed, 12 Jul 2023 04:39:56 +0000 (UTC) X-FDA: 81001707192.12.EEE154E Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) by imf20.hostedemail.com (Postfix) with ESMTP id 9E5981C0007 for ; Wed, 12 Jul 2023 04:39:54 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=XYfimDGo; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of hughd@google.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689136794; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2t2oc+yeedMzdUFXYKBxRHbKcZMzrhHMUkscNN4bTko=; b=z41Czc1Z7Zw9Vz4pmz+tzR22TLPDNoORhEyJ7J49oZ3AAVdvILSX0s+BtozIb4GDaWv+A6 Bcd91LGu5+mun5BzkHkEMJr2k5YcFWIfTb0yNu9OBsWroHM1xXXUgPgLRlX8oyLx63GgR9 NQ6MQnAvxKG3hK+FBKawdvnNPRvBl+0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=XYfimDGo; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of hughd@google.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689136794; a=rsa-sha256; cv=none; b=wHNbmDluYk3MD69BOeM5FcExlEdYKmgGSd7SiYJZ1i4XYkwLniNEeUzlPWJQ1YPKvGGe0y QlwTf/jSxGm7RBEpxxutbXCCqmnnxoprpT26Jsl2SR3+VBaMF5WIcC/mld3/O79/OAMeDa nFIUN98KpR4a2zVUngrgvt3LtsRnl4s= Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-5704fce0f23so76229007b3.3 for ; Tue, 11 Jul 2023 21:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689136793; x=1691728793; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=2t2oc+yeedMzdUFXYKBxRHbKcZMzrhHMUkscNN4bTko=; b=XYfimDGolFEty51TCKUyAZmm69LKogtUNWLIyMCuw81Si1ZjaboZJMyonyZe8BlA1H 4dw7ywAuqviGzR+cqYGZtxV+ytY3jiQkgFAvjXIkhqrYEAwOpEMsZRaoOyR5rEd3GDD+ 6LnUt02lOICM7P8CDg5UfhiUAjvip1TQsKWkNZqXUfOXl95Q2+l607lxyvF7WfoMEkup n8srQ8rKA7PHkhNMQEN57jpX0Q3gk0ZBR2Liws8TUdU4okAi97PytXAsy84/ZZk7I96p LrZbc0ZAQKs4VNt1L3qD7duolPC2jOZozxLhx0DiCnOdCct9HdR1WwipR08docsFv3CN 0s6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689136793; x=1691728793; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2t2oc+yeedMzdUFXYKBxRHbKcZMzrhHMUkscNN4bTko=; b=iarRYjVFzTGnM7sf2Hiu2/oBNpnby1bv9yI+mHe11jnxmflJTh2pF9EcWSPWEMaZKH YThMxCfAEt9wl6Xo5bLzqDC3aBwt7jBcEVYTT3aCfsgxA4/yMWF2QrBUk++YJpm4KDvw 4LjdeeiDS7kpUTHBror8GcEQ/G202NIowtFB07YznoMERJ6YnEtbeecAZ24TWL3RUnz7 22LzOkIXns+cAeXDkhKckTvL6WRgGmFX+1Yw9WK2kfg5M7u6ZkOSiVpz//d4ivUuP5BJ 7VaSj+LkMhDqz+0Coba3wOWsCIioXf1ICoPoUnKIEotvMsKuq+0O2bbT0SE863e/oLpS +ZgA== X-Gm-Message-State: ABy/qLbvyaShhQ28Urj7AapFP7Vm+hN6KJSgvZN0uxBipYOvXmLNmFwR TZX6gvidU+z3gR7cw8ZNrDEZGA== X-Google-Smtp-Source: APBJJlH7ohd7+cM8rUQREK94h0qZle8YpJFolayyIig71Hbg47eK0cJuvgfSIYJvI0oB6MYKn3tkcQ== X-Received: by 2002:a81:a187:0:b0:579:efb4:7d08 with SMTP id y129-20020a81a187000000b00579efb47d08mr15922621ywg.27.1689136793558; Tue, 11 Jul 2023 21:39:53 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id f130-20020a816a88000000b005702bfb19bfsm974843ywc.130.2023.07.11.21.39.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jul 2023 21:39:53 -0700 (PDT) Date: Tue, 11 Jul 2023 21:39:48 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Gerald Schaefer , Vasily Gorbik , Jann Horn , Vishal Moola , Vlastimil Babka , Zi Yan , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 08/13] mm/pgtable: add pte_free_defer() for pgtable as page In-Reply-To: <7cd843a9-aa80-14f-5eb2-33427363c20@google.com> Message-ID: <78e921b0-b681-a1b0-dc20-44c9efa4ef3c@google.com> References: <7cd843a9-aa80-14f-5eb2-33427363c20@google.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9E5981C0007 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: tfoy94h8wzs5cqpzc98a6iae69hzzh7f X-HE-Tag: 1689136794-312195 X-HE-Meta: U2FsdGVkX1/uc+didIQwHPUXJqtXwlkzUa0ZQid1JYkmB/vhN28bJ5D/namX051tTf+1kXKuelq3ZeJY2/xHNxswZL2Yln8zg+UOTfGHfI7NcyGu9hEBIStde4xBweqxymJzeJYtTfzCLleAgXooY9cDSFVfuWqVBVQaJR6aHK8N2fgQjETzuDGyrDqKeDHqmG/+EyJ62HR9elI7Of88FRbKV6cZWfeZqpoHvYGQ3O8wQtnviq+pLJJZFU0xV5AryKNPB5q9SivpPuZFdOrpMjMkkKh2JENsW6ir1Sd0S7eyB/wmHz3Vduckvua3ImjUe+4tR8fgiv8neuIKuMvetJnEwFHwvs5czn4dSCLWna2cmktJT7t0ztgkmAOCaIPYXqy9tpQJ2LZanQAd4gBJ7EjaYifdezIXoQDRiI1JkmfkW2ZRaTep5A42cd33kqMQAcml1UkLOTjiH/qaUajEer6C8ZFABGn1T/sePRSdNqFCTtqxY5CH3zYdMG1UWz8gEqQT9TnS15cBuG5IdBOFoDaa46wZL6NieA9ETEth1cUf7JcVH7G4iE+A+B3ipFa225duAgyAhddgmJ/CxGtwvUOo8SX0Y++dpgkSLt8F6KjDxa5n/nfvx2ldKcWEpW0Twy0tx3ZgNeF5C7G0Kuh4dl8fyi4S6VM43T8101G/KoX/U81X+h1w/irfHiYYibfGk7j32352FVRxBGViO8XsMDAo3Jge7+FEjhGenGzYZbCW8m2QBE8qJ9yFlXD7zOjXKcvLZKXWSmLUTB4EkUC92vUKDsyusu8THCpvz9xXqF4hXA6q19pzvH/zT9r2uTaKjyVfxKdGvdfStoMxHDjyq1FNkksoRoBUYe5zoblYokt/Gy9UkPQyHmCBfLZ89dQZ1FZB0XyAyVKfzmClX/7p71ponfJ1o0FtNlgPXsr1sBsjG9bsJb/robp5EZXjeIjqPje6XhuqfR6PITtIzb5 oN8i1d0+ QBvRjDsWBdMy22AQn8lvDVy6gzKUS2Y8DjUpmUNaKFOqfZni0t4Gd+wPKVRDOq3XJjO8Ec0zB5j82XJElVF2O6vQRzV4ANnAwMdMK/flQ22e4EO+bQl7UWhqtC47nkkMGw2qWimSQHFTVmPEVmz3OIZAqdq/Pa8ZlS0Dkk6OXPgA4f/hSeA7yZwbGK8yaAhzAaiCVf0YIGuiDDK9GCY6ofHQy3miTl8jzJt75JFWy4rXcn9455fnWxBVdkqRX+alCWzVKu+CRQoy4bW9J7H2aw0Sx5xkNPsHaCsxHqZlNxXJYrnK0mVLxw93ox+u0SIELEcib5LCaZP9NIOb234GWfdcdbnPSMp+xy4sh/L1yFDcmGesZBseh4oZNfGICeX6NhufQQGKVIsG0h6/tVaCgUbkPNNnbv6j/4zY+4FlS6xQ/F5Jr/EmTeYmc+lqAl1tuk9n+Ga8x4b+Mru9rkD05fHg7116ul5Uf7dSW4UyYB0PJtpyvbWgzOctk5JoquVn5HVqlrm9DwWz6ao5aEbnmT+HCyDtj4+h/B5QE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the generic pte_free_defer(), to call pte_free() via call_rcu(). pte_free_defer() will be called inside khugepaged's retract_page_tables() loop, where allocating extra memory cannot be relied upon. This version suits all those architectures which use an unfragmented page for one page table (none of whose pte_free()s use the mm arg which was passed to it). Signed-off-by: Hugh Dickins --- include/linux/mm_types.h | 4 ++++ include/linux/pgtable.h | 2 ++ mm/pgtable-generic.c | 20 ++++++++++++++++++++ 3 files changed, 26 insertions(+) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index de10fc797c8e..17a7868f00bd 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -144,6 +144,10 @@ struct page { struct { /* Page table pages */ unsigned long _pt_pad_1; /* compound_head */ pgtable_t pmd_huge_pte; /* protected by page->ptl */ + /* + * A PTE page table page might be freed by use of + * rcu_head: which overlays those two fields above. + */ unsigned long _pt_pad_2; /* mapping */ union { struct mm_struct *pt_mm; /* x86 pgds only */ diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 7f2db400f653..9fa34be65159 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -112,6 +112,8 @@ static inline void pte_unmap(pte_t *pte) } #endif +void pte_free_defer(struct mm_struct *mm, pgtable_t pgtable); + /* Find an entry in the second-level page table.. */ #ifndef pmd_offset static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address) diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index b9a0c2137cc1..fa9d4d084291 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -13,6 +13,7 @@ #include #include #include +#include #include /* @@ -230,6 +231,25 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, return pmd; } #endif + +/* arch define pte_free_defer in asm/pgalloc.h for its own implementation */ +#ifndef pte_free_defer +static void pte_free_now(struct rcu_head *head) +{ + struct page *page; + + page = container_of(head, struct page, rcu_head); + pte_free(NULL /* mm not passed and not used */, (pgtable_t)page); +} + +void pte_free_defer(struct mm_struct *mm, pgtable_t pgtable) +{ + struct page *page; + + page = pgtable; + call_rcu(&page->rcu_head, pte_free_now); +} +#endif /* pte_free_defer */ #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #if defined(CONFIG_GUP_GET_PXX_LOW_HIGH) && \