From patchwork Wed Mar 27 14:10:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13606713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F9CFC47DD9 for ; Wed, 27 Mar 2024 14:13:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1AB866B0087; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BE3C6B0092; Wed, 27 Mar 2024 10:13:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8F3A6B008A; Wed, 27 Mar 2024 10:13:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B3ECB6B0088 for ; Wed, 27 Mar 2024 10:13:21 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1E64E80E26 for ; Wed, 27 Mar 2024 14:13:21 +0000 (UTC) X-FDA: 81943011402.19.029CCF3 Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by imf26.hostedemail.com (Postfix) with ESMTP id 337E7140016 for ; Wed, 27 Mar 2024 14:13:17 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711548799; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AE/HoRl7DV7kTAss42eXnXE/ImpbK0jauu5kNON0ras=; b=n7UlzZn2QfodZaWsudwLyb5XRLywbT0jLJlynuLKfLO9kaBrVBx/hKkukwe3vIUIMyI2+r h1HJwXYE6ewnIafDyD4Cj+FBQwzoFaT0202N+f3Q6qE+TEHZlJR+Ct7cFJnPFCy7WoBBw3 0W+bBPTjZcy3AgmHyZ9fcNUXdqKRl44= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711548799; a=rsa-sha256; cv=none; b=ZsYGutOJp4tqhruddsiABnLs5vcxQNbWiXoqSHnOPanl9AnfG6VhzpZBHw15bDYFAMDlGw 7QjhsgaJYgGTsS8sspD1RkNmgSJHjvYjsYfx0KftlVWiwzKBIMKjeS87WpDoDUXQv0xF5C yN1KOp4+fXPl/uuH8HYOf8Acx7U2qtU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4V4TBx4Zvzz1wnZy; Wed, 27 Mar 2024 22:12:25 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 08C27140259; Wed, 27 Mar 2024 22:13:15 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 27 Mar 2024 22:13:14 +0800 From: Kefeng Wang To: Andrew Morton CC: , Miaohe Lin , Naoya Horiguchi , David Hildenbrand , Oscar Salvador , Zi Yan , Hugh Dickins , Jonathan Corbet , , , Baolin Wang , Kefeng Wang Subject: [PATCH 1/6] mm: migrate: add isolate_movable_folio() Date: Wed, 27 Mar 2024 22:10:29 +0800 Message-ID: <20240327141034.3712697-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> References: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Queue-Id: 337E7140016 X-Rspam-User: X-Stat-Signature: 7c4kwshu1f4mudyjbkpqrzx16rpb38ri X-Rspamd-Server: rspam03 X-HE-Tag: 1711548797-265155 X-HE-Meta: U2FsdGVkX1+foVAKSsE7aJrEJNxdhnyUPrKZwoRcvHGUX8y/Sof/6W8nF6mV0cm2LniM3k1YiZIr71sKb8iNS0v4UYRC0usiY0p7HEJt9f/ZJmI8D5Di6bWfPVRPM10LqNqJL6AbuHPATZDgyQSiYIUbQ0QAa0aQAlnd5AzYmeA2vzlWU1PazgTrIDDLmdsg+9eFEWVJiemuODuN08qAi1+jne+N+fJWl453RZiMsG0LXVka3/F/FlMFew6IUEoNtI+C7Xa5A/6UexksAgjdMedVpMEbiVXQo2Fd0M8rRjOaEp4eklc9OF8R0v4EYmJX2aC+zagcJf5dI1GhDPq/KSQFy3DdypY8yxd0931ZJFvIohMoJB+rDWUd0SqKAgBUy/hSEhM96xNqmrMM1+gt3vxoxAC3e6+NhqzJK+PNG7u5ZlEPxpcQp6SBDANEacLBpKWNZhp63SUBCk8x/sSmjceTab4zhe+Y9K3HEKiD1zcDz3DgRmRueSoCt2v5pIWBNHv3QVPiH2lV1751uifLyVUyijMsBBKIMGxL29/DtceDFu8uRylILhcDGvMscYALhFLBHPck08JnW3MfOBBtONwh6yB//sJOfwC6ObYZ7RaRqCOkNJ3Lyq/ibO3qHwUj1svWEnORRt3WHPuBLtBHB5X/F6Vfu3ib7yccQm2fHlbYhDlPmrjD8zj1KZA3NtOVsIvBxCegrMqQHseQRmUNyyjWO7CswJ3KpkhMzxQiTk6NkOD8fhdpQ/EQpLuHZlNlrAJOqzx4Kqh65yrNckLSwNjm1KCdps1ChxbV0IdfQMUOi943sSFOLrxRX2pzq95bs0vw/nWv+LC1oR3JinERcN6XwqEA9VWfJKbFtTIn69Aldg28Tazk+4LKTPF1CbbrxyqEKufDLZU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Like isolate_lru_page(), make isolate_movable_page() as a wrapper around isolate_lru_folio(), since isolate_movable_page() always fails on a tail page, add a warn for tail page and return immediately. Signed-off-by: Kefeng Wang --- include/linux/migrate.h | 3 +++ mm/migrate.c | 41 +++++++++++++++++++++++------------------ 2 files changed, 26 insertions(+), 18 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index f9d92482d117..a6c38ee7246a 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -70,6 +70,7 @@ int migrate_pages(struct list_head *l, new_folio_t new, free_folio_t free, unsigned int *ret_succeeded); struct folio *alloc_migration_target(struct folio *src, unsigned long private); bool isolate_movable_page(struct page *page, isolate_mode_t mode); +bool isolate_movable_folio(struct folio *folio, isolate_mode_t mode); int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); @@ -91,6 +92,8 @@ static inline struct folio *alloc_migration_target(struct folio *src, { return NULL; } static inline bool isolate_movable_page(struct page *page, isolate_mode_t mode) { return false; } +static inline bool isolate_movable_folio(struct page *page, isolate_mode_t mode) + { return false; } static inline int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src) diff --git a/mm/migrate.c b/mm/migrate.c index 2228ca681afb..b2195b6ff32c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -57,31 +57,29 @@ #include "internal.h" -bool isolate_movable_page(struct page *page, isolate_mode_t mode) +bool isolate_movable_folio(struct folio *folio, isolate_mode_t mode) { - struct folio *folio = folio_get_nontail_page(page); const struct movable_operations *mops; /* - * Avoid burning cycles with pages that are yet under __free_pages(), + * Avoid burning cycles with folios that are yet under __free_pages(), * or just got freed under us. * - * In case we 'win' a race for a movable page being freed under us and + * In case we 'win' a race for a movable folio being freed under us and * raise its refcount preventing __free_pages() from doing its job - * the put_page() at the end of this block will take care of - * release this page, thus avoiding a nasty leakage. + * the folio_put() at the end of this block will take care of + * release this folio, thus avoiding a nasty leakage. */ - if (!folio) - goto out; + folio_get(folio); if (unlikely(folio_test_slab(folio))) goto out_putfolio; /* Pairs with smp_wmb() in slab freeing, e.g. SLUB's __free_slab() */ smp_rmb(); /* - * Check movable flag before taking the page lock because - * we use non-atomic bitops on newly allocated page flags so - * unconditionally grabbing the lock ruins page's owner side. + * Check movable flag before taking the folio lock because + * we use non-atomic bitops on newly allocated folio flags so + * unconditionally grabbing the lock ruins folio's owner side. */ if (unlikely(!__folio_test_movable(folio))) goto out_putfolio; @@ -91,13 +89,13 @@ bool isolate_movable_page(struct page *page, isolate_mode_t mode) goto out_putfolio; /* - * As movable pages are not isolated from LRU lists, concurrent - * compaction threads can race against page migration functions - * as well as race against the releasing a page. + * As movable folios are not isolated from LRU lists, concurrent + * compaction threads can race against folio migration functions + * as well as race against the releasing a folio. * - * In order to avoid having an already isolated movable page + * In order to avoid having an already isolated movable folio * being (wrongly) re-isolated while it is under migration, - * or to avoid attempting to isolate pages being released, + * or to avoid attempting to isolate folios being released, * lets be sure we have the page lock * before proceeding with the movable page isolation steps. */ @@ -113,7 +111,7 @@ bool isolate_movable_page(struct page *page, isolate_mode_t mode) if (!mops->isolate_page(&folio->page, mode)) goto out_no_isolated; - /* Driver shouldn't use PG_isolated bit of page->flags */ + /* Driver shouldn't use PG_isolated bit of folio->flags */ WARN_ON_ONCE(folio_test_isolated(folio)); folio_set_isolated(folio); folio_unlock(folio); @@ -124,10 +122,17 @@ bool isolate_movable_page(struct page *page, isolate_mode_t mode) folio_unlock(folio); out_putfolio: folio_put(folio); -out: return false; } +bool isolate_movable_page(struct page *page, isolate_mode_t mode) +{ + if (WARN_RATELIMIT(PageTail(page), "trying to isolate tail page")) + return false; + + return isolate_movable_folio((struct folio *)page, mode); +} + static void putback_movable_folio(struct folio *folio) { const struct movable_operations *mops = folio_movable_ops(folio); From patchwork Wed Mar 27 14:10:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13606715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BAB6C47DD9 for ; Wed, 27 Mar 2024 14:13:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 864036B0093; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 811366B0095; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 616376B0096; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3FA016B0092 for ; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 0657B1A0DFF for ; Wed, 27 Mar 2024 14:13:22 +0000 (UTC) X-FDA: 81943011444.19.01EB30A Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf13.hostedemail.com (Postfix) with ESMTP id 4BBB820016 for ; Wed, 27 Mar 2024 14:13:18 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf13.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711548800; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8jdoZHQe/eabwCQajw5l4z4BSM48qUV5oi9A/k3PhpM=; b=dXA6uD3+LwZw+9iJYf6DJX6AC2CqxVJOdfylXOQMI/2DKEZ4KlQgxRhHsaqfMs+OyOTMt/ sHhC9kbQIDbSo80m4hND4by2OuEmgy3sAfOsm8dvJHdngd3xydVIY4r1SRypcqTOq6qx6t DXMtLaEZhtx9TRdhPQNPiazkgsSXeo0= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf13.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711548800; a=rsa-sha256; cv=none; b=KZHqCkmMU3JC6nwJiDU4dxRf8FBU2gPzNIiE0wYWNIxrfoPoHwVLsjN4AzxB1GEJbiVPjD vGctIQ93BU86FPrMPB1pUIEmMzyT9V77L2CeMlBrFbMEkLOGXGUDFGiM9GBMW422Q1LXSw lAyvBmm2HTiIyPP2eFaAF20EcHvHRX4= Received: from mail.maildlp.com (unknown [172.19.163.17]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4V4T8q6jGWz29dPD; Wed, 27 Mar 2024 22:10:35 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 9C5C71A0172; Wed, 27 Mar 2024 22:13:15 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 27 Mar 2024 22:13:14 +0800 From: Kefeng Wang To: Andrew Morton CC: , Miaohe Lin , Naoya Horiguchi , David Hildenbrand , Oscar Salvador , Zi Yan , Hugh Dickins , Jonathan Corbet , , , Baolin Wang , Kefeng Wang Subject: [PATCH 2/6] mm: memory_hotplug: use more folio in do_migrate_range() Date: Wed, 27 Mar 2024 22:10:30 +0800 Message-ID: <20240327141034.3712697-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> References: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Queue-Id: 4BBB820016 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: wegg9zx67c4xi784en75io1pjgc1xogf X-HE-Tag: 1711548798-521187 X-HE-Meta: U2FsdGVkX1+BhTdfjEBlSVt92mERv2hJENusxv0nShxoxQ/09fGo3PKDKLoxUOxHeXsNTjzqvOMnhnTqM/dsEzK0+ta0QSQgSoaEwY6Zf4ouyw1m3wflxGyqPtKxNF9Teu75qKJz0O9maBMF4fOlirG4+RGOzVMstmaeXGqxXL/45BK2niCI2qzgBzd/O3PONzufr2K12WKdkR/qXBPq7PWQ0bjyRgKMtN81FpB71OxKUUe0dK2S436NiTRYmgPHixLB8fRvWBkv7wl6dOWnRZQVrH1rBbCNfSXRbJD7nIZ6K3C42J3Nva7s7Lsh98ZgqhueEa6IdTQQrN3zJeeGLLj4qKyWRLa0YCxDcO/HiEmOEmfrG0zmoeo/9sKDoKEoNbRN8x49mWlO76I2IqQwaUcVBBFDJv5y20vg0/h/nwU3OesYYPyPo/hMtSJSgWHpcFA7Jvk2bBpxcUBnaGw2nYzuy1lpuR6AL30YyGFd5kHCVrhR9LMRtyGR8V4JfNuezr1pd6mFMT97pDPop3N5Wz8yMb2wjbvl5JPp8r7QLuf7oV3613OmEsAPLUogUEiJfZbl6aEgRMJ0pxxMCKrE5XtkL1j7+fPWVHuakZ5jeyM+DrXBtQzTSMdKdZnvWGB5GvFPGAnKnU9/gHP47cbQQqUyM47ciwJV5vTlPMcxmT82MJzww28z72HJpbfzzVPe48KC2m1iBXqD2QMtxM9lTSColSnUAPwNRBdivSdAcJuxf+/Q7D+1LbjZWgHoRbOj/0HjBgdkiQbswTPgw+p94c5BsWH9pMFFizSvuecTQa4GXjajzEoR6zky+7TgjpXpwpW3ad3sfxGycdRFqINbXcrL3tCvf3+eDTwSphsIT2res61qLuvCPQlcBpWSlIFkxIKlq9vVbd3ryd2GSMW4rPnY9aAEsMHG27ckCMzRJKqG1G+FmQsSQvTFATiyxxVSL2CzG0qEr2b35cMH6bG 56Xz3tJG hYPUTdCAvOZSUn5+pzbhFRsvshplfNkil3qtxZ6AkOPL9lH2AhWaq/xdoBoPb1lSq/w0/DvUb6avMsvD9y7WXF/FvFxmokTS6QB+/h7ENSHPbacmLImUa+k5ZC78FVJCjFIMS04i50LOXf+fBgwvbo0kWbowOkvOH/b+qsVZqxCs9hcuLGYT+XIi1yMnWTiNcl6cK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With isolate_movable_folio() and folio_isolate_lru(), let's use more folio in do_migrate_range() to save compound_head() calls. Signed-off-by: Kefeng Wang --- mm/memory_hotplug.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index a444e2d7dd2b..bd207772c619 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1774,14 +1774,14 @@ static int scan_movable_pages(unsigned long start, unsigned long end, static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) { + struct folio *folio; unsigned long pfn; - struct page *page, *head; LIST_HEAD(source); static DEFINE_RATELIMIT_STATE(migrate_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); for (pfn = start_pfn; pfn < end_pfn; pfn++) { - struct folio *folio; + struct page *page, *head; bool isolated; if (!pfn_valid(pfn)) @@ -1818,15 +1818,15 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) * We can skip free pages. And we can deal with pages on * LRU and non-lru movable pages. */ - if (PageLRU(page)) - isolated = isolate_lru_page(page); + if (folio_test_lru(folio)) + isolated = folio_isolate_lru(folio); else - isolated = isolate_movable_page(page, ISOLATE_UNEVICTABLE); + isolated = isolate_movable_folio(folio, ISOLATE_UNEVICTABLE); if (isolated) { - list_add_tail(&page->lru, &source); - if (!__PageMovable(page)) - inc_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_lru(page)); + list_add_tail(&folio->lru, &source); + if (!__folio_test_movable(folio)) + node_stat_add_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); } else { if (__ratelimit(&migrate_rs)) { @@ -1834,7 +1834,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) dump_page(page, "isolation failed"); } } - put_page(page); + folio_put(folio); } if (!list_empty(&source)) { nodemask_t nmask = node_states[N_MEMORY]; @@ -1846,9 +1846,9 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) /* * We have checked that migration range is on a single zone so - * we can use the nid of the first page to all the others. + * we can use the nid of the first folio to all the others. */ - mtc.nid = page_to_nid(list_first_entry(&source, struct page, lru)); + mtc.nid = folio_nid(list_first_entry(&source, struct folio, lru)); /* * try to allocate from a different node but reuse this node @@ -1861,11 +1861,11 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) ret = migrate_pages(&source, alloc_migration_target, NULL, (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_HOTPLUG, NULL); if (ret) { - list_for_each_entry(page, &source, lru) { + list_for_each_entry(folio, &source, lru) { if (__ratelimit(&migrate_rs)) { pr_warn("migrating pfn %lx failed ret:%d\n", - page_to_pfn(page), ret); - dump_page(page, "migration failure"); + folio_pfn(folio), ret); + dump_page(&folio->page, "migration failure"); } } putback_movable_pages(&source); From patchwork Wed Mar 27 14:10:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13606714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E612AC54E67 for ; Wed, 27 Mar 2024 14:13:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D80C6B0089; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 309026B008A; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00CBF6B0093; Wed, 27 Mar 2024 10:13:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BA87F6B0089 for ; Wed, 27 Mar 2024 10:13:21 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5B7E940C39 for ; Wed, 27 Mar 2024 14:13:21 +0000 (UTC) X-FDA: 81943011402.20.9388B30 Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by imf05.hostedemail.com (Postfix) with ESMTP id E700E100026 for ; Wed, 27 Mar 2024 14:13:18 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711548799; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I9Gvvqgigvf1QxTd48ZdzuyzG3NsNGJqDA9cyeSmKQM=; b=Gokd34cYdcwfeOJPGUorwTQEYdn5130LEa3+feSUD1XzdusJoLKCE5ViGJd95FMjOPeb4I hLwd3rx/cbv0xTarYLcPFxmzmIDbNPeqaH8AsELZYFjrJ5AOIGnlAMQ5XhZc1OPewx/5sw qcSeFUBrlXb8LYbnoOGMUNhCZMII1Hc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711548799; a=rsa-sha256; cv=none; b=UCm2cFMSW3CjXQwzQBoDPv+hh4zm0EpejSm6tCpkLEkZN/BIhKku6nmSJBoPuUdfhbTMwC lyvJ4PiyNtHbqEAaPa5DwSGFpYYXtmoegeE0ByfqLhUcy6YmlccjDl+6nGO2I8fNhyCq+y Htb45EinhFOqgBNNF7JhlsMZ3zbvmEU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4V4TBy6DS2z1wnbd; Wed, 27 Mar 2024 22:12:26 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 4197B140259; Wed, 27 Mar 2024 22:13:16 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 27 Mar 2024 22:13:15 +0800 From: Kefeng Wang To: Andrew Morton CC: , Miaohe Lin , Naoya Horiguchi , David Hildenbrand , Oscar Salvador , Zi Yan , Hugh Dickins , Jonathan Corbet , , , Baolin Wang , Kefeng Wang Subject: [PATCH 3/6] mm: remove isolate_lru_page() Date: Wed, 27 Mar 2024 22:10:31 +0800 Message-ID: <20240327141034.3712697-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> References: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Stat-Signature: drngx6ay3547fibd66a5ditymoo3oyjy X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: E700E100026 X-Rspam-User: X-HE-Tag: 1711548798-952738 X-HE-Meta: U2FsdGVkX18DGJ6oezaghV73cl5luAILjaUmBaxheNlnA0e/YOH3Q3O3APhYEC24HWy12SQbeTsmXI3wkck4xQKCVmNjdYLf0rIrFNu1Zd5lUS39v4fLtwIsClYPZEW+3zQrkHBf/hQjURFPOlTki8bD4/GFhzj10Dj48YOtFFhV3IWQnJvYquS60AQuKd35zHIe8MuRuyEUGZbjsxqyYDtINjHU5YnemvJ9/fhGcDVaNLP1iPnD196kaSHd69JxtdotA5J72e5jcuWX5zaZWOUVpNks09lt60Jql1NxDYJxoT1tRbyBwomGS5TRYxyQPFMwPDFYxC24QKzFXZ8IX7GEqOoNGsroO4ReXPOOpv1FzLNoGpbdih48IdGYDrXWZlDkCfsMasYR6yR9/uFRSDPHjiZQGXK8u9Fu718q9vuPIhfI2lRzEPZY3AGZtn9XMs9nCKbpyfwvep1M0jdpNDZPUF95bSjxXjpM1aq+KhTaUOr/Y7CvHW2n6S8si3IvAjyiRD3Yu+uMFSkRXcRmLaNccbEM3B9NciyOP1EnbG0LpqG8bcOSt6Sr8Cn08rb5ujK8BwK16B3YB3M2BsvZYJz1rj9EVDmono9qcgJckTx7WUA1iBCrlYx5B8Dzt5Xlm8iadWi+b4blaosnR5OyGMqerqJeei4DkvudVtJko7mgV6GsRvAj6pXk7t7K40jq0JfR2VxcdMja6mRyjY55alJapBOa55hRsP/x8eeYRY8+lK9ob8xu4vrzmH3om+vyUhUcif5vzfYwZzVGStEb76wJzt0Nt4v85WtGJCTjsTBhhj6myC7+kpm3Xg5ybFHvRZqr0hDC5IHMCTmMZ0RoVYzD1LcNq0qvEd3Oo2u/TK1Ec6te0MxejicPmcEfrHHN9ohZrrPUNRVRHZNWwsqwM+NYbOBDsGh7FA78hS1/wcxLS4K31uJdH+eOtgN4B9z/lz18gY9+b3j70PgCJwV SrQC6ao6 3yHP7jwEBf0veCEolLn3syd/mqgwMmIS8jrWvCqKWELQDRKOCWhYmVHLe5Rsebdz+pg9paFe5c9EbyNuMZghEtRA29px5ZRT3Hw9nSOx5OK6KlpgcK66T5IP1e55ZVbkf0hZK2yr3gx6BfumqjE+OuAAyMe3BIbZDfTsdyQ80uHbC20u9kdfCYjOTO0WAShb5mmZEoAN6Ewz9AqNbNi7liDA/5hmJDgHbfw9O X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no more callers of isolate_lru_page(), remove it. Signed-off-by: Kefeng Wang --- Documentation/mm/page_migration.rst | 6 +++--- Documentation/translations/zh_CN/mm/page_migration.rst | 6 +++--- mm/filemap.c | 2 +- mm/folio-compat.c | 7 ------- mm/internal.h | 1 - mm/khugepaged.c | 8 ++++---- mm/migrate_device.c | 2 +- mm/swap.c | 2 +- 8 files changed, 13 insertions(+), 21 deletions(-) diff --git a/Documentation/mm/page_migration.rst b/Documentation/mm/page_migration.rst index f1ce67a26615..0046bbbdc65d 100644 --- a/Documentation/mm/page_migration.rst +++ b/Documentation/mm/page_migration.rst @@ -67,8 +67,8 @@ In kernel use of migrate_pages() Lists of pages to be migrated are generated by scanning over pages and moving them into lists. This is done by - calling isolate_lru_page(). - Calling isolate_lru_page() increases the references to the page + calling folio_isolate_lru(). + Calling folio_isolate_lru() increases the references to the page so that it cannot vanish while the page migration occurs. It also prevents the swapper or other scans from encountering the page. @@ -86,7 +86,7 @@ How migrate_pages() works migrate_pages() does several passes over its list of pages. A page is moved if all references to a page are removable at the time. The page has -already been removed from the LRU via isolate_lru_page() and the refcount +already been removed from the LRU via folio_isolate_lru() and the refcount is increased so that the page cannot be freed while page migration occurs. Steps: diff --git a/Documentation/translations/zh_CN/mm/page_migration.rst b/Documentation/translations/zh_CN/mm/page_migration.rst index f95063826a15..8c8461c6cb9f 100644 --- a/Documentation/translations/zh_CN/mm/page_migration.rst +++ b/Documentation/translations/zh_CN/mm/page_migration.rst @@ -50,8 +50,8 @@ mbind()设置一个新的内存策略。一个进程的页面也可以通过sys_ 1. 从LRU中移除页面。 - 要迁移的页面列表是通过扫描页面并把它们移到列表中来生成的。这是通过调用 isolate_lru_page() - 来完成的。调用isolate_lru_page()增加了对该页的引用，这样在页面迁移发生时它就不会 + 要迁移的页面列表是通过扫描页面并把它们移到列表中来生成的。这是通过调用 folio_isolate_lru() + 来完成的。调用folio_isolate_lru()增加了对该页的引用，这样在页面迁移发生时它就不会 消失。它还可以防止交换器或其他扫描器遇到该页。 @@ -65,7 +65,7 @@ migrate_pages()如何工作 ======================= migrate_pages()对它的页面列表进行了多次处理。如果当时对一个页面的所有引用都可以被移除， -那么这个页面就会被移动。该页已经通过isolate_lru_page()从LRU中移除，并且refcount被 +那么这个页面就会被移动。该页已经通过folio_isolate_lru()从LRU中移除，并且refcount被 增加，以便在页面迁移发生时不释放该页。 步骤: diff --git a/mm/filemap.c b/mm/filemap.c index 7437b2bd75c1..2a03fbbf413a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -113,7 +113,7 @@ * ->private_lock (try_to_unmap_one) * ->i_pages lock (try_to_unmap_one) * ->lruvec->lru_lock (follow_page->mark_page_accessed) - * ->lruvec->lru_lock (check_pte_range->isolate_lru_page) + * ->lruvec->lru_lock (check_pte_range->folio_isolate_lru) * ->private_lock (folio_remove_rmap_pte->set_page_dirty) * ->i_pages lock (folio_remove_rmap_pte->set_page_dirty) * bdi.wb->list_lock (folio_remove_rmap_pte->set_page_dirty) diff --git a/mm/folio-compat.c b/mm/folio-compat.c index 50412014f16f..95ad426b296a 100644 --- a/mm/folio-compat.c +++ b/mm/folio-compat.c @@ -105,13 +105,6 @@ struct page *grab_cache_page_write_begin(struct address_space *mapping, } EXPORT_SYMBOL(grab_cache_page_write_begin); -bool isolate_lru_page(struct page *page) -{ - if (WARN_RATELIMIT(PageTail(page), "trying to isolate tail page")) - return false; - return folio_isolate_lru((struct folio *)page); -} - void putback_lru_page(struct page *page) { folio_putback_lru(page_folio(page)); diff --git a/mm/internal.h b/mm/internal.h index 7e486f2c502c..7cdf7d3d83ea 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -292,7 +292,6 @@ extern unsigned long highest_memmap_pfn; /* * in mm/vmscan.c: */ -bool isolate_lru_page(struct page *page); bool folio_isolate_lru(struct folio *folio); void putback_lru_page(struct page *page); void folio_putback_lru(struct folio *folio); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 38830174608f..e9b8b368f655 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -607,7 +607,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, } /* - * We can do it before isolate_lru_page because the + * We can do it before folio_isolate_lru because the * page can't be freed from under us. NOTE: PG_lock * is needed to serialize against split_huge_page * when invoked from the VM. @@ -1867,7 +1867,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, result = SCAN_FAIL; goto xa_unlocked; } - /* drain lru cache to help isolate_lru_page() */ + /* drain lru cache to help folio_isolate_lru() */ lru_add_drain(); page = folio_file_page(folio, index); } else if (trylock_page(page)) { @@ -1883,7 +1883,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, page_cache_sync_readahead(mapping, &file->f_ra, file, index, end - index); - /* drain lru cache to help isolate_lru_page() */ + /* drain lru cache to help folio_isolate_lru() */ lru_add_drain(); page = find_lock_page(mapping, index); if (unlikely(page == NULL)) { @@ -1990,7 +1990,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, * We control three references to the page: * - we hold a pin on it; * - one reference from page cache; - * - one from isolate_lru_page; + * - one from folio_isolate_lru; * If those are the only references, then any new usage of the * page will have to fetch it from the page cache. That requires * locking the page to handle truncate, so any new usage will be diff --git a/mm/migrate_device.c b/mm/migrate_device.c index c0547271eaaa..3a42624bb590 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -326,7 +326,7 @@ static bool migrate_vma_check_page(struct page *page, struct page *fault_page) { /* * One extra ref because caller holds an extra reference, either from - * isolate_lru_page() for a regular page, or migrate_vma_collect() for + * folio_isolate_lru() for a regular page, or migrate_vma_collect() for * a device page. */ int extra = 1 + (page == fault_page); diff --git a/mm/swap.c b/mm/swap.c index 500a09a48dfd..decd6d44b7ac 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -930,7 +930,7 @@ atomic_t lru_disable_count = ATOMIC_INIT(0); /* * lru_cache_disable() needs to be called before we start compiling - * a list of pages to be migrated using isolate_lru_page(). + * a list of pages to be migrated using folio_isolate_lru(). * It drains pages on LRU cache and then disable on all cpus until * lru_cache_enable is called. * From patchwork Wed Mar 27 14:10:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13606717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02EC1C47DD9 for ; Wed, 27 Mar 2024 14:13:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9002D6B0092; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8ADE56B0095; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7290F6B0096; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5388E6B0092 for ; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 13C6F160DFF for ; Wed, 27 Mar 2024 14:13:23 +0000 (UTC) X-FDA: 81943011486.15.2437F56 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf05.hostedemail.com (Postfix) with ESMTP id C333A100014 for ; Wed, 27 Mar 2024 14:13:20 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711548801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lk/APelPic84s/Ofn0uIL0rf5ARcImvrvKUTuVPHucs=; b=YorKycOztXG5K7ruhW6UhTVtxZW1nHhCO3JPIVKZO9kvwx/3N6Xe/EJVhSnWY62wiTj65z dtRdqlPL+zBeGvmmSgNX80tRwYBebOZLXqsaZdxgzzMkvdpTSmDJaH4skrpCTPzWfy4gag AWkKh2ZrSncE+8Gkdeo0cF84OeG0OyY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711548801; a=rsa-sha256; cv=none; b=L6OKjMHkg0dRBEWHZKLsvDM7lDragVu4W6a+3Bl2IroQ+QmIk2neL1xrCAxgxFhvHHCUXS +fe4IGrDwNKz3mHbSbW/BAWA3P61EtTu/O5sE9+qHPRnxep2VaOmtvhNYyNkbGeFcJzg48 23Q8t8L7fjTOvL5eK44uwNDo7rQ/PPs= Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4V4TCK5yRZz1GD8d; Wed, 27 Mar 2024 22:12:45 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id D64A21400DA; Wed, 27 Mar 2024 22:13:16 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 27 Mar 2024 22:13:16 +0800 From: Kefeng Wang To: Andrew Morton CC: , Miaohe Lin , Naoya Horiguchi , David Hildenbrand , Oscar Salvador , Zi Yan , Hugh Dickins , Jonathan Corbet , , , Baolin Wang , Kefeng Wang Subject: [PATCH 4/6] mm: compaction: use isolate_movable_folio() in isolate_migratepages_block() Date: Wed, 27 Mar 2024 22:10:32 +0800 Message-ID: <20240327141034.3712697-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> References: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Stat-Signature: g17nb9guqzeh9b4aecrsqnnhnnddxj6g X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C333A100014 X-HE-Tag: 1711548800-531842 X-HE-Meta: U2FsdGVkX18UoW8HjUkn9cgs8sTVu+R0z53krVAzCH2FgcK3mnVaLlYwWi1d9RtrX0VWXqNvIhZgNmfYzULYKk+herwbOr2CunRVY2kuLqzKpepHhR5X2NKLPuRX08b/lOxtqJvLbmUJFrJ+qpLJTH4Xa3/jTCJKauPf9oSFep36oIJ/7k56qdFXJfiD4C8lgSQR5XgN3fI+RfZvNyIlHYJ+cnSkyIGchRKQvWCZeg97BlEe0xVTru+YVERso6vFi8EdvkVadgCe+DX0uTHTX+D7O8JyqfWwqKPj4IhS7id8IaslGj69aKt2goaMZMoIngLQFRYPlqNVWkJkBEP4IZHinP4o2fkjKRFOOuYd7RZSu9cnGH6ix3w/CnJG377udKJuy8NgCObrXSmDSo2E9ljj2r4TKVdTO5geSWrqpv7JuD7/Pk5c4SYVAfuLVBvkTqxJoabQ+eEb9LVUM4BJ/PFZJ1ktr45lviW2deqFc3saaHEOCEgHzbKy4dHcRrMHtWOOsChRI/N9I5mO+/T6JpyzUf7/0i8xtZj5RbD2vyZ+V6NxAHJpBocQXzM0OAw9uUf5ftquxep9SqwSBuw/08SXs2KZkrwpHZFeCxgj/aq9Xj0VEqSst51bv7sgL2bX9RH/IrCfOd/EoeFlXj2NICroFLnbF9BBHUo9zOJ0hyMtgM6IEnsBnpNxg7y/E5xM5b+3nYXysKNnUQIMfNadK62KF8TONvW0ItyD3ysg84CsFn3Ks3fHeaOGrQeGKWgh+TFn3AjVj38nFXEvmFJNSVyAB60W5qaShVUx/y2LGVbn+kE+08Dxmc17IMs2Jj6yDkvVKGSt/P94Kn0bnv6j/gsclVRnnWCJ1XaEw/R427iwlyKD2NuPqNRzpZOtLJusz/CqZ0NwjAOeu60nmaN5t+dQnmfuViPUzpiLPJ2VTNKXUqDyXbzJCD7c69oqRn3daNcITtxkYqIg5pVhXSr +w+QwJbw n2kpgI9cj7Z79xNi9j+yFYYOkSZMNn1atEVlFip7Kaszt1gPrh9PL7zOL1VB63WgVy2kwm8+P62U3DmGPQsbR6jN0c8tsYQMT4TjkLdAKsZ24idWeXfHZ62DLIyWusUx0sSBD4TRKiXNHuCbGWxVnRytzBxaYUExo2SNqueedzysI3/RKKhtx9Qpb4RehApP2JnY5skY93RowyhkL+03oc+mpxgByEFm+P5tx8ByKBSldwcE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This moves folio_get_nontail_page() before non-lru movable pages check, and directly call isolate_movable_folio() to save compound_head() calls, since the reference count of the non-lru movable page is increased, a folio_put() is need() whether the folio is isolated or not. Signed-off-by: Kefeng Wang --- mm/compaction.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 807b58e6eb68..74ac65daaed1 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1097,41 +1097,41 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, } } + /* + * Be careful not to clear PageLRU until after we're + * sure the page is not being freed elsewhere -- the + * page release code relies on it. + */ + folio = folio_get_nontail_page(page); + if (unlikely(!folio)) + goto isolate_fail; + /* * Check may be lockless but that's ok as we recheck later. * It's possible to migrate LRU and non-lru movable pages. * Skip any other type of page */ - if (!PageLRU(page)) { + if (!folio_test_lru(folio)) { /* * __PageMovable can return false positive so we need * to verify it under page_lock. */ - if (unlikely(__PageMovable(page)) && - !PageIsolated(page)) { + if (unlikely(__folio_test_movable(folio)) && + !folio_test_isolated(folio)) { if (locked) { unlock_page_lruvec_irqrestore(locked, flags); locked = NULL; } - if (isolate_movable_page(page, mode)) { - folio = page_folio(page); + if (isolate_movable_folio(folio, mode)) { + folio_put(folio); goto isolate_success; } } - goto isolate_fail; + goto isolate_fail_put; } - /* - * Be careful not to clear PageLRU until after we're - * sure the page is not being freed elsewhere -- the - * page release code relies on it. - */ - folio = folio_get_nontail_page(page); - if (unlikely(!folio)) - goto isolate_fail; - /* * Migration will fail if an anonymous page is pinned in memory, * so avoid taking lru_lock and isolating it unnecessarily in an From patchwork Wed Mar 27 14:10:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13606716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2E2BCD11DD for ; Wed, 27 Mar 2024 14:13:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B21F06B008A; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A897B6B0096; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85F5B6B0092; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5E3AE6B0093 for ; Wed, 27 Mar 2024 10:13:22 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2832640C39 for ; Wed, 27 Mar 2024 14:13:22 +0000 (UTC) X-FDA: 81943011444.02.EE03816 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf16.hostedemail.com (Postfix) with ESMTP id 275C318001C for ; Wed, 27 Mar 2024 14:13:19 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf16.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711548800; a=rsa-sha256; cv=none; b=bZctExHM4BQultKAygWTf1Sf1VAzXy1QQ2xi+UkcEoSlMVvI1cnZ1WQVw9GgJQKUK/Hndw pRsXtjngpN8JYEfaU9cIO/EdnuqqB4jWUrOFyhfMqyuvwL4QLfsKIf/gjPMlZPHBsZCJET rep/Z9Xe0TI2bk+vCqzd6shk4tiK5c4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf16.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711548800; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RtqeoRQ1M6tLLawApqTl1tVkzA7i50CQyDlZi82g7Ag=; b=t6v2zQJLdvn6G5I5bHqk9g2iLcuCmtwKO5Nu1qvsImSUInrXORcIjuYygFzvE8IzjqS2Oq HvVSbXNsx4PPkY6VddNMcii9EJNefhFkgU7j8bPeCGdD9WdjDKxi1GI7t/nPgIX/VHFw3E 8Xmyur2xmaAyYViNh4PN4UN5k8wV2j4= Received: from mail.maildlp.com (unknown [172.19.163.174]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4V4T8k11QvzXjxC; Wed, 27 Mar 2024 22:10:30 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 765B61404F6; Wed, 27 Mar 2024 22:13:17 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 27 Mar 2024 22:13:16 +0800 From: Kefeng Wang To: Andrew Morton CC: , Miaohe Lin , Naoya Horiguchi , David Hildenbrand , Oscar Salvador , Zi Yan , Hugh Dickins , Jonathan Corbet , , , Baolin Wang , Kefeng Wang Subject: [PATCH 5/6] mm: memory-failure: use isolate_movable_folio() in mf_isolate_folio() Date: Wed, 27 Mar 2024 22:10:33 +0800 Message-ID: <20240327141034.3712697-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> References: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 275C318001C X-Stat-Signature: oxq9fnj3etn9bgib6ra9aefz3srs8bnw X-Rspam-User: X-HE-Tag: 1711548799-689094 X-HE-Meta: U2FsdGVkX1/+HbGqEgBUYlxJfzCmzG7FlrKhKcMmdcz160hENYXByjBYSuMtoLGHexFvB1RXO7F/GQ6CgFmQQceXpRh+d7qXhGe/vr1mv3euhmHJRb2TFEhCx36jUfAoKZtpbjOCHiy07/q6aN/wfIlVJuOSW8Tkm6DLO4I6pA8I5zTik3oPw/Ri6lY2qqBzGyi0pq3TJS9LCyJXMpD+mfETkaY+7hVM+axKeZCUbRknjqiXmMl7bqa7He9TLJnZ/nEhYG0HhXPBwEuPQZ4WwdjHNWC/rZkLehqjEg29n6T/E5Nk+To6pHm6fo2Y4DWFJiRzA2ZBpGFYNC8qWl7AzLBn92/mUdpsZ4sOYsMATYed/F1Q5pmYF67gWFxCtqM4DsLw8wWC99UKn9Gm8wpAg1eNSjw1tEXh+n6YXxUYRGlbzptrRFVm0+Ze6QbwLm6hbmTbVL9lQiIUcZsKtqaWdcGfOsg6FigdkjzpM2XXRpj+XyPmAIGG03m1Ox/5zIs1y1RTqSRDu8Ylu0LZBTVWHK5tv+8HIJ7aD1ND/TsvjXHtgWSQSf0DhXkA9zd3cAydfqp23kS3SXsuYFXmPqk8ggmEXvSkP8wZZg8dm9vmDtrS41r8iiXjDqKbVZAV2sogw9JLfF5v7Th6HmZ6lAhCCJElkkIExHDvUvl81z4bboS5bIaZaDTIgTzWdQHKqbWmoYeV36ikQwwdGBUkykdflsLb7MjAOu+fU/PQl9hTxcxjnGdgyuoupE3KVtSjnmBeX34pylkXJYO4kzBrXJKlLq/uvmx/1bYLIvLssGTnIF6Km7jS82MnEPQeQMNREhDEBVsXzTc1Szlu5h1iA6HPvzhTpKajLWvTBdwjW11VmkOo29KnxO1cIq6t+iBU7kACTQsqeDy1kKQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Directly use isolate_movable_folio() helper in mf_isolate_folio(). Signed-off-by: Kefeng Wang --- mm/memory-failure.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 9349948f1abf..6f47776df0e1 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2628,8 +2628,8 @@ static bool mf_isolate_folio(struct folio *folio, struct list_head *pagelist) if (lru) isolated = folio_isolate_lru(folio); else - isolated = isolate_movable_page(&folio->page, - ISOLATE_UNEVICTABLE); + isolated = isolate_movable_folio(folio, + ISOLATE_UNEVICTABLE); if (isolated) { list_add(&folio->lru, pagelist); From patchwork Wed Mar 27 14:10:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13606718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CB48C54E67 for ; Wed, 27 Mar 2024 14:13:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF3846B0095; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E79B06B0096; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D411F6B0098; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AD76C6B0096 for ; Wed, 27 Mar 2024 10:13:23 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 66E9B40111 for ; Wed, 27 Mar 2024 14:13:23 +0000 (UTC) X-FDA: 81943011486.30.E7063F4 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf28.hostedemail.com (Postfix) with ESMTP id 1FB39C0011 for ; Wed, 27 Mar 2024 14:13:20 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf28.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711548801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/uUXrt6K3BPM0oU8HqR1j8Mmb0n1YcCC6d199AoQuBU=; b=HnDRXG+xtanyOR1sNRNDJHceLK6X6ggkOvKJX6xvLOY4J2mydTVJctwQrf4NaiVJU+5zQV gj8fTotXr/4UKYSGTiRM/LCIgC3mMrrNQRgJ0816mWSDlAOyJBLu/sPgnQBqqozxHypZ8O z0VNZeJOleZPKT+qyKE0ZktDAn6oMtE= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf28.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711548801; a=rsa-sha256; cv=none; b=OmlhqMOCIyqutTS4BTobqZfJsJhOl9/mXZ/z8RbQDBxmZvBF146XZ+bkQonqjkiSLGsCjq nKFBQ+W5IFEAgm6qVBegcooSBDQt+AnMicB0OZYCzhifL6TjkeHU5QlTJOUGAaUea3gODL cITMvdpcXan+YUEpSXyZkzwZNsv33g8= Received: from mail.maildlp.com (unknown [172.19.88.105]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4V4T9860pXz1QBQx; Wed, 27 Mar 2024 22:10:52 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 1B83A1402C6; Wed, 27 Mar 2024 22:13:18 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 27 Mar 2024 22:13:17 +0800 From: Kefeng Wang To: Andrew Morton CC: , Miaohe Lin , Naoya Horiguchi , David Hildenbrand , Oscar Salvador , Zi Yan , Hugh Dickins , Jonathan Corbet , , , Baolin Wang , Kefeng Wang Subject: [PATCH 6/6] mm: migrate: remove isolate_movable_page() Date: Wed, 27 Mar 2024 22:10:34 +0800 Message-ID: <20240327141034.3712697-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> References: <20240327141034.3712697-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Stat-Signature: unuwbftimc5jtz1etd3ijedrdqw47e3b X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1FB39C0011 X-HE-Tag: 1711548800-53466 X-HE-Meta: U2FsdGVkX19+XanMw1YfAVvmJLOdYSQ9tlDQLmmaUzv0JDNSHZTX74ng4N5zoKg6gtqUDOu/VXKAQKHuVv816HxQSciuo+SqRmGy+2PGsRB97E+GClKUP2gjKHYlRM3/MINU19+7xMWomLvlBgDKdDWL5Fzmb+C5wf4kiuNprD9SCJXqaQhjo62N6mbf7R+G+NHk/Z4bVGY96ur9b6h95aCgoOcL6C59j5IZmgXwcYo+RR6Z9F7lp8PGp/RWz0ARElLeMcpoAZvydeFfN5rqPBL5jdG2bHTr3BdynpGy9lNZISuN0YP0X9C8yN4sUgyG9Qu9sUwlogk0lsK7lHAvIqkp7L9MNfZkI6Zyqe0qmBfRPxbK5TTVOPb4tH8eB6hMv2xqWEStEe38cXfQ5Fv8PTZfp24NvK5RPJNn+NBaxvmR8/Dd64x8qkxSg/baYuz3l6qksq2cmx3qFAsks0Qh4178IlHTjjKthLmc3a4Pn62li64bDdYNB4Bo+BlkDlALrEfXa/p8k9n79/bkXTKg/9zRPY3r+ohQceweVwa5CAb+2Lls2YPI0KC8x5APlfj8SueYK+fVQvOehnHeJ2gjSvvrH3Ab3oOFE49AH4cICHryRfQuKd+z7Sho7OVZviOHeumzeL08KDOy8y0xZKhiy+kbZyEE83LwzhMnFO24q2b1kCSSGLlskKQg3vcIOkkdfkhRTGkyfV+U8oeHNSlT6JwZ/LatxIcjeqeKAaIgCYh1ybE0Tz2Tt/v0bO4vFxrldifvmHFkcP/+UKyYprZePcKitzjAXsmp2ld7aWAUdVhEIsENuE9+XbO9g39k6+5jwqbyHtqvM6g1CsonN+ZOboT3X1kl78APMdtAc1mZjmSY509PcQtK8kNdA3KdHjffoY3XawUyMpaFzxbnTFqzDWOPFjLVmrnR+Cry7Z9IqKZajq3Ln4lTqJQxPCdzteBswvrxgol0xOox7QbxgZa sIiFwZ22 MGznBOefPAglTeU2PLH+g7ZrClr2QqzQotx+ognQKV3DezFQg3WVHoMqE3Kh33bW47mQI8fhfBIXa/JdpNOTN1GLUMoHY+RlPfN+dPjkDKteWjaL7uMJ5zUVQOTI6ZGhqZrTGTLe/nKTrwpOI3zPGahsv4V9SUSF5TcZxCseXMDBOl3qrUfnM3eAnNHrateQW0xXs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no more callers of isolate_movable_page(), remove it. Signed-off-by: Kefeng Wang --- include/linux/migrate.h | 3 --- mm/migrate.c | 8 -------- 2 files changed, 11 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index a6c38ee7246a..2b680b939020 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -69,7 +69,6 @@ int migrate_pages(struct list_head *l, new_folio_t new, free_folio_t free, unsigned long private, enum migrate_mode mode, int reason, unsigned int *ret_succeeded); struct folio *alloc_migration_target(struct folio *src, unsigned long private); -bool isolate_movable_page(struct page *page, isolate_mode_t mode); bool isolate_movable_folio(struct folio *folio, isolate_mode_t mode); int migrate_huge_page_move_mapping(struct address_space *mapping, @@ -90,8 +89,6 @@ static inline int migrate_pages(struct list_head *l, new_folio_t new, static inline struct folio *alloc_migration_target(struct folio *src, unsigned long private) { return NULL; } -static inline bool isolate_movable_page(struct page *page, isolate_mode_t mode) - { return false; } static inline bool isolate_movable_folio(struct page *page, isolate_mode_t mode) { return false; } diff --git a/mm/migrate.c b/mm/migrate.c index b2195b6ff32c..f50ed046ede3 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -125,14 +125,6 @@ bool isolate_movable_folio(struct folio *folio, isolate_mode_t mode) return false; } -bool isolate_movable_page(struct page *page, isolate_mode_t mode) -{ - if (WARN_RATELIMIT(PageTail(page), "trying to isolate tail page")) - return false; - - return isolate_movable_folio((struct folio *)page, mode); -} - static void putback_movable_folio(struct folio *folio) { const struct movable_operations *mops = folio_movable_ops(folio);