From patchwork Tue Aug 27 11:47:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13779374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79E3DC54734 for ; Tue, 27 Aug 2024 11:47:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FACD6B0098; Tue, 27 Aug 2024 07:47:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 09A0E6B0099; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE8836B009A; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 943BD6B0093 for ; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4D1B4161814 for ; Tue, 27 Aug 2024 11:47:44 +0000 (UTC) X-FDA: 82497850848.04.600E830 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf08.hostedemail.com (Postfix) with ESMTP id B6177160006 for ; Tue, 27 Aug 2024 11:47:41 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724759218; a=rsa-sha256; cv=none; b=Yr0pccVvy0W7UvgkDLEBm1F+rCIqKAojY6kiX0a8UNikMU8H7EoCD2HcKJ2ZH75GvXXTyP UyGN5PylgOamoM8C0bxmIkpecK2TRAk5QXCDLvvcUOvIp7V5qMR5IJe8PeQLWrd0nMnXJz /fVgaMoPnvl2xu3DoEyAGEWQ8FEwdYY= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724759218; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1bhUTx7nnx6vHbE5W0f0Ci/xujChDyw8hckCCgIY/zg=; b=X3iv48dFDbNYJY64Avw/fqcX5W63x9OFTJIrJPh4dVnxfN7XMxTABvPHRxnXGTmrjBamBS rVFxLHBXX3O9mUkdgIjUwv/LoXhVpPD130+WhhYnL07jfWby1+naMkbhzU41EUBIqFdfDl EWGON5BNuG7T6Ar0bqsdg1qqUAaw3Do= Received: from mail.maildlp.com (unknown [172.19.163.17]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4WtQdj0Z3bz20mrT; Tue, 27 Aug 2024 19:42:49 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 604981A0188; Tue, 27 Aug 2024 19:47:37 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 27 Aug 2024 19:47:36 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , , Jonathan Cameron , Kefeng Wang Subject: [PATCH v3 4/5] mm: migrate: add isolate_folio_to_list() Date: Tue, 27 Aug 2024 19:47:27 +0800 Message-ID: <20240827114728.3212578-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> References: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf100008.china.huawei.com (7.185.36.138) X-Stat-Signature: 6m9q7zgy89odi7xzm739umcx6i34z4sj X-Rspamd-Queue-Id: B6177160006 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724759261-784442 X-HE-Meta: U2FsdGVkX1+wOOjTVxywjdDi/3pqScNmsPgNx6e/i6C6ATz8KAmHihYyDj7mBvVrERWji8x1bErPKfqN3hdJadXqy1mRG8JGtZKkDVb7GpEa69UwSntimup6w0y7AXmFA8iSFAoNhtExpVdHa/FlWF7O0uC0rMR7l3FVcbSoPvh53NWOFL3MrBLXCLGjz1tKZWImATW5LCLl9B/bSIx0BBSNxVwVNiZvzZ/N4nfTkE9KvRmFDEDZ1tn9kzK/LIkRZlfN7BYaYXY7e5RSREXsBIs7ofEoOCSI2RHBxTMgABcrUIrNHwECfYKZLHfYA6FrK8tS4AiLm7l4g7xj+xDYaZR6AnEaWgBgsWG+wVsjho/grUyxpLnj6A70UeJole1XVppYTanQk2llHwLdtGxv8dI0kyytWSYEsDE8eu+X5c6O4SxtOkG8nixhpnFmKAjcy+1eSmvj9x7gN1XaWW/MK1mqo3gpwzV7wTMCxN10v2rECQgn8WsXlanS1cpkv87PYQ4PnKlH0w4WEa+cjRgX0YSEP/IjWzZ5OETdwWtHBzazTrCINsSB9dmUV45WokTL3M0PjNBS2EJL+CNIAsBeEsclZJH/dFZH/xl7+wK1kgAMlhhjPxSsIITBufYwgH+rtS/qhNGuAVwoFnHOTT4lLUnT79ekVTPQU5vjmYSa+VZ8KADlRZ7njdfRP5dEiYSHlbVntfwHkBiFdJDBPYSpXTnDjF7ADCW9qpOXIAEu9M04mDicvQW9udndDKyADzaonyVLmOU/oXGgBnohSUbs6WF6Eh7VeqRRO5pueqsnOQzavxwdTeq+F6vPxwhRH6WlaMoVxc4luofVGlHG/wHOwWEATDI9w94WpJznU86lnjgmos/ibidFPSyBVB2z8v5JTygfQKZ/jMyvv3s5c+7i2TudeOGg4USONNkWeB8i0Vo+gOeTXdPfdVzkjHU/VJfCX1WOW3UA2hv65eK5vkv 4PQQr3Cd 9LiJeAokYOSjtXyj1iJDeVZgclEUXTGaj44+5aF4/1k64+6PY70VrQnq1CEIb9cjxqK5ZNLnJ70cJB7gdScamc3PnT9jFyYOVOFi4uF1Mq6OZuz2TZTKPIJ0JNmHLqBsvtQV9/wZWIOAKcclyN7xJ9jKceZNh4F8KR6HultKAwvE4XcJkzmqmcwI0F30DOOEhUNi7HVNtisbHKV1+BmN1lornmRoaus5Z0vzxTZL1esn8PIMTLQgbEJrnKyeFIRZkcFsqrOtgSNjBilnGcpLEBwHBtbnZS1+Wj8X8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add isolate_folio_to_list() helper to try to isolate HugeTLB, no-LRU movable and LRU folios to a list, which will be reused by do_migrate_range() from memory hotplug soon, also drop the mf_isolate_folio() since we could directly use new helper in the soft_offline_in_use_page(). Acked-by: David Hildenbrand Acked-by: Miaohe Lin Tested-by: Miaohe Lin Signed-off-by: Kefeng Wang --- include/linux/migrate.h | 3 +++ mm/memory-failure.c | 48 +++++++++++------------------------------ mm/migrate.c | 26 ++++++++++++++++++++++ 3 files changed, 42 insertions(+), 35 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 644be30b69c8..002e49b2ebd9 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -70,6 +70,7 @@ int migrate_pages(struct list_head *l, new_folio_t new, free_folio_t free, unsigned int *ret_succeeded); struct folio *alloc_migration_target(struct folio *src, unsigned long private); bool isolate_movable_page(struct page *page, isolate_mode_t mode); +bool isolate_folio_to_list(struct folio *folio, struct list_head *list); int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); @@ -91,6 +92,8 @@ static inline struct folio *alloc_migration_target(struct folio *src, { return NULL; } static inline bool isolate_movable_page(struct page *page, isolate_mode_t mode) { return false; } +static inline bool isolate_folio_to_list(struct folio *folio, struct list_head *list) + { return false; } static inline int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 67b6b259a75d..7da3697b33f1 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2659,40 +2659,6 @@ EXPORT_SYMBOL(unpoison_memory); #undef pr_fmt #define pr_fmt(fmt) "Soft offline: " fmt -static bool mf_isolate_folio(struct folio *folio, struct list_head *pagelist) -{ - bool isolated = false; - - if (folio_test_hugetlb(folio)) { - isolated = isolate_hugetlb(folio, pagelist); - } else { - bool lru = !__folio_test_movable(folio); - - if (lru) - isolated = folio_isolate_lru(folio); - else - isolated = isolate_movable_page(&folio->page, - ISOLATE_UNEVICTABLE); - - if (isolated) { - list_add(&folio->lru, pagelist); - if (lru) - node_stat_add_folio(folio, NR_ISOLATED_ANON + - folio_is_file_lru(folio)); - } - } - - /* - * If we succeed to isolate the folio, we grabbed another refcount on - * the folio, so we can safely drop the one we got from get_any_page(). - * If we failed to isolate the folio, it means that we cannot go further - * and we will return an error, so drop the reference we got from - * get_any_page() as well. - */ - folio_put(folio); - return isolated; -} - /* * soft_offline_in_use_page handles hugetlb-pages and non-hugetlb pages. * If the page is a non-dirty unmapped page-cache page, it simply invalidates. @@ -2705,6 +2671,7 @@ static int soft_offline_in_use_page(struct page *page) struct folio *folio = page_folio(page); char const *msg_page[] = {"page", "hugepage"}; bool huge = folio_test_hugetlb(folio); + bool isolated; LIST_HEAD(pagelist); struct migration_target_control mtc = { .nid = NUMA_NO_NODE, @@ -2744,7 +2711,18 @@ static int soft_offline_in_use_page(struct page *page) return 0; } - if (mf_isolate_folio(folio, &pagelist)) { + isolated = isolate_folio_to_list(folio, &pagelist); + + /* + * If we succeed to isolate the folio, we grabbed another refcount on + * the folio, so we can safely drop the one we got from get_any_page(). + * If we failed to isolate the folio, it means that we cannot go further + * and we will return an error, so drop the reference we got from + * get_any_page() as well. + */ + folio_put(folio); + + if (isolated) { ret = migrate_pages(&pagelist, alloc_migration_target, NULL, (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_FAILURE, NULL); if (!ret) { diff --git a/mm/migrate.c b/mm/migrate.c index f8777d6fab57..6f9c62c746be 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -178,6 +178,32 @@ void putback_movable_pages(struct list_head *l) } } +/* Must be called with an elevated refcount on the non-hugetlb folio */ +bool isolate_folio_to_list(struct folio *folio, struct list_head *list) +{ + bool isolated, lru; + + if (folio_test_hugetlb(folio)) + return isolate_hugetlb(folio, list); + + lru = !__folio_test_movable(folio); + if (lru) + isolated = folio_isolate_lru(folio); + else + isolated = isolate_movable_page(&folio->page, + ISOLATE_UNEVICTABLE); + + if (!isolated) + return false; + + list_add(&folio->lru, list); + if (lru) + node_stat_add_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); + + return true; +} + /* * Restore a potential migration pte to a working pte entry */