From patchwork Tue Aug 27 11:47:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13779370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CC94C54735 for ; Tue, 27 Aug 2024 11:47:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 248226B008A; Tue, 27 Aug 2024 07:47:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F7B86B008C; Tue, 27 Aug 2024 07:47:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05D0F6B0093; Tue, 27 Aug 2024 07:47:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DC6006B008A for ; Tue, 27 Aug 2024 07:47:42 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 67FEF81768 for ; Tue, 27 Aug 2024 11:47:42 +0000 (UTC) X-FDA: 82497850764.16.85B28C2 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf30.hostedemail.com (Postfix) with ESMTP id 3D1A480007 for ; Tue, 27 Aug 2024 11:47:38 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724759174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3cgPSm+/FxyFHnCmNGNZx2+AN3lgsXCtk/zttMxp4pI=; b=Yr7APrgJhRNz53KBiQPUoLyPEIfUyqvkZeALRaD2cZmCTtOrtvgqQLOEJnpO59P1hCsONr YMEYC6sukHaNi7UqI+wJCOModo6zqD7yo8rhiLrTDFB2/ss22J4jItJeA9UeuWl+QG/wzo obWO+nTFz/ksMqxB8azhQB2FFJsWt3c= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724759174; a=rsa-sha256; cv=none; b=MvU98ri42ckicjDZFIdACH7NAN+cZ3AI0pLSj/gqlUdiVr9mcUvdxMk+B4ouo8g/1JNfgR dictSqtnsAoEgJtdOKoXRKdUav5vFqOrAoj7e6ajbeKRNz5cyNDpJ7IUBx2cgDZDPkx0Pu HmO5Rrq+sEECWVmXeaoY2nsQT72wumc= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com Received: from mail.maildlp.com (unknown [172.19.88.105]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4WtQjH21BPzpTX1; Tue, 27 Aug 2024 19:45:55 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id D1ABE140137; Tue, 27 Aug 2024 19:47:35 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 27 Aug 2024 19:47:35 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , , Jonathan Cameron , Kefeng Wang Subject: [PATCH v3 1/5] mm: memory_hotplug: remove head variable in do_migrate_range() Date: Tue, 27 Aug 2024 19:47:24 +0800 Message-ID: <20240827114728.3212578-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> References: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspamd-Queue-Id: 3D1A480007 X-Stat-Signature: gck4fm5x6zf36rtfd4skc5arjotxf5pj X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1724759258-551733 X-HE-Meta: U2FsdGVkX1+HEzuCKw5tvu9YQDM52ftfWzadRIHMzuPR2ZPFsAYRsbIM0GJa8SebFa7P1eaaKORqldmRH1jdRYj2bjeGx7FqfNdo7KKgL1dgfwQoIhIPBkiZ4CO6+/xZqUHSpgzLipEpN/iVu5rxshoioz8lVWDXXovSXDRptgu16JUQF6eGjxt26s3tkz6iuyppwCe3rbvVuWzKKakK72yv3re5PVQ32fjTcG+nX8BRyDmFgRam2Z5rnkUdFo+N2DyjUVDekYsWSIbgo2P0DQAFJGuOWQoW0m+KgT83DW00xqpd0L8M5Mn2dEc4qV4vU7A7ZjlRLvQBe3XC6ra6qUOPm3NBh7OtnUuY8JeqbiYkYo4JjIQqKhqWRUvsQ22toHcF1iEVpaXAa3zhqUfQnw0CSHwWlutin9ZL3N2m0ja31r0C1oEVz2bkAwNaf+dOJi7T8q7SZnAn/T0B9OFO/VPueF9NdV6ZWG/ayERZ+eAk+uS2K8Vr58LLOOTGJIbjJYXoCPM85jYsC9bX8zSoPTImtbqY3AhQYwvStKVXhcouTirFqKTFDTUR2j+KTicyWJpYV5KW800WO6UOcjYjyzoAWXXTi5nQSR4lcd5I5BiBANcJDCgV3E5/+kMUQvGnSGJkqpe2pd9yZ8cOb+t0+b/9Aq2JdbGOjGwdSJ3mP6lFgFahzqIc4pomLqPUN93QGiyonTgOzpMag8dFCi/R6kDHPOi5vMu38z+8mHjLKJZ25ICv4Zdxn6N4jNXp5JEJYjIEam5W1J35vgYvXvIMaeFfz8e/TVqpST7QCItCwEiBvTZJsepD9iSIRtwovAHoliEPUwisq0GEPlIeLL/8l+vuAYpNM8ai1/tmLG4jdQUg4tetHV2sUFANR1OSEE0jjfMHgPUanJiK7YN3LSONUztwrZRnUq5mNK2FfGzzQZ6aU2wwkmawHxlAIiLHDByZTt+1e77q/xdq43kLSFY jBEV9QtV WQFT6SLEMyiDnlO/BGblH0Kz+amUIEoTYX8lOV5Uy/ks2MOrrfF2O/8aZNMTGUoqawGV2zDDwktsiIhk4WcYe/MUtaf/j2f7X0nUjLIFpyIDmMzDF/SvsVxTD4XxSvnprw5wTILMx+YfBeV7Jynfxv5aVA1HwUpB+tfZQ7Pug3pO1rn9D1i1B/27+GqlqXj95zmzcQg+fCfK43WpH9xLbtx2xo+UveIXQFjWZpHJQ6SVwSScb7AZKX3HYo219L1ch9PjzWPFx65eWItJlWULqouMc4vN+gv94khsG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Directly use a folio for HugeTLB and THP when calculate the next pfn, then remove unused head variable. Reviewed-by: Miaohe Lin Acked-by: David Hildenbrand Signed-off-by: Kefeng Wang --- mm/memory_hotplug.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 38ecd45c3f94..9ef776a25b9d 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1773,7 +1773,7 @@ static int scan_movable_pages(unsigned long start, unsigned long end, static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) { unsigned long pfn; - struct page *page, *head; + struct page *page; LIST_HEAD(source); static DEFINE_RATELIMIT_STATE(migrate_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -1786,14 +1786,20 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) continue; page = pfn_to_page(pfn); folio = page_folio(page); - head = &folio->page; - if (PageHuge(page)) { - pfn = page_to_pfn(head) + compound_nr(head) - 1; - isolate_hugetlb(folio, &source); - continue; - } else if (PageTransHuge(page)) - pfn = page_to_pfn(head) + thp_nr_pages(page) - 1; + /* + * No reference or lock is held on the folio, so it might + * be modified concurrently (e.g. split). As such, + * folio_nr_pages() may read garbage. This is fine as the outer + * loop will revisit the split folio later. + */ + if (folio_test_large(folio)) { + pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1; + if (folio_test_hugetlb(folio)) { + isolate_hugetlb(folio, &source); + continue; + } + } /* * HWPoison pages have elevated reference counts so the migration would From patchwork Tue Aug 27 11:47:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13779375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85A6EC54735 for ; Tue, 27 Aug 2024 11:47:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A44D6B0093; Tue, 27 Aug 2024 07:47:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 556B06B0099; Tue, 27 Aug 2024 07:47:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F8DC6B0095; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B42F26B0095 for ; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 605F5C1683 for ; Tue, 27 Aug 2024 11:47:44 +0000 (UTC) X-FDA: 82497850848.19.61CE56D Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by imf19.hostedemail.com (Postfix) with ESMTP id E2E191A0013 for ; Tue, 27 Aug 2024 11:47:41 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724759175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4LqG7P9tKxA4D8bgnxrOp0BxRnJqnZUSIYcuWvFHJXg=; b=BkMeELlEI+UOH+jrP1DVfZL5ebbLn6t247VlGNjwt9WVMluaAXz+u5fsPV95VyETqWhaJq LmyQy12tQdrk2A1wI9B6x5JlYPpER5ES6RKGU2ayT2AZcgv0mBTaQCd6LtjOh5TArmJn4F M9E+8WIJ9ytXO2oSU2B1kB7XYuQnXJ8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724759175; a=rsa-sha256; cv=none; b=nK69BoAZcJReY4ck44aW/mvD2cVD5ASiVjJ0T81qGIyWx+WEaihAq2C6CLuIAsXf0xmBB3 H90Le8p6b84qH88KW46srt3cznszfJcwXWrwoTRBcP5HG0xOCRiSl0hsOJHx/7kP0SukWg E4EOiSfH4Ra0N8QcDZ9G64DuN7LkQsI= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.88.163]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4WtQhz147xz1xty8; Tue, 27 Aug 2024 19:45:39 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 5DD55180019; Tue, 27 Aug 2024 19:47:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 27 Aug 2024 19:47:35 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , , Jonathan Cameron , Kefeng Wang Subject: [PATCH v3 2/5] mm: memory-failure: add unmap_poisoned_folio() Date: Tue, 27 Aug 2024 19:47:25 +0800 Message-ID: <20240827114728.3212578-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> References: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf100008.china.huawei.com (7.185.36.138) X-Stat-Signature: reod3mb1bmfx1pgt8gxood8hpb65pciu X-Rspamd-Queue-Id: E2E191A0013 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724759261-768890 X-HE-Meta: U2FsdGVkX19nt12nUxP8FLSWS2dio4VOZLA7qYQuDQI8IDS5K5AY4mXSCtPw6C/CVKR9szIjBZicO9neSJN+WJ4UMDi0q9lITNqnpON22IQrfBqn6OgNw+hnMN3LNqi/sipO5aHOfT7A3NKSEA6eBSf70yBkTAbykQDBNrzd89/2V4Y1D+pyafdA+k2n7MdVJykd3cxBhUGStDxg+eK1rqj+7M97mqRbkbmfh/GKVrGaGvvRsXSqXKYIKmAPFQQSLe3pqzdyLyceMpzWttYnWZKb6gfhDHYSOYzUYGLMYFtANF4lWQgm0PWVlS4G66SAdRleqLQnMbjyV1zBMtWpSMejq2beceNCMiOYokGjNmA6+84tW7a0yXGNoOJPaGLx4uDd4u5TXoxfgtBYsi+BhKZB9QcvQg2MdO1CeCl6X6EozSpY7zxOJgW02GOOr/LfXggiK3UIm2hjKKyr10P/H3r/rUP7hsGHBbZoqHQtparE5Y5kA8MjnEyXOUo3/xiCWWocLtqfFGEfn1XBeGjfCMzR2e1fKs4OSKtxCu4DqW55ooeeBbeCk0PVGkCq130PwOJf8LLNz/w1JkejlVql2gR3HPgyeRs4QvUMyXAFEWrSpYBgwxbKjbALPoLA4AaFlCTqVH6BlQ3hvVcjsMipyvaGLJSO4jQ3Wo9SlWJaAjGBWSSo+IpXCrn+Rb2N/NED9eVyPezj6kMiuJcNHaeUvbmiIHsHpaAdqK1+Wt4s11gAJflWNIScldXtAxqSKf9ZIaVsf8ow84nGf6xyQZCtuommysuP2krp4BXitudQX6z8ld4xZNogvzy/VDJGtJXxrREjqmMWjda+l7Xrq5bt0SdvFEeYgk6JhcztQb/3SBrjpHujVJcA8aRRkbRaPxohyJpwqMwzMZwIKJ8CC4G8dFYahialLq4oY3vKDS4l9i3r5objkeIM1FDTtuY/1m1HoUiC+9ILJtj8sGNAqz9 /nDLXZSs VWaRTn2oaNgnavQfHAz6H6VV4IkfA3Aa6PO8yrOlAtCSSbbx965cGsGg7kIhPaOJd39YhYSFPulE0d8UTq77nqtGpwVMsg7BdrzPAHcgVeAEnG+DPoVPxRpwSGlTzxAGIgVfbn4JVOHrKQoVo+73pOoUbXON41ajc5j+OgoImWhiRAbpgJSRCLxBQopjWh/rc0pTLoSgCFH7gn0v+/j78zGTMAJ+EzNGtsJ+6uawK9qWnxw6ai+HArUdaWCWuyX0XV872Z4FsajLY3Hz1Rv5P1ghqUkH0cmiMevRY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add unmap_poisoned_folio() helper which will be reused by do_migrate_range() from memory hotplug soon. Acked-by: David Hildenbrand Signed-off-by: Kefeng Wang Acked-by: Miaohe Lin --- mm/internal.h | 8 ++++++++ mm/memory-failure.c | 43 ++++++++++++++++++++++++++----------------- 2 files changed, 34 insertions(+), 17 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5e6f2abcea28..b00ea4595d18 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1048,6 +1048,8 @@ static inline int find_next_best_node(int node, nodemask_t *used_node_mask) /* * mm/memory-failure.c */ +#ifdef CONFIG_MEMORY_FAILURE +void unmap_poisoned_folio(struct folio *folio, enum ttu_flags ttu); void shake_folio(struct folio *folio); extern int hwpoison_filter(struct page *p); @@ -1068,6 +1070,12 @@ void add_to_kill_ksm(struct task_struct *tsk, struct page *p, unsigned long ksm_addr); unsigned long page_mapped_in_vma(struct page *page, struct vm_area_struct *vma); +#else +static inline void unmap_poisoned_folio(struct folio *folio, enum ttu_flags ttu) +{ +} +#endif + extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 353254537b54..67b6b259a75d 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1554,6 +1554,31 @@ static int get_hwpoison_page(struct page *p, unsigned long flags) return ret; } +void unmap_poisoned_folio(struct folio *folio, enum ttu_flags ttu) +{ + if (folio_test_hugetlb(folio) && !folio_test_anon(folio)) { + struct address_space *mapping; + /* + * For hugetlb folios in shared mappings, try_to_unmap + * could potentially call huge_pmd_unshare. Because of + * this, take semaphore in write mode here and set + * TTU_RMAP_LOCKED to indicate we have taken the lock + * at this higher level. + */ + mapping = hugetlb_folio_mapping_lock_write(folio); + if (!mapping) { + pr_info("%#lx: could not lock mapping for mapped hugetlb folio\n", + folio_pfn(folio)); + return; + } + + try_to_unmap(folio, ttu|TTU_RMAP_LOCKED); + i_mmap_unlock_write(mapping); + } else { + try_to_unmap(folio, ttu); + } +} + /* * Do all that is necessary to remove user space mappings. Unmap * the pages and send SIGBUS to the processes if the data was dirty. @@ -1615,23 +1640,7 @@ static bool hwpoison_user_mappings(struct folio *folio, struct page *p, */ collect_procs(folio, p, &tokill, flags & MF_ACTION_REQUIRED); - if (folio_test_hugetlb(folio) && !folio_test_anon(folio)) { - /* - * For hugetlb pages in shared mappings, try_to_unmap - * could potentially call huge_pmd_unshare. Because of - * this, take semaphore in write mode here and set - * TTU_RMAP_LOCKED to indicate we have taken the lock - * at this higher level. - */ - mapping = hugetlb_folio_mapping_lock_write(folio); - if (mapping) { - try_to_unmap(folio, ttu|TTU_RMAP_LOCKED); - i_mmap_unlock_write(mapping); - } else - pr_info("%#lx: could not lock mapping for mapped huge page\n", pfn); - } else { - try_to_unmap(folio, ttu); - } + unmap_poisoned_folio(folio, ttu); unmap_success = !folio_mapped(folio); if (!unmap_success) From patchwork Tue Aug 27 11:47:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13779373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 693C9C54736 for ; Tue, 27 Aug 2024 11:47:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA6EC6B0096; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E86D6B0098; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7EB836B0096; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5E9F06B0093 for ; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0B120121836 for ; Tue, 27 Aug 2024 11:47:44 +0000 (UTC) X-FDA: 82497850848.22.7623A45 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf26.hostedemail.com (Postfix) with ESMTP id AF18C14000E for ; Tue, 27 Aug 2024 11:47:41 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724759175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dRrcDQgatmZIkVMhpufPY7A0+CG9AIQSYvm0gWw8fYo=; b=rDfnmmqYklzSnTWGWpSl7y9iXltXZ4OyvMrElZJpFzIQy/12ek1HN635DUfNwhbhtiMaJG 5bG/G9aTGBdfLrnryqqsRy6cMONF9uUoIEww8w/JO+Y17ivjpOnpLr08xA7Q9QwqTj8R2P F6ksi+xjKjUcKQ7m00wkcY8WWYg6vXo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724759175; a=rsa-sha256; cv=none; b=J8hMhm8ANe3rgT+bbtbOMP6Z3dYkqvdJgZcoKFj6LjfG5AsVNMXcRLKFlOZNorLDXz4nwD OCEm4XqFUBU8W7Mr/drhssAPLYulqkBTF5dLbYupF9nnxtVKMJK5h9/FQ3Nl08YLzADU1w lobsBg3XH+kOSsEJKfho7u+H9VkVado= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.88.194]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4WtQjJ27v7zpTSk; Tue, 27 Aug 2024 19:45:56 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id D710F140202; Tue, 27 Aug 2024 19:47:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 27 Aug 2024 19:47:36 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , , Jonathan Cameron , Kefeng Wang Subject: [PATCH v3 3/5] mm: memory_hotplug: check hwpoisoned page firstly in do_migrate_range() Date: Tue, 27 Aug 2024 19:47:26 +0800 Message-ID: <20240827114728.3212578-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> References: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: AF18C14000E X-Stat-Signature: khqnwny3b4518dd3x4ddhnqo5b4aswxz X-HE-Tag: 1724759261-699703 X-HE-Meta: U2FsdGVkX19itmPdCxEHTCzsPag/VEzda6KMGKtTyZenhXYF4j/f2UAtMecH3V6ig+pug71T8tp5Uk1ZyyiP3FtqxQCb/fGCUZY4kwW8iVfm/b3j+03eFALd5rOo4gfjlfBLQNNwhW6GhE1bHAHLsfZ0JScp0GPutV7zilOthiwtwA1r/o5IMhsjXuYY3AE/lBS2K3g4794FWG8R0NxZi3/yysIAg6abfe60+S7WtEohdaHzok2uAaJVgKg+VG9ujlxhj5YxtrlITG0yg5qFN/bfwlq8dnbLWXg/heS6HkdYOg4lNys1fsxbN/8MJApG36LeSLWBvviNGj9tJ/a57F/cBqvTC4SQsN5SN3xOOfOdV+U8ODnQMORjaavk1PvGA3jDsIthl03fSCQKK9XfdQKJ7e9/Ip6L8ex5vfnjN5wh7Rzl1A/eAzJfCstX64fE7y7tgOp/nlHFs6YurohApTKy8OnqLuX2pmS4Mjyf5g9rv/6TOcZgtSlueFWAkeroHpIphS20fwP9sOHl4mOa0GHmzeRPQX3Qy0jqTB5XnhHc7nDyDmLkaWXF2ey6JYt/mjAxxQz10kTmbdTDfi5JebEWecGiiP6FbZm3uGS1eXyPLJA4m4Q5SGmLEFqlL/xDZ2FjAb7kg+L1IZTnN4G9EoA8mQN/NsJEpBbMseCcUXRTHcBhjbkQ17IR3Uvj7gSiS7lYb6THBLzwlxvBxhwoFOJSDcT9R6IuepBm4er83YbZbt9DWTO81n+gT3pkdocRdihgKr++8E67NRrTs++9BrMOiBeOyro7jTQUz1B2CbDWkZADT4b2TNpck+rSBOjdZeibknIgHBcNtBMcjsbeJtAF4NuzGeeSmq5h3/tT9Z5UxQXF5w7MeY91Y1Z5Yf6fGvJb5p3c+tZBISsJT0g5aOC1G9KlqAALiEi48IisFefpZpI8C0SEAZ2/EFVDt4N1jqPDZffzVxSLfnqQhio x+Cqt6NY waaY2vfCSnLjTLJvBwJ+jf2CqDFfbrlw26Bh9P7WI5OQ23KQiaGGV1Yo36NhYbsBbnu1RyU9Lz4q+eNGnEgupQVKzCo9PdTI+buYvfD1370/ufc3zywVWsUkKmgV9UdUu08YNHWKG/ELPrG2LiLXYGd16zWtydHyjqMpfF2kytof3K8Gwhfazyp2KCNpB/QtB/COdaQxCAy0SJNFn3FUU5091BRB1PU9JoCHU+8z2IS6iK/VqL3nHnN9P1rcS9yOrD2OmtAr/beR5q9ROv/vzG0TngJ8Vt4orquQx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The commit b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined") don't handle the hugetlb pages, the endless loop still occur if offline a hwpoison hugetlb page, luckly, after the commit e591ef7d96d6 ("mm, hwpoison,hugetlb,memory_hotplug: hotremove memory section with hwpoisoned hugepage"), the HPageMigratable of hugetlb page will be cleared, and the hwpoison hugetlb page will be skipped in scan_movable_pages(), so the endless loop issue is fixed. However if the HPageMigratable() check passed(without reference and lock), the hugetlb page may be hwpoisoned, it won't cause issue since the hwpoisoned page will be handled correctly in the next movable pages scan loop, and it will be isolated in do_migrate_range() but fails to migrate. In order to avoid the unnecessary isolation and unify all hwpoisoned page handling, let's unconditionally check hwpoison firstly, and if it is a hwpoisoned hugetlb page, try to unmap it as the catch all safety net like normal page does. Acked-by: David Hildenbrand Signed-off-by: Kefeng Wang Reviewed-by: Miaohe Lin --- mm/memory_hotplug.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 9ef776a25b9d..1335fb6ef7fa 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1793,26 +1793,26 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) * folio_nr_pages() may read garbage. This is fine as the outer * loop will revisit the split folio later. */ - if (folio_test_large(folio)) { + if (folio_test_large(folio)) pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1; - if (folio_test_hugetlb(folio)) { - isolate_hugetlb(folio, &source); - continue; - } - } /* * HWPoison pages have elevated reference counts so the migration would * fail on them. It also doesn't make any sense to migrate them in the * first place. Still try to unmap such a page in case it is still mapped - * (e.g. current hwpoison implementation doesn't unmap KSM pages but keep - * the unmap as the catch all safety net). + * (keep the unmap as the catch all safety net). */ - if (PageHWPoison(page)) { + if (folio_test_hwpoison(folio) || + (folio_test_large(folio) && folio_test_has_hwpoisoned(folio))) { if (WARN_ON(folio_test_lru(folio))) folio_isolate_lru(folio); if (folio_mapped(folio)) - try_to_unmap(folio, TTU_IGNORE_MLOCK); + unmap_poisoned_folio(folio, TTU_IGNORE_MLOCK); + continue; + } + + if (folio_test_hugetlb(folio)) { + isolate_hugetlb(folio, &source); continue; } From patchwork Tue Aug 27 11:47:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13779374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79E3DC54734 for ; Tue, 27 Aug 2024 11:47:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FACD6B0098; Tue, 27 Aug 2024 07:47:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 09A0E6B0099; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE8836B009A; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 943BD6B0093 for ; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4D1B4161814 for ; Tue, 27 Aug 2024 11:47:44 +0000 (UTC) X-FDA: 82497850848.04.600E830 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf08.hostedemail.com (Postfix) with ESMTP id B6177160006 for ; Tue, 27 Aug 2024 11:47:41 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724759218; a=rsa-sha256; cv=none; b=Yr0pccVvy0W7UvgkDLEBm1F+rCIqKAojY6kiX0a8UNikMU8H7EoCD2HcKJ2ZH75GvXXTyP UyGN5PylgOamoM8C0bxmIkpecK2TRAk5QXCDLvvcUOvIp7V5qMR5IJe8PeQLWrd0nMnXJz /fVgaMoPnvl2xu3DoEyAGEWQ8FEwdYY= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724759218; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1bhUTx7nnx6vHbE5W0f0Ci/xujChDyw8hckCCgIY/zg=; b=X3iv48dFDbNYJY64Avw/fqcX5W63x9OFTJIrJPh4dVnxfN7XMxTABvPHRxnXGTmrjBamBS rVFxLHBXX3O9mUkdgIjUwv/LoXhVpPD130+WhhYnL07jfWby1+naMkbhzU41EUBIqFdfDl EWGON5BNuG7T6Ar0bqsdg1qqUAaw3Do= Received: from mail.maildlp.com (unknown [172.19.163.17]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4WtQdj0Z3bz20mrT; Tue, 27 Aug 2024 19:42:49 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 604981A0188; Tue, 27 Aug 2024 19:47:37 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 27 Aug 2024 19:47:36 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , , Jonathan Cameron , Kefeng Wang Subject: [PATCH v3 4/5] mm: migrate: add isolate_folio_to_list() Date: Tue, 27 Aug 2024 19:47:27 +0800 Message-ID: <20240827114728.3212578-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> References: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf100008.china.huawei.com (7.185.36.138) X-Stat-Signature: 6m9q7zgy89odi7xzm739umcx6i34z4sj X-Rspamd-Queue-Id: B6177160006 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724759261-784442 X-HE-Meta: U2FsdGVkX1+wOOjTVxywjdDi/3pqScNmsPgNx6e/i6C6ATz8KAmHihYyDj7mBvVrERWji8x1bErPKfqN3hdJadXqy1mRG8JGtZKkDVb7GpEa69UwSntimup6w0y7AXmFA8iSFAoNhtExpVdHa/FlWF7O0uC0rMR7l3FVcbSoPvh53NWOFL3MrBLXCLGjz1tKZWImATW5LCLl9B/bSIx0BBSNxVwVNiZvzZ/N4nfTkE9KvRmFDEDZ1tn9kzK/LIkRZlfN7BYaYXY7e5RSREXsBIs7ofEoOCSI2RHBxTMgABcrUIrNHwECfYKZLHfYA6FrK8tS4AiLm7l4g7xj+xDYaZR6AnEaWgBgsWG+wVsjho/grUyxpLnj6A70UeJole1XVppYTanQk2llHwLdtGxv8dI0kyytWSYEsDE8eu+X5c6O4SxtOkG8nixhpnFmKAjcy+1eSmvj9x7gN1XaWW/MK1mqo3gpwzV7wTMCxN10v2rECQgn8WsXlanS1cpkv87PYQ4PnKlH0w4WEa+cjRgX0YSEP/IjWzZ5OETdwWtHBzazTrCINsSB9dmUV45WokTL3M0PjNBS2EJL+CNIAsBeEsclZJH/dFZH/xl7+wK1kgAMlhhjPxSsIITBufYwgH+rtS/qhNGuAVwoFnHOTT4lLUnT79ekVTPQU5vjmYSa+VZ8KADlRZ7njdfRP5dEiYSHlbVntfwHkBiFdJDBPYSpXTnDjF7ADCW9qpOXIAEu9M04mDicvQW9udndDKyADzaonyVLmOU/oXGgBnohSUbs6WF6Eh7VeqRRO5pueqsnOQzavxwdTeq+F6vPxwhRH6WlaMoVxc4luofVGlHG/wHOwWEATDI9w94WpJznU86lnjgmos/ibidFPSyBVB2z8v5JTygfQKZ/jMyvv3s5c+7i2TudeOGg4USONNkWeB8i0Vo+gOeTXdPfdVzkjHU/VJfCX1WOW3UA2hv65eK5vkv 4PQQr3Cd 9LiJeAokYOSjtXyj1iJDeVZgclEUXTGaj44+5aF4/1k64+6PY70VrQnq1CEIb9cjxqK5ZNLnJ70cJB7gdScamc3PnT9jFyYOVOFi4uF1Mq6OZuz2TZTKPIJ0JNmHLqBsvtQV9/wZWIOAKcclyN7xJ9jKceZNh4F8KR6HultKAwvE4XcJkzmqmcwI0F30DOOEhUNi7HVNtisbHKV1+BmN1lornmRoaus5Z0vzxTZL1esn8PIMTLQgbEJrnKyeFIRZkcFsqrOtgSNjBilnGcpLEBwHBtbnZS1+Wj8X8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add isolate_folio_to_list() helper to try to isolate HugeTLB, no-LRU movable and LRU folios to a list, which will be reused by do_migrate_range() from memory hotplug soon, also drop the mf_isolate_folio() since we could directly use new helper in the soft_offline_in_use_page(). Acked-by: David Hildenbrand Acked-by: Miaohe Lin Tested-by: Miaohe Lin Signed-off-by: Kefeng Wang --- include/linux/migrate.h | 3 +++ mm/memory-failure.c | 48 +++++++++++------------------------------ mm/migrate.c | 26 ++++++++++++++++++++++ 3 files changed, 42 insertions(+), 35 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 644be30b69c8..002e49b2ebd9 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -70,6 +70,7 @@ int migrate_pages(struct list_head *l, new_folio_t new, free_folio_t free, unsigned int *ret_succeeded); struct folio *alloc_migration_target(struct folio *src, unsigned long private); bool isolate_movable_page(struct page *page, isolate_mode_t mode); +bool isolate_folio_to_list(struct folio *folio, struct list_head *list); int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); @@ -91,6 +92,8 @@ static inline struct folio *alloc_migration_target(struct folio *src, { return NULL; } static inline bool isolate_movable_page(struct page *page, isolate_mode_t mode) { return false; } +static inline bool isolate_folio_to_list(struct folio *folio, struct list_head *list) + { return false; } static inline int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 67b6b259a75d..7da3697b33f1 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2659,40 +2659,6 @@ EXPORT_SYMBOL(unpoison_memory); #undef pr_fmt #define pr_fmt(fmt) "Soft offline: " fmt -static bool mf_isolate_folio(struct folio *folio, struct list_head *pagelist) -{ - bool isolated = false; - - if (folio_test_hugetlb(folio)) { - isolated = isolate_hugetlb(folio, pagelist); - } else { - bool lru = !__folio_test_movable(folio); - - if (lru) - isolated = folio_isolate_lru(folio); - else - isolated = isolate_movable_page(&folio->page, - ISOLATE_UNEVICTABLE); - - if (isolated) { - list_add(&folio->lru, pagelist); - if (lru) - node_stat_add_folio(folio, NR_ISOLATED_ANON + - folio_is_file_lru(folio)); - } - } - - /* - * If we succeed to isolate the folio, we grabbed another refcount on - * the folio, so we can safely drop the one we got from get_any_page(). - * If we failed to isolate the folio, it means that we cannot go further - * and we will return an error, so drop the reference we got from - * get_any_page() as well. - */ - folio_put(folio); - return isolated; -} - /* * soft_offline_in_use_page handles hugetlb-pages and non-hugetlb pages. * If the page is a non-dirty unmapped page-cache page, it simply invalidates. @@ -2705,6 +2671,7 @@ static int soft_offline_in_use_page(struct page *page) struct folio *folio = page_folio(page); char const *msg_page[] = {"page", "hugepage"}; bool huge = folio_test_hugetlb(folio); + bool isolated; LIST_HEAD(pagelist); struct migration_target_control mtc = { .nid = NUMA_NO_NODE, @@ -2744,7 +2711,18 @@ static int soft_offline_in_use_page(struct page *page) return 0; } - if (mf_isolate_folio(folio, &pagelist)) { + isolated = isolate_folio_to_list(folio, &pagelist); + + /* + * If we succeed to isolate the folio, we grabbed another refcount on + * the folio, so we can safely drop the one we got from get_any_page(). + * If we failed to isolate the folio, it means that we cannot go further + * and we will return an error, so drop the reference we got from + * get_any_page() as well. + */ + folio_put(folio); + + if (isolated) { ret = migrate_pages(&pagelist, alloc_migration_target, NULL, (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_FAILURE, NULL); if (!ret) { diff --git a/mm/migrate.c b/mm/migrate.c index f8777d6fab57..6f9c62c746be 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -178,6 +178,32 @@ void putback_movable_pages(struct list_head *l) } } +/* Must be called with an elevated refcount on the non-hugetlb folio */ +bool isolate_folio_to_list(struct folio *folio, struct list_head *list) +{ + bool isolated, lru; + + if (folio_test_hugetlb(folio)) + return isolate_hugetlb(folio, list); + + lru = !__folio_test_movable(folio); + if (lru) + isolated = folio_isolate_lru(folio); + else + isolated = isolate_movable_page(&folio->page, + ISOLATE_UNEVICTABLE); + + if (!isolated) + return false; + + list_add(&folio->lru, list); + if (lru) + node_stat_add_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); + + return true; +} + /* * Restore a potential migration pte to a working pte entry */ From patchwork Tue Aug 27 11:47:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13779372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A92BC54735 for ; Tue, 27 Aug 2024 11:47:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 381BE6B0092; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30A726B0093; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15D5F6B0095; Tue, 27 Aug 2024 07:47:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E4E9A6B0092 for ; Tue, 27 Aug 2024 07:47:43 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9C900141849 for ; Tue, 27 Aug 2024 11:47:43 +0000 (UTC) X-FDA: 82497850806.09.DCDF69D Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf29.hostedemail.com (Postfix) with ESMTP id 1A168120013 for ; Tue, 27 Aug 2024 11:47:40 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724759218; a=rsa-sha256; cv=none; b=DUMEegSNAoOyhf+SN1Yvct9uqT9ckpaTMIWaCITm7idJ29HKHHthqfQE+5hCE2wEv3UCE0 PD5WfEYiKzYHaOWeXb0T/uYypGxbAK6XxIFKLJyeXevgnD4aJco8H09qcDDIm2FQ1UX8pj pBut8RopP2YWwPiqyX4V+uRM/nSyx0g= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724759218; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jwe2IIHCoCHMWiaxC0pKLadiDVs7qKzatsjBVzmhakM=; b=DLGMzYN34O8tPAWW6Np/1kU/1HyMaRpbRrzSe5n9SkSFMbWB1DbZMbo/8/mYsrJAxmc9Ty f6m0VRPbJD6TAwIY6+aoxPXEh40u5D4AVEVcDo717BQ956e57NAuLFXsFuNliX8dk0kPhM x+ppjIBJ+mezatcMTeuTYo4qT6Z82dE= Received: from mail.maildlp.com (unknown [172.19.163.48]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4WtQkL4v6VzyQZH; Tue, 27 Aug 2024 19:46:50 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id DBEC118007C; Tue, 27 Aug 2024 19:47:37 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 27 Aug 2024 19:47:37 +0800 From: Kefeng Wang To: Andrew Morton CC: David Hildenbrand , Oscar Salvador , Miaohe Lin , Naoya Horiguchi , , , Jonathan Cameron , Kefeng Wang Subject: [PATCH v3 5/5] mm: memory_hotplug: unify Huge/LRU/non-LRU movable folio isolation Date: Tue, 27 Aug 2024 19:47:28 +0800 Message-ID: <20240827114728.3212578-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> References: <20240827114728.3212578-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemf100008.china.huawei.com (7.185.36.138) X-Stat-Signature: 5zahp6485qar4umzef5p4r6q9x1ab7jc X-Rspamd-Queue-Id: 1A168120013 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724759260-916792 X-HE-Meta: U2FsdGVkX19FEnHVuDE48OqKxJ9IxTjjrN4iM8rXCkVxbfY9jQyI+s5dmdRL6tb5peyo13cwrtxc2UC9daZ5rBVEV54a+98yfK9INYOsaGn581eqpQC6omi5/QOrfkgY7KTSXGaKbcsnPZr4Wj9CzYjiMYe5OdaCvdHOMQ70MeBqIx1bmakMxdE2JkeTo/URc2lH3Fhi4EDfD/8PwbV+WBJz/7Kk2co0aer/iTA7N4AUtdlaQZVSIMM6ZQaIWBYRYRZbzrzZ+VNFx6ioC7x265bhkPFYLyt8CRulNcb0SEkUYnBzrI33SuJG8g7fvifNDncVh8nNuqfE62H9/N3Ur0tQjnfbGntPjyySo8gxtwHe7IfF501jaLjA6L3sHkh8e95fNyKWVoiAQ2gIqb94P19FUvjOR33CtWNJ71il1oAqCTedpl7bQGyev28E9B7+4qB6DRiK7woOFyXsz2kGOA4GI9vLmT/BoPFm2nDQVvlC+EUUoK3/KX3wNZ5hz21B4u+hLltw5NSwMnc7cEVa+6cR56Sh9tOe4JIXc8gmpDL9PhlqpNUMuRBd69YfJ6VapX7C/0kKWpA8AXmHgO79oqscMoB2cAAJ7gmtnYpM8qi+QUpLJnlBweCkGwGOJBAYUklQ2o/czndMqupcLs8l1dqzv+O6dgQavSrKnKly3nWsrxHevhGoAKr2Taw1UwN7oQhR11AisNegOmhXdrgTY6OewaT6G+ysuShHLicDrHwj+zNVtF9PGftKkWXlk9cTwGyvh5cWMSUD0iaL/PkBv8Ln26LA+hOGi/kVe5fjt/6TfiZHpCmmjtw/KO7v11ylnBl3a+4kNIRy1GQxkgVgBcmpfFgPTbBMmq9XeSXRDxRaZoFI96kki/qEu3OEHt33BIgI2R0O6lWsl2FMNzJjqzuz3qO6geHbp0NT5t0CKktPpdMNy3Ikh6WYJXsz85r0RQk557mj/VLU0+BhIVH dtXoDw+K nEiYCnrfSROa8XKf4HpaZ+B2Gu6wE8SFFYMyP+WUAMWIeAUsoE+t1DHFVeqZ7btZe7XGBXg/iAr7tHpAF13UhnquiEA147Kv+vFwk9URU9XGI63uRG5gdQBCirc8yOWeJCf3/p8lZWWdxAPtnFj34p1YarM+PXLFtftqTUtHT6HyZy5I8H5XdYHwINI1z5Jewaea32tcQyq8YksDnHjbm+CCdFDay3m2kLVmux9D0LyFApQjYfSL1rdYqOtfg+7tF6bAPPS5TDlzEWAE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use the isolate_folio_to_list() to unify hugetlb/LRU/non-LRU folio isolation, which cleanup code a bit and save a few calls to compound_head(). Signed-off-by: Kefeng Wang Reviewed-by: Miaohe Lin --- mm/memory_hotplug.c | 45 +++++++++++++++++---------------------------- 1 file changed, 17 insertions(+), 28 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 1335fb6ef7fa..5f09866d17cf 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1772,15 +1772,15 @@ static int scan_movable_pages(unsigned long start, unsigned long end, static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) { + struct folio *folio; unsigned long pfn; - struct page *page; LIST_HEAD(source); static DEFINE_RATELIMIT_STATE(migrate_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); for (pfn = start_pfn; pfn < end_pfn; pfn++) { - struct folio *folio; - bool isolated; + struct page *page; + bool hugetlb; if (!pfn_valid(pfn)) continue; @@ -1811,34 +1811,22 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) continue; } - if (folio_test_hugetlb(folio)) { - isolate_hugetlb(folio, &source); - continue; + hugetlb = folio_test_hugetlb(folio); + if (!hugetlb) { + folio = folio_get_nontail_page(page); + if (!folio) + continue; } - if (!get_page_unless_zero(page)) - continue; - /* - * We can skip free pages. And we can deal with pages on - * LRU and non-lru movable pages. - */ - if (PageLRU(page)) - isolated = isolate_lru_page(page); - else - isolated = isolate_movable_page(page, ISOLATE_UNEVICTABLE); - if (isolated) { - list_add_tail(&page->lru, &source); - if (!__PageMovable(page)) - inc_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_lru(page)); - - } else { + if (!isolate_folio_to_list(folio, &source)) { if (__ratelimit(&migrate_rs)) { pr_warn("failed to isolate pfn %lx\n", pfn); dump_page(page, "isolation failed"); } } - put_page(page); + + if (!hugetlb) + folio_put(folio); } if (!list_empty(&source)) { nodemask_t nmask = node_states[N_MEMORY]; @@ -1853,7 +1841,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) * We have checked that migration range is on a single zone so * we can use the nid of the first page to all the others. */ - mtc.nid = page_to_nid(list_first_entry(&source, struct page, lru)); + mtc.nid = folio_nid(list_first_entry(&source, struct folio, lru)); /* * try to allocate from a different node but reuse this node @@ -1866,11 +1854,12 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) ret = migrate_pages(&source, alloc_migration_target, NULL, (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_HOTPLUG, NULL); if (ret) { - list_for_each_entry(page, &source, lru) { + list_for_each_entry(folio, &source, lru) { if (__ratelimit(&migrate_rs)) { pr_warn("migrating pfn %lx failed ret:%d\n", - page_to_pfn(page), ret); - dump_page(page, "migration failure"); + folio_pfn(folio), ret); + dump_page(&folio->page, + "migration failure"); } } putback_movable_pages(&source);