From patchwork Wed Sep 21 09:13:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12983453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDAD7C6FA8E for ; Wed, 21 Sep 2022 09:15:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 245886B0073; Wed, 21 Sep 2022 05:15:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CFAD80008; Wed, 21 Sep 2022 05:15:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 047B26B0075; Wed, 21 Sep 2022 05:15:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E7E346B0073 for ; Wed, 21 Sep 2022 05:15:57 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BD29E120F25 for ; Wed, 21 Sep 2022 09:15:57 +0000 (UTC) X-FDA: 79935535554.21.138600D Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf13.hostedemail.com (Postfix) with ESMTP id 40E9E2003D for ; Wed, 21 Sep 2022 09:15:57 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1663751755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NAK+OfD9y76Qeqp8V3ZaY9yJ/ZmnLYVYzxQlX59hfUo=; b=EqXp6eVohBUwOJlfQsMX5CatWXvACPoYJVgieu9JfuooBH3wIc3sxKU+WaOakYWcn31BXW WV1maFBKpk56Y32dSraaK8ei7ZeeM8DqxYPPz1mX2pNW4EAoBDmHl3VUIIOlOuEVoKnxjP dJWZzMHdVs2RjqoKtspztq7WIiwdTGM= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v3 1/4] mm,hwpoison,hugetlb,memory_hotplug: hotremove memory section with hwpoisoned hugepage Date: Wed, 21 Sep 2022 18:13:56 +0900 Message-Id: <20220921091359.25889-2-naoya.horiguchi@linux.dev> In-Reply-To: <20220921091359.25889-1-naoya.horiguchi@linux.dev> References: <20220921091359.25889-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663751757; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NAK+OfD9y76Qeqp8V3ZaY9yJ/ZmnLYVYzxQlX59hfUo=; b=FnrSXKc4SJG7EC+zMN2qz/x4wpu2bohEfvvMMx2fAo+ncvPI6F3vJak4LKlMrEZDuypslw B2H1CHXPPl6iJ/aAs/Ql8BlZ9DJmzvsTztv5S6sA9s/1G4saktQWumqBaiH8XP9FJxppHa IpoJvV1LPrVFoH2FGFctqT1PXcewqYg= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=EqXp6eVo; spf=pass (imf13.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663751757; a=rsa-sha256; cv=none; b=UJeSE0YVKFapeny+CKrBywsg+ZSKQEUu3SJRNrqKDAFbgXTYgLheMS1V7t3z6dmqmVVqeo 5YfEt5nVE27VQnQOQ2MLLjxdHutxqIDazTSn2740JbaTYoaLzK+UXl8zFKid8smP+I1ML+ geKZNDaHGHKJ+u3mRwpIDqXgUR54T0o= X-Stat-Signature: 8y88i4bbmjsjezcrnj7stk1nwc3c35ks X-Rspamd-Queue-Id: 40E9E2003D X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=EqXp6eVo; spf=pass (imf13.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspamd-Server: rspam09 X-HE-Tag: 1663751757-810186 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi HWPoisoned page is not supposed to be accessed once marked, but currently such accesses can happen during memory hotremove because do_migrate_range() can be called before dissolve_free_huge_pages() is called. Clear HPageMigratable for hwpoisoned hugepages to prevent them from being migrated. This should be done in hugetlb_lock to avoid race against isolate_hugetlb(). get_hwpoison_huge_page() needs to have a flag to show it's called from unpoison to take refcount of hwpoisoned hugepages, so add it. Reported-by: Miaohe Lin Signed-off-by: Naoya Horiguchi --- ChangeLog v2 -> v3 - move to the approach of clearing HPageMigratable instead of shifting dissolve_free_huge_pages. --- include/linux/hugetlb.h | 4 ++-- mm/hugetlb.c | 4 ++-- mm/memory-failure.c | 12 ++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index cfe15b32e2d4..18229402c6d7 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -181,7 +181,7 @@ bool hugetlb_reserve_pages(struct inode *inode, long from, long to, long hugetlb_unreserve_pages(struct inode *inode, long start, long end, long freed); int isolate_hugetlb(struct page *page, struct list_head *list); -int get_hwpoison_huge_page(struct page *page, bool *hugetlb); +int get_hwpoison_huge_page(struct page *page, bool *hugetlb, bool unpoison); int get_huge_page_for_hwpoison(unsigned long pfn, int flags); void putback_active_hugepage(struct page *page); void move_hugetlb_state(struct page *oldpage, struct page *newpage, int reason); @@ -425,7 +425,7 @@ static inline int isolate_hugetlb(struct page *page, struct list_head *list) return -EBUSY; } -static inline int get_hwpoison_huge_page(struct page *page, bool *hugetlb) +static inline int get_hwpoison_huge_page(struct page *page, bool *hugetlb, bool unpoison) { return 0; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 8bcaf66defc5..d3b83c570b56 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -7257,7 +7257,7 @@ int isolate_hugetlb(struct page *page, struct list_head *list) return ret; } -int get_hwpoison_huge_page(struct page *page, bool *hugetlb) +int get_hwpoison_huge_page(struct page *page, bool *hugetlb, bool unpoison) { int ret = 0; @@ -7267,7 +7267,7 @@ int get_hwpoison_huge_page(struct page *page, bool *hugetlb) *hugetlb = true; if (HPageFreed(page)) ret = 0; - else if (HPageMigratable(page)) + else if (HPageMigratable(page) || unpoison) ret = get_page_unless_zero(page); else ret = -EBUSY; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 145bb561ddb3..5942e1c0407e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1244,7 +1244,7 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags) int ret = 0; bool hugetlb = false; - ret = get_hwpoison_huge_page(head, &hugetlb); + ret = get_hwpoison_huge_page(head, &hugetlb, false); if (hugetlb) return ret; @@ -1334,7 +1334,7 @@ static int __get_unpoison_page(struct page *page) int ret = 0; bool hugetlb = false; - ret = get_hwpoison_huge_page(head, &hugetlb); + ret = get_hwpoison_huge_page(head, &hugetlb, true); if (hugetlb) return ret; @@ -1815,6 +1815,13 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) goto out; } + /* + * Clearing HPageMigratable for hwpoisoned hugepages to prevent them + * from being migrated by memory hotremove. + */ + if (count_increased) + ClearHPageMigratable(head); + return ret; out: if (count_increased) @@ -1862,6 +1869,7 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb if (hwpoison_filter(p)) { hugetlb_clear_page_hwpoison(head); + SetHPageMigratable(head); unlock_page(head); if (res == 1) put_page(head); From patchwork Wed Sep 21 09:13:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12983454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B128C6FA90 for ; Wed, 21 Sep 2022 09:16:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3087780009; Wed, 21 Sep 2022 05:16:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2911780008; Wed, 21 Sep 2022 05:16:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10B0080009; Wed, 21 Sep 2022 05:16:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F03CB80008 for ; Wed, 21 Sep 2022 05:16:01 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B0E53120F0D for ; Wed, 21 Sep 2022 09:16:01 +0000 (UTC) X-FDA: 79935535722.17.EDE75D6 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf03.hostedemail.com (Postfix) with ESMTP id 2D33B2009E for ; Wed, 21 Sep 2022 09:16:01 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1663751760; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0Magx0opi1c7C+scjbhIZzuX/5vpGYEOmpE+SoMhVW0=; b=iNAziErxuVm5/to4rBxj+jx3cwJjMgYswcFSNOt6zueXQo8uqqVaIl93Att66gG3X3q19P QrgB7vOhcniJ8CqKxern26nJJrZjvPe/jwlxj1LFACvLeNMG4I6vND4oRG3CVjf2eqAIKf v9GgSgAS+ZhI47iqmwIdEpbsFBlc/VE= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v3 2/4] mm/hwpoison: move definitions of num_poisoned_pages_* to memory-failure.c Date: Wed, 21 Sep 2022 18:13:57 +0900 Message-Id: <20220921091359.25889-3-naoya.horiguchi@linux.dev> In-Reply-To: <20220921091359.25889-1-naoya.horiguchi@linux.dev> References: <20220921091359.25889-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=iNAziErx; spf=pass (imf03.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663751761; a=rsa-sha256; cv=none; b=yJIPa93dDigEYIMfxy9iacLITFxpEwQIUvBDj6pAbJ6r9OZmGcRpZmt9P6Vp50vsuLVk+F KD87+7W79+1rAsnPelbrj+97nxqpdY3+vrnsc1rsetqNJvDAzgzYWH2lv/B1tyjjYmStyV 0Xi9AjN4A0zrHQR9Gtgfer6sCGEmXHs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663751761; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0Magx0opi1c7C+scjbhIZzuX/5vpGYEOmpE+SoMhVW0=; b=lNaGSF8qkZT+ql/OkhYi9ISkJz4r0guZs7KyHlP5SzbCVgQTc66EHovDzlfm7x1DmY/TqR 4FqOo5hsmMYHS4VRL12uDtxKkm6OheA9HwGR5duYr3LSV6k5YQURgDECZ4zS919IT5y7su RVBHrsjYWExqn4iLjuA+4A6JvNHk8xk= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 2D33B2009E X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=iNAziErx; spf=pass (imf03.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Stat-Signature: 4feh9sgiyc9afdx9gmega7io638hm4qh X-HE-Tag: 1663751761-798857 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi These interfaces will be used by drivers/base/core.c by later patch, so as a preparatory work move them to more common header file visible to the file. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- ChangeLog v2 -> v3: - added declaration of num_poisoned_pages_inc() in #ifdef CONFIG_MEMORY_FAILURE --- arch/parisc/kernel/pdt.c | 3 +-- include/linux/mm.h | 5 +++++ include/linux/swapops.h | 24 ++---------------------- mm/memory-failure.c | 10 ++++++++++ 4 files changed, 18 insertions(+), 24 deletions(-) diff --git a/arch/parisc/kernel/pdt.c b/arch/parisc/kernel/pdt.c index e391b175f5ec..fdc880e2575a 100644 --- a/arch/parisc/kernel/pdt.c +++ b/arch/parisc/kernel/pdt.c @@ -18,8 +18,7 @@ #include #include #include -#include -#include +#include #include #include diff --git a/include/linux/mm.h b/include/linux/mm.h index c2277f5aba9e..80a2d800f272 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3279,11 +3279,16 @@ extern atomic_long_t num_poisoned_pages __read_mostly; extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags); +extern void num_poisoned_pages_inc(void); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) { return 0; } + +static inline void num_poisoned_pages_inc(void) +{ +} #endif #ifndef arch_memory_failure diff --git a/include/linux/swapops.h b/include/linux/swapops.h index a91dd08e107b..3e58a812399a 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -581,8 +581,6 @@ static inline int is_pmd_migration_entry(pmd_t pmd) #ifdef CONFIG_MEMORY_FAILURE -extern atomic_long_t num_poisoned_pages __read_mostly; - /* * Support for hardware poisoned pages */ @@ -610,17 +608,7 @@ static inline struct page *hwpoison_entry_to_page(swp_entry_t entry) return p; } -static inline void num_poisoned_pages_inc(void) -{ - atomic_long_inc(&num_poisoned_pages); -} - -static inline void num_poisoned_pages_sub(long i) -{ - atomic_long_sub(i, &num_poisoned_pages); -} - -#else /* CONFIG_MEMORY_FAILURE */ +#else static inline swp_entry_t make_hwpoison_entry(struct page *page) { @@ -636,15 +624,7 @@ static inline struct page *hwpoison_entry_to_page(swp_entry_t entry) { return NULL; } - -static inline void num_poisoned_pages_inc(void) -{ -} - -static inline void num_poisoned_pages_sub(long i) -{ -} -#endif /* CONFIG_MEMORY_FAILURE */ +#endif static inline int non_swap_entry(swp_entry_t entry) { diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 5942e1c0407e..aa6ce685b863 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,6 +74,16 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; +static inline void num_poisoned_pages_inc(void) +{ + atomic_long_inc(&num_poisoned_pages); +} + +static inline void num_poisoned_pages_sub(long i) +{ + atomic_long_sub(i, &num_poisoned_pages); +} + /* * Return values: * 1: the page is dissolved (if needed) and taken off from buddy, From patchwork Wed Sep 21 09:13:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12983455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA651C32771 for ; Wed, 21 Sep 2022 09:16:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4682D8000A; Wed, 21 Sep 2022 05:16:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F12980008; Wed, 21 Sep 2022 05:16:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26AD08000A; Wed, 21 Sep 2022 05:16:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1157B80008 for ; Wed, 21 Sep 2022 05:16:06 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D6FE841101 for ; Wed, 21 Sep 2022 09:16:05 +0000 (UTC) X-FDA: 79935535890.15.8053299 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf28.hostedemail.com (Postfix) with ESMTP id 48844C0013 for ; Wed, 21 Sep 2022 09:16:05 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1663751764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vn/izsX8/VTmLZ+oZfQee4UuxB5JUFLZrbR8EGTGQ0o=; b=ZkeJNyl5tL2nrdhs1hdX4MrIAsdsazG5YNqPGdB/gZ+CinSpGebqMQM19x1sH3SNJ4veMF PaogreoFD+jmzRmDWWwpzojtMCsT4N2Pd0o2O+aS04ruJN5LYvUXiZpdj8EGOqwsgb+/NA +Aho+2wid2OmMPSwh2a9LClBWAGg7qU= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v3 3/4] mm/hwpoison: pass pfn to num_poisoned_pages_*() Date: Wed, 21 Sep 2022 18:13:58 +0900 Message-Id: <20220921091359.25889-4-naoya.horiguchi@linux.dev> In-Reply-To: <20220921091359.25889-1-naoya.horiguchi@linux.dev> References: <20220921091359.25889-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663751765; a=rsa-sha256; cv=none; b=mpkcOaOdQCDg4GW19TOfNK8i3F14AK+I/at2nKjGFjXBtTGoGojMBLvy9utc/dl6TWD6YH /1V3pJdoioopZsqXYW6PWn1jQcOJqcXNUWFFqCg0WaqhjKwcpCWAoQwx2QwVV2mqUCay8w PKsActLIto7c+59HzoiRPfb2EQrRiFY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ZkeJNyl5; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663751765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vn/izsX8/VTmLZ+oZfQee4UuxB5JUFLZrbR8EGTGQ0o=; b=By2CB9qbR5QjpoNT3gWTlC+aY4VbhktZxkCHpUsoPvO3KEju3UGVHtdUXkGz2camk3ID7s wZCoA3gtyc2+qDKGBBhoY6FuA+mgkl8WbbUPFsOhPFFI4DwwjwVTCLy9L/eoKjKMV0QUfE nQHvSETTUi5lFd5/d046yoYmzdV95Xw= X-Rspamd-Queue-Id: 48844C0013 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ZkeJNyl5; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-Rspamd-Server: rspam08 X-Rspam-User: X-Stat-Signature: ok5ajsppb59osc1f6958peotd8sou3y7 X-HE-Tag: 1663751765-28380 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi No functional change. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- ChangeLog v2 -> v3: - added declaration of num_poisoned_pages_inc() in #ifdef CONFIG_MEMORY_FAILURE --- arch/parisc/kernel/pdt.c | 2 +- include/linux/mm.h | 4 ++-- mm/memory-failure.c | 12 ++++++------ 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/parisc/kernel/pdt.c b/arch/parisc/kernel/pdt.c index fdc880e2575a..80943a00e245 100644 --- a/arch/parisc/kernel/pdt.c +++ b/arch/parisc/kernel/pdt.c @@ -231,7 +231,7 @@ void __init pdc_pdt_init(void) /* mark memory page bad */ memblock_reserve(pdt_entry[i] & PAGE_MASK, PAGE_SIZE); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(addr >> PAGE_SHIFT); } } diff --git a/include/linux/mm.h b/include/linux/mm.h index 80a2d800f272..2bb5d1596041 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3279,14 +3279,14 @@ extern atomic_long_t num_poisoned_pages __read_mostly; extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags); -extern void num_poisoned_pages_inc(void); +extern void num_poisoned_pages_inc(unsigned long pfn); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) { return 0; } -static inline void num_poisoned_pages_inc(void) +static inline void num_poisoned_pages_inc(unsigned long pfn) { } #endif diff --git a/mm/memory-failure.c b/mm/memory-failure.c index aa6ce685b863..a069d43bc87f 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,12 +74,12 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; -static inline void num_poisoned_pages_inc(void) +static inline void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); } -static inline void num_poisoned_pages_sub(long i) +static inline void num_poisoned_pages_sub(unsigned long pfn, long i) { atomic_long_sub(i, &num_poisoned_pages); } @@ -125,7 +125,7 @@ static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, boo if (release) put_page(page); page_ref_inc(page); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(page_to_pfn(page)); return true; } @@ -1194,7 +1194,7 @@ static void action_result(unsigned long pfn, enum mf_action_page_type type, { trace_memory_failure_event(pfn, type, result); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(pfn); pr_err("%#lx: recovery action for %s: %s\n", pfn, action_page_types[type], action_name[result]); } @@ -1741,7 +1741,7 @@ static int hugetlb_set_page_hwpoison(struct page *hpage, struct page *page) llist_add(&raw_hwp->node, head); /* the first error event will be counted in action_result(). */ if (ret) - num_poisoned_pages_inc(); + num_poisoned_pages_inc(page_to_pfn(page)); } else { /* * Failed to save raw error info. We no longer trace all @@ -2414,7 +2414,7 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { - num_poisoned_pages_sub(count); + num_poisoned_pages_sub(pfn, count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); } From patchwork Wed Sep 21 09:13:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12983456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFCB7ECAAD8 for ; Wed, 21 Sep 2022 09:16:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79CFD80008; Wed, 21 Sep 2022 05:16:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 725B780007; Wed, 21 Sep 2022 05:16:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59F8380008; Wed, 21 Sep 2022 05:16:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 49EEE80007 for ; Wed, 21 Sep 2022 05:16:10 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 1A660A0DFE for ; Wed, 21 Sep 2022 09:16:10 +0000 (UTC) X-FDA: 79935536100.22.6120B0B Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by imf02.hostedemail.com (Postfix) with ESMTP id 7097580011 for ; Wed, 21 Sep 2022 09:16:09 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1663751768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K3zq9Vv1YEJefxa5eePts00URoMHA6ZGQVN+sMjAFVQ=; b=og2aD95owl2UUJSYB7yl/LK1wbBgowpMLV3NHO5IIDxCxdxa5W0Vkz0XEyYAv/P0nrIkQo QK5+LOPnl5Oa9RceYDFbWZ37OdngjR6EsESWaRcwr+9uYJleD695+ZRJNWqlEqvLlkWUPl HLQoWb324EnTiQP5gTEYaMGvWH6yTtY= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v3 4/4] mm/hwpoison: introduce per-memory_block hwpoison counter Date: Wed, 21 Sep 2022 18:13:59 +0900 Message-Id: <20220921091359.25889-5-naoya.horiguchi@linux.dev> In-Reply-To: <20220921091359.25889-1-naoya.horiguchi@linux.dev> References: <20220921091359.25889-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663751769; a=rsa-sha256; cv=none; b=A02TG9+RFuGdnc+M4HYBku7pBBSRaCE/JN5H/jmAVOqLFyMHqjZHNRtRYg9Q6TEUMM7a/k iLqN2YgdUzoJMFp1PUqSd9hdIXYjjobTvs9CpBNW7g7p7/VpiQT2lfgrXfMSC5pE2U9gti MbfkABbn7YmgQaauYFHPRSlGcyM9wHQ= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=og2aD95o; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf02.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663751769; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=K3zq9Vv1YEJefxa5eePts00URoMHA6ZGQVN+sMjAFVQ=; b=QrxCAzPsdIm2IoQdIIP4SEcYqim7bQw81DeNag0EuZRJpqH8WO4ON+R6FFdY7ME9HWKSNA h8uAzRty+bSqgLDumyb/82Kd0G6Cp99syDC0r8dThsdTQK5jQBP9rKFpFxd4e+5jexRZ6f 1gBqEQxfKAGIvkbGBozXpTOQULZNvyY= X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7097580011 X-Stat-Signature: xsqb8gdrmgzttmypburwmczm7r961qjq Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=og2aD95o; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf02.hostedemail.com: domain of naoya.horiguchi@linux.dev designates 188.165.223.204 as permitted sender) smtp.mailfrom=naoya.horiguchi@linux.dev X-Rspam-User: X-HE-Tag: 1663751769-659360 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Currently PageHWPoison flag does not behave well when experiencing memory hotremove/hotplug. Any data field in struct page is unreliable when the associated memory is offlined, and the current mechanism can't tell whether a memory section is onlined because a new memory devices is installed or because previous failed offline operations are undone. Especially if there's a hwpoisoned memory, it's unclear what the best option is. So introduce a new mechanism to make struct memory_block remember that a memory block has hwpoisoned memory inside it. And make any online event fail if the onlined memory block contains hwpoison. struct memory_block is freed and reallocated over ACPI-based hotremove/hotplug, but not over sysfs-based hotremove/hotplug. So it's desirable to implement hwpoison counter on this struct. Note that clear_hwpoisoned_pages() is relocated to be called earlier than now, just before unregistering struct memory_block. Otherwise, the per-memory_block hwpoison counter is freed and we fail to adjust global hwpoison counter properly. Signed-off-by: Naoya Horiguchi --- drivers/base/memory.c | 36 ++++++++++++++++++++++++++++++++++++ include/linux/memory.h | 3 +++ include/linux/mm.h | 8 ++++++++ mm/internal.h | 8 -------- mm/memory-failure.c | 31 ++++++++++--------------------- mm/sparse.c | 2 -- 6 files changed, 57 insertions(+), 31 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 9aa0da991cfb..c9bde4c4ffdf 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -183,6 +183,9 @@ static int memory_block_online(struct memory_block *mem) struct zone *zone; int ret; + if (atomic_long_read(&mem->nr_hwpoison)) + return -EHWPOISON; + zone = zone_for_pfn_range(mem->online_type, mem->nid, mem->group, start_pfn, nr_pages); @@ -864,6 +867,7 @@ void remove_memory_block_devices(unsigned long start, unsigned long size) mem = find_memory_block_by_id(block_id); if (WARN_ON_ONCE(!mem)) continue; + clear_hwpoisoned_pages(atomic_long_read(&mem->nr_hwpoison)); unregister_memory_block_under_nodes(mem); remove_memory_block(mem); } @@ -1164,3 +1168,35 @@ int walk_dynamic_memory_groups(int nid, walk_memory_groups_func_t func, } return ret; } + +#ifdef CONFIG_MEMORY_FAILURE + +void memblk_nr_poison_inc(unsigned long pfn) +{ + const unsigned long block_id = pfn_to_block_id(pfn); + struct memory_block *mem = find_memory_block_by_id(block_id); + + if (mem) + atomic_long_inc(&mem->nr_hwpoison); +} + +void memblk_nr_poison_sub(unsigned long pfn, long i) +{ + const unsigned long block_id = pfn_to_block_id(pfn); + struct memory_block *mem = find_memory_block_by_id(block_id); + + if (mem) + atomic_long_sub(i, &mem->nr_hwpoison); +} + +unsigned long memblk_nr_poison(unsigned long pfn) +{ + const unsigned long block_id = pfn_to_block_id(pfn); + struct memory_block *mem = find_memory_block_by_id(block_id); + + if (mem) + return atomic_long_read(&mem->nr_hwpoison); + return 0; +} + +#endif diff --git a/include/linux/memory.h b/include/linux/memory.h index aa619464a1df..74e6b3ad947f 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -85,6 +85,9 @@ struct memory_block { unsigned long nr_vmemmap_pages; struct memory_group *group; /* group (if any) for this block */ struct list_head group_next; /* next block inside memory group */ +#ifdef CONFIG_MEMORY_FAILURE + atomic_long_t nr_hwpoison; +#endif }; int arch_get_memory_phys_device(unsigned long start_pfn); diff --git a/include/linux/mm.h b/include/linux/mm.h index 2bb5d1596041..2fe42bb9a517 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3280,6 +3280,10 @@ extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags); extern void num_poisoned_pages_inc(unsigned long pfn); +extern void memblk_nr_poison_inc(unsigned long pfn); +extern void memblk_nr_poison_sub(unsigned long pfn, long i); +extern unsigned long memblk_nr_poison(unsigned long pfn); +extern void clear_hwpoisoned_pages(long nr_poison); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) { @@ -3289,6 +3293,10 @@ static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) static inline void num_poisoned_pages_inc(unsigned long pfn) { } + +static inline void clear_hwpoisoned_pages(long nr_poison) +{ +} #endif #ifndef arch_memory_failure diff --git a/mm/internal.h b/mm/internal.h index b3002e03c28f..42ba8b96cab5 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -708,14 +708,6 @@ extern u64 hwpoison_filter_flags_value; extern u64 hwpoison_filter_memcg; extern u32 hwpoison_filter_enable; -#ifdef CONFIG_MEMORY_FAILURE -void clear_hwpoisoned_pages(struct page *memmap, int nr_pages); -#else -static inline void clear_hwpoisoned_pages(struct page *memmap, int nr_pages) -{ -} -#endif - extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index a069d43bc87f..03479895086d 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,14 +74,17 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; -static inline void num_poisoned_pages_inc(unsigned long pfn) +void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); + memblk_nr_poison_inc(pfn); } static inline void num_poisoned_pages_sub(unsigned long pfn, long i) { atomic_long_sub(i, &num_poisoned_pages); + if (pfn != -1UL) + memblk_nr_poison_sub(pfn, i); } /* @@ -2414,6 +2417,10 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { + /* + * TODO: per-memory_block counter might break when the page + * size to be unpoisoned is larger than a memory_block. + */ num_poisoned_pages_sub(pfn, count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); @@ -2618,25 +2625,7 @@ int soft_offline_page(unsigned long pfn, int flags) return ret; } -void clear_hwpoisoned_pages(struct page *memmap, int nr_pages) +void clear_hwpoisoned_pages(long nr_poison) { - int i, total = 0; - - /* - * A further optimization is to have per section refcounted - * num_poisoned_pages. But that would need more space per memmap, so - * for now just do a quick global check to speed up this routine in the - * absence of bad pages. - */ - if (atomic_long_read(&num_poisoned_pages) == 0) - return; - - for (i = 0; i < nr_pages; i++) { - if (PageHWPoison(&memmap[i])) { - total++; - ClearPageHWPoison(&memmap[i]); - } - } - if (total) - num_poisoned_pages_sub(total); + num_poisoned_pages_sub(-1UL, nr_poison); } diff --git a/mm/sparse.c b/mm/sparse.c index e5a8a3a0edd7..2779b419ef2a 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -926,8 +926,6 @@ void sparse_remove_section(struct mem_section *ms, unsigned long pfn, unsigned long nr_pages, unsigned long map_offset, struct vmem_altmap *altmap) { - clear_hwpoisoned_pages(pfn_to_page(pfn) + map_offset, - nr_pages - map_offset); section_deactivate(pfn, nr_pages, altmap); } #endif /* CONFIG_MEMORY_HOTPLUG */