From patchwork Tue Sep 10 10:30:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139187 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B629276 for ; Tue, 10 Sep 2019 10:31:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 96D6120872 for ; Tue, 10 Sep 2019 10:31:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96D6120872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3A6586B000A; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E1C686B0010; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF2FB6B0008; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 7A8276B0010 for ; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 003A6181AC9AE for ; Tue, 10 Sep 2019 10:30:51 +0000 (UTC) X-FDA: 75918642744.13.jewel02_41ba2b242c902 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30046:30054:30070:30079,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.14.6.2 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:35,LUA_SUMMARY:none X-HE-Tag: jewel02_41ba2b242c902 X-Filterd-Recvd-Size: 1915 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:51 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 9D176B12E; Tue, 10 Sep 2019 10:30:23 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 01/10] mm,hwpoison: cleanup unused PageHuge() check Date: Tue, 10 Sep 2019 12:30:07 +0200 Message-Id: <20190910103016.14290-2-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi memory_failure() forks to memory_failure_hugetlb() for hugetlb pages, so a PageHuge() check after the fork should not be necessary. Signed-off-by: Oscar Salvador Signed-off-by: Naoya Horiguchi Reviewed-by: Anshuman Khandual Reviewed-by: David Hildenbrand --- mm/memory-failure.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 7ef849da8278..e43b61462fd5 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1353,10 +1353,7 @@ int memory_failure(unsigned long pfn, int flags) * page_remove_rmap() in try_to_unmap_one(). So to determine page status * correctly, we save a copy of the page flags at this time. */ - if (PageHuge(p)) - page_flags = hpage->flags; - else - page_flags = p->flags; + page_flags = p->flags; /* * unpoison always clear PG_hwpoison inside page lock From patchwork Tue Sep 10 10:30:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139183 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ABB0416B1 for ; Tue, 10 Sep 2019 10:30:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7FBE02089F for ; Tue, 10 Sep 2019 10:30:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7FBE02089F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E98D76B0003; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BA58D6B000E; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BE1E6B0266; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 4E78D6B000A for ; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 0029B8135 for ; Tue, 10 Sep 2019 10:30:51 +0000 (UTC) X-FDA: 75918642744.11.truck10_41b9b2767545a X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30003:30054:30070:30091,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.14.6.2 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:31,LUA_SUMMARY:none X-HE-Tag: truck10_41b9b2767545a X-Filterd-Recvd-Size: 3221 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:51 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 75BC5ABED; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 02/10] mm,madvise: call soft_offline_page() without MF_COUNT_INCREASED Date: Tue, 10 Sep 2019 12:30:08 +0200 Message-Id: <20190910103016.14290-3-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Currently madvise_inject_error() pins the target via get_user_pages_fast. The call to get_user_pages_fast is only to get the respective page of a given address, but it is the job of the memory-poisoning handler to deal with races, so drop the refcount grabbed by get_user_pages_fast. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador --- mm/madvise.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 6e023414f5c1..fbe6d402232c 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -883,6 +883,16 @@ static int madvise_inject_error(int behavior, ret = get_user_pages_fast(start, 1, 0, &page); if (ret != 1) return ret; + /* + * The get_user_pages_fast() is just to get the pfn of the + * given address, and the refcount has nothing to do with + * what we try to test, so it should be released immediately. + * This is racy but it's intended because the real hardware + * errors could happen at any moment and memory error handlers + * must properly handle the race. + */ + put_page(page); + pfn = page_to_pfn(page); /* @@ -892,16 +902,11 @@ static int madvise_inject_error(int behavior, */ order = compound_order(compound_head(page)); - if (PageHWPoison(page)) { - put_page(page); - continue; - } - if (behavior == MADV_SOFT_OFFLINE) { pr_info("Soft offlining pfn %#lx at process virtual address %#lx\n", pfn, start); - ret = soft_offline_page(page, MF_COUNT_INCREASED); + ret = soft_offline_page(page, 0); if (ret) return ret; continue; @@ -909,14 +914,6 @@ static int madvise_inject_error(int behavior, pr_info("Injecting memory failure for pfn %#lx at process virtual address %#lx\n", pfn, start); - - /* - * Drop the page reference taken by get_user_pages_fast(). In - * the absence of MF_COUNT_INCREASED the memory_failure() - * routine is responsible for pinning the page to prevent it - * from being released back to the page allocator. - */ - put_page(page); ret = memory_failure(pfn, 0); if (ret) return ret; From patchwork Tue Sep 10 10:30:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139179 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C52BF76 for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 917BC2171F for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 917BC2171F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 984F36B0269; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 90BB36B0008; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 738FA6B000E; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0230.hostedemail.com [216.40.44.230]) by kanga.kvack.org (Postfix) with ESMTP id 4B76E6B0008 for ; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 0C028181AC9B6 for ; Tue, 10 Sep 2019 10:30:52 +0000 (UTC) X-FDA: 75918642744.22.part19_41b9c497efd35 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.14.6.2 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: part19_41b9c497efd35 X-Filterd-Recvd-Size: 2752 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:51 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 8374FAE5E; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 03/10] mm,hwpoison-inject: don't pin for hwpoison_filter Date: Tue, 10 Sep 2019 12:30:09 +0200 Message-Id: <20190910103016.14290-4-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Another memory error injection interface debugfs:hwpoison/corrupt-pfn also takes bogus refcount for hwpoison_filter(). It's justified because this does a coarse filter, expecting that memory_failure() redoes the check for sure. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador --- mm/hwpoison-inject.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c index 5b7430bd83a6..0c8cdb80fd7d 100644 --- a/mm/hwpoison-inject.c +++ b/mm/hwpoison-inject.c @@ -26,11 +26,6 @@ static int hwpoison_inject(void *data, u64 val) p = pfn_to_page(pfn); hpage = compound_head(p); - /* - * This implies unable to support free buddy pages. - */ - if (!get_hwpoison_page(p)) - return 0; if (!hwpoison_filter_enable) goto inject; @@ -40,23 +35,20 @@ static int hwpoison_inject(void *data, u64 val) * This implies unable to support non-LRU pages. */ if (!PageLRU(hpage) && !PageHuge(p)) - goto put_out; + return 0; /* - * do a racy check with elevated page count, to make sure PG_hwpoison - * will only be set for the targeted owner (or on a free page). + * do a racy check to make sure PG_hwpoison will only be set for + * the targeted owner (or on a free page). * memory_failure() will redo the check reliably inside page lock. */ err = hwpoison_filter(hpage); if (err) - goto put_out; + return 0; inject: pr_info("Injecting memory failure at pfn %#lx\n", pfn); - return memory_failure(pfn, MF_COUNT_INCREASED); -put_out: - put_hwpoison_page(p); - return 0; + return memory_failure(pfn, 0); } static int hwpoison_unpoison(void *data, u64 val) From patchwork Tue Sep 10 10:30:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139191 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01ADB76 for ; Tue, 10 Sep 2019 10:31:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC66220872 for ; Tue, 10 Sep 2019 10:31:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC66220872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4F4E96B0010; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 231836B000C; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F00A36B0010; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0062.hostedemail.com [216.40.44.62]) by kanga.kvack.org (Postfix) with ESMTP id B8AF26B000C for ; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 6808B813F for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-FDA: 75918642786.17.fuel52_41efec64e613b X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30051:30054:30070:30090,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:30,LUA_SUMMARY:none X-HE-Tag: fuel52_41efec64e613b X-Filterd-Recvd-Size: 3947 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 9D768AF0B; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 04/10] mm,hwpoison: remove MF_COUNT_INCREASED Date: Tue, 10 Sep 2019 12:30:10 +0200 Message-Id: <20190910103016.14290-5-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Now there's no user of MF_COUNT_INCREASED, so we can safely remove it from all calling points. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador Acked-by: David Hildenbrand --- include/linux/mm.h | 7 +++---- mm/memory-failure.c | 16 +++------------- 2 files changed, 6 insertions(+), 17 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ad6766a08f9b..fb36a4165a4e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2814,10 +2814,9 @@ void register_page_bootmem_memmap(unsigned long section_nr, struct page *map, unsigned long nr_pages); enum mf_flags { - MF_COUNT_INCREASED = 1 << 0, - MF_ACTION_REQUIRED = 1 << 1, - MF_MUST_KILL = 1 << 2, - MF_SOFT_OFFLINE = 1 << 3, + MF_ACTION_REQUIRED = 1 << 0, + MF_MUST_KILL = 1 << 1, + MF_SOFT_OFFLINE = 1 << 2, }; extern int memory_failure(unsigned long pfn, int flags); extern void memory_failure_queue(unsigned long pfn, int flags); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index e43b61462fd5..1be785b25324 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1092,7 +1092,7 @@ static int memory_failure_hugetlb(unsigned long pfn, int flags) num_poisoned_pages_inc(); - if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p)) { + if (!get_hwpoison_page(p)) { /* * Check "filter hit" and "race with other subpage." */ @@ -1286,7 +1286,7 @@ int memory_failure(unsigned long pfn, int flags) * In fact it's dangerous to directly bump up page count from 0, * that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ - if (!(flags & MF_COUNT_INCREASED) && !get_hwpoison_page(p)) { + if (!get_hwpoison_page(p)) { if (is_free_buddy_page(p)) { action_result(pfn, MF_MSG_BUDDY, MF_DELAYED); return 0; @@ -1327,10 +1327,7 @@ int memory_failure(unsigned long pfn, int flags) shake_page(p, 0); /* shake_page could have turned it free. */ if (!PageLRU(p) && is_free_buddy_page(p)) { - if (flags & MF_COUNT_INCREASED) - action_result(pfn, MF_MSG_BUDDY, MF_DELAYED); - else - action_result(pfn, MF_MSG_BUDDY_2ND, MF_DELAYED); + action_result(pfn, MF_MSG_BUDDY_2ND, MF_DELAYED); return 0; } @@ -1618,9 +1615,6 @@ static int __get_any_page(struct page *p, unsigned long pfn, int flags) { int ret; - if (flags & MF_COUNT_INCREASED) - return 1; - /* * When the target page is a free hugepage, just remove it * from free hugepage list. @@ -1890,15 +1884,11 @@ int soft_offline_page(struct page *page, int flags) if (is_zone_device_page(page)) { pr_debug_ratelimited("soft_offline: %#lx page is device page\n", pfn); - if (flags & MF_COUNT_INCREASED) - put_page(page); return -EIO; } if (PageHWPoison(page)) { pr_info("soft offline: %#lx page already poisoned\n", pfn); - if (flags & MF_COUNT_INCREASED) - put_hwpoison_page(page); return -EBUSY; } From patchwork Tue Sep 10 10:30:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139185 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29CF376 for ; Tue, 10 Sep 2019 10:31:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E796721019 for ; Tue, 10 Sep 2019 10:31:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E796721019 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 184016B0008; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CDE346B000C; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 981F26B0003; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id 5D3FA6B000C for ; Tue, 10 Sep 2019 06:30:52 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 11E2D824376E for ; Tue, 10 Sep 2019 10:30:52 +0000 (UTC) X-FDA: 75918642744.19.match90_41b91e8a4cd27 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30003:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.14.6.2 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: match90_41b91e8a4cd27 X-Filterd-Recvd-Size: 6671 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:51 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 9D29BAE84; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 05/10] mm: remove flag argument from soft offline functions Date: Tue, 10 Sep 2019 12:30:11 +0200 Message-Id: <20190910103016.14290-6-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi The argument @flag no longer affects the behavior of soft_offline_page() and its variants, so let's remove them. Signed-off-by: Naoya Horiguchi Signed-off-by: Oscar Salvador Acked-by: David Hildenbrand --- drivers/base/memory.c | 2 +- include/linux/mm.h | 2 +- mm/madvise.c | 2 +- mm/memory-failure.c | 27 +++++++++++++-------------- 4 files changed, 16 insertions(+), 17 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 6bea4f3f8040..e5485c22ef77 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -540,7 +540,7 @@ static ssize_t soft_offline_page_store(struct device *dev, pfn >>= PAGE_SHIFT; if (!pfn_valid(pfn)) return -ENXIO; - ret = soft_offline_page(pfn_to_page(pfn), 0); + ret = soft_offline_page(pfn_to_page(pfn)); return ret == 0 ? count : ret; } diff --git a/include/linux/mm.h b/include/linux/mm.h index fb36a4165a4e..3cc800d9f57a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2827,7 +2827,7 @@ extern int sysctl_memory_failure_early_kill; extern int sysctl_memory_failure_recovery; extern void shake_page(struct page *p, int access); extern atomic_long_t num_poisoned_pages __read_mostly; -extern int soft_offline_page(struct page *page, int flags); +extern int soft_offline_page(struct page *page); /* diff --git a/mm/madvise.c b/mm/madvise.c index fbe6d402232c..ece128211400 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -906,7 +906,7 @@ static int madvise_inject_error(int behavior, pr_info("Soft offlining pfn %#lx at process virtual address %#lx\n", pfn, start); - ret = soft_offline_page(page, 0); + ret = soft_offline_page(page); if (ret) return ret; continue; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 1be785b25324..5071d39bdfef 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1478,7 +1478,7 @@ static void memory_failure_work_func(struct work_struct *work) if (!gotten) break; if (entry.flags & MF_SOFT_OFFLINE) - soft_offline_page(pfn_to_page(entry.pfn), entry.flags); + soft_offline_page(pfn_to_page(entry.pfn)); else memory_failure(entry.pfn, entry.flags); } @@ -1611,7 +1611,7 @@ static struct page *new_page(struct page *p, unsigned long private) * that is not free, and 1 for any other page type. * For 1 the page is returned with increased page count, otherwise not. */ -static int __get_any_page(struct page *p, unsigned long pfn, int flags) +static int __get_any_page(struct page *p, unsigned long pfn) { int ret; @@ -1638,9 +1638,9 @@ static int __get_any_page(struct page *p, unsigned long pfn, int flags) return ret; } -static int get_any_page(struct page *page, unsigned long pfn, int flags) +static int get_any_page(struct page *page, unsigned long pfn) { - int ret = __get_any_page(page, pfn, flags); + int ret = __get_any_page(page, pfn); if (ret == 1 && !PageHuge(page) && !PageLRU(page) && !__PageMovable(page)) { @@ -1653,7 +1653,7 @@ static int get_any_page(struct page *page, unsigned long pfn, int flags) /* * Did it turn free? */ - ret = __get_any_page(page, pfn, 0); + ret = __get_any_page(page, pfn); if (ret == 1 && !PageLRU(page)) { /* Drop page reference which is from __get_any_page() */ put_hwpoison_page(page); @@ -1665,7 +1665,7 @@ static int get_any_page(struct page *page, unsigned long pfn, int flags) return ret; } -static int soft_offline_huge_page(struct page *page, int flags) +static int soft_offline_huge_page(struct page *page) { int ret; unsigned long pfn = page_to_pfn(page); @@ -1724,7 +1724,7 @@ static int soft_offline_huge_page(struct page *page, int flags) return ret; } -static int __soft_offline_page(struct page *page, int flags) +static int __soft_offline_page(struct page *page) { int ret; unsigned long pfn = page_to_pfn(page); @@ -1804,7 +1804,7 @@ static int __soft_offline_page(struct page *page, int flags) return ret; } -static int soft_offline_in_use_page(struct page *page, int flags) +static int soft_offline_in_use_page(struct page *page) { int ret; int mt; @@ -1834,9 +1834,9 @@ static int soft_offline_in_use_page(struct page *page, int flags) mt = get_pageblock_migratetype(page); set_pageblock_migratetype(page, MIGRATE_ISOLATE); if (PageHuge(page)) - ret = soft_offline_huge_page(page, flags); + ret = soft_offline_huge_page(page); else - ret = __soft_offline_page(page, flags); + ret = __soft_offline_page(page); set_pageblock_migratetype(page, mt); return ret; } @@ -1857,7 +1857,6 @@ static int soft_offline_free_page(struct page *page) /** * soft_offline_page - Soft offline a page. * @page: page to offline - * @flags: flags. Same as memory_failure(). * * Returns 0 on success, otherwise negated errno. * @@ -1876,7 +1875,7 @@ static int soft_offline_free_page(struct page *page) * This is not a 100% solution for all memory, but tries to be * ``good enough'' for the majority of memory. */ -int soft_offline_page(struct page *page, int flags) +int soft_offline_page(struct page *page) { int ret; unsigned long pfn = page_to_pfn(page); @@ -1893,11 +1892,11 @@ int soft_offline_page(struct page *page, int flags) } get_online_mems(); - ret = get_any_page(page, pfn, flags); + ret = get_any_page(page, pfn); put_online_mems(); if (ret > 0) - ret = soft_offline_in_use_page(page, flags); + ret = soft_offline_in_use_page(page); else if (ret == 0) ret = soft_offline_free_page(page); From patchwork Tue Sep 10 10:30:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139193 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6A5776 for ; Tue, 10 Sep 2019 10:31:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9A74420872 for ; Tue, 10 Sep 2019 10:31:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A74420872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7D9DB6B000C; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 678326B026F; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A1506B000E; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id DD3276B0266 for ; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 973FC180AD7C3 for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-FDA: 75918642786.23.hands72_41f6981c85528 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30012:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: hands72_41f6981c85528 X-Filterd-Recvd-Size: 3523 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id C4C37AF68; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 06/10] mm,hwpoison: Unify THP handling for hard and soft offline Date: Tue, 10 Sep 2019 12:30:12 +0200 Message-Id: <20190910103016.14290-7-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Place the THP's page handling in a helper and use it from both hard and soft-offline machinery, so we get rid of some duplicated code. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 48 ++++++++++++++++++++++-------------------------- 1 file changed, 22 insertions(+), 26 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 5071d39bdfef..820742035402 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1077,6 +1077,25 @@ static int identify_page_state(unsigned long pfn, struct page *p, return page_action(ps, p, pfn); } +static int try_to_split_thp_page(struct page *page, const char *msg) +{ + lock_page(page); + if (!PageAnon(page) || unlikely(split_huge_page(page))) { + unsigned long pfn = page_to_pfn(page); + + unlock_page(page); + if (!PageAnon(page)) + pr_info("%s: %#lx: non anonymous thp\n", msg, pfn); + else + pr_info("%s: %#lx: thp split failed\n", msg, pfn); + put_hwpoison_page(page); + return -EBUSY; + } + unlock_page(page); + + return 0; +} + static int memory_failure_hugetlb(unsigned long pfn, int flags) { struct page *p = pfn_to_page(pfn); @@ -1297,21 +1316,8 @@ int memory_failure(unsigned long pfn, int flags) } if (PageTransHuge(hpage)) { - lock_page(p); - if (!PageAnon(p) || unlikely(split_huge_page(p))) { - unlock_page(p); - if (!PageAnon(p)) - pr_err("Memory failure: %#lx: non anonymous thp\n", - pfn); - else - pr_err("Memory failure: %#lx: thp split failed\n", - pfn); - if (TestClearPageHWPoison(p)) - num_poisoned_pages_dec(); - put_hwpoison_page(p); + if (try_to_split_thp_page(p, "Memory Failure") < 0) return -EBUSY; - } - unlock_page(p); VM_BUG_ON_PAGE(!page_count(p), p); hpage = compound_head(p); } @@ -1810,19 +1816,9 @@ static int soft_offline_in_use_page(struct page *page) int mt; struct page *hpage = compound_head(page); - if (!PageHuge(page) && PageTransHuge(hpage)) { - lock_page(page); - if (!PageAnon(page) || unlikely(split_huge_page(page))) { - unlock_page(page); - if (!PageAnon(page)) - pr_info("soft offline: %#lx: non anonymous thp\n", page_to_pfn(page)); - else - pr_info("soft offline: %#lx: thp split failed\n", page_to_pfn(page)); - put_hwpoison_page(page); + if (!PageHuge(page) && PageTransHuge(hpage)) + if (try_to_split_thp_page(page, "soft offline") < 0) return -EBUSY; - } - unlock_page(page); - } /* * Setting MIGRATE_ISOLATE here ensures that the page will be linked From patchwork Tue Sep 10 10:30:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139195 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D2521709 for ; Tue, 10 Sep 2019 10:31:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4716420872 for ; Tue, 10 Sep 2019 10:31:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4716420872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A6A0B6B026A; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 872456B0266; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42C6D6B0266; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 041316B026B for ; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id A9B8E8130 for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-FDA: 75918642786.01.trick57_41f539d83380d X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30012:30051:30054:30056:30070:30075:30083:30091,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:33,LUA_SUMMARY:none X-HE-Tag: trick57_41f539d83380d X-Filterd-Recvd-Size: 11492 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id C4AF8AF59; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 07/10] mm,hwpoison: Rework soft offline for in-use pages Date: Tue, 10 Sep 2019 12:30:13 +0200 Message-Id: <20190910103016.14290-8-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch changes the way we set and handle in-use poisoned pages. Until now, poisoned pages were released to the buddy allocator, trusting that the checks that take place prior to hand the page to userspace would act as a safe net and would skip that page. This has proved to be wrong, as we got some pfn walkers out there, like compaction, that all they care is the page to be PageBuddy and be in a freelist. Although this might not be the only user, having poisoned pages in the buddy allocator seems a bad idea as we should only have free pages that are ready and meant to be used as such. Before explainaing the taken approach, let us break down the kind of pages we can soft offline. - Anonymous THP (after the split, they end up being 4K pages) - Hugetlb - Order-0 pages (that can be either migrated or invalited) The following will only refer to in-use pages, free pages will be explained in patch#9. * Normal pages (order-0 and anon-THP) - If they are clean and unmapped page cache pages, we detach the page from its mapping. - If they are mapped/dirty, we do the isolate-and-migrate dance. Either way, do not call put_page directly from those paths. Instead, we keep the page and send it to page_set_poison. page_set_poison sets the HWPoison flag and calls put_page. This call to put_page is mainly to be able to call __page_cache_release, since this function is not exported. Down the chain, we placed a check for HWPoison page in free_pages_prepare, that just skips any poisoned page, so those pages do not end up in any pcplist/freelist. [[Now, I think that we would be better off if we duplicated/exported __page_cache_release in/to the hwpoison code, so this last put_page could go]] After that, we set the refcount on the page to 1 and we increment the poisoned pages counter. * Hugetlb pages - we isolate-and-migrate them After the migration has been succesful, we call page_set_poison that sets the HWPoison flag and actually calls dissolve_free_huge_page for hugetlb pages. When dissolving a non-gigantib hugetlb page and we know that the memory range contains poisoned pages, we free the pages as order-0 pages, so free_pages_prepare will skip them accordingly. poisoned page. Since the infrastructure is already there because that is the way we free gigantic hugetlb pages, it does not take any effort to adapt it for non-gigantic hugetlb pages. Because of the way we handle now in-use pages, we can safely drop the put-as-isolation-migratetype dance, that was guarding for the poisoned pages to end up in pcplists. Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 35 +++++++++++++++++++++++++------ mm/memory-failure.c | 60 ++++++++++++++++++++++++++--------------------------- mm/migrate.c | 11 +++------- mm/page_alloc.c | 3 +++ 4 files changed, 64 insertions(+), 45 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ef37c85423a5..139e1c05c9a1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1045,16 +1045,17 @@ static int hstate_next_node_to_free(struct hstate *h, nodemask_t *nodes_allowed) ((node = hstate_next_node_to_free(hs, mask)) || 1); \ nr_nodes--) -#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE -static void destroy_compound_gigantic_page(struct page *page, - unsigned int order) +static void destroy_compound_page(struct page *page, unsigned int order) { int i; int nr_pages = 1 << order; struct page *p = page + 1; + bool gigantic = order > MAX_ORDER - 1; atomic_set(compound_mapcount_ptr(page), 0); for (i = 1; i < nr_pages; i++, p = mem_map_next(p, page, i)) { + if (!gigantic) + p->mapping = NULL; clear_compound_head(p); set_page_refcounted(p); } @@ -1063,6 +1064,13 @@ static void destroy_compound_gigantic_page(struct page *page, __ClearPageHead(page); } +#ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE +static void destroy_compound_gigantic_page(struct page *page, + unsigned int order) +{ + destroy_compound_page(page, order); +} + static void free_gigantic_page(struct page *page, unsigned int order) { free_contig_range(page_to_pfn(page), 1 << order); @@ -1175,6 +1183,8 @@ static inline void destroy_compound_gigantic_page(struct page *page, static void update_and_free_page(struct hstate *h, struct page *page) { int i; + bool poisoned = false; + unsigned int order = huge_page_order(h); if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) return; @@ -1182,6 +1192,8 @@ static void update_and_free_page(struct hstate *h, struct page *page) h->nr_huge_pages--; h->nr_huge_pages_node[page_to_nid(page)]--; for (i = 0; i < pages_per_huge_page(h); i++) { + if (unlikely(PageHWPoison(page))) + poisoned = true; page[i].flags &= ~(1 << PG_locked | 1 << PG_error | 1 << PG_referenced | 1 << PG_dirty | 1 << PG_active | 1 << PG_private | @@ -1191,10 +1203,21 @@ static void update_and_free_page(struct hstate *h, struct page *page) set_compound_page_dtor(page, NULL_COMPOUND_DTOR); set_page_refcounted(page); if (hstate_is_gigantic(h)) { - destroy_compound_gigantic_page(page, huge_page_order(h)); - free_gigantic_page(page, huge_page_order(h)); + destroy_compound_gigantic_page(page, order); + free_gigantic_page(page, order); } else { - __free_pages(page, huge_page_order(h)); + if (poisoned) { + unsigned long pfn = page_to_pfn(page); + /* + * If we have poisoned pages in the range, + * we free them up as order-0 pages, so + * free_pages_prepare will skip them accordingly. + */ + destroy_compound_page(page, order); + free_contig_range(pfn, 1 << order); + } else { + __free_pages(page, order); + } } } diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 820742035402..d44dacb8e2ea 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -78,6 +78,24 @@ EXPORT_SYMBOL_GPL(hwpoison_filter_dev_minor); EXPORT_SYMBOL_GPL(hwpoison_filter_flags_mask); EXPORT_SYMBOL_GPL(hwpoison_filter_flags_value); +static bool page_set_poison(struct page *page) +{ + SetPageHWPoison(page); + + if (PageHuge(page) && dissolve_free_huge_page(page)) + goto error; + else if (!PageHuge(page) && page_count(page)) + put_page(page); + + set_page_refcounted(page); + num_poisoned_pages_inc(); + + return true; +error: + ClearPageHWPoison(page); + return false; +} + static int hwpoison_filter_dev(struct page *p) { struct address_space *mapping; @@ -1704,28 +1722,16 @@ static int soft_offline_huge_page(struct page *page) ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, MIGRATE_SYNC, MR_MEMORY_FAILURE); - if (ret) { + if (!ret) { + if (!page_set_poison(page)) + ret = -EBUSY; + } else { pr_info("soft offline: %#lx: hugepage migration failed %d, type %lx (%pGp)\n", pfn, ret, page->flags, &page->flags); if (!list_empty(&pagelist)) putback_movable_pages(&pagelist); if (ret > 0) ret = -EIO; - } else { - /* - * We set PG_hwpoison only when the migration source hugepage - * was successfully dissolved, because otherwise hwpoisoned - * hugepage remains on free hugepage list, then userspace will - * find it as SIGBUS by allocation failure. That's not expected - * in soft-offlining. - */ - ret = dissolve_free_huge_page(page); - if (!ret) { - if (set_hwpoison_free_buddy_page(page)) - num_poisoned_pages_inc(); - else - ret = -EBUSY; - } } return ret; } @@ -1760,10 +1766,8 @@ static int __soft_offline_page(struct page *page) * would need to fix isolation locking first. */ if (ret == 1) { - put_hwpoison_page(page); pr_info("soft_offline: %#lx: invalidated\n", pfn); - SetPageHWPoison(page); - num_poisoned_pages_inc(); + page_set_poison(page); return 0; } @@ -1794,7 +1798,12 @@ static int __soft_offline_page(struct page *page) list_add(&page->lru, &pagelist); ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, MIGRATE_SYNC, MR_MEMORY_FAILURE); - if (ret) { + if (!ret) { + /* + * Page was succesfully migrated. + */ + page_set_poison(page); + } else { if (!list_empty(&pagelist)) putback_movable_pages(&pagelist); @@ -1813,27 +1822,16 @@ static int __soft_offline_page(struct page *page) static int soft_offline_in_use_page(struct page *page) { int ret; - int mt; struct page *hpage = compound_head(page); if (!PageHuge(page) && PageTransHuge(hpage)) if (try_to_split_thp_page(page, "soft offline") < 0) return -EBUSY; - /* - * Setting MIGRATE_ISOLATE here ensures that the page will be linked - * to free list immediately (not via pcplist) when released after - * successful page migration. Otherwise we can't guarantee that the - * page is really free after put_page() returns, so - * set_hwpoison_free_buddy_page() highly likely fails. - */ - mt = get_pageblock_migratetype(page); - set_pageblock_migratetype(page, MIGRATE_ISOLATE); if (PageHuge(page)) ret = soft_offline_huge_page(page); else ret = __soft_offline_page(page); - set_pageblock_migratetype(page, mt); return ret; } diff --git a/mm/migrate.c b/mm/migrate.c index bdd1e95a459e..c396a019b2a4 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1223,16 +1223,11 @@ static ICE_noinline int unmap_and_move(new_page_t get_new_page, * we want to retry. */ if (rc == MIGRATEPAGE_SUCCESS) { - put_page(page); - if (reason == MR_MEMORY_FAILURE) { + if (reason != MR_MEMORY_FAILURE) /* - * Set PG_HWPoison on just freed page - * intentionally. Although it's rather weird, - * it's how HWPoison flag works at the moment. + * We handle poisoned pages in hwpoison code */ - if (set_hwpoison_free_buddy_page(page)) - num_poisoned_pages_inc(); - } + put_page(page); } else { if (rc != -EAGAIN) { if (likely(!__PageMovable(page))) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c5d62f1c2851..fe38229d0a77 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1134,6 +1134,9 @@ static __always_inline bool free_pages_prepare(struct page *page, trace_mm_page_free(page, order); + if (unlikely(PageHWPoison(page))) + return false; + /* * Check tail pages before head page information is cleared to * avoid checking PageCompound for order-0 pages. From patchwork Tue Sep 10 10:30:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139197 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A439016B1 for ; Tue, 10 Sep 2019 10:31:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 716FC20872 for ; Tue, 10 Sep 2019 10:31:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 716FC20872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CDA0C6B0266; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9D6E66B026C; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 767006B026A; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 04D6C6B026C for ; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id AEBD58141 for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-FDA: 75918642786.26.cakes93_41f2bd5800f25 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30012:30034:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:28,LUA_SUMMARY:none X-HE-Tag: cakes93_41f2bd5800f25 X-Filterd-Recvd-Size: 7863 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id E9153AFE4; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 08/10] mm,hwpoison: Refactor soft_offline_huge_page and __soft_offline_page Date: Tue, 10 Sep 2019 12:30:14 +0200 Message-Id: <20190910103016.14290-9-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: soft_offline_huge_page and __soft_offline_page share quite some code, and it can be re-joined into one single function without losing neither functionatilty nor readibility. This allows us of getting rid of quite some duplicated code. Signed-off-by: Oscar Salvador --- mm/memory-failure.c | 145 ++++++++++++++++++++-------------------------------- 1 file changed, 56 insertions(+), 89 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index d44dacb8e2ea..ce017a0d79a6 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1689,57 +1689,48 @@ static int get_any_page(struct page *page, unsigned long pfn) return ret; } -static int soft_offline_huge_page(struct page *page) +static bool isolate_page(struct page *page, struct list_head *pagelist) { - int ret; - unsigned long pfn = page_to_pfn(page); - struct page *hpage = compound_head(page); - LIST_HEAD(pagelist); + bool isolated = false; + bool lru = PageLRU(page); - /* - * This double-check of PageHWPoison is to avoid the race with - * memory_failure(). See also comment in __soft_offline_page(). - */ - lock_page(hpage); - if (PageHWPoison(hpage)) { - unlock_page(hpage); - put_hwpoison_page(hpage); - pr_info("soft offline: %#lx hugepage already poisoned\n", pfn); - return -EBUSY; + if (PageHuge(page)) { + isolated = isolate_huge_page(page, pagelist); + } else { + if (PageLRU(page)) + isolated = !isolate_lru_page(page); + else + isolated = !isolate_movable_page(page, ISOLATE_UNEVICTABLE); + + if (isolated) { + if (lru) + inc_node_page_state(page, NR_ISOLATED_ANON + + page_is_file_cache(page)); + list_add(&page->lru, pagelist); + } } - unlock_page(hpage); - ret = isolate_huge_page(hpage, &pagelist); /* - * get_any_page() and isolate_huge_page() takes a refcount each, - * so need to drop one here. + * If we succeed to isolate the page, we grabbed another refcount on + * the page, so we can safely drop the one we got from get_any_pages(). + * If we failed to isolate the page, it means that we cannot go further + * any further and we will return an error, so drop the reference we got + * from get_any_pages() as well. */ - put_hwpoison_page(hpage); - if (!ret) { - pr_info("soft offline: %#lx hugepage failed to isolate\n", pfn); - return -EBUSY; - } - - ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, - MIGRATE_SYNC, MR_MEMORY_FAILURE); - if (!ret) { - if (!page_set_poison(page)) - ret = -EBUSY; - } else { - pr_info("soft offline: %#lx: hugepage migration failed %d, type %lx (%pGp)\n", - pfn, ret, page->flags, &page->flags); - if (!list_empty(&pagelist)) - putback_movable_pages(&pagelist); - if (ret > 0) - ret = -EIO; - } - return ret; + put_hwpoison_page(page); + return isolated; } - +/* + * This routine handles both hugetlb and normal pages + */ static int __soft_offline_page(struct page *page) { - int ret; + int ret = 0; unsigned long pfn = page_to_pfn(page); + struct page *hpage = compound_head(page); + const char *msg_page[] = { "page", "hugepage"}; + bool huge = PageHuge(page); + LIST_HEAD(pagelist); /* * Check PageHWPoison again inside page lock because PageHWPoison @@ -1747,92 +1738,68 @@ static int __soft_offline_page(struct page *page) * memory_failure() also double-checks PageHWPoison inside page lock, * so there's no race between soft_offline_page() and memory_failure(). */ - lock_page(page); - wait_on_page_writeback(page); - if (PageHWPoison(page)) { - unlock_page(page); - put_hwpoison_page(page); + lock_page(hpage); + if (!PageHuge(page)) + wait_on_page_writeback(page); + if (PageHWPoison(hpage)) { + unlock_page(hpage); + put_hwpoison_page(hpage); pr_info("soft offline: %#lx page already poisoned\n", pfn); return -EBUSY; } - /* - * Try to invalidate first. This should work for - * non dirty unmapped page cache pages. - */ - ret = invalidate_inode_page(page); + + if (!PageHuge(page)) + /* + * Try to invalidate first. This should work for + * non dirty unmapped page cache pages. + */ + ret = invalidate_inode_page(page); unlock_page(page); /* * RED-PEN would be better to keep it isolated here, but we * would need to fix isolation locking first. */ - if (ret == 1) { + if (ret) { pr_info("soft_offline: %#lx: invalidated\n", pfn); page_set_poison(page); return 0; } - /* - * Simple invalidation didn't work. - * Try to migrate to a new page instead. migrate.c - * handles a large number of cases for us. - */ - if (PageLRU(page)) - ret = isolate_lru_page(page); - else - ret = isolate_movable_page(page, ISOLATE_UNEVICTABLE); - /* - * Drop page reference which is came from get_any_page() - * successful isolate_lru_page() already took another one. - */ - put_hwpoison_page(page); - if (!ret) { - LIST_HEAD(pagelist); - /* - * After isolated lru page, the PageLRU will be cleared, - * so use !__PageMovable instead for LRU page's mapping - * cannot have PAGE_MAPPING_MOVABLE. - */ - if (!__PageMovable(page)) - inc_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_cache(page)); - list_add(&page->lru, &pagelist); + if (isolate_page(hpage, &pagelist)) { ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, - MIGRATE_SYNC, MR_MEMORY_FAILURE); + MIGRATE_SYNC, MR_MEMORY_FAILURE); if (!ret) { /* * Page was succesfully migrated. */ - page_set_poison(page); + if (!page_set_poison(page)) + ret = -EBUSY; } else { if (!list_empty(&pagelist)) putback_movable_pages(&pagelist); - pr_info("soft offline: %#lx: migration failed %d, type %lx (%pGp)\n", - pfn, ret, page->flags, &page->flags); + pr_info("soft offline: %#lx: %s migration failed %d, type %lx (%pGp)\n", + pfn, msg_page[huge], ret, page->flags, &page->flags); if (ret > 0) ret = -EIO; } } else { - pr_info("soft offline: %#lx: isolation failed: %d, page count %d, type %lx (%pGp)\n", - pfn, ret, page_count(page), page->flags, &page->flags); + pr_info("soft offline: %#lx %s isolation failed: %d, page count %d, type %lx (%pGp)\n", + pfn, msg_page[huge], ret, page_count(page), page->flags, &page->flags); } + return ret; } static int soft_offline_in_use_page(struct page *page) { - int ret; struct page *hpage = compound_head(page); if (!PageHuge(page) && PageTransHuge(hpage)) if (try_to_split_thp_page(page, "soft offline") < 0) return -EBUSY; - if (PageHuge(page)) - ret = soft_offline_huge_page(page); - else - ret = __soft_offline_page(page); - return ret; + return __soft_offline_page(page); } static int soft_offline_free_page(struct page *page) From patchwork Tue Sep 10 10:30:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139199 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD26676 for ; Tue, 10 Sep 2019 10:31:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A3F2A20872 for ; Tue, 10 Sep 2019 10:31:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3F2A20872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EFBF56B000E; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C71BF6B026D; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8261E6B026B; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id 22B3C6B026D for ; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id BEF87181AC9AE for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-FDA: 75918642786.09.pail72_41f88ead7ef4c X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30054:30070,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: pail72_41f88ead7ef4c X-Filterd-Recvd-Size: 5368 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id E54D1AF93; Tue, 10 Sep 2019 10:30:49 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 09/10] mm,hwpoison: Rework soft offline for free pages Date: Tue, 10 Sep 2019 12:30:15 +0200 Message-Id: <20190910103016.14290-10-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: take_page_off_buddy will be used to take a page meant to be poisoned off the buddy allocator. take_page_off_buddy calls break_down_buddy_pages, which will split a higher-order page in case our page belongs to one. Once we grab the page, we call page_set_poison to set it as poisoned and grab a refcount on it. Signed-off-by: Oscar Salvador --- include/linux/page-flags.h | 5 ---- mm/memory-failure.c | 6 +++-- mm/page_alloc.c | 59 +++++++++++++++++++++++++++++++++++++++------- 3 files changed, 54 insertions(+), 16 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f91cb8898ff0..21df81c9ea57 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -414,13 +414,8 @@ PAGEFLAG_FALSE(Uncached) PAGEFLAG(HWPoison, hwpoison, PF_ANY) TESTSCFLAG(HWPoison, hwpoison, PF_ANY) #define __PG_HWPOISON (1UL << PG_hwpoison) -extern bool set_hwpoison_free_buddy_page(struct page *page); #else PAGEFLAG_FALSE(HWPoison) -static inline bool set_hwpoison_free_buddy_page(struct page *page) -{ - return 0; -} #define __PG_HWPOISON 0 #endif diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ce017a0d79a6..03f07015a106 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -78,6 +78,8 @@ EXPORT_SYMBOL_GPL(hwpoison_filter_dev_minor); EXPORT_SYMBOL_GPL(hwpoison_filter_flags_mask); EXPORT_SYMBOL_GPL(hwpoison_filter_flags_value); +extern bool take_page_off_buddy(struct page *page); + static bool page_set_poison(struct page *page) { SetPageHWPoison(page); @@ -1807,8 +1809,8 @@ static int soft_offline_free_page(struct page *page) int rc = dissolve_free_huge_page(page); if (!rc) { - if (set_hwpoison_free_buddy_page(page)) - num_poisoned_pages_inc(); + if (take_page_off_buddy(page)) + page_set_poison(page); else rc = -EBUSY; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index fe38229d0a77..68f6c2cda512 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8605,30 +8605,71 @@ bool is_free_buddy_page(struct page *page) #ifdef CONFIG_MEMORY_FAILURE /* - * Set PG_hwpoison flag if a given page is confirmed to be a free page. This - * test is performed under the zone lock to prevent a race against page - * allocation. + * Break down a higher-order page in sub-pages, and keep our target out of + * buddy allocator. */ -bool set_hwpoison_free_buddy_page(struct page *page) +static void break_down_buddy_pages(struct zone *zone, struct page *page, + struct page *target, int low, int high, + struct free_area *area, int migratetype) +{ + unsigned long size = 1 << high; + struct page *current_buddy, *next_page; + + while (high > low) { + area--; + high--; + size >>= 1; + + if (target >= &page[size]) { + next_page = page + size; + current_buddy = page; + } else { + next_page = page; + current_buddy = page + size; + } + + if (set_page_guard(zone, current_buddy, high, migratetype)) + continue; + + if (current_buddy != target) { + add_to_free_area(current_buddy, area, migratetype); + set_page_order(current_buddy, high); + page = next_page; + } + } +} + +/* + * Take a page that will be marked as poisoned off the buddy allocator. + */ +bool take_page_off_buddy(struct page *page) { struct zone *zone = page_zone(page); unsigned long pfn = page_to_pfn(page); unsigned long flags; unsigned int order; - bool hwpoisoned = false; + bool ret = false; spin_lock_irqsave(&zone->lock, flags); for (order = 0; order < MAX_ORDER; order++) { struct page *page_head = page - (pfn & ((1 << order) - 1)); + int buddy_order = page_order(page_head); + struct free_area *area = &(zone->free_area[buddy_order]); + + if (PageBuddy(page_head) && buddy_order >= order) { + unsigned long pfn_head = page_to_pfn(page_head); + int migratetype = get_pfnblock_migratetype(page_head, + pfn_head); - if (PageBuddy(page_head) && page_order(page_head) >= order) { - if (!TestSetPageHWPoison(page)) - hwpoisoned = true; + del_page_from_free_area(page_head, area); + break_down_buddy_pages(zone, page_head, page, 0, + buddy_order, area, migratetype); + ret = true; break; } } spin_unlock_irqrestore(&zone->lock, flags); - return hwpoisoned; + return ret; } #endif From patchwork Tue Sep 10 10:30:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 11139189 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B68376 for ; Tue, 10 Sep 2019 10:31:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4C10120872 for ; Tue, 10 Sep 2019 10:31:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C10120872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 233776B026E; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 196DB6B000E; Tue, 10 Sep 2019 06:30:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6FBF6B026A; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id BCAD86B000E for ; Tue, 10 Sep 2019 06:30:53 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 7C488824376D for ; Tue, 10 Sep 2019 10:30:53 +0000 (UTC) X-FDA: 75918642786.18.bulb57_41f2ee7033a08 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,osalvador@suse.de,:n-horiguchi@ah.jp.nec.com:mhocko@kernel.org:mike.kravetz@oracle.com::linux-kernel@vger.kernel.org:osalvador@suse.de,RULES_HIT:30036:30045:30054,0,RBL:195.135.220.15:@suse.de:.lbl8.mailshell.net-62.2.6.2 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:29,LUA_SUMMARY:none X-HE-Tag: bulb57_41f2ee7033a08 X-Filterd-Recvd-Size: 3455 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Sep 2019 10:30:52 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 02ADFB008; Tue, 10 Sep 2019 10:30:50 +0000 (UTC) From: Oscar Salvador To: n-horiguchi@ah.jp.nec.com Cc: mhocko@kernel.org, mike.kravetz@oracle.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH 10/10] mm,hwpoison: Use hugetlb_replace_page to replace free hugetlb pages Date: Tue, 10 Sep 2019 12:30:16 +0200 Message-Id: <20190910103016.14290-11-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20190910103016.14290-1-osalvador@suse.de> References: <20190910103016.14290-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When soft offlining a free hugtlb, try first to allocate a new hugetlb to the pool and pass the old state to the new one by move_hugetlb_state(). Either we succeed or not, we dissolve the poisoned hugetlb page. Worst-scenario case is that we cannot allocate a new fresh hugetlb page as a replacement. Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 16 ++++++++++++++++ mm/memory-failure.c | 34 ++++++++++++++++++++++++++++------ 2 files changed, 44 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 139e1c05c9a1..d0844aec7531 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5154,3 +5154,19 @@ void move_hugetlb_state(struct page *oldpage, struct page *newpage, int reason) spin_unlock(&hugetlb_lock); } } + +#ifdef CONFIG_MEMORY_FAILURE +int hugetlb_replace_page(struct page *page, int reason) +{ + int nid = page_to_nid(page); + struct hstate *h = page_hstate(page); + struct page *new_page; + + new_page = alloc_huge_page_nodemask(h, nid, &node_states[N_MEMORY]); + if (!new_page) + return -ENOMEM; + + move_hugetlb_state(page, new_page, reason); + return 0; +} +#endif diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 03f07015a106..fe73fe19c6e9 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -79,6 +79,7 @@ EXPORT_SYMBOL_GPL(hwpoison_filter_flags_mask); EXPORT_SYMBOL_GPL(hwpoison_filter_flags_value); extern bool take_page_off_buddy(struct page *page); +extern int hugetlb_replace_page(struct page *page, int reason); static bool page_set_poison(struct page *page) { @@ -1804,16 +1805,37 @@ static int soft_offline_in_use_page(struct page *page) return __soft_offline_page(page); } +static int soft_offline_free_huge_page(struct page *page) +{ + struct page *hpage = compound_head(page); + + /* + * Try to add a new hugetlb page to the pool + */ + if (hugetlb_replace_page(hpage, MR_MEMORY_FAILURE)) + return -EBUSY; + + /* + * Remove old hugetlb from the pool + */ + if (!page_set_poison(hpage)) + return -EBUSY; + + return 0; +} + static int soft_offline_free_page(struct page *page) { - int rc = dissolve_free_huge_page(page); + int rc = -EBUSY; - if (!rc) { - if (take_page_off_buddy(page)) + if (PageHuge(page)) + rc = soft_offline_free_huge_page(page); + else + if (take_page_off_buddy(page)) { page_set_poison(page); - else - rc = -EBUSY; - } + rc = 0; + } + return rc; }