From patchwork Thu Jun 23 23:51:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E1C7C43334 for ; Thu, 23 Jun 2022 23:52:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ADA298E01A6; Thu, 23 Jun 2022 19:52:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A8A278E01A1; Thu, 23 Jun 2022 19:52:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9521F8E01A6; Thu, 23 Jun 2022 19:52:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 87DCA8E01A1 for ; Thu, 23 Jun 2022 19:52:32 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 68263607CE for ; Thu, 23 Jun 2022 23:52:32 +0000 (UTC) X-FDA: 79611152544.04.BFF0ED9 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf10.hostedemail.com (Postfix) with ESMTP id 17BFAC0033 for ; Thu, 23 Jun 2022 23:52:31 +0000 (UTC) Received: by mail-pj1-f49.google.com with SMTP id p5so1135529pjt.2 for ; Thu, 23 Jun 2022 16:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=guCk9+fmq044uYhThIHO4V934ZX2nYq8f94Emgz8GLM=; b=SUGUBrBFD30w/LODLPszAIWFJuyzy0h8yKc3XwFCVTjTpmAqYwYtNybMHRZ+ghsnt6 oFA4URHrauTUulNAvk0QCM67P/nkIVpijwPqu2IdFqBueiiaMSAQRPwpv6i3r+6CFCg5 plfgyMDGoKKdV7nNouU2+OWjZzQEV8GM9uqudK4CYp5wzLwGI5Uhh02fmt9opzL/hOuQ CHb2xiFPkQFAEUzJn7vUfUw/CjDKDoPj4aI+IY/+fKamjEPVD1pKTCLT8ivMhoUVoJNj HHF6gHkYy+hsHX3Jl6+v2yJCbU1tOMvmcYoLzEGZK1goInD5vx4s9WLc/yecqlQhAZqt O+OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=guCk9+fmq044uYhThIHO4V934ZX2nYq8f94Emgz8GLM=; b=Xy3rEc8iMe1Qrtdsm9DI1Pko+K9p7tlyWavxma7tl4R9tcVEDkn39dIDOnHcfx/Yoz GmYSTgdaREgWhx3zLHliquWAE4EPJy3nCo+nI4jnNWEyXbmhrHroEXKk1bb/NOyoqEaX bTU/O1lt0jHVQ7jPLhARhvrgjJoKlDU76ydmN4Q4J7TcSKRzbGGTNiJJSHIYnTbHR0Hk K3Jn49e0Io0c3YMt/Nn8QMZZkf/vEmAQZ2PCXm4NBFSl30K8mCXzq3V0fTUVFYjkoMWq M5RGgyPOxny8qKW7PkC2QlZyWlqtpXUyvnkhI+adQFnRDYSLr3YRH8t/mYxMXI8WXUgq GZZA== X-Gm-Message-State: AJIora/arii6IHo4NTtuBRDdVOMDL9mcdTuPajcu6uGpX+XzcdSaEad+ c/isTUZfFlQLu8e3RqAd+1edqO/ZLQ== X-Google-Smtp-Source: AGRyM1vkLEUzFkCo4X+WONweFksRrl4oFKOzMvGjQX9HHb/bP3CF+/JL7adCPfO4KW6Vp/9kEJ5QLg== X-Received: by 2002:a17:90b:1a8c:b0:1ed:1afb:7a73 with SMTP id ng12-20020a17090b1a8c00b001ed1afb7a73mr571077pjb.144.1656028351028; Thu, 23 Jun 2022 16:52:31 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:30 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/9] mm/hugetlb: remove checking hstate_is_gigantic() in return_unused_surplus_pages() Date: Fri, 24 Jun 2022 08:51:45 +0900 Message-Id: <20220623235153.2623702-2-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SUGUBrBF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028352; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=guCk9+fmq044uYhThIHO4V934ZX2nYq8f94Emgz8GLM=; b=VNM/QkA7AoZIs55iO9STdvs+4lvRoZrrUWpYXIOcETX3LBj1E6dYPAtWapneZ3abNP4l7s 2VVdVus4Eq+ATEyqC5YMWwvNLEJHKl8xBM4znw651Ot76+ckBI9rYwoTx9rD4eF9cCKB1u g+p2QS16W7BiVXC92Zc8giNfhXjtzGc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028352; a=rsa-sha256; cv=none; b=SoOl3uzh8/clCp9GPORVmG2goHzTRVuGvFGGlbZPcMohT/C8BL23C7/Vynww/ty3CycwjC W/dKeSyXKL2dS3pn+8nQpmFcnXcqFoKw+efqzvw02R8rUDlLIzXjfFLahGri8kZijwHEUS 2Bk1ozRKZnAnOnifhXFgHHtvtTsDBlQ= X-Stat-Signature: a3ip1j9pmop93e6equ1zwihij9b398a4 X-Rspamd-Queue-Id: 17BFAC0033 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SUGUBrBF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-Rspamd-Server: rspam12 X-HE-Tag: 1656028351-245513 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi I found a weird state of 1GB hugepage pool, caused by the following procedure: - run a process reserving all free 1GB hugepages, - shrink free 1GB hugepage pool to zero (i.e. writing 0 to /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages), then - kill the reserving process. , then all the hugepages are free *and* surplus at the same time. $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages 3 $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages 3 $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/resv_hugepages 0 $ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/surplus_hugepages 3 This state is resolved by reserving and allocating the pages then freeing them again, so this seems not to result in serious problem. But it's a little surprizing (shrinking pool suddenly fails). This behavior is caused by hstate_is_gigantic() check in return_unused_surplus_pages(). This was introduced so long ago in 2008 by commit aa888a74977a ("hugetlb: support larger than MAX_ORDER"), and it seems to me that this check is no longer unnecessary. Let's remove it. Signed-off-by: Naoya Horiguchi --- mm/hugetlb.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a57e1be41401..c538278170a2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2432,10 +2432,6 @@ static void return_unused_surplus_pages(struct hstate *h, /* Uncommit the reservation */ h->resv_huge_pages -= unused_resv_pages; - /* Cannot return gigantic pages currently */ - if (hstate_is_gigantic(h)) - goto out; - /* * Part (or even all) of the reservation could have been backed * by pre-allocated pages. Only free surplus pages. From patchwork Thu Jun 23 23:51:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FD35C43334 for ; Thu, 23 Jun 2022 23:52:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D25678E01A7; Thu, 23 Jun 2022 19:52:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD4128E01A1; Thu, 23 Jun 2022 19:52:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9C898E01A7; Thu, 23 Jun 2022 19:52:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AC2FB8E01A1 for ; Thu, 23 Jun 2022 19:52:35 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 87AA132B for ; Thu, 23 Jun 2022 23:52:35 +0000 (UTC) X-FDA: 79611152670.30.5C33FBF Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf24.hostedemail.com (Postfix) with ESMTP id 30DD5180034 for ; Thu, 23 Jun 2022 23:52:35 +0000 (UTC) Received: by mail-pf1-f178.google.com with SMTP id bo5so1057591pfb.4 for ; Thu, 23 Jun 2022 16:52:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l+d0OGtRtkY09M0VLBofFywNSItSfbSemC3EkYMtQyY=; b=ou1qFSPl8LCvTmp9oUsKtEu26AbVApYmilkjIi5Rqdhn7tRx0awXZilRcMS07UUquu QsIs67X8QqsYXNyNzHjQNz4bMesy8pxAo3Bce4yXtWS4IsYr//Q6obb1scOMH/WtDmL/ SsxBh2QpvXr8qN7vUjWsuvyZg8bCvfUzCU06EOdhNks5pav3NCoN2wDpgXepSFDFWoJh 8gCeRcS9v75oR4dEJJeo6B0GAHI6EUGSyxBzaS1PBsydIbR84FKq3cCjokSPygKkHr3b DoJiHIvWanhxYmJRFwaglGt55RhnE69wEJocNsqgwWlBFy7k6z/n2lV6xFjGI2JJQole GCsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l+d0OGtRtkY09M0VLBofFywNSItSfbSemC3EkYMtQyY=; b=IMoKxtO90/KVbUn1ZyTluL3vyqtVynJO4/LHseuS3fISxVA2v56OCzIiYqnhu6w7gR 62Q68aSBCyBj8GIHA3CNbVnjqF2zAgxdqLTuEYfrdd/qOkGS1mPD0yguYSsl+uc3TbZr y80ozxJ20o125Q2t3S3INuQS8UXQdtHepHyjXqEs8lGqvlOQV9Z0BixnxIr1BNwz0Q7R Q4d92pXoUNKL3X1SVViANbycPbDfcGy3Z+f140g3wENGCzqe/qq2/LRavk3AZXjdVQeV 8x2iWTp8NaP/5zM29UwAUsYX84ppBGfjim07AAxbih2JJdgF9i7eDN13tCy61SjA8wVD HL8Q== X-Gm-Message-State: AJIora+KgDYZ2yhk2DVsHYzW3EvYyZmGMCTP2tfiRezouCVqt+Z2+JcP hL1NXfn1NHUSCpXWUJte6egfe08TYA== X-Google-Smtp-Source: AGRyM1sCQntLUzCfsXoFtqsZy7Xh7dUH0cStwNLtXgYEfOK9ftLkc8zvJEC5P2/PdCc5Fs8haLDBMg== X-Received: by 2002:a05:6a00:134f:b0:51c:4c92:1dae with SMTP id k15-20020a056a00134f00b0051c4c921daemr43342643pfu.32.1656028354057; Thu, 23 Jun 2022 16:52:34 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:33 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 2/9] mm/hugetlb: separate path for hwpoison entry in copy_hugetlb_page_range() Date: Fri, 24 Jun 2022 08:51:46 +0900 Message-Id: <20220623235153.2623702-3-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028355; a=rsa-sha256; cv=none; b=0y8kNcXXmlh325NWPUswjxqujzqwrJwEJh3DH2p08fVqiU3uy2bZLp3hVDK5wJy9J0bqdV 4C1SJHAp1TFRIK5L0/AIghkqTpHPYMHNPvkUlmztEyikKioy70gjYPYxTum9p+6j3GX1i2 TeiJ8xxe8O4yG+CTvq4gDxlHxFMMFNk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ou1qFSPl; spf=pass (imf24.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028355; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l+d0OGtRtkY09M0VLBofFywNSItSfbSemC3EkYMtQyY=; b=8kEd8hVmidUOIs+MLHlEVShy/nq4u5fOPVip99yVO82TghMHGnYZ64hF78rnjkq6OXR+vf mXqOsIZipYOfVSENiP+0kaKXw46IEd6F0EOFb4eTKfvoAPmTd918yEwdoKw4QRs77r/HzU OjD6U03/3IAcQKre20jR25ccGR2zocU= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 30DD5180034 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ou1qFSPl; spf=pass (imf24.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Stat-Signature: ui5pahokj7wgi5gwxdu4a9ejeegzou98 X-HE-Tag: 1656028355-948104 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Originally copy_hugetlb_page_range() handles migration entries and hwpoisone entries in similar manner. But recently the related code path has more code for migration entries, and when is_writable_migration_entry() was converted to !is_readable_migration_entry(), hwpoison entries on source processes got to be unexpectedly updated (which is legitimate for migration entries, but not for hwpoison entries). This results in unexpected serious issues like kernel panic when foking processes with hwpoison entries in pmd. Separate the if branch into one for hwpoison entries and one for migration entries. Fixes: 6c287605fd56 ("mm: remember exclusively mapped anonymous pages with PG_anon_exclusive") Signed-off-by: Naoya Horiguchi Cc: # 5.18 Reviewed-by: Miaohe Lin Reviewed-by: Mike Kravetz Reviewed-by: Muchun Song --- mm/hugetlb.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c538278170a2..f59f43c06601 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4784,8 +4784,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, * sharing with another vma. */ ; - } else if (unlikely(is_hugetlb_entry_migration(entry) || - is_hugetlb_entry_hwpoisoned(entry))) { + } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { + bool uffd_wp = huge_pte_uffd_wp(entry); + + if (!userfaultfd_wp(dst_vma) && uffd_wp) + entry = huge_pte_clear_uffd_wp(entry); + set_huge_swap_pte_at(dst, addr, dst_pte, entry, sz); + } else if (unlikely(is_hugetlb_entry_migration(entry))) { swp_entry_t swp_entry = pte_to_swp_entry(entry); bool uffd_wp = huge_pte_uffd_wp(entry); From patchwork Thu Jun 23 23:51:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BEEEC43334 for ; Thu, 23 Jun 2022 23:52:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 164458E01A8; Thu, 23 Jun 2022 19:52:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 114918E01A1; Thu, 23 Jun 2022 19:52:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1FF18E01A8; Thu, 23 Jun 2022 19:52:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E62428E01A1 for ; Thu, 23 Jun 2022 19:52:38 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B389A204B0 for ; Thu, 23 Jun 2022 23:52:38 +0000 (UTC) X-FDA: 79611152796.01.A489A82 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf22.hostedemail.com (Postfix) with ESMTP id 523BCC000D for ; Thu, 23 Jun 2022 23:52:38 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id go6so1171899pjb.0 for ; Thu, 23 Jun 2022 16:52:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gLXI0bFmtdpL1fm2aZ9vhvRpJm3p+mQNTKIlYsIrOjU=; b=n3EpWNne89DpkqcHeRXrJ15yy+IykIai7ornumZPTFP3oBklmi1A9IfZR2B/qtHRBE fEhmAFan4uUC7OrYWIDjV7OSRH6ALcWMbKnDZSnGBAIPyEKvGE2GD8iLzFiQWbhjOx8P IFNuSd9GajJZepS7jAVhRX8+KQZnAlpgD4YGn66q6n+VtWWRkfFQljjadM7UsZJhdNcu W5am1TruDf29uTRDkrxWzSbMs3OV9pweR6PMVYdkYTxMG98IlJmK7MRM3Pe2tLTOpxMP w9fq/O2WZelM6ft+W/kTXDjoHIMwggETk7IWGuE//bekvG8z5qhjUlCrGxuudfCbY2gV /lcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gLXI0bFmtdpL1fm2aZ9vhvRpJm3p+mQNTKIlYsIrOjU=; b=YedezYLqQC6HIXGTge0WRmXo7LYpTxbiZ7DWdrohGP3PG6u2pXGmnKBSCkaM9HNgZ5 TicMCpwdFZLEfmcxvICbCHMeGEYbONhdD0FUunMUkXUnQ1yr/a/TvDn5mgeE6NdJ5jSo 56HIHpMShyTSLd++w//dET1Q4cTtMficEx7nDPVXBJQZtv4Vkv1FR/QbDE3Nfb2YT3Bg K0Da9L0QwGXKdYndFL4aNxBfefP3flNT2duhX7C7suleeIxiKAtvVNoZinO3tTjnvp5E ljvYScsD1jP3cl2lDCH8dTvNR//LBJ1dKEdvHNaF2TYerYojgZftIGwvIel9WWuonvTx tP/A== X-Gm-Message-State: AJIora/i0QU+u4i9jMtii/liXTA9ao/NmDtneZDjRpfiPc88sudoq+fX 3U5hqVHlDdcFBJB0UdnrZIhntvlasQ== X-Google-Smtp-Source: AGRyM1swwZN4zwkDXsQnlRX7chaljh2k4qwXaQviyOxNxdAje8fNl06jiLeDO1ZeRVZ07VAF4OC9+g== X-Received: by 2002:a17:903:2350:b0:16a:4246:f1b6 with SMTP id c16-20020a170903235000b0016a4246f1b6mr12420841plh.35.1656028357207; Thu, 23 Jun 2022 16:52:37 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:36 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 3/9] mm/hugetlb: make pud_huge() and huge_pud() aware of non-present pud entry Date: Fri, 24 Jun 2022 08:51:47 +0900 Message-Id: <20220623235153.2623702-4-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=n3EpWNne; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028358; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gLXI0bFmtdpL1fm2aZ9vhvRpJm3p+mQNTKIlYsIrOjU=; b=QrrK1kV7xfODK1M0eReCLThoBUrBg3tKfN8XVnxFC4+0rCmLoe2cW1o3q7mQpq74maFZFW mB/ZfxCxkMzXq7E82+vAXs+MR5MjXFJekELiUQnjsl2n0rexwuF+Dhd1H38/ZyNQOAS69s lDa619G2gIZGJsJmJeikeLJ/knU95Ug= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028358; a=rsa-sha256; cv=none; b=dUV7lyTFU2dEDfxizoI3chje/c98zdo/emCgNvnrMSjJEPbK9L9lB/1G6GH/i3zm0/iYyX 6X1yVpTAmHeg9JcDfT9zbcf9OGsyPZeDQicMMEbMC45oyc7jYty1RavgoOeZayH8+Wg7Dj hu7vACCFQL+TbLrr2DKKCqbKtp9nzVg= X-Stat-Signature: xtootxmrnnddtafd89huk79h15bic78t X-Rspamd-Queue-Id: 523BCC000D X-Rspam-User: Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=n3EpWNne; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-Rspamd-Server: rspam12 X-HE-Tag: 1656028358-204362 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi follow_pud_mask() does not support non-present pud entry now. As long as I tested on x86_64 server, follow_pud_mask() still simply returns no_page_table() for non-present_pud_entry() due to pud_bad(), so no severe user-visible effect should happen. But generally we should call follow_huge_pud() for non-present pud entry for 1GB hugetlb page. Update pud_huge() and huge_pud() to handle non-present pud entries. The changes are similar to previous works for pud entries commit e66f17ff7177 ("mm/hugetlb: take page table lock in follow_huge_pmd()") and commit cbef8478bee5 ("mm/hugetlb: pmd_huge() returns true for non-present hugepage"). Signed-off-by: Naoya Horiguchi --- arch/x86/mm/hugetlbpage.c | 3 ++- mm/hugetlb.c | 26 +++++++++++++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index a0d023cb4292..5fb86fb49ba8 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -70,7 +70,8 @@ int pmd_huge(pmd_t pmd) int pud_huge(pud_t pud) { - return !!(pud_val(pud) & _PAGE_PSE); + return !pud_none(pud) && + (pud_val(pud) & (_PAGE_PRESENT|_PAGE_PSE)) != _PAGE_PRESENT; } #endif diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f59f43c06601..b7ae5f73f3b2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6946,10 +6946,34 @@ struct page * __weak follow_huge_pud(struct mm_struct *mm, unsigned long address, pud_t *pud, int flags) { + struct page *page = NULL; + spinlock_t *ptl; + pte_t pte; + if (flags & (FOLL_GET | FOLL_PIN)) return NULL; - return pte_page(*(pte_t *)pud) + ((address & ~PUD_MASK) >> PAGE_SHIFT); +retry: + ptl = huge_pte_lock(hstate_sizelog(PUD_SHIFT), mm, (pte_t *)pud); + if (!pud_huge(*pud)) + goto out; + pte = huge_ptep_get((pte_t *)pud); + if (pte_present(pte)) { + page = pud_page(*pud) + ((address & ~PUD_MASK) >> PAGE_SHIFT); + if (WARN_ON_ONCE(!try_grab_page(page, flags))) { + page = NULL; + goto out; + } + } else { + if (is_hugetlb_entry_migration(pte)) { + spin_unlock(ptl); + __migration_entry_wait(mm, (pte_t *)pud, ptl); + goto retry; + } + } +out: + spin_unlock(ptl); + return page; } struct page * __weak From patchwork Thu Jun 23 23:51:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 756C8C43334 for ; Thu, 23 Jun 2022 23:52:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 13D528E01A9; Thu, 23 Jun 2022 19:52:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EF198E01A1; Thu, 23 Jun 2022 19:52:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECFDF8E01A9; Thu, 23 Jun 2022 19:52:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DE60D8E01A1 for ; Thu, 23 Jun 2022 19:52:41 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id BE735120659 for ; Thu, 23 Jun 2022 23:52:41 +0000 (UTC) X-FDA: 79611152922.09.FD281FA Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf17.hostedemail.com (Postfix) with ESMTP id 6DE8F4001D for ; Thu, 23 Jun 2022 23:52:41 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id h9-20020a17090a648900b001ecb8596e43so1147442pjj.5 for ; Thu, 23 Jun 2022 16:52:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TAJYwYrd1tyojXaywF9IiUbYLQjEtu9FiUYYeWzF2Rw=; b=JeyytKAAkynnoiRRZ20ZhrJ8TRPHzLJyJkx/Jq//4aqdCxJhvcNvrEhL8eZxORpykD VfSuX/slc2Ow7ch6vlL5mPA4RAiwka4tR58+U/vdGSvfTJBsrseDamxhzL+KUgw/ckER aSLeIn1FWF3zIFKFytgoAkHWEL2TFzH5pA0CrWAmPOvjfRgA5Zayz/WioK5ADPF4lb5z Z6LpXej3Nn6tM3Vc7jG/GfmqEBBJHsr0FD/aliT8s0MshxIMI7gegBsXSja0MmR5lDzd R/cQT8tLU31XqinTg5uY/ZLNUIf4ltCU7MX4e2EH8RWQMHDEHZ1AZRkIojs/odgQwM5X z/rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TAJYwYrd1tyojXaywF9IiUbYLQjEtu9FiUYYeWzF2Rw=; b=Iw3FsKbyTqDlraI4iA4In1F7PWEl9kaMjHHbnyG6Snx+mY1RWUNm7XbBi8T+d/RCg2 g7z0UWloE1GM6rOoZoxh5FE+S47k/vukNRYvJFwsfUreq+cku8lqrRa+C3+8gOtE237U VwKtlb+cEnFsQihCuHu1mu3QZ/mS00mB1Iy2MbJ8MWqiO1Ib2V6zCEFHHo2jEFvwmhIU cBC+4VcpquADCuWCRV1YTLuOETJVTVnHebrXHmPbNE6OPr62hvSBU/0V0TCZAETmdPjP FXHcm4oBeLEA1pc9eA2CrzWwdr/Zgx3WwdcbSR9U34lVij4klAsANk3qUi5QWBWH7Aec aqqQ== X-Gm-Message-State: AJIora+gdnTva6H2v4vJnnalHzjQbRE1ApN3IyWz0u71Qf+ONIEbCbzk zt6tUB6WZhnSoLpTJg2SFOJFRVt7+w== X-Google-Smtp-Source: AGRyM1s8Qo2wZLu117yPTVjGPSDeSHkvrMoIz0nMpNtrh4bomIN/aKXR67bO9Nr5UDZnTcjQXHxSPw== X-Received: by 2002:a17:90b:1c87:b0:1ca:f4e:4fbe with SMTP id oo7-20020a17090b1c8700b001ca0f4e4fbemr540195pjb.159.1656028360332; Thu, 23 Jun 2022 16:52:40 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:40 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 4/9] mm, hwpoison, hugetlb: support saving mechanism of raw error pages Date: Fri, 24 Jun 2022 08:51:48 +0900 Message-Id: <20220623235153.2623702-5-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028361; a=rsa-sha256; cv=none; b=CCVAtmE5IP01V6xViuAKquauckOmTOJrfRjoHEo220gjaocQ8ZbEFonvfY17ZyKSHy/LHM 5jq2gBp3qfyaOBhQ3rwyMzHhPkHiYwmBjoXjIUyBPE6UPdiDfi0Asf9848cLF8L4gpDPDk uM3q/uabnBTCMmfQZ5PKtV81O1t5VVw= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=JeyytKAA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TAJYwYrd1tyojXaywF9IiUbYLQjEtu9FiUYYeWzF2Rw=; b=Jzhon4HhCM8+36H0hKoxSibXsLt00sC/IodZSV8LOYWNVhy1FWDIUlLOK5o2dZRK/Y3eqJ ktTS1uWUcYwzUOnfA1jEE6inTageS1n9cQC5OqJ1Xk9jVYwGNK1HKQc/7YHsjge0lNce8t xhXmUNKW0nIrORPOT+OZZXjRGfVlbGU= X-Stat-Signature: tg3jjshy1nzp31y9aprk3kztzy4we7em X-Rspamd-Queue-Id: 6DE8F4001D Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=JeyytKAA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1656028361-446690 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi When handling memory error on a hugetlb page, the error handler tries to dissolve and turn it into 4kB pages. If it's successfully dissolved, PageHWPoison flag is moved to the raw error page, so that's all right. However, dissolve sometimes fails, then the error page is left as hwpoisoned hugepage. It's useful if we can retry to dissolve it to save healthy pages, but that's not possible now because the information about where the raw error pages is lost. Use the private field of a few tail pages to keep that information. The code path of shrinking hugepage pool uses this info to try delayed dissolve. In order to remember multiple errors in a hugepage, a singly-linked list originated from SUBPAGE_INDEX_HWPOISON-th tail page is constructed. Only simple operations (adding an entry or clearing all) are required and the list is assumed not to be very long, so this simple data structure should be enough. If we failed to save raw error info, the hwpoison hugepage has errors on unknown subpage, then this new saving mechanism does not work any more, so disable saving new raw error info and freeing hwpoison hugepages. Signed-off-by: Naoya Horiguchi --- v1 -> v2: - support hwpoison hugepage with multiple errors, - moved the new interface functions to mm/memory-failure.c, - define additional subpage index SUBPAGE_INDEX_HWPOISON_UNRELIABLE, - stop freeing/dissolving hwpoison hugepages with unreliable raw error info, - drop hugetlb_clear_page_hwpoison() in dissolve_free_huge_page() because that's done in update_and_free_page(), - move setting/clearing PG_hwpoison flag to the new interfaces, - checking already hwpoisoned or not on a subpage basis. ChangeLog since previous post on 4/27: - fixed typo in patch description (by Miaohe) - fixed config value in #ifdef statement (by Miaohe) - added sentences about "multiple hwpoison pages" scenario in patch description Signed-off-by: Naoya Horiguchi --- include/linux/hugetlb.h | 13 ++++++ mm/hugetlb.c | 39 +++++++++-------- mm/memory-failure.c | 95 ++++++++++++++++++++++++++++++++++++++++- 3 files changed, 125 insertions(+), 22 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index e4cff27d1198..ac13c2022517 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -42,6 +42,10 @@ enum { SUBPAGE_INDEX_CGROUP, /* reuse page->private */ SUBPAGE_INDEX_CGROUP_RSVD, /* reuse page->private */ __MAX_CGROUP_SUBPAGE_INDEX = SUBPAGE_INDEX_CGROUP_RSVD, +#endif +#ifdef CONFIG_MEMORY_FAILURE + SUBPAGE_INDEX_HWPOISON, + SUBPAGE_INDEX_HWPOISON_UNRELIABLE, #endif __NR_USED_SUBPAGE, }; @@ -798,6 +802,15 @@ extern int dissolve_free_huge_page(struct page *page); extern int dissolve_free_huge_pages(unsigned long start_pfn, unsigned long end_pfn); +#ifdef CONFIG_MEMORY_FAILURE +extern int hugetlb_clear_page_hwpoison(struct page *hpage); +#else +static inline int hugetlb_clear_page_hwpoison(struct page *hpage) +{ + return 0; +} +#endif + #ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION #ifndef arch_hugetlb_migration_supported static inline bool arch_hugetlb_migration_supported(struct hstate *h) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index b7ae5f73f3b2..19ef427aa1e8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1541,17 +1541,15 @@ static void __update_and_free_page(struct hstate *h, struct page *page) if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) return; - if (hugetlb_vmemmap_alloc(h, page)) { - spin_lock_irq(&hugetlb_lock); - /* - * If we cannot allocate vmemmap pages, just refuse to free the - * page and put the page back on the hugetlb free list and treat - * as a surplus page. - */ - add_hugetlb_page(h, page, true); - spin_unlock_irq(&hugetlb_lock); - return; - } + if (hugetlb_vmemmap_alloc(h, page)) + goto fail; + + /* + * Move PageHWPoison flag from head page to the raw error pages, + * which makes any healthy subpages reusable. + */ + if (unlikely(PageHWPoison(page) && hugetlb_clear_page_hwpoison(page))) + goto fail; for (i = 0; i < pages_per_huge_page(h); i++, subpage = mem_map_next(subpage, page, i)) { @@ -1572,6 +1570,16 @@ static void __update_and_free_page(struct hstate *h, struct page *page) } else { __free_pages(page, huge_page_order(h)); } + return; +fail: + spin_lock_irq(&hugetlb_lock); + /* + * If we cannot allocate vmemmap pages or cannot identify raw hwpoison + * subpages reliably, just refuse to free the page and put the page + * back on the hugetlb free list and treat as a surplus page. + */ + add_hugetlb_page(h, page, true); + spin_unlock_irq(&hugetlb_lock); } /* @@ -2115,15 +2123,6 @@ int dissolve_free_huge_page(struct page *page) */ rc = hugetlb_vmemmap_alloc(h, head); if (!rc) { - /* - * Move PageHWPoison flag from head page to the raw - * error page, which makes any subpages rather than - * the error page reusable. - */ - if (PageHWPoison(head) && page != head) { - SetPageHWPoison(page); - ClearPageHWPoison(head); - } update_and_free_page(h, head, false); } else { spin_lock_irq(&hugetlb_lock); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index fb3feb1f363e..af571fa6f2af 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1499,6 +1499,97 @@ static int try_to_split_thp_page(struct page *page, const char *msg) } #ifdef CONFIG_HUGETLB_PAGE +/* + * Struct raw_hwp_page represents information about "raw error page", + * constructing singly linked list originated from ->private field of + * SUBPAGE_INDEX_HWPOISON-th tail page. + */ +struct raw_hwp_page { + struct llist_node node; + struct page *page; +}; + +static inline struct llist_head *raw_hwp_list_head(struct page *hpage) +{ + return (struct llist_head *)&page_private(hpage + SUBPAGE_INDEX_HWPOISON); +} + +static inline int raw_hwp_unreliable(struct page *hpage) +{ + return page_private(hpage + SUBPAGE_INDEX_HWPOISON_UNRELIABLE); +} + +static inline void set_raw_hwp_unreliable(struct page *hpage) +{ + set_page_private(hpage + SUBPAGE_INDEX_HWPOISON_UNRELIABLE, 1); +} + +/* + * about race consideration + */ +static inline int hugetlb_set_page_hwpoison(struct page *hpage, + struct page *page) +{ + struct llist_head *head; + struct raw_hwp_page *raw_hwp; + struct llist_node *t, *tnode; + int ret; + + /* + * Once the hwpoison hugepage has lost reliable raw error info, + * there is little mean to keep additional error info precisely, + * so skip to add additional raw error info. + */ + if (raw_hwp_unreliable(hpage)) + return -EHWPOISON; + head = raw_hwp_list_head(hpage); + llist_for_each_safe(tnode, t, head->first) { + struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); + + if (p->page == page) + return -EHWPOISON; + } + + ret = TestSetPageHWPoison(hpage) ? -EHWPOISON : 0; + /* the first error event will be counted in action_result(). */ + if (ret) + num_poisoned_pages_inc(); + + raw_hwp = kmalloc(sizeof(struct raw_hwp_page), GFP_KERNEL); + if (raw_hwp) { + raw_hwp->page = page; + llist_add(&raw_hwp->node, head); + } else { + /* + * Failed to save raw error info. We no longer trace all + * hwpoisoned subpages, and we need refuse to free/dissolve + * this hwpoisoned hugepage. + */ + set_raw_hwp_unreliable(hpage); + return ret; + } + return ret; +} + +inline int hugetlb_clear_page_hwpoison(struct page *hpage) +{ + struct llist_head *head; + struct llist_node *t, *tnode; + + if (raw_hwp_unreliable(hpage)) + return -EBUSY; + ClearPageHWPoison(hpage); + head = raw_hwp_list_head(hpage); + llist_for_each_safe(tnode, t, head->first) { + struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); + + SetPageHWPoison(p->page); + kfree(p); + } + llist_del_all(head); + return 0; +} + /* * Called from hugetlb code with hugetlb_lock held. * @@ -1533,7 +1624,7 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) goto out; } - if (TestSetPageHWPoison(head)) { + if (hugetlb_set_page_hwpoison(head, page)) { ret = -EHWPOISON; goto out; } @@ -1585,7 +1676,7 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb lock_page(head); if (hwpoison_filter(p)) { - ClearPageHWPoison(head); + hugetlb_clear_page_hwpoison(head); res = -EOPNOTSUPP; goto out; } From patchwork Thu Jun 23 23:51:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83999C43334 for ; Thu, 23 Jun 2022 23:52:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 24F2A8E01AA; Thu, 23 Jun 2022 19:52:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1FEEF8E01A1; Thu, 23 Jun 2022 19:52:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C6E88E01AA; Thu, 23 Jun 2022 19:52:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F2A8D8E01A1 for ; Thu, 23 Jun 2022 19:52:44 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D1C2820762 for ; Thu, 23 Jun 2022 23:52:44 +0000 (UTC) X-FDA: 79611153048.10.8369598 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by imf19.hostedemail.com (Postfix) with ESMTP id 866941A001F for ; Thu, 23 Jun 2022 23:52:44 +0000 (UTC) Received: by mail-pj1-f42.google.com with SMTP id i8-20020a17090aee8800b001ecc929d14dso4585561pjz.0 for ; Thu, 23 Jun 2022 16:52:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9qDM1SEwhJHNMkiAW+9hnn1MZNaGCub7B2p3sdAevGo=; b=a7P7HnrlqqsSBAv8Rh9l5P0qKEPodg3DsKsYr1Ea4M2Kn+QsYmKyxzF6uMI7LIyD90 D7dQ6KAxhdboYEDc65+ew8MRy8m7dAcgAw/XN9mFluukHhigzZK9AlkGJoXVhUYCDYxB iLNdAr+Lt1+Hhj13KaJVx531dr/ndkA5+wn+Xx0ZMBms/VzRGeN93f23HV3Fivmq0x4S wekPTJ7XFlXQdG1L1/GEkg4Y9ZXaUdt5cHqskGHTeZ1REkKnnlA7yUZkP2sRj28BTcsr 1SpvYWYgizD/bL6oe+cCC39Of/ccsD5GQvut0an6oon8bsbqSxFnCig0gv8wbdTZOacr re2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9qDM1SEwhJHNMkiAW+9hnn1MZNaGCub7B2p3sdAevGo=; b=cZhur1u5LCirxWu2e0EDCGpEbxFHVXpdkiZAm36XQ//JlNTb5M+5bCQZ5liE5nmjLU k9Xs6gWx73qWQl1ginxV/lwwyi2CQuMCmhj+CyUjo2/UsA8JzTsUgJuQ/pqQouTAhEdH cXZ0ZJBxZJno8/9Gpi6WKej4OLKR2MldZl0craAd90ulgjaPasZ/f3A1ejDGd2Jj3gYe r48B4jJO8P2zgcBlZxYwrsFM+pHdEvmNvaK4/C5ns00EtcEI5QTf2VT1j1NgrXqttWlm jz59hbLGvTt9BrRgoeWEN/UZAV/YyAeObTBiC1/h+XP9XOmWjLXaAb8PtJBY+69xGzRo kc8Q== X-Gm-Message-State: AJIora+JsKqhgQpLWEjz2zQMnS/dmeQtz3QS2FG/jMsNUkR4cItE75WY BOI2Vy8h4kq+t5dw2dMl5KbbQFSVxA== X-Google-Smtp-Source: AGRyM1to5MI42NBV7wBN2fu1N3JmUZLbmtHx8xzIu63b/rMAvK85lhxS+WOmvPyJdXJd+RprgZCMnw== X-Received: by 2002:a17:90a:1588:b0:1e0:a45c:5c1 with SMTP id m8-20020a17090a158800b001e0a45c05c1mr550853pja.65.1656028363425; Thu, 23 Jun 2022 16:52:43 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:43 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 5/9] mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage Date: Fri, 24 Jun 2022 08:51:49 +0900 Message-Id: <20220623235153.2623702-6-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=a7P7Hnrl; spf=pass (imf19.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028364; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9qDM1SEwhJHNMkiAW+9hnn1MZNaGCub7B2p3sdAevGo=; b=vacpEnJViJ8TYVdwew0HTHVgXSSNwd5D2RcDwkadr2fF2XRpiYwlGXb939n936kKN/2ryn tfN9QTjkaUIayb2CtnmmHOnAme3J+RF03HFTCJh+UOs4gFUP9rCimVO7p9Hi8yelohyULb GtlPDGzpbI0ULIgIdQ11NPdA0HCJOEo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028364; a=rsa-sha256; cv=none; b=h6zqHZ1jGPa/mQWe/LdDWVh3YMUV1aUlgzjPVTpfmQ26kWoHc5WYyuX0k5djxoCebpFdYd aB2jFr/ZIn7Vo8wyTzETVTT/wMdbpM65DhLVutxEh6il5zN7Jbx15tHw9GyT+C0PUJbyV9 IFFxpdnteRs+5zT8gCol6RvQhYWtJu0= X-Stat-Signature: tywmq88pxiz49ut3nyd51w8cbge59j5n X-Rspamd-Queue-Id: 866941A001F X-Rspam-User: Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=a7P7Hnrl; spf=pass (imf19.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-HE-Tag: 1656028364-260869 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Raw error info list needs to be removed when hwpoisoned hugetlb is unpoisioned. And unpoison handler needs to know how many errors there are in the target hugepage. So add them. Signed-off-by: Naoya Horiguchi --- include/linux/swapops.h | 9 +++++++++ mm/memory-failure.c | 34 +++++++++++++++++++++++++++------- 2 files changed, 36 insertions(+), 7 deletions(-) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 9584c2e1c54a..ad62776ee99c 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -494,6 +494,11 @@ static inline void num_poisoned_pages_dec(void) atomic_long_dec(&num_poisoned_pages); } +static inline void num_poisoned_pages_sub(long i) +{ + atomic_long_sub(i, &num_poisoned_pages); +} + #else static inline swp_entry_t make_hwpoison_entry(struct page *page) @@ -514,6 +519,10 @@ static inline struct page *hwpoison_entry_to_page(swp_entry_t entry) static inline void num_poisoned_pages_inc(void) { } + +static inline void num_poisoned_pages_sub(long i) +{ +} #endif static inline int non_swap_entry(swp_entry_t entry) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index af571fa6f2af..6005e953d011 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1571,23 +1571,34 @@ static inline int hugetlb_set_page_hwpoison(struct page *hpage, return ret; } -inline int hugetlb_clear_page_hwpoison(struct page *hpage) +static inline long free_raw_hwp_pages(struct page *hpage, bool move_flag) { struct llist_head *head; struct llist_node *t, *tnode; + long count = 0; - if (raw_hwp_unreliable(hpage)) - return -EBUSY; - ClearPageHWPoison(hpage); head = raw_hwp_list_head(hpage); llist_for_each_safe(tnode, t, head->first) { struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); - SetPageHWPoison(p->page); + if (move_flag) + SetPageHWPoison(p->page); kfree(p); + count++; } llist_del_all(head); - return 0; + return count; +} + +inline int hugetlb_clear_page_hwpoison(struct page *hpage) +{ + int ret = -EBUSY; + + if (raw_hwp_unreliable(hpage)) + return ret; + ret = !TestClearPageHWPoison(hpage); + free_raw_hwp_pages(hpage, true); + return ret; } /* @@ -1729,6 +1740,10 @@ static inline int try_memory_failure_hugetlb(unsigned long pfn, int flags, int * { return 0; } + +static inline void free_raw_hwp_pages(struct page *hpage, bool move_flag) +{ +} #endif static int memory_failure_dev_pagemap(unsigned long pfn, int flags, @@ -2188,6 +2203,7 @@ int unpoison_memory(unsigned long pfn) struct page *p; int ret = -EBUSY; int freeit = 0; + long count = 1; static DEFINE_RATELIMIT_STATE(unpoison_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -2235,6 +2251,8 @@ int unpoison_memory(unsigned long pfn) ret = get_hwpoison_page(p, MF_UNPOISON); if (!ret) { + if (PageHuge(p)) + count = free_raw_hwp_pages(page, false); ret = TestClearPageHWPoison(page) ? 0 : -EBUSY; } else if (ret < 0) { if (ret == -EHWPOISON) { @@ -2243,6 +2261,8 @@ int unpoison_memory(unsigned long pfn) unpoison_pr_info("Unpoison: failed to grab page %#lx\n", pfn, &unpoison_rs); } else { + if (PageHuge(p)) + count = free_raw_hwp_pages(page, false); freeit = !!TestClearPageHWPoison(p); put_page(page); @@ -2255,7 +2275,7 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { - num_poisoned_pages_dec(); + num_poisoned_pages_sub(count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); } From patchwork Thu Jun 23 23:51:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B7D3C433EF for ; Thu, 23 Jun 2022 23:52:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B4F9F8E01AB; Thu, 23 Jun 2022 19:52:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFFD28E01A1; Thu, 23 Jun 2022 19:52:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C7E88E01AB; Thu, 23 Jun 2022 19:52:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8E8B98E01A1 for ; Thu, 23 Jun 2022 19:52:48 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 72D49607CE for ; Thu, 23 Jun 2022 23:52:48 +0000 (UTC) X-FDA: 79611153216.07.BA25061 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf13.hostedemail.com (Postfix) with ESMTP id BCEA72001C for ; Thu, 23 Jun 2022 23:52:47 +0000 (UTC) Received: by mail-pl1-f171.google.com with SMTP id o18so634818plg.2 for ; Thu, 23 Jun 2022 16:52:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KvUi4xXUaw0Gsdez/sII2VYp9q4T8l1odrgep5/r9aE=; b=b71HZZHM2aY5ylc8U6Uu7sxHM5Q4Dz7cL/mOReiZJuAaeu2rHNu+hnQmQAv38dDmwP pOKIk5yY/X85zQfa1Ys2n7GhIPD0SVPxsFL/z9/OGvRegbXwb/U4EHvDpjg7cQ1S0pyP KWtLwypFgkCY/db06rZ/D2urozJab3xD2VJxRbQ0no53NC4CzSsNbu/G6NHqRh9A8X+F 8OTJbDheCebOpKua3A0uEj2qHKjRhFb2+wCy2g910KB5HQ2VrIMCFd/2r8H0V9axz7PU b2cZtNIaDaxWhsGU+tHFIrLWR/bKzQEwuNlwc7ldCcwha2zy4xep6e7bfU6vHKhpsNqV 7Cvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KvUi4xXUaw0Gsdez/sII2VYp9q4T8l1odrgep5/r9aE=; b=k7/h6bYnzuAGyY5jluzm+niCx1e8yaQ5BKryo2dVCAOGNzg9U40l3PnYpYIlXbrZsL K+CP6TMmIeg5uXS6Z2w3sHC3rbOjJxuOAF4DzfmGOyMY5pmE21yamm0y/Myc/mDndH2h rs1JLX+TBL2aqScb7fRupSsOYdtfT6lWhKbysqxH3N4Whz6Nhv1bydXyRyfFkCvlAleE D01Glz1XgiGM8Wxer+3NT0bfQXCSvE/GoAHpevbmaovcNGkxSiBNnScONt+llqaGwJDf N6aWgONQI3wH1qb/drzx0/P6nTSbA3+cJbklNsFRMzQAV0b0OFFgjt7cZArEx4CgC9xS SOhA== X-Gm-Message-State: AJIora+ngoVhUJlziYDiZdiIJmOJMpa4SieF4oYIM4SaPG+vxw5cMxnt XLon35VrXTj4rbkqhpSzi34L5b0qQg== X-Google-Smtp-Source: AGRyM1u3RWXKYbvMbZlXJV9HgWdbCWIpY9vHZiP0MBdnlmG8yhoNWv12HEJym6YydHWar1VwhEIpYw== X-Received: by 2002:a17:902:ebc1:b0:168:fd13:8adc with SMTP id p1-20020a170902ebc100b00168fd138adcmr41512651plg.161.1656028366487; Thu, 23 Jun 2022 16:52:46 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:46 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 6/9] mm, hwpoison: set PG_hwpoison for busy hugetlb pages Date: Fri, 24 Jun 2022 08:51:50 +0900 Message-Id: <20220623235153.2623702-7-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=b71HZZHM; spf=pass (imf13.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028367; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KvUi4xXUaw0Gsdez/sII2VYp9q4T8l1odrgep5/r9aE=; b=Ncg+CeelNr5x4wiHgv3kWKyCq0zQ6LCxCJYhP8rC46cE02bk9op+VLYWhVAgLa+l/v8oyI Q8VNGO83pplAswINV4uFuhz/oTwQOuDbcTBPIywjSCP5FNqzWo34deJwp2IAVwdqKpKU65 wCdZc9WMFRtDJX+dwnRSb8a+QhUQYHY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028367; a=rsa-sha256; cv=none; b=D7hNbmPeR4HFV0Mygu4CTe2j3++84MsjRHznc3qco/+Va/ElG7rAdIjO+eEco8HkaYWwKM Mx42A8QJraVjqysIh+MptU36Xj+e2N2Y/I3NMD8C3PLxMG52DHzYGW8B0VlsVevqh/q80q IOV2Q/GGl1AAG0m0uV0vi9HD8X2fMfQ= X-Stat-Signature: q73s4g7unr8iinr13kb1jdscpayxh8nq X-Rspamd-Queue-Id: BCEA72001C X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=b71HZZHM; spf=pass (imf13.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-HE-Tag: 1656028367-677854 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi If memory_failure() fails to grab page refcount on a hugetlb page because it's busy, it returns without setting PG_hwpoison on it. This not only loses a chance of error containment, but breaks the rule that action_result() should be called only when memory_failure() do any of handling work (even if that's just setting PG_hwpoison). This inconsistency could harm code maintainability. So set PG_hwpoison and call hugetlb_set_page_hwpoison() for such a case. Fixes: 405ce051236c ("mm/hwpoison: fix race between hugetlb free/demotion and memory_failure_hugetlb()") Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- include/linux/mm.h | 1 + mm/memory-failure.c | 8 ++++---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 4346e51484ba..044dc5a2e361 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3233,6 +3233,7 @@ enum mf_flags { MF_SOFT_OFFLINE = 1 << 3, MF_UNPOISON = 1 << 4, MF_SW_SIMULATED = 1 << 5, + MF_NO_RETRY = 1 << 6, }; extern int memory_failure(unsigned long pfn, int flags); extern void memory_failure_queue(unsigned long pfn, int flags); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 6005e953d011..ce045d0d6115 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1632,7 +1632,8 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) count_increased = true; } else { ret = -EBUSY; - goto out; + if (!(flags & MF_NO_RETRY)) + goto out; } if (hugetlb_set_page_hwpoison(head, page)) { @@ -1659,7 +1660,6 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb struct page *p = pfn_to_page(pfn); struct page *head; unsigned long page_flags; - bool retry = true; *hugetlb = 1; retry: @@ -1675,8 +1675,8 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb } return res; } else if (res == -EBUSY) { - if (retry) { - retry = false; + if (!(flags & MF_NO_RETRY)) { + flags |= MF_NO_RETRY; goto retry; } action_result(pfn, MF_MSG_UNKNOWN, MF_IGNORED); From patchwork Thu Jun 23 23:51:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB9A2C433EF for ; Thu, 23 Jun 2022 23:52:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 646FB8E01AC; Thu, 23 Jun 2022 19:52:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61D378E01A1; Thu, 23 Jun 2022 19:52:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50D228E01AC; Thu, 23 Jun 2022 19:52:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4208C8E01A1 for ; Thu, 23 Jun 2022 19:52:51 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 22261604BB for ; Thu, 23 Jun 2022 23:52:51 +0000 (UTC) X-FDA: 79611153342.15.D8188DE Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf23.hostedemail.com (Postfix) with ESMTP id AC1C614002E for ; Thu, 23 Jun 2022 23:52:50 +0000 (UTC) Received: by mail-pf1-f179.google.com with SMTP id bo5so1058017pfb.4 for ; Thu, 23 Jun 2022 16:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5pQfND038h5QWcn78qM/I1ai1qmbxoygS2JmZLRC1mQ=; b=iXRbqsMPfof+NUltQHH89pGwAzb42H/dstNMxQgGGt6Yt//W8LgTGOTG0OaoSTeNRz dZWUB59vT4SDDN3iALIsxg16784gK20+rZPhBd+mZNDSqMB9O7U2WxsqjohKmBB6pXUG nibweuhhc5VKaNupCNP0N0YPwZxrsIyX/Bb9vW1e68rAyv5FmPBsYHd9kQFfKyW1Ez5f w5un26afpOlcnOq5acBDDRhfwfXCkdcFPIehGQOKnQhnUuw02P3Yd7V2Oqo0X9yKyjgn RWPTgZnrtjJ7VlJaB/hzIuIdKb1hqY3+8AgAQyB9hZ3NMamUhCbmytLPWpfDRAdW+V2/ eftw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5pQfND038h5QWcn78qM/I1ai1qmbxoygS2JmZLRC1mQ=; b=cMe8aKwM1X2+OQhlLXaeCM1A/snCiF4ukNHvbtPcIB7dpVvMESvt84Q/XblFVGaG7c 3oWZBFTWkFCOT6DJ+mwmkkjlEeuZlH4RIotDMW2FSWd4IwX+0oURbYlZOmfUb1mebJ6v GrtnrxvuBuzxOx84yiVk7oXU/QcvqbVFqtwYwV5UgXOocBGJkADECCeAMS3h2+CT7ntV DkuwveI424FtjDpsSzQUl/3rvQpYCtrxkkZvB6z3+tUUvmJgDt9azLIL1LHn/XZ3hmYg WpKq528OgU3BPF4FzOABzrnn2xbzx+cn7KinyBNK2egYYrQPj3KBJe8rPgdkNbQG1WL5 K+UQ== X-Gm-Message-State: AJIora/bj29MXYIyhahAvhYa4Qep6YCZ28RLBmhvc2lQ2MOXRKfJwEcE bkkCvRcHsbIuuGzW/lcpsepXCGYZ1w== X-Google-Smtp-Source: AGRyM1s96G2FFH66JXa8bc7eH4qd+fIDxeMhCkK8flgwP1JhK2XMzt6hxEJTmkotzQNPWlNmHQYnig== X-Received: by 2002:a63:7b5c:0:b0:40d:684:b760 with SMTP id k28-20020a637b5c000000b0040d0684b760mr9535270pgn.323.1656028369640; Thu, 23 Jun 2022 16:52:49 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:49 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 7/9] mm, hwpoison: make __page_handle_poison returns int Date: Fri, 24 Jun 2022 08:51:51 +0900 Message-Id: <20220623235153.2623702-8-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028370; a=rsa-sha256; cv=none; b=DtnaTEtEmGcLADe4ayqMXNFWI9NsB660Ov1uB/3ZCoi7NbW3IvUFPb+uRS37JHabNuh7VP K0DJ2nWHSbdMjgRqKf+h9VUCBRJcJgmIUzoruRL9xxAEXqtkSxFjeYyP352+erZu60Zvhw PIvxoA0X/st5uB47lOtmkKFVLNb3ri4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028370; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5pQfND038h5QWcn78qM/I1ai1qmbxoygS2JmZLRC1mQ=; b=Y5um0eEcIFkOVB57S5C6Pogj308h9FGOV04VABAIIAGDef9y4+CMg+gwJRt4YiHDEYJHD2 t5cx08G9chqozX3GOXt2JTm0YbuQRdxQB7WUinXqjsdfjhvyCSe+Uf/zvj23aej7pWHC6E cbVUJ/cj8G6TJ3F5+vW4ggPOZmiRz3o= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=iXRbqsMP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-Stat-Signature: 1di557orptjgjhnjjw5eort5c6ofkd5j X-Rspamd-Queue-Id: AC1C614002E Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=iXRbqsMP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1656028370-498680 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi __page_handle_poison() returns bool that shows whether take_page_off_buddy() has passed or not now. But we will want to distinguish another case of "dissolve has passed but taking off failed" by its return value. So change the type of the return value. No functional change. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- mm/memory-failure.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ce045d0d6115..db85f644a1e3 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -71,7 +71,13 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; -static bool __page_handle_poison(struct page *page) +/* + * Return values: + * 1: the page is dissolved (if needed) and taken off from buddy, + * 0: the page is dissolved (if needed) and not taken off from buddy, + * < 0: failed to dissolve. + */ +static int __page_handle_poison(struct page *page) { int ret; @@ -81,7 +87,7 @@ static bool __page_handle_poison(struct page *page) ret = take_page_off_buddy(page); zone_pcp_enable(page_zone(page)); - return ret > 0; + return ret; } static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, bool release) @@ -91,7 +97,7 @@ static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, boo * Doing this check for free pages is also fine since dissolve_free_huge_page * returns 0 for non-hugetlb pages as well. */ - if (!__page_handle_poison(page)) + if (__page_handle_poison(page) <= 0) /* * We could fail to take off the target page from buddy * for example due to racy page allocation, but that's @@ -1048,7 +1054,7 @@ static int me_huge_page(struct page_state *ps, struct page *p) * subpages. */ put_page(hpage); - if (__page_handle_poison(p)) { + if (__page_handle_poison(p) > 0) { page_ref_inc(p); res = MF_RECOVERED; } @@ -1698,8 +1704,7 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb */ if (res == 0) { unlock_page(head); - res = MF_FAILED; - if (__page_handle_poison(p)) { + if (__page_handle_poison(p) > 0) { page_ref_inc(p); res = MF_RECOVERED; } From patchwork Thu Jun 23 23:51:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1778C433EF for ; Thu, 23 Jun 2022 23:52:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8DFE88E01AD; Thu, 23 Jun 2022 19:52:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 88EDE8E01A1; Thu, 23 Jun 2022 19:52:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 730418E01AD; Thu, 23 Jun 2022 19:52:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 653428E01A1 for ; Thu, 23 Jun 2022 19:52:54 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 3082C120575 for ; Thu, 23 Jun 2022 23:52:54 +0000 (UTC) X-FDA: 79611153468.25.B3048BA Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf06.hostedemail.com (Postfix) with ESMTP id DCD31180012 for ; Thu, 23 Jun 2022 23:52:53 +0000 (UTC) Received: by mail-pg1-f181.google.com with SMTP id g186so887193pgc.1 for ; Thu, 23 Jun 2022 16:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LG1rSzGEsPK4iv+N/lwp5PXQGNIYUtfM/aAuTYkg0vc=; b=kp5Fq94l6Q8wgGQCHb0XGJwN4XF5JHuSExD/vq+W4HulxPu4y7NpXpzk0sC4IbGOF0 M8KAGgZc5kObwch9yTED6BTWncP2yvMtbr+bH+KSPK3guq1ThRAzdHSa+W71rWfvYi5Y ysRgIgyiMWh4MKt7L42JrZc5NkKFssOV2e5P4ICRr7BACAKuoQ/eu+QswjxOjsF8Qh9L /9q19zgN4ONQWZgUIzp2zk9FPdJ7T/61qXbmDITGTyaGcabR3hZpeQjiZAfpcM4Id7Ms ZxiNLZtA8OmC0bfzSoEwTbnAiH5hXBWlOsZ/GgJaKy63TSCZUNEzfRNHl4QZ+uHeTTH/ Cbxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LG1rSzGEsPK4iv+N/lwp5PXQGNIYUtfM/aAuTYkg0vc=; b=AKxmLZhzoJ/bc3M+ixkv4nJCjFrxqwNeauajAEnMGulbp5KQGfbFIePRMNkc/ZqEc+ /STpcmDWDJyMryQQuM25BNKKVzyy/PJ0znwW3yz5kWYYeDS0xcDGT+5I/3pzANgUVyBV AYECCU+fIq33yVDkEw8teNbfcNmTikAX0sJHWNHkCAkjvMbzumQ8LB+dOci1tVwQ/KEd PnaAX79Cd9G6s6fVSyscOE4vQmJV1Xypyi+EPMze0CciXLvjPBc0N7gO6U0Xul0URqKb /Uu2DyfACsHZLlkoc6yqPIs0WTG1JPiopz+F83ImZz5pst8N8rOlfJc9bUfTZ9bUUbwq IRBA== X-Gm-Message-State: AJIora+X462VdsWImJiCLHVDERtAFblZQrIBFQThfauYlXp0tQ7O4INs uDFse6wkEmbvHutzBLJ2+KgsceBNdg== X-Google-Smtp-Source: AGRyM1tcKr8yX9/tffiNeAaqH/kIj/EK53FgF4ggMpoeeio6m/TIPEIsDaxtlmNYN+u9v8W4FmnXEg== X-Received: by 2002:a62:6410:0:b0:4f3:9654:266d with SMTP id y16-20020a626410000000b004f39654266dmr43077948pfb.59.1656028372725; Thu, 23 Jun 2022 16:52:52 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:52 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 8/9] mm, hwpoison: skip raw hwpoison page in freeing 1GB hugepage Date: Fri, 24 Jun 2022 08:51:52 +0900 Message-Id: <20220623235153.2623702-9-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028373; a=rsa-sha256; cv=none; b=RfyywpZ3yZl59Y+36bGZvia7F7Odf9/1aoFQyZkIm2U9y3kETX++niuIjkRhY4umDaNmgK ek0lpC5eHr28MrXml7OP6Ja0nH25xOI04RLHtQXe3bYO0UVFD54oExsOEUff3TQEKatV/c Ep++Rv+qdfH8KdF0w+YP8Uaiew+nolY= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kp5Fq94l; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028373; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LG1rSzGEsPK4iv+N/lwp5PXQGNIYUtfM/aAuTYkg0vc=; b=45m/NMjDvsAceEBMmv7qlNsFMsyciWS2M8WdjrkM7C6alX+nhwCoYaEQKWmOIBoB2fYYeS f8G6oCNTkJx5JdEqbxmuFIWUtUdN3EuYx47mSt5qVrF0JMQN+N7KlDt3Lj2+AH37TRH6ql NL5rXWKyHjyay2SWqz+LjhYixyyWBM4= X-Rspam-User: X-Stat-Signature: tzhqz8jb3nfyoq9qcki9othupt4yd6yj X-Rspamd-Queue-Id: DCD31180012 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kp5Fq94l; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com X-Rspamd-Server: rspam03 X-HE-Tag: 1656028373-551821 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Currently if memory_failure() (modified to remove blocking code with subsequent patch) is called on a page in some 1GB hugepage, memory error handling fails and the raw error page gets into leaked state. The impact is small in production systems (just leaked single 4kB page), but this limits the testability because unpoison doesn't work for it. We can no longer create 1GB hugepage on the 1GB physical address range with such leaked pages, that's not useful when testing on small systems. When a hwpoison page in a 1GB hugepage is handled, it's caught by the PageHWPoison check in free_pages_prepare() because the 1GB hugepage is broken down into raw error pages before coming to this point: if (unlikely(PageHWPoison(page)) && !order) { ... return false; } Then, the page is not sent to buddy and the page refcount is left 0. Originally this check is supposed to work when the error page is freed from page_handle_poison() (that is called from soft-offline), but now we are opening another path to call it, so the callers of __page_handle_poison() need to handle the case by considering the return value 0 as success. Then page refcount for hwpoison is properly incremented so unpoison works. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- mm/memory-failure.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index db85f644a1e3..fc7b83cb6468 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1046,7 +1046,6 @@ static int me_huge_page(struct page_state *ps, struct page *p) res = truncate_error_page(hpage, page_to_pfn(p), mapping); unlock_page(hpage); } else { - res = MF_FAILED; unlock_page(hpage); /* * migration entry prevents later access on error hugepage, @@ -1054,9 +1053,11 @@ static int me_huge_page(struct page_state *ps, struct page *p) * subpages. */ put_page(hpage); - if (__page_handle_poison(p) > 0) { + if (__page_handle_poison(p) >= 0) { page_ref_inc(p); res = MF_RECOVERED; + } else { + res = MF_FAILED; } } @@ -1704,9 +1705,11 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb */ if (res == 0) { unlock_page(head); - if (__page_handle_poison(p) > 0) { + if (__page_handle_poison(p) >= 0) { page_ref_inc(p); res = MF_RECOVERED; + } else { + res = MF_FAILED; } action_result(pfn, MF_MSG_FREE_HUGE, res); return res == MF_RECOVERED ? 0 : -EBUSY; From patchwork Thu Jun 23 23:51:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 12893263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D54C4C433EF for ; Thu, 23 Jun 2022 23:52:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A63D8E01AE; Thu, 23 Jun 2022 19:52:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7564C8E01A1; Thu, 23 Jun 2022 19:52:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61E7B8E01AE; Thu, 23 Jun 2022 19:52:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5504B8E01A1 for ; Thu, 23 Jun 2022 19:52:57 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 31AFD6040E for ; Thu, 23 Jun 2022 23:52:57 +0000 (UTC) X-FDA: 79611153594.21.BDAB20A Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf20.hostedemail.com (Postfix) with ESMTP id C71741C001D for ; Thu, 23 Jun 2022 23:52:56 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id g10-20020a17090a708a00b001ea8aadd42bso1217475pjk.0 for ; Thu, 23 Jun 2022 16:52:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xlh5+JHPCpuQ09KawzKkUOlMCn4aWXR34ra2vrdN9hI=; b=MEmORMw9CoAhwNYfHfOlrZjmyaV1kxLa2ly8Bz617AoWw6gyQfA+eYRxSScDsayqSE vEeb04MRVgwYo6YZU8ahEfefgc7Zo/NiMU+DYL4YDwHrcNbicueEUnbIqQFX5doIy1jm /kCAQATYrsmfNaVqrDVcb6DJfAGw87zSOMdMLh1lLF7YOPPH/73bvPg34HTyM7tpRGUQ nOjgsZK4t5kglXK6qoK3JAYx7AQQTcRuk8Mj17qhHxTNgnF+Y7GasXDcNgFynTQE9W9L 95RsE1CJ9sfNpVvHVI83dPlXVdq3OhKVdl2cpSil4MWjqDhLx+Kv/eTTwb9LdbB6ZrXn bNEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xlh5+JHPCpuQ09KawzKkUOlMCn4aWXR34ra2vrdN9hI=; b=4q9FPMm5Jm6KygvRFEuHMrk8TQ5u32NjGrU3dhoVNNYSnkH0PZQ28MZBjgYx0Dns0x RmA3p+GCxr40y0fGMZsbbtfXcPR/k92NsDTob55eIBReaLRmTurjCnq18g+dS7C+CVLZ kzjKP05QmeCcKrbrZXvqrezMfMW+SGi2pbVwfNYkqi2MPrQl2hPQMTWSCXZkoW2rtkNp smw/iYjCPRen/ibHBu6+TEjWjQzw0auDtJO7TNH2i3TN7kmE0B6kX1mkGjV2yuZ7yToj o9hvBZGNuNrWKUcRrvIiuS8/yG+75yMnRmJ8kMQaym8yt0T0WRskCV/9mrECfuQhqiUK micw== X-Gm-Message-State: AJIora/EA0FJnUslRNRgLB2V3xkiQtPeKPR2vkEpq579rXxShw27sx7Y 1Q9Tmp9IWbiX1sDivIc0Bz4kDbr0YQ== X-Google-Smtp-Source: AGRyM1soNngnpVcfxmXaErJFMeKvhtVFxNIlwB4ItZCs2wn5DW3s5YIJ6hdZc7kgW4lJHxcSAIlZDg== X-Received: by 2002:a17:90b:1808:b0:1ec:9559:3060 with SMTP id lw8-20020a17090b180800b001ec95593060mr542906pjb.163.1656028375788; Thu, 23 Jun 2022 16:52:55 -0700 (PDT) Received: from ik1-406-35019.vs.sakura.ne.jp (ik1-406-35019.vs.sakura.ne.jp. [153.127.16.23]) by smtp.gmail.com with ESMTPSA id r10-20020a170903020a00b00168eab11f67sm362571plh.94.2022.06.23.16.52.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 16:52:55 -0700 (PDT) From: Naoya Horiguchi X-Google-Original-From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , David Hildenbrand , Mike Kravetz , Miaohe Lin , Liu Shixin , Yang Shi , Oscar Salvador , Muchun Song , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v2 9/9] mm, hwpoison: enable memory error handling on 1GB hugepage Date: Fri, 24 Jun 2022 08:51:53 +0900 Message-Id: <20220623235153.2623702-10-naoya.horiguchi@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> References: <20220623235153.2623702-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MEmORMw9; spf=pass (imf20.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656028376; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xlh5+JHPCpuQ09KawzKkUOlMCn4aWXR34ra2vrdN9hI=; b=GXTe5FsT2r6i/pQB9+ar7DE0+FQrctbeUH2HioFhDvaW5Fyfm67uIG1oP/7ZceBJMRo00I 6SOAVijPRGJxIvN44moCKQklvcqMzqED3vqznXD0xm1YraSyS84V/1CHYJ0+mVixGiQFoI AZbvrhGoG3QgberBaKqq6oRhMkI+R8I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656028376; a=rsa-sha256; cv=none; b=HkJfrh19fJ0hFPkHlG9rNXfByqOj8p6qtZ5X7J3lIuCKjBcRk/S76wFKCFU1mOKlmDp5UC HpMQEgVkpEF7jKvd6sk/TG1ee78lHxTUaeeWIOY6fb360LFLXeFz99ECOGlgiZhpo2b17J nbxudYeLfQ1cH8/M+cUWEVctj+wMOHw= X-Stat-Signature: 1869metimhdms9xe1jfqgndgs4wrsw9x X-Rspamd-Queue-Id: C71741C001D X-Rspam-User: Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MEmORMw9; spf=pass (imf20.hostedemail.com: domain of nao.horiguchi@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=nao.horiguchi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-HE-Tag: 1656028376-937905 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Now error handling code is prepared, so remove the blocking code and enable memory error handling on 1GB hugepage. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- include/linux/mm.h | 1 - include/ras/ras_event.h | 1 - mm/memory-failure.c | 16 ---------------- 3 files changed, 18 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 044dc5a2e361..9d7e9b5a4d1d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3284,7 +3284,6 @@ enum mf_action_page_type { MF_MSG_DIFFERENT_COMPOUND, MF_MSG_HUGE, MF_MSG_FREE_HUGE, - MF_MSG_NON_PMD_HUGE, MF_MSG_UNMAP_FAILED, MF_MSG_DIRTY_SWAPCACHE, MF_MSG_CLEAN_SWAPCACHE, diff --git a/include/ras/ras_event.h b/include/ras/ras_event.h index d0337a41141c..cbd3ddd7c33d 100644 --- a/include/ras/ras_event.h +++ b/include/ras/ras_event.h @@ -360,7 +360,6 @@ TRACE_EVENT(aer_event, EM ( MF_MSG_DIFFERENT_COMPOUND, "different compound page after locking" ) \ EM ( MF_MSG_HUGE, "huge page" ) \ EM ( MF_MSG_FREE_HUGE, "free huge page" ) \ - EM ( MF_MSG_NON_PMD_HUGE, "non-pmd-sized huge page" ) \ EM ( MF_MSG_UNMAP_FAILED, "unmapping failed page" ) \ EM ( MF_MSG_DIRTY_SWAPCACHE, "dirty swapcache page" ) \ EM ( MF_MSG_CLEAN_SWAPCACHE, "clean swapcache page" ) \ diff --git a/mm/memory-failure.c b/mm/memory-failure.c index fc7b83cb6468..33521e059f7f 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -728,7 +728,6 @@ static const char * const action_page_types[] = { [MF_MSG_DIFFERENT_COMPOUND] = "different compound page after locking", [MF_MSG_HUGE] = "huge page", [MF_MSG_FREE_HUGE] = "free huge page", - [MF_MSG_NON_PMD_HUGE] = "non-pmd-sized huge page", [MF_MSG_UNMAP_FAILED] = "unmapping failed page", [MF_MSG_DIRTY_SWAPCACHE] = "dirty swapcache page", [MF_MSG_CLEAN_SWAPCACHE] = "clean swapcache page", @@ -1717,21 +1716,6 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb page_flags = head->flags; - /* - * TODO: hwpoison for pud-sized hugetlb doesn't work right now, so - * simply disable it. In order to make it work properly, we need - * make sure that: - * - conversion of a pud that maps an error hugetlb into hwpoison - * entry properly works, and - * - other mm code walking over page table is aware of pud-aligned - * hwpoison entries. - */ - if (huge_page_size(page_hstate(head)) > PMD_SIZE) { - action_result(pfn, MF_MSG_NON_PMD_HUGE, MF_IGNORED); - res = -EBUSY; - goto out; - } - if (!hwpoison_user_mappings(p, pfn, flags, head)) { action_result(pfn, MF_MSG_UNMAP_FAILED, MF_IGNORED); res = -EBUSY;