From patchwork Thu Jul 13 00:18:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13311064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33AECEB64DD for ; Thu, 13 Jul 2023 00:18:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42D168E000B; Wed, 12 Jul 2023 20:18:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 403EC8E0001; Wed, 12 Jul 2023 20:18:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A67C8E000B; Wed, 12 Jul 2023 20:18:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1D4778E0001 for ; Wed, 12 Jul 2023 20:18:42 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id DC553403D0 for ; Thu, 13 Jul 2023 00:18:41 +0000 (UTC) X-FDA: 81004677642.16.772D21D Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf12.hostedemail.com (Postfix) with ESMTP id 188D340009 for ; Thu, 13 Jul 2023 00:18:39 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=OYL0bSj4; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 33kKvZAgKCE0yxp5xDp2v33v0t.r310x29C-11zAprz.36v@flex--jiaqiyan.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=33kKvZAgKCE0yxp5xDp2v33v0t.r310x29C-11zAprz.36v@flex--jiaqiyan.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689207520; a=rsa-sha256; cv=none; b=fHUwMCl9zNZTxR+McmKx/WssmxheiTZ06mcJQFnCNbyAL+9TIRw6AL+rLFElc5IL8jpyMt edY4GqklZMcGL0QfE9RnsFXZ2RcGd+Qsyx40nOMFu95rqIGOz1htgfPl46zhWwWJ0RFI2a 5JABv9+zJYtZPbWpVqg5kSgHOuqxh1E= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=OYL0bSj4; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 33kKvZAgKCE0yxp5xDp2v33v0t.r310x29C-11zAprz.36v@flex--jiaqiyan.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=33kKvZAgKCE0yxp5xDp2v33v0t.r310x29C-11zAprz.36v@flex--jiaqiyan.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689207520; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8FNetjBnnWs4aWcoESXEPv5EQNod9M4H8ra/lwVlWng=; b=fhZVxCZLb9xhfOeVajonCeARUrxdWNPj60a9PseVsu6mFzqrtm1/NC5KQPFtnF/RHgfq9E zKdpzZfJ73xxrJj8JZ411zGE2aL/H8drjCXJCxT8L09T9A3Uc2wlLsh/NVWOOhX4UoZRzJ zxSZZ5BMZoDXxyWOewqekQ1xmz26h5M= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-262d8993033so20281a91.0 for ; Wed, 12 Jul 2023 17:18:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689207519; x=1691799519; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8FNetjBnnWs4aWcoESXEPv5EQNod9M4H8ra/lwVlWng=; b=OYL0bSj4G5DjZWuCKYJh1mgYYyb7btYF2LGzKY9Bk+oN0orukveShF4mERnDT12wOV 59jdagioq6JUZlITmhCB22KSsxcpFF6nRLpZZr2dAeg1q60Px50PCZI4BiVQB4x4I1Nv D7UtKoG3pLXisajm6f5fiIvw4mAM/GvHvlla0HmYIzBfJMVspK8P3f7Zh2oU4gWLY038 pzkJZ7lT76qFOdxc22MN7AWRmoNR4Jxt1plUazR3Kx2CHPPAlPjFhmATOWQzVnQ1H54p mEraCl7FGtI0EzT6W0V7qICdMdx0dnTi3JpZXGu/u6Ttzha9Zt0Uc8uTuZemcL9A24wm 4ruw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689207519; x=1691799519; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8FNetjBnnWs4aWcoESXEPv5EQNod9M4H8ra/lwVlWng=; b=fxXD+PKvq3WSNhb5OB9606FQvT/5YIZ/tO2SOaxpsnxiQdbAM+Z1Rez5rdjk7UTGzj 8LxT1fZWRc+wWroYcpuQN/3qlz1L53IXVyimQtpwnY45M51SPme8SvlF/I12Q21ETKLo FAZpm5ZjmDdx/vXggorCEuWbO6MR4KVJp/ItiIjCpaZLzu/ADHu0dnNdSj9G3zXoMSM2 L1GxAy/J7gpzhMcm/Pac+uMSKVeiyBgM0V8wBTfpEoOUGHUrrsfK5fA/g9qHmHgD5CK5 l4EXhVjFWOtTcFHAu+e9222FmrKj5liaTj5ytF+bxQ4xQ0svPR3i/X7QhGyQXse7tBo/ GjpQ== X-Gm-Message-State: ABy/qLbFpEkogZUHKYA3VIOCoRJmXa1xlUXc99j/KFMy1hRXqWYRd9nb hT5MIBPkn60Pc0j6vsx02+Y6T7HPBUqALw== X-Google-Smtp-Source: APBJJlHqRdIoAACpRH8a6t9G+zRijJgrhccuFIBvQtuMsvEmpFO+C6Om+9Yz4ShU/cZRQoH/R02E4OT9ko/qNQ== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:902:8bcb:b0:1ab:18eb:17c8 with SMTP id r11-20020a1709028bcb00b001ab18eb17c8mr720plo.2.1689207518956; Wed, 12 Jul 2023 17:18:38 -0700 (PDT) Date: Thu, 13 Jul 2023 00:18:30 +0000 In-Reply-To: <20230713001833.3778937-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230713001833.3778937-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230713001833.3778937-2-jiaqiyan@google.com> Subject: [PATCH v4 1/4] mm/hwpoison: delete all entries before traversal in __folio_free_raw_hwp From: Jiaqi Yan To: linmiaohe@huawei.com, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: akpm@linux-foundation.org, songmuchun@bytedance.com, shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jthoughton@google.com, Jiaqi Yan X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 188D340009 X-Stat-Signature: 4thtjaf3uhuojzujz8ag1aoscekt4hh7 X-HE-Tag: 1689207519-881009 X-HE-Meta: U2FsdGVkX1/1K138fsUjoO/AtCNKog6nzfYh++kMKe6F1YH7RjCq3aYrUgq2yhCvhELDec4HN5Rh4VQuEea78U9jft3/4XwlDlZvLMg/OnWTwjJiYhnfgPKVLb0IAnJqNdqxswZqGdwhTQkm3VYl/9FEbm/moJKYedYhY84XFSafqCnsbyjwTOTyJQ5e7M7SCNXdZmBgdSnFf4rk7uvy+jLnrb4puMgPyHkDX+oYbRqUUVmuwKZly3IDqClzA6FDJrc0jqrFovdpU67t4MUS5OusVtvwhf6MIcPN7WP0wIz2UF/jrXZhmilVkq7oNoh4Z7I+ODhCqG5gwo2vYKR7EJsMG2Z0pggwEfFIL/dSeLpWtknXVdXYoEW+5Ruxwx5K7iuIoQBdmxkDDXNPSBkw41hO7O4I5mZWX2+cLqZJDwfNLM1lmUD7ahFPrTKL/grr/kiMgYcI5iiAmXU5zsrWFGj6FvZYWHx+PWdXFNDCuq4qHbNshoQ/9X40c5sC5v94D6nGPoed1HXnGMtbpydD0QW95qbpiiWBxkcMKP05BHQMl/hvrVNUyn9nHgnagQW4JMNfl0N3j+C/dnoqK1DU0gHMT/w+ZYRDzuPPn6KFctYxIUsOoLy79E5DuYt982Vzysqtr0fAMAJSbqVr5rdMLIv1qEev5tPzIcCvHyE7bFwhDkosRNnlROSE7HAkECJE2vCkyUsGo1irYK+i7FdDx5nLWdr5GUAvs4vdjrHttnegxe1SfIQ5hH65kkbU1ThFAcE+rChJVeHS2GnRCIpJNXRrZbk2er42UFce6AUXVB8s8ceO1LUG5ndRBwFwI3USXfthBie44k60JTLTQql68g2jQTPnPPa8oOMH7J3XYKWF9sP249wrp6WXkz6FENl650EV6eUUF+BDP1uy5Ci9H9pcFC/mJu20cpWMWoShU8TpIZ6EUSILojwe0PostDUfGgFJu1AF9ZEGNHFTZkV Fa1U1u2i TT8DfhwssgjvNWHX8SFNLH7Q5Mtg5n428AQsqMroetWVw8mc5g0q8aY+yUXMR1epvbDDNXKu/A+xzA1hfPPN4Ru7SU5X6tddJmHwKy9qrJEOs1J9gUDRPVa2muh+FVTPC0dJWPZpB2uoCFokb4jlvg7cICq7ZwHUKkdB/37xlsSvyi4LAPawn7uUWjMHEfMse7C3nMvtVVvy6FUjNKF3OHzVsDTaAoWXh+2TFYszD7CvDMKGD4/cgQZlYJ/pQyyJIy93JQ8SvB2RotcxuC1KrnrEeckrBNQrYVW7/kqRXefdDD8+9LHPdu5nBDJOa8ud0kqpSb37yPb3q0YSj579Ss31zDOXXUrQ5w/9BnBSJRjwck10Rq//hhCoAeyKPIXc2ilS+i6ctvb9of76TweS0gLoNbVq9LxeA6RzOERfAfh5cNrUMRyuAHxCjoi0WLSRaCWI0Dq6AfZR2RkCDkKTleVN/MGnGuCjxiTTn/z/TaCZddg7eHD/6FTcCli8rFozPZLG52Cyp007Ynml2+O7FeEsEKndBsDMkY06N5LnjGILdDv4R7o9tQ3AZYdwJMBz1+PgOeXHVmb7a3U3pCYSKr23KV1DLvxOXF37uen93Q8x0DavhII/pTzx+HfUuebdl7vTEL8lR0I3u/CjXz6uF8n+VPj+bGI2osll+C0LyJn8+wrTw4wWIAY2EbR1i4WFYenFODaA7YFURzOpzyg32+Cqo/w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.002459, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Traversal on llist (e.g. llist_for_each_safe) is only safe AFTER entries are deleted from the llist. Correct the way __folio_free_raw_hwp deletes and frees raw_hwp_page entries in raw_hwp_list: first llist_del_all, then kfree within llist_for_each_safe. As of today, concurrent adding, deleting, and traversal on raw_hwp_list from hugetlb.c and/or memory-failure.c are fine with each other. Note this is guaranteed partly by the lock-free nature of llist, and partly by holding hugetlb_lock and/or mf_mutex. For example, as llist_del_all is lock-free with itself, folio_clear_hugetlb_hwpoison()s from __update_and_free_hugetlb_folio and memory_failure won't need explicit locking when freeing the raw_hwp_list. New code that manipulates raw_hwp_list must be careful to ensure the concurrency correctness. Acked-by: Mike Kravetz Acked-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan --- mm/memory-failure.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index e245191e6b04..a08677dcf953 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1829,12 +1829,11 @@ static inline struct llist_head *raw_hwp_list_head(struct folio *folio) static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) { - struct llist_head *head; - struct llist_node *t, *tnode; + struct llist_node *t, *tnode, *head; unsigned long count = 0; - head = raw_hwp_list_head(folio); - llist_for_each_safe(tnode, t, head->first) { + head = llist_del_all(raw_hwp_list_head(folio)); + llist_for_each_safe(tnode, t, head) { struct raw_hwp_page *p = container_of(tnode, struct raw_hwp_page, node); if (move_flag) @@ -1844,7 +1843,6 @@ static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) kfree(p); count++; } - llist_del_all(head); return count; } From patchwork Thu Jul 13 00:18:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13311066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D72CBC001B0 for ; Thu, 13 Jul 2023 00:18:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A22718E000C; Wed, 12 Jul 2023 20:18:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9830E8E0001; Wed, 12 Jul 2023 20:18:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B00E8E000C; Wed, 12 Jul 2023 20:18:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6D7F18E0001 for ; Wed, 12 Jul 2023 20:18:43 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3DE0E1203FD for ; Thu, 13 Jul 2023 00:18:43 +0000 (UTC) X-FDA: 81004677726.05.0CA68BB Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 6DB7BC0013 for ; Thu, 13 Jul 2023 00:18:41 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=OxVY6Z7l; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 34EKvZAgKCE80zr7zFr4x55x2v.t532z4BE-331Crt1.58x@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=34EKvZAgKCE80zr7zFr4x55x2v.t532z4BE-331Crt1.58x@flex--jiaqiyan.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689207521; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Iz4QSmIvTSTUsAdxwi3pCBw4pt8RugCrQx5KDc9ZhJs=; b=2GKyPxLxYL5DVDYyXkj8USqCAEskpibIDcPGpNeRWN6CcwALagQs5Shw/GVGBuvFUUtCt6 CrObDHYaj28NR5c3XbkSvXo9EmR5TGy7N2fGK42TVpttR239QVkf23fviJ0wrdO54tWVK1 U43n5JeJFmsDFRJHtyiv34REPV5T4S0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=OxVY6Z7l; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 34EKvZAgKCE80zr7zFr4x55x2v.t532z4BE-331Crt1.58x@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=34EKvZAgKCE80zr7zFr4x55x2v.t532z4BE-331Crt1.58x@flex--jiaqiyan.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689207521; a=rsa-sha256; cv=none; b=ouZ3jN3lsGgkdo3SDyWZ+8n3uPerWShhFTkBZ1j/Jqy2vfWbvhh/6z9AWSIgSSfS6hxl1b m7LmgooeBZCLirkzDbyA4SV+P417/is0qqTz2/v6KGzxVK3QYtmNi3M+4D/MdLMmxPyQp1 8Pw453haKjncPZpsQLiW+4mcGLgHvdI= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1b8b2a2e720so824425ad.3 for ; Wed, 12 Jul 2023 17:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689207520; x=1691799520; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Iz4QSmIvTSTUsAdxwi3pCBw4pt8RugCrQx5KDc9ZhJs=; b=OxVY6Z7lXNfLpGGQxHTUt1swejOIFF85FUW0wE2VE8vdLB5aDIEaplEBhJovE6UIuU K4MI2omqzLcf0878ohV/qbCyn1o1btoxNO1zmo4XpzgNCwhqbsR5g9lPMhF/oc6c5OSH FLFQ8rcC3HlG6YKuWPQCFnkeyENr45NxeouUwj5SMJayn/oY+aa5qTUD8r4awoQKd282 07sAZWdt7BPtWzt9aI4AGI/dz9iMygnhJ+O+zdqcVmDbmZTctgty2teAKbz/LapOZrLR jUhKEwIXQ2hO2jhE5Geirl2yikNH3XAhBveTKUwqbOhLRH3RICugfJqK+TjpYtt4DoV1 LizA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689207520; x=1691799520; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Iz4QSmIvTSTUsAdxwi3pCBw4pt8RugCrQx5KDc9ZhJs=; b=MKW1mRTgklhFuvLdJ2lCEKXQ+iQso1emsmeVofGjjw4noYj0HgSfbsE+/JdjqWTOn2 Rd/T0vpOpRIwmYDUjkrjPhucLrA711Z1kHLrQWrG6vDy6wTg3PRGlcnpIjVBkjfoeFKl xjV7Eq5NSMahxLodKG5SRUk1R9ZSJ8Nfk87Cy/QgGKw4ZyTYyvOfWoGMY4MYzEI7wKRT AcciytctaFb/e/koxzdmR/w7A/BlNulMeu9HC3hq5wwxSbDfKtVuY3wSiuxIQbM72o1b STrNTRbVxVqhFExOdcY+LOgVx9mfpAHIQVhka+yqkwlbS9zQUjdsulJb736Zmp2i+JBB cOAQ== X-Gm-Message-State: ABy/qLZ8wibBF0UwqlYTpxbFwudWTx6nxieWz4XTbw6FrBoHnx7bNVnX XqIz/GQifxh03k+u9HHzZT34e27YR7xOOQ== X-Google-Smtp-Source: APBJJlFBNAQ4nTEEROlscxAMFHxffimTCXNSjblZZoYT2wX9TgjL8pV7vT6yXRoYce4lKt7cw6slHJX3L0xABg== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:902:e54e:b0:1b5:1fe8:a91 with SMTP id n14-20020a170902e54e00b001b51fe80a91mr611plf.3.1689207520379; Wed, 12 Jul 2023 17:18:40 -0700 (PDT) Date: Thu, 13 Jul 2023 00:18:31 +0000 In-Reply-To: <20230713001833.3778937-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230713001833.3778937-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230713001833.3778937-3-jiaqiyan@google.com> Subject: [PATCH v4 2/4] mm/hwpoison: check if a raw page in a hugetlb folio is raw HWPOISON From: Jiaqi Yan To: linmiaohe@huawei.com, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: akpm@linux-foundation.org, songmuchun@bytedance.com, shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jthoughton@google.com, Jiaqi Yan X-Rspamd-Queue-Id: 6DB7BC0013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: jp4fk84phw17raj3c3z8cajk7dhcpegu X-HE-Tag: 1689207521-290287 X-HE-Meta: U2FsdGVkX18h7qEwyHTOWZIMEX3zxKtF8/m42Fa5/P3ePWVZxAZCFn6IJcW+YRb23IL2+S29YaWy6oI15YILWm/19uzNIdz3dTHHjUxTHLQ6x0W3wqCyMc192dXReRoVSjNPwEcAy5Nuk7+ZAgItheV0MfGh2M69OtRaZdyds7UeUbJwSB1BhGfVe/LKeqc6JewwjQIrNenjSSQYJjMg4aRY44R4NeYDkgY3foW7629KVlQE5dXs7dTRtB7lm6bfzBJrug9+s4kv6zL5mXEKCNxkJ8GprFGwgDASlVbJ6TBLN2ydxxa3Ei3At2IcjL0IASmk5RWtgJKcJTuFyFzU9qbcfD2y/Gr9xMjF0BmVVKDsc4bP2+4mYV5RYhwZxm72x+vUHp78JY2qArBepNQyKx9Ao/IcCkVrBmiqg2xK+4lDtjZAymgmagMrgIK1R4GRZ7VdKZtUnpq8UmkNsywCvXEFS7cdglliNy6cFi6zzt7fwLA1E3DsSlsNZy9Kn1ewNCPR76xLJ3BX2UtlLX0euQdDu1HTy212ea/tARR7+LtMrGc/oIflPzstagK68mMmFwa2H3RpR1rU/8TJlk8ggIUJOfAMEk3nhlAzFhoHlQf6ate4sLLcbDbExL6JsVqOn5oFcUtcGIsxYAat9XV5C8LkIIOp+TRyEnwsHu3tObb+4/x9xJyZyI7tBAT/0tFZTLQLUM6LCIiPeBgPfkTDiINCzNqOMeYOxUZbZlw4CKQV9B9cx2/QyqMl0JOpgzusHida/kZOWZusa7Uszk7buCGyadxFPboDN4rYEmqV4QoxB5TAXysGqiXqyPjaJ7bXl3eCGDPktR+Ly3VI92jINU/EvMu2Gh1Scfk+cGRs/2u4aDhVz0AKllmm7rIhtRQ+54RXmnWB1hRYB6V02CThxZ0WH1pEFeuvlurt97yEH0QQmFS3Lr2wMurLU8KDSRlRVGyayLSZ7ishPvyxFOc Sk4f0NEm hhaXfoPVZQYgNNbs6gVHRg88euDT0OfU6QjbaperuwGG7aQ1Rj50Qm+TRJcQ25b1gE6mNxHwVQJV2QHfKBE0tXfH4EaRwqH9azpk4K+ahgKAJ7+FZLoYIMIDATX5FbtPXSINmqI1/L70DULM+zSeYrwjNaPk1Qcrx1WMaAtsI0trVY4GVM2fofQ3xzPGV0K+7+lAWaGlyQwKObW0wTFCQoNrghuTmv5ZExkzgBj8C7LPed81KhFcfggnsOH5a8tO81L02JierVu5BvXEiReFJziD8Y8aCgxZDKpa25QYqFMzfCtAJaR3nNFfsInwjwmH2TKdtOXLauCWrxoMqoRAaMTQStUKhWtPbvl43k1JC2kMSrxxMxBvBkMwUmUkUZ3fxFskTYySki7T0tHdfIXOt+zzjkvPyHwEehBSMCPZo9OKEdAACl/KV7ZDUaJy2OCAFGcnaVglCz6wXoAmrUaf+VSvabGPdQpHHqkBtHQHE859ZZx8kAQKjsYSd0EceC6yFlVi6DChiMVaZ7JKkmZAE3i2JCW8BpIuExc9GozMYdKJCwyWQP5paePBF5kEIVmVufEA3G8h2cS0zXHrNKaXGfBtzKVInjEYbMQ5AW+CE9Dfyw8dNf1KmVo1l4PXtjxF7bpXKPdyJ11P9Udbap7ztl5O0uO/c1m8xBvstU0PB3DXksydlCA4jYJfHycIYlzew57FO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the functionality, is_raw_hwpoison_page_in_hugepage, to tell if a raw page in a hugetlb folio is HWPOISON. This functionality relies on RawHwpUnreliable to be not set; otherwise hugepage's raw HWPOISON list becomes meaningless. is_raw_hwpoison_page_in_hugepage holds mf_mutex in order to synchronize with folio_set_hugetlb_hwpoison and folio_free_raw_hwp who iterate, insert, or delete entry in raw_hwp_list. llist itself doesn't ensure insertion and removal are synchornized with the llist_for_each_entry used by is_raw_hwpoison_page_in_hugepage (unless iterated entries are already deleted from the list). Caller can minimize the overhead of lock cycles by first checking HWPOISON flag of the folio. Exports this functionality to be immediately used in the read operation for hugetlbfs. Reviewed-by: Mike Kravetz Reviewed-by: Naoya Horiguchi Reviewed-by: Miaohe Lin Signed-off-by: Jiaqi Yan --- include/linux/hugetlb.h | 5 +++++ mm/memory-failure.c | 40 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ca3c8e10f24a..0a96cfacb746 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1007,6 +1007,11 @@ void hugetlb_register_node(struct node *node); void hugetlb_unregister_node(struct node *node); #endif +/* + * Check if a given raw @page in a hugepage is HWPOISON. + */ +bool is_raw_hwpoison_page_in_hugepage(struct page *page); + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index a08677dcf953..d610d8f03f69 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -75,6 +75,8 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; +static DEFINE_MUTEX(mf_mutex); + inline void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); @@ -1813,6 +1815,7 @@ EXPORT_SYMBOL_GPL(mf_dax_kill_procs); #endif /* CONFIG_FS_DAX */ #ifdef CONFIG_HUGETLB_PAGE + /* * Struct raw_hwp_page represents information about "raw error page", * constructing singly linked list from ->_hugetlb_hwpoison field of folio. @@ -1827,6 +1830,41 @@ static inline struct llist_head *raw_hwp_list_head(struct folio *folio) return (struct llist_head *)&folio->_hugetlb_hwpoison; } +bool is_raw_hwpoison_page_in_hugepage(struct page *page) +{ + struct llist_head *raw_hwp_head; + struct raw_hwp_page *p; + struct folio *folio = page_folio(page); + bool ret = false; + + if (!folio_test_hwpoison(folio)) + return false; + + if (!folio_test_hugetlb(folio)) + return PageHWPoison(page); + + /* + * When RawHwpUnreliable is set, kernel lost track of which subpages + * are HWPOISON. So return as if ALL subpages are HWPOISONed. + */ + if (folio_test_hugetlb_raw_hwp_unreliable(folio)) + return true; + + mutex_lock(&mf_mutex); + + raw_hwp_head = raw_hwp_list_head(folio); + llist_for_each_entry(p, raw_hwp_head->first, node) { + if (page == p->page) { + ret = true; + break; + } + } + + mutex_unlock(&mf_mutex); + + return ret; +} + static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) { struct llist_node *t, *tnode, *head; @@ -2106,8 +2144,6 @@ static int memory_failure_dev_pagemap(unsigned long pfn, int flags, return rc; } -static DEFINE_MUTEX(mf_mutex); - /** * memory_failure - Handle memory failure of a page. * @pfn: Page Number of the corrupted page From patchwork Thu Jul 13 00:18:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13311065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D011C001DE for ; Thu, 13 Jul 2023 00:18:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDF218E000D; Wed, 12 Jul 2023 20:18:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8DFE8E0001; Wed, 12 Jul 2023 20:18:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D05C88E000D; Wed, 12 Jul 2023 20:18:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C35BC8E0001 for ; Wed, 12 Jul 2023 20:18:44 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 91513C03D3 for ; Thu, 13 Jul 2023 00:18:44 +0000 (UTC) X-FDA: 81004677768.29.A15C281 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf29.hostedemail.com (Postfix) with ESMTP id CA08412000E for ; Thu, 13 Jul 2023 00:18:42 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=dypHe3K1; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf29.hostedemail.com: domain of 34UKvZAgKCFA10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=34UKvZAgKCFA10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689207522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6XARlF0BXnO6/4Atr6/Ka8Rsp7ITnYdH8gSSlCIsHmQ=; b=d1QD+vNkMkFC2qaG7ZppjVJYC3++1RBODducVLrdwrqV+e4xmAvVAsCNqkLWOClEqv/f6C CKbhKQ+rVji8y2Z7J46e/ODUK10+6BLh+8FamCn3xBFJA0Pwkwk8ivWz9GAfSeD7jHbomx 039+81wTqn8VCi/DQC5Twnii8x2pwH8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=dypHe3K1; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf29.hostedemail.com: domain of 34UKvZAgKCFA10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=34UKvZAgKCFA10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689207522; a=rsa-sha256; cv=none; b=lCj2LIZi/ysxqammwaU+ipvYiSeGIGIqG/OwjTpI1VpiIXagBgvDmRAaWkF3aGLV0jflbK /S+fazN/b7mMhk571xIUgosVEGkuql6gIzBnS/4gaqwZvLuJB3TiCdZDwuxF6bsLRtyhJE SAenPIqGwSucOUKSV3wb/elGsLrGL1o= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1b8a44ee130so895445ad.1 for ; Wed, 12 Jul 2023 17:18:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689207522; x=1691799522; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6XARlF0BXnO6/4Atr6/Ka8Rsp7ITnYdH8gSSlCIsHmQ=; b=dypHe3K1l/c1oaj2HHVrHq7fKVxuhlr/Ku9wHIlgid2l5uxeWU6gepVzV2p5HJ+Mtn HQbc73phPu16+ziP3vR3iq444D/tI3BAZT+QtWryBgLc3sVCy5eU27hzqaAtG1F8EqMl o/uPkLyFyiabmK6g1nZ8jFcEXuWM0VnyC/xfa332OEOupMY+ygXwWjny1snuRDFi0JkG s3xf57LVHLyYM8qLCEOrGRoywDSE6krtphd9CuaI+egnsX7ey3Kuc8nYch5I/S9m9RGe skHC5e2MOBIE+ANgafHaV29E/DQKCyjKboQ/a6THaV0HVxs4uK2tWwdUET7s3EB7CyXK CWZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689207522; x=1691799522; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6XARlF0BXnO6/4Atr6/Ka8Rsp7ITnYdH8gSSlCIsHmQ=; b=efKwAW9mbGDcdPJ2dXoLQ8OP2ze6IzzuDKkz24wHR+E9UXmIMpubIP+8VrXFKiZgHe jHmgKky58jOt13rcc0pbwpdRdFoY1U5YPUVI0s0abXJnrGlVrZe8mcU0Z4IKh98Aou/j iqTkqFJx3VNVB0LQ8Z6ZC1vn3rlmRjFEj0+8e3vWOYmApERxRb3mRc6QmCAqDkVwiwYb BipOHKEMeofUnxN/yQcR9OqXpnsq4hYW3j6tdCtX83qXoEXTYPFj8PcgvpSPy1U1vKHX tTkNzGT5OuXScFCbb6d2gYBhlBkC964mziE86jAG8KcVYODxvQYdIF6Wr9FHfjoe1Lm0 PUUw== X-Gm-Message-State: ABy/qLYmz0XTQ29Yhivn3ElSTFBidgm7LvA7jVkaBwJvxLw8nHn1BBVG mS7MENCy7y01Yudl2lXmlf579uzFD4iYoQ== X-Google-Smtp-Source: APBJJlG1dH5gtGtj7+R2nOPMgHiIgop3mdrBpT2PYcsdRrW+V38DX7do6vdw3iO2pKOF0QHuX8mPhg685Rik7A== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:902:7448:b0:1b7:dbb0:782b with SMTP id e8-20020a170902744800b001b7dbb0782bmr496plt.10.1689207521799; Wed, 12 Jul 2023 17:18:41 -0700 (PDT) Date: Thu, 13 Jul 2023 00:18:32 +0000 In-Reply-To: <20230713001833.3778937-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230713001833.3778937-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230713001833.3778937-4-jiaqiyan@google.com> Subject: [PATCH v4 3/4] hugetlbfs: improve read HWPOISON hugepage From: Jiaqi Yan To: linmiaohe@huawei.com, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: akpm@linux-foundation.org, songmuchun@bytedance.com, shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jthoughton@google.com, Jiaqi Yan X-Rspam-User: X-Stat-Signature: dj5jfejstt5uhos3n4zqoda9fhasn1zx X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CA08412000E X-HE-Tag: 1689207522-388640 X-HE-Meta: U2FsdGVkX1+8bNc6ogbEVLN14WJVxKzTkhZT10oWS2Jxj2Ax/cl/5WK+3GDxLBdJUxyruTbMqktsTfbY17MYUPtgjIuUCWBM5dG+ATl6tu9D9Maut4DVMAHlYr+FrQ2IGiS0ZkAt2ouo5/bP61VRSLz7wzR+Dj8zMXayFHAqdRBenrgX3CjFR3mB5HgrClV1E6r6R+bAHJM3kQ4Xf8b1QuFTK9lAO9LjQaXuHZmXfLmgPzntg1WVK0uMKQNfVO0ZFVQBaFu4hRWj8p7fu6kaWScOWgIlNFNICahGeVpyfXprZSqM1+DIFm0waVDZV8P+YLsZiYDopLowyTTLFrvEPaUa8aXp2isLvl6We2JbvNBvDxu/TdpUXu61pzhdQnnGhqJsOC0u0+I0ba0cVgIPtLR0DtSxGKQlLkKYPZt6hBXnqKA0Q+AlNfI0w+8fVfygFrmm3R+hRmd+yld2boCHdiJX7mPi8VCBvA7f/PPkWoMve6AAkPTLwyHVZ0YtZA2fcahQ3jfZZg3IEO+KkgGQMNysXYD8l0v+reHcAyOrky1TQIJfFyN3tAB8HUeLQYYLnnVfEhIKt2Hv6IaiqhoEXFg1qUpz1brr2zWyi/8erMaAj+HGoyBhWNMHcm+IuosCvuRxTxZQqH4/piIQ037b4/JsV6ex0LwaxPvvDM+Sddn/c8xFq6QK4Z2IlakkplJvi8y4nnguXE0CW+Ebdmr7H06Jxig5/4hL0P/tOOJL73YDLHMGuSJIeirauVMSh/XGpyEo2Qb247QmgndI4d0DoVQGd72kDjBhtEjpxSOzz4F86tt2wMQHUE5EoSgeyyjd/o+tGI3ta3GcvUOLpURXNxsSbtFiC4vp9C2vPyYHr5E5yq6uM/M5o1i3dq5+ev3F3IB/2VsrM8uM46kYuyagmUv/kES06E9FLsBRxpToPcjOPEjAdmil3tBi3wT/8Azcb0WoQPn9SPiNTWZI8/F BiWkpLGz YckLnYdS3TqBySzOYMkpwx5vKspsKlsbu5Y53Q2mWvWzb+CtZwmZm3Wp8O5nFUvxL5RqjDqBR7S9g4oMzO4cd6FBlS4zq2adohzKN+UXCtbjRW/z1uI9ixcoYBYFppIqycLoiuJRoGIqun4WP3CPTAIW+A5iwZXzO/QqS8tPuKt6F/dddRbVpk46pGaa7fkwTMr2UB/q6lwgk38a8kjFsWP0JkpIRO5EyZH/aDRmXzEsshRpOcGVx4tcLlbNSANk35qX9uilBn7jNU0aG8Py4xB8xr9kgcpufzpo8hOrXEvcsKShF2W95f7tqfgt0kVYeXaziGKM6RZccIAOYNDnWiPSCUWFlfHaM+B5i760/FX2qg6aKBdqAysVT66cHZq4JTy5GtA2nTGcbt+JKQ4P1ziBXuCQzeRLZ7D6w1MOmg3RmDjftFtiBHUgwVOEp/kml2vPK69fn8toCbCb2+6Tj4ZKEu9qnzFOQ7+hhFEf8bFdrZeJsM2Asu+BN+KIO+HgufBgRC65SE2tchRUP9dyicjlZGMlHt51Mw3unxNxPv0tC9UtJ/ES1BHCW7DDLB5fqjtPDnNC6DwY+fOWawJi+9l+K3mmt6frP0e/BvXga39B0KKqixfJz50cXBR10NoYU9pjSCXfdzWhKTAgg6DG+mgBmXOQfojmM3I1763dRJPVX468= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000483, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a hugepage contains HWPOISON pages, read() fails to read any byte of the hugepage and returns -EIO, although many bytes in the HWPOISON hugepage are readable. Improve this by allowing hugetlbfs_read_iter returns as many bytes as possible. For a requested range [offset, offset + len) that contains HWPOISON page, return [offset, first HWPOISON page addr); the next read attempt will fail and return -EIO. Reviewed-by: Mike Kravetz Reviewed-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan --- fs/hugetlbfs/inode.c | 57 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 51 insertions(+), 6 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 7b17ccfa039d..e7611ae1e612 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -282,6 +282,41 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, } #endif +/* + * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. + * Returns the maximum number of bytes one can read without touching the 1st raw + * HWPOISON subpage. + * + * The implementation borrows the iteration logic from copy_page_to_iter*. + */ +static size_t adjust_range_hwpoison(struct page *page, size_t offset, size_t bytes) +{ + size_t n = 0; + size_t res = 0; + + /* First subpage to start the loop. */ + page += offset / PAGE_SIZE; + offset %= PAGE_SIZE; + while (1) { + if (is_raw_hwpoison_page_in_hugepage(page)) + break; + + /* Safe to read n bytes without touching HWPOISON subpage. */ + n = min(bytes, (size_t)PAGE_SIZE - offset); + res += n; + bytes -= n; + if (!bytes || !n) + break; + offset += n; + if (offset == PAGE_SIZE) { + page++; + offset = 0; + } + } + + return res; +} + /* * Support for read() - Find the page attached to f_mapping and copy out the * data. This provides functionality similar to filemap_read(). @@ -300,7 +335,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) while (iov_iter_count(to)) { struct page *page; - size_t nr, copied; + size_t nr, copied, want; /* nr is the maximum number of bytes to copy from this page */ nr = huge_page_size(h); @@ -328,16 +363,26 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) } else { unlock_page(page); - if (PageHWPoison(page)) { - put_page(page); - retval = -EIO; - break; + if (!PageHWPoison(page)) + want = nr; + else { + /* + * Adjust how many bytes safe to read without + * touching the 1st raw HWPOISON subpage after + * offset. + */ + want = adjust_range_hwpoison(page, offset, nr); + if (want == 0) { + put_page(page); + retval = -EIO; + break; + } } /* * We have the page, copy it to user space buffer. */ - copied = copy_page_to_iter(page, offset, nr, to); + copied = copy_page_to_iter(page, offset, want, to); put_page(page); } offset += copied; From patchwork Thu Jul 13 00:18:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13311067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B1E6C00528 for ; Thu, 13 Jul 2023 00:18:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D08178E000E; Wed, 12 Jul 2023 20:18:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C92958E0001; Wed, 12 Jul 2023 20:18:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1FC88E000E; Wed, 12 Jul 2023 20:18:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9386D8E0001 for ; Wed, 12 Jul 2023 20:18:46 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 602BBB0066 for ; Thu, 13 Jul 2023 00:18:46 +0000 (UTC) X-FDA: 81004677852.15.167C131 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf17.hostedemail.com (Postfix) with ESMTP id 8AFDA4000B for ; Thu, 13 Jul 2023 00:18:44 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=FlZRL9jU; spf=pass (imf17.hostedemail.com: domain of 340KvZAgKCFI32uA2Iu708805y.w86527EH-664Fuw4.8B0@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=340KvZAgKCFI32uA2Iu708805y.w86527EH-664Fuw4.8B0@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689207524; a=rsa-sha256; cv=none; b=z7md5hv5w4+ysAmA5ZHfrsg9A7YozhPSfvSI8cFwLw/VkK3nXwLr5ow/GGkB6NhIAXcy2s +k+sd9TthJ7MAv6yyRoH7BGg4+ScapiB3M0/uEwAM0wvUhDfNStNXJ/RpVaV6tX5oxiFJ0 gwqENKneP245UAGqy/5AdaDbiXxLPD4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=FlZRL9jU; spf=pass (imf17.hostedemail.com: domain of 340KvZAgKCFI32uA2Iu708805y.w86527EH-664Fuw4.8B0@flex--jiaqiyan.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=340KvZAgKCFI32uA2Iu708805y.w86527EH-664Fuw4.8B0@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689207524; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dmSXGhNgjtW0Yb8+yIyK70NmvAjyR1vjV+RIYl0ULq4=; b=XfhT/Rt9/IvmomwUQHfkToQs9heJDmjcRD70pdW/zZRKqRU8MNQg+bXNWbwYroebbVAGAo fjGbaKfiqp9CjmtSnuYYDjWGLGCRFuqiXnCLA1ntGGbWuLRf77ehWybtmNs/880JwXaTwX i2rJ3/b+IZnR48vSkanF0RJ1S9NZ99o= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1b888bdacbcso870895ad.2 for ; Wed, 12 Jul 2023 17:18:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689207523; x=1691799523; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dmSXGhNgjtW0Yb8+yIyK70NmvAjyR1vjV+RIYl0ULq4=; b=FlZRL9jUxJFKgY4MqDtKMkc0rMg/7jFvumz8fttZ1mweOBBWCGzvC8lc/FCQJuLkTZ A3qVeyiodQbUroyDwmQMCZw9jh+ovjD2C6URvGmE5mQg2qAG3rIFa3u5O3utY9wqWDdh y2yPK+CMKrOvd+Vf4PsWNx1mpzo4cScAJX/G/GmXtN3BrOtfw2lROeQROcur5BX+u/nI HMmRA4ujFCbKxeSV9RFjI91KjjN5dVl+5reqgQrM5L+aNYcjlbm1Kknrzn4l69z7WPtO K7sCo37UfV1f9414ZzedguXvQ6M3kjBSF+jGOr6vSzlnQwTqTeW0uoCbjObPBWxUQGY6 wV8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689207523; x=1691799523; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dmSXGhNgjtW0Yb8+yIyK70NmvAjyR1vjV+RIYl0ULq4=; b=hw8++29+e8dp+efOeqjZCTEM0Jj38ZxJvCaBqeIu/gnVFo2APLqb8oHoqOvtnTgL3x 3vMRl1r7BqdAJl05bXn/Cu29g2XLVLm+XnYL/JAywwWYM9he30GF9Hsa6RrBmSP9NUuf T23pEoVMRaRWcS4lsi9FkYH1vv4l8DAbKDXiWy9aqeyd3RHgx7Jd1RxIimw0lUx8CLCc igvLzZunvkNdugisV8G9MsRwSfmbEg3t3CHsq+BaZn91+SBKoYaZMyKTnJKvWo688w4J dbcB468F6jFpgcG1+pZvMlMz4fV6O2fBRloPlsjGYdlXLBTFxcmj7kA+TU8BIQRf0Kpc Fjyw== X-Gm-Message-State: ABy/qLZluTaPrA2SP6DvPNuPMWISJD2Hlk834yfE7HvKBI35LNNH+w3T G4YLeSZlLP1csFaU99Ia3Q/ESY1ecBIVfA== X-Google-Smtp-Source: APBJJlG1yocyNKvw7nuw2+n9bzHNP/TDAk4jjb7zrKQsqMxgJBPSYO7jxfm4d0oFBf2YofHbmBKExQhGGJ0LSw== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:902:d50f:b0:1b8:a555:385d with SMTP id b15-20020a170902d50f00b001b8a555385dmr624plg.9.1689207523281; Wed, 12 Jul 2023 17:18:43 -0700 (PDT) Date: Thu, 13 Jul 2023 00:18:33 +0000 In-Reply-To: <20230713001833.3778937-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230713001833.3778937-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230713001833.3778937-5-jiaqiyan@google.com> Subject: [PATCH v4 4/4] selftests/mm: add tests for HWPOISON hugetlbfs read From: Jiaqi Yan To: linmiaohe@huawei.com, mike.kravetz@oracle.com, naoya.horiguchi@nec.com Cc: akpm@linux-foundation.org, songmuchun@bytedance.com, shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, jthoughton@google.com, Jiaqi Yan X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 8AFDA4000B X-Stat-Signature: 6i1rbrtktgpirz4oox7qe96u174skn5x X-Rspam-User: X-HE-Tag: 1689207524-302007 X-HE-Meta: U2FsdGVkX1//Lh0aymEtmpDNvCYFgxEWubZvCYeiTZifN9sg28CzCsPm2rRHThEWs8Lab8rX2IpWpZVFDlh8s2dfhka3GD5foIw/ZxE/TAwLnBe1IxkCk3NzhibQi6TqVMK+F7+4X1oCd/q/2k/mhXvpYYdOL/UZCsG0V92kSb6D07/KLl2j0l45TUSfSecMHqTpMcQpD3fQBzHM0pNjJ6ognDKPcvHGz34G6KxXMBv/QJAeOlMpBDrn8bs+bd+kzpoVxzM7UISzxwON2DYSbq5Af0UPOvVKXiLSPRsoZZfWbHhW8IMaHEPE6qm0Q3dhtE25f3hK8IqyCpjuyE8pdp2R3mpjXPDV9p2l+j6LTKQwaREp3wso6JijsMYu7w5fcrSa8BrWDyGDkjYIYAyvdQkx0UANpUpMC6FL54WG0HXaOFnwcYljJ79AUDLiVKWiYub7MzBuf7PX/nsepsGtF69rwpiqntnqD4XzUCXcfIzcRsIaesyml1hlnZM1wvuwOgBT/s74y36juqYU89fnXThJcLYQfyVHD47Z5LZ0Ogwa2mClQl5DsMOwX6YzT9r/EKAIdwFpoe3WXAyNz4qrWOF+wBQSkfEzTtQOR5aKhCZJsGV6K7xZ3B5Fe7b3/EGfZQdPbn2DVclIrcXlZeGwC5D4eVd+reUPH9eAzJ2B03wHSNM2HJgyTZUtHz0lbUiwl6MzPQNkR5ErDHfLebLyTP8VtSToeNgIhsVW/YqD6Faj8StI1NDDWcxqKp6EDSQo9F84qGqdg9PC59vgouyn8mLxsDGuKP8MY0ys9tOAs/e54tQN9qxLjYTuvIkIO/a8ku29TKyN5o69OJGq27b71ppq9UBvbH2Ft/H7NGGAL3AmK43o9mUJOPcYRskSWjOrbLjoeqBKfk76CODxaPLys35lwaGI9bYydatPVuXJ/1vPYp/Buj6xCgcdgTd9djNtEqFMrjOe3dteNglu9mf G2TXKCfv vPUHmjB4XsjNDnCFdLoLp+VT/Mc0/lVBMXs/9WHxzBjMIeSGa4Fk+BzVzesi1J59zRXV/TGKVihgD4C6r+pXx80DoGG1WVk5m/TJfKAH4gUK6aJFHU1hi+lvTDQ9jsBp1og8vzg6IgpiAz6DSuHpIpOwBh4MkPaJIZGGnGTQZR25wqbdElZQJ4nJBQUuQYTIuvTtXjjiTuuK9/ysnRDatr8tMPvcrTeatotkcKBYfeBMQWA1/cUhdJJjxkzCU7ubaWDbJ3KXdDKtu2/BDnwQuXc08o7S8lRk79B9gZ+W9eXiiiYQ0bc8E8N0CQLEYTJONJwHXW/ic8uvof2aqXi4sEOc95wX/1JQOFn1y156fKODbtwza6FlSqkYoNlqrPWJ/A3fNvwLmJWMIc2hL8bz5YqA9DT4FsTDw/4fAYXXWskUO6wUZS6mtG2uVtm2nMEmUuTzDATkJ5V7tO0uVBz/h13GAl27Z6Hlptl3+AW1Zih/RLkjKTZi5hGnFQ2xHc+ymyZJReWGrULmPunFMLOw6ze094mDbQQpBojCbydJUWbSXxwKDVId8Pd9zCYqJ7vIVjeFnfzQwLXljbmfg7g/+4QxvRt0Ojgeki3tESGv8BK6WsKd74qyDRB8Grtrv2niSFz45/5MAAtbeYth9txvk1R7u6fmr2wNnNAQEjCKObWwQBq4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add tests for the improvement made to read operation on HWPOISON hugetlb page with different read granularities. For each chunk size, three read scenarios are tested: 1. Simple regression test on read without HWPOISON. 2. Sequential read page by page should succeed until encounters the 1st raw HWPOISON subpage. 3. After skip a raw HWPOISON subpage by lseek, read()s always succeed. Acked-by: Mike Kravetz Reviewed-by: Naoya Horiguchi Signed-off-by: Jiaqi Yan Tested-by: Muhammad Usama Anjum --- tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + .../selftests/mm/hugetlb-read-hwpoison.c | 322 ++++++++++++++++++ 3 files changed, 324 insertions(+) create mode 100644 tools/testing/selftests/mm/hugetlb-read-hwpoison.c diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore index 7e2a982383c0..cdc9ce4426b9 100644 --- a/tools/testing/selftests/mm/.gitignore +++ b/tools/testing/selftests/mm/.gitignore @@ -5,6 +5,7 @@ hugepage-mremap hugepage-shm hugepage-vmemmap hugetlb-madvise +hugetlb-read-hwpoison khugepaged map_hugetlb map_populate diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile index 66d7c07dc177..b7fce9073279 100644 --- a/tools/testing/selftests/mm/Makefile +++ b/tools/testing/selftests/mm/Makefile @@ -41,6 +41,7 @@ TEST_GEN_PROGS += gup_longterm TEST_GEN_PROGS += gup_test TEST_GEN_PROGS += hmm-tests TEST_GEN_PROGS += hugetlb-madvise +TEST_GEN_PROGS += hugetlb-read-hwpoison TEST_GEN_PROGS += hugepage-mmap TEST_GEN_PROGS += hugepage-mremap TEST_GEN_PROGS += hugepage-shm diff --git a/tools/testing/selftests/mm/hugetlb-read-hwpoison.c b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c new file mode 100644 index 000000000000..ba6cc6f9cabc --- /dev/null +++ b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "../kselftest.h" + +#define PREFIX " ... " +#define ERROR_PREFIX " !!! " + +#define MAX_WRITE_READ_CHUNK_SIZE (getpagesize() * 16) +#define MAX(a, b) (((a) > (b)) ? (a) : (b)) + +enum test_status { + TEST_PASSED = 0, + TEST_FAILED = 1, + TEST_SKIPPED = 2, +}; + +static char *status_to_str(enum test_status status) +{ + switch (status) { + case TEST_PASSED: + return "TEST_PASSED"; + case TEST_FAILED: + return "TEST_FAILED"; + case TEST_SKIPPED: + return "TEST_SKIPPED"; + default: + return "TEST_???"; + } +} + +static int setup_filemap(char *filemap, size_t len, size_t wr_chunk_size) +{ + char iter = 0; + + for (size_t offset = 0; offset < len; + offset += wr_chunk_size) { + iter++; + memset(filemap + offset, iter, wr_chunk_size); + } + + return 0; +} + +static bool verify_chunk(char *buf, size_t len, char val) +{ + size_t i; + + for (i = 0; i < len; ++i) { + if (buf[i] != val) { + printf(PREFIX ERROR_PREFIX "check fail: buf[%lu] = %u != %u\n", + i, buf[i], val); + return false; + } + } + + return true; +} + +static bool seek_read_hugepage_filemap(int fd, size_t len, size_t wr_chunk_size, + off_t offset, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = offset / wr_chunk_size + offset % wr_chunk_size; + + printf(PREFIX PREFIX "init val=%u with offset=0x%lx\n", val, offset); + printf(PREFIX PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + if (lseek(fd, offset, SEEK_SET) < 0) { + perror(PREFIX ERROR_PREFIX "seek failed"); + return false; + } + + while (offset + total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(PREFIX ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static bool read_hugepage_filemap(int fd, size_t len, + size_t wr_chunk_size, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = 0; + + printf(PREFIX PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + while (total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(PREFIX ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static enum test_status +test_hugetlb_read(int fd, size_t len, size_t wr_chunk_size) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + + if (ftruncate(fd, len) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(PREFIX ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + if (read_hugepage_filemap(fd, len, wr_chunk_size, len)) + status = TEST_PASSED; + + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static enum test_status +test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size, + bool skip_hwpoison_page) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + char *hwp_addr = NULL; + const unsigned long pagesize = getpagesize(); + + if (ftruncate(fd, len) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(PREFIX ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + /* + * Poisoned hugetlb page layout (assume hugepagesize=2MB): + * |<---------------------- 1MB ---------------------->| + * |<---- healthy page ---->|<---- HWPOISON page ----->| + * |<------------------- (1MB - 8KB) ----------------->| + */ + hwp_addr = filemap + len / 2 + pagesize; + if (madvise(hwp_addr, pagesize, MADV_HWPOISON) < 0) { + perror(PREFIX ERROR_PREFIX "MADV_HWPOISON failed"); + goto unmap; + } + + if (!skip_hwpoison_page) { + /* + * Userspace should be able to read (1MB + 1 page) from + * the beginning of the HWPOISONed hugepage. + */ + if (read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + pagesize)) + status = TEST_PASSED; + } else { + /* + * Userspace should be able to read (1MB - 2 pages) from + * HWPOISONed hugepage. + */ + if (seek_read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + MAX(2 * pagesize, wr_chunk_size), + len / 2 - MAX(2 * pagesize, wr_chunk_size))) + status = TEST_PASSED; + } + +unmap: + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(PREFIX ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static int create_hugetlbfs_file(struct statfs *file_stat) +{ + int fd; + + fd = memfd_create("hugetlb_tmp", MFD_HUGETLB); + if (fd < 0) { + perror(PREFIX ERROR_PREFIX "could not open hugetlbfs file"); + return -1; + } + + memset(file_stat, 0, sizeof(*file_stat)); + if (fstatfs(fd, file_stat)) { + perror(PREFIX ERROR_PREFIX "fstatfs failed"); + goto close; + } + if (file_stat->f_type != HUGETLBFS_MAGIC) { + printf(PREFIX ERROR_PREFIX "not hugetlbfs file\n"); + goto close; + } + + return fd; +close: + close(fd); + return -1; +} + +int main(void) +{ + int fd; + struct statfs file_stat; + enum test_status status; + /* Test read() in different granularity. */ + size_t wr_chunk_sizes[] = { + getpagesize() / 2, getpagesize(), + getpagesize() * 2, getpagesize() * 4 + }; + size_t i; + + for (i = 0; i < ARRAY_SIZE(wr_chunk_sizes); ++i) { + printf("Write/read chunk size=0x%lx\n", + wr_chunk_sizes[i]); + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf(PREFIX "HugeTLB read regression test...\n"); + status = test_hugetlb_read(fd, file_stat.f_bsize, + wr_chunk_sizes[i]); + printf(PREFIX "HugeTLB read regression test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf(PREFIX "HugeTLB read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], false); + printf(PREFIX "HugeTLB read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf(PREFIX "HugeTLB seek then read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], true); + printf(PREFIX "HugeTLB seek then read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + } + + return 0; + +create_failure: + printf(ERROR_PREFIX "Abort test: failed to create hugetlbfs file\n"); + return -1; +}