From patchwork Wed May 17 16:09:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13245279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90FB4C77B75 for ; Wed, 17 May 2023 16:10:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C94B900006; Wed, 17 May 2023 12:10:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2793D900003; Wed, 17 May 2023 12:10:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13FBA900006; Wed, 17 May 2023 12:10:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 04E08900003 for ; Wed, 17 May 2023 12:10:41 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BA8E61C6D09 for ; Wed, 17 May 2023 16:10:40 +0000 (UTC) X-FDA: 80800235040.29.B4B2533 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf05.hostedemail.com (Postfix) with ESMTP id B54EB10001E for ; Wed, 17 May 2023 16:10:02 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="YE1b/zym"; spf=pass (imf05.hostedemail.com: domain of 3WfxkZAgKCBAzyq6yEq3w44w1u.s421y3AD-220Bqs0.47w@flex--jiaqiyan.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3WfxkZAgKCBAzyq6yEq3w44w1u.s421y3AD-220Bqs0.47w@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684339802; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=woRAdaWYV8FD8vab19hJA4MPyBeuK73rImQoSJfr/I0=; b=xGP54tzHfJehhtJEG7BhV5cEXk47o3uzDoZOCDBG2skSw0VeR7uqSJfazMA++bPMYFOrx7 OkfGVlWtjjzq6qFJuS+YO9xjC0fY9Saa7D3ifxpHIqCGPqKhRik2vtO6zvZ062E5m/e9Q7 kMFRWzKBqlxgBA98HNKvLAa2BLLd2z0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684339802; a=rsa-sha256; cv=none; b=nrP6z2LsRFDMqoRnFNO8uVgixU40l1Oscl2rqt8wGOJhE3l3OcfqECxGq+H068frvvURnN l4Efy1+Sgq5BJvJccf3kJVOnBDbJT6csxnSdDjm84e4Ed8VcfhVyYrudQvyiVdT7R00f5f vzC7Kr8KFK/tfzD7T/5j8GGCvlN6zx4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="YE1b/zym"; spf=pass (imf05.hostedemail.com: domain of 3WfxkZAgKCBAzyq6yEq3w44w1u.s421y3AD-220Bqs0.47w@flex--jiaqiyan.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3WfxkZAgKCBAzyq6yEq3w44w1u.s421y3AD-220Bqs0.47w@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1ae5a30a9c4so964725ad.1 for ; Wed, 17 May 2023 09:10:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684339801; x=1686931801; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=woRAdaWYV8FD8vab19hJA4MPyBeuK73rImQoSJfr/I0=; b=YE1b/zymZ9vc5qJrU2yZ6Y6Sk4upJCBRZPNMxGSUkSJ4NmR9re5sc7FtxyFCA6op5Z NYFItdPqItmJDGSzZeWAH3tLZk1Wm8Afz5r3LdakqUg7OhhCWIIdGn7UMn0FZjneF7a+ bisIrHgG+Aut0UuT0TMLzqNQqD7aRu51rTrV0uDZzw1l+caeFdr0GilMHPWzBABPv15M X4eJsAFnhsiuhrdvQNPjyZTVUft46X+7oX5ruBdVoJ1jwWqEudjOGdVLWk0A5g6E9K6h emxl51IemdopSKTESt1Ud6k+QmEevHfSkIuOBN+uVG3cweipz2hw8QLMUNEJwvHVbkK9 kVdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684339801; x=1686931801; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=woRAdaWYV8FD8vab19hJA4MPyBeuK73rImQoSJfr/I0=; b=RgeM88BFRuzh7ecsGXnbFcm+ufPaPNzPugn4eHefaIWG5C7IZFpPFkoigOwEoXfSC3 aXzWNqLu53nt5ioLe57nTwLP+tjjZMsvn/KTT9MZpwJskdDicxW3JEinG03/MOuxnBBa TVcaoN8t3deS8Z+ogRKbNkxouN1lDhPHtNI32PysSAKtVidzSl+oxEQ0bcHqjr9ZK64U UUbVgn8py8uKDNRkIJK+2yAL2lGsbaDrJXppQeqkFJ5YNPidF3IO82vxijRwKrepKQ06 YtgbsSTfWDA8Zrrg4W/5m2POAk+fFuv7MxqYXnG8donqaHLXuXmYLFmAVoKRzPySRUI9 UnMQ== X-Gm-Message-State: AC+VfDxvgGz6b58ED8QYOi+OcLufm/9JMkKNYTQ2VzO8O2GVvpdZ5mdF ikIjvaJu4R3mHTFCXoVVZZm6Jh4hshCjHQ== X-Google-Smtp-Source: ACHHUZ4THj5Qkl03JCHCjTlSWp/o7bV04vsyL93TtgiK33g3S4UGmuzEa+WOM6LmC0yaPTlWwNF+duECY3Pn7A== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:903:334f:b0:1ae:531f:366a with SMTP id ka15-20020a170903334f00b001ae531f366amr637380plb.5.1684339801269; Wed, 17 May 2023 09:10:01 -0700 (PDT) Date: Wed, 17 May 2023 16:09:46 +0000 In-Reply-To: <20230517160948.811355-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230517160948.811355-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230517160948.811355-2-jiaqiyan@google.com> Subject: [PATCH v1 1/3] mm/hwpoison: find subpage in hugetlb HWPOISON list From: Jiaqi Yan To: mike.kravetz@oracle.com, songmuchun@bytedance.com, naoya.horiguchi@nec.com, shy828301@gmail.com, linmiaohe@huawei.com Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Rspamd-Queue-Id: B54EB10001E X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: xucx5kx1xy5gmep9d1ymmg63epng1d9x X-HE-Tag: 1684339802-600515 X-HE-Meta: U2FsdGVkX1+Llwwt70UiB54y6QYWuiwo2jwjRo+CVGEeDaow0x+m0PSWETnU0fbAAAvVT6XACTBVW+TEb2ptsV45KAHYEpi0qL2pJ98FS8Id4kbcZy+m6kQdAktbQLK9PjqbpXqt2EJXkKBV0KsW9EZr1uctUawlIc13u19vfUxisvSYKZBBp60CnT+tBKD1cWc4gHFijwkorZJuqEXo29/b5f839om3iqQbbqR/4prnUgGHwLjP2mMCrZuXdTQQRIIvh3A81EfED7GjGWUeVJYIbUIKCxlB48GYiJ/hDgjdpsC4f/uhqXycjq1uFtLzeUrnWU3a8nqt9gozLt8FcaruGVoFao+xKEUZ5kLb1n0xYMl2O/+1kWXpHQTfOKC2TqefAHFYSzoJbDm9ZPzJLMC4kNu8QctEXyLwA/wwHnSrtUDGbUem1/+52bgzr1HWxiKkQau8bg9KqGcGLUu6l4nbNpm6J+vo57RaIBlBL8Hu3UBTnE1p5GTGGgw4zI0me53CrVMowYW3D0ybSB0SykSkzNnWKjhC18iYs861mXdC3Bm+o7vilMBa9k+RbrhazNSg4+lRXuIvI92QsbHpP2RDR57clEr+PAv5WgTF0O1B/7kWRimn2BRhKHbFiWbHFOJZRnRP+SQGb910Jhm35daZ87In4Am/SCAWmZWKpgos7eGvcRGd7rwMLeUfIhF1ik/hiLgYcQTDz1XnhXb9LCTRnE5DEHKeULdbG8SXQ65F1pUgGc+nnl4Ji0Izgz1pYv6oCK2uM9nYXbfC5yvQtvBSF5BN+0HNGw7skV+0d32LbNKaqVNjenbxLwHE4GkF7FLsZSr5HnC9mOs72agr1zIYguvYRDaUiwymM8uZLvTSa6WznBQ8ppmqcMThIeViD+MeVzBgkeBPPtpsof85NFLPW0l0T8+SoTt62HQyWuqprtQSK3RkGOSsVKiAaXOb8npYqT/RKm00RQ4NCyd a8eexBXr abdcfVGwiIbDz2xIM6+AVNT/GQzEjwRhWQH1lB18StHNcbTMA8w0SIMrF92yoiMOQnnyni/d8YXvDsypI1LqGAsbAI8F/VDDH1m3O5/wzIM+6lX862hcI9TOPdG/iPsAOvig/fAzVLbCCf+KQw6U1aK4fiNq+cGEsDfsKLfWd5f4GrDgIwN+JruzwQIPJQ7lxIdKsAo3Kerq2VAZB6dJn+nIFuuafvY7x/UPo3GAZLyPGJFpWFUSTiF+8rFNBcfzfKtpnWQeFzZQ/x/xht+EBEoMjnkWjv53fk8pE7R+y799Zmpdj9Nzi89iWKrQp7Q0/LMTgsidmzLVy6qgkeQPjpDAtfpohSlyPYsbkQ7pEkXYufqg1lOifEkzn2+c1AVkY0WpRM2F9lkxwP9Ol4upy/jQ7lAP/DffnupwNWx48RvkkdsRdj4naNdQHlg+qbs6Qt3cj6DrhnwBmFlU+/hoQwp/ooXeatN4elWcpOdTmRNMxa8IMV8SFtHJK7tlLaGpADxUUbjpM/Md+1YmPO/mucJlgnJOTJDR5Z+aFnqVDnuRtsNiPbroXRvwCiMmvdfpVv1uCOuKVy+2k0KrBAyEGZ9+enVH2DmRfG8cSKwxO/LnI9Pc7frKBg6G17g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Adds the functionality to search a subpage's corresponding raw_hwp_page in hugetlb page's HWPOISON list. This functionality can also tell if a subpage is a raw HWPOISON page. Exports this functionality to be immediately used in the read operation for hugetlbfs. Signed-off-by: Jiaqi Yan Signed-off-by: Mike Kravetz --- include/linux/mm.h | 23 +++++++++++++++++++++++ mm/memory-failure.c | 26 ++++++++++++++++---------- 2 files changed, 39 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 27ce77080c79..f191a4119719 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3683,6 +3683,29 @@ enum mf_action_page_type { */ extern const struct attribute_group memory_failure_attr_group; +#ifdef CONFIG_HUGETLB_PAGE +/* + * Struct raw_hwp_page represents information about "raw error page", + * constructing singly linked list from ->_hugetlb_hwpoison field of folio. + */ +struct raw_hwp_page { + struct llist_node node; + struct page *page; +}; + +static inline struct llist_head *raw_hwp_list_head(struct folio *folio) +{ + return (struct llist_head *)&folio->_hugetlb_hwpoison; +} + +/* + * Given @subpage, a raw page in a hugepage, find its location in @folio's + * _hugetlb_hwpoison list. Return NULL if @subpage is not in the list. + */ +struct raw_hwp_page *find_raw_hwp_page(struct folio *folio, + struct page *subpage); +#endif + #if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLBFS) extern void clear_huge_page(struct page *page, unsigned long addr_hint, diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 5b663eca1f29..c49e6c2d1f07 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1818,18 +1818,24 @@ EXPORT_SYMBOL_GPL(mf_dax_kill_procs); #endif /* CONFIG_FS_DAX */ #ifdef CONFIG_HUGETLB_PAGE -/* - * Struct raw_hwp_page represents information about "raw error page", - * constructing singly linked list from ->_hugetlb_hwpoison field of folio. - */ -struct raw_hwp_page { - struct llist_node node; - struct page *page; -}; -static inline struct llist_head *raw_hwp_list_head(struct folio *folio) +struct raw_hwp_page *find_raw_hwp_page(struct folio *folio, + struct page *subpage) { - return (struct llist_head *)&folio->_hugetlb_hwpoison; + struct llist_node *t, *tnode; + struct llist_head *raw_hwp_head = raw_hwp_list_head(folio); + struct raw_hwp_page *hwp_page = NULL; + struct raw_hwp_page *p; + + llist_for_each_safe(tnode, t, raw_hwp_head->first) { + p = container_of(tnode, struct raw_hwp_page, node); + if (subpage == p->page) { + hwp_page = p; + break; + } + } + + return hwp_page; } static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) From patchwork Wed May 17 16:09:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13245280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F911C77B7F for ; Wed, 17 May 2023 16:10:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C844900007; Wed, 17 May 2023 12:10:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 25249900003; Wed, 17 May 2023 12:10:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F1E7900007; Wed, 17 May 2023 12:10:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D6050900003 for ; Wed, 17 May 2023 12:10:41 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C4B1EC0618 for ; Wed, 17 May 2023 16:10:40 +0000 (UTC) X-FDA: 80800235040.07.BAED20F Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf05.hostedemail.com (Postfix) with ESMTP id 0D3E010001F for ; Wed, 17 May 2023 16:10:03 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=axQ41qX1; spf=pass (imf05.hostedemail.com: domain of 3W_xkZAgKCBI10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3W_xkZAgKCBI10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684339804; a=rsa-sha256; cv=none; b=iPqLDcJJeg+MFqGsGmmJyByyYyfFfEQijAjJ8ch2Mxhh01ABcQNJfwmDQ42emo3DzLkxY/ ++881FHJOhyc7NfRW+pn+2WfxDUBuepoO3/mMoZd5m61LUcxVrXpvH+TV5WSsx4Itd2U5X 9MaToxDtuB9cVNsUr8jQSf+3L9fwjQk= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=axQ41qX1; spf=pass (imf05.hostedemail.com: domain of 3W_xkZAgKCBI10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3W_xkZAgKCBI10s80Gs5y66y3w.u64305CF-442Dsu2.69y@flex--jiaqiyan.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684339804; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4afwjJMNRVLZWsRS4KvH1UWrNK7pSNXsAEU4xOupb6Y=; b=6jsnkzLyUHmy1Gj9vgivGkqQYbcwPRqEJZRCbHVEEaWZv8wIgWUXaCMJUey52C/amqS0Oz ZcdGGkW3aHBmH/dIakidySiq9aMxq9TRHQNvvqLaYBiWX2Rle/kQhQVDiy/4W4sUh0Tbt9 S914BUUONBgdkpXnNfcPQ9BOvO+70bA= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-ba81b24b878so1836353276.3 for ; Wed, 17 May 2023 09:10:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684339803; x=1686931803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=4afwjJMNRVLZWsRS4KvH1UWrNK7pSNXsAEU4xOupb6Y=; b=axQ41qX1bwiHJbGU6rrt3d0A/FAXog8UjScM7eAjhvO/VZz1uMpCiXfvNRTVLE3/8S OZBGnYY6NkXRzHCcXWb2deoRCp4d/IhzJGUh0wL1C9VUN2hWUXoDFRR7iXDO/6Wl+fp6 uGwIan2D0pw5vLh98Q1bRnbHw7ENy/oETdEpQ8m8wE2UMa7QnS7eUD2E3CuA2ie5z6mc Q3h9U6cGrBmDrF/iHiQeKEXmB/AqVVSqZ2+ND4BoFj+Gq0VO1MO60ZY7L3bL5eaUrqOU INdMQsIXL5q34h0EXxp3baPOV9jVqICirZUX4wvP0xgrgao+63OSBIe1/yr8wrkYoNgo iHsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684339803; x=1686931803; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4afwjJMNRVLZWsRS4KvH1UWrNK7pSNXsAEU4xOupb6Y=; b=h8jXMrF09Va95lMyAl+jq5V9RcusPY+lx9N69jrenoezY4Lp3FQjG9n5xMCr/13G9U 8qqgZ0yfYC3X4VBR8rUNpzklxVU4gVRmZWEqbf9eWYiQr6oK8zRg++S+n+TskJjJl+8a 4InwE2v80ulQfYJ4Mjqy1ulKMeUg+LUR0uXyNvbTDL/pDPnHA7J1kX+oq6eu3wmGCwIY vVzkjZU2BgGzmQu1umrzNc/S2wi/rnTAao8oNBfA1SzZx2H7lMsPinbviAHu3ybkJhsQ kgiujJ0IgQacXuOK5FNo2IGOrpuQqus+rsHdBWscjzWbgrhUTjMd7+99OMTLw0XDGR24 lK1g== X-Gm-Message-State: AC+VfDxGDyuZWyoCefHJlOr8GmKVvTVMze5VoOnq2lZsnHL6ha3DUga1 0W+yU6i0flnfPthzm7ZLiEHDjpUGriWq3Q== X-Google-Smtp-Source: ACHHUZ5jetsUCam+N66UMpWQp6kuy86/2g1XYzaVSDPJC584NE14V4DeuwZ0XEKGYqDKMI3zAZXmMykVQH43Wg== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a25:ce12:0:b0:ba8:1fab:4f99 with SMTP id x18-20020a25ce12000000b00ba81fab4f99mr2620511ybe.9.1684339803118; Wed, 17 May 2023 09:10:03 -0700 (PDT) Date: Wed, 17 May 2023 16:09:47 +0000 In-Reply-To: <20230517160948.811355-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230517160948.811355-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230517160948.811355-3-jiaqiyan@google.com> Subject: [PATCH v1 2/3] hugetlbfs: improve read HWPOISON hugepage From: Jiaqi Yan To: mike.kravetz@oracle.com, songmuchun@bytedance.com, naoya.horiguchi@nec.com, shy828301@gmail.com, linmiaohe@huawei.com Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0D3E010001F X-Stat-Signature: 58rxgbpm7hc1ioah93en5do19nfs9qtr X-HE-Tag: 1684339803-339617 X-HE-Meta: U2FsdGVkX1/3g0E0YNGzOJlrBPpD/b2jwT5d0Hhc8cH4YFYQu0rnVB7JRmqDg2MlgcgzP+nsMph3JxuBUhak51ncINdUKzDRTWZ97CVFsyMiuSbDG8pmGOTDSSMz2zpBNv8KF1CwRjPZIQM448mf5XwURqpX256kdLW4CGjFg1Sq0rwbq5afspF0kWtrOqCu4ALH9hYjdnaJj7DiRg5TCannd6umshiBaBudWAdypT34mcEo34p3wZxrkqUZ3HRp6CpZ5AmU4MW5k92CoDsDne2K6JJTBNAs566+FpdXsdCloAGhcfAXuBehpnG/HmTItqePOi3ed6kwtENTsJDbVLR0TPVugK/NVsgHtyhD+OZffrM2pdwxVS6BySYRMpCDmzm2g4No71Z/Lyb6M3fp5ht/1w3exTz3Dt2UA1WSd8cDL/xoF0oJkAYVptPqW0za6JGzaJGCKnGEIxS54F5cgIzMIxHwOQ0dMU70EqwClmTudamBw2dgZKehqwzDsEhkkRaRx0ZYi5ZXj0NvTaCytw2yC9zWdq12iF88T9UDlto8ap070adou8DizwTFmz8gPIYi8FQw452SCs+VH6Wl5mreUzIRzU4Ev8N7mQHmj1GXJw0mCKUiPfvYx0IJ/xAnIKt4iHp6IimE74kb8r9qj5SaHeZmNMQTfpUHrsuiiZq1oE9iPbZwLxhQ7GFcQ1uGFskAgT3vzik3OaA/oDkOtpVYJnIHTB+uCgOOfsEytXKDVtB6QwDTT//pAEerpzzA6QM4kRyC4JvGzu05Q/nhRKcpxYlP32kg4SskNKZhij4tmtowVY3jL7S06cCfrBdX3C/hYS5D8INbr1pTAXC76iUJzTzqXpoXZy4SDN5F4uFbztbI9FdbN0pGdcZLtOtVABPQtqXMQ00qPnJ8gfCQEoSgc2i8uamA426jIHXpXXqDPjCVL7+MwohAoxw9d8PBG70i3GrNxT5skmm7ep6 wXfHXPF2 nkgk1+B/NtjWvPe9KBLPe8Xo08EXcjH8Bu6MDe71Tg43WouR2KDWskKk9RiYUQ240partc6oHd3fMDkSPxlOgaFaEqvJPEwdKIYifZIdIOaFV2nAceXb7gwnoJkoJKXJ3KdF/7AaZ2P8HvzLbKWRuZUlD1MXQCaysWG6nDH+TPrGuxetEEUxsDoiE0Yy6+YRt7kIXM40+qCh/MuMBjcGG3ZwMjXdsfl6/xzcwysTWECpRjHlTonzLj2qm6XyVhfDigRU9L8m4X+uy0ZMXXLge9moZKvwtI8TB2Ekco0x1pQsKGFuJforPlhQbrPxlyHa3FW70vLBon91XHazSEisSfpgLPZ4ByARtAHerU+yvO1pr/jSuMMnHtYQK08+VCoQy2jz/o8e7EuGiuuta6uVYpR/zbZHtmr4KH4C3SlqsaaLcJidB4Rk1Io4w6wsSw+LkI4jv354CSk/5lgjfAD5df3/yJHsooaYJsz1PNUhJi2XTq2OXvnwI3Tor4iRb9zounoMMPNs+IkaHZe9rVTKILYAOxaaOTQY11eKB41pwZA5LTQDU1FXVY8onf7Dp/6/qktuRyKZItHRxd1So1/tlRuL5QzmyZkoVgE+WIhZkm6xbq9GLoXDNAPRQwg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a hugepage contains HWPOISON pages, read() fails to read any byte of the hugepage and returns -EIO, although many bytes in the HWPOISON hugepage are readable. Improve this by allowing hugetlbfs_read_iter returns as many bytes as possible. For a requested range [offset, offset + len) that contains HWPOISON page, return [offset, first HWPOISON page addr); the next read attempt will fail and return -EIO. Signed-off-by: Jiaqi Yan --- fs/hugetlbfs/inode.c | 62 +++++++++++++++++++++++++++++++++++++++----- 1 file changed, 56 insertions(+), 6 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index ecfdfb2529a3..1baa08ec679f 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -282,6 +282,46 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr, } #endif +/* + * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. + * Returns the maximum number of bytes one can read without touching the 1st raw + * HWPOISON subpage. + * + * The implementation borrows the iteration logic from copy_page_to_iter*. + */ +static size_t adjust_range_hwpoison(struct page *page, size_t offset, size_t bytes) +{ + size_t n = 0; + size_t res = 0; + struct folio *folio = page_folio(page); + + folio_lock(folio); + + /* First subpage to start the loop. */ + page += offset / PAGE_SIZE; + offset %= PAGE_SIZE; + while (1) { + if (find_raw_hwp_page(folio, page) != NULL) + break; + + /* Safe to read n bytes without touching HWPOISON subpage. */ + n = min(bytes, (size_t)PAGE_SIZE - offset); + res += n; + bytes -= n; + if (!bytes || !n) + break; + offset += n; + if (offset == PAGE_SIZE) { + page++; + offset = 0; + } + } + + folio_unlock(folio); + + return res; +} + /* * Support for read() - Find the page attached to f_mapping and copy out the * data. This provides functionality similar to filemap_read(). @@ -300,7 +340,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) while (iov_iter_count(to)) { struct page *page; - size_t nr, copied; + size_t nr, copied, want; /* nr is the maximum number of bytes to copy from this page */ nr = huge_page_size(h); @@ -328,16 +368,26 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) } else { unlock_page(page); - if (PageHWPoison(page)) { - put_page(page); - retval = -EIO; - break; + if (!PageHWPoison(page)) + want = nr; + else { + /* + * Adjust how many bytes safe to read without + * touching the 1st raw HWPOISON subpage after + * offset. + */ + want = adjust_range_hwpoison(page, offset, nr); + if (want == 0) { + put_page(page); + retval = -EIO; + break; + } } /* * We have the page, copy it to user space buffer. */ - copied = copy_page_to_iter(page, offset, nr, to); + copied = copy_page_to_iter(page, offset, want, to); put_page(page); } offset += copied; From patchwork Wed May 17 16:09:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaqi Yan X-Patchwork-Id: 13245277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6D8C77B7D for ; Wed, 17 May 2023 16:10:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2EEF7900004; Wed, 17 May 2023 12:10:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 29F1C900003; Wed, 17 May 2023 12:10:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 140AE900004; Wed, 17 May 2023 12:10:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 02087900003 for ; Wed, 17 May 2023 12:10:10 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BF498A0672 for ; Wed, 17 May 2023 16:10:08 +0000 (UTC) X-FDA: 80800233696.04.104F25C Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf01.hostedemail.com (Postfix) with ESMTP id 22E814001C for ; Wed, 17 May 2023 16:10:05 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=KnKAovPQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3XPxkZAgKCBM21t91Ht6z77z4x.v75416DG-553Etv3.7Az@flex--jiaqiyan.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3XPxkZAgKCBM21t91Ht6z77z4x.v75416DG-553Etv3.7Az@flex--jiaqiyan.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684339806; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kGxg+zhhm/tdTqkE4MZxwuMTE52MydtCt+SDG4bfPzg=; b=h56i9hl3MbSdhT8BwZrb/vverfCRpk0dX0gFk86wt0aYylbZtue1n8iLirfi/4A6kUggXx RModtsmvyHiF+z8CRrF7VM2Vc+inSNtdFJhWIJ2r6bD7wSksLDWQKRo6b214ZwNgT1o61k xziyKDESzXqTwA5aUGD/5Xe7QEtfnZ0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=KnKAovPQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3XPxkZAgKCBM21t91Ht6z77z4x.v75416DG-553Etv3.7Az@flex--jiaqiyan.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3XPxkZAgKCBM21t91Ht6z77z4x.v75416DG-553Etv3.7Az@flex--jiaqiyan.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684339806; a=rsa-sha256; cv=none; b=ZuumBtG/XzSlXMnYhsg+gwgv0PRjtXDGhSLEYj1xOojjbHawNyFWaeg67dyjWds8g/0zsC pmrcQOdkp2pxWokJvgcC0fwGiX3cYz5FZcl6gdDh6Zas/UdTbVXLKlTB6OKQCEa6aaf3fQ WJOA05nAnZvR6SIRzWTaMxQ4QfBnGa8= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2505b80d5b3so98424a91.0 for ; Wed, 17 May 2023 09:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684339805; x=1686931805; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kGxg+zhhm/tdTqkE4MZxwuMTE52MydtCt+SDG4bfPzg=; b=KnKAovPQXFj1RExR67Af1qVGSh2sLTfcDMR3AcMJtvl3707WaSzfh6aMkZSZmDD1X+ uzn/EpGaYm2cxuC0FxRYA+GlUxMapzpXiQSZLwZhacfAiuPjYYlLDj85PEJfe1PEIV2b 1D9DlD2BU/7jhp1KwyV52RHdPFRtnFHBB5wK3f6YVen8fubFlK4YWeCIctalUfZwKjKw YTKrF3QNZkBgvpjwSv9cFl6QHcB+0zWqTgPqRHQcoyEbHy4QQTQGIDt1d3FcwkENmt1n Fco52N7YFesAPXg4TnvA5SWtsXJwC7EHDj/K9pELCkr7bQaLZFqUya9X/vOAu+zxDYAE Ou6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684339805; x=1686931805; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kGxg+zhhm/tdTqkE4MZxwuMTE52MydtCt+SDG4bfPzg=; b=Iw8z+2F88jWwcVFq6N3/6ljtpJfuh32gwQwHS+7IapijCuxGWKkNIm06/8cT9U2BW7 PCQiTCSWvlRspx4yVncCH+CsgI1zhgnYy0sZBB7Jm7ZywiDLAzaaVucgdPuEh1epEnj3 OAUQ1eZt2MU/RxHvRbNTDF4amcSNbrCYJKMPovXNNrlXKUMbuIdnyjt1HVcmlZcuJazn IMxr1GggSx3BNRuPs0LLmaDbwquniJqyZ1VRYuStYcs9aaZevu8P7y06vZofpi3vPbWH 83f08xSeJS9UQareka0IoVr+ZaTZIZuNngd6mLjSpnLPB3Wogw23UK3pWcw4FOeQSM1y 6TOA== X-Gm-Message-State: AC+VfDx2bhdliXWgJMcn63QmM9I7c+dOmxDidPVAfge8Rr3cA9d42gLO cVdkn3p4I1VHXmSnqeKKTtqRIFqGgUaPCA== X-Google-Smtp-Source: ACHHUZ5+d60PcaMXbxsAMlias9eLlwRUCq3s5d4aGQz1cuagr41Ek8tZIoO9lZquq6XMY0bgQXJgoSJwjv+7Jg== X-Received: from yjq3.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:272f]) (user=jiaqiyan job=sendgmr) by 2002:a17:90b:89:b0:253:4800:438b with SMTP id bb9-20020a17090b008900b002534800438bmr7216pjb.2.1684339804718; Wed, 17 May 2023 09:10:04 -0700 (PDT) Date: Wed, 17 May 2023 16:09:48 +0000 In-Reply-To: <20230517160948.811355-1-jiaqiyan@google.com> Mime-Version: 1.0 References: <20230517160948.811355-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230517160948.811355-4-jiaqiyan@google.com> Subject: [PATCH v1 3/3] selftests/mm: add tests for HWPOISON hugetlbfs read From: Jiaqi Yan To: mike.kravetz@oracle.com, songmuchun@bytedance.com, naoya.horiguchi@nec.com, shy828301@gmail.com, linmiaohe@huawei.com Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, duenwen@google.com, axelrasmussen@google.com, jthoughton@google.com, Jiaqi Yan X-Stat-Signature: 3gigqch91cgfxazjjtyuc4bnh4qutgz6 X-Rspam-User: X-Rspamd-Queue-Id: 22E814001C X-Rspamd-Server: rspam07 X-HE-Tag: 1684339805-834597 X-HE-Meta: U2FsdGVkX1915DGnWj+Ns1O1RbHZgXfTpJ3CkPbOZBK9PsdS56DfZ9mZM9HctYQ9Jp34GmKE4s6855V0b6FulvFTWd5+Vhf7Xe/PGgRuc67HFoBLYP5buEM82SZmS6NQuMl01+DrHhLapxeAiK0uJFChsCOnDHPEpZfOMNicaqWTHcRvFsmCfeF2+ZY8sB0RQuzLH5jvSDuyaDKf4jEOMadAWYS1yW8s7NTUio8nR4iPCLgYXeFHqKet1GrBpl9QD6KlkHuZHO4p9mqGDO6CInjr1ZXwn2yWLre2zAqAGjHVkvKMOLg2tpGAK9CsDQNQ+z3yOFpPu8Dkt3mRz8eEazcgi6EkDyhnUzVJZy0HBxrhks0QwJTnGXKrNkJ9Dl6Dwy47Y5Od5pxecQ1/zB+P4B0wiLr6lNR888yvr8Kzlovng/sEsAnBcTHzfB0CJRgyATl9etjBHuGV+criZhOC1iDSIrQBiclxKHBCeDL393ikhw071qJhPD3JVkStyNcnqJbEgFOQ0kD4DzHdmkRLyRvf8zM3KOvpDDU6EKH8eKERmjlikYzpexCzVA5PiF9UQxPZRYV8Yr4vSCYWs9cxzxqTZPCfYN8e6lVjgZ4Ef8/KNz+yJgF8vep/W0bVxrUoKY+GBeJz0oo33qZIC0eDUvTrkBhaRsV/t2xWg1eQikg9GE/lrsaJo+aPudjnT/zHFEbxdzLAEZ5jrPdtqbdzSyk96RDDvV8p395uoB+UZZHX+JSxK23jNCxfaJO3R+mjajaQt1w9ki/8fxXmAIAIxPt2gpcZOXAvAuC6KgcNAjNn46yyRMgzcm+Ho5Phfx6DpHFiQJeJObUXjhcHyqO7dSpjNfyevzY9+d7udBxokacBLcoV7CQxTKYxTGLu/xcFreN2nDnT0qP5utY0RePS8ecYngCM0ysCDsRuy2fR2+W2K82Ui/AOkqIKjulJUz8akXj1CmhpmuHpqi2516R ok1vpLOJ ua7Br0zeq231NTWmqLTcBREgMksq7EtkyIocthCVaQDwd1J57vD7TeDW8JB2TvUDdFaWT6E3gZgGQNz1mT6ozCQkXM/+Ym/+ZjKgh5qV8BoREfT9tgrwfgOqiHC74xZLw72UgBSNYUlGifOZtfaMj5gVXWdXsgP3YJcGCpU6YY0N4WklPr9xhm9QgrZWkWZg/8JFez2XOuFUBbzRI16PWa+ZlYlI8vr+kUahBey8ahsC9hyGIsVVuJ8ph4fdSzPsCo/3jShgBrp/6uJRj8hUH2embaZKQWlVYUTMUAmK3C9fp0bIkVGyXMI0WuvbMWlWRhLIglytnsN+U8/NPwgxckQYvUtxP1eJFexzHc3SP9agkVMR/GRLRRfOXDH6uEcOI8h0nD21LuRTjf3Yg/TET0pYFTZlzEalbf7BneGtPfkLDc12lAX6gI4CWJ+qvDYg289R94D6Mp3IX0Avg/eIYfmPMao9vAFBj62BZI4P2CjladiXhOdDQ2xk5LErGntuG7wK04eOjJHNCh+csE4HTy8U4qpudtJSZGEeojGxnkLlqMs60CGo8UTQiedMPMMrtjQxK2sBGNFFkeknY1CmiMfl4FcJXZMQisIzesEZ+Ja0qAk/+IgfSf+m44A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add tests for the improvement made to read operations on HWPOISON hugetlb page with different read granularities. 0) Simple regression test on read. 1) Sequential read page by page should succeed until encounters the 1st raw HWPOISON subpage. 2) After skip raw HWPOISON subpage by lseek, read always succeeds. Signed-off-by: Jiaqi Yan --- tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + .../selftests/mm/hugetlb-read-hwpoison.c | 322 ++++++++++++++++++ 3 files changed, 324 insertions(+) create mode 100644 tools/testing/selftests/mm/hugetlb-read-hwpoison.c diff --git a/tools/testing/selftests/mm/.gitignore b/tools/testing/selftests/mm/.gitignore index 8917455f4f51..fe8224d2ee06 100644 --- a/tools/testing/selftests/mm/.gitignore +++ b/tools/testing/selftests/mm/.gitignore @@ -5,6 +5,7 @@ hugepage-mremap hugepage-shm hugepage-vmemmap hugetlb-madvise +hugetlb-read-hwpoison khugepaged map_hugetlb map_populate diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile index 23af4633f0f4..6cc63668c50e 100644 --- a/tools/testing/selftests/mm/Makefile +++ b/tools/testing/selftests/mm/Makefile @@ -37,6 +37,7 @@ TEST_GEN_PROGS += compaction_test TEST_GEN_PROGS += gup_test TEST_GEN_PROGS += hmm-tests TEST_GEN_PROGS += hugetlb-madvise +TEST_GEN_PROGS += hugetlb-read-hwpoison TEST_GEN_PROGS += hugepage-mmap TEST_GEN_PROGS += hugepage-mremap TEST_GEN_PROGS += hugepage-shm diff --git a/tools/testing/selftests/mm/hugetlb-read-hwpoison.c b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c new file mode 100644 index 000000000000..2f8e84eceb3d --- /dev/null +++ b/tools/testing/selftests/mm/hugetlb-read-hwpoison.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include + +#include +#include +#include +#include +#include + +#include "../kselftest.h" + +#define PREFIX " ... " +#define ERROR_PREFIX " !!! " + +#define MAX_WRITE_READ_CHUNK_SIZE (getpagesize() * 16) +#define MAX(a, b) (((a) > (b)) ? (a) : (b)) + +enum test_status { + TEST_PASSED = 0, + TEST_FAILED = 1, + TEST_SKIPPED = 2, +}; + +static char *status_to_str(enum test_status status) +{ + switch (status) { + case TEST_PASSED: + return "TEST_PASSED"; + case TEST_FAILED: + return "TEST_FAILED"; + case TEST_SKIPPED: + return "TEST_SKIPPED"; + default: + return "TEST_???"; + } +} + +static int setup_filemap(char *filemap, size_t len, size_t wr_chunk_size) +{ + char iter = 0; + + for (size_t offset = 0; offset < len; + offset += wr_chunk_size) { + iter++; + memset(filemap + offset, iter, wr_chunk_size); + } + + return 0; +} + +static bool verify_chunk(char *buf, size_t len, char val) +{ + size_t i; + + for (i = 0; i < len; ++i) { + if (buf[i] != val) { + printf(ERROR_PREFIX "check fail: buf[%lu] = %u != %u\n", + i, buf[i], val); + return false; + } + } + + return true; +} + +static bool seek_read_hugepage_filemap(int fd, size_t len, size_t wr_chunk_size, + off_t offset, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = offset / wr_chunk_size + offset % wr_chunk_size; + + printf(PREFIX "init val=%u with offset=0x%lx\n", val, offset); + printf(PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + if (lseek(fd, offset, SEEK_SET) < 0) { + perror(ERROR_PREFIX "seek failed"); + return false; + } + + while (offset + total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static bool read_hugepage_filemap(int fd, size_t len, + size_t wr_chunk_size, size_t expected) +{ + char buf[MAX_WRITE_READ_CHUNK_SIZE]; + ssize_t ret_count = 0; + ssize_t total_ret_count = 0; + char val = 0; + + printf(PREFIX "expect to read 0x%lx bytes of data in total\n", + expected); + while (total_ret_count < len) { + ret_count = read(fd, buf, wr_chunk_size); + if (ret_count == 0) { + printf(PREFIX "read reach end of the file\n"); + break; + } else if (ret_count < 0) { + perror(ERROR_PREFIX "read failed"); + break; + } + ++val; + if (!verify_chunk(buf, ret_count, val)) + return false; + + total_ret_count += ret_count; + } + printf(PREFIX "actually read 0x%lx bytes of data in total\n", + total_ret_count); + + return total_ret_count == expected; +} + +static enum test_status +test_hugetlb_read(int fd, size_t len, size_t wr_chunk_size) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + + if (ftruncate(fd, len) < 0) { + perror(ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + if (read_hugepage_filemap(fd, len, wr_chunk_size, len)) + status = TEST_PASSED; + + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static enum test_status +test_hugetlb_read_hwpoison(int fd, size_t len, size_t wr_chunk_size, + bool skip_hwpoison_page) +{ + enum test_status status = TEST_SKIPPED; + char *filemap = NULL; + char *hwp_addr = NULL; + const unsigned long pagesize = getpagesize(); + + if (ftruncate(fd, len) < 0) { + perror(ERROR_PREFIX "ftruncate failed"); + return status; + } + + filemap = mmap(NULL, len, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + if (filemap == MAP_FAILED) { + perror(ERROR_PREFIX "mmap for primary mapping failed"); + goto done; + } + + setup_filemap(filemap, len, wr_chunk_size); + status = TEST_FAILED; + + /* + * Poisoned hugetlb page layout (assume hugepagesize=2MB): + * |<---------------------- 1MB ---------------------->| + * |<---- healthy page ---->|<---- HWPOISON page ----->| + * |<------------------- (1MB - 8KB) ----------------->| + */ + hwp_addr = filemap + len / 2 + pagesize; + if (madvise(hwp_addr, pagesize, MADV_HWPOISON) < 0) { + perror(ERROR_PREFIX "MADV_HWPOISON failed"); + goto unmap; + } + + if (!skip_hwpoison_page) { + /* + * Userspace should be able to read (1MB + 1 page) from + * the beginning of the HWPOISONed hugepage. + */ + if (read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + pagesize)) + status = TEST_PASSED; + } else { + /* + * Userspace should be able to read (1MB - 2 pages) from + * HWPOISONed hugepage. + */ + if (seek_read_hugepage_filemap(fd, len, wr_chunk_size, + len / 2 + MAX(2 * pagesize, wr_chunk_size), + len / 2 - MAX(2 * pagesize, wr_chunk_size))) + status = TEST_PASSED; + } + +unmap: + munmap(filemap, len); +done: + if (ftruncate(fd, 0) < 0) { + perror(ERROR_PREFIX "ftruncate back to 0 failed"); + status = TEST_FAILED; + } + + return status; +} + +static int create_hugetlbfs_file(struct statfs *file_stat) +{ + int fd; + + fd = memfd_create("hugetlb_tmp", MFD_HUGETLB); + if (fd < 0) { + perror(ERROR_PREFIX "could not open hugetlbfs file"); + return -1; + } + + memset(file_stat, 0, sizeof(*file_stat)); + if (fstatfs(fd, file_stat)) { + perror(ERROR_PREFIX "fstatfs failed"); + goto close; + } + if (file_stat->f_type != HUGETLBFS_MAGIC) { + printf(ERROR_PREFIX "not hugetlbfs file\n"); + goto close; + } + + return fd; +close: + close(fd); + return -1; +} + +int main(void) +{ + int fd; + struct statfs file_stat; + enum test_status status; + /* Test read() in different granularity. */ + size_t wr_chunk_sizes[] = { + getpagesize() / 2, getpagesize(), + getpagesize() * 2, getpagesize() * 4 + }; + size_t i; + + for (i = 0; i < ARRAY_SIZE(wr_chunk_sizes); ++i) { + printf(PREFIX "Write/read chunk size=0x%lx\n", + wr_chunk_sizes[i]); + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf("HugeTLB read regression test...\n"); + status = test_hugetlb_read(fd, file_stat.f_bsize, + wr_chunk_sizes[i]); + printf("HugeTLB read regression test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf("HugeTLB read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], false); + printf("HugeTLB read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + + fd = create_hugetlbfs_file(&file_stat); + if (fd < 0) + goto create_failure; + printf("HugeTLB seek then read HWPOISON test...\n"); + status = test_hugetlb_read_hwpoison(fd, file_stat.f_bsize, + wr_chunk_sizes[i], true); + printf("HugeTLB seek then read HWPOISON test...%s\n", + status_to_str(status)); + close(fd); + if (status == TEST_FAILED) + return -1; + } + + return 0; + +create_failure: + printf(ERROR_PREFIX "Abort test: failed to create hugetlbfs file\n"); + return -1; +}