From patchwork Fri Sep 8 22:23:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jane Chu X-Patchwork-Id: 13377844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 436BEEEB56E for ; Fri, 8 Sep 2023 22:24:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 88C366B0101; Fri, 8 Sep 2023 18:24:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 813586B0102; Fri, 8 Sep 2023 18:24:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B4356B0103; Fri, 8 Sep 2023 18:24:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 575C16B0101 for ; Fri, 8 Sep 2023 18:24:02 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B63C2A11AC for ; Fri, 8 Sep 2023 22:24:01 +0000 (UTC) X-FDA: 81214859082.11.C4F9936 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf03.hostedemail.com (Postfix) with ESMTP id D83402000D for ; Fri, 8 Sep 2023 22:23:59 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=ZGybkZ8J; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf03.hostedemail.com: domain of jane.chu@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=jane.chu@oracle.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694211840; a=rsa-sha256; cv=none; b=KxvUDzAy/MAQysyJhMpwrfpbAERHxiXFNeDCvw6qtePVp5xuKvNnpOCQjk4cwfUFunFEhD zkQ4YlOPBiWkBxBJcDtGN6/Pp6tYtd9gF/T1CoSs7lY2oaozereE2c2Osp7hOgKpElxmmh 0yg/adyAqN3q+tazo096FFtU8Ls05sA= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=ZGybkZ8J; dmarc=pass (policy=none) header.from=oracle.com; spf=pass (imf03.hostedemail.com: domain of jane.chu@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=jane.chu@oracle.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694211840; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references:dkim-signature; bh=VvoPQjhxkCH2kY1NovE1Z4TOJ9QuhiNmRQBHKzACSx0=; b=lJ39ZFaWpVJVjgn0mxcvzmBbdgKwhxvrLXD9YGV7Py3ygcFaVom4Wc1qL011Mv+1DL/6Ns YK79PKh8L2bLoSxw9fcxSTQ2RNfxRXhaChgSYJkCJZYz1Pi9wzNOBDSUeUPF0R/0yayxDX ymFDrzpLI1Aih6esw5uhg6xR/TWXU5Q= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 388MEl88020463; Fri, 8 Sep 2023 22:23:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id; s=corp-2023-03-30; bh=VvoPQjhxkCH2kY1NovE1Z4TOJ9QuhiNmRQBHKzACSx0=; b=ZGybkZ8J5tSdyjaQ63L3uAquxvQ30iwtxcHsgRu/sE1Sst5gEu6R/1SpU6jnFVVQCYbY eS4SBYubegA8KeFw2Jxy/nIRAbAHEP4Z69MIIdFHcght2TErrDgT6I3XGVmk/0PS9vYv tR2dAikU9/6lKx234bFnGZG8t4bv95tHKOpePzjjWDJ1TlnDMqeH0i0SIwzA/y7ERGvz EfQhrnRqQnNnNC0d/9m6Lh0dFzs8pX7eId1sFP0WtHZ8R78q4I4nrHh8BTpFQAowYUc/ KVbYvj9BK6srbpWpBBwPitpZFa6S9akkMY9Q1wI2qssYPnGN5T6Sx+3HI0XIlDVqvtq+ uw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3t0ba1r38c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 08 Sep 2023 22:23:48 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 388KKhd2017419; Fri, 8 Sep 2023 22:23:48 GMT Received: from brm-x62-16.us.oracle.com (brm-x62-16.us.oracle.com [10.80.150.37]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3suuga6p2g-1; Fri, 08 Sep 2023 22:23:48 +0000 From: Jane Chu To: willy@infradead.org, akpm@linux-foundation.org, nvdimm@lists.linux.dev, dan.j.williams@intel.com, naoya.horiguchi@nec.com, linux-mm@kvack.org Subject: [PATCH v3] mm: Convert DAX lock/unlock page to lock/unlock folio Date: Fri, 8 Sep 2023 16:23:36 -0600 Message-Id: <20230908222336.186313-1-jane.chu@oracle.com> X-Mailer: git-send-email 2.18.4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-09-08_18,2023-09-05_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 bulkscore=0 adultscore=0 suspectscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2309080204 X-Proofpoint-GUID: R_48R7R2JysCnNt8vXjC9XTAdCStV5x1 X-Proofpoint-ORIG-GUID: R_48R7R2JysCnNt8vXjC9XTAdCStV5x1 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D83402000D X-Stat-Signature: 78t9iptxiawctqdhxfgdq9w1pg56mqot X-HE-Tag: 1694211839-831307 X-HE-Meta: U2FsdGVkX1866+RnlFWrMlNbFvTWlqXIB4viYUuhDeCImXVV1lRPkMaDi9zwQeLDO9ObLGppgCdRoCAJ3IIiePmBG2GC++Lrsv8ah/9Ig8msvFWB6AkT8fhSZOGP1NmM8xmQDUsbMKBVmajexqMsJmu1tkyEkQXdFf/FvOVQYDyQRBjS0RLfhbpDN51CfMgUIbX9GJDd5AS4UEbLHCG8EantFqtnD9vnlyQlpk1diEP7BcrkJwvVYp8HND6KsjHEef8uTX++eNMRLeM6WjZhZx6EBZUhLfkF468jrv/bm36LsRcP4HIn95lqL3CekZwN6BbyJZsmpxTZIh8+mvimPNdI9uhl+Z9omF9ddZ2QRNRN9BBBitIu3paLqvPyPLXSp6Ew02fOMeJXaEHp1VAgm/eDCNY98PY0fiZ8MlqPdhy6JnGaVqnd9uf8gq8+JXYqxw1jRV1x+BlM3bmSi47OiPb8eRiOhNH0mSGuso1m1GpXEeoxKtbUG9plpVIWM9xpVQezVOf8Bhb0Ajpw2RS1W0OTQWX4Cq33+qBvv7Tdi/79BuHj7gphgcse0WL5SRGnHyoYL8Yi9qKt4XfL0nzWnBGLSQQQt6lCE/jRs9UsmnKe9RwOB3QzmSOIudnX5PLyb7bNdHOXjVYwni47TmNdZ48Zl0p750cHPXfhKriaZJTc9BQBlqh0jMbGDHXj07i9Qan9dm4yNorF5F39hlkLN/7ktwQ86F1OGl1Tj9H26BmkLCipeZZWqWLn2lmd+h63zbzSDww5Wvku/TQsG9w6MLCD686oWTpRF7N/FKdEnSFfn1D8vnGyOFpw7HpVIw4wBUBgMHVDtlYDotckW+zPjFbx5rHQ2RWS1e6t0qSAzUcPSn0cj9lpoNadKRuzvi/zuoo7YD3K+S1uWZSAi24ihD9/OP1Y+m1CQP2Bd5fH/S0ZuF1YMTeMt0BRWnROQggendAImOCHwR0lWotuwtS 7og0ClXQ sBhoe/mJbUg7RFFaqAfnbTm1YNZsRDWndcVJejG30sgqzWbHJr/riLnNhfRcO/JK25LHZy8I7MnICHW+aRyrVJKtf5fsmf+NBUGcB8/bVNICtPTd9AqdCP2Q6Az3PHejvKep7+CMWO6JT9T6bMtwCmAMQC8uNm8fiBk7TGZmekIexOKYJrLgq2u6P+5AvoFAA9lOzDhHpGINfZpx+zhYhzOdTWfb6zGWmehnK7mekA+x3t7XRtxwvhHC1zyNABOHGWYMb7RJYKjAjbIx1MAWo3zdmJQLMwiTrcfAnmyAcYaVbU4A63vKWB74vUYOa7YvSFlF4SdMjNuIFF/GtKNlLQtZFHq8VSZkJVJCK82eRSkiD/HI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From Matthew Wilcox: The one caller of DAX lock/unlock page already calls compound_head(), so use page_folio() instead, then use a folio throughout the DAX code to remove uses of page->mapping and page->index. [1] The additional change to [1] is comments added to mf_generic_kill_procs(). [1] https://lore.kernel.org/linux-mm/b2b0fce8-b7f8-420e-0945-ab9581b23d9a@oracle.com/T/ Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Naoya Horiguchi Signed-off-by: Jane Chu --- fs/dax.c | 24 ++++++++++++------------ include/linux/dax.h | 10 +++++----- mm/memory-failure.c | 29 ++++++++++++++++------------- 3 files changed, 33 insertions(+), 30 deletions(-) base-commit: 727dbda16b83600379061c4ca8270ef3e2f51922 diff --git a/fs/dax.c b/fs/dax.c index 906ecbd541a3..c70d4da047db 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -426,23 +426,23 @@ static struct page *dax_busy_page(void *entry) return NULL; } -/* - * dax_lock_page - Lock the DAX entry corresponding to a page - * @page: The page whose entry we want to lock +/** + * dax_lock_folio - Lock the DAX entry corresponding to a folio + * @folio: The folio whose entry we want to lock * * Context: Process context. - * Return: A cookie to pass to dax_unlock_page() or 0 if the entry could + * Return: A cookie to pass to dax_unlock_folio() or 0 if the entry could * not be locked. */ -dax_entry_t dax_lock_page(struct page *page) +dax_entry_t dax_lock_folio(struct folio *folio) { XA_STATE(xas, NULL, 0); void *entry; - /* Ensure page->mapping isn't freed while we look at it */ + /* Ensure folio->mapping isn't freed while we look at it */ rcu_read_lock(); for (;;) { - struct address_space *mapping = READ_ONCE(page->mapping); + struct address_space *mapping = READ_ONCE(folio->mapping); entry = NULL; if (!mapping || !dax_mapping(mapping)) @@ -461,11 +461,11 @@ dax_entry_t dax_lock_page(struct page *page) xas.xa = &mapping->i_pages; xas_lock_irq(&xas); - if (mapping != page->mapping) { + if (mapping != folio->mapping) { xas_unlock_irq(&xas); continue; } - xas_set(&xas, page->index); + xas_set(&xas, folio->index); entry = xas_load(&xas); if (dax_is_locked(entry)) { rcu_read_unlock(); @@ -481,10 +481,10 @@ dax_entry_t dax_lock_page(struct page *page) return (dax_entry_t)entry; } -void dax_unlock_page(struct page *page, dax_entry_t cookie) +void dax_unlock_folio(struct folio *folio, dax_entry_t cookie) { - struct address_space *mapping = page->mapping; - XA_STATE(xas, &mapping->i_pages, page->index); + struct address_space *mapping = folio->mapping; + XA_STATE(xas, &mapping->i_pages, folio->index); if (S_ISCHR(mapping->host->i_mode)) return; diff --git a/include/linux/dax.h b/include/linux/dax.h index 261944ec0887..711deb72c109 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -159,8 +159,8 @@ int dax_writeback_mapping_range(struct address_space *mapping, struct page *dax_layout_busy_page(struct address_space *mapping); struct page *dax_layout_busy_page_range(struct address_space *mapping, loff_t start, loff_t end); -dax_entry_t dax_lock_page(struct page *page); -void dax_unlock_page(struct page *page, dax_entry_t cookie); +dax_entry_t dax_lock_folio(struct folio *folio); +void dax_unlock_folio(struct folio *folio, dax_entry_t cookie); dax_entry_t dax_lock_mapping_entry(struct address_space *mapping, unsigned long index, struct page **page); void dax_unlock_mapping_entry(struct address_space *mapping, @@ -182,14 +182,14 @@ static inline int dax_writeback_mapping_range(struct address_space *mapping, return -EOPNOTSUPP; } -static inline dax_entry_t dax_lock_page(struct page *page) +static inline dax_entry_t dax_lock_folio(struct folio *folio) { - if (IS_DAX(page->mapping->host)) + if (IS_DAX(folio->mapping->host)) return ~0UL; return 0; } -static inline void dax_unlock_page(struct page *page, dax_entry_t cookie) +static inline void dax_unlock_folio(struct folio *folio, dax_entry_t cookie) { } diff --git a/mm/memory-failure.c b/mm/memory-failure.c index fe121fdb05f7..6c2c9af0caa0 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1710,20 +1710,23 @@ static void unmap_and_kill(struct list_head *to_kill, unsigned long pfn, kill_procs(to_kill, flags & MF_MUST_KILL, false, pfn, flags); } +/* + * Only dev_pagemap pages get here, such as fsdax when the filesystem + * either do not claim or fails to claim a hwpoison event, or devdax. + * The fsdax pages are initialized per base page, and the devdax pages + * could be initialized either as base pages, or as compound pages with + * vmemmap optimization enabled. Devdax is simplistic in its dealing with + * hwpoison, such that, if a subpage of a compound page is poisoned, + * simply mark the compound head page is by far sufficient. + */ static int mf_generic_kill_procs(unsigned long long pfn, int flags, struct dev_pagemap *pgmap) { - struct page *page = pfn_to_page(pfn); + struct folio *folio = pfn_folio(pfn); LIST_HEAD(to_kill); dax_entry_t cookie; int rc = 0; - /* - * Pages instantiated by device-dax (not filesystem-dax) - * may be compound pages. - */ - page = compound_head(page); - /* * Prevent the inode from being freed while we are interrogating * the address_space, typically this would be handled by @@ -1731,11 +1734,11 @@ static int mf_generic_kill_procs(unsigned long long pfn, int flags, * also prevents changes to the mapping of this pfn until * poison signaling is complete. */ - cookie = dax_lock_page(page); + cookie = dax_lock_folio(folio); if (!cookie) return -EBUSY; - if (hwpoison_filter(page)) { + if (hwpoison_filter(&folio->page)) { rc = -EOPNOTSUPP; goto unlock; } @@ -1757,7 +1760,7 @@ static int mf_generic_kill_procs(unsigned long long pfn, int flags, * Use this flag as an indication that the dax page has been * remapped UC to prevent speculative consumption of poison. */ - SetPageHWPoison(page); + SetPageHWPoison(&folio->page); /* * Unlike System-RAM there is no possibility to swap in a @@ -1766,11 +1769,11 @@ static int mf_generic_kill_procs(unsigned long long pfn, int flags, * SIGBUS (i.e. MF_MUST_KILL) */ flags |= MF_ACTION_REQUIRED | MF_MUST_KILL; - collect_procs(page, &to_kill, true); + collect_procs(&folio->page, &to_kill, true); - unmap_and_kill(&to_kill, pfn, page->mapping, page->index, flags); + unmap_and_kill(&to_kill, pfn, folio->mapping, folio->index, flags); unlock: - dax_unlock_page(page, cookie); + dax_unlock_folio(folio, cookie); return rc; }