From patchwork Wed Jun 28 10:48:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13295553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41DF4EB64D7 for ; Wed, 28 Jun 2023 10:49:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A53368D0003; Wed, 28 Jun 2023 06:49:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A03118D0001; Wed, 28 Jun 2023 06:49:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A3D88D0003; Wed, 28 Jun 2023 06:49:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7B9D48D0001 for ; Wed, 28 Jun 2023 06:49:11 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 04E4C140DAE for ; Wed, 28 Jun 2023 10:49:10 +0000 (UTC) X-FDA: 80951834502.27.11F0B34 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf07.hostedemail.com (Postfix) with ESMTP id A66BE40016 for ; Wed, 28 Jun 2023 10:49:07 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cc8z2A57; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687949347; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TOeQPebD91AEi5r7Bvbf8K+TmmzdkyidP0hZg85tDnQ=; b=oNWxNRpArcSFCzGIiLX6oDZxKpIWfNV4ZLatJqHAA/3godoUMuoY/XzYbCY5vM1lIf/BMD CNGfHIWfIXaCNys2l6cMKRKdjiz83WPNeK6lCj2B9lgggmkh6Movs5ao0DUIkL+V9do/G+ L4Trho5RgNpL1r7C7N4TAJgwb89M1rA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cc8z2A57; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687949347; a=rsa-sha256; cv=none; b=HZ3YiEeJ7ZASV0qVlzRMmPrq7IdIX3TGXX3Yq0g34iLKhEi4UirHRX40hrQS3GV1utsk1p KVMw2++WGibi9ZJuOCx+XwNHHk4Z771whCjKci4bKBteEaGIDQXvASconxFa+F3M/vqNly UckcJ2mUhzJBCVSZplydFIWsNR/hbLM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687949346; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TOeQPebD91AEi5r7Bvbf8K+TmmzdkyidP0hZg85tDnQ=; b=cc8z2A57juF3ao1E5QB0/sHI19T2zIiziGvB95KI3VDLMHfLcJVv5pOv9hI46AX7/bCTUM VkvLXSF0nmJ2CetbjEYQpLV8CnCiI/+ZfASDNU4RLeQnMvpU+K02fnxd4HOOFPLp8Idij2 udG7st50RPF4k95s876qZ4Iz8CYuoD8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-618-L4wgRJtDMGublEOWLuvnxw-1; Wed, 28 Jun 2023 06:49:01 -0400 X-MC-Unique: L4wgRJtDMGublEOWLuvnxw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3363E85A58A; Wed, 28 Jun 2023 10:49:00 +0000 (UTC) Received: from warthog.procyon.org.uk.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6153F40C2063; Wed, 28 Jun 2023 10:48:57 +0000 (UTC) From: David Howells To: Andrew Morton Cc: David Howells , Matthew Wilcox , Linus Torvalds , Jeff Layton , Christoph Hellwig , linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net, linux-erofs@lists.ozlabs.org, linux-ext4@vger.kernel.org, linux-cachefs@redhat.com, linux-fsdevel@vger.kernel.org, Rohith Surabattula , Steve French , Shyam Prasad N , Dave Wysochanski , Dominique Martinet , Ilya Dryomov , "Theodore Ts'o" , Andreas Dilger , linux-mm@kvack.org Subject: [PATCH v7 1/2] mm: Merge folio_has_private()/filemap_release_folio() call pairs Date: Wed, 28 Jun 2023 11:48:51 +0100 Message-ID: <20230628104852.3391651-2-dhowells@redhat.com> In-Reply-To: <20230628104852.3391651-1-dhowells@redhat.com> References: <20230628104852.3391651-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Queue-Id: A66BE40016 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: m74gsuxuzpendkiu5mbbi9ynduxez3rs X-HE-Tag: 1687949347-807656 X-HE-Meta: U2FsdGVkX18X0kXV+PoRM2nSpOFXkOBrUS4gBveggmF36KbWpMDM9KIN66n9cxUn7rzG/DbtOoeRem+/C72hMv48N5L9dFw51SumMolEwl/xwcHYxqy6lZgiQW5wi5zU7vP2Uf355LEziAQFHyWP0q8XUeIkOu7F7Axgs2+NokWO4fYpk+rC3T74bxiSDNXgAF8na31rZVTE19YRfvkZe0ceCufD0UUEX4B6THxurkhtPvj4Id3UCGmYo66b07hU5xVKyGtLwW131RIrImxYl8IYvAeVGbi4pWHo0TvAkPa/zmLC94uXXlUJtcOV0YuK6aOHszP4KvGPpIX1QIN17vyl4o2IkqU9NUxpxg0tD8HOy6l8kbABfoaGzXZroFygs+uNsASNOch+Vqj0siWQJmuhHizVW6revWIxx+Bbmfx8+tx329zDP1OH9N8IByxPTHzFqDCxfhI5Xwlj+DzegtFLX3ZaumNl8Z5neMSd8Pvd4LYtHWdx/HGRxfnfyuW32CrHwzi7srZs+F5j1EkLZMJ1wLZQ1EROnsjyXZT5M7vCoVv8dr1FVlWa0e2vMSWdS+Dvy7nSHby6yQUI4Rj0fFUApAJ2iDMt0w15sMHQmeqU1wxgf9PwK4TuCavvOV7yRtIp2hqtSn9evvydZp5PBuUU+HS+Z6r+wR+2WVX08tdJg2kzBJLXUjZBwmVmgZ3q3bCUQEKIYfHhZHm4s1wrBrpVM71NcGTiEw1AuMmicaOhUI722BZOkt+gZX6BK7YYli0CJq2H/xEGWETnB7D/LiPxaXAmwD9w8ku7cveRIXwWPfZAdcHIUC2vNMGmDck5VBnU8PiNFi7LDqSQr6tgMKV242zjul+IpQMYTwHW4t2EaaICC9MZT9Bse1baFzqsnuh6bkoDybqIcYLMZU1OokCt+eUoPH5Be6zN3buW6SWJ41n8VKi7FRNlkq3wFg1O0tPBvDw/2GZAkw4Z3D/ GYpx90hv KBmQZe+fKl4U8gbvPZI+jXmG+kCpgoMOvIYjvmwBWEuRMAG1y04Efj3Rnlu+bg6WFPG7Or6npij2iE7wPwJKqH3l2LJR5DHZGmJf+IyMOmOOB4qQS2UDOpWhMghBTx76lTnOQ+LJfjAoL0Yn5NVT7MM3Ga87XRQgvyM+Nmss7hX4vyqV3yDlmr6wA+FXeG1nMTCZ1dPWlIW4NzqLQYt/yv3N1OpzgrnwbfawF9CfU86b9zOPG9RzOS5eJCoBMHcW0bt+pJsbjX3+dGYGEn9oqZMBaqIHEsfx5ZrqzvJ8lC3wuDdn1rQ7+J8YZj6aX7ljxhDOaK+jVWQVPyfBOPJCQjOvich+yFb69o2X0SU/N5WlHjGDk1rPtlrfY9+LrGavm14VooOl8kQK/H3m3UKI3ELggLMWgDDsMZUCNDK1LbVDziTIF5exbsUKHzkYEHYVwT3t2n/l1KUw/3TG/eA1KYjlNUQ9yKnm6OW/mnQ6bE9czeCjJtRfleXDZqcoz/BNniM9qljUVY8Uh7mpb8iaM0fG3ewvd1/XnOfDaT/klnD6IgGatrdbDsZ5uBXRFlnPE8xNPT8/E1uAJ+C1Wpbod0DLGUr7+FCTHk30Q0tyaAOlhOxjDpP9KMvsvlr8DNnKOoA/YzXLA/nFvo/45yn8WgK4T9ysE5/+QQ5BtTXfkRqWzSwOY43ITzduIK0gAoBIBJ4UABYPLBCIYbfRkXdGu2dAode2HlaqUER+5AZuJdytX0Mzqu7FzvWxuYt2UIMkFvOdy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make filemap_release_folio() check folio_has_private(). Then, in most cases, where a call to folio_has_private() is immediately followed by a call to filemap_release_folio(), we can get rid of the test in the pair. There are a couple of sites in mm/vscan.c that this can't so easily be done. In shrink_folio_list(), there are actually three cases (something different is done for incompletely invalidated buffers), but filemap_release_folio() elides two of them. In shrink_active_list(), we don't have have the folio lock yet, so the check allows us to avoid locking the page unnecessarily. A wrapper function to check if a folio needs release is provided for those places that still need to do it in the mm/ directory. This will acquire additional parts to the condition in a future patch. After this, the only remaining caller of folio_has_private() outside of mm/ is a check in fuse. Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: "Theodore Ts'o" cc: Andreas Dilger cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-afs@lists.infradead.org cc: v9fs-developer@lists.sourceforge.net cc: ceph-devel@vger.kernel.org cc: linux-nfs@vger.kernel.org cc: linux-ext4@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- Notes: ver #5) - Rebased on linus/master. try_to_release_page() has now been entirely replaced by filemap_release_folio(), barring one comment. - Cleaned up some pairs in ext4. ver #4) - Split from fscache fix. - Moved folio_needs_release() to mm/internal.h and removed open-coded version from filemap_release_folio(). ver #3) - Fixed mapping_clear_release_always() to use clear_bit() not set_bit(). - Moved a '&&' to the correct line. ver #2) - Rewrote entirely according to Willy's suggestion[1]. fs/ext4/move_extent.c | 12 ++++-------- fs/splice.c | 3 +-- mm/filemap.c | 2 ++ mm/huge_memory.c | 3 +-- mm/internal.h | 8 ++++++++ mm/khugepaged.c | 3 +-- mm/memory-failure.c | 8 +++----- mm/migrate.c | 3 +-- mm/truncate.c | 6 ++---- mm/vmscan.c | 8 ++++---- 10 files changed, 27 insertions(+), 29 deletions(-) diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c index b5af2fc03b2f..251584a23d05 100644 --- a/fs/ext4/move_extent.c +++ b/fs/ext4/move_extent.c @@ -340,10 +340,8 @@ move_extent_per_page(struct file *o_filp, struct inode *donor_inode, ext4_double_up_write_data_sem(orig_inode, donor_inode); goto data_copy; } - if ((folio_has_private(folio[0]) && - !filemap_release_folio(folio[0], 0)) || - (folio_has_private(folio[1]) && - !filemap_release_folio(folio[1], 0))) { + if (!filemap_release_folio(folio[0], 0) || + !filemap_release_folio(folio[1], 0)) { *err = -EBUSY; goto drop_data_sem; } @@ -362,10 +360,8 @@ move_extent_per_page(struct file *o_filp, struct inode *donor_inode, /* At this point all buffers in range are uptodate, old mapping layout * is no longer required, try to drop it now. */ - if ((folio_has_private(folio[0]) && - !filemap_release_folio(folio[0], 0)) || - (folio_has_private(folio[1]) && - !filemap_release_folio(folio[1], 0))) { + if (!filemap_release_folio(folio[0], 0) || + !filemap_release_folio(folio[1], 0)) { *err = -EBUSY; goto unlock_folios; } diff --git a/fs/splice.c b/fs/splice.c index 7a9565d8ec4f..6412848891df 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -82,8 +82,7 @@ static bool page_cache_pipe_buf_try_steal(struct pipe_inode_info *pipe, */ folio_wait_writeback(folio); - if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, GFP_KERNEL)) goto out_unlock; /* diff --git a/mm/filemap.c b/mm/filemap.c index 00f01d8ead47..31d07c2f8d32 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4134,6 +4134,8 @@ bool filemap_release_folio(struct folio *folio, gfp_t gfp) struct address_space * const mapping = folio->mapping; BUG_ON(!folio_test_locked(folio)); + if (!folio_needs_release(folio)) + return true; if (folio_test_writeback(folio)) return false; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 624671aaa60d..a14b3d1af9d7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2688,8 +2688,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) gfp = current_gfp_context(mapping_gfp_mask(mapping) & GFP_RECLAIM_MASK); - if (folio_test_private(folio) && - !filemap_release_folio(folio, gfp)) { + if (!filemap_release_folio(folio, gfp)) { ret = -EBUSY; goto out; } diff --git a/mm/internal.h b/mm/internal.h index e6029d94bdb2..a76314764d8c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -170,6 +170,14 @@ static inline void set_page_refcounted(struct page *page) set_page_count(page, 1); } +/* + * Return true if a folio needs ->release_folio() calling upon it. + */ +static inline bool folio_needs_release(struct folio *folio) +{ + return folio_has_private(folio); +} + extern unsigned long highest_memmap_pfn; /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 2d0d58fb4e7f..1e6e6a25cd52 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2058,8 +2058,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, goto out_unlock; } - if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_KERNEL)) { + if (!filemap_release_folio(folio, GFP_KERNEL)) { result = SCAN_PAGE_HAS_PRIVATE; folio_putback_lru(folio); goto out_unlock; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 5b663eca1f29..38321eb85fb2 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -944,14 +944,12 @@ static int truncate_error_page(struct page *p, unsigned long pfn, struct folio *folio = page_folio(p); int err = mapping->a_ops->error_remove_page(mapping, p); - if (err != 0) { + if (err != 0) pr_info("%#lx: Failed to punch page: %d\n", pfn, err); - } else if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_NOIO)) { + else if (!filemap_release_folio(folio, GFP_NOIO)) pr_info("%#lx: failed to release buffers\n", pfn); - } else { + else ret = MF_RECOVERED; - } } else { /* * If the file system doesn't support it just invalidate diff --git a/mm/migrate.c b/mm/migrate.c index 01cac26a3127..5fc27d1a7eaf 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -929,8 +929,7 @@ static int fallback_migrate_folio(struct address_space *mapping, * Buffers may be managed in a filesystem specific way. * We must have no buffers or drop them. */ - if (folio_test_private(src) && - !filemap_release_folio(src, GFP_KERNEL)) + if (!filemap_release_folio(src, GFP_KERNEL)) return mode == MIGRATE_SYNC ? -EAGAIN : -EBUSY; return migrate_folio(mapping, dst, src, mode); diff --git a/mm/truncate.c b/mm/truncate.c index 86de31ed4d32..6fb830369829 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -19,7 +19,6 @@ #include #include #include -#include /* grr. try_to_release_page */ #include #include #include "internal.h" @@ -276,7 +275,7 @@ static long mapping_evict_folio(struct address_space *mapping, if (folio_ref_count(folio) > folio_nr_pages(folio) + folio_has_private(folio) + 1) return 0; - if (folio_has_private(folio) && !filemap_release_folio(folio, 0)) + if (!filemap_release_folio(folio, 0)) return 0; return remove_mapping(mapping, folio); @@ -574,8 +573,7 @@ static int invalidate_complete_folio2(struct address_space *mapping, if (folio->mapping != mapping) return 0; - if (folio_has_private(folio) && - !filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, GFP_KERNEL)) return 0; spin_lock(&mapping->host->i_lock); diff --git a/mm/vmscan.c b/mm/vmscan.c index 5bf98d0a22c9..49bc412b31a2 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2058,7 +2058,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, * (refcount == 1) it can be freed. Otherwise, leave * the folio on the LRU so it is swappable. */ - if (folio_has_private(folio)) { + if (folio_needs_release(folio)) { if (!filemap_release_folio(folio, sc->gfp_mask)) goto activate_locked; if (!mapping && folio_ref_count(folio) == 1) { @@ -2703,9 +2703,9 @@ static void shrink_active_list(unsigned long nr_to_scan, } if (unlikely(buffer_heads_over_limit)) { - if (folio_test_private(folio) && folio_trylock(folio)) { - if (folio_test_private(folio)) - filemap_release_folio(folio, 0); + if (folio_needs_release(folio) && + folio_trylock(folio)) { + filemap_release_folio(folio, 0); folio_unlock(folio); } } From patchwork Wed Jun 28 10:48:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13295554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA00EB64DD for ; Wed, 28 Jun 2023 10:49:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 598668D0005; Wed, 28 Jun 2023 06:49:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5483C8D0001; Wed, 28 Jun 2023 06:49:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 39DBD8D0005; Wed, 28 Jun 2023 06:49:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2D3288D0001 for ; Wed, 28 Jun 2023 06:49:12 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 06BAEA0B6A for ; Wed, 28 Jun 2023 10:49:12 +0000 (UTC) X-FDA: 80951834544.19.1D28F53 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 329361A000D for ; Wed, 28 Jun 2023 10:49:09 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TUkV7kyc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687949350; a=rsa-sha256; cv=none; b=uFBQfxAyS4y/KsxzWLdRoiZnKaTOp2hzgHkB+1JjflajIrNi6Qfk6zmEe/ns9Jp3FtEx/s 2NRQGLHl4k6yQQR02m1vwYQ0g1/dDW+PguqQAJSY/xm4tvaqznt+mKZB6xftJSzCq/KERM mst0VdNCXw7Y+cgVc8NbNpgU8SAJbmE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TUkV7kyc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687949350; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=35xC16dMlQgwE6owW0vdiolQSi7usOC9ELk4zbmo8S8=; b=eb4nf84tw9rISccUsQ/qNNVrj9WNqs/O9iryKqOEfHivgOpBJrE0lNGowSSOdNgzi1E/zI Ek3hUxjsT9HhOvV2v1ldsZ9l8AYQnyhWMcFogmAwK0vUnyWHJT+lesRxC29KVi872WzGA7 pQbAWPOzzvHKtiE0hw3mIlBtDjS1aEA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687949349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=35xC16dMlQgwE6owW0vdiolQSi7usOC9ELk4zbmo8S8=; b=TUkV7kyc8Zp+Ku6ga4J19npSixVjtAFMYAl1BtKGrhQyhIckZEwmiaULNLEBWms/M/+gWK VWjTOCdgwQ7yDtYcrAfnzpwse/YtWUyp3tg4ZLqZ2BR8PkHzt3oeET776riuqDhWbEhn8G wnhj694Uqk1RJcwb1ur/EPceUu5KxNY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-178-etuXAlI0MLCPNHRgBXUBRA-1; Wed, 28 Jun 2023 06:49:04 -0400 X-MC-Unique: etuXAlI0MLCPNHRgBXUBRA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3433C3C02B6D; Wed, 28 Jun 2023 10:49:03 +0000 (UTC) Received: from warthog.procyon.org.uk.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id D68A8C09A07; Wed, 28 Jun 2023 10:49:00 +0000 (UTC) From: David Howells To: Andrew Morton Cc: David Howells , Matthew Wilcox , Linus Torvalds , Jeff Layton , Christoph Hellwig , linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net, linux-erofs@lists.ozlabs.org, linux-ext4@vger.kernel.org, linux-cachefs@redhat.com, linux-fsdevel@vger.kernel.org, Rohith Surabattula , Steve French , Shyam Prasad N , Dave Wysochanski , Dominique Martinet , Ilya Dryomov , linux-mm@kvack.org Subject: [PATCH v7 2/2] mm, netfs, fscache: Stop read optimisation when folio removed from pagecache Date: Wed, 28 Jun 2023 11:48:52 +0100 Message-ID: <20230628104852.3391651-3-dhowells@redhat.com> In-Reply-To: <20230628104852.3391651-1-dhowells@redhat.com> References: <20230628104852.3391651-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 329361A000D X-Stat-Signature: m5ioyw8fnnbjarrmf4nfryey7o4s3tj5 X-HE-Tag: 1687949349-716149 X-HE-Meta: U2FsdGVkX1+NbvQN76I/vNs9aNx0pnn+YIA8lUOSi5Y/r3/YBkHHUfdLL1vK8FxsEP1kHCszB5f0PX4T9XhuZbtJolxhAldb0JcO1jK/TrAmiVdkzUM8V73ZIpN/4kbWzN7WwtoFJsuGMm8MdTsl1+6Ax1xi3PnmsE2DehyylmI3NqqVL1JQSOKpRcqBSjakdbB9dDehgx3oDImhlQuJOwaD4B6i4D0Jdt41DYI6Fd0ZZXg1Y3hinp1TX6UPiMTCCpcla++QEScZF0ay5evQDp/qHoVLD/Aeo1vqlqIyFexb/MBs13PGqldnRndkR+S+W4Ds+Ycfksxvai5Ozjrsp/uzfTq9fCRd0zxV0wfsjkTWnE8upnCb4tmBlO1OfYY8XKnGNqwsHO3l9Czp5UOafaTt6NJVhJO3ocfM1h19dgwDZaval/reD5Ox6pygE0cUPFzp7cak0Ie7BqtGTEWKvZPi1WBYDOl+hjlgEziv4lT9rERo1Z2xBVkMvoM8sP+cbM4bbertefE/mND5Um9jIrzq2gjQgbnAAc25PKFvZlsW2J5WmyUBSJ+MiywOr8wkbnKwQuvKFaBae7wHWgYwxTpBl+/cD94aXHmRELTIaV+U/DpSIchAFjcLJGZ50TD4T5hnmg0thUknmyzCZ9DJ/tFMBKr7PYYQvvrabTVblDFVwn0vEJldkwZ2J7GBNmwnXPaArC8ZACt4udk2JFp5bRMsRjvjSPCrt/m77f3KBTkOsBQuYO0JRkKLNVm95OopMof9mW2ehCCaK5D8bYT8h1CQeMXwJ5f42aip3gwy1p2Le0bmvdt0H52FoLA336p5Vb2fCKVoJ6vl95k/NvqWCbQR/SJigvCNcaHOLE+3Ghdm6ZRR98sXIs1RqayKjbeDA6uJJCWtB0ltDrR2oqvC75I1m57KM2ATXGgvIHkPsGtppN+Qq16Sw7toSOLB3Xgjuyu3YTzjVBPFWwbgSeg QLEU958/ 1eouKjbtQQ7bjWddUM/qMLRlQoPBNKS7HWHJkwFsATE4bF2Nih1RnMxgAuvj6KbEB4Vu3pQtIloYWfRl6+Riq3fHav5F72JlP34eRrajaZuMmNm0yx/XbdIel1F+V53mO/BHNVQQZNYeb76e8J0XKR3KS8nZNgXrGVntqA0p/h6EiQLBbbvC+jOjLVU1YRUXnlkDD+m5KopIaZdv1Cm0tLQfsPIi1eV/qLSE1ELeew3P341UUyoDNy0YZP5loIVauxmughIqFqvH5YEWYNsLK4mq/SzerWm9a+WmCaJ7qMQ68aXe4W9CPzPRDsDZcjYovHAKwtJyOzgnt8aKGRwSooZ9p86e1FClcFnk8SAnV5Zwtho9OYE4ysZ15ESqJxogvGA3DqlXoTMNHtb6WncTGnrIPUrb8hhqefT8RAsN4JB/eF7U9zdJDVAD4M0VpDKKYEcVMhHqC6g86aogbBkZrkpbcFPhSgEElNsvc6EUrJ9e+5z2fBNTFZOB3AiCg+MXgCcI3QBeo3DIf8VW3pvQGami4fEg2JuXf//krCwIfl/X1lH81hv+7ZrqvM+krK4D1ig+rLqPelngWQmYyiZgMw0oX8RsijbBjWTeViHj2HVcvvE0DWSZxQyfWzYT0IoAzsKVAld0MUIMH+nw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fscache has an optimisation by which reads from the cache are skipped until we know that (a) there's data there to be read and (b) that data isn't entirely covered by pages resident in the netfs pagecache. This is done with two flags manipulated by fscache_note_page_release(): if (... test_bit(FSCACHE_COOKIE_HAVE_DATA, &cookie->flags) && test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags); where the NO_DATA_TO_READ flag causes cachefiles_prepare_read() to indicate that netfslib should download from the server or clear the page instead. The fscache_note_page_release() function is intended to be called from ->releasepage() - but that only gets called if PG_private or PG_private_2 is set - and currently the former is at the discretion of the network filesystem and the latter is only set whilst a page is being written to the cache, so sometimes we miss clearing the optimisation. Fix this by following Willy's suggestion[1] and adding an address_space flag, AS_RELEASE_ALWAYS, that causes filemap_release_folio() to always call ->release_folio() if it's set, even if PG_private or PG_private_2 aren't set. Note that this would require folio_test_private() and page_has_private() to become more complicated. To avoid that, in the places[*] where these are used to conditionalise calls to filemap_release_folio() and try_to_release_page(), the tests are removed the those functions just jumped to unconditionally and the test is performed there. [*] There are some exceptions in vmscan.c where the check guards more than just a call to the releaser. I've added a function, folio_needs_release() to wrap all the checks for that. AS_RELEASE_ALWAYS should be set if a non-NULL cookie is obtained from fscache and cleared in ->evict_inode() before truncate_inode_pages_final() is called. Additionally, the FSCACHE_COOKIE_NO_DATA_TO_READ flag needs to be cleared and the optimisation cancelled if a cachefiles object already contains data when we open it. Fixes: 1f67e6d0b188 ("fscache: Provide a function to note the release of a page") Fixes: 047487c947e8 ("cachefiles: Implement the I/O routines") Reported-by: Rohith Surabattula Suggested-by: Matthew Wilcox Signed-off-by: David Howells cc: Matthew Wilcox cc: Linus Torvalds cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Dave Wysochanski cc: Dominique Martinet cc: Ilya Dryomov cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-afs@lists.infradead.org cc: v9fs-developer@lists.sourceforge.net cc: ceph-devel@vger.kernel.org cc: linux-nfs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Tested-by: SeongJae Park --- Notes: ver #7) - Make NFS set AS_RELEASE_ALWAYS. ver #4) - Split out merging of folio_has_private()/filemap_release_folio() call pairs into a preceding patch. - Don't need to clear AS_RELEASE_ALWAYS in ->evict_inode(). ver #3) - Fixed mapping_clear_release_always() to use clear_bit() not set_bit(). - Moved a '&&' to the correct line. ver #2) - Rewrote entirely according to Willy's suggestion[1]. fs/9p/cache.c | 2 ++ fs/afs/internal.h | 2 ++ fs/cachefiles/namei.c | 2 ++ fs/ceph/cache.c | 2 ++ fs/nfs/fscache.c | 3 +++ fs/smb/client/fscache.c | 2 ++ include/linux/pagemap.h | 16 ++++++++++++++++ mm/internal.h | 5 ++++- 8 files changed, 33 insertions(+), 1 deletion(-) diff --git a/fs/9p/cache.c b/fs/9p/cache.c index cebba4eaa0b5..12c0ae29f185 100644 --- a/fs/9p/cache.c +++ b/fs/9p/cache.c @@ -68,6 +68,8 @@ void v9fs_cache_inode_get_cookie(struct inode *inode) &path, sizeof(path), &version, sizeof(version), i_size_read(&v9inode->netfs.inode)); + if (v9inode->netfs.cache) + mapping_set_release_always(inode->i_mapping); p9_debug(P9_DEBUG_FSC, "inode %p get cookie %p\n", inode, v9fs_inode_cookie(v9inode)); diff --git a/fs/afs/internal.h b/fs/afs/internal.h index 9d3d64921106..da73b97e19a9 100644 --- a/fs/afs/internal.h +++ b/fs/afs/internal.h @@ -681,6 +681,8 @@ static inline void afs_vnode_set_cache(struct afs_vnode *vnode, { #ifdef CONFIG_AFS_FSCACHE vnode->netfs.cache = cookie; + if (cookie) + mapping_set_release_always(vnode->netfs.inode.i_mapping); #endif } diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c index d9d22d0ec38a..7bf7a5fcc045 100644 --- a/fs/cachefiles/namei.c +++ b/fs/cachefiles/namei.c @@ -585,6 +585,8 @@ static bool cachefiles_open_file(struct cachefiles_object *object, if (ret < 0) goto check_failed; + clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &object->cookie->flags); + object->file = file; /* Always update the atime on an object we've just looked up (this is diff --git a/fs/ceph/cache.c b/fs/ceph/cache.c index 177d8e8d73fe..de1dee46d3df 100644 --- a/fs/ceph/cache.c +++ b/fs/ceph/cache.c @@ -36,6 +36,8 @@ void ceph_fscache_register_inode_cookie(struct inode *inode) &ci->i_vino, sizeof(ci->i_vino), &ci->i_version, sizeof(ci->i_version), i_size_read(inode)); + if (ci->netfs.cache) + mapping_set_release_always(inode->i_mapping); } void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info *ci) diff --git a/fs/nfs/fscache.c b/fs/nfs/fscache.c index 8c35d88a84b1..b05717fe0d4e 100644 --- a/fs/nfs/fscache.c +++ b/fs/nfs/fscache.c @@ -180,6 +180,9 @@ void nfs_fscache_init_inode(struct inode *inode) &auxdata, /* aux_data */ sizeof(auxdata), i_size_read(inode)); + + if (netfs_inode(inode)->cache) + mapping_set_release_always(inode->i_mapping); } /* diff --git a/fs/smb/client/fscache.c b/fs/smb/client/fscache.c index 8f6909d633da..3677525ee993 100644 --- a/fs/smb/client/fscache.c +++ b/fs/smb/client/fscache.c @@ -108,6 +108,8 @@ void cifs_fscache_get_inode_cookie(struct inode *inode) &cifsi->uniqueid, sizeof(cifsi->uniqueid), &cd, sizeof(cd), i_size_read(&cifsi->netfs.inode)); + if (cifsi->netfs.cache) + mapping_set_release_always(inode->i_mapping); } void cifs_fscache_unuse_inode_cookie(struct inode *inode, bool update) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index a56308a9d1a4..a1176ceb4a0c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -199,6 +199,7 @@ enum mapping_flags { /* writeback related tags are not used */ AS_NO_WRITEBACK_TAGS = 5, AS_LARGE_FOLIO_SUPPORT = 6, + AS_RELEASE_ALWAYS, /* Call ->release_folio(), even if no private data */ }; /** @@ -269,6 +270,21 @@ static inline int mapping_use_writeback_tags(struct address_space *mapping) return !test_bit(AS_NO_WRITEBACK_TAGS, &mapping->flags); } +static inline bool mapping_release_always(const struct address_space *mapping) +{ + return test_bit(AS_RELEASE_ALWAYS, &mapping->flags); +} + +static inline void mapping_set_release_always(struct address_space *mapping) +{ + set_bit(AS_RELEASE_ALWAYS, &mapping->flags); +} + +static inline void mapping_clear_release_always(struct address_space *mapping) +{ + clear_bit(AS_RELEASE_ALWAYS, &mapping->flags); +} + static inline gfp_t mapping_gfp_mask(struct address_space * mapping) { return mapping->gfp_mask; diff --git a/mm/internal.h b/mm/internal.h index a76314764d8c..86aef26df905 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -175,7 +175,10 @@ static inline void set_page_refcounted(struct page *page) */ static inline bool folio_needs_release(struct folio *folio) { - return folio_has_private(folio); + struct address_space *mapping = folio->mapping; + + return folio_has_private(folio) || + (mapping && mapping_release_always(mapping)); } extern unsigned long highest_memmap_pfn;