From patchwork Tue Apr 25 04:18:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Theodore Ts'o X-Patchwork-Id: 13222797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 818D4C77B61 for ; Tue, 25 Apr 2023 04:18:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CAED06B0071; Tue, 25 Apr 2023 00:18:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5EE66B0074; Tue, 25 Apr 2023 00:18:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B00766B0075; Tue, 25 Apr 2023 00:18:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9A9F36B0071 for ; Tue, 25 Apr 2023 00:18:44 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5D0E9C049A for ; Tue, 25 Apr 2023 04:18:44 +0000 (UTC) X-FDA: 80718607368.10.4462BFB Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by imf05.hostedemail.com (Postfix) with ESMTP id 36A44100005 for ; Tue, 25 Apr 2023 04:18:41 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=fail ("headers rsa verify failed") header.d=mit.edu header.s=outgoing header.b="HJ8qIrr/"; spf=pass (imf05.hostedemail.com: domain of tytso@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=tytso@mit.edu; dmarc=pass (policy=none) header.from=mit.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682396322; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=YZbPL+yfZ3UvCq6sEE8ugNH4HyKTMjz6JeeTlf68zBk=; b=qRI+zcJqlzTG0SmQKIu0mnyVfvNSf0hkOoYWMgk4QXAutRrOIFyIXqrQJUQPhtEZAIhz7t Vomg8cPmpsJrCS+SC0VAbEVSma9EaqsnVLPBFXEMSD3GwzuEWictldKYytY3nNZs8G8NwO ptj9pQPf/V4jZJK+9/RTEYMSFWtatWU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=fail ("headers rsa verify failed") header.d=mit.edu header.s=outgoing header.b="HJ8qIrr/"; spf=pass (imf05.hostedemail.com: domain of tytso@mit.edu designates 18.9.28.11 as permitted sender) smtp.mailfrom=tytso@mit.edu; dmarc=pass (policy=none) header.from=mit.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682396322; a=rsa-sha256; cv=none; b=YuKTzzs2geKorVX3AcnB9fYn8kVz01aVohRSHgTw3RHFjlNyj2Rj/D7P7NuWtwtedcKXEr XpdeKrZ/GWtXJYDzlXOrU/ArSpD7DvoksKPbhP5nYBZeDOn5ccLjvu474tO6lKxnj52P7y YxQo1jcqEPKKiaIMbk5ngi/dWi6rht8= Received: from cwcc.thunk.org (pool-173-48-120-46.bstnma.fios.verizon.net [173.48.120.46]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 33P4IcYp007538 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 25 Apr 2023 00:18:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1682396320; bh=YZbPL+yfZ3UvCq6sEE8ugNH4HyKTMjz6JeeTlf68zBk=; h=Date:From:To:Cc:Subject; b=HJ8qIrr/GthdjDCMCypF336LridIFVPSzOkeBYzcnvHWmTuFqbjw9mGgIUY++uMsV SJCzSk5WxgeLbF1JPbn1P/f6jLAeRbXELSLzuN7GdeWD/aWsc+Tmco02vO0arkKacr 9kbbWGG0UntrkLsrmTehlxgGbg361LwEq+HE+hYVaRoBvJzQAVeDnzwwGkly7L1rWa AXeMWTqMJuGvALQhqB7opVI9P8R7my3l0FW0L8z00VdI8cIFPLF6/YOIzRMCzRN4pE DDyzwLVMtNCV1BYFAl6Mhl2uTpNkDBcEPt+giuVqIRV7R8jUsczkGwW+eeCI7FJB2D YGYTto73g08ZA== Received: by cwcc.thunk.org (Postfix, from userid 15806) id A61DB15C5464; Tue, 25 Apr 2023 00:18:38 -0400 (EDT) Date: Tue, 25 Apr 2023 00:18:38 -0400 From: "Theodore Ts'o" To: Linus Torvalds Cc: linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [GIT PULL] ext4 changes for the 6.4 merge window Message-ID: <20230425041838.GA150312@mit.edu> MIME-Version: 1.0 Content-Disposition: inline X-Stat-Signature: w8wcajf6rp4ydbrqb6x81ougq878czot X-Rspam-User: X-Rspamd-Queue-Id: 36A44100005 X-Rspamd-Server: rspam06 X-HE-Tag: 1682396321-813535 X-HE-Meta: U2FsdGVkX19kSS0qaDf+Ri3u3elbngTgs6HBMOVHQACogTsAMn9BXA2UH3fy/eIWTbCwSYIc/8k40ER4coQinprD+eUzCf+y81APze72d00F+mFNhUapHYmABnO8vk1xEKHtdOXH0ZPu3P2Vb1H831ii1KiLD31wcDNzFKNsmOTeko9/x3xpt2lZtK7nHNeAyIEteoz1jU33d+z+Cpi59uNqxS+/dpFcV+E7A+f/Jq20ZOdxKLniy7buFbNt0+WWPAbIHgJ5kMvhQ3cbB+sRXTCmtN26DnrYTPfRj2nBFO5GorzhLdSuCJCrCoQY/XXTiyWOxEgTZp4MKwwOX3tF2ZnwCzbqS6vm66jOogN/hN1372kC+StKwwjESUwXR3ISyzmtEdLvFLHd1FOkrA/j70VPbFhUhcbXCrMlt2u3IUmPMb3StmknNHVgzS2qdgnRCPKFKNH0kzl8LbzMB7NyobZidtsKv1dgMIt4Amiuf9DBDfX+r6lDfaJxXB4jIl5UODYzhsjP6YO/XHx5AFO1i9L5V0KOMgjItaGdSCyXZS5nkDC9MxrEJ+As1x7gHP3ocgEcA6tmSQ5pr+UTO/JP9874Du24zdH/uMeEbKU4vWLyuJ8DTMXszJHZZLV782obM+rRuZ6x2vxgZZlHjmhVMLPlcjjwunD8NLhfmNTH+vzuIkajh2SuZosRXnkFKmdL1UtAWj2pjQkcpxS95LRIKmCAgiZGWc2XWpa/WCWjEduKXZrwbVjNTMMazS5hz0tcJ+e9br4NAsuY3DkN7Ptn+7xQ4xepNfWpuLxKGPgub2E7IdyatBvKyDbuBA+A1gMFNjYeqtxaXOXrowdGQQAADPUR7vCjZjZiTU7siqH1wlxRkmh1FelBcEXYtzYKJDvGsEKoqNxsWMxzPxazbUkFXzUZSiUlfCeOmnHJEdhMa5abstsTuOVar5MPmEhXNvVXk63bx49poXUTaSPkxR5 IG4Sz0J5 T34Wlf5F5FiCGQgj0Nnp9bWN44erLlKkivtHidrhxxrJ6hrSzHJnN7DZVFg/LakHStcxeI8+ouRkUVwprSRfnreK4oGhfUDtjmoAQlnUIVT7AWdj9mxtuXGns4qvtJ2lP8sj/zLPDQcW3cWrQF89lGP+QhZCBPGzBg0FUcA2rz/QracIYNBpjbkhraPRsortGqPw3Dc2+LVLGns6iHLbAAVvj2XZ+kBtpJx5PEWlEfPAYyZis4xVOaPisrtBm416/wNQ1x+/eFHh07SSiNbh/MhOUq0bRlNsEB7kEuHsfdOZHSwxg8L79sttbRIX4RwkP4Ka51Pp2bdpCHqyJ/0FXcB7ceA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The following changes since commit e8d018dd0257f744ca50a729e3d042cf2ec9da65: Linux 6.3-rc3 (2023-03-19 13:27:55 -0700) are available in the Git repository at: https://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git tags/ext4_for_linus for you to fetch changes up to 519fe1bae7e20fc4e7f179d50b6102b49980e85d: ext4: Add a uapi header for ext4 userspace APIs (2023-04-19 23:39:42 -0400) ------- Please note that after merging the mm and ext4 trees you will need to apply the patch found here[1]. [1] https://lore.kernel.org/r/20230419120923.3152939-1-willy@infradead.org This is due to a patch in the mm tree, "mm: return an ERR_PTR from __filemap_get_folio" changing that function to returning an ERR_PTR instead of returning NULL on an error. Because we have a lot of folio changes flowing in various trees, this patch can't be applied to either the ext4 or mm trees, but only *after* the two trees are merged together. There are also some minor merge conflicts again caused by other folio changes. Commit 5854356ab539 from linux-next's next-20230424 has the merge conflicts resolved as well as the patch in [1]. For more details/discussion, please see [2] and the prediction in [3], "I'm sure Linus will love it". :-P [2] https://lore.kernel.org/r/ZD6WNSKCjGGEFLB3@casper.infradead.org [3] https://lore.kernel.org/r/ZD8F3qV6eLHZpagX@casper.infradead.org ---------------------------------------------------------------- There are a number of major cleanups in ext4 this cycle: * The data=journal writepath has been significantly cleaned up and simplified, and reduces a large number of data=journal special cases by Jan Kara. * Ojaswin Muhoo has replaced linked list used to track extents that have been used for inode preallocation with a red-black tree in the multi-block allocator. This improves performance for workloads which do a large number of random allocating writes. * Thanks to Kemeng Shi for a lot of cleanup and bug fixes in the multi-block allocator. * Matthew wilcox has converted the code paths for reading and writing ext4 pages to use folios. * Jason Yan has continued to factor out ext4_fill_super() into smaller functions for improve ease of maintenance and comprehension. * Josh Triplett has created an uapi header for ext4 userspace API's. ---------------------------------------------------------------- Jan Kara (21): ext4: Update stale comment about write constraints ext4: Use nr_to_write directly in mpage_prepare_extent_to_map() ext4: Mark page for delayed dirtying only if it is pinned ext4: Don't unlock page in ext4_bio_write_page() ext4: Move page unlocking out of mpage_submit_page() ext4: Move mpage_page_done() calls after error handling ext4: Convert data=journal writeback to use ext4_writepages() ext4: Fix warnings when freezing filesystem with journaled data jdb2: Don't refuse invalidation of already invalidated buffers ext4: Mark pages with journalled data dirty ext4: Keep pages with journalled data dirty ext4: Clear dirty bit from pages without data to write ext4: Commit transaction before writing back pages in data=journal mode ext4: Drop special handling of journalled data from ext4_sync_file() ext4: Drop special handling of journalled data from extent shifting operations ext4: Fix special handling of journalled data from extent zeroing ext4: Drop special handling of journalled data from ext4_evict_inode() ext4: Drop special handling of journalled data from ext4_quota_on() ext4: Simplify handling of journalled data in ext4_bmap() ext4: Update comment in mpage_prepare_extent_to_map() Revert "ext4: Fix warnings when freezing filesystem with journaled data" Jason Yan (8): ext4: factor out ext4_hash_info_init() ext4: factor out ext4_percpu_param_init() and ext4_percpu_param_destroy() ext4: use ext4_group_desc_free() in ext4_put_super() to save some duplicated code ext4: factor out ext4_flex_groups_free() ext4: rename two functions with 'check' ext4: move s_reserved_gdt_blocks and addressable checking into ext4_check_geometry() ext4: factor out ext4_block_group_meta_init() ext4: move dax and encrypt checking into ext4_check_feature_compatibility() Josh Triplett (1): ext4: Add a uapi header for ext4 userspace APIs Kemeng Shi (33): ext4: properly handle error of ext4_init_block_bitmap in ext4_read_block_bitmap_nowait ext4: correct validation check of inode table in ext4_valid_block_bitmap ext4: call ext4_bg_num_gdb_[no]meta directly in ext4_num_base_meta_clusters ext4: remove unnecessary check in ext4_bg_num_gdb_nometa ext4: remove stale comment in ext4_init_block_bitmap ext4: stop trying to verify just initialized bitmap in ext4_read_block_bitmap_nowait ext4: improve inode table blocks counting in ext4_num_overhead_clusters ext4: remove unused group parameter in ext4_inode_bitmap_csum_verify ext4: remove unused group parameter in ext4_inode_bitmap_csum_set ext4: remove unused group parameter in ext4_block_bitmap_csum_verify ext4: remove unused group parameter in ext4_block_bitmap_csum_set ext4: set goal start correctly in ext4_mb_normalize_request ext4: allow to find by goal if EXT4_MB_HINT_GOAL_ONLY is set ext4: get correct ext4_group_info in ext4_mb_prefetch_fini ext4: correct calculation of s_mb_preallocated ext4: correct start of used group pa for debug in ext4_mb_use_group_pa ext4: protect pa->pa_free in ext4_discard_allocated_blocks ext4: add missed brelse in ext4_free_blocks_simple ext4: remove unused return value of ext4_mb_try_best_found and ext4_mb_free_metadata ext4: Remove unnecessary release when memory allocation failed in ext4_mb_init_cache ext4: remove unnecessary e4b->bd_buddy_page check in ext4_mb_load_buddy_gfp ext4: remove unnecessary check in ext4_mb_new_blocks ext4: remove dead check in mb_buddy_mark_free ext4: remove ac->ac_found > sbi->s_mb_min_to_scan dead check in ext4_mb_check_limits ext4: use best found when complex scan of group finishs ext4: remove unnecessary exit_meta_group_info tag ext4: remove unnecessary count2 in ext4_free_data_in_buddy ext4: remove unnecessary goto in ext4_mb_mark_diskspace_used ext4: remove repeat assignment to ac_f_ex ext4: remove comment code ext4_discard_preallocations ext4: simplify calculation of blkoff in ext4_mb_new_blocks_simple ext4: fix typos in mballoc ext4: avoid unnecessary pointer dereference in ext4_mb_normalize_request Matthew Wilcox (29): fs: Add FGP_WRITEBEGIN fscrypt: Add some folio helper functions ext4: Convert ext4_bio_write_page() to use a folio ext4: Convert ext4_finish_bio() to use folios ext4: Turn mpage_process_page() into mpage_process_folio() ext4: Convert mpage_submit_page() to mpage_submit_folio() ext4: Convert mpage_page_done() to mpage_folio_done() ext4: Convert ext4_bio_write_page() to ext4_bio_write_folio() ext4: Convert ext4_readpage_inline() to take a folio ext4: Convert ext4_convert_inline_data_to_extent() to use a folio ext4: Convert ext4_try_to_write_inline_data() to use a folio ext4: Convert ext4_da_convert_inline_data_to_extent() to use a folio ext4: Convert ext4_da_write_inline_data_begin() to use a folio ext4: Convert ext4_read_inline_page() to ext4_read_inline_folio() ext4: Convert ext4_write_inline_data_end() to use a folio ext4: Convert ext4_write_begin() to use a folio ext4: Convert ext4_write_end() to use a folio ext4: Use a folio in ext4_journalled_write_end() ext4: Convert ext4_journalled_zero_new_buffers() to use a folio ext4: Convert __ext4_block_zero_page_range() to use a folio ext4: Convert ext4_page_nomap_can_writeout to ext4_folio_nomap_can_writeout ext4: Use a folio in ext4_da_write_begin() ext4: Convert ext4_mpage_readpages() to work on folios ext4: Convert ext4_block_write_begin() to take a folio ext4: Use a folio in ext4_page_mkwrite() ext4: Use a folio iterator in __read_end_io() ext4: Convert mext_page_mkuptodate() to take a folio ext4: Convert pagecache_read() to use a folio ext4: Use a folio in ext4_read_merkle_tree_page Ojaswin Mujoo (9): ext4: Stop searching if PA doesn't satisfy non-extent file ext4: Refactor code related to freeing PAs ext4: Refactor code in ext4_mb_normalize_request() and ext4_mb_use_preallocated() ext4: Move overlap assert logic into a separate function ext4: Abstract out overlap fix/check logic in ext4_mb_normalize_request() ext4: Fix best extent lstart adjustment logic in ext4_mb_new_inode_pa() ext4: Convert pa->pa_inode_list and pa->pa_obj_lock into a union ext4: Use rbtrees to manage PAs instead of inode i_prealloc_list ext4: Remove the logic to trim inode PAs Theodore Ts'o (2): ext4: fix comment: "start start" -> "start" in mpage_prepare_extent_to_map() ext4: convert some BUG_ON's in mballoc to use WARN_RATELIMITED instead Tom Rix (1): ext4: remove unneeded check of nr_to_submit wuchi (1): ext4: remove useless conditional branch code Documentation/admin-guide/ext4.rst | 3 - MAINTAINERS | 1 + block/bio.c | 1 + fs/ext4/balloc.c | 124 ++++----- fs/ext4/bitmap.c | 13 +- fs/ext4/ext4.h | 114 +------- fs/ext4/extents.c | 35 +-- fs/ext4/fsync.c | 11 - fs/ext4/ialloc.c | 14 +- fs/ext4/inline.c | 171 ++++++------ fs/ext4/inode.c | 810 +++++++++++++++++++++----------------------------------- fs/ext4/mballoc.c | 691 +++++++++++++++++++++++++++++------------------ fs/ext4/mballoc.h | 17 +- fs/ext4/move_extent.c | 33 +-- fs/ext4/page-io.c | 116 ++++---- fs/ext4/readpage.c | 72 +++-- fs/ext4/resize.c | 7 +- fs/ext4/super.c | 413 +++++++++++++++-------------- fs/ext4/sysfs.c | 2 - fs/ext4/verity.c | 30 +-- fs/iomap/buffered-io.c | 2 +- fs/jbd2/transaction.c | 3 + fs/netfs/buffered_read.c | 3 +- fs/nfs/file.c | 12 +- include/linux/fscrypt.h | 21 ++ include/linux/page-flags.h | 5 - include/linux/pagemap.h | 2 + include/trace/events/ext4.h | 7 - include/uapi/linux/ext4.h | 117 ++++++++ mm/folio-compat.c | 4 +- 30 files changed, 1413 insertions(+), 1441 deletions(-) create mode 100644 include/uapi/linux/ext4.h