| Message ID | 20210219041244.GZ7193@magnolia (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL] xfs: new code for 5.12 | expand |
The pull request you sent on Thu, 18 Feb 2021 20:12:44 -0800:
> git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-5.12-merge-5
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/b52bb135aad99deea9bfe5f050c3295b049adc87
Thank you!
diff --cc fs/xfs/xfs_inode.c index 636ac13b1df2,95b7f2ba4e06..000000000000 --- a/fs/xfs/xfs_inode.c diff --cc fs/xfs/xfs_ioctl.c index 248083ea0276,3d4c7ca080fb..000000000000 --- a/fs/xfs/xfs_ioctl.c diff --cc fs/xfs/xfs_inode.c index 636ac13b1df2,95b7f2ba4e06..000000000000 --- a/fs/xfs/xfs_inode.c
Hi Linus, Please pull the following branch containing all the new xfs code for 5.12. There's a lot going on this time, which seems about right for this drama-filled year. Community developers added some code to speed up freezing when read-only workloads are still running, refactored the logging code, added checks to prevent file extent counter overflow, reduced iolock cycling to speed up fsync and gc scans, and started the slow march towards supporting filesystem shrinking. There's a huge refactoring of the internal speculative preallocation garbage collection code which fixes a bunch of bugs, makes the gc scheduling per-AG and hence multithreaded, and standardizes the retry logic when we try to reserve space or quota, can't, and want to trigger a gc scan. We also enable multithreaded quotacheck to reduce mount times further. This is also preparation for background file gc, which may or may not land for 5.13. We also fixed some deadlocks in the rename code, fixed a quota accounting leak when FSSETXATTR fails, restored the behavior that write faults to an mmap'd region actually cause a SIGBUS, fixed a bug where sgid directory inheritance wasn't quite working properly, and fixed a bug where symlinks weren't working properly in ecryptfs. We also now advertise the inode btree counters feature that was introduced two cycles ago. This branch merges cleanly with 5.11, but there were a few merge conflicts with the pidfd tree that Stephen Rothwell noticed in for-next. Christian Brauner is trying to create per-mount id mappings, which apparently requires passing the per-mount user namespace deep into the filesystems, either directly or through struct files. The first conflict arises from Christoph's fix for gid inheritance; I think it can be resolved as follows: +++ b/fs/xfs/xfs_inode.c @@@ -809,13 -810,13 +810,13 @@@ xfs_init_new_inode inode->i_rdev = rdev; ip->i_d.di_projid = prid; - if (pip && XFS_INHERIT_GID(pip)) { - inode->i_gid = VFS_I(pip)->i_gid; - if ((VFS_I(pip)->i_mode & S_ISGID) && S_ISDIR(mode)) - inode->i_mode |= S_ISGID; + if (dir && !(dir->i_mode & S_ISGID) && + (mp->m_flags & XFS_MOUNT_GRPID)) { - inode->i_uid = current_fsuid(); ++ inode->i_uid = fsuid_into_mnt(mnt_userns); + inode->i_gid = dir->i_gid; + inode->i_mode = mode; } else { - inode_init_owner(inode, dir, mode); - inode->i_gid = fsgid_into_mnt(mnt_userns); ++ inode_init_owner(mnt_userns, inode, dir, mode); } /* I think the important bits here are making sure the previous current_fs[ug]id() calls get turned into fs[ug]id_into_mnt() calls, and making sure the mnt_userns pointer gets passed to inode_init_owner(). The second conflict involves the quota reservation rework patchset, and I think it can be resolved as follows: +++ b/fs/xfs/xfs_ioctl.c @@@ -1275,9 -1280,9 +1280,10 @@@ xfs_ioctl_setattr_prepare_dax */ static struct xfs_trans * xfs_ioctl_setattr_get_trans( - struct xfs_inode *ip, - struct file *file) ++ struct file *file, + struct xfs_dquot *pdqp) { + struct xfs_inode *ip = XFS_I(file_inode(file)); struct xfs_mount *mp = ip->i_mount; struct xfs_trans *tp; int error = -EROFS; @@@ -1461,9 -1470,9 +1469,9 @@@ xfs_ioctl_setattr xfs_ioctl_setattr_prepare_dax(ip, fa); - tp = xfs_ioctl_setattr_get_trans(ip, pdqp); - tp = xfs_ioctl_setattr_get_trans(file); ++ tp = xfs_ioctl_setattr_get_trans(file, pdqp); if (IS_ERR(tp)) { - code = PTR_ERR(tp); + error = PTR_ERR(tp); goto error_free_dquots; } @@@ -1599,7 -1615,7 +1606,7 @@@ xfs_ioc_setxflags xfs_ioctl_setattr_prepare_dax(ip, &fa); - tp = xfs_ioctl_setattr_get_trans(ip, NULL); - tp = xfs_ioctl_setattr_get_trans(filp); ++ tp = xfs_ioctl_setattr_get_trans(filp, NULL); if (IS_ERR(tp)) { error = PTR_ERR(tp); goto out_drop_write; Mr. Brauner swapped the xfs_inode pointer in the first argument of xfs_ioctl_setattr_get_trans for a struct file, and I added a second argument to pass a xfs_dquot that we're making reservations against into the get_trans function. The rest of the diff updates the callsite parameters. After the merge, the function signature should be: static struct xfs_trans * xfs_ioctl_setattr_get_trans( struct file *file, struct xfs_dquot *pdqp) {...} The third conflict is also from the quota rework patchset, and (AFAICT) auto-resolved like this: +++ b/fs/xfs/xfs_inode.c @@@ -1159,12 -1167,16 +1166,12 @@@ xfs_create_tmpfile resblks = XFS_IALLOC_SPACE_RES(mp); tres = &M_RES(mp)->tr_create_tmpfile; - error = xfs_trans_alloc(mp, tres, resblks, 0, 0, &tp); + error = xfs_trans_alloc_icreate(mp, tres, udqp, gdqp, pdqp, resblks, + &tp); if (error) - goto out_release_inode; - - error = xfs_trans_reserve_quota(tp, mp, udqp, gdqp, - pdqp, resblks, 1, 0); - if (error) - goto out_trans_cancel; + goto out_release_dquots; - error = xfs_dir_ialloc(&tp, dp, mode, 0, 0, prid, &ip); + error = xfs_dir_ialloc(mnt_userns, &tp, dp, mode, 0, 0, prid, &ip); if (error) goto out_trans_cancel; All that is going on here is adding the mnt_userns parameter as the first argument to xfs_dir_ialloc; I think the only reason my test merge noticed it is because it's adjacent to a different change that I made. With those pieces fixed up, the tree builds and seems to pass the simple fstest run that I did. Please let me know if anything else strange happens during the merge process, particularly since there usually aren't merge conflicts. :) I will probably follow this up in a day or two with a couple more fixes that have trickled in, but I am still catching up after the same ice storm that knocked you in the dark(?) last Sunday also knocked me offline until yesterday afternoon. It's a bit disconcerting that a single evening's ice storm could cut power to 10% of the state's population and overload the cell phone networks to the point of unusability. --D The following changes since commit 19c329f6808995b142b3966301f217c831e7cf31: Linux 5.11-rc4 (2021-01-17 16:37:05 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-5.12-merge-5 for you to fetch changes up to 1cd738b13ae9b29e03d6149f0246c61f76e81fcf: xfs: consider shutdown in bmapbt cursor delete assert (2021-02-11 08:46:38 -0800) ---------------------------------------------------------------- New code for 5.12: - Fix an ABBA deadlock when renaming files on overlayfs. - Make sure that we can't overflow the inode extent counters when adding to or removing extents from a file. - Make directory sgid inheritance work the same way as all the other filesystems. - Don't drain the buffer cache on freeze and ro remount, which should reduce the amount of time if read-only workloads are continuing during the freeze. - Fix a bug where symlink size isn't reported to the vfs in ecryptfs. - Disentangle log cleaning from log covering. This refactoring sets us up for future changes to the log, though for now it simply means that we can use covering for freezes, and cleaning becomes something we only do at unmount. - Speed up file fsyncs by reducing iolock cycling. - Fix delalloc blocks leaking when changing the project id fails because of input validation errors in FSSETXATTR. - Fix oversized quota reservation when converting unwritten extents during a DAX write. - Create a transaction allocation helper function to standardize the idiom of allocating a transaction, reserving blocks, locking inodes, and reserving quota. Replace all the open-coded logic for file creation, file ownership changes, and file modifications to use them. - Actually shut down the fs if the incore quota reservations get corrupted. - Fix background block garbage collection scans to not block and to actually clean out CoW staging extents properly. - Run block gc scans when we run low on project quota. - Use the standardized transaction allocation helpers to make it so that ENOSPC and EDQUOT errors during reservation will back out, invoke the block gc scanner, and try again. This is preparation for introducing background inode garbage collection in the next cycle. - Combine speculative post-EOF block garbage collection with speculative copy on write block garbage collection. - Enable multithreaded quotacheck. - Allow sysadmins to tweak the CPU affinities and maximum concurrency levels of quotacheck and background blockgc worker pools. - Expose the inode btree counter feature in the fs geometry ioctl. - Cleanups of the growfs code in preparation for starting work on filesystem shrinking. - Fix all the bloody gcc warnings that the maintainer knows about. :P - Fix a RST syntax error. - Don't trigger bmbt corruption assertions after the fs shuts down. - Restore behavior of forcing SIGBUS on a shut down filesystem when someone triggers a mmap write fault (or really, any buffered write). ---------------------------------------------------------------- Brian Foster (14): xfs: rename xfs_wait_buftarg() to xfs_buftarg_drain() xfs: don't drain buffer lru on freeze and read-only remount xfs: sync lazy sb accounting on quiesce of read-only mounts xfs: lift writable fs check up into log worker task xfs: separate log cleaning from log quiesce xfs: cover the log during log quiesce xfs: don't reset log idle state on covering checkpoints xfs: fold sbcount quiesce logging into log covering xfs: remove duplicate wq cancel and log force from attr quiesce xfs: remove xfs_quiesce_attr() xfs: cover the log on freeze instead of cleaning it xfs: fix unused log variable in xfs_log_cover() xfs: restore shutdown check in mapped write fault path xfs: consider shutdown in bmapbt cursor delete assert Chandan Babu R (17): xfs: Add helper for checking per-inode extent count overflow xfs: Check for extent overflow when trivally adding a new extent xfs: Check for extent overflow when punching a hole xfs: Check for extent overflow when adding dir entries xfs: Check for extent overflow when removing dir entries xfs: Check for extent overflow when renaming dir entries xfs: Check for extent overflow when adding/removing xattrs xfs: Check for extent overflow when writing to unwritten extent xfs: Check for extent overflow when moving extent from cow to data fork xfs: Check for extent overflow when remapping an extent xfs: Check for extent overflow when swapping extents xfs: Introduce error injection to reduce maximum inode fork extent count xfs: Remove duplicate assert statement in xfs_bmap_btalloc() xfs: Compute bmap extent alignments in a separate function xfs: Process allocated extent in a separate function xfs: Introduce error injection to allocate only minlen size extents for files xfs: Fix 'set but not used' warning in xfs_bmap_compute_alignments() Christoph Hellwig (3): xfs: fix up non-directory creation in SGID directories xfs: refactor xfs_file_fsync xfs: reduce ilock acquisitions in xfs_file_fsync Darrick J. Wong (44): xfs: fix an ABBA deadlock in xfs_rename xfs: fix chown leaking delalloc quota blocks when fssetxattr fails xfs: reduce quota reservation when doing a dax unwritten extent conversion xfs: clean up quota reservation callsites xfs: create convenience wrappers for incore quota block reservations xfs: remove xfs_trans_unreserve_quota_nblks completely xfs: clean up icreate quota reservation calls xfs: fix up build warnings when quotas are disabled xfs: reserve data and rt quota at the same time xfs: refactor common transaction/inode/quota allocation idiom xfs: allow reservation of rtblocks with xfs_trans_alloc_inode xfs: refactor reflink functions to use xfs_trans_alloc_inode xfs: refactor inode creation transaction/inode/quota allocation idiom xfs: refactor inode ownership change transaction/inode/quota allocation idiom xfs: remove xfs_qm_vop_chown_reserve xfs: rename code to error in xfs_ioctl_setattr xfs: shut down the filesystem if we screw up quota reservation xfs: trigger all block gc scans when low on quota space xfs: don't stall cowblocks scan if we can't take locks xfs: xfs_inode_free_quota_blocks should scan project quota xfs: move and rename xfs_inode_free_quota_blocks to avoid conflicts xfs: pass flags and return gc errors from xfs_blockgc_free_quota xfs: try worst case space reservation upfront in xfs_reflink_remap_extent xfs: flush eof/cowblocks if we can't reserve quota for file blocks xfs: flush eof/cowblocks if we can't reserve quota for inode creation xfs: flush eof/cowblocks if we can't reserve quota for chown xfs: add a tracepoint for blockgc scans xfs: refactor xfs_icache_free_{eof,cow}blocks call sites xfs: flush speculative space allocations when we run out of space xfs: increase the default parallelism levels of pwork clients xfs: set WQ_SYSFS on all workqueues in debug mode xfs: relocate the eofb/cowb workqueue functions xfs: hide xfs_icache_free_eofblocks xfs: hide xfs_icache_free_cowblocks xfs: remove trivial eof/cowblocks functions xfs: consolidate incore inode radix tree posteof/cowblocks tags xfs: consolidate the eofblocks and cowblocks workers xfs: only walk the incore inode tree once per blockgc scan xfs: rename block gc start and stop functions xfs: parallelize block preallocation garbage collection xfs: expose the blockgc workqueue knobs publicly xfs: don't bounce the iolock between free_{eof,cow}blocks xfs: fix incorrect root dquot corruption error when switching group/project quota types xfs: fix rst syntax error in admin guide Eric Biggers (1): xfs: remove a stale comment from xfs_file_aio_write_checks() Gao Xiang (2): xfs: rename `new' to `delta' in xfs_growfs_data_private() xfs: get rid of xfs_growfs_{data,log}_t Jeffrey Mitchell (1): xfs: set inode size after creating symlink Yumei Huang (1): xfs: Fix assert failure in xfs_setattr_size() Zorro Lang (1): libxfs: expose inobtcount in xfs geometry kernel test robot (1): xfs: fix boolreturn.cocci warnings Documentation/admin-guide/xfs.rst | 42 ++++ fs/xfs/libxfs/xfs_alloc.c | 50 +++++ fs/xfs/libxfs/xfs_alloc.h | 3 + fs/xfs/libxfs/xfs_attr.c | 22 +- fs/xfs/libxfs/xfs_bmap.c | 315 ++++++++++++++++++--------- fs/xfs/libxfs/xfs_btree.c | 33 ++- fs/xfs/libxfs/xfs_dir2.h | 2 - fs/xfs/libxfs/xfs_dir2_sf.c | 2 +- fs/xfs/libxfs/xfs_errortag.h | 6 +- fs/xfs/libxfs/xfs_fs.h | 1 + fs/xfs/libxfs/xfs_inode_fork.c | 27 +++ fs/xfs/libxfs/xfs_inode_fork.h | 63 ++++++ fs/xfs/libxfs/xfs_sb.c | 2 + fs/xfs/scrub/common.c | 4 +- fs/xfs/xfs_bmap_item.c | 10 + fs/xfs/xfs_bmap_util.c | 81 ++++--- fs/xfs/xfs_buf.c | 30 ++- fs/xfs/xfs_buf.h | 11 +- fs/xfs/xfs_dquot.c | 47 +++- fs/xfs/xfs_error.c | 6 + fs/xfs/xfs_file.c | 114 +++++----- fs/xfs/xfs_fsops.c | 32 +-- fs/xfs/xfs_fsops.h | 4 +- fs/xfs/xfs_globals.c | 7 +- fs/xfs/xfs_icache.c | 438 ++++++++++++++++++++------------------ fs/xfs/xfs_icache.h | 24 +-- fs/xfs/xfs_inode.c | 134 ++++++++---- fs/xfs/xfs_ioctl.c | 75 +++---- fs/xfs/xfs_iomap.c | 53 ++--- fs/xfs/xfs_iops.c | 28 +-- fs/xfs/xfs_iwalk.c | 5 +- fs/xfs/xfs_linux.h | 3 +- fs/xfs/xfs_log.c | 142 +++++++++--- fs/xfs/xfs_log.h | 4 +- fs/xfs/xfs_mount.c | 43 +--- fs/xfs/xfs_mount.h | 10 +- fs/xfs/xfs_mru_cache.c | 2 +- fs/xfs/xfs_pwork.c | 25 +-- fs/xfs/xfs_pwork.h | 4 +- fs/xfs/xfs_qm.c | 116 ++-------- fs/xfs/xfs_quota.h | 49 +++-- fs/xfs/xfs_reflink.c | 109 ++++++---- fs/xfs/xfs_rtalloc.c | 5 + fs/xfs/xfs_super.c | 82 +++---- fs/xfs/xfs_super.h | 6 + fs/xfs/xfs_symlink.c | 15 +- fs/xfs/xfs_sysctl.c | 15 +- fs/xfs/xfs_sysctl.h | 3 +- fs/xfs/xfs_trace.c | 1 + fs/xfs/xfs_trace.h | 50 ++++- fs/xfs/xfs_trans.c | 195 +++++++++++++++++ fs/xfs/xfs_trans.h | 13 ++ fs/xfs/xfs_trans_dquot.c | 73 +++++-- 53 files changed, 1654 insertions(+), 982 deletions(-)