| Message ID | 20221218100306.76408-1-allison.henderson@oracle.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Parent Pointers | expand |
On Sun, Dec 18, 2022 at 03:02:39AM -0700, allison.henderson@oracle.com wrote: > From: Allison Henderson <allison.henderson@oracle.com> > > Hi all, > > This is the latest parent pointer attributes for xfs. > The goal of this patch set is to add a parent pointer attribute to each inode. > The attribute name containing the parent inode, generation, and directory > offset, while the attribute value contains the file name. This feature will > enable future optimizations for online scrub, shrink, nfs handles, verity, or > any other feature that could make use of quickly deriving an inodes path from > the mount point. > > This set can be viewed on github here > https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv7 > > And the corresponding xfsprogs code is here > https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v7 > > This set has been tested with the below parent pointers tests > https://lore.kernel.org/fstests/20221012013812.82161-1-catherine.hoang@oracle.com/T/#t I pulled v7 and rebased your patches atop 6.2-rc3. generic/476 failed with quotas enabled: XFS: Assertion failed: q[j].qt_dquot->q_id > d->q_id, file: fs/xfs/xfs_dquot.c, line: 1351 <repeat> I suspect this is becuase xfs_dqlockn needs to sort the dqtrx structures by quotaid? So I turned off quotas and the kernel quieted down. Good. :) Next up was xfs_scrub, which complained about *something* in the xattr structures. I'm guessing that it probably doesn't know how to deal with the new pptr namespace or the zeroes that can be present in the pptr attr name? Other than that, looks promising. I'll fix up the scrub and quota problems and start prototyping fsck code from there. --D > Updates since v6: > Andry had reported intermittent hangs on unmount while running stress tests. > Reviewed and corrected unlocks in the case of error conditions. > > xfs: parent pointer attribute creation > Fixed dp to unlock on error > > xfs: Hold inode locks in xfs_ialloc > Fixed ip to unlock on create error > > xfs: add parent attributes to symlink > Hold and release dp across pptr operation > > Questions comments and feedback appreciated! > > Thanks all! > Allison > > > Allison Henderson (27): > xfs: Add new name to attri/d > xfs: Increase XFS_DEFER_OPS_NR_INODES to 5 > xfs: Increase XFS_QM_TRANS_MAXDQS to 5 > xfs: Hold inode locks in xfs_ialloc > xfs: Hold inode locks in xfs_trans_alloc_dir > xfs: Hold inode locks in xfs_rename > xfs: Expose init_xattrs in xfs_create_tmpfile > xfs: get directory offset when adding directory name > xfs: get directory offset when removing directory name > xfs: get directory offset when replacing a directory name > xfs: add parent pointer support to attribute code > xfs: define parent pointer xattr format > xfs: Add xfs_verify_pptr > xfs: extend transaction reservations for parent attributes > xfs: parent pointer attribute creation > xfs: add parent attributes to link > xfs: add parent attributes to symlink > xfs: remove parent pointers in unlink > xfs: Indent xfs_rename > xfs: Add parent pointers to rename > xfs: Add parent pointers to xfs_cross_rename > xfs: Add the parent pointer support to the superblock version 5. > xfs: Add helper function xfs_attr_list_context_init > xfs: Filter XFS_ATTR_PARENT for getfattr > xfs: Add parent pointer ioctl > xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res > xfs: drop compatibility minimum log size computations for reflink > > fs/xfs/Makefile | 2 + > fs/xfs/libxfs/xfs_attr.c | 71 +++++- > fs/xfs/libxfs/xfs_attr.h | 13 +- > fs/xfs/libxfs/xfs_da_btree.h | 3 + > fs/xfs/libxfs/xfs_da_format.h | 38 ++- > fs/xfs/libxfs/xfs_defer.c | 28 ++- > fs/xfs/libxfs/xfs_defer.h | 8 +- > fs/xfs/libxfs/xfs_dir2.c | 21 +- > fs/xfs/libxfs/xfs_dir2.h | 7 +- > fs/xfs/libxfs/xfs_dir2_block.c | 9 +- > fs/xfs/libxfs/xfs_dir2_leaf.c | 8 +- > fs/xfs/libxfs/xfs_dir2_node.c | 8 +- > fs/xfs/libxfs/xfs_dir2_sf.c | 6 + > fs/xfs/libxfs/xfs_format.h | 4 +- > fs/xfs/libxfs/xfs_fs.h | 75 ++++++ > fs/xfs/libxfs/xfs_log_format.h | 7 +- > fs/xfs/libxfs/xfs_log_rlimit.c | 53 ++++ > fs/xfs/libxfs/xfs_parent.c | 207 ++++++++++++++++ > fs/xfs/libxfs/xfs_parent.h | 46 ++++ > fs/xfs/libxfs/xfs_sb.c | 4 + > fs/xfs/libxfs/xfs_trans_resv.c | 324 ++++++++++++++++++++---- > fs/xfs/libxfs/xfs_trans_space.h | 8 - > fs/xfs/scrub/attr.c | 2 +- > fs/xfs/xfs_attr_item.c | 142 +++++++++-- > fs/xfs/xfs_attr_item.h | 1 + > fs/xfs/xfs_attr_list.c | 17 +- > fs/xfs/xfs_dquot.c | 25 ++ > fs/xfs/xfs_dquot.h | 1 + > fs/xfs/xfs_file.c | 1 + > fs/xfs/xfs_inode.c | 427 +++++++++++++++++++++++++------- > fs/xfs/xfs_inode.h | 3 +- > fs/xfs/xfs_ioctl.c | 148 +++++++++-- > fs/xfs/xfs_ioctl.h | 2 + > fs/xfs/xfs_iops.c | 3 +- > fs/xfs/xfs_ondisk.h | 4 + > fs/xfs/xfs_parent_utils.c | 125 ++++++++++ > fs/xfs/xfs_parent_utils.h | 11 + > fs/xfs/xfs_qm.c | 4 +- > fs/xfs/xfs_qm.h | 2 +- > fs/xfs/xfs_super.c | 4 + > fs/xfs/xfs_symlink.c | 58 ++++- > fs/xfs/xfs_trans.c | 6 +- > fs/xfs/xfs_trans_dquot.c | 15 +- > fs/xfs/xfs_xattr.c | 5 +- > fs/xfs/xfs_xattr.h | 1 + > 45 files changed, 1712 insertions(+), 245 deletions(-) > create mode 100644 fs/xfs/libxfs/xfs_parent.c > create mode 100644 fs/xfs/libxfs/xfs_parent.h > create mode 100644 fs/xfs/xfs_parent_utils.c > create mode 100644 fs/xfs/xfs_parent_utils.h > > -- > 2.25.1 >
On Wed, Jan 11, 2023 at 04:31:04PM -0800, Darrick J. Wong wrote: > On Sun, Dec 18, 2022 at 03:02:39AM -0700, allison.henderson@oracle.com wrote: > > From: Allison Henderson <allison.henderson@oracle.com> > > > > Hi all, > > > > This is the latest parent pointer attributes for xfs. > > The goal of this patch set is to add a parent pointer attribute to each inode. > > The attribute name containing the parent inode, generation, and directory > > offset, while the attribute value contains the file name. This feature will > > enable future optimizations for online scrub, shrink, nfs handles, verity, or > > any other feature that could make use of quickly deriving an inodes path from > > the mount point. > > > > This set can be viewed on github here > > https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv7 > > > > And the corresponding xfsprogs code is here > > https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v7 > > > > This set has been tested with the below parent pointers tests > > https://lore.kernel.org/fstests/20221012013812.82161-1-catherine.hoang@oracle.com/T/#t > > I pulled v7 and rebased your patches atop 6.2-rc3. generic/476 failed > with quotas enabled: > > XFS: Assertion failed: q[j].qt_dquot->q_id > d->q_id, file: fs/xfs/xfs_dquot.c, line: 1351 > <repeat> > > I suspect this is becuase xfs_dqlockn needs to sort the dqtrx structures > by quotaid? > > So I turned off quotas and the kernel quieted down. Good. :) > > Next up was xfs_scrub, which complained about *something* in the xattr > structures. I'm guessing that it probably doesn't know how to deal with > the new pptr namespace or the zeroes that can be present in the pptr > attr name? > > Other than that, looks promising. I'll fix up the scrub and quota > problems and start prototyping fsck code from there. Second followup: two of the fstest vms are stuck spinning in unmount on xfs/113; and the other two are stuck trying to grab ilocks in generic/681 and generic/682. --D > --D > > > Updates since v6: > > Andry had reported intermittent hangs on unmount while running stress tests. > > Reviewed and corrected unlocks in the case of error conditions. > > > > xfs: parent pointer attribute creation > > Fixed dp to unlock on error > > > > xfs: Hold inode locks in xfs_ialloc > > Fixed ip to unlock on create error > > > > xfs: add parent attributes to symlink > > Hold and release dp across pptr operation > > > > Questions comments and feedback appreciated! > > > > Thanks all! > > Allison > > > > > > Allison Henderson (27): > > xfs: Add new name to attri/d > > xfs: Increase XFS_DEFER_OPS_NR_INODES to 5 > > xfs: Increase XFS_QM_TRANS_MAXDQS to 5 > > xfs: Hold inode locks in xfs_ialloc > > xfs: Hold inode locks in xfs_trans_alloc_dir > > xfs: Hold inode locks in xfs_rename > > xfs: Expose init_xattrs in xfs_create_tmpfile > > xfs: get directory offset when adding directory name > > xfs: get directory offset when removing directory name > > xfs: get directory offset when replacing a directory name > > xfs: add parent pointer support to attribute code > > xfs: define parent pointer xattr format > > xfs: Add xfs_verify_pptr > > xfs: extend transaction reservations for parent attributes > > xfs: parent pointer attribute creation > > xfs: add parent attributes to link > > xfs: add parent attributes to symlink > > xfs: remove parent pointers in unlink > > xfs: Indent xfs_rename > > xfs: Add parent pointers to rename > > xfs: Add parent pointers to xfs_cross_rename > > xfs: Add the parent pointer support to the superblock version 5. > > xfs: Add helper function xfs_attr_list_context_init > > xfs: Filter XFS_ATTR_PARENT for getfattr > > xfs: Add parent pointer ioctl > > xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res > > xfs: drop compatibility minimum log size computations for reflink > > > > fs/xfs/Makefile | 2 + > > fs/xfs/libxfs/xfs_attr.c | 71 +++++- > > fs/xfs/libxfs/xfs_attr.h | 13 +- > > fs/xfs/libxfs/xfs_da_btree.h | 3 + > > fs/xfs/libxfs/xfs_da_format.h | 38 ++- > > fs/xfs/libxfs/xfs_defer.c | 28 ++- > > fs/xfs/libxfs/xfs_defer.h | 8 +- > > fs/xfs/libxfs/xfs_dir2.c | 21 +- > > fs/xfs/libxfs/xfs_dir2.h | 7 +- > > fs/xfs/libxfs/xfs_dir2_block.c | 9 +- > > fs/xfs/libxfs/xfs_dir2_leaf.c | 8 +- > > fs/xfs/libxfs/xfs_dir2_node.c | 8 +- > > fs/xfs/libxfs/xfs_dir2_sf.c | 6 + > > fs/xfs/libxfs/xfs_format.h | 4 +- > > fs/xfs/libxfs/xfs_fs.h | 75 ++++++ > > fs/xfs/libxfs/xfs_log_format.h | 7 +- > > fs/xfs/libxfs/xfs_log_rlimit.c | 53 ++++ > > fs/xfs/libxfs/xfs_parent.c | 207 ++++++++++++++++ > > fs/xfs/libxfs/xfs_parent.h | 46 ++++ > > fs/xfs/libxfs/xfs_sb.c | 4 + > > fs/xfs/libxfs/xfs_trans_resv.c | 324 ++++++++++++++++++++---- > > fs/xfs/libxfs/xfs_trans_space.h | 8 - > > fs/xfs/scrub/attr.c | 2 +- > > fs/xfs/xfs_attr_item.c | 142 +++++++++-- > > fs/xfs/xfs_attr_item.h | 1 + > > fs/xfs/xfs_attr_list.c | 17 +- > > fs/xfs/xfs_dquot.c | 25 ++ > > fs/xfs/xfs_dquot.h | 1 + > > fs/xfs/xfs_file.c | 1 + > > fs/xfs/xfs_inode.c | 427 +++++++++++++++++++++++++------- > > fs/xfs/xfs_inode.h | 3 +- > > fs/xfs/xfs_ioctl.c | 148 +++++++++-- > > fs/xfs/xfs_ioctl.h | 2 + > > fs/xfs/xfs_iops.c | 3 +- > > fs/xfs/xfs_ondisk.h | 4 + > > fs/xfs/xfs_parent_utils.c | 125 ++++++++++ > > fs/xfs/xfs_parent_utils.h | 11 + > > fs/xfs/xfs_qm.c | 4 +- > > fs/xfs/xfs_qm.h | 2 +- > > fs/xfs/xfs_super.c | 4 + > > fs/xfs/xfs_symlink.c | 58 ++++- > > fs/xfs/xfs_trans.c | 6 +- > > fs/xfs/xfs_trans_dquot.c | 15 +- > > fs/xfs/xfs_xattr.c | 5 +- > > fs/xfs/xfs_xattr.h | 1 + > > 45 files changed, 1712 insertions(+), 245 deletions(-) > > create mode 100644 fs/xfs/libxfs/xfs_parent.c > > create mode 100644 fs/xfs/libxfs/xfs_parent.h > > create mode 100644 fs/xfs/xfs_parent_utils.c > > create mode 100644 fs/xfs/xfs_parent_utils.h > > > > -- > > 2.25.1 > >
From: Allison Henderson <allison.henderson@oracle.com> Hi all, This is the latest parent pointer attributes for xfs. The goal of this patch set is to add a parent pointer attribute to each inode. The attribute name containing the parent inode, generation, and directory offset, while the attribute value contains the file name. This feature will enable future optimizations for online scrub, shrink, nfs handles, verity, or any other feature that could make use of quickly deriving an inodes path from the mount point. This set can be viewed on github here https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv7 And the corresponding xfsprogs code is here https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v7 This set has been tested with the below parent pointers tests https://lore.kernel.org/fstests/20221012013812.82161-1-catherine.hoang@oracle.com/T/#t Updates since v6: Andry had reported intermittent hangs on unmount while running stress tests. Reviewed and corrected unlocks in the case of error conditions. xfs: parent pointer attribute creation Fixed dp to unlock on error xfs: Hold inode locks in xfs_ialloc Fixed ip to unlock on create error xfs: add parent attributes to symlink Hold and release dp across pptr operation Questions comments and feedback appreciated! Thanks all! Allison Allison Henderson (27): xfs: Add new name to attri/d xfs: Increase XFS_DEFER_OPS_NR_INODES to 5 xfs: Increase XFS_QM_TRANS_MAXDQS to 5 xfs: Hold inode locks in xfs_ialloc xfs: Hold inode locks in xfs_trans_alloc_dir xfs: Hold inode locks in xfs_rename xfs: Expose init_xattrs in xfs_create_tmpfile xfs: get directory offset when adding directory name xfs: get directory offset when removing directory name xfs: get directory offset when replacing a directory name xfs: add parent pointer support to attribute code xfs: define parent pointer xattr format xfs: Add xfs_verify_pptr xfs: extend transaction reservations for parent attributes xfs: parent pointer attribute creation xfs: add parent attributes to link xfs: add parent attributes to symlink xfs: remove parent pointers in unlink xfs: Indent xfs_rename xfs: Add parent pointers to rename xfs: Add parent pointers to xfs_cross_rename xfs: Add the parent pointer support to the superblock version 5. xfs: Add helper function xfs_attr_list_context_init xfs: Filter XFS_ATTR_PARENT for getfattr xfs: Add parent pointer ioctl xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res xfs: drop compatibility minimum log size computations for reflink fs/xfs/Makefile | 2 + fs/xfs/libxfs/xfs_attr.c | 71 +++++- fs/xfs/libxfs/xfs_attr.h | 13 +- fs/xfs/libxfs/xfs_da_btree.h | 3 + fs/xfs/libxfs/xfs_da_format.h | 38 ++- fs/xfs/libxfs/xfs_defer.c | 28 ++- fs/xfs/libxfs/xfs_defer.h | 8 +- fs/xfs/libxfs/xfs_dir2.c | 21 +- fs/xfs/libxfs/xfs_dir2.h | 7 +- fs/xfs/libxfs/xfs_dir2_block.c | 9 +- fs/xfs/libxfs/xfs_dir2_leaf.c | 8 +- fs/xfs/libxfs/xfs_dir2_node.c | 8 +- fs/xfs/libxfs/xfs_dir2_sf.c | 6 + fs/xfs/libxfs/xfs_format.h | 4 +- fs/xfs/libxfs/xfs_fs.h | 75 ++++++ fs/xfs/libxfs/xfs_log_format.h | 7 +- fs/xfs/libxfs/xfs_log_rlimit.c | 53 ++++ fs/xfs/libxfs/xfs_parent.c | 207 ++++++++++++++++ fs/xfs/libxfs/xfs_parent.h | 46 ++++ fs/xfs/libxfs/xfs_sb.c | 4 + fs/xfs/libxfs/xfs_trans_resv.c | 324 ++++++++++++++++++++---- fs/xfs/libxfs/xfs_trans_space.h | 8 - fs/xfs/scrub/attr.c | 2 +- fs/xfs/xfs_attr_item.c | 142 +++++++++-- fs/xfs/xfs_attr_item.h | 1 + fs/xfs/xfs_attr_list.c | 17 +- fs/xfs/xfs_dquot.c | 25 ++ fs/xfs/xfs_dquot.h | 1 + fs/xfs/xfs_file.c | 1 + fs/xfs/xfs_inode.c | 427 +++++++++++++++++++++++++------- fs/xfs/xfs_inode.h | 3 +- fs/xfs/xfs_ioctl.c | 148 +++++++++-- fs/xfs/xfs_ioctl.h | 2 + fs/xfs/xfs_iops.c | 3 +- fs/xfs/xfs_ondisk.h | 4 + fs/xfs/xfs_parent_utils.c | 125 ++++++++++ fs/xfs/xfs_parent_utils.h | 11 + fs/xfs/xfs_qm.c | 4 +- fs/xfs/xfs_qm.h | 2 +- fs/xfs/xfs_super.c | 4 + fs/xfs/xfs_symlink.c | 58 ++++- fs/xfs/xfs_trans.c | 6 +- fs/xfs/xfs_trans_dquot.c | 15 +- fs/xfs/xfs_xattr.c | 5 +- fs/xfs/xfs_xattr.h | 1 + 45 files changed, 1712 insertions(+), 245 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_parent.c create mode 100644 fs/xfs/libxfs/xfs_parent.h create mode 100644 fs/xfs/xfs_parent_utils.c create mode 100644 fs/xfs/xfs_parent_utils.h