| Message ID | cover.1709918025.git.sweettea-kernel@dorminy.me (mailing list archive) |
|---|---|
| Headers | show |
| Series | fiemap extension to add physical extent length | expand |
On Fri, Mar 08, 2024 at 01:03:17PM -0500, Sweet Tea Dorminy wrote: > For many years, various btrfs users have written programs to discover > the actual disk space used by files, using root-only interfaces. > However, this information is a great fit for fiemap: it is inherently > tied to extent information, all filesystems can use it, and the > capabilities required for FIEMAP make sense for this additional > information also. > > Hence, this patchset adds physical extent length information to fiemap, > and extends btrfs to return it. This uses some of the reserved padding > in the fiemap extent structure, so programs unaware of the new field > will be unaffected by its presence. > > This is based on next-20240307. I've tested the btrfs part of this with > the standard btrfs testing matrix locally, and verified that the physical extent > information returned there is correct, but I'm still waiting on more > tests. Please let me know what you think of the general idea! Seems useful! Any chance you'd be willing to pick up this old proposal to report the dev_t through iomap? iirc the iomap wrappers for fiemap can export that pretty easily. https://lore.kernel.org/linux-fsdevel/20190211094306.fjr6gfehcstm7eqq@hades.usersys.redhat.com/ (Not sure what we do for pmem filesystems) --D > Sweet Tea Dorminy (3): > fs: add physical_length field to fiemap extents > fs: update fiemap_fill_next_extent() signature > btrfs: fiemap: return extent physical size > > Documentation/filesystems/fiemap.rst | 29 +++++++++---- > fs/bcachefs/fs.c | 6 ++- > fs/btrfs/extent_io.c | 63 +++++++++++++++++----------- > fs/ext4/extents.c | 1 + > fs/f2fs/data.c | 8 ++-- > fs/f2fs/inline.c | 3 +- > fs/ioctl.c | 8 ++-- > fs/iomap/fiemap.c | 2 +- > fs/nilfs2/inode.c | 8 ++-- > fs/ntfs3/frecord.c | 6 ++- > fs/ocfs2/extent_map.c | 4 +- > fs/smb/client/smb2ops.c | 1 + > include/linux/fiemap.h | 2 +- > include/uapi/linux/fiemap.h | 24 +++++++---- > 14 files changed, 108 insertions(+), 57 deletions(-) > > > base-commit: 1843e16d2df9d98427ef8045589571749d627cf7 > -- > 2.44.0 > >
On Thu, Mar 14, 2024 at 08:03:34PM -0700, Darrick J. Wong wrote: > On Fri, Mar 08, 2024 at 01:03:17PM -0500, Sweet Tea Dorminy wrote: > > For many years, various btrfs users have written programs to discover > > the actual disk space used by files, using root-only interfaces. > > However, this information is a great fit for fiemap: it is inherently > > tied to extent information, all filesystems can use it, and the > > capabilities required for FIEMAP make sense for this additional > > information also. > > > > Hence, this patchset adds physical extent length information to fiemap, > > and extends btrfs to return it. This uses some of the reserved padding > > in the fiemap extent structure, so programs unaware of the new field > > will be unaffected by its presence. > > > > This is based on next-20240307. I've tested the btrfs part of this with > > the standard btrfs testing matrix locally, and verified that the physical extent > > information returned there is correct, but I'm still waiting on more > > tests. Please let me know what you think of the general idea! > > Seems useful! Any chance you'd be willing to pick up this old proposal > to report the dev_t through iomap? iirc the iomap wrappers for fiemap > can export that pretty easily. > > https://lore.kernel.org/linux-fsdevel/20190211094306.fjr6gfehcstm7eqq@hades.usersys.redhat.com/ I think this is not too useful for btrfs (in general) due to the block group profiles that store copies on multiple devices, we'd need more than one device identifier per extent.
On Mar 21, 2024, at 12:58 PM, David Sterba <dsterba@suse.cz> wrote: > > On Thu, Mar 14, 2024 at 08:03:34PM -0700, Darrick J. Wong wrote: >> On Fri, Mar 08, 2024 at 01:03:17PM -0500, Sweet Tea Dorminy wrote: >>> For many years, various btrfs users have written programs to discover >>> the actual disk space used by files, using root-only interfaces. >>> However, this information is a great fit for fiemap: it is inherently >>> tied to extent information, all filesystems can use it, and the >>> capabilities required for FIEMAP make sense for this additional >>> information also. >>> >>> Hence, this patchset adds physical extent length information to fiemap, >>> and extends btrfs to return it. This uses some of the reserved padding >>> in the fiemap extent structure, so programs unaware of the new field >>> will be unaffected by its presence. >>> >>> This is based on next-20240307. I've tested the btrfs part of this with >>> the standard btrfs testing matrix locally, and verified that the physical extent >>> information returned there is correct, but I'm still waiting on more >>> tests. Please let me know what you think of the general idea! >> >> Seems useful! Any chance you'd be willing to pick up this old proposal >> to report the dev_t through iomap? iirc the iomap wrappers for fiemap >> can export that pretty easily. >> >> https://lore.kernel.org/linux-fsdevel/20190211094306.fjr6gfehcstm7eqq@hades.usersys.redhat.com/ > > I think this is not too useful for btrfs (in general) due to the block > group profiles that store copies on multiple devices, we'd need more > than one device identifier per extent. My thought would be that there are multiple overlapping extents with the same logical offset returned in this case. It wouldn't just be the device that would be different in this case, but also the physical offset may be different on each device (depending on how allocation is done), and maybe even the length and flags are different if one device stores compressed data and another one does not. Having multiple overlapping extents for files with built-in mirrors allows freedom for all of the extent parameters to be different, doesn't have any limits on the number of copies/devices that could fit in one extent, etc. Cheers, Andreas