| Message ID | 20240530053754.4115449-1-srivathsa.d.dara@oracle.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [RESEND] btrfs-progs: convert: Add 64 bit block numbers support | expand |
On 30/05/2024 13:37, Srivathsa Dara wrote: > In ext4, number of blocks can be greater than 2^32. Therefore, if > btrfs-convert is used on filesystems greater than or equal to 16TiB > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > convert. > > Example: > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 > btrfs-convert from btrfs-progs v5.15.1 > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` failed, value 0 > btrfs-convert(+0xfd04)[0xaaaaba44fd04] > btrfs-convert(main+0x258)[0xaaaaba44d278] > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > Aborted (core dumped) > > Fix it by considering 64 bit block numbers. > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> > --- > convert/source-ext2.c | 6 +++--- > convert/source-ext2.h | 2 +- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/convert/source-ext2.c b/convert/source-ext2.c > index 2186b252..afa48606 100644 > --- a/convert/source-ext2.c > +++ b/convert/source-ext2.c > @@ -288,8 +288,8 @@ error: > return -1; > } > > -static int ext2_block_iterate_proc(ext2_filsys fs, blk_t *blocknr, > - e2_blkcnt_t blockcnt, blk_t ref_block, > +static int ext2_block_iterate_proc(ext2_filsys fs, blk64_t *blocknr, > + e2_blkcnt_t blockcnt, blk64_t ref_block, > int ref_offset, void *priv_data) > { > int ret; > @@ -323,7 +323,7 @@ static int ext2_create_file_extents(struct btrfs_trans_handle *trans, > init_blk_iterate_data(&data, trans, root, btrfs_inode, objectid, > convert_flags & CONVERT_FLAG_DATACSUM); > > - err = ext2fs_block_iterate2(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > + err = ext2fs_block_iterate3(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, Are there any ext2progs library version dependencies when using ext2fs_block_iterate3()? Thanks, Anand > NULL, ext2_block_iterate_proc, &data); > if (err) > goto error; > diff --git a/convert/source-ext2.h b/convert/source-ext2.h > index d204aac5..73c39e23 100644 > --- a/convert/source-ext2.h > +++ b/convert/source-ext2.h > @@ -46,7 +46,7 @@ struct btrfs_trans_handle; > #define ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range > #define ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks > #define ext2fs_read_ext_attr2 ext2fs_read_ext_attr > -#define ext2fs_blocks_count(s) ((s)->s_blocks_count) > +#define ext2fs_blocks_count(s) ((s)->s_blocks_count_hi << 32) | (s)->s_blocks_count > #define EXT2FS_CLUSTER_RATIO(fs) (1) > #define EXT2_CLUSTERS_PER_GROUP(s) (EXT2_BLOCKS_PER_GROUP(s)) > #define EXT2FS_B2C(fs, blk) (blk)
On Thu, May 30, 2024 at 05:37:54AM +0000, Srivathsa Dara wrote: > In ext4, number of blocks can be greater than 2^32. Therefore, if > btrfs-convert is used on filesystems greater than or equal to 16TiB > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > convert. > > Example: > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 > btrfs-convert from btrfs-progs v5.15.1 > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` failed, value 0 > btrfs-convert(+0xfd04)[0xaaaaba44fd04] > btrfs-convert(main+0x258)[0xaaaaba44d278] > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > Aborted (core dumped) > > Fix it by considering 64 bit block numbers. > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> > --- > convert/source-ext2.c | 6 +++--- > convert/source-ext2.h | 2 +- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/convert/source-ext2.c b/convert/source-ext2.c > index 2186b252..afa48606 100644 > --- a/convert/source-ext2.c > +++ b/convert/source-ext2.c > @@ -288,8 +288,8 @@ error: > return -1; > } > > -static int ext2_block_iterate_proc(ext2_filsys fs, blk_t *blocknr, > - e2_blkcnt_t blockcnt, blk_t ref_block, > +static int ext2_block_iterate_proc(ext2_filsys fs, blk64_t *blocknr, > + e2_blkcnt_t blockcnt, blk64_t ref_block, > int ref_offset, void *priv_data) > { > int ret; > @@ -323,7 +323,7 @@ static int ext2_create_file_extents(struct btrfs_trans_handle *trans, > init_blk_iterate_data(&data, trans, root, btrfs_inode, objectid, > convert_flags & CONVERT_FLAG_DATACSUM); > > - err = ext2fs_block_iterate2(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > + err = ext2fs_block_iterate3(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > NULL, ext2_block_iterate_proc, &data); > if (err) > goto error; > diff --git a/convert/source-ext2.h b/convert/source-ext2.h > index d204aac5..73c39e23 100644 > --- a/convert/source-ext2.h > +++ b/convert/source-ext2.h > @@ -46,7 +46,7 @@ struct btrfs_trans_handle; > #define ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range > #define ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks > #define ext2fs_read_ext_attr2 ext2fs_read_ext_attr > -#define ext2fs_blocks_count(s) ((s)->s_blocks_count) > +#define ext2fs_blocks_count(s) ((s)->s_blocks_count_hi << 32) | (s)->s_blocks_count Looks like there's missing closing )
On Thu, May 30, 2024 at 07:00:34PM +0800, Anand Jain wrote: > On 30/05/2024 13:37, Srivathsa Dara wrote: > > In ext4, number of blocks can be greater than 2^32. Therefore, if > > btrfs-convert is used on filesystems greater than or equal to 16TiB > > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > > convert. > > > > Example: > > > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 > > btrfs-convert from btrfs-progs v5.15.1 > > > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` failed, value 0 > > btrfs-convert(+0xfd04)[0xaaaaba44fd04] > > btrfs-convert(main+0x258)[0xaaaaba44d278] > > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > > Aborted (core dumped) > > > > Fix it by considering 64 bit block numbers. > > > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> > > --- > > convert/source-ext2.c | 6 +++--- > > convert/source-ext2.h | 2 +- > > 2 files changed, 4 insertions(+), 4 deletions(-) > > > > diff --git a/convert/source-ext2.c b/convert/source-ext2.c > > index 2186b252..afa48606 100644 > > --- a/convert/source-ext2.c > > +++ b/convert/source-ext2.c > > @@ -288,8 +288,8 @@ error: > > return -1; > > } > > > > -static int ext2_block_iterate_proc(ext2_filsys fs, blk_t *blocknr, > > - e2_blkcnt_t blockcnt, blk_t ref_block, > > +static int ext2_block_iterate_proc(ext2_filsys fs, blk64_t *blocknr, > > + e2_blkcnt_t blockcnt, blk64_t ref_block, > > int ref_offset, void *priv_data) > > { > > int ret; > > @@ -323,7 +323,7 @@ static int ext2_create_file_extents(struct btrfs_trans_handle *trans, > > init_blk_iterate_data(&data, trans, root, btrfs_inode, objectid, > > convert_flags & CONVERT_FLAG_DATACSUM); > > > > - err = ext2fs_block_iterate2(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > > + err = ext2fs_block_iterate3(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > > > Are there any ext2progs library version dependencies when using > ext2fs_block_iterate3()? There are but we don't have to worry, the function was added in 1998 https://github.com/tytso/e2fsprogs/commit/674a4ee1e3e05133ddad701730bfc21c283272a4
On Thu, May 30, 2024 at 05:37:54AM +0000, Srivathsa Dara wrote: > In ext4, number of blocks can be greater than 2^32. Therefore, if > btrfs-convert is used on filesystems greater than or equal to 16TiB > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > convert. > > Example: > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 > btrfs-convert from btrfs-progs v5.15.1 > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` failed, value 0 > btrfs-convert(+0xfd04)[0xaaaaba44fd04] > btrfs-convert(main+0x258)[0xaaaaba44d278] > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > Aborted (core dumped) > > Fix it by considering 64 bit block numbers. > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> The current conver tests passed, can you please also send test case for this fix? Thanks.
-----Original Message----- From: David Sterba <dsterba@suse.cz> Sent: Thursday, May 30, 2024 11:16 PM To: Srivathsa Dara <srivathsa.d.dara@oracle.com> Cc: linux-btrfs@vger.kernel.org; Rajesh Sivaramasubramaniom <rajesh.sivaramasubramaniom@oracle.com>; Junxiao Bi <junxiao.bi@oracle.com>; clm@fb.com; josef@toxicpanda.com; dsterba@suse.com Subject: Re: [RESEND PATCH] btrfs-progs: convert: Add 64 bit block numbers support Hi David, Thanks for answering Anand's question on my behalf. > On Thu, May 30, 2024 at 05:37:54AM +0000, Srivathsa Dara wrote: > > In ext4, number of blocks can be greater than 2>32. Therefore, if > > btrfs-convert is used on filesystems greater than or equal to 16TiB > > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > > convert. > > > > Example: > > > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 btrfs-convert > > from btrfs-progs v5.15.1 > > > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` > > failed, value 0 btrfs-convert(+0xfd04)[0xaaaaba44fd04] > > btrfs-convert(main+0x258)[0xaaaaba44d278] > > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > > Aborted (core dumped) > > > > Fix it by considering 64 bit block numbers. > > > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> > > The current conver tests passed, can you please also send test case for this fix? Thanks. Sure, would you like me to send that in v2 or as a separate patch?
-----Original Message----- From: David Sterba <dsterba@suse.cz> Sent: Thursday, May 30, 2024 10:59 PM To: Srivathsa Dara <srivathsa.d.dara@oracle.com> Cc: linux-btrfs@vger.kernel.org; Rajesh Sivaramasubramaniom <rajesh.sivaramasubramaniom@oracle.com>; Junxiao Bi <junxiao.bi@oracle.com>; clm@fb.com; josef@toxicpanda.com; dsterba@suse.com Subject: Re: [RESEND PATCH] btrfs-progs: convert: Add 64 bit block numbers support > On Thu, May 30, 2024 at 05:37:54AM +0000, Srivathsa Dara wrote: > > In ext4, number of blocks can be greater than 2^32. Therefore, if > > btrfs-convert is used on filesystems greater than or equal to 16TiB > > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > > convert. > > > > Example: > > > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 btrfs-convert > > from btrfs-progs v5.15.1 > > > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` > > failed, value 0 btrfs-convert(+0xfd04)[0xaaaaba44fd04] > > btrfs-convert(main+0x258)[0xaaaaba44d278] > > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > > Aborted (core dumped) > > > > Fix it by considering 64 bit block numbers. > > > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> > > --- > > convert/source-ext2.c | 6 +++--- > > convert/source-ext2.h | 2 +- > > 2 files changed, 4 insertions(+), 4 deletions(-) > > > > diff --git a/convert/source-ext2.c b/convert/source-ext2.c index > > 2186b252..afa48606 100644 > > --- a/convert/source-ext2.c > > +++ b/convert/source-ext2.c > > @@ -288,8 +288,8 @@ error: > > return -1; > > } > > > > -static int ext2_block_iterate_proc(ext2_filsys fs, blk_t *blocknr, > > - e2_blkcnt_t blockcnt, blk_t ref_block, > > +static int ext2_block_iterate_proc(ext2_filsys fs, blk64_t *blocknr, > > + e2_blkcnt_t blockcnt, blk64_t ref_block, > > int ref_offset, void *priv_data) { > > int ret; > > @@ -323,7 +323,7 @@ static int ext2_create_file_extents(struct btrfs_trans_handle *trans, > > init_blk_iterate_data(&data, trans, root, btrfs_inode, objectid, > > convert_flags & CONVERT_FLAG_DATACSUM); > > > > - err = ext2fs_block_iterate2(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > > + err = ext2fs_block_iterate3(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, > > NULL, ext2_block_iterate_proc, &data); > > if (err) > > goto error; > > diff --git a/convert/source-ext2.h b/convert/source-ext2.h index > > d204aac5..73c39e23 100644 > > --- a/convert/source-ext2.h > > +++ b/convert/source-ext2.h > > @@ -46,7 +46,7 @@ struct btrfs_trans_handle; #define > > ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range #define > > ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks #define > > ext2fs_read_ext_attr2 ext2fs_read_ext_attr > > -#define ext2fs_blocks_count(s) ((s)->s_blocks_count) > > +#define ext2fs_blocks_count(s) ((s)->s_blocks_count_hi << 32) | (s)->s_blocks_count > > Looks like there's missing closing ) No, all parenthesis are balanced.
On Fri, May 31, 2024 at 09:04:13AM +0000, Srivathsa Dara wrote: > > > @@ -46,7 +46,7 @@ struct btrfs_trans_handle; #define > > > ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range #define > > > ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks #define > > > ext2fs_read_ext_attr2 ext2fs_read_ext_attr > > > -#define ext2fs_blocks_count(s) ((s)->s_blocks_count) > > > +#define ext2fs_blocks_count(s) ((s)->s_blocks_count_hi << 32) | (s)->s_blocks_count > > > > Looks like there's missing closing ) > > No, all parenthesis are balanced. Oh right but it's confusing because the expression should be enclosed in a pair of ( ) as it's an expression in a macro and it could change result once expanded in random locations.
On Fri, May 31, 2024 at 08:58:08AM +0000, Srivathsa Dara wrote: > > > -----Original Message----- > From: David Sterba <dsterba@suse.cz> > Sent: Thursday, May 30, 2024 11:16 PM > To: Srivathsa Dara <srivathsa.d.dara@oracle.com> > Cc: linux-btrfs@vger.kernel.org; Rajesh Sivaramasubramaniom <rajesh.sivaramasubramaniom@oracle.com>; Junxiao Bi <junxiao.bi@oracle.com>; clm@fb.com; josef@toxicpanda.com; dsterba@suse.com > Subject: Re: [RESEND PATCH] btrfs-progs: convert: Add 64 bit block numbers support > > Hi David, > Thanks for answering Anand's question on my behalf. > > > On Thu, May 30, 2024 at 05:37:54AM +0000, Srivathsa Dara wrote: > > > In ext4, number of blocks can be greater than 2>32. Therefore, if > > > btrfs-convert is used on filesystems greater than or equal to 16TiB > > > (Staring from 16TiB, number of blocks overflow 32 bits), it fails to > > > convert. > > > > > > Example: > > > > > > Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. > > > > > > [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 btrfs-convert > > > from btrfs-progs v5.15.1 > > > > > > convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` > > > failed, value 0 btrfs-convert(+0xfd04)[0xaaaaba44fd04] > > > btrfs-convert(main+0x258)[0xaaaaba44d278] > > > /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] > > > btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] > > > Aborted (core dumped) > > > > > > Fix it by considering 64 bit block numbers. > > > > > > Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> > > > > The current conver tests passed, can you please also send test case for this fix? Thanks. > > Sure, would you like me to send that in v2 or as a separate patch? Separate patch for test is ok. Thanks.
> -----Original Message----- > From: David Sterba <dsterba@suse.cz> > Sent: Friday, May 31, 2024 9:03 PM > To: Srivathsa Dara <srivathsa.d.dara@oracle.com> > Cc: linux-btrfs@vger.kernel.org; Rajesh Sivaramasubramaniom <rajesh.sivaramasubramaniom@oracle.com>; Junxiao Bi <junxiao.bi@oracle.com>; clm@fb.com; josef@toxicpanda.com; dsterba@suse.com > Subject: [External] : Re: [RESEND PATCH] btrfs-progs: convert: Add 64 bit block numbers support > > On Fri, May 31, 2024 at 09:04:13AM +0000, Srivathsa Dara wrote: > > > > @@ -46,7 +46,7 @@ struct btrfs_trans_handle; #define > > > > ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range > > > > #define > > > > ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks #define > > > > ext2fs_read_ext_attr2 ext2fs_read_ext_attr > > > > -#define ext2fs_blocks_count(s) ((s)->s_blocks_count) > > > > +#define ext2fs_blocks_count(s) ((s)->s_blocks_count_hi << 32) | (s)->s_blocks_count > > > > > > Looks like there's missing closing ) > > > > No, all parenthesis are balanced. > > Oh right but it's confusing because the expression should be enclosed in a pair of ( ) as it's an expression in a macro and it could change result once expanded in random locations. Okay, I will send a V2.
diff --git a/convert/source-ext2.c b/convert/source-ext2.c index 2186b252..afa48606 100644 --- a/convert/source-ext2.c +++ b/convert/source-ext2.c @@ -288,8 +288,8 @@ error: return -1; } -static int ext2_block_iterate_proc(ext2_filsys fs, blk_t *blocknr, - e2_blkcnt_t blockcnt, blk_t ref_block, +static int ext2_block_iterate_proc(ext2_filsys fs, blk64_t *blocknr, + e2_blkcnt_t blockcnt, blk64_t ref_block, int ref_offset, void *priv_data) { int ret; @@ -323,7 +323,7 @@ static int ext2_create_file_extents(struct btrfs_trans_handle *trans, init_blk_iterate_data(&data, trans, root, btrfs_inode, objectid, convert_flags & CONVERT_FLAG_DATACSUM); - err = ext2fs_block_iterate2(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, + err = ext2fs_block_iterate3(ext2_fs, ext2_ino, BLOCK_FLAG_DATA_ONLY, NULL, ext2_block_iterate_proc, &data); if (err) goto error; diff --git a/convert/source-ext2.h b/convert/source-ext2.h index d204aac5..73c39e23 100644 --- a/convert/source-ext2.h +++ b/convert/source-ext2.h @@ -46,7 +46,7 @@ struct btrfs_trans_handle; #define ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range #define ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks #define ext2fs_read_ext_attr2 ext2fs_read_ext_attr -#define ext2fs_blocks_count(s) ((s)->s_blocks_count) +#define ext2fs_blocks_count(s) ((s)->s_blocks_count_hi << 32) | (s)->s_blocks_count #define EXT2FS_CLUSTER_RATIO(fs) (1) #define EXT2_CLUSTERS_PER_GROUP(s) (EXT2_BLOCKS_PER_GROUP(s)) #define EXT2FS_B2C(fs, blk) (blk)
In ext4, number of blocks can be greater than 2^32. Therefore, if btrfs-convert is used on filesystems greater than or equal to 16TiB (Staring from 16TiB, number of blocks overflow 32 bits), it fails to convert. Example: Here, /dev/sdc1 is 16TiB partition intitialized with an ext4 filesystem. [root@rasivara-arm2 opc]# btrfs-convert -d -p /dev/sdc1 btrfs-convert from btrfs-progs v5.15.1 convert/main.c:1164: do_convert: Assertion `cctx.total_bytes != 0` failed, value 0 btrfs-convert(+0xfd04)[0xaaaaba44fd04] btrfs-convert(main+0x258)[0xaaaaba44d278] /lib64/libc.so.6(__libc_start_main+0xdc)[0xffffb962777c] btrfs-convert(+0xd4fc)[0xaaaaba44d4fc] Aborted (core dumped) Fix it by considering 64 bit block numbers. Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com> --- convert/source-ext2.c | 6 +++--- convert/source-ext2.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)