Message ID | 20240704025740.527171-1-shengyong@oppo.com (mailing list archive) |
---|---|
Headers | show |
Series | f2fs-tools: introduce inject.f2fs | expand |
On 2024/7/4 10:57, Sheng Yong wrote: > This patchset introduces a new tool inject.f2fs to modify metadata or > data (directory entry) of f2fs image offline flexibly. > > With inject.f2fs, it is easier to generate a corrupted f2fs image, which > can help verify fsck or reproduce userspace behaviors of some a fault. > If option `--dry-run' is used, nothing really gets changed, and that > could be used to get the value of a specified field. > > inject.f2fs allows injecting some members in sb, cp, nat, sit, ssa, node > and dentry for now. The available members of each part can be listed by > executing command like: > inject.f2fs --sb 0 --help > > [...] > [mb]: > magic: inject magic number > s_stop_reason: inject s_stop_reason array selected by --idx <index> > s_errors: inject s_errors array selected by --idx <index> > devs.path: inject path in devs array selected by --idx <index> specified by --str <string> > > More injection fields are still work-in-progress. The TODO list includes: > * sb: features > * cp: fsync dnodes > * inode: extent, extra attrs, xattr > * data: fsverify? > * other fields which is needed to verify fsck What about including todo list into manual? > > v3: > * patch 3, fix error handling of inject sb->devs.path > * patch 3, do not ASSERT devs.path when inject is executed > * patch 3, allow inject to execute if image is umounted unclean > * patch 9, check whether blkaddr is valid before reading dentry block > * add is_digits and strtol entptr check when parsing numeric options > > v2: > * change print format of s_errors > * add write_raw_cp_blocks to write the first & last cp blocks directly > to avoid updating ckpt_flags by write_checkpoint > * call update_block if i_inode_checksum is injected to avoid updating > i_inode_checksum by update_inode > * go through all dentry blocks to find the target dir entry > > Examples: > > Inject sb's magic > ================= > inject.f2fs --sb 0 --mb magic --val 0x12345 $DEV > > Info: inject sb auto > Info: inject member magic > Info: inject value 74565 : 0x12345 > [...] > Info: inject magic of sb 1: 0xf2f52010 -> 0x12345 > [update_superblock: 890] Info: Done to update superblock > > Inject cp's cur_data_segno > ========================== > inject.f2fs --cp 0 --mb cur_data_segno --idx 1 --val 0x12345 $DEV > > Info: inject cp pack auto > Info: inject member cur_data_segno > Info: inject slot index 1 > Info: inject value 74565 : 0x12345 > [...] > Info: inject cur_data_segno[1] of cp 1: 0x4 -> 0x12345 > Info: write_checkpoint() cur_cp:1 > > Inject nat's ino > ================ > inject.f2fs --nat 0 --mb ino --nid $INO --val 0x12345 $DEV > > Info: inject nat pack auto > Info: inject nid 4 : 0x4 > Info: inject member ino > Info: inject value 74565 : 0x12345 > [...] > Info: inject nat entry ino of nid 4 in pack 1: 4 -> 74565 > > Inject ssa's nid > ================ > inject.f2fs --ssa --blk $BLK --mb nid --val 0x12345 $DEV > > Info: inject ssa > Info: inject blkaddr 7511 : 0x1d57 > Info: inject member nid > Info: inject value 74565 : 0x12345 > [...] > Info: auto idx = 343 > Info: inject summary entry nid of block 0x1d57: 0x4 -> 0x12345 > > Inject inode's i_addr > ===================== > inject.f2fs --node --nid $INO --mb i_addr --idx 100 --val 0x12345 $DEV > > Info: inject node > Info: inject nid 4 : 0x4 > Info: inject member i_addr > Info: inject slot index 100 > Info: inject value 74565 : 0x12345 > [...] > Info: inject inode i_addr[100] of nid 4: 0x20864 -> 0x12345 > > Inject inode's dentry hash > ========================== > inject.f2fs --dent --nid $INO --mb d_hash --val 0x12345 $DEV > > Info: inject dentry > Info: inject nid 4 : 0x4 > Info: inject member d_hash > Info: inject value 74565 : 0x12345 > [..] > Info: inject dentry d_hash of nid 4: 0xc77b804e -> 0x12345 Ditto, better to include examples in manual, since it's a little bit complicated to use inject.f2fs. Thanks, > > Sheng Yong (10): > f2fs-tools: export is_digits > inject.f2fs: introduce inject.f2fs > inject.f2fs: add sb injection > inject.f2fs: add cp injection > inject.f2fs: add nat injection > inject.f2fs: add sit injection > inject.f2fs: add ssa injection > inject.f2fs: add node injection > inject.f2fs: add dentry injection > man: add inject.f2fs man page > > fsck/Makefile.am | 5 +- > fsck/fsck.h | 6 + > fsck/inject.c | 1104 +++++++++++++++++++++++++++++++++++++++++++++ > fsck/inject.h | 41 ++ > fsck/main.c | 38 +- > fsck/mount.c | 30 +- > include/f2fs_fs.h | 2 + > man/Makefile.am | 2 +- > man/inject.f2fs.8 | 225 +++++++++ > 9 files changed, 1447 insertions(+), 6 deletions(-) > create mode 100644 fsck/inject.c > create mode 100644 fsck/inject.h > create mode 100644 man/inject.f2fs.8 >
Hi Sheng, Can we have a list of tests to check the expected behavior of given inject.f2fs followed by fsck.f2fs? Thanks, On 07/04, Sheng Yong wrote: > This patchset introduces a new tool inject.f2fs to modify metadata or > data (directory entry) of f2fs image offline flexibly. > > With inject.f2fs, it is easier to generate a corrupted f2fs image, which > can help verify fsck or reproduce userspace behaviors of some a fault. > If option `--dry-run' is used, nothing really gets changed, and that > could be used to get the value of a specified field. > > inject.f2fs allows injecting some members in sb, cp, nat, sit, ssa, node > and dentry for now. The available members of each part can be listed by > executing command like: > inject.f2fs --sb 0 --help > > [...] > [mb]: > magic: inject magic number > s_stop_reason: inject s_stop_reason array selected by --idx <index> > s_errors: inject s_errors array selected by --idx <index> > devs.path: inject path in devs array selected by --idx <index> specified by --str <string> > > More injection fields are still work-in-progress. The TODO list includes: > * sb: features > * cp: fsync dnodes > * inode: extent, extra attrs, xattr > * data: fsverify? > * other fields which is needed to verify fsck > > v3: > * patch 3, fix error handling of inject sb->devs.path > * patch 3, do not ASSERT devs.path when inject is executed > * patch 3, allow inject to execute if image is umounted unclean > * patch 9, check whether blkaddr is valid before reading dentry block > * add is_digits and strtol entptr check when parsing numeric options > > v2: > * change print format of s_errors > * add write_raw_cp_blocks to write the first & last cp blocks directly > to avoid updating ckpt_flags by write_checkpoint > * call update_block if i_inode_checksum is injected to avoid updating > i_inode_checksum by update_inode > * go through all dentry blocks to find the target dir entry > > Examples: > > Inject sb's magic > ================= > inject.f2fs --sb 0 --mb magic --val 0x12345 $DEV > > Info: inject sb auto > Info: inject member magic > Info: inject value 74565 : 0x12345 > [...] > Info: inject magic of sb 1: 0xf2f52010 -> 0x12345 > [update_superblock: 890] Info: Done to update superblock > > Inject cp's cur_data_segno > ========================== > inject.f2fs --cp 0 --mb cur_data_segno --idx 1 --val 0x12345 $DEV > > Info: inject cp pack auto > Info: inject member cur_data_segno > Info: inject slot index 1 > Info: inject value 74565 : 0x12345 > [...] > Info: inject cur_data_segno[1] of cp 1: 0x4 -> 0x12345 > Info: write_checkpoint() cur_cp:1 > > Inject nat's ino > ================ > inject.f2fs --nat 0 --mb ino --nid $INO --val 0x12345 $DEV > > Info: inject nat pack auto > Info: inject nid 4 : 0x4 > Info: inject member ino > Info: inject value 74565 : 0x12345 > [...] > Info: inject nat entry ino of nid 4 in pack 1: 4 -> 74565 > > Inject ssa's nid > ================ > inject.f2fs --ssa --blk $BLK --mb nid --val 0x12345 $DEV > > Info: inject ssa > Info: inject blkaddr 7511 : 0x1d57 > Info: inject member nid > Info: inject value 74565 : 0x12345 > [...] > Info: auto idx = 343 > Info: inject summary entry nid of block 0x1d57: 0x4 -> 0x12345 > > Inject inode's i_addr > ===================== > inject.f2fs --node --nid $INO --mb i_addr --idx 100 --val 0x12345 $DEV > > Info: inject node > Info: inject nid 4 : 0x4 > Info: inject member i_addr > Info: inject slot index 100 > Info: inject value 74565 : 0x12345 > [...] > Info: inject inode i_addr[100] of nid 4: 0x20864 -> 0x12345 > > Inject inode's dentry hash > ========================== > inject.f2fs --dent --nid $INO --mb d_hash --val 0x12345 $DEV > > Info: inject dentry > Info: inject nid 4 : 0x4 > Info: inject member d_hash > Info: inject value 74565 : 0x12345 > [..] > Info: inject dentry d_hash of nid 4: 0xc77b804e -> 0x12345 > > Sheng Yong (10): > f2fs-tools: export is_digits > inject.f2fs: introduce inject.f2fs > inject.f2fs: add sb injection > inject.f2fs: add cp injection > inject.f2fs: add nat injection > inject.f2fs: add sit injection > inject.f2fs: add ssa injection > inject.f2fs: add node injection > inject.f2fs: add dentry injection > man: add inject.f2fs man page > > fsck/Makefile.am | 5 +- > fsck/fsck.h | 6 + > fsck/inject.c | 1104 +++++++++++++++++++++++++++++++++++++++++++++ > fsck/inject.h | 41 ++ > fsck/main.c | 38 +- > fsck/mount.c | 30 +- > include/f2fs_fs.h | 2 + > man/Makefile.am | 2 +- > man/inject.f2fs.8 | 225 +++++++++ > 9 files changed, 1447 insertions(+), 6 deletions(-) > create mode 100644 fsck/inject.c > create mode 100644 fsck/inject.h > create mode 100644 man/inject.f2fs.8 > > -- > 2.40.1
On 07/10, Jaegeuk Kim wrote: > Hi Sheng, > > Can we have a list of tests to check the expected behavior of given > inject.f2fs followed by fsck.f2fs? And we need this. https://lore.kernel.org/linux-f2fs-devel/20240710230319.33025-1-jaegeuk@kernel.org/T/#u > > Thanks, > > On 07/04, Sheng Yong wrote: > > This patchset introduces a new tool inject.f2fs to modify metadata or > > data (directory entry) of f2fs image offline flexibly. > > > > With inject.f2fs, it is easier to generate a corrupted f2fs image, which > > can help verify fsck or reproduce userspace behaviors of some a fault. > > If option `--dry-run' is used, nothing really gets changed, and that > > could be used to get the value of a specified field. > > > > inject.f2fs allows injecting some members in sb, cp, nat, sit, ssa, node > > and dentry for now. The available members of each part can be listed by > > executing command like: > > inject.f2fs --sb 0 --help > > > > [...] > > [mb]: > > magic: inject magic number > > s_stop_reason: inject s_stop_reason array selected by --idx <index> > > s_errors: inject s_errors array selected by --idx <index> > > devs.path: inject path in devs array selected by --idx <index> specified by --str <string> > > > > More injection fields are still work-in-progress. The TODO list includes: > > * sb: features > > * cp: fsync dnodes > > * inode: extent, extra attrs, xattr > > * data: fsverify? > > * other fields which is needed to verify fsck > > > > v3: > > * patch 3, fix error handling of inject sb->devs.path > > * patch 3, do not ASSERT devs.path when inject is executed > > * patch 3, allow inject to execute if image is umounted unclean > > * patch 9, check whether blkaddr is valid before reading dentry block > > * add is_digits and strtol entptr check when parsing numeric options > > > > v2: > > * change print format of s_errors > > * add write_raw_cp_blocks to write the first & last cp blocks directly > > to avoid updating ckpt_flags by write_checkpoint > > * call update_block if i_inode_checksum is injected to avoid updating > > i_inode_checksum by update_inode > > * go through all dentry blocks to find the target dir entry > > > > Examples: > > > > Inject sb's magic > > ================= > > inject.f2fs --sb 0 --mb magic --val 0x12345 $DEV > > > > Info: inject sb auto > > Info: inject member magic > > Info: inject value 74565 : 0x12345 > > [...] > > Info: inject magic of sb 1: 0xf2f52010 -> 0x12345 > > [update_superblock: 890] Info: Done to update superblock > > > > Inject cp's cur_data_segno > > ========================== > > inject.f2fs --cp 0 --mb cur_data_segno --idx 1 --val 0x12345 $DEV > > > > Info: inject cp pack auto > > Info: inject member cur_data_segno > > Info: inject slot index 1 > > Info: inject value 74565 : 0x12345 > > [...] > > Info: inject cur_data_segno[1] of cp 1: 0x4 -> 0x12345 > > Info: write_checkpoint() cur_cp:1 > > > > Inject nat's ino > > ================ > > inject.f2fs --nat 0 --mb ino --nid $INO --val 0x12345 $DEV > > > > Info: inject nat pack auto > > Info: inject nid 4 : 0x4 > > Info: inject member ino > > Info: inject value 74565 : 0x12345 > > [...] > > Info: inject nat entry ino of nid 4 in pack 1: 4 -> 74565 > > > > Inject ssa's nid > > ================ > > inject.f2fs --ssa --blk $BLK --mb nid --val 0x12345 $DEV > > > > Info: inject ssa > > Info: inject blkaddr 7511 : 0x1d57 > > Info: inject member nid > > Info: inject value 74565 : 0x12345 > > [...] > > Info: auto idx = 343 > > Info: inject summary entry nid of block 0x1d57: 0x4 -> 0x12345 > > > > Inject inode's i_addr > > ===================== > > inject.f2fs --node --nid $INO --mb i_addr --idx 100 --val 0x12345 $DEV > > > > Info: inject node > > Info: inject nid 4 : 0x4 > > Info: inject member i_addr > > Info: inject slot index 100 > > Info: inject value 74565 : 0x12345 > > [...] > > Info: inject inode i_addr[100] of nid 4: 0x20864 -> 0x12345 > > > > Inject inode's dentry hash > > ========================== > > inject.f2fs --dent --nid $INO --mb d_hash --val 0x12345 $DEV > > > > Info: inject dentry > > Info: inject nid 4 : 0x4 > > Info: inject member d_hash > > Info: inject value 74565 : 0x12345 > > [..] > > Info: inject dentry d_hash of nid 4: 0xc77b804e -> 0x12345 > > > > Sheng Yong (10): > > f2fs-tools: export is_digits > > inject.f2fs: introduce inject.f2fs > > inject.f2fs: add sb injection > > inject.f2fs: add cp injection > > inject.f2fs: add nat injection > > inject.f2fs: add sit injection > > inject.f2fs: add ssa injection > > inject.f2fs: add node injection > > inject.f2fs: add dentry injection > > man: add inject.f2fs man page > > > > fsck/Makefile.am | 5 +- > > fsck/fsck.h | 6 + > > fsck/inject.c | 1104 +++++++++++++++++++++++++++++++++++++++++++++ > > fsck/inject.h | 41 ++ > > fsck/main.c | 38 +- > > fsck/mount.c | 30 +- > > include/f2fs_fs.h | 2 + > > man/Makefile.am | 2 +- > > man/inject.f2fs.8 | 225 +++++++++ > > 9 files changed, 1447 insertions(+), 6 deletions(-) > > create mode 100644 fsck/inject.c > > create mode 100644 fsck/inject.h > > create mode 100644 man/inject.f2fs.8 > > > > -- > > 2.40.1 > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
On 2024/7/11 7:04, Jaegeuk Kim wrote: > On 07/10, Jaegeuk Kim wrote: >> Hi Sheng, >> >> Can we have a list of tests to check the expected behavior of given >> inject.f2fs followed by fsck.f2fs? Hi, Jaegeuk, Yes, I'm working on the testcases of fsck.f2fs and inject.f2fs. The basic idea is: 1. creates an image by mkfs, and prepare some files for injection 2. inject faults into the image 3. fsck fixes the image 4. compare the output with the expected output (which is filtered to remove information that changes frequently, like line number or block address). Some testcases may requiremounting the image again to check file states. For some complex testcases where there is no simple way to create the scenario, we have to prepare the image file (with injected faults) in advance. > > And we need this. > > https://lore.kernel.org/linux-f2fs-devel/20240710230319.33025-1-jaegeuk@kernel.org/T/#u > Thanks for the fix. Thanks, shengyong >> >> Thanks, >> >> On 07/04, Sheng Yong wrote: >>> This patchset introduces a new tool inject.f2fs to modify metadata or >>> data (directory entry) of f2fs image offline flexibly. >>> >>> With inject.f2fs, it is easier to generate a corrupted f2fs image, which >>> can help verify fsck or reproduce userspace behaviors of some a fault. >>> If option `--dry-run' is used, nothing really gets changed, and that >>> could be used to get the value of a specified field. >>> >>> inject.f2fs allows injecting some members in sb, cp, nat, sit, ssa, node >>> and dentry for now. The available members of each part can be listed by >>> executing command like: >>> inject.f2fs --sb 0 --help >>> >>> [...] >>> [mb]: >>> magic: inject magic number >>> s_stop_reason: inject s_stop_reason array selected by --idx <index> >>> s_errors: inject s_errors array selected by --idx <index> >>> devs.path: inject path in devs array selected by --idx <index> specified by --str <string> >>> >>> More injection fields are still work-in-progress. The TODO list includes: >>> * sb: features >>> * cp: fsync dnodes >>> * inode: extent, extra attrs, xattr >>> * data: fsverify? >>> * other fields which is needed to verify fsck >>> >>> v3: >>> * patch 3, fix error handling of inject sb->devs.path >>> * patch 3, do not ASSERT devs.path when inject is executed >>> * patch 3, allow inject to execute if image is umounted unclean >>> * patch 9, check whether blkaddr is valid before reading dentry block >>> * add is_digits and strtol entptr check when parsing numeric options >>> >>> v2: >>> * change print format of s_errors >>> * add write_raw_cp_blocks to write the first & last cp blocks directly >>> to avoid updating ckpt_flags by write_checkpoint >>> * call update_block if i_inode_checksum is injected to avoid updating >>> i_inode_checksum by update_inode >>> * go through all dentry blocks to find the target dir entry >>> >>> Examples: >>> >>> Inject sb's magic >>> ================= >>> inject.f2fs --sb 0 --mb magic --val 0x12345 $DEV >>> >>> Info: inject sb auto >>> Info: inject member magic >>> Info: inject value 74565 : 0x12345 >>> [...] >>> Info: inject magic of sb 1: 0xf2f52010 -> 0x12345 >>> [update_superblock: 890] Info: Done to update superblock >>> >>> Inject cp's cur_data_segno >>> ========================== >>> inject.f2fs --cp 0 --mb cur_data_segno --idx 1 --val 0x12345 $DEV >>> >>> Info: inject cp pack auto >>> Info: inject member cur_data_segno >>> Info: inject slot index 1 >>> Info: inject value 74565 : 0x12345 >>> [...] >>> Info: inject cur_data_segno[1] of cp 1: 0x4 -> 0x12345 >>> Info: write_checkpoint() cur_cp:1 >>> >>> Inject nat's ino >>> ================ >>> inject.f2fs --nat 0 --mb ino --nid $INO --val 0x12345 $DEV >>> >>> Info: inject nat pack auto >>> Info: inject nid 4 : 0x4 >>> Info: inject member ino >>> Info: inject value 74565 : 0x12345 >>> [...] >>> Info: inject nat entry ino of nid 4 in pack 1: 4 -> 74565 >>> >>> Inject ssa's nid >>> ================ >>> inject.f2fs --ssa --blk $BLK --mb nid --val 0x12345 $DEV >>> >>> Info: inject ssa >>> Info: inject blkaddr 7511 : 0x1d57 >>> Info: inject member nid >>> Info: inject value 74565 : 0x12345 >>> [...] >>> Info: auto idx = 343 >>> Info: inject summary entry nid of block 0x1d57: 0x4 -> 0x12345 >>> >>> Inject inode's i_addr >>> ===================== >>> inject.f2fs --node --nid $INO --mb i_addr --idx 100 --val 0x12345 $DEV >>> >>> Info: inject node >>> Info: inject nid 4 : 0x4 >>> Info: inject member i_addr >>> Info: inject slot index 100 >>> Info: inject value 74565 : 0x12345 >>> [...] >>> Info: inject inode i_addr[100] of nid 4: 0x20864 -> 0x12345 >>> >>> Inject inode's dentry hash >>> ========================== >>> inject.f2fs --dent --nid $INO --mb d_hash --val 0x12345 $DEV >>> >>> Info: inject dentry >>> Info: inject nid 4 : 0x4 >>> Info: inject member d_hash >>> Info: inject value 74565 : 0x12345 >>> [..] >>> Info: inject dentry d_hash of nid 4: 0xc77b804e -> 0x12345 >>> >>> Sheng Yong (10): >>> f2fs-tools: export is_digits >>> inject.f2fs: introduce inject.f2fs >>> inject.f2fs: add sb injection >>> inject.f2fs: add cp injection >>> inject.f2fs: add nat injection >>> inject.f2fs: add sit injection >>> inject.f2fs: add ssa injection >>> inject.f2fs: add node injection >>> inject.f2fs: add dentry injection >>> man: add inject.f2fs man page >>> >>> fsck/Makefile.am | 5 +- >>> fsck/fsck.h | 6 + >>> fsck/inject.c | 1104 +++++++++++++++++++++++++++++++++++++++++++++ >>> fsck/inject.h | 41 ++ >>> fsck/main.c | 38 +- >>> fsck/mount.c | 30 +- >>> include/f2fs_fs.h | 2 + >>> man/Makefile.am | 2 +- >>> man/inject.f2fs.8 | 225 +++++++++ >>> 9 files changed, 1447 insertions(+), 6 deletions(-) >>> create mode 100644 fsck/inject.c >>> create mode 100644 fsck/inject.h >>> create mode 100644 man/inject.f2fs.8 >>> >>> -- >>> 2.40.1 >> >> >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel