mbox series

[00/13] btrfs-progs: fixes of file extent in original and lowmem check

Message ID 20181023094147.7906-1-suy.fnst@cn.fujitsu.com (mailing list archive)
Headers show
Series btrfs-progs: fixes of file extent in original and lowmem check | expand

Message

Su Yue Oct. 23, 2018, 9:41 a.m. UTC 
This patchset can be fetched from my repo:
https://github.com/Damenly/btrfs-progs/tree/file_extent_fixes
which is based on kdave/devel whose HEAD is:
commit 4f20c27ab33aab3efffe13cdae1b8837c821d0d7 (kdave/devel)
Author: Nikolay Borisov <nborisov@suse.com>
Date:   Fri Jun 15 07:13:50 2018 +0000

    btrfs-progs: tests: test for FST corruption detection/repair

This set fixes bugs of checking unaligned disk_bytenr extent_data and
orphan extent. Lowmem part patches can be used in common, so I put
this two fixes together.

For unaligned disk_bytenr extent_data, now original and lowmem check
both delete the corrupted part and punch a hole.

For orphan extent, lowmem mode has no change here.
Original mode discards function of detect and repair orphan extents
according extent items. Because as corruptions the community reported,
extent tree is more problematic than fs trees.

patch[1-2] fix minor bugs in lowmem repair.
patch[3] fixes false alert about repaired extent item in lowmem mode.
patch[4] fixes annoying alerts about gap in file extent in lowmem mode.
patch[5,6,7] enable check and repair unaligned disk_bytenr file extent
	     in lowmem mode.
patch[8,9] revert support for orphan extent in original mode.
patch[10] fixes bug about finding right backrefs in original mode.
patch[11] adds support to detect and delete unaligned file extents
	       in original.
patch[12] adds a test image which lacks of a file extent.
patch[13] enables lowmem repair of test case fsck-tests/001 which has
	  	  unaligned disk_bytenr file extent.


Lu Fengqi (2):
  btrfs-progs: lowmem: fix false alert about the existence of gaps in
    the check_file_extent
  btrfs-progs: tests: add case for inode lose one file extent

Su Yanjun (4):
  btrfs-progs: Revert "btrfs-progs: Add repair and report function for
    orphan file extent."
  btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to
    corresponding root."
  btrfs-progs: check: fix bug in find_possible_backrefs
  btrfs-progs: check: Delete file extent item with unaligned extent
    backref

Su Yue (7):
  btrfs-progs: lowmem: add argument path to punch_extent_hole()
  btrfs-progs: lowmem: move nbytes check before isize check
  btrfs-progs: lowmem: fix false alert if extent item has been repaired
  btrfs-progs: lowmem: check unaligned disk_bytenr for extent_data
  btrfs-progs: lowmem: rename delete_extent_tree_item() to delete_item()
  btrfs-progs: lowmem: delete unaligned bytes extent data under repair
  btrfs-progs: fsck-test: enable lowmem repair for case 001

 check/main.c                                  | 581 +++++++++++-------
 check/mode-lowmem.c                           | 272 ++++----
 check/mode-original.h                         |  30 +-
 ctree.h                                       |  10 +-
 disk-io.c                                     |   2 +-
 .../.lowmem_repairable                        |   0
 .../.lowmem_repairable                        |   0
 .../default_case.img                          | Bin 0 -> 3072 bytes
 8 files changed, 538 insertions(+), 357 deletions(-)
 create mode 100644 tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable
 create mode 100644 tests/fsck-tests/038-missing-one-file-extent/.lowmem_repairable
 create mode 100644 tests/fsck-tests/038-missing-one-file-extent/default_case.img

Comments

Su Yue Oct. 23, 2018, 9:41 a.m. UTC | #1 
From: Lu Fengqi <lufq.fnst@cn.fujitsu.com>

The missing extent will lead to the existence of the gap between adjacent
extents. The fsck should can detect the gap correctly and repair by punch
a hole.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
---
 .../.lowmem_repairable                           |   0
 .../038-missing-one-file-extent/default_case.img | Bin 0 -> 3072 bytes
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/038-missing-one-file-extent/.lowmem_repairable
 create mode 100644 tests/fsck-tests/038-missing-one-file-extent/default_case.img

diff --git a/tests/fsck-tests/038-missing-one-file-extent/.lowmem_repairable b/tests/fsck-tests/038-missing-one-file-extent/.lowmem_repairable
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tests/fsck-tests/038-missing-one-file-extent/default_case.img b/tests/fsck-tests/038-missing-one-file-extent/default_case.img
new file mode 100644
index 0000000000000000000000000000000000000000..7cc2cd403977a805a58c61f007d517f24412800c
GIT binary patch
literal 3072
zcmeH{i!<Bl8pqKRLc?y;hF+rDQnDI!b>kMH-NSCzC9O-$A};5kw5V%@DiYed7p<*T
zBCb_x>r!!xoRp>As<<aENgC8$98|M}{EnJ4d#0T^f54gPGw(d__j%vvo%z0>dFM^_
zeGodS=|-3OPwd~FrT50J;D21S!0y&kQVM&yZ!a5s?6Mf%(*qwjzr+^;UkLm^5{Mcb
z=ZD!ANGr<=Cz{?bgfa@di095{Dd%#|>FbkzSEH`2+QBXTnXB!`ZXAbcT!BcZ8Kos^
zAM%uOxQ?}K@|M1b*Ose0ngIO<LjF1{(DQp*JT>7^&<Uj}S@uwb@K%dpGho?nsy;{Y
z&pZRVd7ZTB5%i>#VKUmwGrnwDPXAS6`2~d}mb_iGjrNbwVs$iAV_MJiLB$=}5>IFv
z>9Ri9im7d}52u!uU;RuA*+v|@A?qZOI>MK>1rih=-)ty1IMv*HJ)YB%@wx!iHx2ru
zVDDgptW-h_?q1Z`@;K(&3k%im)!FTRCg;f|Inb8yu+XcA9<^L8>Khow-FyDFC;^gG
zOwJ+OT{;7@rp79iArF~CkGX{pp^9c4vsD}>80t+E?+RPRbW+8i_X2*nD`tBEv-J1Q
zE9F)8^}KYb&GT_wOZ6C|e&%(H8I6H4p=5e@i$uj@;~5j(!H$!^z$%*%VOh2HE)yva
zd8FPL-eFz-0lz#m=>K>!e#yr$5N>bm{3aiAk#e~^cCh(ebBn~)b|sg5`9qbD`}gh}
zE?D<cElXk^u<jVeb1BHFkXgI$=RspodUFyB16r888|hdHY-eZtsEUm{e-{MT{_bI)
zW1xTe;kg^`#NTb@zczd5&+&qh|4509)l9T_d_u!p1AGOXh#al#XwAlY4rypB-@fxA
z5%R$r@z_l!*b;HQNQJ0k(s@$Ve}>6p6Dh>mf)GEPo96tw@shabCWYu*m?z?LU#}4D
z>1~rwv4T7r2KdL;+nks)n4dSg!u-tPih`(tQA%k@-O5lNLA1R&G|I)q(EzbuhcH^y
zlptKhz7%DY;Ni*wiy-?`-}R;wzKdoyp0RRvn=`x>01{1;U;$hzB!XNTl|dC6XHObS
z{O!GoNRC>j(Fg>-r086cF1p0L-C)pv;4``xRMXN0y!O+HQ)LC4%dP}$^#EMn6hGR$
zBVkTDp%j0khkX0=)mk-sWR^ha%3aQ%Ym>zC{Rmwj)~nc=ZdKfQ2gFGhvH?w=UVzT@
zYX-4j%*_>F_^r%KsK&aEWSv%EqoziEDZzxB5kaF_PgXB&0B;ty03#MvxLCYB%wkC5
z-nrCC-o77M|0UB<%TEhAY!C1t3+BSloc45Bc$s<t9s^7KM^%!z_8KR^V;F9CYxK%_
ze5Hip<PLbSW(osJ_2v;@QM!CIa)ZHM!N`YTYQ*!95VOSAn#MxAEfU|m&xU+)RVMB#
zh){8H10+^tI<`GQk7AQf;Zv*|z|1Lwhlu^x>cSq)DQBx1Dl+qGRc%Bd>%V{IM!(f1
zAI=y@k3_2D-R7P(aRieO9G9@w5X_^Vh!o8vNZD5bVd*N2WOJL1IK`c@W-GXMxT<#K
z%K&ly4w~J$)3GB?rTIU*LDH-HR_h%I3&G)%I-CTz@lkVvF_aUl-4Vd--bsCSNHl>A
z=gU2HSpk^kJonFh!~-;C0=-@Mdj9Oj6Z5hyK#1b9oB?%E>Q4uBm$&bJqt&KnjUI>3
zE|${dEDooc<;sP;B3ZfFdpemmoqA7Ny1SfNZViPw$aljP1Qv6$qgZ|;*$T;tT<b1b
z2|uzzBS%nak(QAe9;otJRfL?#{JvaBL8@-lMuOodUcf04sInD*x)meu4NbUp7^YW5
zEC#ak*Yj~9Mw!dyM|9DRQ>ZnL9J<j}%qiB%JUb26X)Q~|MfYc8WBkP#Fv9Z9v%<=x
z{{Fw!r-$bhu5;>Q53&dG@glD(z+AUFdR``IE>`u9t8s2M#)!VLoz*qn(^<APzbG;4
z2^|N1<rrl%=&14RzE~DFl41Yr;4R9w2vEBJWWrT(ZMBuV6A~I;iqIPdP*sb-6sDy1
zWP2_nd6eE}RyZ=#jZWtnp?1Z-Pi`gkPg3)7XWLDfQnc6f$Di{l_ytzAYn6a#m9Fad
zY)Do2#4%6c?+h0m?6A;3YN+E|yKr2-MwcTO4bsXVUsb|rT~w*vj`LBD;f1L#1~YqS
zPFoG)&rbLR4pDqs0)z4QQ?Bh_Np}sjf$*ZDYADq^zAb^tJ_}(wJBI3dt#b|oI;qn_
z+Dp66%hUzf=4>%F)Z}L)g!)hY2a``!^Pvk>axI@a{;S%yJf|cuX~L_EDg#&9=npHi
cM@=@SQR^9<Z{oH*YHS^F;J<pmi2oA!H;vM1aR2}S

literal 0
HcmV?d00001
Su Yue Oct. 23, 2018, 9:41 a.m. UTC | #2 
Lowmem can repair after commit
       'btrfs-progs: lowmem: move nbytes check before isize check',
so add the beacon file.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
---
 tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable

diff --git a/tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable b/tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable
new file mode 100644
index 000000000000..e69de29bb2d1
Qu Wenruo Oct. 23, 2018, 9:45 a.m. UTC | #3 
On 2018/10/23 下午5:41, Su Yue wrote:
> This patchset can be fetched from my repo:
> https://github.com/Damenly/btrfs-progs/tree/file_extent_fixes
> which is based on kdave/devel whose HEAD is:
> commit 4f20c27ab33aab3efffe13cdae1b8837c821d0d7 (kdave/devel)
> Author: Nikolay Borisov <nborisov@suse.com>
> Date:   Fri Jun 15 07:13:50 2018 +0000
> 
>     btrfs-progs: tests: test for FST corruption detection/repair
> 
> This set fixes bugs of checking unaligned disk_bytenr extent_data and
> orphan extent. Lowmem part patches can be used in common, so I put
> this two fixes together.
> 
> For unaligned disk_bytenr extent_data, now original and lowmem check
> both delete the corrupted part and punch a hole.

This method looks pretty valid.

> 
> For orphan extent, lowmem mode has no change here.
> Original mode discards function of detect and repair orphan extents
> according extent items. Because as corruptions the community reported,
> extent tree is more problematic than fs trees.

This also looks pretty nice to me.

Extent tree can't really be trusted, not to mention under most case it's
extent tree itself corrupted.

More over, even we lost every tree block of extent tree, it's still
possible for btrfs-progs to recover the whole fs using btrfs-restore.

(In theory, we should provide a mount option to allow RO mount to read
the fs without a valid extent tree).

So this is the correct way to do.

Thanks,
Qu
> 
> patch[1-2] fix minor bugs in lowmem repair.
> patch[3] fixes false alert about repaired extent item in lowmem mode.
> patch[4] fixes annoying alerts about gap in file extent in lowmem mode.
> patch[5,6,7] enable check and repair unaligned disk_bytenr file extent
> 	     in lowmem mode.
> patch[8,9] revert support for orphan extent in original mode.
> patch[10] fixes bug about finding right backrefs in original mode.
> patch[11] adds support to detect and delete unaligned file extents
> 	       in original.
> patch[12] adds a test image which lacks of a file extent.
> patch[13] enables lowmem repair of test case fsck-tests/001 which has
> 	  	  unaligned disk_bytenr file extent.
> 
> 
> Lu Fengqi (2):
>   btrfs-progs: lowmem: fix false alert about the existence of gaps in
>     the check_file_extent
>   btrfs-progs: tests: add case for inode lose one file extent
> 
> Su Yanjun (4):
>   btrfs-progs: Revert "btrfs-progs: Add repair and report function for
>     orphan file extent."
>   btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to
>     corresponding root."
>   btrfs-progs: check: fix bug in find_possible_backrefs
>   btrfs-progs: check: Delete file extent item with unaligned extent
>     backref
> 
> Su Yue (7):
>   btrfs-progs: lowmem: add argument path to punch_extent_hole()
>   btrfs-progs: lowmem: move nbytes check before isize check
>   btrfs-progs: lowmem: fix false alert if extent item has been repaired
>   btrfs-progs: lowmem: check unaligned disk_bytenr for extent_data
>   btrfs-progs: lowmem: rename delete_extent_tree_item() to delete_item()
>   btrfs-progs: lowmem: delete unaligned bytes extent data under repair
>   btrfs-progs: fsck-test: enable lowmem repair for case 001
> 
>  check/main.c                                  | 581 +++++++++++-------
>  check/mode-lowmem.c                           | 272 ++++----
>  check/mode-original.h                         |  30 +-
>  ctree.h                                       |  10 +-
>  disk-io.c                                     |   2 +-
>  .../.lowmem_repairable                        |   0
>  .../.lowmem_repairable                        |   0
>  .../default_case.img                          | Bin 0 -> 3072 bytes
>  8 files changed, 538 insertions(+), 357 deletions(-)
>  create mode 100644 tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable
>  create mode 100644 tests/fsck-tests/038-missing-one-file-extent/.lowmem_repairable
>  create mode 100644 tests/fsck-tests/038-missing-one-file-extent/default_case.img
>