| Message ID | 20220317122839.44404-1-p.raghav@samsung.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] btrfs/237: Use zone cap instead of zone size in fill_size and rest calculation | expand |
On Thu, Mar 17, 2022 at 01:28:39PM +0100, Pankaj Raghav wrote: > This test will break when zone capacity != zone size because the > calculation of the size to be filled is done using zone_size instead of > the actual capacity available per zone. Fix it by using zone capacity. > > As a zoned device can have variable capacity, use the capacity from zone 5 > as it is the first data block group. > > The support to extract zone capacity was added to blkzone only from > version 2.37. So zcap will be used only when the blkzone version is > greater or equal to 2.37. > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > --- > Changes since v1: > - Add a filter that can append a 'cap' column for blkzone version <2.37 > (Naohiro) > - Extract the capacity from the zone on which the test is performed as a > zoned device can have variable zone capacity(Naohiro) > > common/filter | 12 ++++++++++++ > tests/btrfs/237 | 13 ++++++++----- > 2 files changed, 20 insertions(+), 5 deletions(-) > > diff --git a/common/filter b/common/filter > index 257227c2..5acd57dd 100644 > --- a/common/filter > +++ b/common/filter > @@ -634,5 +634,17 @@ _filter_bash() > sed -e "s/^bash: line 1: /bash: /" > } > > +# > +#blkzone report added zone capacity to be printed from v2.37. > +#This filter will add an extra column 'cap' with the same value of > +#'len'(zone size) for blkzone version < 2.37 > +# > +#Before: start: 0x000100000, len 0x040000, wptr 0x000000 .. > +#After: start: 0x000100000, len 0x040000, cap 0x040000, wptr 0x000000 .. ^- nit: need a space after '#' > +_filter_blkzone_report() > +{ > + $AWK_PROG -F "," 'BEGIN{OFS=",";} $3 !~ /cap/ {$2=$2","$2;} {print;}' | sed -e 's/len/cap/2' > +} > + > # make sure this script returns success > /bin/true > diff --git a/tests/btrfs/237 b/tests/btrfs/237 > index 96940549..33a0d81d 100755 > --- a/tests/btrfs/237 > +++ b/tests/btrfs/237 > @@ -35,9 +35,6 @@ get_data_bg() > grep -Eo "CHUNK_ITEM [[:digit:]]+" | cut -d ' ' -f 2 > } > > -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) > -zonesize=$((zonesize << 9)) > - > _scratch_mkfs >/dev/null 2>&1 > _scratch_mount -o commit=1 # 1s commit time to speed up test > > @@ -49,12 +46,18 @@ if [[ "$uuid" == "" ]]; then > exit 1 > fi > > +data_bg_zone_idx=5 Hmm. Yeah, it's currently there. But, I'd hesitate to use that like a constant number. Instead, how about querying "btrfs inspect-internal" like in get_data_bg() do? E.g, like this: # Assumes SINGLE data profile btrfs inspect-internal dump-tree -t CHUNK $SCRATCH_DEV | grep -A 4 CHUNK_ITEM |\ grep -A 3 'type DATA\|SINGLE' |\ grep -Eo 'offset [[:digit:]]+'| cut -d ' ' -f 2 This command dumps the location of data BG in a byte unit. Then, we can calculate the sector number and can supply it to "blkzone report -o". > +size=$($BLKZONE_PROG report -c $data_bg_zone_idx $SCRATCH_DEV | tail -1 |\ > + _filter_blkzone_report |\ > + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) > +size=$((size << 9)) > + > reclaim_threshold=75 > echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold > fill_percent=$((reclaim_threshold + 2)) > rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG > -fill_size=$((zonesize * fill_percent / 100)) > -rest=$((zonesize * rest_percent / 100)) > +fill_size=$((size * fill_percent / 100)) > +rest=$((size * rest_percent / 100)) > > # step 1, fill FS over $fillsize > $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full > -- > 2.25.1 >
On 2022-03-18 08:54, Naohiro Aota wrote: >> >> -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) >> -zonesize=$((zonesize << 9)) >> - >> _scratch_mkfs >/dev/null 2>&1 >> _scratch_mount -o commit=1 # 1s commit time to speed up test >> >> @@ -49,12 +46,18 @@ if [[ "$uuid" == "" ]]; then >> exit 1 >> fi >> >> +data_bg_zone_idx=5 > > Hmm. Yeah, it's currently there. But, I'd hesitate to use that like a > constant number. > > Instead, how about querying "btrfs inspect-internal" like in > get_data_bg() do? E.g, like this: > > # Assumes SINGLE data profile > btrfs inspect-internal dump-tree -t CHUNK $SCRATCH_DEV | grep -A 4 CHUNK_ITEM |\ > grep -A 3 'type DATA\|SINGLE' |\ > grep -Eo 'offset [[:digit:]]+'| cut -d ' ' -f 2 > > This command dumps the location of data BG in a byte unit. > > Then, we can calculate the sector number and can supply it to "blkzone > report -o". > Got it. I also prefer this over hardcoding. I think we can reuse the get_data_bg function as it also outputs the same data BG offset. Like this: start_data_bg_zone=$(get_data_bg) start_data_bg_zone=$((data_bg_zone >> 9)) size=$($BLKZONE_PROG report -o $start_data_bg_zone -l 1 $SCRATCH_DEV |\ ... Let me know your thoughts. >> +size=$($BLKZONE_PROG report -c $data_bg_zone_idx $SCRATCH_DEV | tail -1 |\ >> + _filter_blkzone_report |\ >> + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) >> +size=$((size << 9)) >> + >> reclaim_threshold=75 >> echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold >> fill_percent=$((reclaim_threshold + 2)) >> rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG >> -fill_size=$((zonesize * fill_percent / 100)) >> -rest=$((zonesize * rest_percent / 100)) >> +fill_size=$((size * fill_percent / 100)) >> +rest=$((size * rest_percent / 100)) >> >> # step 1, fill FS over $fillsize >> $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full >> -- >> 2.25.1
On Fri, Mar 18, 2022 at 10:28:44AM +0100, Pankaj Raghav wrote: > > > On 2022-03-18 08:54, Naohiro Aota wrote: > >> > >> -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) > >> -zonesize=$((zonesize << 9)) > >> - > >> _scratch_mkfs >/dev/null 2>&1 > >> _scratch_mount -o commit=1 # 1s commit time to speed up test > >> > >> @@ -49,12 +46,18 @@ if [[ "$uuid" == "" ]]; then > >> exit 1 > >> fi > >> > >> +data_bg_zone_idx=5 > > > > Hmm. Yeah, it's currently there. But, I'd hesitate to use that like a > > constant number. > > > > Instead, how about querying "btrfs inspect-internal" like in > > get_data_bg() do? E.g, like this: > > > > # Assumes SINGLE data profile > > btrfs inspect-internal dump-tree -t CHUNK $SCRATCH_DEV | grep -A 4 CHUNK_ITEM |\ > > grep -A 3 'type DATA\|SINGLE' |\ > > grep -Eo 'offset [[:digit:]]+'| cut -d ' ' -f 2 > > > > This command dumps the location of data BG in a byte unit. > > > > Then, we can calculate the sector number and can supply it to "blkzone > > report -o". > > > Got it. I also prefer this over hardcoding. > I think we can reuse the get_data_bg function as it also outputs the > same data BG offset. No, we cannot. They look the same in number, but they are different in type. The get_data_bg returns the logical address (virtual btrfs's internal address) of the BG. And, my command above returns the physical address (on-disk address) of the BG. > Like this: > > start_data_bg_zone=$(get_data_bg) > start_data_bg_zone=$((data_bg_zone >> 9)) > > size=$($BLKZONE_PROG report -o $start_data_bg_zone -l 1 $SCRATCH_DEV |\ > ... > Let me know your thoughts. > >> +size=$($BLKZONE_PROG report -c $data_bg_zone_idx $SCRATCH_DEV | tail -1 |\ > >> + _filter_blkzone_report |\ > >> + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) > >> +size=$((size << 9)) > >> + > >> reclaim_threshold=75 > >> echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold > >> fill_percent=$((reclaim_threshold + 2)) > >> rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG > >> -fill_size=$((zonesize * fill_percent / 100)) > >> -rest=$((zonesize * rest_percent / 100)) > >> +fill_size=$((size * fill_percent / 100)) > >> +rest=$((size * rest_percent / 100)) > >> > >> # step 1, fill FS over $fillsize > >> $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full > >> -- > >> 2.25.1 > > -- > Regards, > Pankaj
diff --git a/common/filter b/common/filter index 257227c2..5acd57dd 100644 --- a/common/filter +++ b/common/filter @@ -634,5 +634,17 @@ _filter_bash() sed -e "s/^bash: line 1: /bash: /" } +# +#blkzone report added zone capacity to be printed from v2.37. +#This filter will add an extra column 'cap' with the same value of +#'len'(zone size) for blkzone version < 2.37 +# +#Before: start: 0x000100000, len 0x040000, wptr 0x000000 .. +#After: start: 0x000100000, len 0x040000, cap 0x040000, wptr 0x000000 .. +_filter_blkzone_report() +{ + $AWK_PROG -F "," 'BEGIN{OFS=",";} $3 !~ /cap/ {$2=$2","$2;} {print;}' | sed -e 's/len/cap/2' +} + # make sure this script returns success /bin/true diff --git a/tests/btrfs/237 b/tests/btrfs/237 index 96940549..33a0d81d 100755 --- a/tests/btrfs/237 +++ b/tests/btrfs/237 @@ -35,9 +35,6 @@ get_data_bg() grep -Eo "CHUNK_ITEM [[:digit:]]+" | cut -d ' ' -f 2 } -zonesize=$(cat /sys/block/$(_short_dev $SCRATCH_DEV)/queue/chunk_sectors) -zonesize=$((zonesize << 9)) - _scratch_mkfs >/dev/null 2>&1 _scratch_mount -o commit=1 # 1s commit time to speed up test @@ -49,12 +46,18 @@ if [[ "$uuid" == "" ]]; then exit 1 fi +data_bg_zone_idx=5 +size=$($BLKZONE_PROG report -c $data_bg_zone_idx $SCRATCH_DEV | tail -1 |\ + _filter_blkzone_report |\ + grep -Po "cap ([0x\d]+)+" | cut -d ' ' -f 2) +size=$((size << 9)) + reclaim_threshold=75 echo $reclaim_threshold > /sys/fs/btrfs/"$uuid"/bg_reclaim_threshold fill_percent=$((reclaim_threshold + 2)) rest_percent=$((90 - fill_percent)) # make sure we're not creating a new BG -fill_size=$((zonesize * fill_percent / 100)) -rest=$((zonesize * rest_percent / 100)) +fill_size=$((size * fill_percent / 100)) +rest=$((size * rest_percent / 100)) # step 1, fill FS over $fillsize $XFS_IO_PROG -fc "pwrite 0 $fill_size" $SCRATCH_MNT/$seq.test1 >> $seqres.full
This test will break when zone capacity != zone size because the calculation of the size to be filled is done using zone_size instead of the actual capacity available per zone. Fix it by using zone capacity. As a zoned device can have variable capacity, use the capacity from zone 5 as it is the first data block group. The support to extract zone capacity was added to blkzone only from version 2.37. So zcap will be used only when the blkzone version is greater or equal to 2.37. Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> --- Changes since v1: - Add a filter that can append a 'cap' column for blkzone version <2.37 (Naohiro) - Extract the capacity from the zone on which the test is performed as a zoned device can have variable zone capacity(Naohiro) common/filter | 12 ++++++++++++ tests/btrfs/237 | 13 ++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-)