| Message ID | 20190212152203.37434-1-bfoster@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | fstests/generic: cgroup aware writeback accounting test | expand |
On Tue, Feb 12, 2019 at 10:22:03AM -0500, Brian Foster wrote: > A test to perform reads/writes under various cgroups and verify that > I/Os are accounted properly according to cgroup aware writeback. > This is a generic test, but not all commonly used local filesystems > support cgroup aware writeback at the moment (i.e., XFS). Therefore, > this test currently requires ext4 or btrfs for the time being. > > The common/cgroup2 file is copied from a separate cgroup related > patch from Shaohua Li that never made it upstream. > > Signed-off-by: Brian Foster <bfoster@redhat.com> > Signed-off-by: Shaohua Li <shli@fb.com> > --- > > This is a test I had lying around from previous cgawb on XFS discussions > that I never got back to finishing off. This currently passes on > ext4/btrfs, fails on upstream XFS and survives XFS with the old cgawb > enablement patch applied. > > Shaohua's patch referenced in the commit log is available here[1]. > > Brian > > [1] https://marc.info/?l=fstests&m=152175327503759&w=2 > > common/cgroup2 | 15 +++++ > tests/generic/999 | 141 ++++++++++++++++++++++++++++++++++++++++++ > tests/generic/999.out | 14 +++++ > tests/generic/group | 1 + > 4 files changed, 171 insertions(+) > create mode 100644 common/cgroup2 > create mode 100755 tests/generic/999 > create mode 100644 tests/generic/999.out > > diff --git a/common/cgroup2 b/common/cgroup2 > new file mode 100644 > index 00000000..f89825e2 > --- /dev/null > +++ b/common/cgroup2 > @@ -0,0 +1,15 @@ > +# cgroup2 specific common functions > + > +export CGROUP2_PATH="${CGROUP2_PATH:-/sys/fs/cgroup}" > + > +_require_cgroup2() > +{ > + if [ ! -f "${CGROUP2_PATH}/cgroup.subtree_control" ]; then > + _notrun "Test requires cgroup2 enabled" > + fi > + if [[ ! $(cat ${CGROUP2_PATH}/cgroup.controllers) =~ $1 ]]; then > + _notrun "Cgroup2 doesn't support $1 controller $1" > + fi > +} > + > +/bin/true > diff --git a/tests/generic/999 b/tests/generic/999 > new file mode 100755 > index 00000000..1664da60 > --- /dev/null > +++ b/tests/generic/999 > @@ -0,0 +1,141 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. > +# > +# FS QA Test 999 > +# > +# This test verifies that cgroup aware writeback properly accounts I/Os in > +# various scenarios. We perform reads/writes from different combinations of > +# cgroups and verify that pages are accounted against the group that brought > +# them into cache. > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > + > + echo $$ > $cgdir/cgroup.procs > + rmdir $cgdir/$seq-cg* > /dev/null 2>&1 > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/filter > +. ./common/cgroup2 > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > + > +# Modify as appropriate. > +_supported_fs ext4 btrfs Then this should be a 'shared' test, I renamed it to shared/011. > +_supported_os Linux > +_require_scratch > +_require_cgroup2 io > + > +smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) > +sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) > +cgdir=$CGROUP2_PATH > + > +iosize=$((1024 * 1024 * 8)) > + > +# Check cgroup read/write charges against expected values. Allow for some > +# tolerance as different filesystems seem to account slightly differently. > +check_cg() > +{ > + cgroot=$1 > + cgname=$(basename $cgroot) > + expectedread=$2 > + expectedwrite=$3 > + rbytes=0 > + wbytes=0 > + > + iobytes=`cat $cgroot/io.stat | grep $smajor:$sminor` > + if [ $? == 0 ]; then > + rbytes=`echo $iobytes | awk '{ print $2 }' | \ > + awk -F = '{ print $2 }'` > + wbytes=`echo $iobytes | awk '{ print $3 }' | \ > + awk -F = '{ print $2 }'` > + fi > + > + _within_tolerance "read" $rbytes $expectedread 5% -v > + _within_tolerance "write" $wbytes $expectedwrite 5% -v > +} > + > +# Move current process to another cgroup. > +switch_cg() > +{ > + mkdir -p $1 > + echo $$ > $1/cgroup.procs > +} > + > +# Reset cgroup state for a new test. > +reset() > +{ > + echo $$ > $cgdir/cgroup.procs > + rmdir $cgdir/$seq-cg* > /dev/null 2>&1 > + $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ > + >> $seqres.full 2>&1 > + _scratch_cycle_mount || _fail "mount failed" _scratch_cycle_mount will call _fail too, no need to _fail explicitly here. Otherwise looks good to me. Thanks, Eryu > + stat $SCRATCH_MNT/file > /dev/null > +} > + > +_scratch_mkfs >> $seqres.full 2>&1 > +_scratch_mount > + > +echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control" > + > +# Read and write from a single group. > +echo "read/write" > +reset > +switch_cg $cgdir/$seq-cg > +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" -c fsync \ > + $SCRATCH_MNT/file >> $seqres.full 2>&1 > +switch_cg $cgdir > +$XFS_IO_PROG -c fsync $SCRATCH_MNT/file > +check_cg $cgdir/$seq-cg $iosize $iosize > + > +# Write from one cgroup then read and write from a second. Writes are charged to > +# the first group and nothing to the second. > +echo "write -> read/write" > +reset > +switch_cg $cgdir/$seq-cg > +$XFS_IO_PROG -c "pwrite 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1 > +switch_cg $cgdir/$seq-cg-2 > +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \ > + >> $seqres.full 2>&1 > +switch_cg $cgdir > +$XFS_IO_PROG -c fsync $SCRATCH_MNT/file > +check_cg $cgdir/$seq-cg 0 $iosize > +check_cg $cgdir/$seq-cg-2 0 0 > + > +# Read from one cgroup, read & write from a second. Both reads and writes are > +# charged to the first group and nothing to the second. > +echo "read -> read/write" > +reset > +switch_cg $cgdir/$seq-cg > +$XFS_IO_PROG -c "pread 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1 > +switch_cg $cgdir/$seq-cg-2 > +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \ > + >> $seqres.full 2>&1 > +switch_cg $cgdir > +$XFS_IO_PROG -c fsync $SCRATCH_MNT/file > +check_cg $cgdir/$seq-cg $iosize $iosize > +check_cg $cgdir/$seq-cg-2 0 0 > + > +echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" > + > +# success, all done > +status=0 > +exit > diff --git a/tests/generic/999.out b/tests/generic/999.out > new file mode 100644 > index 00000000..d93b7089 > --- /dev/null > +++ b/tests/generic/999.out > @@ -0,0 +1,14 @@ > +QA output created by 999 > +read/write > +read is in range > +write is in range > +write -> read/write > +read is in range > +write is in range > +read is in range > +write is in range > +read -> read/write > +read is in range > +write is in range > +read is in range > +write is in range > diff --git a/tests/generic/group b/tests/generic/group > index cfd003d3..9d7d7613 100644 > --- a/tests/generic/group > +++ b/tests/generic/group > @@ -528,3 +528,4 @@ > 523 auto quick attr > 524 auto quick > 525 auto quick rw > +999 auto quick > -- > 2.17.2 >
diff --git a/common/cgroup2 b/common/cgroup2 new file mode 100644 index 00000000..f89825e2 --- /dev/null +++ b/common/cgroup2 @@ -0,0 +1,15 @@ +# cgroup2 specific common functions + +export CGROUP2_PATH="${CGROUP2_PATH:-/sys/fs/cgroup}" + +_require_cgroup2() +{ + if [ ! -f "${CGROUP2_PATH}/cgroup.subtree_control" ]; then + _notrun "Test requires cgroup2 enabled" + fi + if [[ ! $(cat ${CGROUP2_PATH}/cgroup.controllers) =~ $1 ]]; then + _notrun "Cgroup2 doesn't support $1 controller $1" + fi +} + +/bin/true diff --git a/tests/generic/999 b/tests/generic/999 new file mode 100755 index 00000000..1664da60 --- /dev/null +++ b/tests/generic/999 @@ -0,0 +1,141 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 999 +# +# This test verifies that cgroup aware writeback properly accounts I/Os in +# various scenarios. We perform reads/writes from different combinations of +# cgroups and verify that pages are accounted against the group that brought +# them into cache. +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* + + echo $$ > $cgdir/cgroup.procs + rmdir $cgdir/$seq-cg* > /dev/null 2>&1 +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/cgroup2 + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs ext4 btrfs +_supported_os Linux +_require_scratch +_require_cgroup2 io + +smajor=$((0x`stat -L -c %t $SCRATCH_DEV`)) +sminor=$((0x`stat -L -c %T $SCRATCH_DEV`)) +cgdir=$CGROUP2_PATH + +iosize=$((1024 * 1024 * 8)) + +# Check cgroup read/write charges against expected values. Allow for some +# tolerance as different filesystems seem to account slightly differently. +check_cg() +{ + cgroot=$1 + cgname=$(basename $cgroot) + expectedread=$2 + expectedwrite=$3 + rbytes=0 + wbytes=0 + + iobytes=`cat $cgroot/io.stat | grep $smajor:$sminor` + if [ $? == 0 ]; then + rbytes=`echo $iobytes | awk '{ print $2 }' | \ + awk -F = '{ print $2 }'` + wbytes=`echo $iobytes | awk '{ print $3 }' | \ + awk -F = '{ print $2 }'` + fi + + _within_tolerance "read" $rbytes $expectedread 5% -v + _within_tolerance "write" $wbytes $expectedwrite 5% -v +} + +# Move current process to another cgroup. +switch_cg() +{ + mkdir -p $1 + echo $$ > $1/cgroup.procs +} + +# Reset cgroup state for a new test. +reset() +{ + echo $$ > $cgdir/cgroup.procs + rmdir $cgdir/$seq-cg* > /dev/null 2>&1 + $XFS_IO_PROG -fc "pwrite 0 $iosize" $SCRATCH_MNT/file \ + >> $seqres.full 2>&1 + _scratch_cycle_mount || _fail "mount failed" + stat $SCRATCH_MNT/file > /dev/null +} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control" + +# Read and write from a single group. +echo "read/write" +reset +switch_cg $cgdir/$seq-cg +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" -c fsync \ + $SCRATCH_MNT/file >> $seqres.full 2>&1 +switch_cg $cgdir +$XFS_IO_PROG -c fsync $SCRATCH_MNT/file +check_cg $cgdir/$seq-cg $iosize $iosize + +# Write from one cgroup then read and write from a second. Writes are charged to +# the first group and nothing to the second. +echo "write -> read/write" +reset +switch_cg $cgdir/$seq-cg +$XFS_IO_PROG -c "pwrite 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1 +switch_cg $cgdir/$seq-cg-2 +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \ + >> $seqres.full 2>&1 +switch_cg $cgdir +$XFS_IO_PROG -c fsync $SCRATCH_MNT/file +check_cg $cgdir/$seq-cg 0 $iosize +check_cg $cgdir/$seq-cg-2 0 0 + +# Read from one cgroup, read & write from a second. Both reads and writes are +# charged to the first group and nothing to the second. +echo "read -> read/write" +reset +switch_cg $cgdir/$seq-cg +$XFS_IO_PROG -c "pread 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1 +switch_cg $cgdir/$seq-cg-2 +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \ + >> $seqres.full 2>&1 +switch_cg $cgdir +$XFS_IO_PROG -c fsync $SCRATCH_MNT/file +check_cg $cgdir/$seq-cg $iosize $iosize +check_cg $cgdir/$seq-cg-2 0 0 + +echo "-io" > $cgdir/cgroup.subtree_control || _fail "subtree control" + +# success, all done +status=0 +exit diff --git a/tests/generic/999.out b/tests/generic/999.out new file mode 100644 index 00000000..d93b7089 --- /dev/null +++ b/tests/generic/999.out @@ -0,0 +1,14 @@ +QA output created by 999 +read/write +read is in range +write is in range +write -> read/write +read is in range +write is in range +read is in range +write is in range +read -> read/write +read is in range +write is in range +read is in range +write is in range diff --git a/tests/generic/group b/tests/generic/group index cfd003d3..9d7d7613 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -528,3 +528,4 @@ 523 auto quick attr 524 auto quick 525 auto quick rw +999 auto quick