[V3,0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup

Hi, In order to test the non-trivial I/O path in the block layer for REQ_OP_WRITE_ZEROES allow write-zeroes on null_blk so we can write testcases with and without non-memory backed mode followed by few cleanup patches. Below is the test report with ext2/ext4 mkfs and a blktest waiting to get upstream for this patch-series. -ck v2->v3:- 1. Fix the commit log in patch 3. (Bart) 2. Fix the commit log in patch 4. (Bart) v1->v2:- * Merge patch 1 and 2 (Damien). * Change SECT_INX_IN_PAGE->SECT_OFFSET_IN_PAGE. (Damien) * Add reviewed-by tags. (Damien) * Add discard/write-zeroes module param to set the respective max sector length. Chaitanya Kulkarni (7): null_blk: allow REQ_OP_WRITE_ZEROES null_blk: code cleaup null_blk: initialize cmd->bio in __alloc_cmd() null_blk: don't use magic numbers in the code null_blk: remove extra space in switch condition null_blk: add param to set max disacrd sectors null_blk: add param to set max write-zeroes sects drivers/block/null_blk/main.c | 149 ++++++++++++++++++++++-------- drivers/block/null_blk/null_blk.h | 3 + 2 files changed, 116 insertions(+), 36 deletions(-) linux-block (for-next) # gitlog -8 5868a750b828 (HEAD -> for-next) null_blk: add param to set max write-zeroes sects 1dad765791bf null_blk: add param to set max disacrd sectors 28ff18cb23a0 null_blk: remove extra space in switch condition 574bbf59c69f null_blk: don't use magic numbers in the code 77fb0410ab98 null_blk: initialize cmd->bio in __alloc_cmd() 2f9c00f2e37c null_blk: code cleaup da21346d4ec1 null_blk: allow REQ_OP_WRITE_ZEROES c223decdb316 Merge branch 'for-next' of git://git.kernel.dk/linux-block into for-next linux-block (for-next) # ./compile_nullb.sh + umount /mnt/nullb0 umount: /mnt/nullb0: not mounted. + rmdir 'config/nullb/nullb*' rmdir: failed to remove 'config/nullb/nullb*': No such file or directory + dmesg -c + modprobe -r null_blk + lsmod + grep null_blk ++ nproc + make -j 48 M=drivers/block modules CC [M] drivers/block/null_blk/main.o CC [M] drivers/block/null_blk/trace.o CC [M] drivers/block/null_blk/zoned.o LD [M] drivers/block/null_blk/null_blk.o MODPOST drivers/block/Module.symvers LD [M] drivers/block/null_blk/null_blk.ko + HOST=drivers/block/null_blk/ ++ uname -r + HOST_DEST=/lib/modules/6.1.0-rc3blk+/kernel/drivers/block/null_blk/ + cp drivers/block/null_blk//null_blk.ko /lib/modules/6.1.0-rc3blk+/kernel/drivers/block/null_blk// + ls -lrth /lib/modules/6.1.0-rc3blk+/kernel/drivers/block/null_blk//null_blk.ko -rw-r--r--. 1 root root 1.2M Nov 15 11:32 /lib/modules/6.1.0-rc3blk+/kernel/drivers/block/null_blk//null_blk.ko + sleep 1 + dmesg -c linux-block (for-next) # cd ../blktests/ blktests (master) # ./check block/ block/001 (stress device hotplugging) [passed] runtime 49.525s ... 52.179s block/002 (remove a device while running blktrace) [passed] runtime 1.259s ... 1.236s block/006 (run null-blk in blocking mode) [passed] read iops 8698266 ... 8846254 runtime 9.888s ... 9.754s block/007 => nullb1 (test classic and hybrid IO polling) [passed] auto hybrid poll read iops 621194 ... 619725 auto hybrid poll read lat mean 1.359473 ... 1.366760 auto hybrid poll system cpu 73.634204% ... 74.170616% classic poll read iops 598502 ... 595781 classic poll read lat mean 1.418514 ... 1.409480 classic poll system cpu 73.455378% ... 68.109339% fixed hybrid poll read iops 618264 ... 611058 fixed hybrid poll read lat mean 1.363924 ... 1.388329 fixed hybrid poll system cpu 72.813239% ... 75.000000% no poll read iops 606814 ... 650481 no poll read lat mean 1.395035 ... 1.301867 no poll system cpu 72.389791% ... 76.616915% runtime 2.532s ... 2.507s block/009 (check page-cache coherency after BLKDISCARD) [passed] runtime 0.570s ... 0.580s block/010 (run I/O on null_blk with shared and non-shared tags) [passed] Individual tags read iops 7780737 ... 7903528 runtime 21.407s ... 20.915s Shared tags read iops 7602680 ... 7854501 block/014 (run null-blk with blk-mq and timeout injection configured) [passed] runtime 54.296s ... 49.470s block/015 (run null-blk on different schedulers with requeue injection configured) [passed] runtime 44.127s ... 45.185s block/016 (send a signal to a process waiting on a frozen queue) [passed] runtime 7.080s ... 7.081s block/017 (do I/O and check the inflight counter) [passed] runtime 1.681s ... 1.690s block/018 (do I/O and check iostats times) [passed] runtime 5.112s ... 5.122s block/020 (run null-blk on different schedulers with only one hardware tag) [passed] runtime 41.557s ... 41.539s block/021 (read/write nr_requests on null-blk with different schedulers) [passed] runtime 5.846s ... 5.477s block/022 (Test hang caused by freeze/unfreeze sequence) [passed] runtime 30.121s ... 30.181s block/023 (do I/O on all null_blk queue modes) [passed] runtime 0.266s ... 0.252s block/024 (do I/O faster than a jiffy and check iostats times) [passed] runtime 2.642s ... 2.655s block/025 (do a huge discard with 4k sector size) [passed] runtime 3.934s ... 3.938s block/027 (stress device hotplugging with running fio jobs and different schedulers) [passed] runtime 18.416s ... 18.700s block/028 (do I/O on scsi_debug with DIF/DIX enabled) [passed] runtime 13.616s ... 13.438s block/029 (trigger blk_mq_update_nr_hw_queues()) [failed] runtime 30.284s ... 30.287s --- tests/block/029.out 2022-07-18 16:14:37.196230036 -0700 +++ /root/blktests/results/nodev/block/029.out.bad 2022-11-15 11:38:15.571089141 -0800 @@ -1 +1,127 @@ +tests/block/029: line 24: echo: write error: Cannot allocate memory +tests/block/029: line 24: echo: write error: Cannot allocate memory +tests/block/029: line 24: echo: write error: Cannot allocate memory +tests/block/029: line 24: echo: write error: Cannot allocate memory +tests/block/029: line 24: echo: write error: Cannot allocate memory +tests/block/029: line 24: echo: write error: Cannot allocate memory +tests/block/029: line 24: echo: write error: Cannot allocate memory ... (Run 'diff -u tests/block/029.out /root/blktests/results/nodev/block/029.out.bad' to see the entire diff) block/030 (trigger the blk_mq_realloc_hw_ctxs() error path) [failed] runtime 2.192s ... 3.057s --- tests/block/030.out 2022-07-18 16:14:37.196230036 -0700 +++ /root/blktests/results/nodev/block/030.out.bad 2022-11-15 11:38:18.687161284 -0800 @@ -1 +1,101 @@ +tests/block/030: line 44: echo: write error: Cannot allocate memory +tests/block/030: line 44: echo: write error: Cannot allocate memory +tests/block/030: line 44: echo: write error: Cannot allocate memory +tests/block/030: line 44: echo: write error: Cannot allocate memory +tests/block/030: line 44: echo: write error: Cannot allocate memory +tests/block/030: line 44: echo: write error: Cannot allocate memory +tests/block/030: line 44: echo: write error: Cannot allocate memory ... (Run 'diff -u tests/block/030.out /root/blktests/results/nodev/block/030.out.bad' to see the entire diff) block/031 (do IO on null-blk with a host tag set) [passed] runtime 30.334s ... 30.334s blktests (master) # ls check CONTRIBUTING.md LICENSES Makefile nvme1n1 results tests common Documentation local-block-031-0-verify.state new README.md src blktests (master) # blktests (master) # blktests (master) # cd - /root/linux-block linux-block (for-next) # lsmod | grep null linux-block (for-next) # ./zeroout.sh 1 #################### BLKISZ 512 ##################### config/nullb/nullb0 ├── badblocks ├── blocking ├── blocksize ├── cache_size ├── completion_nsec ├── discard ├── home_node ├── hw_queue_depth ├── index ├── irqmode ├── max_discard_sectors ├── max_sectors ├── max_write_zeroes_sectors ├── mbps ├── memory_backed ├── no_sched ├── poll_queues ├── power ├── queue_mode ├── shared_tag_bitmap ├── size ├── submit_queues ├── use_per_node_hctx ├── virt_boundary ├── write_zeroes ├── zone_capacity ├── zoned ├── zone_max_active ├── zone_max_open ├── zone_nr_conv └── zone_size 0 directories, 31 files ODD:- 19+1 records in 19+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.001324 s, 7.6 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0000512 a a a a a a a a a a a a a a a a 0000528 0001024 a a a a a a a a a a a a a a a a 0001040 0001536 a a a a a a a a a a a a a a a a 0001552 0002048 a a a a a a a a a a a a a a a a 0002064 0002560 a a a a a a a a a a a a a a a a 0002576 0003072 a a a a a a a a a a a a a a a a 0003088 0003584 a a a a a a a a a a a a a a a a 0003600 0004096 a a a a a a a a a a a a a a a a 0004112 0004608 a a a a a a a a a a a a a a a a 0004624 ------------------------------------------------------ 0000000 a a a a a a a a a a a a a a a a 0000016 0000512 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000528 0001024 a a a a a a a a a a a a a a a a 0001040 0001536 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0001552 0002048 a a a a a a a a a a a a a a a a 0002064 0002560 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0002576 0003072 a a a a a a a a a a a a a a a a 0003088 0003584 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0003600 0004096 a a a a a a a a a a a a a a a a 0004112 0004608 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0004624 ------------------------------------------------------ EVEN:- 19+1 records in 19+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000401131 s, 24.9 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0000512 a a a a a a a a a a a a a a a a 0000528 0001024 a a a a a a a a a a a a a a a a 0001040 0001536 a a a a a a a a a a a a a a a a 0001552 0002048 a a a a a a a a a a a a a a a a 0002064 0002560 a a a a a a a a a a a a a a a a 0002576 0003072 a a a a a a a a a a a a a a a a 0003088 0003584 a a a a a a a a a a a a a a a a 0003600 0004096 a a a a a a a a a a a a a a a a 0004112 0004608 a a a a a a a a a a a a a a a a 0004624 ------------------------------------------------------ 0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000016 0000512 a a a a a a a a a a a a a a a a 0000528 0001024 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0001040 0001536 a a a a a a a a a a a a a a a a 0001552 0002048 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0002064 0002560 a a a a a a a a a a a a a a a a 0002576 0003072 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0003088 0003584 a a a a a a a a a a a a a a a a 0003600 0004096 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0004112 0004608 a a a a a a a a a a a a a a a a 0004624 ------------------------------------------------------ MKFS:- mke2fs 1.45.6 (20-Mar-2020) Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: fb802f1b-5d92-487b-9ebd-e936a5952f44 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel) [66860.364528] null_blk: disk nullb0 created [66861.496865] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem [66861.497385] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][80.0%][r=1083MiB/s][r=277k IOPS][eta 00m:04s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181274: Tue Nov 15 11:40:08 2022 read: IOPS=272k, BW=1064MiB/s (1115MB/s)(2529MiB/2378msec) slat (nsec): min=2154, max=158802, avg=2591.49, stdev=736.38 clat (nsec): min=1313, max=238742, avg=55360.06, stdev=6883.70 lat (usec): min=3, max=241, avg=58.00, stdev= 7.17 clat percentiles (usec): | 1.00th=[ 52], 5.00th=[ 53], 10.00th=[ 53], 20.00th=[ 53], | 30.00th=[ 54], 40.00th=[ 54], 50.00th=[ 55], 60.00th=[ 55], | 70.00th=[ 55], 80.00th=[ 56], 90.00th=[ 59], 95.00th=[ 62], | 99.00th=[ 104], 99.50th=[ 105], 99.90th=[ 118], 99.95th=[ 120], | 99.99th=[ 188] write: IOPS=74.8k, BW=292MiB/s (306MB/s)(4000MiB/13688msec); 0 zone resets slat (usec): min=3, max=538, avg=12.66, stdev= 9.63 clat (nsec): min=801, max=1015.1k, avg=200837.48, stdev=44069.44 lat (usec): min=7, max=1077, avg=213.56, stdev=45.83 clat percentiles (usec): | 1.00th=[ 137], 5.00th=[ 153], 10.00th=[ 161], 20.00th=[ 174], | 30.00th=[ 182], 40.00th=[ 190], 50.00th=[ 196], 60.00th=[ 204], | 70.00th=[ 210], 80.00th=[ 217], 90.00th=[ 227], 95.00th=[ 285], | 99.00th=[ 371], 99.50th=[ 388], 99.90th=[ 553], 99.95th=[ 619], | 99.99th=[ 709] bw ( KiB/s): min=123440, max=351160, per=97.76%, avg=292548.86, stdev=41626.13, samples=28 iops : min=30860, max=87790, avg=73137.21, stdev=10406.53, samples=28 lat (nsec) : 1000=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=38.28% lat (usec) : 250=58.26%, 500=3.35%, 750=0.10%, 1000=0.01% lat (msec) : 2=0.01% cpu : usr=23.45%, sys=74.46%, ctx=3377, majf=0, minf=15192 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647428,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1064MiB/s (1115MB/s), 1064MiB/s-1064MiB/s (1115MB/s-1115MB/s), io=2529MiB (2652MB), run=2378-2378msec WRITE: bw=292MiB/s (306MB/s), 292MiB/s-292MiB/s (306MB/s-306MB/s), io=4000MiB (4194MB), run=13688-13688msec Disk stats (read/write): nullb0: ios=598210/1024000, merge=0/0, ticks=674/1775, in_queue=2448, util=99.41% mke2fs 1.45.6 (20-Mar-2020) /dev/nullb0 contains a ext2 file system last mounted on /mnt/nullb0 on Tue Nov 15 11:39:52 2022 Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: 14eefd9e-aef7-4500-b5ef-f3fea6f1e126 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel) [66877.927718] EXT4-fs (nullb0): unmounting filesystem. [66877.956419] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][70.6%][r=1075MiB/s][r=275k IOPS][eta 00m:05s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181294: Tue Nov 15 11:40:21 2022 read: IOPS=275k, BW=1075MiB/s (1127MB/s)(2528MiB/2352msec) slat (nsec): min=2154, max=55936, avg=2582.05, stdev=492.16 clat (nsec): min=1112, max=117372, avg=54781.62, stdev=2453.47 lat (usec): min=3, max=119, avg=57.41, stdev= 2.52 clat percentiles (nsec): | 1.00th=[51968], 5.00th=[52480], 10.00th=[52992], 20.00th=[53504], | 30.00th=[53504], 40.00th=[54016], 50.00th=[54016], 60.00th=[54528], | 70.00th=[55040], 80.00th=[55552], 90.00th=[57600], 95.00th=[60672], | 99.00th=[63232], 99.50th=[64768], 99.90th=[69120], 99.95th=[71168], | 99.99th=[86528] write: IOPS=98.2k, BW=384MiB/s (402MB/s)(4000MiB/10423msec); 0 zone resets slat (usec): min=3, max=4111, avg= 9.54, stdev= 9.29 clat (nsec): min=892, max=4287.6k, avg=152969.66, stdev=37959.01 lat (usec): min=8, max=4294, avg=162.57, stdev=39.33 clat percentiles (usec): | 1.00th=[ 119], 5.00th=[ 128], 10.00th=[ 133], 20.00th=[ 139], | 30.00th=[ 145], 40.00th=[ 149], 50.00th=[ 153], 60.00th=[ 157], | 70.00th=[ 161], 80.00th=[ 165], 90.00th=[ 172], 95.00th=[ 180], | 99.00th=[ 204], 99.50th=[ 219], 99.90th=[ 281], 99.95th=[ 437], | 99.99th=[ 603] bw ( KiB/s): min=350488, max=433056, per=99.27%, avg=390095.24, stdev=25613.79, samples=21 iops : min=87622, max=108264, avg=97523.81, stdev=6403.43, samples=21 lat (nsec) : 1000=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=38.73% lat (usec) : 250=61.09%, 500=0.15%, 750=0.02% lat (msec) : 4=0.01%, 10=0.01% cpu : usr=27.73%, sys=72.09%, ctx=37, majf=0, minf=15189 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647282,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1075MiB/s (1127MB/s), 1075MiB/s-1075MiB/s (1127MB/s-1127MB/s), io=2528MiB (2651MB), run=2352-2352msec WRITE: bw=384MiB/s (402MB/s), 384MiB/s-384MiB/s (402MB/s-402MB/s), io=4000MiB (4194MB), run=10423-10423msec Disk stats (read/write): nullb0: ios=611304/1024241, merge=0/5921, ticks=676/1838, in_queue=2514, util=99.29% #################### BLKISZ 1024 ##################### config/nullb/nullb0 ├── badblocks ├── blocking ├── blocksize ├── cache_size ├── completion_nsec ├── discard ├── home_node ├── hw_queue_depth ├── index ├── irqmode ├── max_discard_sectors ├── max_sectors ├── max_write_zeroes_sectors ├── mbps ├── memory_backed ├── no_sched ├── poll_queues ├── power ├── queue_mode ├── shared_tag_bitmap ├── size ├── submit_queues ├── use_per_node_hctx ├── virt_boundary ├── write_zeroes ├── zone_capacity ├── zoned ├── zone_max_active ├── zone_max_open ├── zone_nr_conv └── zone_size 0 directories, 31 files ODD:- 9+1 records in 9+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000408326 s, 24.5 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0001024 a a a a a a a a a a a a a a a a 0001040 0002048 a a a a a a a a a a a a a a a a 0002064 0003072 a a a a a a a a a a a a a a a a 0003088 0004096 a a a a a a a a a a a a a a a a 0004112 0005120 a a a a a a a a a a a a a a a a 0005136 0006144 a a a a a a a a a a a a a a a a 0006160 0007168 a a a a a a a a a a a a a a a a 0007184 0008192 a a a a a a a a a a a a a a a a 0008208 0009216 a a a a a a a a a a a a a a a a 0009232 ------------------------------------------------------ 0000000 a a a a a a a a a a a a a a a a 0000016 0001024 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0001040 0002048 a a a a a a a a a a a a a a a a 0002064 0003072 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0003088 0004096 a a a a a a a a a a a a a a a a 0004112 0005120 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0005136 0006144 a a a a a a a a a a a a a a a a 0006160 0007168 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0007184 0008192 a a a a a a a a a a a a a a a a 0008208 0009216 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0009232 ------------------------------------------------------ EVEN:- 9+1 records in 9+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000351117 s, 28.5 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0001024 a a a a a a a a a a a a a a a a 0001040 0002048 a a a a a a a a a a a a a a a a 0002064 0003072 a a a a a a a a a a a a a a a a 0003088 0004096 a a a a a a a a a a a a a a a a 0004112 0005120 a a a a a a a a a a a a a a a a 0005136 0006144 a a a a a a a a a a a a a a a a 0006160 0007168 a a a a a a a a a a a a a a a a 0007184 0008192 a a a a a a a a a a a a a a a a 0008208 0009216 a a a a a a a a a a a a a a a a 0009232 ------------------------------------------------------ 0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000016 0001024 a a a a a a a a a a a a a a a a 0001040 0002048 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0002064 0003072 a a a a a a a a a a a a a a a a 0003088 0004096 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0004112 0005120 a a a a a a a a a a a a a a a a 0005136 0006144 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0006160 0007168 a a a a a a a a a a a a a a a a 0007184 0008192 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0008208 0009216 a a a a a a a a a a a a a a a a 0009232 ------------------------------------------------------ MKFS:- mke2fs 1.45.6 (20-Mar-2020) Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: 535605e4-52bf-41d8-9695-198f156dd440 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel) [66891.591141] null_blk: disk nullb0 created [66892.708280] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem [66892.708830] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][75.0%][r=1135MiB/s][r=291k IOPS][eta 00m:05s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181405: Tue Nov 15 11:40:39 2022 read: IOPS=289k, BW=1130MiB/s (1185MB/s)(2530MiB/2238msec) slat (nsec): min=2004, max=87557, avg=2412.56, stdev=493.35 clat (nsec): min=1383, max=261085, avg=52107.29, stdev=2767.75 lat (usec): min=3, max=266, avg=54.57, stdev= 2.85 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 51], | 30.00th=[ 51], 40.00th=[ 51], 50.00th=[ 52], 60.00th=[ 52], | 70.00th=[ 52], 80.00th=[ 53], 90.00th=[ 56], 95.00th=[ 59], | 99.00th=[ 62], 99.50th=[ 65], 99.90th=[ 70], 99.95th=[ 76], | 99.99th=[ 110] write: IOPS=77.1k, BW=301MiB/s (316MB/s)(4000MiB/13285msec); 0 zone resets slat (usec): min=3, max=439, avg=12.25, stdev= 9.62 clat (nsec): min=691, max=966015, avg=194939.47, stdev=42572.14 lat (usec): min=7, max=1328, avg=207.26, stdev=44.18 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 151], 10.00th=[ 159], 20.00th=[ 169], | 30.00th=[ 178], 40.00th=[ 184], 50.00th=[ 190], 60.00th=[ 196], | 70.00th=[ 202], 80.00th=[ 208], 90.00th=[ 219], 95.00th=[ 285], | 99.00th=[ 363], 99.50th=[ 375], 99.90th=[ 545], 99.95th=[ 594], | 99.99th=[ 693] bw ( KiB/s): min=188448, max=339888, per=98.41%, avg=303407.41, stdev=31231.18, samples=27 iops : min=47112, max=84972, avg=75851.85, stdev=7807.79, samples=27 lat (nsec) : 750=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=1.12%, 100=37.62% lat (usec) : 250=58.02%, 500=3.14%, 750=0.10%, 1000=0.01% cpu : usr=23.84%, sys=73.97%, ctx=3272, majf=0, minf=15197 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647639,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1130MiB/s (1185MB/s), 1130MiB/s-1130MiB/s (1185MB/s-1185MB/s), io=2530MiB (2653MB), run=2238-2238msec WRITE: bw=301MiB/s (316MB/s), 301MiB/s-301MiB/s (316MB/s-316MB/s), io=4000MiB (4194MB), run=13285-13285msec Disk stats (read/write): nullb0: ios=607879/1024000, merge=0/0, ticks=609/1640, in_queue=2250, util=99.37% mke2fs 1.45.6 (20-Mar-2020) /dev/nullb0 contains a ext2 file system last mounted on /mnt/nullb0 on Tue Nov 15 11:40:23 2022 Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: c1a58ae3-4c3c-4707-9413-249ec935e0a0 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel) [66908.577976] EXT4-fs (nullb0): unmounting filesystem. [66908.604562] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=0): [f(1)][100.0%][r=1071MiB/s][r=274k IOPS][eta 00m:00s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181431: Tue Nov 15 11:40:52 2022 read: IOPS=283k, BW=1104MiB/s (1158MB/s)(2526MiB/2288msec) slat (nsec): min=2064, max=55737, avg=2477.85, stdev=522.77 clat (nsec): min=1113, max=115029, avg=53321.61, stdev=2517.96 lat (usec): min=3, max=117, avg=55.84, stdev= 2.58 clat percentiles (usec): | 1.00th=[ 51], 5.00th=[ 52], 10.00th=[ 52], 20.00th=[ 52], | 30.00th=[ 52], 40.00th=[ 53], 50.00th=[ 53], 60.00th=[ 53], | 70.00th=[ 53], 80.00th=[ 54], 90.00th=[ 58], 95.00th=[ 60], | 99.00th=[ 62], 99.50th=[ 63], 99.90th=[ 71], 99.95th=[ 75], | 99.99th=[ 101] write: IOPS=96.9k, BW=378MiB/s (397MB/s)(4000MiB/10570msec); 0 zone resets slat (usec): min=3, max=4143, avg= 9.69, stdev= 9.14 clat (nsec): min=902, max=8152.9k, avg=155134.15, stdev=42347.61 lat (usec): min=6, max=8165, avg=164.88, stdev=43.97 clat percentiles (usec): | 1.00th=[ 119], 5.00th=[ 128], 10.00th=[ 135], 20.00th=[ 141], | 30.00th=[ 145], 40.00th=[ 151], 50.00th=[ 155], 60.00th=[ 157], | 70.00th=[ 163], 80.00th=[ 167], 90.00th=[ 176], 95.00th=[ 184], | 99.00th=[ 215], 99.50th=[ 243], 99.90th=[ 330], 99.95th=[ 408], | 99.99th=[ 482] bw ( KiB/s): min=51824, max=440064, per=96.09%, avg=372363.64, stdev=75777.57, samples=22 iops : min=12954, max=110018, avg=93090.91, stdev=18944.88, samples=22 lat (nsec) : 1000=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=38.70% lat (usec) : 250=61.04%, 500=0.25%, 750=0.01% lat (msec) : 4=0.01%, 10=0.01% cpu : usr=27.04%, sys=72.74%, ctx=327, majf=0, minf=15177 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=646761,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1104MiB/s (1158MB/s), 1104MiB/s-1104MiB/s (1158MB/s-1158MB/s), io=2526MiB (2649MB), run=2288-2288msec WRITE: bw=378MiB/s (397MB/s), 378MiB/s-378MiB/s (397MB/s-397MB/s), io=4000MiB (4194MB), run=10570-10570msec Disk stats (read/write): nullb0: ios=585939/1024234, merge=0/5702, ticks=587/1738, in_queue=2325, util=99.31% #################### BLKISZ 2048 ##################### config/nullb/nullb0 ├── badblocks ├── blocking ├── blocksize ├── cache_size ├── completion_nsec ├── discard ├── home_node ├── hw_queue_depth ├── index ├── irqmode ├── max_discard_sectors ├── max_sectors ├── max_write_zeroes_sectors ├── mbps ├── memory_backed ├── no_sched ├── poll_queues ├── power ├── queue_mode ├── shared_tag_bitmap ├── size ├── submit_queues ├── use_per_node_hctx ├── virt_boundary ├── write_zeroes ├── zone_capacity ├── zoned ├── zone_max_active ├── zone_max_open ├── zone_nr_conv └── zone_size 0 directories, 31 files ODD:- 4+1 records in 4+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000362418 s, 27.6 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0002048 a a a a a a a a a a a a a a a a 0002064 0004096 a a a a a a a a a a a a a a a a 0004112 0006144 a a a a a a a a a a a a a a a a 0006160 0008192 a a a a a a a a a a a a a a a a 0008208 0010240 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0010256 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0014336 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0014352 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0018432 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0018448 ------------------------------------------------------ 0000000 a a a a a a a a a a a a a a a a 0000016 0002048 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0002064 0004096 a a a a a a a a a a a a a a a a 0004112 0006144 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0006160 0008192 a a a a a a a a a a a a a a a a 0008208 0010240 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0010256 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0014336 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0014352 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0018432 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0018448 ------------------------------------------------------ EVEN:- 4+1 records in 4+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000310148 s, 32.2 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0002048 a a a a a a a a a a a a a a a a 0002064 0004096 a a a a a a a a a a a a a a a a 0004112 0006144 a a a a a a a a a a a a a a a a 0006160 0008192 a a a a a a a a a a a a a a a a 0008208 0010240 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0010256 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0014336 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0014352 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0018432 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0018448 ------------------------------------------------------ 0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000016 0002048 a a a a a a a a a a a a a a a a 0002064 0004096 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0004112 0006144 a a a a a a a a a a a a a a a a 0006160 0008192 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0008208 0010240 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0010256 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0014336 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0014352 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0018432 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0018448 ------------------------------------------------------ MKFS:- mke2fs 1.45.6 (20-Mar-2020) Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: 52d082f5-badf-4ed4-8a01-32e24ae621b1 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel) [66922.293478] null_blk: disk nullb0 created [66923.411765] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem [66923.412345] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][71.4%][r=1143MiB/s][r=293k IOPS][eta 00m:06s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181524: Tue Nov 15 11:41:10 2022 read: IOPS=294k, BW=1150MiB/s (1206MB/s)(2531MiB/2200msec) slat (nsec): min=1914, max=94730, avg=2331.47, stdev=563.62 clat (nsec): min=1102, max=167378, avg=51209.99, stdev=3492.57 lat (usec): min=3, max=172, avg=53.59, stdev= 3.61 clat percentiles (usec): | 1.00th=[ 49], 5.00th=[ 49], 10.00th=[ 50], 20.00th=[ 50], | 30.00th=[ 50], 40.00th=[ 50], 50.00th=[ 51], 60.00th=[ 51], | 70.00th=[ 51], 80.00th=[ 52], 90.00th=[ 56], 95.00th=[ 58], | 99.00th=[ 65], 99.50th=[ 72], 99.90th=[ 88], 99.95th=[ 95], | 99.99th=[ 108] write: IOPS=75.3k, BW=294MiB/s (309MB/s)(4000MiB/13590msec); 0 zone resets slat (usec): min=3, max=524, avg=12.54, stdev= 9.34 clat (nsec): min=821, max=1017.9k, avg=199408.01, stdev=44731.27 lat (usec): min=6, max=1250, avg=212.02, stdev=46.63 clat percentiles (usec): | 1.00th=[ 135], 5.00th=[ 149], 10.00th=[ 159], 20.00th=[ 172], | 30.00th=[ 180], 40.00th=[ 188], 50.00th=[ 196], 60.00th=[ 202], | 70.00th=[ 208], 80.00th=[ 217], 90.00th=[ 231], 95.00th=[ 281], | 99.00th=[ 375], 99.50th=[ 408], 99.90th=[ 553], 99.95th=[ 619], | 99.99th=[ 725] bw ( KiB/s): min=58240, max=361120, per=97.07%, avg=292571.14, stdev=54664.73, samples=28 iops : min=14560, max=90280, avg=73142.86, stdev=13666.23, samples=28 lat (nsec) : 1000=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=14.75%, 100=24.00% lat (usec) : 250=57.42%, 500=3.73%, 750=0.11%, 1000=0.01% lat (msec) : 2=0.01% cpu : usr=23.75%, sys=74.27%, ctx=3340, majf=0, minf=15201 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647840,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1150MiB/s (1206MB/s), 1150MiB/s-1150MiB/s (1206MB/s-1206MB/s), io=2531MiB (2654MB), run=2200-2200msec WRITE: bw=294MiB/s (309MB/s), 294MiB/s-294MiB/s (309MB/s-309MB/s), io=4000MiB (4194MB), run=13590-13590msec Disk stats (read/write): nullb0: ios=601631/1024000, merge=0/0, ticks=544/1629, in_queue=2173, util=99.38% mke2fs 1.45.6 (20-Mar-2020) /dev/nullb0 contains a ext2 file system last mounted on /mnt/nullb0 on Tue Nov 15 11:40:53 2022 Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: 2bea2344-e903-42fb-bea3-5812319229ac Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel) [66939.549992] EXT4-fs (nullb0): unmounting filesystem. [66939.572566] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][75.0%][r=1138MiB/s][r=291k IOPS][eta 00m:04s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181545: Tue Nov 15 11:41:22 2022 read: IOPS=291k, BW=1137MiB/s (1192MB/s)(2528MiB/2223msec) slat (nsec): min=1924, max=102094, avg=2374.71, stdev=533.20 clat (nsec): min=1503, max=433523, avg=51807.71, stdev=3883.92 lat (usec): min=3, max=435, avg=54.23, stdev= 3.99 clat percentiles (usec): | 1.00th=[ 50], 5.00th=[ 50], 10.00th=[ 50], 20.00th=[ 51], | 30.00th=[ 51], 40.00th=[ 51], 50.00th=[ 51], 60.00th=[ 52], | 70.00th=[ 52], 80.00th=[ 52], 90.00th=[ 55], 95.00th=[ 58], | 99.00th=[ 62], 99.50th=[ 68], 99.90th=[ 96], 99.95th=[ 104], | 99.99th=[ 131] write: IOPS=99.0k, BW=387MiB/s (406MB/s)(4000MiB/10340msec); 0 zone resets slat (usec): min=3, max=3807, avg= 9.47, stdev= 9.02 clat (nsec): min=882, max=6505.2k, avg=151752.51, stdev=38984.09 lat (usec): min=12, max=6587, avg=161.28, stdev=40.70 clat percentiles (usec): | 1.00th=[ 117], 5.00th=[ 126], 10.00th=[ 133], 20.00th=[ 139], | 30.00th=[ 143], 40.00th=[ 147], 50.00th=[ 151], 60.00th=[ 155], | 70.00th=[ 159], 80.00th=[ 163], 90.00th=[ 172], 95.00th=[ 178], | 99.00th=[ 202], 99.50th=[ 215], 99.90th=[ 277], 99.95th=[ 334], | 99.99th=[ 416] bw ( KiB/s): min=295752, max=434984, per=98.48%, avg=390095.24, stdev=30689.19, samples=21 iops : min=73938, max=108746, avg=97523.81, stdev=7672.30, samples=21 lat (nsec) : 1000=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=4.43%, 100=34.28% lat (usec) : 250=61.16%, 500=0.13% lat (msec) : 4=0.01%, 10=0.01% cpu : usr=27.09%, sys=72.66%, ctx=449, majf=0, minf=15187 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647098,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1137MiB/s (1192MB/s), 1137MiB/s-1137MiB/s (1192MB/s-1192MB/s), io=2528MiB (2651MB), run=2223-2223msec WRITE: bw=387MiB/s (406MB/s), 387MiB/s-387MiB/s (406MB/s-406MB/s), io=4000MiB (4194MB), run=10340-10340msec Disk stats (read/write): nullb0: ios=598120/1024237, merge=0/5791, ticks=540/1647, in_queue=2186, util=99.27% #################### BLKISZ 4096 ##################### config/nullb/nullb0 ├── badblocks ├── blocking ├── blocksize ├── cache_size ├── completion_nsec ├── discard ├── home_node ├── hw_queue_depth ├── index ├── irqmode ├── max_discard_sectors ├── max_sectors ├── max_write_zeroes_sectors ├── mbps ├── memory_backed ├── no_sched ├── poll_queues ├── power ├── queue_mode ├── shared_tag_bitmap ├── size ├── submit_queues ├── use_per_node_hctx ├── virt_boundary ├── write_zeroes ├── zone_capacity ├── zoned ├── zone_max_active ├── zone_max_open ├── zone_nr_conv └── zone_size 0 directories, 31 files ODD:- 2+1 records in 2+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000394609 s, 25.3 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0004096 a a a a a a a a a a a a a a a a 0004112 0008192 a a a a a a a a a a a a a a a a 0008208 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0020480 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0020496 0024576 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0024592 0028672 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0028688 0032768 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0032784 0036864 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0036880 ------------------------------------------------------ 0000000 a a a a a a a a a a a a a a a a 0000016 0004096 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0004112 0008192 a a a a a a a a a a a a a a a a 0008208 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0020480 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0020496 0024576 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0024592 0028672 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0028688 0032768 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0032784 0036864 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0036880 ------------------------------------------------------ EVEN:- 2+1 records in 2+1 records out 10002 bytes (10 kB, 9.8 KiB) copied, 0.000231178 s, 43.3 MB/s 0000000 a a a a a a a a a a a a a a a a 0000016 0004096 a a a a a a a a a a a a a a a a 0004112 0008192 a a a a a a a a a a a a a a a a 0008208 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0020480 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0020496 0024576 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0024592 0028672 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0028688 0032768 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0032784 0036864 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0036880 ------------------------------------------------------ 0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000016 0004096 a a a a a a a a a a a a a a a a 0004112 0008192 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0008208 0012288 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0012304 0016384 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0016400 0020480 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0020496 0024576 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0024592 0028672 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0028688 0032768 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0032784 0036864 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0036880 ------------------------------------------------------ MKFS:- mke2fs 1.45.6 (20-Mar-2020) Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: 472150a1-06d4-43bb-9ec8-f9855439ef51 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel) [66952.985308] null_blk: disk nullb0 created [66954.103807] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem [66954.104413] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][78.9%][r=1164MiB/s][r=298k IOPS][eta 00m:04s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181633: Tue Nov 15 11:41:40 2022 read: IOPS=297k, BW=1161MiB/s (1217MB/s)(2528MiB/2178msec) slat (nsec): min=1963, max=68339, avg=2318.80, stdev=478.10 clat (nsec): min=1392, max=154052, avg=50753.94, stdev=2410.44 lat (usec): min=4, max=156, avg=53.12, stdev= 2.47 clat percentiles (nsec): | 1.00th=[48384], 5.00th=[48896], 10.00th=[48896], 20.00th=[49408], | 30.00th=[49408], 40.00th=[49920], 50.00th=[49920], 60.00th=[50432], | 70.00th=[50432], 80.00th=[50944], 90.00th=[54528], 95.00th=[56576], | 99.00th=[58624], 99.50th=[60160], 99.90th=[64768], 99.95th=[67072], | 99.99th=[84480] write: IOPS=78.4k, BW=306MiB/s (321MB/s)(4000MiB/13062msec); 0 zone resets slat (usec): min=3, max=459, avg=12.06, stdev= 9.33 clat (nsec): min=781, max=878889, avg=191647.26, stdev=41358.52 lat (usec): min=6, max=1256, avg=203.78, stdev=42.97 clat percentiles (usec): | 1.00th=[ 133], 5.00th=[ 147], 10.00th=[ 155], 20.00th=[ 165], | 30.00th=[ 174], 40.00th=[ 182], 50.00th=[ 188], 60.00th=[ 194], | 70.00th=[ 200], 80.00th=[ 208], 90.00th=[ 217], 95.00th=[ 239], | 99.00th=[ 355], 99.50th=[ 371], 99.90th=[ 529], 99.95th=[ 586], | 99.99th=[ 685] bw ( KiB/s): min=35760, max=361560, per=96.75%, avg=303382.70, stdev=59886.47, samples=27 iops : min= 8938, max=90392, avg=75845.67, stdev=14972.03, samples=27 lat (nsec) : 1000=0.01% lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=18.50%, 100=20.23% lat (usec) : 250=58.30%, 500=2.90%, 750=0.08%, 1000=0.01% cpu : usr=23.28%, sys=74.59%, ctx=3190, majf=0, minf=15184 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647124,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1161MiB/s (1217MB/s), 1161MiB/s-1161MiB/s (1217MB/s-1217MB/s), io=2528MiB (2651MB), run=2178-2178msec WRITE: bw=306MiB/s (321MB/s), 306MiB/s-306MiB/s (321MB/s-321MB/s), io=4000MiB (4194MB), run=13062-13062msec Disk stats (read/write): nullb0: ios=616272/1024000, merge=0/0, ticks=547/1593, in_queue=2141, util=99.37% mke2fs 1.45.6 (20-Mar-2020) /dev/nullb0 contains a ext2 file system last mounted on /mnt/nullb0 on Tue Nov 15 11:41:24 2022 Creating filesystem with 1310720 4k blocks and 327680 inodes Filesystem UUID: f17569a8-1b75-430a-97bb-4b4b8f23406a Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done /dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel) [66969.684331] EXT4-fs (nullb0): unmounting filesystem. [66969.706596] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none. write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.27 Starting 1 process write-and-verify: Laying out IO file (1 file / 4000MiB) Jobs: 1 (f=1): [V(1)][75.0%][r=1143MiB/s][r=293k IOPS][eta 00m:04s] write-and-verify: (groupid=0, jobs=1): err= 0: pid=181656: Tue Nov 15 11:41:53 2022 read: IOPS=292k, BW=1141MiB/s (1196MB/s)(2530MiB/2218msec) slat (nsec): min=1943, max=59723, avg=2357.85, stdev=610.78 clat (nsec): min=1122, max=126250, avg=51644.27, stdev=3519.72 lat (usec): min=3, max=128, avg=54.05, stdev= 3.63 clat percentiles (usec): | 1.00th=[ 49], 5.00th=[ 49], 10.00th=[ 50], 20.00th=[ 50], | 30.00th=[ 50], 40.00th=[ 51], 50.00th=[ 51], 60.00th=[ 51], | 70.00th=[ 52], 80.00th=[ 52], 90.00th=[ 57], 95.00th=[ 59], | 99.00th=[ 65], 99.50th=[ 69], 99.90th=[ 77], 99.95th=[ 81], | 99.99th=[ 109] write: IOPS=98.4k, BW=385MiB/s (403MB/s)(4000MiB/10403msec); 0 zone resets slat (usec): min=3, max=3347, avg= 9.53, stdev= 8.05 clat (nsec): min=1182, max=3537.4k, avg=152680.35, stdev=33311.37 lat (usec): min=13, max=3548, avg=162.27, stdev=34.53 clat percentiles (usec): | 1.00th=[ 119], 5.00th=[ 128], 10.00th=[ 133], 20.00th=[ 139], | 30.00th=[ 145], 40.00th=[ 149], 50.00th=[ 153], 60.00th=[ 155], | 70.00th=[ 159], 80.00th=[ 165], 90.00th=[ 172], 95.00th=[ 180], | 99.00th=[ 204], 99.50th=[ 219], 99.90th=[ 285], 99.95th=[ 355], | 99.99th=[ 474] bw ( KiB/s): min=349128, max=431920, per=99.08%, avg=390094.48, stdev=22615.43, samples=21 iops : min=87282, max=107980, avg=97523.81, stdev=5653.77, samples=21 lat (usec) : 2=0.01%, 10=0.01%, 20=0.01%, 50=11.55%, 100=27.19% lat (usec) : 250=61.10%, 500=0.15%, 750=0.01% lat (msec) : 4=0.01% cpu : usr=28.99%, sys=70.86%, ctx=27, majf=0, minf=15197 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=647655,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=1141MiB/s (1196MB/s), 1141MiB/s-1141MiB/s (1196MB/s-1196MB/s), io=2530MiB (2653MB), run=2218-2218msec WRITE: bw=385MiB/s (403MB/s), 385MiB/s-385MiB/s (403MB/s-403MB/s), io=4000MiB (4194MB), run=10403-10403msec Disk stats (read/write): nullb0: ios=580338/1024235, merge=0/5744, ticks=513/1640, in_queue=2154, util=99.29%

Message ID	20221115195253.182484-1-kch@nvidia.com (mailing list archive)
Headers	show Return-Path: <linux-block-owner@kernel.org> Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Chaitanya Kulkarni <kch@nvidia.com> To: <linux-block@vger.kernel.org>, <linux-kernel@vger.kernel.org> CC: <axboe@kernel.dk>, <kch@nvidia.com>, <damien.lemoal@opensource.wdc.com>, <johannes.thumshirn@wdc.com>, <bvanassche@acm.org>, <ming.lei@redhat.com>, <shinichiro.kawasaki@wdc.com>, <vincent.fu@samsung.com>, <yukuai3@huawei.com> Subject: [PATCH V3 0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup Date: Tue, 15 Nov 2022 11:52:46 -0800 Message-ID: <20221115195253.182484-1-kch@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="a" Content-Transfer-Encoding: 8bit Precedence: bulk
Series	null_blk: allow REQ_OP_WRITE_ZEROES and cleanup \| expand [V3,0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup [V3,1/7] null_blk: allow REQ_OP_WRITE_ZEROES [V3,2/7] null_blk: code cleaup [V3,3/7] null_blk: initialize cmd->bio in __alloc_cmd() [V3,4/7] null_blk: don't use magic numbers in the code [V3,5/7] null_blk: remove extra space in switch condition [V3,6/7] null_blk: add param to set max disacrd sectors [V3,7/7] null_blk: add param to set max write-zeroes sects

[V3,0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup

Message