| Message ID | 20241022111059.2566137-1-yi.zhang@huaweicloud.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | ext4: use iomap for regular file's buffered I/O path and enable large folio | expand |
On Tue, Oct 22, 2024 at 5:13 AM Zhang Yi <yi.zhang@huaweicloud.com> wrote: > > From: Zhang Yi <yi.zhang@huawei.com> > > Hello！ > > This patch series is the latest version based on my previous RFC > series[1], which converts the buffered I/O path of ext4 regular files to > iomap and enables large folios. After several months of work, almost all > preparatory changes have been upstreamed, thanks a lot for the review > and comments from Jan, Dave, Christoph, Darrick and Ritesh. Now it is > time for the main implementation of this conversion. > > This series is the main part of iomap buffered iomap conversion, it's > based on 6.12-rc4, and the code context is also depend on my anohter > cleanup series[1] (I've put that in this seris so we can merge it > directly), fixed all minor bugs found in my previous RFC v4 series. > Additionally, I've update change logs in each patch and also includes > some code modifications as Dave's suggestions. This series implements > the core iomap APIs on ext4 and introduces a mount option called > "buffered_iomap" to enable the iomap buffered I/O path. We have already > supported the default features, default mount options and bigalloc > feature. However, we do not yet support online defragmentation, inline > data, fs_verify, fs_crypt, ext3, and data=journal mode, ext4 will fall > to buffered_head I/O path automatically if you use those features and > options. Some of these features should be supported gradually in the > near future. > > Most of the implementations resemble the original buffered_head path; > however, there are four key differences. > > 1. The first aspect is the block allocation in the writeback path. The > iomap frame will invoke ->map_blocks() at least once for each dirty > folio. To ensure optimal writeback performance, we aim to allocate a > range of delalloc blocks that is as long as possible within the > writeback length for each invocation. In certain situations, we may > allocate a range of blocks that exceeds the amount we will actually > write back. Therefore, > 1) we cannot allocate a written extent for those blocks because it may > expose stale data in such short write cases. Instead, we should > allocate an unwritten extent, which means we must always enable the > dioread_nolock option. This change could also bring many other > benefits. > 2) We should postpone updating the 'i_disksize' until the end of the I/O > process, based on the actual written length. This approach can also > prevent the exposure of zero data, which may occur if there is a > power failure during an append write. > 3) We do not need to pre-split extents during write-back, we can > postpone this task until the end I/O process while converting > unwritten extents. > > 2. The second reason is that since we always allocate unwritten space > for new blocks, there is no risk of exposing stale data. As a result, > we do not need to order the data, which allows us to disable the > data=ordered mode. Consequently, we also do not require the reserved > handle when converting the unwritten extent in the final I/O worker, > we can directly start with the normal handle. > > Series details: > > Patch 1-10 is just another series of mine that refactors the fallocate > functions[1]. This series relies on the code context of that but has no > logical dependencies. I put this here just for easy access and merge. > > Patch 11-21 implement the iomap buffered read/write path, dirty folio > write back path and mmap path for ext4 regular file. > > Patch 22-23 disable the unsupported online-defragmentation function and > disable the changing of the inode journal flag to data=journal mode. > Please look at the following patch for details. > > Patch 24-27 introduce "buffered_iomap" mount option (is not enabled by > default now) to partially enable the iomap buffered I/O path and also > enable large folio. > > > About performance: > > Fio tests with psync on my machine with Intel Xeon Gold 6240 CPU with > 400GB system ram, 200GB ramdisk and 4TB nvme ssd disk. > > fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ > -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ > -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ > -group_reportin -name=$name --output=/tmp/test_log > Hi Zhang Yi, can you clarify about the FIO values for the diverse parameters? Thanks. BR, -Sedat- > == buffer read == > > buffer_head iomap + large folio > type bs IOPS BW(MiB/s) IOPS BW(MiB/s) > ------------------------------------------------------- > hole 4K 576k 2253 762k 2975 +32% > hole 64K 48.7k 3043 77.8k 4860 +60% > hole 1M 2960 2960 4942 4942 +67% > ramdisk 4K 443k 1732 530k 2069 +19% > ramdisk 64K 34.5k 2156 45.6k 2850 +32% > ramdisk 1M 2093 2093 2841 2841 +36% > nvme 4K 339k 1323 364k 1425 +8% > nvme 64K 23.6k 1471 25.2k 1574 +7% > nvme 1M 2012 2012 2153 2153 +7% > > > == buffer write == > > buffer_head iomap + large folio > type Overwrite Sync Writeback bs IOPS BW IOPS BW(MiB/s) > ---------------------------------------------------------------------- > cache N N N 4K 417k 1631 440k 1719 +5% > cache N N N 64K 33.4k 2088 81.5k 5092 +144% > cache N N N 1M 2143 2143 5716 5716 +167% > cache Y N N 4K 449k 1755 469k 1834 +5% > cache Y N N 64K 36.6k 2290 82.3k 5142 +125% > cache Y N N 1M 2352 2352 5577 5577 +137% > ramdisk N N Y 4K 365k 1424 354k 1384 -3% > ramdisk N N Y 64K 31.2k 1950 74.2k 4640 +138% > ramdisk N N Y 1M 1968 1968 5201 5201 +164% > ramdisk N Y N 4K 9984 39 12.9k 51 +29% > ramdisk N Y N 64K 5936 371 8960 560 +51% > ramdisk N Y N 1M 1050 1050 1835 1835 +75% > ramdisk Y N Y 4K 411k 1609 443k 1731 +8% > ramdisk Y N Y 64K 34.1k 2134 77.5k 4844 +127% > ramdisk Y N Y 1M 2248 2248 5372 5372 +139% > ramdisk Y Y N 4K 182k 711 186k 730 +3% > ramdisk Y Y N 64K 18.7k 1170 34.7k 2171 +86% > ramdisk Y Y N 1M 1229 1229 2269 2269 +85% > nvme N N Y 4K 373k 1458 387k 1512 +4% > nvme N N Y 64K 29.2k 1827 70.9k 4431 +143% > nvme N N Y 1M 1835 1835 4919 4919 +168% > nvme N Y N 4K 11.7k 46 11.7k 46 0% > nvme N Y N 64K 6453 403 8661 541 +34% > nvme N Y N 1M 649 649 1351 1351 +108% > nvme Y N Y 4K 372k 1456 433k 1693 +16% > nvme Y N Y 64K 33.0k 2064 74.7k 4669 +126% > nvme Y N Y 1M 2131 2131 5273 5273 +147% > nvme Y Y N 4K 56.7k 222 56.4k 220 -1% > nvme Y Y N 64K 13.4k 840 19.4k 1214 +45% > nvme Y Y N 1M 714 714 1504 1504 +111% > > Thanks, > Yi. > > Major changes since RFC v4: > - Disable unsupported online defragmentation, do not fall back to > buffer_head path. > - Wite and wait data back while doing partial block truncate down to > fix a stale data problem. > - Disable the online changing of the inode journal flag to data=journal > mode. > - Since iomap can zero out dirty pages with unwritten extent, do not > write data before zeroing out in ext4_zero_range(), and also do not > zero partial blocks under a started journal handle. > > [1] https://lore.kernel.org/linux-ext4/20241010133333.146793-1-yi.zhang@huawei.com/ > > --- > RFC v4: https://lore.kernel.org/linux-ext4/20240410142948.2817554-1-yi.zhang@huaweicloud.com/ > RFC v3: https://lore.kernel.org/linux-ext4/20240127015825.1608160-1-yi.zhang@huaweicloud.com/ > RFC v2: https://lore.kernel.org/linux-ext4/20240102123918.799062-1-yi.zhang@huaweicloud.com/ > RFC v1: https://lore.kernel.org/linux-ext4/20231123125121.4064694-1-yi.zhang@huaweicloud.com/ > > > Zhang Yi (27): > ext4: remove writable userspace mappings before truncating page cache > ext4: don't explicit update times in ext4_fallocate() > ext4: don't write back data before punch hole in nojournal mode > ext4: refactor ext4_punch_hole() > ext4: refactor ext4_zero_range() > ext4: refactor ext4_collapse_range() > ext4: refactor ext4_insert_range() > ext4: factor out ext4_do_fallocate() > ext4: move out inode_lock into ext4_fallocate() > ext4: move out common parts into ext4_fallocate() > ext4: use reserved metadata blocks when splitting extent on endio > ext4: introduce seq counter for the extent status entry > ext4: add a new iomap aops for regular file's buffered IO path > ext4: implement buffered read iomap path > ext4: implement buffered write iomap path > ext4: don't order data for inode with EXT4_STATE_BUFFERED_IOMAP > ext4: implement writeback iomap path > ext4: implement mmap iomap path > ext4: do not always order data when partial zeroing out a block > ext4: do not start handle if unnecessary while partial zeroing out a > block > ext4: implement zero_range iomap path > ext4: disable online defrag when inode using iomap buffered I/O path > ext4: disable inode journal mode when using iomap buffered I/O path > ext4: partially enable iomap for the buffered I/O path of regular > files > ext4: enable large folio for regular file with iomap buffered I/O path > ext4: change mount options code style > ext4: introduce a mount option for iomap buffered I/O path > > fs/ext4/ext4.h | 17 +- > fs/ext4/ext4_jbd2.c | 3 +- > fs/ext4/ext4_jbd2.h | 8 + > fs/ext4/extents.c | 568 +++++++++++---------------- > fs/ext4/extents_status.c | 13 +- > fs/ext4/file.c | 19 +- > fs/ext4/ialloc.c | 5 + > fs/ext4/inode.c | 755 ++++++++++++++++++++++++++++++------ > fs/ext4/move_extent.c | 7 + > fs/ext4/page-io.c | 105 +++++ > fs/ext4/super.c | 185 ++++----- > include/trace/events/ext4.h | 57 +-- > 12 files changed, 1153 insertions(+), 589 deletions(-) > > -- > 2.46.1 > >
On 2024/10/22 14:59, Sedat Dilek wrote: > On Tue, Oct 22, 2024 at 5:13 AM Zhang Yi <yi.zhang@huaweicloud.com> wrote: >> >> From: Zhang Yi <yi.zhang@huawei.com> >> >> Hello！ >> >> This patch series is the latest version based on my previous RFC >> series[1], which converts the buffered I/O path of ext4 regular files to >> iomap and enables large folios. After several months of work, almost all >> preparatory changes have been upstreamed, thanks a lot for the review >> and comments from Jan, Dave, Christoph, Darrick and Ritesh. Now it is >> time for the main implementation of this conversion. >> >> This series is the main part of iomap buffered iomap conversion, it's >> based on 6.12-rc4, and the code context is also depend on my anohter >> cleanup series[1] (I've put that in this seris so we can merge it >> directly), fixed all minor bugs found in my previous RFC v4 series. >> Additionally, I've update change logs in each patch and also includes >> some code modifications as Dave's suggestions. This series implements >> the core iomap APIs on ext4 and introduces a mount option called >> "buffered_iomap" to enable the iomap buffered I/O path. We have already >> supported the default features, default mount options and bigalloc >> feature. However, we do not yet support online defragmentation, inline >> data, fs_verify, fs_crypt, ext3, and data=journal mode, ext4 will fall >> to buffered_head I/O path automatically if you use those features and >> options. Some of these features should be supported gradually in the >> near future. >> >> Most of the implementations resemble the original buffered_head path; >> however, there are four key differences. >> >> 1. The first aspect is the block allocation in the writeback path. The >> iomap frame will invoke ->map_blocks() at least once for each dirty >> folio. To ensure optimal writeback performance, we aim to allocate a >> range of delalloc blocks that is as long as possible within the >> writeback length for each invocation. In certain situations, we may >> allocate a range of blocks that exceeds the amount we will actually >> write back. Therefore, >> 1) we cannot allocate a written extent for those blocks because it may >> expose stale data in such short write cases. Instead, we should >> allocate an unwritten extent, which means we must always enable the >> dioread_nolock option. This change could also bring many other >> benefits. >> 2) We should postpone updating the 'i_disksize' until the end of the I/O >> process, based on the actual written length. This approach can also >> prevent the exposure of zero data, which may occur if there is a >> power failure during an append write. >> 3) We do not need to pre-split extents during write-back, we can >> postpone this task until the end I/O process while converting >> unwritten extents. >> >> 2. The second reason is that since we always allocate unwritten space >> for new blocks, there is no risk of exposing stale data. As a result, >> we do not need to order the data, which allows us to disable the >> data=ordered mode. Consequently, we also do not require the reserved >> handle when converting the unwritten extent in the final I/O worker, >> we can directly start with the normal handle. >> >> Series details: >> >> Patch 1-10 is just another series of mine that refactors the fallocate >> functions[1]. This series relies on the code context of that but has no >> logical dependencies. I put this here just for easy access and merge. >> >> Patch 11-21 implement the iomap buffered read/write path, dirty folio >> write back path and mmap path for ext4 regular file. >> >> Patch 22-23 disable the unsupported online-defragmentation function and >> disable the changing of the inode journal flag to data=journal mode. >> Please look at the following patch for details. >> >> Patch 24-27 introduce "buffered_iomap" mount option (is not enabled by >> default now) to partially enable the iomap buffered I/O path and also >> enable large folio. >> >> >> About performance: >> >> Fio tests with psync on my machine with Intel Xeon Gold 6240 CPU with >> 400GB system ram, 200GB ramdisk and 4TB nvme ssd disk. >> >> fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ >> -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ >> -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ >> -group_reportin -name=$name --output=/tmp/test_log >> > > Hi Zhang Yi, > > can you clarify about the FIO values for the diverse parameters? > Hi Sedat, Sure, the test I present here is a simple single-thread and single-I/O depth case with psync ioengine. Most of the FIO parameters are shown in the tables below. For the rest, the 'iodepth' and 'numjobs' are always set to 1 and the 'size' is 40GB. During the write cache test, I also disable the write back process through: echo 0 > /proc/sys/vm/dirty_writeback_centisecs echo 100 > /proc/sys/vm/dirty_background_ratio echo 100 > /proc/sys/vm/dirty_ratio Thanks, Yi. > >> == buffer read == >> >> buffer_head iomap + large folio >> type bs IOPS BW(MiB/s) IOPS BW(MiB/s) >> ------------------------------------------------------- >> hole 4K 576k 2253 762k 2975 +32% >> hole 64K 48.7k 3043 77.8k 4860 +60% >> hole 1M 2960 2960 4942 4942 +67% >> ramdisk 4K 443k 1732 530k 2069 +19% >> ramdisk 64K 34.5k 2156 45.6k 2850 +32% >> ramdisk 1M 2093 2093 2841 2841 +36% >> nvme 4K 339k 1323 364k 1425 +8% >> nvme 64K 23.6k 1471 25.2k 1574 +7% >> nvme 1M 2012 2012 2153 2153 +7% >> >> >> == buffer write == >> >> buffer_head iomap + large folio >> type Overwrite Sync Writeback bs IOPS BW IOPS BW(MiB/s) >> ---------------------------------------------------------------------- >> cache N N N 4K 417k 1631 440k 1719 +5% >> cache N N N 64K 33.4k 2088 81.5k 5092 +144% >> cache N N N 1M 2143 2143 5716 5716 +167% >> cache Y N N 4K 449k 1755 469k 1834 +5% >> cache Y N N 64K 36.6k 2290 82.3k 5142 +125% >> cache Y N N 1M 2352 2352 5577 5577 +137% >> ramdisk N N Y 4K 365k 1424 354k 1384 -3% >> ramdisk N N Y 64K 31.2k 1950 74.2k 4640 +138% >> ramdisk N N Y 1M 1968 1968 5201 5201 +164% >> ramdisk N Y N 4K 9984 39 12.9k 51 +29% >> ramdisk N Y N 64K 5936 371 8960 560 +51% >> ramdisk N Y N 1M 1050 1050 1835 1835 +75% >> ramdisk Y N Y 4K 411k 1609 443k 1731 +8% >> ramdisk Y N Y 64K 34.1k 2134 77.5k 4844 +127% >> ramdisk Y N Y 1M 2248 2248 5372 5372 +139% >> ramdisk Y Y N 4K 182k 711 186k 730 +3% >> ramdisk Y Y N 64K 18.7k 1170 34.7k 2171 +86% >> ramdisk Y Y N 1M 1229 1229 2269 2269 +85% >> nvme N N Y 4K 373k 1458 387k 1512 +4% >> nvme N N Y 64K 29.2k 1827 70.9k 4431 +143% >> nvme N N Y 1M 1835 1835 4919 4919 +168% >> nvme N Y N 4K 11.7k 46 11.7k 46 0% >> nvme N Y N 64K 6453 403 8661 541 +34% >> nvme N Y N 1M 649 649 1351 1351 +108% >> nvme Y N Y 4K 372k 1456 433k 1693 +16% >> nvme Y N Y 64K 33.0k 2064 74.7k 4669 +126% >> nvme Y N Y 1M 2131 2131 5273 5273 +147% >> nvme Y Y N 4K 56.7k 222 56.4k 220 -1% >> nvme Y Y N 64K 13.4k 840 19.4k 1214 +45% >> nvme Y Y N 1M 714 714 1504 1504 +111% >> >> Thanks, >> Yi. >> >> Major changes since RFC v4: >> - Disable unsupported online defragmentation, do not fall back to >> buffer_head path. >> - Wite and wait data back while doing partial block truncate down to >> fix a stale data problem. >> - Disable the online changing of the inode journal flag to data=journal >> mode. >> - Since iomap can zero out dirty pages with unwritten extent, do not >> write data before zeroing out in ext4_zero_range(), and also do not >> zero partial blocks under a started journal handle. >> >> [1] https://lore.kernel.org/linux-ext4/20241010133333.146793-1-yi.zhang@huawei.com/ >> >> --- >> RFC v4: https://lore.kernel.org/linux-ext4/20240410142948.2817554-1-yi.zhang@huaweicloud.com/ >> RFC v3: https://lore.kernel.org/linux-ext4/20240127015825.1608160-1-yi.zhang@huaweicloud.com/ >> RFC v2: https://lore.kernel.org/linux-ext4/20240102123918.799062-1-yi.zhang@huaweicloud.com/ >> RFC v1: https://lore.kernel.org/linux-ext4/20231123125121.4064694-1-yi.zhang@huaweicloud.com/ >> >> >> Zhang Yi (27): >> ext4: remove writable userspace mappings before truncating page cache >> ext4: don't explicit update times in ext4_fallocate() >> ext4: don't write back data before punch hole in nojournal mode >> ext4: refactor ext4_punch_hole() >> ext4: refactor ext4_zero_range() >> ext4: refactor ext4_collapse_range() >> ext4: refactor ext4_insert_range() >> ext4: factor out ext4_do_fallocate() >> ext4: move out inode_lock into ext4_fallocate() >> ext4: move out common parts into ext4_fallocate() >> ext4: use reserved metadata blocks when splitting extent on endio >> ext4: introduce seq counter for the extent status entry >> ext4: add a new iomap aops for regular file's buffered IO path >> ext4: implement buffered read iomap path >> ext4: implement buffered write iomap path >> ext4: don't order data for inode with EXT4_STATE_BUFFERED_IOMAP >> ext4: implement writeback iomap path >> ext4: implement mmap iomap path >> ext4: do not always order data when partial zeroing out a block >> ext4: do not start handle if unnecessary while partial zeroing out a >> block >> ext4: implement zero_range iomap path >> ext4: disable online defrag when inode using iomap buffered I/O path >> ext4: disable inode journal mode when using iomap buffered I/O path >> ext4: partially enable iomap for the buffered I/O path of regular >> files >> ext4: enable large folio for regular file with iomap buffered I/O path >> ext4: change mount options code style >> ext4: introduce a mount option for iomap buffered I/O path >> >> fs/ext4/ext4.h | 17 +- >> fs/ext4/ext4_jbd2.c | 3 +- >> fs/ext4/ext4_jbd2.h | 8 + >> fs/ext4/extents.c | 568 +++++++++++---------------- >> fs/ext4/extents_status.c | 13 +- >> fs/ext4/file.c | 19 +- >> fs/ext4/ialloc.c | 5 + >> fs/ext4/inode.c | 755 ++++++++++++++++++++++++++++++------ >> fs/ext4/move_extent.c | 7 + >> fs/ext4/page-io.c | 105 +++++ >> fs/ext4/super.c | 185 ++++----- >> include/trace/events/ext4.h | 57 +-- >> 12 files changed, 1153 insertions(+), 589 deletions(-) >> >> -- >> 2.46.1 >> >>
On Tue, Oct 22, 2024 at 11:22 AM Zhang Yi <yi.zhang@huaweicloud.com> wrote: > > On 2024/10/22 14:59, Sedat Dilek wrote: > > On Tue, Oct 22, 2024 at 5:13 AM Zhang Yi <yi.zhang@huaweicloud.com> wrote: > >> > >> From: Zhang Yi <yi.zhang@huawei.com> > >> > >> Hello！ > >> > >> This patch series is the latest version based on my previous RFC > >> series[1], which converts the buffered I/O path of ext4 regular files to > >> iomap and enables large folios. After several months of work, almost all > >> preparatory changes have been upstreamed, thanks a lot for the review > >> and comments from Jan, Dave, Christoph, Darrick and Ritesh. Now it is > >> time for the main implementation of this conversion. > >> > >> This series is the main part of iomap buffered iomap conversion, it's > >> based on 6.12-rc4, and the code context is also depend on my anohter > >> cleanup series[1] (I've put that in this seris so we can merge it > >> directly), fixed all minor bugs found in my previous RFC v4 series. > >> Additionally, I've update change logs in each patch and also includes > >> some code modifications as Dave's suggestions. This series implements > >> the core iomap APIs on ext4 and introduces a mount option called > >> "buffered_iomap" to enable the iomap buffered I/O path. We have already > >> supported the default features, default mount options and bigalloc > >> feature. However, we do not yet support online defragmentation, inline > >> data, fs_verify, fs_crypt, ext3, and data=journal mode, ext4 will fall > >> to buffered_head I/O path automatically if you use those features and > >> options. Some of these features should be supported gradually in the > >> near future. > >> > >> Most of the implementations resemble the original buffered_head path; > >> however, there are four key differences. > >> > >> 1. The first aspect is the block allocation in the writeback path. The > >> iomap frame will invoke ->map_blocks() at least once for each dirty > >> folio. To ensure optimal writeback performance, we aim to allocate a > >> range of delalloc blocks that is as long as possible within the > >> writeback length for each invocation. In certain situations, we may > >> allocate a range of blocks that exceeds the amount we will actually > >> write back. Therefore, > >> 1) we cannot allocate a written extent for those blocks because it may > >> expose stale data in such short write cases. Instead, we should > >> allocate an unwritten extent, which means we must always enable the > >> dioread_nolock option. This change could also bring many other > >> benefits. > >> 2) We should postpone updating the 'i_disksize' until the end of the I/O > >> process, based on the actual written length. This approach can also > >> prevent the exposure of zero data, which may occur if there is a > >> power failure during an append write. > >> 3) We do not need to pre-split extents during write-back, we can > >> postpone this task until the end I/O process while converting > >> unwritten extents. > >> > >> 2. The second reason is that since we always allocate unwritten space > >> for new blocks, there is no risk of exposing stale data. As a result, > >> we do not need to order the data, which allows us to disable the > >> data=ordered mode. Consequently, we also do not require the reserved > >> handle when converting the unwritten extent in the final I/O worker, > >> we can directly start with the normal handle. > >> > >> Series details: > >> > >> Patch 1-10 is just another series of mine that refactors the fallocate > >> functions[1]. This series relies on the code context of that but has no > >> logical dependencies. I put this here just for easy access and merge. > >> > >> Patch 11-21 implement the iomap buffered read/write path, dirty folio > >> write back path and mmap path for ext4 regular file. > >> > >> Patch 22-23 disable the unsupported online-defragmentation function and > >> disable the changing of the inode journal flag to data=journal mode. > >> Please look at the following patch for details. > >> > >> Patch 24-27 introduce "buffered_iomap" mount option (is not enabled by > >> default now) to partially enable the iomap buffered I/O path and also > >> enable large folio. > >> > >> > >> About performance: > >> > >> Fio tests with psync on my machine with Intel Xeon Gold 6240 CPU with > >> 400GB system ram, 200GB ramdisk and 4TB nvme ssd disk. > >> > >> fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ > >> -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ > >> -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ > >> -group_reportin -name=$name --output=/tmp/test_log > >> > > > > Hi Zhang Yi, > > > > can you clarify about the FIO values for the diverse parameters? > > > > Hi Sedat, > > Sure, the test I present here is a simple single-thread and single-I/O > depth case with psync ioengine. Most of the FIO parameters are shown > in the tables below. > Hi Zhang Yi, Thanks for your reply. Can you share a FIO config file with all (relevant) settings? Maybe it is in the below link? Link: https://packages.debian.org/sid/all/fio-examples/filelist > For the rest, the 'iodepth' and 'numjobs' are always set to 1 and the > 'size' is 40GB. During the write cache test, I also disable the write > back process through: > > echo 0 > /proc/sys/vm/dirty_writeback_centisecs > echo 100 > /proc/sys/vm/dirty_background_ratio > echo 100 > /proc/sys/vm/dirty_ratio > ^^ Ist this info in one of the patches? If not - can you add this info to the next version's cover-letter? The patchset and improvements are valid only for powerful servers or has a notebook user any benefits of this? If you have benchmark data, please share this. I can NOT promise if I will give that patchset a try. Best thanks. Best regards, -Sedat- > Thanks, > Yi. > > > > >> == buffer read == > >> > >> buffer_head iomap + large folio > >> type bs IOPS BW(MiB/s) IOPS BW(MiB/s) > >> ------------------------------------------------------- > >> hole 4K 576k 2253 762k 2975 +32% > >> hole 64K 48.7k 3043 77.8k 4860 +60% > >> hole 1M 2960 2960 4942 4942 +67% > >> ramdisk 4K 443k 1732 530k 2069 +19% > >> ramdisk 64K 34.5k 2156 45.6k 2850 +32% > >> ramdisk 1M 2093 2093 2841 2841 +36% > >> nvme 4K 339k 1323 364k 1425 +8% > >> nvme 64K 23.6k 1471 25.2k 1574 +7% > >> nvme 1M 2012 2012 2153 2153 +7% > >> > >> > >> == buffer write == > >> > >> buffer_head iomap + large folio > >> type Overwrite Sync Writeback bs IOPS BW IOPS BW(MiB/s) > >> ---------------------------------------------------------------------- > >> cache N N N 4K 417k 1631 440k 1719 +5% > >> cache N N N 64K 33.4k 2088 81.5k 5092 +144% > >> cache N N N 1M 2143 2143 5716 5716 +167% > >> cache Y N N 4K 449k 1755 469k 1834 +5% > >> cache Y N N 64K 36.6k 2290 82.3k 5142 +125% > >> cache Y N N 1M 2352 2352 5577 5577 +137% > >> ramdisk N N Y 4K 365k 1424 354k 1384 -3% > >> ramdisk N N Y 64K 31.2k 1950 74.2k 4640 +138% > >> ramdisk N N Y 1M 1968 1968 5201 5201 +164% > >> ramdisk N Y N 4K 9984 39 12.9k 51 +29% > >> ramdisk N Y N 64K 5936 371 8960 560 +51% > >> ramdisk N Y N 1M 1050 1050 1835 1835 +75% > >> ramdisk Y N Y 4K 411k 1609 443k 1731 +8% > >> ramdisk Y N Y 64K 34.1k 2134 77.5k 4844 +127% > >> ramdisk Y N Y 1M 2248 2248 5372 5372 +139% > >> ramdisk Y Y N 4K 182k 711 186k 730 +3% > >> ramdisk Y Y N 64K 18.7k 1170 34.7k 2171 +86% > >> ramdisk Y Y N 1M 1229 1229 2269 2269 +85% > >> nvme N N Y 4K 373k 1458 387k 1512 +4% > >> nvme N N Y 64K 29.2k 1827 70.9k 4431 +143% > >> nvme N N Y 1M 1835 1835 4919 4919 +168% > >> nvme N Y N 4K 11.7k 46 11.7k 46 0% > >> nvme N Y N 64K 6453 403 8661 541 +34% > >> nvme N Y N 1M 649 649 1351 1351 +108% > >> nvme Y N Y 4K 372k 1456 433k 1693 +16% > >> nvme Y N Y 64K 33.0k 2064 74.7k 4669 +126% > >> nvme Y N Y 1M 2131 2131 5273 5273 +147% > >> nvme Y Y N 4K 56.7k 222 56.4k 220 -1% > >> nvme Y Y N 64K 13.4k 840 19.4k 1214 +45% > >> nvme Y Y N 1M 714 714 1504 1504 +111% > >> > >> Thanks, > >> Yi. > >> > >> Major changes since RFC v4: > >> - Disable unsupported online defragmentation, do not fall back to > >> buffer_head path. > >> - Wite and wait data back while doing partial block truncate down to > >> fix a stale data problem. > >> - Disable the online changing of the inode journal flag to data=journal > >> mode. > >> - Since iomap can zero out dirty pages with unwritten extent, do not > >> write data before zeroing out in ext4_zero_range(), and also do not > >> zero partial blocks under a started journal handle. > >> > >> [1] https://lore.kernel.org/linux-ext4/20241010133333.146793-1-yi.zhang@huawei.com/ > >> > >> --- > >> RFC v4: https://lore.kernel.org/linux-ext4/20240410142948.2817554-1-yi.zhang@huaweicloud.com/ > >> RFC v3: https://lore.kernel.org/linux-ext4/20240127015825.1608160-1-yi.zhang@huaweicloud.com/ > >> RFC v2: https://lore.kernel.org/linux-ext4/20240102123918.799062-1-yi.zhang@huaweicloud.com/ > >> RFC v1: https://lore.kernel.org/linux-ext4/20231123125121.4064694-1-yi.zhang@huaweicloud.com/ > >> > >> > >> Zhang Yi (27): > >> ext4: remove writable userspace mappings before truncating page cache > >> ext4: don't explicit update times in ext4_fallocate() > >> ext4: don't write back data before punch hole in nojournal mode > >> ext4: refactor ext4_punch_hole() > >> ext4: refactor ext4_zero_range() > >> ext4: refactor ext4_collapse_range() > >> ext4: refactor ext4_insert_range() > >> ext4: factor out ext4_do_fallocate() > >> ext4: move out inode_lock into ext4_fallocate() > >> ext4: move out common parts into ext4_fallocate() > >> ext4: use reserved metadata blocks when splitting extent on endio > >> ext4: introduce seq counter for the extent status entry > >> ext4: add a new iomap aops for regular file's buffered IO path > >> ext4: implement buffered read iomap path > >> ext4: implement buffered write iomap path > >> ext4: don't order data for inode with EXT4_STATE_BUFFERED_IOMAP > >> ext4: implement writeback iomap path > >> ext4: implement mmap iomap path > >> ext4: do not always order data when partial zeroing out a block > >> ext4: do not start handle if unnecessary while partial zeroing out a > >> block > >> ext4: implement zero_range iomap path > >> ext4: disable online defrag when inode using iomap buffered I/O path > >> ext4: disable inode journal mode when using iomap buffered I/O path > >> ext4: partially enable iomap for the buffered I/O path of regular > >> files > >> ext4: enable large folio for regular file with iomap buffered I/O path > >> ext4: change mount options code style > >> ext4: introduce a mount option for iomap buffered I/O path > >> > >> fs/ext4/ext4.h | 17 +- > >> fs/ext4/ext4_jbd2.c | 3 +- > >> fs/ext4/ext4_jbd2.h | 8 + > >> fs/ext4/extents.c | 568 +++++++++++---------------- > >> fs/ext4/extents_status.c | 13 +- > >> fs/ext4/file.c | 19 +- > >> fs/ext4/ialloc.c | 5 + > >> fs/ext4/inode.c | 755 ++++++++++++++++++++++++++++++------ > >> fs/ext4/move_extent.c | 7 + > >> fs/ext4/page-io.c | 105 +++++ > >> fs/ext4/super.c | 185 ++++----- > >> include/trace/events/ext4.h | 57 +-- > >> 12 files changed, 1153 insertions(+), 589 deletions(-) > >> > >> -- > >> 2.46.1 > >> > >> >
On 2024/10/23 20:13, Sedat Dilek wrote: > On Tue, Oct 22, 2024 at 11:22 AM Zhang Yi <yi.zhang@huaweicloud.com> wrote: >> >> On 2024/10/22 14:59, Sedat Dilek wrote: >>> On Tue, Oct 22, 2024 at 5:13 AM Zhang Yi <yi.zhang@huaweicloud.com> wrote: >>>> >>>> From: Zhang Yi <yi.zhang@huawei.com> >>>> >>>> Hello！ >>>> >>>> This patch series is the latest version based on my previous RFC >>>> series[1], which converts the buffered I/O path of ext4 regular files to >>>> iomap and enables large folios. After several months of work, almost all >>>> preparatory changes have been upstreamed, thanks a lot for the review >>>> and comments from Jan, Dave, Christoph, Darrick and Ritesh. Now it is >>>> time for the main implementation of this conversion. >>>> >>>> This series is the main part of iomap buffered iomap conversion, it's >>>> based on 6.12-rc4, and the code context is also depend on my anohter >>>> cleanup series[1] (I've put that in this seris so we can merge it >>>> directly), fixed all minor bugs found in my previous RFC v4 series. >>>> Additionally, I've update change logs in each patch and also includes >>>> some code modifications as Dave's suggestions. This series implements >>>> the core iomap APIs on ext4 and introduces a mount option called >>>> "buffered_iomap" to enable the iomap buffered I/O path. We have already >>>> supported the default features, default mount options and bigalloc >>>> feature. However, we do not yet support online defragmentation, inline >>>> data, fs_verify, fs_crypt, ext3, and data=journal mode, ext4 will fall >>>> to buffered_head I/O path automatically if you use those features and >>>> options. Some of these features should be supported gradually in the >>>> near future. >>>> >>>> Most of the implementations resemble the original buffered_head path; >>>> however, there are four key differences. >>>> >>>> 1. The first aspect is the block allocation in the writeback path. The >>>> iomap frame will invoke ->map_blocks() at least once for each dirty >>>> folio. To ensure optimal writeback performance, we aim to allocate a >>>> range of delalloc blocks that is as long as possible within the >>>> writeback length for each invocation. In certain situations, we may >>>> allocate a range of blocks that exceeds the amount we will actually >>>> write back. Therefore, >>>> 1) we cannot allocate a written extent for those blocks because it may >>>> expose stale data in such short write cases. Instead, we should >>>> allocate an unwritten extent, which means we must always enable the >>>> dioread_nolock option. This change could also bring many other >>>> benefits. >>>> 2) We should postpone updating the 'i_disksize' until the end of the I/O >>>> process, based on the actual written length. This approach can also >>>> prevent the exposure of zero data, which may occur if there is a >>>> power failure during an append write. >>>> 3) We do not need to pre-split extents during write-back, we can >>>> postpone this task until the end I/O process while converting >>>> unwritten extents. >>>> >>>> 2. The second reason is that since we always allocate unwritten space >>>> for new blocks, there is no risk of exposing stale data. As a result, >>>> we do not need to order the data, which allows us to disable the >>>> data=ordered mode. Consequently, we also do not require the reserved >>>> handle when converting the unwritten extent in the final I/O worker, >>>> we can directly start with the normal handle. >>>> >>>> Series details: >>>> >>>> Patch 1-10 is just another series of mine that refactors the fallocate >>>> functions[1]. This series relies on the code context of that but has no >>>> logical dependencies. I put this here just for easy access and merge. >>>> >>>> Patch 11-21 implement the iomap buffered read/write path, dirty folio >>>> write back path and mmap path for ext4 regular file. >>>> >>>> Patch 22-23 disable the unsupported online-defragmentation function and >>>> disable the changing of the inode journal flag to data=journal mode. >>>> Please look at the following patch for details. >>>> >>>> Patch 24-27 introduce "buffered_iomap" mount option (is not enabled by >>>> default now) to partially enable the iomap buffered I/O path and also >>>> enable large folio. >>>> >>>> >>>> About performance: >>>> >>>> Fio tests with psync on my machine with Intel Xeon Gold 6240 CPU with >>>> 400GB system ram, 200GB ramdisk and 4TB nvme ssd disk. >>>> >>>> fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ >>>> -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ >>>> -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ >>>> -group_reportin -name=$name --output=/tmp/test_log >>>> >>> >>> Hi Zhang Yi, >>> >>> can you clarify about the FIO values for the diverse parameters? >>> >> >> Hi Sedat, >> >> Sure, the test I present here is a simple single-thread and single-I/O >> depth case with psync ioengine. Most of the FIO parameters are shown >> in the tables below. >> > > Hi Zhang Yi, > > Thanks for your reply. > > Can you share a FIO config file with all (relevant) settings? > Maybe it is in the below link? > > Link: https://packages.debian.org/sid/all/fio-examples/filelist No, I didn't have this configuration file. I simply wrote two straightforward scripts to do this test. This serves as a reference, primarily used for performance analysis in basic read/write operations with different backends. More complex cases should be adjusted based on the actual circumstances. I have attached the scripts, feel free to use them. I suggest adjusting the parameters according to your machine configuration and service I/O model. > >> For the rest, the 'iodepth' and 'numjobs' are always set to 1 and the >> 'size' is 40GB. During the write cache test, I also disable the write >> back process through: >> >> echo 0 > /proc/sys/vm/dirty_writeback_centisecs >> echo 100 > /proc/sys/vm/dirty_background_ratio >> echo 100 > /proc/sys/vm/dirty_ratio >> > > ^^ Ist this info in one of the patches? If not - can you add this info > to the next version's cover-letter? > > The patchset and improvements are valid only for powerful servers or > has a notebook user any benefits of this? The performance improvement is primarily attributed to the cost savings of the kernel software stack with large I/O. Therefore, when the CPU becomes a bottleneck, performance should improves, i.e. the faster the disk, the more pronounced the benefits, regardless of whether the system is a server or a notebook. Thanks, Yi. > If you have benchmark data, please share this. > > I can NOT promise if I will give that patchset a try. > > Best thanks. > > Best regards, > -Sedat- > >> Thanks, >> Yi. >> >>> >>>> == buffer read == >>>> >>>> buffer_head iomap + large folio >>>> type bs IOPS BW(MiB/s) IOPS BW(MiB/s) >>>> ------------------------------------------------------- >>>> hole 4K 576k 2253 762k 2975 +32% >>>> hole 64K 48.7k 3043 77.8k 4860 +60% >>>> hole 1M 2960 2960 4942 4942 +67% >>>> ramdisk 4K 443k 1732 530k 2069 +19% >>>> ramdisk 64K 34.5k 2156 45.6k 2850 +32% >>>> ramdisk 1M 2093 2093 2841 2841 +36% >>>> nvme 4K 339k 1323 364k 1425 +8% >>>> nvme 64K 23.6k 1471 25.2k 1574 +7% >>>> nvme 1M 2012 2012 2153 2153 +7% >>>> >>>> >>>> == buffer write == >>>> >>>> buffer_head iomap + large folio >>>> type Overwrite Sync Writeback bs IOPS BW IOPS BW(MiB/s) >>>> ---------------------------------------------------------------------- >>>> cache N N N 4K 417k 1631 440k 1719 +5% >>>> cache N N N 64K 33.4k 2088 81.5k 5092 +144% >>>> cache N N N 1M 2143 2143 5716 5716 +167% >>>> cache Y N N 4K 449k 1755 469k 1834 +5% >>>> cache Y N N 64K 36.6k 2290 82.3k 5142 +125% >>>> cache Y N N 1M 2352 2352 5577 5577 +137% >>>> ramdisk N N Y 4K 365k 1424 354k 1384 -3% >>>> ramdisk N N Y 64K 31.2k 1950 74.2k 4640 +138% >>>> ramdisk N N Y 1M 1968 1968 5201 5201 +164% >>>> ramdisk N Y N 4K 9984 39 12.9k 51 +29% >>>> ramdisk N Y N 64K 5936 371 8960 560 +51% >>>> ramdisk N Y N 1M 1050 1050 1835 1835 +75% >>>> ramdisk Y N Y 4K 411k 1609 443k 1731 +8% >>>> ramdisk Y N Y 64K 34.1k 2134 77.5k 4844 +127% >>>> ramdisk Y N Y 1M 2248 2248 5372 5372 +139% >>>> ramdisk Y Y N 4K 182k 711 186k 730 +3% >>>> ramdisk Y Y N 64K 18.7k 1170 34.7k 2171 +86% >>>> ramdisk Y Y N 1M 1229 1229 2269 2269 +85% >>>> nvme N N Y 4K 373k 1458 387k 1512 +4% >>>> nvme N N Y 64K 29.2k 1827 70.9k 4431 +143% >>>> nvme N N Y 1M 1835 1835 4919 4919 +168% >>>> nvme N Y N 4K 11.7k 46 11.7k 46 0% >>>> nvme N Y N 64K 6453 403 8661 541 +34% >>>> nvme N Y N 1M 649 649 1351 1351 +108% >>>> nvme Y N Y 4K 372k 1456 433k 1693 +16% >>>> nvme Y N Y 64K 33.0k 2064 74.7k 4669 +126% >>>> nvme Y N Y 1M 2131 2131 5273 5273 +147% >>>> nvme Y Y N 4K 56.7k 222 56.4k 220 -1% >>>> nvme Y Y N 64K 13.4k 840 19.4k 1214 +45% >>>> nvme Y Y N 1M 714 714 1504 1504 +111% >>>> >>>> Thanks, >>>> Yi. >>>> >>>> Major changes since RFC v4: >>>> - Disable unsupported online defragmentation, do not fall back to >>>> buffer_head path. >>>> - Wite and wait data back while doing partial block truncate down to >>>> fix a stale data problem. >>>> - Disable the online changing of the inode journal flag to data=journal >>>> mode. >>>> - Since iomap can zero out dirty pages with unwritten extent, do not >>>> write data before zeroing out in ext4_zero_range(), and also do not >>>> zero partial blocks under a started journal handle. >>>> >>>> [1] https://lore.kernel.org/linux-ext4/20241010133333.146793-1-yi.zhang@huawei.com/ >>>> >>>> --- >>>> RFC v4: https://lore.kernel.org/linux-ext4/20240410142948.2817554-1-yi.zhang@huaweicloud.com/ >>>> RFC v3: https://lore.kernel.org/linux-ext4/20240127015825.1608160-1-yi.zhang@huaweicloud.com/ >>>> RFC v2: https://lore.kernel.org/linux-ext4/20240102123918.799062-1-yi.zhang@huaweicloud.com/ >>>> RFC v1: https://lore.kernel.org/linux-ext4/20231123125121.4064694-1-yi.zhang@huaweicloud.com/ >>>> >>>> >>>> Zhang Yi (27): >>>> ext4: remove writable userspace mappings before truncating page cache >>>> ext4: don't explicit update times in ext4_fallocate() >>>> ext4: don't write back data before punch hole in nojournal mode >>>> ext4: refactor ext4_punch_hole() >>>> ext4: refactor ext4_zero_range() >>>> ext4: refactor ext4_collapse_range() >>>> ext4: refactor ext4_insert_range() >>>> ext4: factor out ext4_do_fallocate() >>>> ext4: move out inode_lock into ext4_fallocate() >>>> ext4: move out common parts into ext4_fallocate() >>>> ext4: use reserved metadata blocks when splitting extent on endio >>>> ext4: introduce seq counter for the extent status entry >>>> ext4: add a new iomap aops for regular file's buffered IO path >>>> ext4: implement buffered read iomap path >>>> ext4: implement buffered write iomap path >>>> ext4: don't order data for inode with EXT4_STATE_BUFFERED_IOMAP >>>> ext4: implement writeback iomap path >>>> ext4: implement mmap iomap path >>>> ext4: do not always order data when partial zeroing out a block >>>> ext4: do not start handle if unnecessary while partial zeroing out a >>>> block >>>> ext4: implement zero_range iomap path >>>> ext4: disable online defrag when inode using iomap buffered I/O path >>>> ext4: disable inode journal mode when using iomap buffered I/O path >>>> ext4: partially enable iomap for the buffered I/O path of regular >>>> files >>>> ext4: enable large folio for regular file with iomap buffered I/O path >>>> ext4: change mount options code style >>>> ext4: introduce a mount option for iomap buffered I/O path >>>> >>>> fs/ext4/ext4.h | 17 +- >>>> fs/ext4/ext4_jbd2.c | 3 +- >>>> fs/ext4/ext4_jbd2.h | 8 + >>>> fs/ext4/extents.c | 568 +++++++++++---------------- >>>> fs/ext4/extents_status.c | 13 +- >>>> fs/ext4/file.c | 19 +- >>>> fs/ext4/ialloc.c | 5 + >>>> fs/ext4/inode.c | 755 ++++++++++++++++++++++++++++++------ >>>> fs/ext4/move_extent.c | 7 + >>>> fs/ext4/page-io.c | 105 +++++ >>>> fs/ext4/super.c | 185 ++++----- >>>> include/trace/events/ext4.h | 57 +-- >>>> 12 files changed, 1153 insertions(+), 589 deletions(-) >>>> >>>> -- >>>> 2.46.1 >>>> >>>> >> #!/bin/bash ramdev=$1 nvmedev=$2 MOUNT_OPT="" test_size=40G function run_fio() { local rw=read local sync=$1 local bs=$2 local iodepth=$3 local numjobs=$4 local overwrite=$5 local name=1 local size=$6 fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ -group_reportin -name=$name --output=/tmp/log cat /tmp/log >> /tmp/fio_result } function init_env() { local hole=$1 local size=$2 local dev=$3 rm -rf /mnt/* if [[ "$hole" == "1" ]]; then truncate -s $size /mnt/1.0.0 else xfs_io -f -c "pwrite 0 $size" /mnt/1.0.0 fi umount /mnt mount -o $MOUNT_OPT $dev /mnt } function reset_env() { local dev=$1 umount /mnt mount -o $MOUNT_OPT $dev /mnt } function do_one_test() { local sync=0 local hole=$1 local size=$2 local dev=$3 echo "-------------------" | tee -a /tmp/fio_result echo "=== 4K:" | tee -a /tmp/fio_result reset_env $dev run_fio $sync 4k 1 1 0 $size echo "=== 64K:" | tee -a /tmp/fio_result reset_env $dev run_fio $sync 64k 1 1 0 $size echo "=== 1M:" | tee -a /tmp/fio_result reset_env $dev run_fio $sync 1M 1 1 0 $size echo "-------------------" | tee -a /tmp/fio_result } function run_one_round() { local hole=$1 local size=$2 local dev=$3 init_env $hole $size $dev do_one_test $hole $size $dev } function run_test() { echo "---- TEST RAMDEV ----" | tee -a /tmp/fio_result mount -o $MOUNT_OPT $ramdev /mnt echo "----- 1. READ HOLE" | tee -a /tmp/fio_result run_one_round 1 $test_size $ramdev echo "----- 2. READ RAM DATA" | tee -a /tmp/fio_result run_one_round 0 $test_size $ramdev umount /mnt echo "---- TEST NVMEDEV ----" | tee -a /tmp/fio_result echo "----- 3. READ NVME DATA" | tee -a /tmp/fio_result mount -o $MOUNT_OPT $nvmedev /mnt run_one_round 0 $test_size $nvmedev umount /mnt } if [ -z "$ramdev" ] || [ -z "$nvmedev" ]; then echo "$0 <ramdev> <nvmedev>" exit fi umount /mnt mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 -F $ramdev mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 -F $nvmedev cp /tmp/fio_result /tmp/fio_result.old rm -f /tmp/fio_result ## TEST base ramdev echo "==== TEST BASE ====" | tee -a /tmp/fio_result MOUNT_OPT="nobuffered_iomap" run_test ## TEST iomap ramdev echo "==== TEST IOMAP ====" | tee -a /tmp/fio_result MOUNT_OPT="buffered_iomap" run_test #!/bin/bash ramdev=$1 nvmedev=$2 MOUNT_OPT="" test_size=40G function run_fio() { local rw=write local sync=$1 local bs=$2 local iodepth=$3 local numjobs=$4 local overwrite=$5 local name=1 local size=$6 fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ -group_reportin -name=$name --output=/tmp/log cat /tmp/log >> /tmp/fio_result } function init_env() { local dev=$1 rm -rf /mnt/* umount /mnt mount -o $MOUNT_OPT $dev /mnt } function reset_env() { local overwrite=$1 local dev=$2 if [[ "$overwrite" == "0" ]]; then rm -rf /mnt/* fi umount /mnt mount -o $MOUNT_OPT $dev /mnt } function do_one_test() { local sync=$1 local overwrite=$2 local size=$3 local dev=$4 echo "-------------------" | tee -a /tmp/fio_result echo "=== 4K:" | tee -a /tmp/fio_result reset_env $overwrite $dev run_fio $sync 4k 1 1 $overwrite $size echo "=== 64K:" | tee -a /tmp/fio_result reset_env $overwrite $dev run_fio $sync 64k 1 1 $overwrite $size echo "=== 1M:" | tee -a /tmp/fio_result reset_env $overwrite $dev run_fio $sync 1M 1 1 $overwrite $size echo "-------------------" | tee -a /tmp/fio_result } function run_one_round() { local sync=$1 local overwrite=$2 local size=$3 local dev=$4 echo "Sync:$sync, Overwrite:$overwrite" | tee -a /tmp/fio_result init_env $dev do_one_test $sync $overwrite $size $dev } function run_test() { echo "---- TEST RAMDEV ----" | tee -a /tmp/fio_result mount -o $MOUNT_OPT $ramdev /mnt echo "----- 1. WRITE CACHE" | tee -a /tmp/fio_result # Stop writeback echo 0 > /proc/sys/vm/dirty_writeback_centisecs echo 30000 > /proc/sys/vm/dirty_expire_centisecs echo 100 > /proc/sys/vm/dirty_background_ratio echo 100 > /proc/sys/vm/dirty_ratio run_one_round 0 0 $test_size $ramdev run_one_round 0 1 $test_size $ramdev echo "----- 2. WRITE RAM DISK" | tee -a /tmp/fio_result # Restore writeback echo 500 > /proc/sys/vm/dirty_writeback_centisecs echo 3000 > /proc/sys/vm/dirty_expire_centisecs echo 10 > /proc/sys/vm/dirty_background_ratio echo 20 > /proc/sys/vm/dirty_ratio run_one_round 0 0 $test_size $ramdev run_one_round 0 1 $test_size $ramdev run_one_round 1 0 $test_size $ramdev run_one_round 1 1 $test_size $ramdev umount /mnt echo "---- TEST NVMEDEV ----" | tee -a /tmp/fio_result echo "----- 3. WRITE NVME DISK" | tee -a /tmp/fio_result mount -o $MOUNT_OPT $nvmedev /mnt run_one_round 0 0 $test_size $nvmedev run_one_round 0 1 $test_size $nvmedev run_one_round 1 0 $test_size $nvmedev run_one_round 1 1 $test_size $nvmedev umount /mnt } if [ -z "$ramdev" ] || [ -z "$nvmedev" ]; then echo "$0 <ramdev> <nvmedev>" exit fi umount /mnt mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 -F $ramdev mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 -F $nvmedev cp /tmp/fio_result /tmp/fio_result.old rm -f /tmp/fio_result ## TEST base echo "==== TEST BASE ====" | tee -a /tmp/fio_result MOUNT_OPT="nobuffered_iomap" run_test ## TEST iomap echo "==== TEST IOMAP ====" | tee -a /tmp/fio_result MOUNT_OPT="buffered_iomap" run_test
From: Zhang Yi <yi.zhang@huawei.com> Hello！ This patch series is the latest version based on my previous RFC series[1], which converts the buffered I/O path of ext4 regular files to iomap and enables large folios. After several months of work, almost all preparatory changes have been upstreamed, thanks a lot for the review and comments from Jan, Dave, Christoph, Darrick and Ritesh. Now it is time for the main implementation of this conversion. This series is the main part of iomap buffered iomap conversion, it's based on 6.12-rc4, and the code context is also depend on my anohter cleanup series[1] (I've put that in this seris so we can merge it directly), fixed all minor bugs found in my previous RFC v4 series. Additionally, I've update change logs in each patch and also includes some code modifications as Dave's suggestions. This series implements the core iomap APIs on ext4 and introduces a mount option called "buffered_iomap" to enable the iomap buffered I/O path. We have already supported the default features, default mount options and bigalloc feature. However, we do not yet support online defragmentation, inline data, fs_verify, fs_crypt, ext3, and data=journal mode, ext4 will fall to buffered_head I/O path automatically if you use those features and options. Some of these features should be supported gradually in the near future. Most of the implementations resemble the original buffered_head path; however, there are four key differences. 1. The first aspect is the block allocation in the writeback path. The iomap frame will invoke ->map_blocks() at least once for each dirty folio. To ensure optimal writeback performance, we aim to allocate a range of delalloc blocks that is as long as possible within the writeback length for each invocation. In certain situations, we may allocate a range of blocks that exceeds the amount we will actually write back. Therefore, 1) we cannot allocate a written extent for those blocks because it may expose stale data in such short write cases. Instead, we should allocate an unwritten extent, which means we must always enable the dioread_nolock option. This change could also bring many other benefits. 2) We should postpone updating the 'i_disksize' until the end of the I/O process, based on the actual written length. This approach can also prevent the exposure of zero data, which may occur if there is a power failure during an append write. 3) We do not need to pre-split extents during write-back, we can postpone this task until the end I/O process while converting unwritten extents. 2. The second reason is that since we always allocate unwritten space for new blocks, there is no risk of exposing stale data. As a result, we do not need to order the data, which allows us to disable the data=ordered mode. Consequently, we also do not require the reserved handle when converting the unwritten extent in the final I/O worker, we can directly start with the normal handle. Series details: Patch 1-10 is just another series of mine that refactors the fallocate functions[1]. This series relies on the code context of that but has no logical dependencies. I put this here just for easy access and merge. Patch 11-21 implement the iomap buffered read/write path, dirty folio write back path and mmap path for ext4 regular file. Patch 22-23 disable the unsupported online-defragmentation function and disable the changing of the inode journal flag to data=journal mode. Please look at the following patch for details. Patch 24-27 introduce "buffered_iomap" mount option (is not enabled by default now) to partially enable the iomap buffered I/O path and also enable large folio. About performance: Fio tests with psync on my machine with Intel Xeon Gold 6240 CPU with 400GB system ram, 200GB ramdisk and 4TB nvme ssd disk. fio -directory=/mnt -direct=0 -iodepth=$iodepth -fsync=$sync -rw=$rw \ -numjobs=${numjobs} -bs=${bs} -ioengine=psync -size=$size \ -runtime=60 -norandommap=0 -fallocate=none -overwrite=$overwrite \ -group_reportin -name=$name --output=/tmp/test_log == buffer read == buffer_head iomap + large folio type bs IOPS BW(MiB/s) IOPS BW(MiB/s) ------------------------------------------------------- hole 4K 576k 2253 762k 2975 +32% hole 64K 48.7k 3043 77.8k 4860 +60% hole 1M 2960 2960 4942 4942 +67% ramdisk 4K 443k 1732 530k 2069 +19% ramdisk 64K 34.5k 2156 45.6k 2850 +32% ramdisk 1M 2093 2093 2841 2841 +36% nvme 4K 339k 1323 364k 1425 +8% nvme 64K 23.6k 1471 25.2k 1574 +7% nvme 1M 2012 2012 2153 2153 +7% == buffer write == buffer_head iomap + large folio type Overwrite Sync Writeback bs IOPS BW IOPS BW(MiB/s) ---------------------------------------------------------------------- cache N N N 4K 417k 1631 440k 1719 +5% cache N N N 64K 33.4k 2088 81.5k 5092 +144% cache N N N 1M 2143 2143 5716 5716 +167% cache Y N N 4K 449k 1755 469k 1834 +5% cache Y N N 64K 36.6k 2290 82.3k 5142 +125% cache Y N N 1M 2352 2352 5577 5577 +137% ramdisk N N Y 4K 365k 1424 354k 1384 -3% ramdisk N N Y 64K 31.2k 1950 74.2k 4640 +138% ramdisk N N Y 1M 1968 1968 5201 5201 +164% ramdisk N Y N 4K 9984 39 12.9k 51 +29% ramdisk N Y N 64K 5936 371 8960 560 +51% ramdisk N Y N 1M 1050 1050 1835 1835 +75% ramdisk Y N Y 4K 411k 1609 443k 1731 +8% ramdisk Y N Y 64K 34.1k 2134 77.5k 4844 +127% ramdisk Y N Y 1M 2248 2248 5372 5372 +139% ramdisk Y Y N 4K 182k 711 186k 730 +3% ramdisk Y Y N 64K 18.7k 1170 34.7k 2171 +86% ramdisk Y Y N 1M 1229 1229 2269 2269 +85% nvme N N Y 4K 373k 1458 387k 1512 +4% nvme N N Y 64K 29.2k 1827 70.9k 4431 +143% nvme N N Y 1M 1835 1835 4919 4919 +168% nvme N Y N 4K 11.7k 46 11.7k 46 0% nvme N Y N 64K 6453 403 8661 541 +34% nvme N Y N 1M 649 649 1351 1351 +108% nvme Y N Y 4K 372k 1456 433k 1693 +16% nvme Y N Y 64K 33.0k 2064 74.7k 4669 +126% nvme Y N Y 1M 2131 2131 5273 5273 +147% nvme Y Y N 4K 56.7k 222 56.4k 220 -1% nvme Y Y N 64K 13.4k 840 19.4k 1214 +45% nvme Y Y N 1M 714 714 1504 1504 +111% Thanks, Yi. Major changes since RFC v4: - Disable unsupported online defragmentation, do not fall back to buffer_head path. - Wite and wait data back while doing partial block truncate down to fix a stale data problem. - Disable the online changing of the inode journal flag to data=journal mode. - Since iomap can zero out dirty pages with unwritten extent, do not write data before zeroing out in ext4_zero_range(), and also do not zero partial blocks under a started journal handle. [1] https://lore.kernel.org/linux-ext4/20241010133333.146793-1-yi.zhang@huawei.com/ --- RFC v4: https://lore.kernel.org/linux-ext4/20240410142948.2817554-1-yi.zhang@huaweicloud.com/ RFC v3: https://lore.kernel.org/linux-ext4/20240127015825.1608160-1-yi.zhang@huaweicloud.com/ RFC v2: https://lore.kernel.org/linux-ext4/20240102123918.799062-1-yi.zhang@huaweicloud.com/ RFC v1: https://lore.kernel.org/linux-ext4/20231123125121.4064694-1-yi.zhang@huaweicloud.com/ Zhang Yi (27): ext4: remove writable userspace mappings before truncating page cache ext4: don't explicit update times in ext4_fallocate() ext4: don't write back data before punch hole in nojournal mode ext4: refactor ext4_punch_hole() ext4: refactor ext4_zero_range() ext4: refactor ext4_collapse_range() ext4: refactor ext4_insert_range() ext4: factor out ext4_do_fallocate() ext4: move out inode_lock into ext4_fallocate() ext4: move out common parts into ext4_fallocate() ext4: use reserved metadata blocks when splitting extent on endio ext4: introduce seq counter for the extent status entry ext4: add a new iomap aops for regular file's buffered IO path ext4: implement buffered read iomap path ext4: implement buffered write iomap path ext4: don't order data for inode with EXT4_STATE_BUFFERED_IOMAP ext4: implement writeback iomap path ext4: implement mmap iomap path ext4: do not always order data when partial zeroing out a block ext4: do not start handle if unnecessary while partial zeroing out a block ext4: implement zero_range iomap path ext4: disable online defrag when inode using iomap buffered I/O path ext4: disable inode journal mode when using iomap buffered I/O path ext4: partially enable iomap for the buffered I/O path of regular files ext4: enable large folio for regular file with iomap buffered I/O path ext4: change mount options code style ext4: introduce a mount option for iomap buffered I/O path fs/ext4/ext4.h | 17 +- fs/ext4/ext4_jbd2.c | 3 +- fs/ext4/ext4_jbd2.h | 8 + fs/ext4/extents.c | 568 +++++++++++---------------- fs/ext4/extents_status.c | 13 +- fs/ext4/file.c | 19 +- fs/ext4/ialloc.c | 5 + fs/ext4/inode.c | 755 ++++++++++++++++++++++++++++++------ fs/ext4/move_extent.c | 7 + fs/ext4/page-io.c | 105 +++++ fs/ext4/super.c | 185 ++++----- include/trace/events/ext4.h | 57 +-- 12 files changed, 1153 insertions(+), 589 deletions(-)