[0/5] brd: Allow to change block sizes

Message

Hannes Reinecke March 6, 2023, 12:01 p.m. UTC

Hi all,

meat to the bone: with this patchset one can change the physical and
logical block size of the 'brd' ramdisk driver.
Default is 512 (for both); one can easily increase the physical block
size to 16k and the logical block size to 4k.
Increasing the logcial block size beyond 4k gives some 'interesting'
crashes.
It should also be possible to use the resulting ram disk as a backend
for nvme target, thereby exercising the NVMe stack for larger block
sizes, too.

Happy hacking!

Hannes Reinecke (5):
  brd: convert to folios
  brd: abstract page_size conventions
  brd: make sector size configurable
  brd: limit maximal block size to 32M
  brd: make logical sector size configurable

 drivers/block/brd.c | 244 ++++++++++++++++++++++++--------------------
 1 file changed, 136 insertions(+), 108 deletions(-)

Comments

Pankaj Raghav March 7, 2023, 11:33 a.m. UTC | #1

Hi Hannes,

On Mon, Mar 06, 2023 at 01:01:22PM +0100, Hannes Reinecke wrote:
> Hi all,
> 
> meat to the bone: with this patchset one can change the physical and
> logical block size of the 'brd' ramdisk driver.
> Default is 512 (for both); one can easily increase the physical block
> size to 16k and the logical block size to 4k.
> Increasing the logcial block size beyond 4k gives some 'interesting'
> crashes.

I did something similar for an internal prototype to test large block
size. I ran the perf test suite I created on your changes and I can
clearly see a perf increase for large block IOs.

I enabled huge pages and used the iomem=mmaphuge option in fio to test
large block IOs. Here are my results:

base: next-20230307
new: base with your changes on top

bw: bandwith in MiB/s

For each set of rd_blksize and rd_logical_blksize, I ran the test with
different fio blocksize as indicated by `io_uring_iod_128_bs_4k`

rd_blksize=8192 and rd_logical_blksize=4096
+------------------------+-------------+------------+-------+----------+---------+-------+
| io_uring_iod_128_bs_4k | base[kiops] | new[kiops] | delta | base[bw] | new[bw] | delta |
+------------------------+-------------+------------+-------+----------+---------+-------+
|          read          |     567     |    605     |  6.7  |   2214   |  2362   | 6.68  |
|        randread        |     517     |    529     | 2.32  |   2019   |  2066   | 2.33  |
|         write          |     551     |    558     | 1.27  |   2154   |  2179   | 1.16  |
|       randwrite        |     481     |    502     | 4.37  |   1880   |  1962   | 4.36  |
+------------------------+-------------+------------+-------+----------+---------+-------+
+------------------------+-------------+------------+-------+----------+---------+-------+
| io_uring_iod_128_bs_8k | base[kiops] | new[kiops] | delta | base[bw] | new[bw] | delta |
+------------------------+-------------+------------+-------+----------+---------+-------+
|          read          |     462     |    512     | 10.82 |   3611   |  3997   | 10.69 |
|        randread        |     426     |    445     | 4.46  |   3326   |  3480   | 4.63  |
|         write          |     442     |    472     | 6.79  |   3454   |  3687   | 6.75  |
|       randwrite        |     401     |    426     | 6.23  |   3134   |  3328   | 6.19  |
+------------------------+-------------+------------+-------+----------+---------+-------+
+-------------------------+-------------+------------+-------+----------+---------+-------+
| io_uring_iod_128_bs_16k | base[kiops] | new[kiops] | delta | base[bw] | new[bw] | delta |
+-------------------------+-------------+------------+-------+----------+---------+-------+
|          read           |     343     |    390     | 13.7  |   5360   |  6086   | 13.54 |
|        randread         |     317     |    364     | 14.83 |   4946   |  5694   | 15.12 |
|          write          |     335     |    346     | 3.28  |   5235   |  5414   | 3.42  |
|        randwrite        |     305     |    327     | 7.21  |   4759   |  5106   | 7.29  |
+-------------------------+-------------+------------+-------+----------+---------+-------+

rd_blksize=16384 and rd_logical_blksize=4096
+------------------------+-------------+------------+-------+----------+---------+-------+
| io_uring_iod_128_bs_4k | base[kiops] | new[kiops] | delta | base[bw] | new[bw] | delta |
+------------------------+-------------+------------+-------+----------+---------+-------+
|          read          |     576     |    586     | 1.74  |   2250   |  2291   | 1.82  |
|        randread        |     524     |    548     | 4.58  |   2046   |  2139   | 4.55  |
|         write          |     533     |    545     | 2.25  |   2081   |  2129   | 2.31  |
|       randwrite        |     484     |    496     | 2.48  |   1892   |  1938   | 2.43  |
+------------------------+-------------+------------+-------+----------+---------+-------+
+------------------------+-------------+------------+-------+----------+---------+-------+
| io_uring_iod_128_bs_8k | base[kiops] | new[kiops] | delta | base[bw] | new[bw] | delta |
+------------------------+-------------+------------+-------+----------+---------+-------+
|          read          |     461     |    491     | 6.51  |   3601   |  3836   | 6.53  |
|        randread        |     425     |    472     | 11.06 |   3323   |  3684   | 10.86 |
|         write          |     454     |    465     | 2.42  |   3543   |  3632   | 2.51  |
|       randwrite        |     395     |    430     | 8.86  |   3086   |  3357   | 8.78  |
+------------------------+-------------+------------+-------+----------+---------+-------+
+-------------------------+-------------+------------+-------+----------+---------+-------+
| io_uring_iod_128_bs_16k | base[kiops] | new[kiops] | delta | base[bw] | new[bw] | delta |
+-------------------------+-------------+------------+-------+----------+---------+-------+
|          read           |     338     |    400     | 18.34 |   5282   |  6255   | 18.42 |
|        randread         |     317     |    384     | 21.14 |   4959   |  5997   | 20.93 |
|          write          |     335     |    354     | 5.67  |   5239   |  5525   | 5.46  |
|        randwrite        |     303     |    326     | 7.59  |   4728   |  5097   |  7.8  |
+-------------------------+-------------+------------+-------+----------+---------+-------+