| Message ID | 20200214211227.407836-1-Anna.Schumaker@Netapp.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | NFS: Add support for the v4.2 READ_PLUS operation | expand |
Hi Anna- > On Feb 14, 2020, at 4:12 PM, schumaker.anna@gmail.com wrote: > > From: Anna Schumaker <Anna.Schumaker@Netapp.com> > > These patches add client support for the READ_PLUS operation, which > breaks read requests into several "data" and "hole" segments when > replying to the client. I also add a "noreadplus" mount option to allow > users to disable the new operation if it becomes a problem, similar to > the "nordirplus" mount option that we already have. Hrm, I went looking for the patch that adds "noreadplus", but I don't see it in this series? Wondering if, to start off, the default should be "noreadplus" until our feet are under us. Just a thought. > Here are the results of some performance tests I ran on Netapp lab > machines. I tested by reading various 2G files from a few different > undelying filesystems and across several NFS versions. I used the > `vmtouch` utility to make sure files were only cached when we wanted > them to be. In addition to 100% data and 100% hole cases, I also tested > with files that alternate between data and hole segments. These files > have either 4K, 8K, 16K, or 32K segment sizes and start with either data > or hole segments. So the file mixed-4d has a 4K segment size beginning > with a data segment, but mixed-32h hase 32K segments beginning with a > hole. The units are in seconds, with the first number for each NFS > version being the uncached read time and the second number is for when > the file is cached on the server. > > ext4 | v3 | v4.0 | v4.1 | v4.2 | > ----------|-----------------|-----------------|-----------------|-----------------| > data | 22.909 : 18.253 | 22.934 : 18.252 | 22.902 : 18.253 | 23.485 : 18.253 | > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.708 : 0.709 | > mixed-4d | 28.261 : 18.253 | 29.616 : 18.252 | 28.341 : 18.252 | 24.508 : 9.150 | > mixed-8d | 27.956 : 18.253 | 28.404 : 18.252 | 28.320 : 18.252 | 23.967 : 9.140 | > mixed-16d | 28.172 : 18.253 | 27.946 : 18.252 | 27.627 : 18.252 | 23.043 : 9.134 | > mixed-32d | 25.350 : 18.253 | 24.406 : 18.252 | 24.384 : 18.253 | 20.698 : 9.132 | > mixed-4h | 28.913 : 18.253 | 28.564 : 18.252 | 27.996 : 18.252 | 21.837 : 9.150 | > mixed-8h | 28.625 : 18.253 | 27.833 : 18.252 | 27.798 : 18.253 | 21.710 : 9.140 | > mixed-16h | 27.975 : 18.253 | 27.662 : 18.252 | 27.795 : 18.253 | 20.585 : 9.134 | > mixed-32h | 25.958 : 18.253 | 25.491 : 18.252 | 24.856 : 18.252 | 21.018 : 9.132 | > > xfs | v3 | v4.0 | v4.1 | v4.2 | > ----------|-----------------|-----------------|-----------------|-----------------| > data | 22.041 : 18.253 | 22.618 : 18.252 | 23.067 : 18.253 | 23.496 : 18.253 | > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.723 : 0.708 | > mixed-4d | 29.417 : 18.253 | 28.503 : 18.252 | 28.671 : 18.253 | 24.957 : 9.150 | > mixed-8d | 29.080 : 18.253 | 29.401 : 18.252 | 29.251 : 18.252 | 24.625 : 9.140 | > mixed-16d | 27.638 : 18.253 | 28.606 : 18.252 | 27.871 : 18.253 | 25.511 : 9.135 | > mixed-32d | 24.967 : 18.253 | 25.239 : 18.252 | 25.434 : 18.252 | 21.728 : 9.132 | > mixed-4h | 34.816 : 18.253 | 36.243 : 18.252 | 35.837 : 18.252 | 32.332 : 9.150 | > mixed-8h | 43.469 : 18.253 | 44.009 : 18.252 | 43.810 : 18.253 | 37.962 : 9.140 | > mixed-16h | 29.280 : 18.253 | 28.563 : 18.252 | 28.241 : 18.252 | 22.116 : 9.134 | > mixed-32h | 29.428 : 18.253 | 29.378 : 18.252 | 28.808 : 18.253 | 27.378 : 9.134 | > > btrfs | v3 | v4.0 | v4.1 | v4.2 | > ----------|-----------------|-----------------|-----------------|-----------------| > data | 25.547 : 18.253 | 25.053 : 18.252 | 24.209 : 18.253 | 32.121 : 18.253 | > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.252 | 0.702 : 0.724 | > mixed-4d | 19.016 : 18.253 | 18.822 : 18.252 | 18.955 : 18.253 | 18.697 : 9.150 | > mixed-8d | 19.186 : 18.253 | 19.444 : 18.252 | 18.841 : 18.253 | 18.452 : 9.140 | > mixed-16d | 18.480 : 18.253 | 19.010 : 18.252 | 19.167 : 18.252 | 16.000 : 9.134 | > mixed-32d | 18.635 : 18.253 | 18.565 : 18.252 | 18.550 : 18.252 | 15.930 : 9.132 | > mixed-4h | 19.079 : 18.253 | 18.990 : 18.252 | 19.157 : 18.253 | 27.834 : 9.150 | > mixed-8h | 18.613 : 18.253 | 19.234 : 18.252 | 18.616 : 18.253 | 20.177 : 9.140 | > mixed-16h | 18.590 : 18.253 | 19.221 : 18.252 | 19.654 : 18.253 | 17.273 : 9.135 | > mixed-32h | 18.768 : 18.253 | 19.122 : 18.252 | 18.535 : 18.252 | 15.791 : 9.132 | > > ext3 | v3 | v4.0 | v4.1 | v4.2 | > ----------|-----------------|-----------------|-----------------|-----------------| > data | 34.292 : 18.253 | 33.810 : 18.252 | 33.450 : 18.253 | 33.390 : 18.254 | > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.718 : 0.728 | > mixed-4d | 46.818 : 18.253 | 47.140 : 18.252 | 48.385 : 18.253 | 42.887 : 9.150 | > mixed-8d | 58.554 : 18.253 | 59.277 : 18.252 | 59.673 : 18.253 | 56.760 : 9.140 | > mixed-16d | 44.631 : 18.253 | 44.291 : 18.252 | 44.729 : 18.253 | 40.237 : 9.135 | > mixed-32d | 39.110 : 18.253 | 38.735 : 18.252 | 38.902 : 18.252 | 35.270 : 9.132 | > mixed-4h | 56.396 : 18.253 | 56.387 : 18.252 | 56.573 : 18.253 | 67.661 : 9.150 | > mixed-8h | 58.483 : 18.253 | 58.484 : 18.252 | 59.099 : 18.253 | 77.958 : 9.140 | > mixed-16h | 42.511 : 18.253 | 42.338 : 18.252 | 42.356 : 18.252 | 51.805 : 9.135 | > mixed-32h | 38.419 : 18.253 | 38.504 : 18.252 | 38.643 : 18.252 | 40.411 : 9.132 | > > > Changes since v1: > - Rebase to 5.6-rc1 > - Drop the mount option patch for now > - Fix fallback to READ when the server doesn't support READ_PLUS > > Any questions? > Anna > > > Anna Schumaker (6): > SUNRPC: Split out a function for setting current page > SUNRPC: Add the ability to expand holes in data pages > SUNRPC: Add the ability to shift data to a specific offset > NFS: Add READ_PLUS data segment support > NFS: Add READ_PLUS hole segment decoding > NFS: Decode multiple READ_PLUS segments > > fs/nfs/nfs42xdr.c | 169 +++++++++++++++++++++++++ > fs/nfs/nfs4proc.c | 43 ++++++- > fs/nfs/nfs4xdr.c | 1 + > include/linux/nfs4.h | 2 +- > include/linux/nfs_fs_sb.h | 1 + > include/linux/nfs_xdr.h | 2 +- > include/linux/sunrpc/xdr.h | 2 + > net/sunrpc/xdr.c | 244 ++++++++++++++++++++++++++++++++++++- > 8 files changed, 457 insertions(+), 7 deletions(-) > > -- > 2.25.0 > -- Chuck Lever chucklever@gmail.com
On Thu, 2020-02-20 at 12:40 -0500, Chuck Lever wrote: > Hi Anna- > > > On Feb 14, 2020, at 4:12 PM, schumaker.anna@gmail.com wrote: > > > > From: Anna Schumaker <Anna.Schumaker@Netapp.com> > > > > These patches add client support for the READ_PLUS operation, which > > breaks read requests into several "data" and "hole" segments when > > replying to the client. I also add a "noreadplus" mount option to allow > > users to disable the new operation if it becomes a problem, similar to > > the "nordirplus" mount option that we already have. > > Hrm, I went looking for the patch that adds "noreadplus", but I > don't see it in this series? You suggested dropping that patch in the v1 posting and waiting to see if anybody asks for it. > > Wondering if, to start off, the default should be "noreadplus" > until our feet are under us. Just a thought. I could re-add the patch with this as the default if that's the way everybody wants to go. Anna > > > > Here are the results of some performance tests I ran on Netapp lab > > machines. I tested by reading various 2G files from a few different > > undelying filesystems and across several NFS versions. I used the > > `vmtouch` utility to make sure files were only cached when we wanted > > them to be. In addition to 100% data and 100% hole cases, I also tested > > with files that alternate between data and hole segments. These files > > have either 4K, 8K, 16K, or 32K segment sizes and start with either data > > or hole segments. So the file mixed-4d has a 4K segment size beginning > > with a data segment, but mixed-32h hase 32K segments beginning with a > > hole. The units are in seconds, with the first number for each NFS > > version being the uncached read time and the second number is for when > > the file is cached on the server. > > > > ext4 | v3 | v4.0 | v4.1 | v4.2 > > | > > ----------|-----------------|-----------------|-----------------|----------- > > ------| > > data | 22.909 : 18.253 | 22.934 : 18.252 | 22.902 : 18.253 | 23.485 : > > 18.253 | > > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.708 > > : 0.709 | > > mixed-4d | 28.261 : 18.253 | 29.616 : 18.252 | 28.341 : 18.252 | 24.508 > > : 9.150 | > > mixed-8d | 27.956 : 18.253 | 28.404 : 18.252 | 28.320 : 18.252 | 23.967 > > : 9.140 | > > mixed-16d | 28.172 : 18.253 | 27.946 : 18.252 | 27.627 : 18.252 | 23.043 > > : 9.134 | > > mixed-32d | 25.350 : 18.253 | 24.406 : 18.252 | 24.384 : 18.253 | 20.698 > > : 9.132 | > > mixed-4h | 28.913 : 18.253 | 28.564 : 18.252 | 27.996 : 18.252 | 21.837 > > : 9.150 | > > mixed-8h | 28.625 : 18.253 | 27.833 : 18.252 | 27.798 : 18.253 | 21.710 > > : 9.140 | > > mixed-16h | 27.975 : 18.253 | 27.662 : 18.252 | 27.795 : 18.253 | 20.585 > > : 9.134 | > > mixed-32h | 25.958 : 18.253 | 25.491 : 18.252 | 24.856 : 18.252 | 21.018 > > : 9.132 | > > > > xfs | v3 | v4.0 | v4.1 | v4.2 > > | > > ----------|-----------------|-----------------|-----------------|----------- > > ------| > > data | 22.041 : 18.253 | 22.618 : 18.252 | 23.067 : 18.253 | 23.496 : > > 18.253 | > > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.723 > > : 0.708 | > > mixed-4d | 29.417 : 18.253 | 28.503 : 18.252 | 28.671 : 18.253 | 24.957 > > : 9.150 | > > mixed-8d | 29.080 : 18.253 | 29.401 : 18.252 | 29.251 : 18.252 | 24.625 > > : 9.140 | > > mixed-16d | 27.638 : 18.253 | 28.606 : 18.252 | 27.871 : 18.253 | 25.511 > > : 9.135 | > > mixed-32d | 24.967 : 18.253 | 25.239 : 18.252 | 25.434 : 18.252 | 21.728 > > : 9.132 | > > mixed-4h | 34.816 : 18.253 | 36.243 : 18.252 | 35.837 : 18.252 | 32.332 > > : 9.150 | > > mixed-8h | 43.469 : 18.253 | 44.009 : 18.252 | 43.810 : 18.253 | 37.962 > > : 9.140 | > > mixed-16h | 29.280 : 18.253 | 28.563 : 18.252 | 28.241 : 18.252 | 22.116 > > : 9.134 | > > mixed-32h | 29.428 : 18.253 | 29.378 : 18.252 | 28.808 : 18.253 | 27.378 > > : 9.134 | > > > > btrfs | v3 | v4.0 | v4.1 | v4.2 > > | > > ----------|-----------------|-----------------|-----------------|----------- > > ------| > > data | 25.547 : 18.253 | 25.053 : 18.252 | 24.209 : 18.253 | 32.121 : > > 18.253 | > > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.252 | 0.702 > > : 0.724 | > > mixed-4d | 19.016 : 18.253 | 18.822 : 18.252 | 18.955 : 18.253 | 18.697 > > : 9.150 | > > mixed-8d | 19.186 : 18.253 | 19.444 : 18.252 | 18.841 : 18.253 | 18.452 > > : 9.140 | > > mixed-16d | 18.480 : 18.253 | 19.010 : 18.252 | 19.167 : 18.252 | 16.000 > > : 9.134 | > > mixed-32d | 18.635 : 18.253 | 18.565 : 18.252 | 18.550 : 18.252 | 15.930 > > : 9.132 | > > mixed-4h | 19.079 : 18.253 | 18.990 : 18.252 | 19.157 : 18.253 | 27.834 > > : 9.150 | > > mixed-8h | 18.613 : 18.253 | 19.234 : 18.252 | 18.616 : 18.253 | 20.177 > > : 9.140 | > > mixed-16h | 18.590 : 18.253 | 19.221 : 18.252 | 19.654 : 18.253 | 17.273 > > : 9.135 | > > mixed-32h | 18.768 : 18.253 | 19.122 : 18.252 | 18.535 : 18.252 | 15.791 > > : 9.132 | > > > > ext3 | v3 | v4.0 | v4.1 | v4.2 > > | > > ----------|-----------------|-----------------|-----------------|----------- > > ------| > > data | 34.292 : 18.253 | 33.810 : 18.252 | 33.450 : 18.253 | 33.390 : > > 18.254 | > > hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.718 > > : 0.728 | > > mixed-4d | 46.818 : 18.253 | 47.140 : 18.252 | 48.385 : 18.253 | 42.887 > > : 9.150 | > > mixed-8d | 58.554 : 18.253 | 59.277 : 18.252 | 59.673 : 18.253 | 56.760 > > : 9.140 | > > mixed-16d | 44.631 : 18.253 | 44.291 : 18.252 | 44.729 : 18.253 | 40.237 > > : 9.135 | > > mixed-32d | 39.110 : 18.253 | 38.735 : 18.252 | 38.902 : 18.252 | 35.270 > > : 9.132 | > > mixed-4h | 56.396 : 18.253 | 56.387 : 18.252 | 56.573 : 18.253 | 67.661 > > : 9.150 | > > mixed-8h | 58.483 : 18.253 | 58.484 : 18.252 | 59.099 : 18.253 | 77.958 > > : 9.140 | > > mixed-16h | 42.511 : 18.253 | 42.338 : 18.252 | 42.356 : 18.252 | 51.805 > > : 9.135 | > > mixed-32h | 38.419 : 18.253 | 38.504 : 18.252 | 38.643 : 18.252 | 40.411 > > : 9.132 | > > > > > > Changes since v1: > > - Rebase to 5.6-rc1 > > - Drop the mount option patch for now > > - Fix fallback to READ when the server doesn't support READ_PLUS > > > > Any questions? > > Anna > > > > > > Anna Schumaker (6): > > SUNRPC: Split out a function for setting current page > > SUNRPC: Add the ability to expand holes in data pages > > SUNRPC: Add the ability to shift data to a specific offset > > NFS: Add READ_PLUS data segment support > > NFS: Add READ_PLUS hole segment decoding > > NFS: Decode multiple READ_PLUS segments > > > > fs/nfs/nfs42xdr.c | 169 +++++++++++++++++++++++++ > > fs/nfs/nfs4proc.c | 43 ++++++- > > fs/nfs/nfs4xdr.c | 1 + > > include/linux/nfs4.h | 2 +- > > include/linux/nfs_fs_sb.h | 1 + > > include/linux/nfs_xdr.h | 2 +- > > include/linux/sunrpc/xdr.h | 2 + > > net/sunrpc/xdr.c | 244 ++++++++++++++++++++++++++++++++++++- > > 8 files changed, 457 insertions(+), 7 deletions(-) > > > > -- > > 2.25.0 > > > > -- > Chuck Lever > chucklever@gmail.com > > >
> On Feb 20, 2020, at 1:25 PM, Anna Schumaker <schumaker.anna@gmail.com> wrote: > > On Thu, 2020-02-20 at 12:40 -0500, Chuck Lever wrote: >> Hi Anna- >> >>> On Feb 14, 2020, at 4:12 PM, schumaker.anna@gmail.com wrote: >>> >>> From: Anna Schumaker <Anna.Schumaker@Netapp.com> >>> >>> These patches add client support for the READ_PLUS operation, which >>> breaks read requests into several "data" and "hole" segments when >>> replying to the client. I also add a "noreadplus" mount option to allow >>> users to disable the new operation if it becomes a problem, similar to >>> the "nordirplus" mount option that we already have. >> >> Hrm, I went looking for the patch that adds "noreadplus", but I >> don't see it in this series? > > You suggested dropping that patch in the v1 posting and waiting to see if > anybody asks for it. Yes, recalling that now. I requested that because I don't like to add administrative interfaces if there isn't a clear need. I think we might have one now! >> Wondering if, to start off, the default should be "noreadplus" >> until our feet are under us. Just a thought. > > I could re-add the patch with this as the default if that's the way everybody > wants to go. Yep, I'm interested in other opinions too. > Anna > >> >> >>> Here are the results of some performance tests I ran on Netapp lab >>> machines. I tested by reading various 2G files from a few different >>> undelying filesystems and across several NFS versions. I used the >>> `vmtouch` utility to make sure files were only cached when we wanted >>> them to be. In addition to 100% data and 100% hole cases, I also tested >>> with files that alternate between data and hole segments. These files >>> have either 4K, 8K, 16K, or 32K segment sizes and start with either data >>> or hole segments. So the file mixed-4d has a 4K segment size beginning >>> with a data segment, but mixed-32h hase 32K segments beginning with a >>> hole. The units are in seconds, with the first number for each NFS >>> version being the uncached read time and the second number is for when >>> the file is cached on the server. >>> >>> ext4 | v3 | v4.0 | v4.1 | v4.2 >>> | >>> ----------|-----------------|-----------------|-----------------|----------- >>> ------| >>> data | 22.909 : 18.253 | 22.934 : 18.252 | 22.902 : 18.253 | 23.485 : >>> 18.253 | >>> hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.708 >>> : 0.709 | >>> mixed-4d | 28.261 : 18.253 | 29.616 : 18.252 | 28.341 : 18.252 | 24.508 >>> : 9.150 | >>> mixed-8d | 27.956 : 18.253 | 28.404 : 18.252 | 28.320 : 18.252 | 23.967 >>> : 9.140 | >>> mixed-16d | 28.172 : 18.253 | 27.946 : 18.252 | 27.627 : 18.252 | 23.043 >>> : 9.134 | >>> mixed-32d | 25.350 : 18.253 | 24.406 : 18.252 | 24.384 : 18.253 | 20.698 >>> : 9.132 | >>> mixed-4h | 28.913 : 18.253 | 28.564 : 18.252 | 27.996 : 18.252 | 21.837 >>> : 9.150 | >>> mixed-8h | 28.625 : 18.253 | 27.833 : 18.252 | 27.798 : 18.253 | 21.710 >>> : 9.140 | >>> mixed-16h | 27.975 : 18.253 | 27.662 : 18.252 | 27.795 : 18.253 | 20.585 >>> : 9.134 | >>> mixed-32h | 25.958 : 18.253 | 25.491 : 18.252 | 24.856 : 18.252 | 21.018 >>> : 9.132 | >>> >>> xfs | v3 | v4.0 | v4.1 | v4.2 >>> | >>> ----------|-----------------|-----------------|-----------------|----------- >>> ------| >>> data | 22.041 : 18.253 | 22.618 : 18.252 | 23.067 : 18.253 | 23.496 : >>> 18.253 | >>> hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.723 >>> : 0.708 | >>> mixed-4d | 29.417 : 18.253 | 28.503 : 18.252 | 28.671 : 18.253 | 24.957 >>> : 9.150 | >>> mixed-8d | 29.080 : 18.253 | 29.401 : 18.252 | 29.251 : 18.252 | 24.625 >>> : 9.140 | >>> mixed-16d | 27.638 : 18.253 | 28.606 : 18.252 | 27.871 : 18.253 | 25.511 >>> : 9.135 | >>> mixed-32d | 24.967 : 18.253 | 25.239 : 18.252 | 25.434 : 18.252 | 21.728 >>> : 9.132 | >>> mixed-4h | 34.816 : 18.253 | 36.243 : 18.252 | 35.837 : 18.252 | 32.332 >>> : 9.150 | >>> mixed-8h | 43.469 : 18.253 | 44.009 : 18.252 | 43.810 : 18.253 | 37.962 >>> : 9.140 | >>> mixed-16h | 29.280 : 18.253 | 28.563 : 18.252 | 28.241 : 18.252 | 22.116 >>> : 9.134 | >>> mixed-32h | 29.428 : 18.253 | 29.378 : 18.252 | 28.808 : 18.253 | 27.378 >>> : 9.134 | >>> >>> btrfs | v3 | v4.0 | v4.1 | v4.2 >>> | >>> ----------|-----------------|-----------------|-----------------|----------- >>> ------| >>> data | 25.547 : 18.253 | 25.053 : 18.252 | 24.209 : 18.253 | 32.121 : >>> 18.253 | >>> hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.252 | 0.702 >>> : 0.724 | >>> mixed-4d | 19.016 : 18.253 | 18.822 : 18.252 | 18.955 : 18.253 | 18.697 >>> : 9.150 | >>> mixed-8d | 19.186 : 18.253 | 19.444 : 18.252 | 18.841 : 18.253 | 18.452 >>> : 9.140 | >>> mixed-16d | 18.480 : 18.253 | 19.010 : 18.252 | 19.167 : 18.252 | 16.000 >>> : 9.134 | >>> mixed-32d | 18.635 : 18.253 | 18.565 : 18.252 | 18.550 : 18.252 | 15.930 >>> : 9.132 | >>> mixed-4h | 19.079 : 18.253 | 18.990 : 18.252 | 19.157 : 18.253 | 27.834 >>> : 9.150 | >>> mixed-8h | 18.613 : 18.253 | 19.234 : 18.252 | 18.616 : 18.253 | 20.177 >>> : 9.140 | >>> mixed-16h | 18.590 : 18.253 | 19.221 : 18.252 | 19.654 : 18.253 | 17.273 >>> : 9.135 | >>> mixed-32h | 18.768 : 18.253 | 19.122 : 18.252 | 18.535 : 18.252 | 15.791 >>> : 9.132 | >>> >>> ext3 | v3 | v4.0 | v4.1 | v4.2 >>> | >>> ----------|-----------------|-----------------|-----------------|----------- >>> ------| >>> data | 34.292 : 18.253 | 33.810 : 18.252 | 33.450 : 18.253 | 33.390 : >>> 18.254 | >>> hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.718 >>> : 0.728 | >>> mixed-4d | 46.818 : 18.253 | 47.140 : 18.252 | 48.385 : 18.253 | 42.887 >>> : 9.150 | >>> mixed-8d | 58.554 : 18.253 | 59.277 : 18.252 | 59.673 : 18.253 | 56.760 >>> : 9.140 | >>> mixed-16d | 44.631 : 18.253 | 44.291 : 18.252 | 44.729 : 18.253 | 40.237 >>> : 9.135 | >>> mixed-32d | 39.110 : 18.253 | 38.735 : 18.252 | 38.902 : 18.252 | 35.270 >>> : 9.132 | >>> mixed-4h | 56.396 : 18.253 | 56.387 : 18.252 | 56.573 : 18.253 | 67.661 >>> : 9.150 | >>> mixed-8h | 58.483 : 18.253 | 58.484 : 18.252 | 59.099 : 18.253 | 77.958 >>> : 9.140 | >>> mixed-16h | 42.511 : 18.253 | 42.338 : 18.252 | 42.356 : 18.252 | 51.805 >>> : 9.135 | >>> mixed-32h | 38.419 : 18.253 | 38.504 : 18.252 | 38.643 : 18.252 | 40.411 >>> : 9.132 | >>> >>> >>> Changes since v1: >>> - Rebase to 5.6-rc1 >>> - Drop the mount option patch for now >>> - Fix fallback to READ when the server doesn't support READ_PLUS >>> >>> Any questions? >>> Anna >>> >>> >>> Anna Schumaker (6): >>> SUNRPC: Split out a function for setting current page >>> SUNRPC: Add the ability to expand holes in data pages >>> SUNRPC: Add the ability to shift data to a specific offset >>> NFS: Add READ_PLUS data segment support >>> NFS: Add READ_PLUS hole segment decoding >>> NFS: Decode multiple READ_PLUS segments >>> >>> fs/nfs/nfs42xdr.c | 169 +++++++++++++++++++++++++ >>> fs/nfs/nfs4proc.c | 43 ++++++- >>> fs/nfs/nfs4xdr.c | 1 + >>> include/linux/nfs4.h | 2 +- >>> include/linux/nfs_fs_sb.h | 1 + >>> include/linux/nfs_xdr.h | 2 +- >>> include/linux/sunrpc/xdr.h | 2 + >>> net/sunrpc/xdr.c | 244 ++++++++++++++++++++++++++++++++++++- >>> 8 files changed, 457 insertions(+), 7 deletions(-) >>> >>> -- >>> 2.25.0 >>> >> >> -- >> Chuck Lever >> chucklever@gmail.com -- Chuck Lever chucklever@gmail.com
From: Anna Schumaker <Anna.Schumaker@Netapp.com> These patches add client support for the READ_PLUS operation, which breaks read requests into several "data" and "hole" segments when replying to the client. I also add a "noreadplus" mount option to allow users to disable the new operation if it becomes a problem, similar to the "nordirplus" mount option that we already have. Here are the results of some performance tests I ran on Netapp lab machines. I tested by reading various 2G files from a few different undelying filesystems and across several NFS versions. I used the `vmtouch` utility to make sure files were only cached when we wanted them to be. In addition to 100% data and 100% hole cases, I also tested with files that alternate between data and hole segments. These files have either 4K, 8K, 16K, or 32K segment sizes and start with either data or hole segments. So the file mixed-4d has a 4K segment size beginning with a data segment, but mixed-32h hase 32K segments beginning with a hole. The units are in seconds, with the first number for each NFS version being the uncached read time and the second number is for when the file is cached on the server. ext4 | v3 | v4.0 | v4.1 | v4.2 | ----------|-----------------|-----------------|-----------------|-----------------| data | 22.909 : 18.253 | 22.934 : 18.252 | 22.902 : 18.253 | 23.485 : 18.253 | hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.708 : 0.709 | mixed-4d | 28.261 : 18.253 | 29.616 : 18.252 | 28.341 : 18.252 | 24.508 : 9.150 | mixed-8d | 27.956 : 18.253 | 28.404 : 18.252 | 28.320 : 18.252 | 23.967 : 9.140 | mixed-16d | 28.172 : 18.253 | 27.946 : 18.252 | 27.627 : 18.252 | 23.043 : 9.134 | mixed-32d | 25.350 : 18.253 | 24.406 : 18.252 | 24.384 : 18.253 | 20.698 : 9.132 | mixed-4h | 28.913 : 18.253 | 28.564 : 18.252 | 27.996 : 18.252 | 21.837 : 9.150 | mixed-8h | 28.625 : 18.253 | 27.833 : 18.252 | 27.798 : 18.253 | 21.710 : 9.140 | mixed-16h | 27.975 : 18.253 | 27.662 : 18.252 | 27.795 : 18.253 | 20.585 : 9.134 | mixed-32h | 25.958 : 18.253 | 25.491 : 18.252 | 24.856 : 18.252 | 21.018 : 9.132 | xfs | v3 | v4.0 | v4.1 | v4.2 | ----------|-----------------|-----------------|-----------------|-----------------| data | 22.041 : 18.253 | 22.618 : 18.252 | 23.067 : 18.253 | 23.496 : 18.253 | hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.723 : 0.708 | mixed-4d | 29.417 : 18.253 | 28.503 : 18.252 | 28.671 : 18.253 | 24.957 : 9.150 | mixed-8d | 29.080 : 18.253 | 29.401 : 18.252 | 29.251 : 18.252 | 24.625 : 9.140 | mixed-16d | 27.638 : 18.253 | 28.606 : 18.252 | 27.871 : 18.253 | 25.511 : 9.135 | mixed-32d | 24.967 : 18.253 | 25.239 : 18.252 | 25.434 : 18.252 | 21.728 : 9.132 | mixed-4h | 34.816 : 18.253 | 36.243 : 18.252 | 35.837 : 18.252 | 32.332 : 9.150 | mixed-8h | 43.469 : 18.253 | 44.009 : 18.252 | 43.810 : 18.253 | 37.962 : 9.140 | mixed-16h | 29.280 : 18.253 | 28.563 : 18.252 | 28.241 : 18.252 | 22.116 : 9.134 | mixed-32h | 29.428 : 18.253 | 29.378 : 18.252 | 28.808 : 18.253 | 27.378 : 9.134 | btrfs | v3 | v4.0 | v4.1 | v4.2 | ----------|-----------------|-----------------|-----------------|-----------------| data | 25.547 : 18.253 | 25.053 : 18.252 | 24.209 : 18.253 | 32.121 : 18.253 | hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.252 | 0.702 : 0.724 | mixed-4d | 19.016 : 18.253 | 18.822 : 18.252 | 18.955 : 18.253 | 18.697 : 9.150 | mixed-8d | 19.186 : 18.253 | 19.444 : 18.252 | 18.841 : 18.253 | 18.452 : 9.140 | mixed-16d | 18.480 : 18.253 | 19.010 : 18.252 | 19.167 : 18.252 | 16.000 : 9.134 | mixed-32d | 18.635 : 18.253 | 18.565 : 18.252 | 18.550 : 18.252 | 15.930 : 9.132 | mixed-4h | 19.079 : 18.253 | 18.990 : 18.252 | 19.157 : 18.253 | 27.834 : 9.150 | mixed-8h | 18.613 : 18.253 | 19.234 : 18.252 | 18.616 : 18.253 | 20.177 : 9.140 | mixed-16h | 18.590 : 18.253 | 19.221 : 18.252 | 19.654 : 18.253 | 17.273 : 9.135 | mixed-32h | 18.768 : 18.253 | 19.122 : 18.252 | 18.535 : 18.252 | 15.791 : 9.132 | ext3 | v3 | v4.0 | v4.1 | v4.2 | ----------|-----------------|-----------------|-----------------|-----------------| data | 34.292 : 18.253 | 33.810 : 18.252 | 33.450 : 18.253 | 33.390 : 18.254 | hole | 18.256 : 18.253 | 18.255 : 18.252 | 18.256 : 18.253 | 0.718 : 0.728 | mixed-4d | 46.818 : 18.253 | 47.140 : 18.252 | 48.385 : 18.253 | 42.887 : 9.150 | mixed-8d | 58.554 : 18.253 | 59.277 : 18.252 | 59.673 : 18.253 | 56.760 : 9.140 | mixed-16d | 44.631 : 18.253 | 44.291 : 18.252 | 44.729 : 18.253 | 40.237 : 9.135 | mixed-32d | 39.110 : 18.253 | 38.735 : 18.252 | 38.902 : 18.252 | 35.270 : 9.132 | mixed-4h | 56.396 : 18.253 | 56.387 : 18.252 | 56.573 : 18.253 | 67.661 : 9.150 | mixed-8h | 58.483 : 18.253 | 58.484 : 18.252 | 59.099 : 18.253 | 77.958 : 9.140 | mixed-16h | 42.511 : 18.253 | 42.338 : 18.252 | 42.356 : 18.252 | 51.805 : 9.135 | mixed-32h | 38.419 : 18.253 | 38.504 : 18.252 | 38.643 : 18.252 | 40.411 : 9.132 | Changes since v1: - Rebase to 5.6-rc1 - Drop the mount option patch for now - Fix fallback to READ when the server doesn't support READ_PLUS Any questions? Anna Anna Schumaker (6): SUNRPC: Split out a function for setting current page SUNRPC: Add the ability to expand holes in data pages SUNRPC: Add the ability to shift data to a specific offset NFS: Add READ_PLUS data segment support NFS: Add READ_PLUS hole segment decoding NFS: Decode multiple READ_PLUS segments fs/nfs/nfs42xdr.c | 169 +++++++++++++++++++++++++ fs/nfs/nfs4proc.c | 43 ++++++- fs/nfs/nfs4xdr.c | 1 + include/linux/nfs4.h | 2 +- include/linux/nfs_fs_sb.h | 1 + include/linux/nfs_xdr.h | 2 +- include/linux/sunrpc/xdr.h | 2 + net/sunrpc/xdr.c | 244 ++++++++++++++++++++++++++++++++++++- 8 files changed, 457 insertions(+), 7 deletions(-)