| Message ID | 20221017035231.51112-1-wangyugui@e16-tech.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | btrfs-progs: send: sync splice buf size with kernel when proto 2 | expand |
On Mon, Oct 17, 2022 at 11:52:31AM +0800, Wang Yugui wrote: > When 'btrfs send --proto 2', the max buffer in kernel is changed from > BTRFS_SEND_BUF_SIZE_V1(SZ_64K) to (SZ_16K + BTRFS_MAX_COMPRESSED). > > The performance is improved when we use the same buf size in btrfs-progs. > without this patch: 57.96s > with this patch: 48.44s > > bigger buf size(SZ_512K) is tested too. but it help 'btrfs send --proto 2' / > 'btrfs send --proto 1' just little. so we will not use bigger buffer. > > Signed-off-by: Wang Yugui <wangyugui@e16-tech.com> Added to devel, thanks. > cmds/send.c | 17 +++++++++++++++-- > 1 file changed, 15 insertions(+), 2 deletions(-) > > diff --git a/cmds/send.c b/cmds/send.c > index ec61aaf4..d18a31a1 100644 > --- a/cmds/send.c > +++ b/cmds/send.c > @@ -37,7 +37,10 @@ > #include "cmds/commands.h" > #include "ioctl.h" > > -#define SEND_BUFFER_SIZE SZ_64K > +#define BTRFS_SEND_BUF_SIZE_V1 (SZ_64K) > +#define BTRFS_MAX_COMPRESSED (SZ_128K) > +#define BTRFS_SEND_BUF_SIZE_V2 (SZ_16K + BTRFS_MAX_COMPRESSED) Such define is missing from kernel, right? It's hidden in btrfs_ioctl_send as sctx->send_max_size = ALIGN(SZ_16K + BTRFS_MAX_COMPRESSED, PAGE_SIZE); would be nice to have that as a separate define.
diff --git a/cmds/send.c b/cmds/send.c index ec61aaf4..d18a31a1 100644 --- a/cmds/send.c +++ b/cmds/send.c @@ -37,7 +37,10 @@ #include "cmds/commands.h" #include "ioctl.h" -#define SEND_BUFFER_SIZE SZ_64K +#define BTRFS_SEND_BUF_SIZE_V1 (SZ_64K) +#define BTRFS_MAX_COMPRESSED (SZ_128K) +#define BTRFS_SEND_BUF_SIZE_V2 (SZ_16K + BTRFS_MAX_COMPRESSED) + struct btrfs_send { int send_fd; @@ -201,11 +204,18 @@ static void *read_sent_data(void *arg) struct btrfs_send *sctx = (struct btrfs_send*)arg; while (1) { + size_t splice_buf_size = BTRFS_SEND_BUF_SIZE_V1; ssize_t sbytes; + if(sctx->proto > 1){ + splice_buf_size = BTRFS_SEND_BUF_SIZE_V2; + + /* try to change pipe buffer size */ + fcntl(sctx->dump_fd, F_SETPIPE_SZ, splice_buf_size); + } /* Source is a pipe, output is either file or stdout */ sbytes = splice(sctx->send_fd, NULL, sctx->dump_fd, - NULL, SEND_BUFFER_SIZE, SPLICE_F_MORE); + NULL, splice_buf_size, SPLICE_F_MORE); if (sbytes < 0) { ret = -errno; error("failed to read stream from kernel: %m"); @@ -261,6 +271,9 @@ static int do_send(struct btrfs_send *send, u64 parent_root_id, */ io_send.version = send->proto; io_send.flags |= BTRFS_SEND_FLAG_VERSION; + + fcntl(pipefd[0], F_SETPIPE_SZ, BTRFS_SEND_BUF_SIZE_V2); + fcntl(pipefd[1], F_SETPIPE_SZ, BTRFS_SEND_BUF_SIZE_V2); } if (!ret)
When 'btrfs send --proto 2', the max buffer in kernel is changed from BTRFS_SEND_BUF_SIZE_V1(SZ_64K) to (SZ_16K + BTRFS_MAX_COMPRESSED). The performance is improved when we use the same buf size in btrfs-progs. without this patch: 57.96s with this patch: 48.44s bigger buf size(SZ_512K) is tested too. but it help 'btrfs send --proto 2' / 'btrfs send --proto 1' just little. so we will not use bigger buffer. Signed-off-by: Wang Yugui <wangyugui@e16-tech.com> --- cmds/send.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)