From patchwork Mon Dec 4 07:02:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: robbieko X-Patchwork-Id: 10089731 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 368336035E for ; Mon, 4 Dec 2017 07:03:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CE6C28F83 for ; Mon, 4 Dec 2017 07:03:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 115FA29024; Mon, 4 Dec 2017 07:03:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A7FC28F83 for ; Mon, 4 Dec 2017 07:03:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752956AbdLDHDT (ORCPT ); Mon, 4 Dec 2017 02:03:19 -0500 Received: from synology.com ([59.124.61.242]:37287 "EHLO synology.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752860AbdLDHDS (ORCPT ); Mon, 4 Dec 2017 02:03:18 -0500 Received: from localhost.localdomain (unknown [10.13.20.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by synology.com (Postfix) with ESMTPSA id 09A48A8D2580; Mon, 4 Dec 2017 15:03:04 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=synology.com; s=123; t=1512370984; bh=W997lH9g/KAGBVLMD9yIWyvqu3vm9bS0wfsNux3d1G0=; h=From:To:Cc:Subject:Date; b=t+mraatRICCLRzmMXHwhw7zZ5oVrXytKWDLGgM3sfL0R+3iBeqssljKNYQU2ZMp84 b1DSqMbTYdy5NjffWc5GSbdxdG30+fjWtEjyOJyyBf3+7HlfP5tM5JWRP+om0I8lCn jdfEqq7U5p8O/vhDWNMlmsUJ6/lx4rhUQWPCEzq4= From: robbieko To: linux-btrfs@vger.kernel.org Cc: Robbie Ko Subject: [PATCH] Btrfs : send, truncate first to enhance many small files Date: Mon, 4 Dec 2017 15:02:39 +0800 Message-Id: <1512370959-17182-1-git-send-email-robbieko@synology.com> X-Mailer: git-send-email 1.9.1 X-MailScanner-ID: 09A48A8D2580.AB6F5 X-MailScanner: Found to be clean X-MailScanner-MCPCheck: MCP-Clean, MCP-Checker (score=0, required 80) X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached, score=-0.299, required 4.5, ALL_TRUSTED -1.00, BAYES_50 0.80, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, URIBL_BLOCKED 0.00) X-MailScanner-From: robbieko@synology.com Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Robbie Ko The commands generated by send contain the following step: 1. mkfile o1851-19-0 2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c 3. set_xattr alsa-driver/alsa-kernel/isa/es1688/es1688.c - name=user.xattr data_len=4 data=test 4. write alsa-driver/alsa-kernel/isa/es1688/es1688.c - offset=0, len=10458 5. truncate alsa-driver/alsa-kernel/isa/es1688/es1688.c size=10458 6. chown alsa-driver/alsa-kernel/isa/es1688/es1688.c - uid=1024, gid=100 7. chmod alsa-driver/alsa-kernel/isa/es1688/es1688.c - mode=0644 8. utimes alsa-driver/alsa-kernel/isa/es1688/es1688.c After writing file content, it will truncate file to the correct size. Btrfs truncate will flush last page if size does not align to sectorsize, and this will cause receive process to wait until flush finishes. In order to avoid waiting flushing data.This patch changes the order so that truncate command is sent before write command. Overall performance improves by 102 percent when sending 790000 small files. original: 32m45.311s patch: 16m8.387s Signed-off-by: Robbie Ko --- fs/btrfs/send.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 20d3300..7ae2347 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -5857,10 +5857,6 @@ static int finish_inode_if_needed(struct send_ctx *sctx, int at_end) goto out; } } - ret = send_truncate(sctx, sctx->cur_ino, sctx->cur_inode_gen, - sctx->cur_inode_size); - if (ret < 0) - goto out; } if (need_chown) { @@ -6044,6 +6040,15 @@ static int changed_inode(struct send_ctx *sctx, sctx->left_path->nodes[0], left_ii); } } + if (result == BTRFS_COMPARE_TREE_NEW || + result == BTRFS_COMPARE_TREE_CHANGED) { + if (S_ISREG(sctx->cur_inode_mode)) { + ret = send_truncate(sctx, sctx->cur_ino, sctx->cur_inode_gen, + sctx->cur_inode_size); + if (ret < 0) + goto out; + } + } out: return ret;