From patchwork Mon Feb 9 20:03:11 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 5803151 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 58C4E9F336 for ; Mon, 9 Feb 2015 20:03:58 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7D11520120 for ; Mon, 9 Feb 2015 20:03:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 98A9D2011E for ; Mon, 9 Feb 2015 20:03:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760866AbbBIUDs (ORCPT ); Mon, 9 Feb 2015 15:03:48 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:40615 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760826AbbBIUDl (ORCPT ); Mon, 9 Feb 2015 15:03:41 -0500 Received: from pps.filterd (m0044010 [127.0.0.1]) by mx0a-00082601.pphosted.com (8.14.5/8.14.5) with SMTP id t19JsvKZ031695; Mon, 9 Feb 2015 12:03:39 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wit.ai; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=mx2; bh=QoodmvFohMX+6viG2FPzlBoMzLUGxVPVOhp/5JM7Ihc=; b=kxppI9H9ZlG4djavS+bBdBRoa1otyB/O22M10fltL+rSDTMJ0GYCG3RqO6bBQwB/S73A IEKfKk2nVEa2OVpVbj3ujqL9R9xL2WEUm8T7c0fNKujP6t6kRGVvh1Jeu9lmqyvDfKA/ Qw4gdqcmGtELF4kvWr+Qzf/XsDathgxFv3GaqOndaKYW1Vpg0V3xPiY7PmW8v80sbGAp 01cesiql+hnkebZhSQ8uLL+yxNS2lHhUX59I+5lUqWvbU2UWGbcs7md2dn9gFZhoVHrh kFNT4ifEWiY+SPxs3pGtwnv4GQ7ewqTVo2q3L+RcB5LliujJJ69jf6J7Nb8lSzGzh+6S Pw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=QoodmvFohMX+6viG2FPzlBoMzLUGxVPVOhp/5JM7Ihc=; b=K5vzb3nWZmPWa34pL5F51Z1aU/7r2GPLplQvEfFqZS5u1bSYOhCuD2rW72gp/vpMbVaT rLfrPe18K+8t0AoTWbBSlCYnNSv+ozEAUWr46JeHZ2ExQ5nSOBj3RQ7jT9Az9djbxfLA 1PZUwZXUlQczAyKo/Jkyth+H/NLxCyrD0SM= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 1sf0648fux-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Mon, 09 Feb 2015 12:03:39 -0800 Received: from localhost (192.168.57.29) by mail.thefacebook.com (192.168.16.12) with Microsoft SMTP Server (TLS) id 14.3.195.1; Mon, 9 Feb 2015 12:03:37 -0800 From: Josef Bacik To: , Subject: [PATCH 08/16] Btrfs-progs: multi-thread btrfs-image restore Date: Mon, 9 Feb 2015 15:03:11 -0500 Message-ID: <1423512199-16552-9-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1423512199-16552-1-git-send-email-jbacik@fb.com> References: <1423512199-16552-1-git-send-email-jbacik@fb.com> MIME-Version: 1.0 X-Originating-IP: [192.168.57.29] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68, 1.0.33, 0.0.0000 definitions=2015-02-09_02:2015-02-09, 2015-02-08, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 kscore.is_bulkscore=0 kscore.compositescore=0 circleOfTrustscore=2.00421954885187 compositescore=0.980601336099369 urlsuspect_oldscore=0.980601336099369 suspectscore=0 recipient_domain_to_sender_totalscore=0 phishscore=0 bulkscore=0 kscore.is_spamscore=0 recipient_to_sender_totalscore=0 recipient_domain_to_sender_domain_totalscore=62764 rbsscore=0.980601336099369 spamscore=0 recipient_to_sender_domain_totalscore=0 urlsuspectscore=0.9 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1502090196 X-FB-Internal: deliver Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For some reason we only allow btrfs-image restore to have one thread, which is incredibly slow with large images. So allow us to do work with more than just one thread. This made my restore go from 16 minutes to 3 minutes. Thanks, Signed-off-by: Josef Bacik --- btrfs-image.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/btrfs-image.c b/btrfs-image.c index aaff26d..ea85542 100644 --- a/btrfs-image.c +++ b/btrfs-image.c @@ -1922,7 +1922,6 @@ static int add_cluster(struct meta_cluster *cluster, u32 i, nritems; int ret; - BUG_ON(mdres->num_items); mdres->compress_method = header->compress; bytenr = le64_to_cpu(header->bytenr) + BLOCK_SIZE; @@ -2433,7 +2432,7 @@ static int __restore_metadump(const char *input, FILE *out, int old_restore, goto out; } - while (1) { + while (!mdrestore.error) { ret = fread(cluster, BLOCK_SIZE, 1, in); if (!ret) break; @@ -2450,14 +2449,8 @@ static int __restore_metadump(const char *input, FILE *out, int old_restore, fprintf(stderr, "Error adding cluster\n"); break; } - - ret = wait_for_worker(&mdrestore); - if (ret) { - fprintf(stderr, "One of the threads errored out %d\n", - ret); - break; - } } + ret = wait_for_worker(&mdrestore); out: mdrestore_destroy(&mdrestore, num_threads); failed_cluster: @@ -2598,7 +2591,7 @@ int main(int argc, char *argv[]) { char *source; char *target; - u64 num_threads = 0; + u64 num_threads = 1; u64 compress_level = 0; int create = 1; int old_restore = 0; @@ -2689,7 +2682,7 @@ int main(int argc, char *argv[]) } } - if (num_threads == 0 && compress_level > 0) { + if (num_threads == 1 && compress_level > 0) { num_threads = sysconf(_SC_NPROCESSORS_ONLN); if (num_threads <= 0) num_threads = 1; @@ -2708,7 +2701,7 @@ int main(int argc, char *argv[]) ret = create_metadump(source, out, num_threads, compress_level, sanitize, walk_trees); } else { - ret = restore_metadump(source, out, old_restore, 1, + ret = restore_metadump(source, out, old_restore, num_threads, multi_devices); } if (ret) {