From patchwork Tue Feb 24 21:37:33 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 5875631 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id BDDA4BF440 for ; Tue, 24 Feb 2015 21:37:42 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E2407201BC for ; Tue, 24 Feb 2015 21:37:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 04C05201B9 for ; Tue, 24 Feb 2015 21:37:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752998AbbBXVhh (ORCPT ); Tue, 24 Feb 2015 16:37:37 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:6079 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750955AbbBXVhg (ORCPT ); Tue, 24 Feb 2015 16:37:36 -0500 Received: from pps.filterd (m0044010 [127.0.0.1]) by mx0a-00082601.pphosted.com (8.14.5/8.14.5) with SMTP id t1OLZWL2014845 for ; Tue, 24 Feb 2015 13:37:36 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : subject : date : message-id : mime-version : content-type; s=facebook; bh=mcwzkzl5Xoj8ExiMk3q9MfnpbC2IK9rq1t6Oagvt6dg=; b=K8EuVVNDXJ2UGma+u+pCRkU0Rafql1yyBzUGGs8BXAvIgADUiWRUslOcCL4GJNnHTBNT bEECHzWuwLJI9HQ3QgcijU5nwfCwx1MVIOtx5sOLC2ykxHQ1bPhtYMr4a3Nj8wB2S4HV HyoZNRCpqmWncREPQO+0Ls7Cx1TNc1ZvDbs= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 1srvwfrk49-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) for ; Tue, 24 Feb 2015 13:37:36 -0800 Received: from localhost (192.168.57.29) by mail.thefacebook.com (192.168.16.16) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 24 Feb 2015 13:37:35 -0800 From: Josef Bacik To: Subject: [PATCH] Btrfs: throttle transactions based on dirty bgs Date: Tue, 24 Feb 2015 16:37:33 -0500 Message-ID: <1424813853-26382-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 X-Originating-IP: [192.168.57.29] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68, 1.0.33, 0.0.0000 definitions=2015-02-24_06:2015-02-24, 2015-02-24, 1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 kscore.is_bulkscore=4.93938223655732e-13 kscore.compositescore=0 circleOfTrustscore=0 compositescore=0.996183534634169 suspectscore=1 recipient_domain_to_sender_totalscore=0 phishscore=0 bulkscore=0 kscore.is_spamscore=0 rbsscore=0.996183534634169 recipient_to_sender_totalscore=0 recipient_domain_to_sender_domain_totalscore=0 spamscore=0 recipient_to_sender_domain_totalscore=0 urlsuspectscore=0.996183534634169 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1502240188 X-FB-Internal: deliver Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When we delete large files that span many block groups we can run short on the global reserve as we use it to account for space in the csum tree as well as the extent tree. If our files span many block groups then we can get into a situation where we don't have enough room to update all the block groups and we abort the transaction. So keep track of the number of dirty block groups and use that space in seeing if we have enough global reserve. Thanks, Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 3 +++ fs/btrfs/transaction.c | 1 + fs/btrfs/transaction.h | 1 + 3 files changed, 5 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 89f1726..b2f7cf4 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2632,6 +2632,7 @@ int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, { struct btrfs_block_rsv *global_rsv; u64 num_heads = trans->transaction->delayed_refs.num_heads_ready; + u64 num_dirty_bgs = trans->transaction->num_dirty_bgs; u64 num_bytes; int ret = 0; @@ -2640,6 +2641,7 @@ int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans, if (num_heads > 1) num_bytes += (num_heads - 1) * root->nodesize; num_bytes <<= 1; + num_bytes += btrfs_calc_trans_metadata_size(root, num_dirty_bgs); global_rsv = &root->fs_info->global_block_rsv; /* @@ -5365,6 +5367,7 @@ static int update_block_group(struct btrfs_trans_handle *trans, if (list_empty(&cache->dirty_list)) { list_add_tail(&cache->dirty_list, &trans->transaction->dirty_bgs); + trans->transaction->num_dirty_bgs++; btrfs_get_block_group(cache); } spin_unlock(&trans->transaction->dirty_bgs_lock); diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index e0faf80..822a26e 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -249,6 +249,7 @@ loop: INIT_LIST_HEAD(&cur_trans->switch_commits); INIT_LIST_HEAD(&cur_trans->pending_ordered); INIT_LIST_HEAD(&cur_trans->dirty_bgs); + cur_trans->num_dirty_bgs = 0; spin_lock_init(&cur_trans->dirty_bgs_lock); list_add_tail(&cur_trans->list, &fs_info->trans_list); extent_io_tree_init(&cur_trans->dirty_pages, diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h index 3305451..8ac29f0 100644 --- a/fs/btrfs/transaction.h +++ b/fs/btrfs/transaction.h @@ -59,6 +59,7 @@ struct btrfs_transaction { struct list_head pending_ordered; struct list_head switch_commits; struct list_head dirty_bgs; + u64 num_dirty_bgs; spinlock_t dirty_bgs_lock; struct btrfs_delayed_ref_root delayed_refs; int aborted;