From patchwork Wed May 25 14:51:34 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 9135569 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 01C8A607D7 for ; Wed, 25 May 2016 14:52:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E624C28164 for ; Wed, 25 May 2016 14:52:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA957282D8; Wed, 25 May 2016 14:52:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EACC28164 for ; Wed, 25 May 2016 14:52:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754731AbcEYOwB (ORCPT ); Wed, 25 May 2016 10:52:01 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:52357 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754875AbcEYOv5 (ORCPT ); Wed, 25 May 2016 10:51:57 -0400 Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u4PEpda1014667 for ; Wed, 25 May 2016 07:51:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : subject : date : message-id : mime-version : content-type; s=facebook; bh=MpGIWCyNnUDcSRvTHY0fGXjxam/TX5JKZb6cepx/2MA=; b=Fb/zfi1p055o9Jr6iv4d8eHjCEhsmigMyNC1h3uj1KDNx5owJFg3LfNSQ6haldarlfCw fxqDCEgrE0ANRd+3kAGne2GRgx9Ffm0mhWz9KeXBFTr7el6nSu0jjhOYWlEkQ2Szhnsa qPm+/zZhIEj9aKNvW2uQkDVZrutUh0YJ5Zc= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0b-00082601.pphosted.com with ESMTP id 234j8qyw5d-2 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 25 May 2016 07:51:55 -0700 Received: from na01-bl2-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.12) with Microsoft SMTP Server (TLS) id 14.3.294.0; Wed, 25 May 2016 07:51:42 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Y18pIhGd2q0g8fu5+R/x5QmE9Vwl9US9BvnPzNJ7AyA=; b=WdUt8xpI53c8wtVA/RcCRu5kV1V1st32FmoIKNUyNPuqnqGLUCC/3JspeDbss/18X4eXE7ibWd+AW3xVZkwBsAErZOlF19WHY8Yc2mBTmObI5SCZFC4A63rPqu9KS3ytvmMuqZcaNhTlrDnMb3YX952bwd/IMcHaJWwMsiriFzQ= Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=fb.com; Received: from localhost (107.15.72.49) by BY2PR15MB0840.namprd15.prod.outlook.com (10.164.171.134) with Microsoft SMTP Server (TLS) id 15.1.497.12; Wed, 25 May 2016 14:51:39 +0000 From: Josef Bacik To: , Subject: [PATCH] Btrfs: change how we calculate the global block rsv Date: Wed, 25 May 2016 10:51:34 -0400 Message-ID: <1464187894-16902-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 X-Originating-IP: [107.15.72.49] X-ClientProxiedBy: BN3PR16CA0051.namprd16.prod.outlook.com (10.165.72.147) To BY2PR15MB0840.namprd15.prod.outlook.com (10.164.171.134) X-MS-Office365-Filtering-Correlation-Id: ec53a02d-8ace-4959-2d0f-08d384ac13dd X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0840; 2:2D1zDM2Il2iq4T+dEN5oV40e9CUMsPxuK0AarxQKZSFXpCblyY4CGuA+pkS64vs/umiwLnjE+b5jYUXEpUOfdBLsbz1muvSTEw3gpHMYVf2rqCtOIl9PR8bTruF2A2ckqlS0bHz2hhmZtiRtp/pkqcRttCdyAsX2mtdL9l+v++S0Sh4QfAyHyr12Ti6I15wu; 3:l3NIzBLqnbl4g3XlXwHQrPTSHvQDWGbio/np2yDs6QvYDDhH9taqaub42D8gV+nqm+19oQJSxg9EXw96AwCBVJdu7cPQyKIWVaY25fV4TP1xpJa+ZQAoGENZzaXtdjE3 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR15MB0840; X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0840; 25:eWp51qSfcX0rjDfUu0OyrWOEtj+rHwVUCAWcTlDnxWdlhnf78pd5SEL4CRJIwMbY0CVmxu3Fg0jWeuCWbpg/6u27vYjy7A8CNayFE77PZe5TmTsZKgDLYvpkgiky2OxxTXhYqSdzkmYbnwOf9yNdIPG0qjO3k4pDE6RgX4LTitCLftNVJcOVGz+9cRWHRB5BkIA7+NW2ErB85PoN8ALFJ2yGqpIGenYYsa2xUAIgV6X1acOzodMxrgiTzY3+Ng1dGP0B/J6p1QxJyHqn2aJxfhjVrPsMk9nLgZiNb3neWNfVesmmIekU1DjsbFmHVytj+sQhbi0Hvsd8Aa8flhif10pqT4gBq9c8buUpyAVDyS5kkHoCcNxjqOele0anXu0vmo7jIxOGOFntoEP0EUerRapEEz92ToUcCsluyDicEtgVzePv6QwIDuyDlB3TrNWl8CdaS/Z0mcdlzBiiDq+sMTLiD87Ibv8D1BOHzXq38D+yVbkYc3XdmLhIjHJCMXCblp6h9Y1ruoZXedVDDyrb5683L4d2Wm+pUaF/xdnslkr77tm1If67S291RwMwrdWxSnhivBY7V1VmH5dDePW92PBEhCzFv/ZrcQqPocsp+hJWYOrj7ds1hqmVtX3sOJmsub0aCZeTBaFE5YpJbB4c2vXajJMkruvRg9LgMuI0KFKinuloIu0dKAXCYrIHEiBQltfpuRfkWDPPsE6CWB8aiq+HPEmXyK11QT2Kkb6CEWg=; 20:XYAgG+fIbWrO1iSrAQuEWCkKP6NpxEZJ2ZOkHVjjhfSN2U3Az25WbiaAeqGcDYirtKY03cs0+zVQeOImNEBTPNAEdM6owtxTJzyBLyv0VJvxjxQdrlotPetymKbQ3bG3hO6n1eUxyg2x0ssrKL0wi9o/4sRLiRlf9diltbZQMT0= X-LD-Processed: 8ae927fe-1255-47a7-a2af-5f3a069daaa2,ExtAddr X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:BY2PR15MB0840; BCL:0; PCL:0; RULEID:; SRVR:BY2PR15MB0840; X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0840; 4:r+3sFzV1HVI3z8W3SpZ+2UQozDwzJAaTJpUcS+xPIYif5UZz5RIwQyrUgVmT3yvXqddskkaVC9aou+vxQq93TmptZYOgXorKWnT2HDNf+auGkMEMpH1legHZzIp8ne4PdEX5Oa9NpsocRGX1yqlVIoCvDBLISPO6Sy5XUO9L0r1czWqBpH42t45qd9w6eyPGGfZIIDb9sBfJNcqroeO7YSxx2Q6yX45kB12SKWUA8Xxvv+ll8fxGB0sggQAHwsfAwP3MXLIh78CU4W6/Bi6FP8CTE2RGr1mWiIghsqtBHMrNcRiJuXOuKoB2pQ+8YX7LPz/18Gofw7JE18dFeLItV+tCpQQ1og0fqiVcMchZir0KUv0MUZp6gPzWxBZUErp2 X-Forefront-PRVS: 09538D3531 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(6069001)(19580395003)(33646002)(5001770100001)(19580405001)(107886002)(551934003)(36756003)(50466002)(66066001)(48376002)(50226002)(47776003)(450100001)(189998001)(3846002)(5008740100001)(76506005)(42186005)(586003)(5003940100001)(81166006)(5004730100002)(77096005)(8676002)(229853001)(50986999)(6116002)(92566002)(2906002)(86362001)(4720700001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY2PR15MB0840; H:localhost; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0840; 23:dJEZeCKeDHAoGfBaeE7rj6kH/pplXkTzHTK4Yj6e5S07cA5/BZQGiM+MREwGR6YGIutA+wrYTg8vEqd8r9eqQrhmjaWG2NTUc66NliIGNhC4Gt/CWGNDo6GhYztq3735htSYLKBk/zGhrvLn5kNKSOGkIVmknr3Q5dy1gI1hQxek+Za7CBSKs67TJfKB5ggGWzkSemAbXm/ixiPBhQjfqCEDwQO212RjcAp9SHcP5iFiBet0GyJNSxCWScgi+66bHax0AGERiGlUlQ5S87Ra1XI4InVycYRNu1oLTKK0VCPBiB0mwi6wXgX5EaTi2yfF5R8yX37pL3apF0lK/7NDko8LbGKHQrWlxxP9umNAWkJudky1rOr591YftiNFVLynELQj6PlyQskX7X/z35OI6cQsT8BNATKX1NA9sZSQE3Wonmnly9/Q/g/eog4YYCbWxOnKg16lD4a0Zyt2i/+XNhDYc0tlH3ptwg/pBX7TiVuVavp0iLYQsVUQOF3l3yotnl+bDJF4he0ktRZvuEoDF6VVcnLslelyW+3iGwhXashodnSZPEzHf0uyYRZK+quSFnVRKladDA5jQ5CRnGwcrQTu1AHzP8gdSzWlgfZGXEqso4Vv9bR7EfZeMq3X24qTZyEkCMwa6YvOTHRXtCxObP7pUkVX6XFMhyYo/brR/d9o9heHsLJFJGum37tW7fUenfRgyjQhXegohvtBlrkemE4FZY9dqALyFl9OmEWdHcWL5bFFbyRNw9y3PNc8FEHmTkfV1TOV6droWB2drWALvdjaOJ92o2p15cwHdLgPhI7ii5PNlItXufhQwIuJmH6QDB/+LIuucJZUIOhhzCf1yzmwXpOiuDqWzNS6pKxvas+HfDyYL+qCzEDhkom/WVIrV99m8yvStlgcM9WP7EOo+3jvjDWDkQSJ8aXq9LfdR XD26vEvyGTmqHpQDH4FbnaX X-Microsoft-Exchange-Diagnostics: 1; BY2PR15MB0840; 5:6UmoNIAFzouo7tGSyoDrgrdKBp6XJk+Mk3jky+p54YpGIvczq1CM7KTMdjQQYBygeff1XrMEamNL5EtvcsQAeXi9heKYRIe9mrEDM0Blpp2046IPUeGEO2dlKcb6ZfZJroYdMP7irJLJyrOqV/N4TQ==; 24:2/AWoN+Sx7geKbpg0PuKO8G8DSZfkpskJ1aoj4LWCVTPdmPcEKp/Z5n+WXB02oWVqajfb0hGhzWuezdVHQ42wokt1Pl/Hd8lgGN+uRnxysU=; 7:K5v1/xoDa9CgU/rwEmrOvKR9AFa96KT+jdAIiihs0u7FoNZlCS4dOTmyxsm2kdSWDpajuHXZenvyWikipFSfsInPy0T05hBvsnsXD97vwjPmUEQGVir5IW0EA06fYuJFne6SgruF7e6Td5zAm541p2EOKk22GzF+nTn2RaImIQ7hOLxV2CRpulDbx+/oxeF6; 20:R5brVlwKcRFi0Pu825N47OjOvdaUexWtZ5q/9aehncyWR60cEUKgYGzm7uLzj5kDK6hnWa+u3D+zIcEiliCSMv3HFKdxpj/zbCoX6DwMCykySr4y+fbpJFD4bL5DKBJyLXPH00Cpa4Ojh+EnVu25SHKLwhR8BncCxiqEFRlNngc= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2016 14:51:39.0000 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR15MB0840 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-05-25_09:, , signatures=0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Traditionally we've calculated the global block rsv by guessing how much of the metadata used amount was the extent tree, and then taking the data size and figuring out how large the csum tree would have to be to hold that much data. This is imprecise and falls down on MIXED file systems as we can't trust the data used amount. This resulted in failures for xfstests generic/333 because it creates lots of clones, which explodes out the extent tree. Our global reserve calculations were woefully inaccurate in this case which meant we got into a situation where we did not have enough reserved to do our work. We know we only use the global block rsv for the extent, csum, and root trees, so just get the bytes used for these trees and use that as the basis of our global reserve. Since these are not reference counted trees the bytes_used value will be accurate. This fixed the transaction aborts seen with generic/333. Thanks, Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 43 +++++++------------------------------------ 1 file changed, 7 insertions(+), 36 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 1213a59..fd11cd0 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5550,48 +5550,19 @@ void btrfs_block_rsv_release(struct btrfs_root *root, num_bytes); } -/* - * helper to calculate size of global block reservation. - * the desired value is sum of space used by extent tree, - * checksum tree and root tree - */ -static u64 calc_global_metadata_size(struct btrfs_fs_info *fs_info) -{ - struct btrfs_space_info *sinfo; - u64 num_bytes; - u64 meta_used; - u64 data_used; - int csum_size = btrfs_super_csum_size(fs_info->super_copy); - - sinfo = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_DATA); - spin_lock(&sinfo->lock); - data_used = sinfo->bytes_used; - spin_unlock(&sinfo->lock); - - sinfo = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA); - spin_lock(&sinfo->lock); - if (sinfo->flags & BTRFS_BLOCK_GROUP_DATA) - data_used = 0; - meta_used = sinfo->bytes_used; - spin_unlock(&sinfo->lock); - - num_bytes = (data_used >> fs_info->sb->s_blocksize_bits) * - csum_size * 2; - num_bytes += div_u64(data_used + meta_used, 50); - - if (num_bytes * 3 > meta_used) - num_bytes = div_u64(meta_used, 3); - - return ALIGN(num_bytes, fs_info->extent_root->nodesize << 10); -} - static void update_global_block_rsv(struct btrfs_fs_info *fs_info) { struct btrfs_block_rsv *block_rsv = &fs_info->global_block_rsv; struct btrfs_space_info *sinfo = block_rsv->space_info; u64 num_bytes; - num_bytes = calc_global_metadata_size(fs_info); + /* + * The global block rsv is based on the size of the extent tree, the + * checksum tree and the root tree. + */ + num_bytes = btrfs_root_used(&fs_info->extent_root->root_item) + + btrfs_root_used(&fs_info->csum_root->root_item) + + btrfs_root_used(&fs_info->tree_root->root_item); spin_lock(&sinfo->lock); spin_lock(&block_rsv->lock);