From patchwork Mon Sep 4 06:41:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 9936723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EC17F60237 for ; Mon, 4 Sep 2017 06:41:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DC5C72871E for ; Mon, 4 Sep 2017 06:41:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CFDA72872D; Mon, 4 Sep 2017 06:41:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_HI,RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DED8A2871E for ; Mon, 4 Sep 2017 06:41:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753237AbdIDGlW (ORCPT ); Mon, 4 Sep 2017 02:41:22 -0400 Received: from mout.gmx.net ([212.227.15.15]:49572 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751666AbdIDGlU (ORCPT ); Mon, 4 Sep 2017 02:41:20 -0400 Received: from localhost.localdomain ([45.32.39.184]) by mail.gmx.com (mrgmx002 [212.227.17.184]) with ESMTPSA (Nemesis) id 0LnwxU-1dHc7347q4-00g0GN; Mon, 04 Sep 2017 08:41:18 +0200 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Cc: dsterba@suse.cz Subject: [PATCH 2/5] btrfs-progs: mkfs: Rework rootdir option to avoid custom chunk layout Date: Mon, 4 Sep 2017 15:41:07 +0900 Message-Id: <20170904064110.25951-3-quwenruo.btrfs@gmx.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170904064110.25951-1-quwenruo.btrfs@gmx.com> References: <20170904064110.25951-1-quwenruo.btrfs@gmx.com> X-Provags-ID: V03:K0:N7Epl1A6aizQg59nqlFaqkQ2ND8Okq3dYpwNMXgkTrnKV74pmfH 8PQtSIBZZU+YdgcImS9tqSlwjMLk7YetbK9PzsO4xfsOxD27T0zd7014Ai35vkmo1NdQTZ3 Z5WsCnh98l3ksyDYxwqKf0Ql+XpoQjPz/xHyad9ALzDfK8g+8XeVOG83p+NTcUE83s2fYpG KNH8dwyntVDMG/lzx4Mcg== X-UI-Out-Filterresults: notjunk:1; V01:K0:DdiN0iERdtw=:douai5TmXYdIhBvHt7LD0x iHDM5qhX9u5WY59cLjB7iI0TA7z37SLhvRrO9raUwFX+O4Guix3zpJ8XAKUVV8QbWRs2KhcKu tSyQKzprr5jHKWJ7gqjUwC5p3B3Ui6VwTXLmk0VcPEepT2IC0oX316NERP7jgFXZbWbTOeOoh YbVsSD3Mlg0ZK8PPy3SvCpLQyJUdvBm6HNynQUnhziCTeuEiCLEVH/Uh1QdDLKYBKdxY7l7ru m1wcaK8chGs7iLXMjDnJ3HoTLaTRhzUPghJBh0HZty3JQ26rIbMwpspYuy1QSB1zHXXN8PMTd qefyrqoGzmDGDm2u1OCpwDi7Va0wTCe9dZNnpKKsk9DjU1X3TpWeifmVne7SOT1g+jXEiTIFO gTVYgSqwXiFuGsdb8wndjX93JrXr77UIuWyZKIoo7J40z/bnWj5c8bhFbtZDXWizc98OoM70O kU6213yRRdBtaUxt9H54CVeLC+sL12F9yKUC7ob8lbaw7SEf0XpYjBHd9DxbjhUaaWkZay99r cZoEs/F9KRWZiSNslQPk1kj9C+x1JT+tmrrhLaZTbAZwUIudhwX+iFD7gd18tTL/qhm6XfDgk 5nj7AP3NuMgQHpFD7sQfoCeTRP+4xt8m6PpWGk7tZonaM9mREB1WaG+FtGsZ/FpOeY9B2nZ2J rkolnCSHdE0rOuz5ktDk9Zat1KdZrYSlaBb48F5Why8YSELlcBUyeWqGNZ1mCxQ3pHUbuhpi+ Y98MkzDuywwhfVjonA9pzPmEfBop08VRAfY/80d+sRMMiW0duynGmdqZX/VW219tDfACP8oQ2 TZE7BSSdJVOFKDRiBuEMj4HTzzdcQ== Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP mkfs.btrfs --rootdir uses its own custom chunk layout. This provides the possibility to limit the filesystem to a minimal size. However this custom chunk allocation has several problems. The most obvious problem is that it will allocate chunk from device offset 0. Both kernel and normal mkfs will reserve 0~1M range for each device. This rework will remove all related custom chunk allocation and size calculation. Less code to maintain is always a good thing, especially for minor or less maintained code. So all --rootdir operation will result the same result as mkfs.btrfs + mount + cp. (Same result as mkfs.ext4 -d) Signed-off-by: Qu Wenruo --- mkfs/main.c | 218 ++++++++---------------------------------------------------- 1 file changed, 28 insertions(+), 190 deletions(-) diff --git a/mkfs/main.c b/mkfs/main.c index 84ff300b..bca836b5 100644 --- a/mkfs/main.c +++ b/mkfs/main.c @@ -990,53 +990,6 @@ fail_no_dir: goto out; } -static int create_chunks(struct btrfs_trans_handle *trans, - struct btrfs_root *root, u64 num_of_meta_chunks, - u64 size_of_data, - struct mkfs_allocation *allocation) -{ - struct btrfs_fs_info *fs_info = root->fs_info; - u64 chunk_start; - u64 chunk_size; - u64 meta_type = BTRFS_BLOCK_GROUP_METADATA; - u64 data_type = BTRFS_BLOCK_GROUP_DATA; - u64 minimum_data_chunk_size = 8 * 1024 * 1024; - u64 i; - int ret; - - for (i = 0; i < num_of_meta_chunks; i++) { - ret = btrfs_alloc_chunk(trans, fs_info, - &chunk_start, &chunk_size, meta_type); - if (ret) - return ret; - ret = btrfs_make_block_group(trans, fs_info, 0, - meta_type, BTRFS_FIRST_CHUNK_TREE_OBJECTID, - chunk_start, chunk_size); - allocation->metadata += chunk_size; - if (ret) - return ret; - set_extent_dirty(&root->fs_info->free_space_cache, - chunk_start, chunk_start + chunk_size - 1); - } - - if (size_of_data < minimum_data_chunk_size) - size_of_data = minimum_data_chunk_size; - - ret = btrfs_alloc_data_chunk(trans, fs_info, - &chunk_start, size_of_data, data_type, 0); - if (ret) - return ret; - ret = btrfs_make_block_group(trans, fs_info, 0, - data_type, BTRFS_FIRST_CHUNK_TREE_OBJECTID, - chunk_start, size_of_data); - allocation->data += size_of_data; - if (ret) - return ret; - set_extent_dirty(&root->fs_info->free_space_cache, - chunk_start, chunk_start + size_of_data - 1); - return ret; -} - static int make_image(const char *source_dir, struct btrfs_root *root) { int ret; @@ -1080,86 +1033,6 @@ out: return ret; } -/* - * This ignores symlinks with unreadable targets and subdirs that can't - * be read. It's a best-effort to give a rough estimate of the size of - * a subdir. It doesn't guarantee that prepopulating btrfs from this - * tree won't still run out of space. - */ -static u64 global_total_size; -static u64 fs_block_size; -static int ftw_add_entry_size(const char *fpath, const struct stat *st, - int type) -{ - if (type == FTW_F || type == FTW_D) - global_total_size += round_up(st->st_size, fs_block_size); - - return 0; -} - -static u64 size_sourcedir(const char *dir_name, u64 sectorsize, - u64 *num_of_meta_chunks_ret, u64 *size_of_data_ret) -{ - u64 dir_size = 0; - u64 total_size = 0; - int ret; - u64 default_chunk_size = 8 * 1024 * 1024; /* 8MB */ - u64 allocated_meta_size = 8 * 1024 * 1024; /* 8MB */ - u64 allocated_total_size = 20 * 1024 * 1024; /* 20MB */ - u64 num_of_meta_chunks = 0; - u64 num_of_data_chunks = 0; - u64 num_of_allocated_meta_chunks = - allocated_meta_size / default_chunk_size; - - global_total_size = 0; - fs_block_size = sectorsize; - ret = ftw(dir_name, ftw_add_entry_size, 10); - dir_size = global_total_size; - if (ret < 0) { - error("ftw subdir walk of %s failed: %s", dir_name, - strerror(errno)); - exit(1); - } - - num_of_data_chunks = (dir_size + default_chunk_size - 1) / - default_chunk_size; - - num_of_meta_chunks = (dir_size / 2) / default_chunk_size; - if (((dir_size / 2) % default_chunk_size) != 0) - num_of_meta_chunks++; - if (num_of_meta_chunks <= num_of_allocated_meta_chunks) - num_of_meta_chunks = 0; - else - num_of_meta_chunks -= num_of_allocated_meta_chunks; - - total_size = allocated_total_size + - (num_of_data_chunks * default_chunk_size) + - (num_of_meta_chunks * default_chunk_size); - - *num_of_meta_chunks_ret = num_of_meta_chunks; - *size_of_data_ret = num_of_data_chunks * default_chunk_size; - return total_size; -} - -static int zero_output_file(int out_fd, u64 size) -{ - int loop_num; - u64 location = 0; - char buf[4096]; - int ret = 0, i; - ssize_t written; - - memset(buf, 0, 4096); - loop_num = size / 4096; - for (i = 0; i < loop_num; i++) { - written = pwrite64(out_fd, buf, 4096, location); - if (written != 4096) - ret = -EIO; - location += 4096; - } - return ret; -} - static int is_ssd(const char *file) { blkid_probe probe; @@ -1429,9 +1302,6 @@ int main(int argc, char **argv) int force_overwrite = 0; char *source_dir = NULL; int source_dir_set = 0; - u64 num_of_meta_chunks = 0; - u64 size_of_data = 0; - u64 source_dir_size = 0; int dev_cnt = 0; int saved_optind; char fs_uuid[BTRFS_UUID_UNPARSED_SIZE] = { 0 }; @@ -1682,44 +1552,27 @@ int main(int argc, char **argv) * (btrfs_register_one_device() fails if O_EXCL is on) */ fd = open(file, O_RDWR); - if (fd < 0) { - error("unable to open %s: %s", file, strerror(errno)); - goto error; - } - ret = btrfs_prepare_device(fd, file, &dev_block_count, - block_count, - (zero_end ? PREP_DEVICE_ZERO_END : 0) | - (discard ? PREP_DEVICE_DISCARD : 0) | - (verbose ? PREP_DEVICE_VERBOSE : 0)); - if (ret) { - goto error; - } - if (block_count && block_count > dev_block_count) { - error("%s is smaller than requested size, expected %llu, found %llu", - file, - (unsigned long long)block_count, - (unsigned long long)dev_block_count); - goto error; - } } else { fd = open(file, O_CREAT | O_RDWR, S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH); - if (fd < 0) { - error("unable to open %s: %s", file, strerror(errno)); - goto error; - } + } + if (fd < 0) { + error("unable to open %s: %s", file, strerror(errno)); + goto error; + } + ret = btrfs_prepare_device(fd, file, &dev_block_count, + block_count, + (zero_end ? PREP_DEVICE_ZERO_END : 0) | + (discard ? PREP_DEVICE_DISCARD : 0) | + (verbose ? PREP_DEVICE_VERBOSE : 0)); + if (ret) + goto error; - source_dir_size = size_sourcedir(source_dir, sectorsize, - &num_of_meta_chunks, &size_of_data); - if(block_count < source_dir_size) - block_count = source_dir_size; - ret = zero_output_file(fd, block_count); - if (ret) { - error("unable to zero the output file"); - goto error; - } - /* our "device" is the new image file */ - dev_block_count = block_count; + if (block_count && block_count > dev_block_count) { + error("%s is smaller than requested size, expected %llu, found %llu", + file, (unsigned long long)block_count, + (unsigned long long)dev_block_count); + goto error; } /* To create the first block group and chunk 0 in make_btrfs */ @@ -1845,13 +1698,11 @@ int main(int argc, char **argv) } raid_groups: - if (!source_dir_set) { - ret = create_raid_groups(trans, root, data_profile, + ret = create_raid_groups(trans, root, data_profile, metadata_profile, mixed, &allocation); - if (ret) { - error("unable to create raid groups: %d", ret); - goto out; - } + if (ret) { + error("unable to create raid groups: %d", ret); + goto out; } ret = create_data_reloc_tree(trans, root); @@ -1866,33 +1717,20 @@ raid_groups: goto out; } - if (source_dir_set) { - trans = btrfs_start_transaction(root, 1); - ret = create_chunks(trans, root, - num_of_meta_chunks, size_of_data, - &allocation); - if (ret) { - error("unable to create chunks: %d", ret); - goto out; - } - ret = btrfs_commit_transaction(trans, root); - if (ret) { - error("transaction commit failed: %d", ret); - goto out; - } + ret = cleanup_temp_chunks(fs_info, &allocation, data_profile, + metadata_profile, metadata_profile); + if (ret < 0) { + error("failed to cleanup temporary chunks: %d", ret); + goto out; + } + if (source_dir_set) { ret = make_image(source_dir, root); if (ret) { error("error wihle filling filesystem: %d", ret); goto out; } } - ret = cleanup_temp_chunks(fs_info, &allocation, data_profile, - metadata_profile, metadata_profile); - if (ret < 0) { - error("failed to cleanup temporary chunks: %d", ret); - goto out; - } if (verbose) { char features_buf[64];