From patchwork Wed Apr 24 14:47:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 10914979 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E39F13B5 for ; Wed, 24 Apr 2019 14:50:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E9212892E for ; Wed, 24 Apr 2019 14:50:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 02DD0289FC; Wed, 24 Apr 2019 14:50:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83F7D289E5 for ; Wed, 24 Apr 2019 14:50:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731754AbfDXOuN (ORCPT ); Wed, 24 Apr 2019 10:50:13 -0400 Received: from mx2.suse.de ([195.135.220.15]:51692 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731194AbfDXOsD (ORCPT ); Wed, 24 Apr 2019 10:48:03 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 4D9FAADEA for ; Wed, 24 Apr 2019 14:48:02 +0000 (UTC) From: Johannes Thumshirn To: David Sterba Cc: Linux BTRFS Mailinglist , Johannes Thumshirn Subject: [PATCH v2 1/3] btrfs-progs: factor out super_block reading from load_and_dump_sb Date: Wed, 24 Apr 2019 16:47:52 +0200 Message-Id: <20190424144754.4612-2-jthumshirn@suse.de> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190424144754.4612-1-jthumshirn@suse.de> References: <20190424144754.4612-1-jthumshirn@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP inspect-internal dump-superblock's load_and_dump_sb() already reads a super block from a file descriptor and places it into a 'struct btrfs_super_block'. For inspect-internal dump-csum we need this super block as well but don't care about printing it. Separate the read from the dump phase so we can re-use it elsewhere. Signed-off-by: Johannes Thumshirn --- cmds-inspect-dump-super.c | 6 +++--- utils.c | 17 +++++++++++++++++ utils.h | 2 ++ 3 files changed, 22 insertions(+), 3 deletions(-) diff --git a/cmds-inspect-dump-super.c b/cmds-inspect-dump-super.c index 7815c863f2ed..516760fad3da 100644 --- a/cmds-inspect-dump-super.c +++ b/cmds-inspect-dump-super.c @@ -483,10 +483,10 @@ static int load_and_dump_sb(char *filename, int fd, u64 sb_bytenr, int full, sb = (struct btrfs_super_block *)super_block_data; - ret = pread64(fd, super_block_data, BTRFS_SUPER_INFO_SIZE, sb_bytenr); - if (ret != BTRFS_SUPER_INFO_SIZE) { + ret = load_sb(fd, sb_bytenr, sb, BTRFS_SUPER_INFO_SIZE); + if (ret) { /* check if the disk if too short for further superblock */ - if (ret == 0 && errno == 0) + if (ret == -ENOSPC) return 0; error("failed to read the superblock on %s at %llu", diff --git a/utils.c b/utils.c index 9e26c884cc6c..b15cfcf5a434 100644 --- a/utils.c +++ b/utils.c @@ -2593,3 +2593,20 @@ void print_all_devices(struct list_head *devices) print_device_info(dev, "\t"); printf("\n"); } + +int load_sb(int fd, u64 bytenr, struct btrfs_super_block *sb, size_t size) +{ + int ret; + + if (size != BTRFS_SUPER_INFO_SIZE) + return -EINVAL; + + ret = pread64(fd, sb, size, bytenr); + if (ret != size) { + if (ret == 0 && errno == 0) + return -ENOSPC; + + return -errno; + } + return 0; +} diff --git a/utils.h b/utils.h index 7c5eb798557d..65549374b19c 100644 --- a/utils.h +++ b/utils.h @@ -171,6 +171,8 @@ unsigned long total_memory(void); void print_device_info(struct btrfs_device *device, char *prefix); void print_all_devices(struct list_head *devices); +int load_sb(int fd, u64 bytenr, struct btrfs_super_block *sb, size_t size); + /* * Global program state, configurable by command line and available to * functions without extra context passing. From patchwork Wed Apr 24 14:47:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 10914975 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB7F313B5 for ; Wed, 24 Apr 2019 14:50:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AB1C128852 for ; Wed, 24 Apr 2019 14:50:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9F2C4289C0; Wed, 24 Apr 2019 14:50:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7CB3287DE for ; Wed, 24 Apr 2019 14:50:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732388AbfDXOsI (ORCPT ); Wed, 24 Apr 2019 10:48:08 -0400 Received: from mx2.suse.de ([195.135.220.15]:51698 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731184AbfDXOsE (ORCPT ); Wed, 24 Apr 2019 10:48:04 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id A4C34AE17 for ; Wed, 24 Apr 2019 14:48:02 +0000 (UTC) From: Johannes Thumshirn To: David Sterba Cc: Linux BTRFS Mailinglist , Johannes Thumshirn Subject: [PATCH v2 2/3] btrfs-progs: add 'btrfs inspect-internal csum-dump' command Date: Wed, 24 Apr 2019 16:47:53 +0200 Message-Id: <20190424144754.4612-3-jthumshirn@suse.de> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190424144754.4612-1-jthumshirn@suse.de> References: <20190424144754.4612-1-jthumshirn@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a 'btrfs inspect-internal csum-dump' command to dump the on-disk checksums of a file. The dump command first uses the FIEMAP ioctl() to get a map of the file's extents and then uses the BTRFS_TREE_SEARCH_V2 ioctl() to get the checksums for these extents. Using FIEMAP instead of the BTRFS_TREE_SEARCH_V2 ioctl() to get the extents allows us to quickly filter out any holes in the file, as this is already done for us in the kernel. Signed-off-by: Johannes Thumshirn --- Makefile | 3 +- cmds-inspect-dump-csum.c | 253 +++++++++++++++++++++++++++++++++++++++++++++++ cmds-inspect.c | 2 + commands.h | 2 + 4 files changed, 259 insertions(+), 1 deletion(-) create mode 100644 cmds-inspect-dump-csum.c diff --git a/Makefile b/Makefile index e25e256f96af..f5d0c0532faf 100644 --- a/Makefile +++ b/Makefile @@ -130,7 +130,8 @@ cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \ cmds-restore.o cmds-rescue.o chunk-recover.o super-recover.o \ cmds-property.o cmds-fi-usage.o cmds-inspect-dump-tree.o \ cmds-inspect-dump-super.o cmds-inspect-tree-stats.o cmds-fi-du.o \ - mkfs/common.o check/mode-common.o check/mode-lowmem.o + cmds-inspect-dump-csum.o mkfs/common.o check/mode-common.o \ + check/mode-lowmem.o libbtrfs_objects = send-stream.o send-utils.o kernel-lib/rbtree.o btrfs-list.o \ kernel-lib/crc32c.o messages.o \ uuid-tree.o utils-lib.o rbtree-utils.o diff --git a/cmds-inspect-dump-csum.c b/cmds-inspect-dump-csum.c new file mode 100644 index 000000000000..67e14fde6ec7 --- /dev/null +++ b/cmds-inspect-dump-csum.c @@ -0,0 +1,253 @@ +/* + * Copyright (C) 2019 SUSE. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public + * License v2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; if not, write to the + * Free Software Foundation, Inc., 59 Temple Place - Suite 330, + * Boston, MA 021110-1307, USA. + */ + +#include +#include + +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "kerncompat.h" +#include "ctree.h" +#include "messages.h" +#include "help.h" +#include "ioctl.h" +#include "utils.h" +#include "disk-io.h" + +static bool debug = false; + +static int btrfs_lookup_csum_for_phys(int fd, struct btrfs_super_block *sb, + u64 phys, u64 extent_csums) +{ + struct btrfs_ioctl_search_args_v2 *search; + struct btrfs_ioctl_search_key *sk; + int bufsz = 1024; + char buf[bufsz], *bp; + unsigned int off = 0; + const int csum_size = btrfs_super_csum_size(sb); + const int sector_size = btrfs_super_sectorsize(sb); + int ret, i, j; + u64 needle = phys; + u64 pending_csum_count = extent_csums; + + memset(buf, 0, sizeof(buf)); + search = (struct btrfs_ioctl_search_args_v2 *)buf; + sk = &search->key; + +again: + if (debug) + printf( +"Looking up checksums for extent at physial offset: %llu (searching at %llu), looking for %llu csums\n", + phys, needle, pending_csum_count); + + sk->tree_id = BTRFS_CSUM_TREE_OBJECTID; + sk->min_objectid = BTRFS_EXTENT_CSUM_OBJECTID; + sk->max_objectid = BTRFS_EXTENT_CSUM_OBJECTID; + sk->max_type = BTRFS_EXTENT_CSUM_KEY; + sk->min_type = BTRFS_EXTENT_CSUM_KEY; + sk->min_offset = needle; + sk->max_offset = (u64)-1; + sk->max_transid = (u64)-1; + sk->nr_items = 1; + search->buf_size = bufsz - sizeof(*search); + + ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, search); + if (ret < 0) + return ret; + + /* + * If we don't find the csum item at @needle go back by @sector_size and + * retry until we've found it. + */ + if (sk->nr_items == 0) { + needle -= sector_size; + goto again; + } + + + bp = (char *) search->buf; + + for (i = 0; i < sk->nr_items; i++) { + struct btrfs_ioctl_search_header *sh; + u64 csums_in_item; + + sh = (struct btrfs_ioctl_search_header *) (bp + off); + off += sizeof(*sh); + + csums_in_item = btrfs_search_header_len(sh) / csum_size; + csums_in_item = min(csums_in_item, pending_csum_count); + + for (j = 0; j < csums_in_item; j++) { + struct btrfs_csum_item *csum_item; + + csum_item = (struct btrfs_csum_item *) + (bp + off + j * csum_size); + + printf("Offset: %llu, checksum: 0x%08x\n", + phys + j * sector_size, *(u32 *)csum_item); + } + + off += btrfs_search_header_len(sh); + pending_csum_count -= csums_in_item; + + } + + return ret; +} + +static int btrfs_get_extent_csum(int fd, struct btrfs_super_block *sb) +{ + struct fiemap *fiemap, *tmp; + struct fiemap_extent *fe; + size_t ext_size; + int ret, i; + + fiemap = calloc(1, sizeof(*fiemap)); + if (!fiemap) + return -ENOMEM; + + fiemap->fm_length = ~0; + + ret = ioctl(fd, FS_IOC_FIEMAP, fiemap); + if (ret) + goto free_fiemap; + + ext_size = fiemap->fm_mapped_extents * sizeof(struct fiemap_extent); + + tmp = realloc(fiemap, sizeof(*fiemap) + ext_size); + if (!tmp) { + ret = -ENOMEM; + goto free_fiemap; + } + + fiemap = tmp; + fiemap->fm_extent_count = fiemap->fm_mapped_extents; + fiemap->fm_mapped_extents = 0; + + ret = ioctl(fd, FS_IOC_FIEMAP, fiemap); + if (ret) + goto free_fiemap; + + for (i = 0; i < fiemap->fm_mapped_extents; i++) { + u64 extent_csums; + + fe = &fiemap->fm_extents[i]; + extent_csums = fe->fe_length / btrfs_super_sectorsize(sb); + + if (debug) + printf( +"Found extent at physial offset: %llu, length %llu, looking for %llu csums\n", + fe->fe_physical, fe->fe_length, extent_csums); + + ret = btrfs_lookup_csum_for_phys(fd, sb, fe->fe_physical, + extent_csums); + if (ret) + break; + + if(fe->fe_flags & FIEMAP_EXTENT_LAST) + break; + } + + +free_fiemap: + free(fiemap); + return ret; +} + +const char * const cmd_inspect_dump_csum_usage[] = { + "btrfs inspect-internal dump-csum ", + "Get Checksums for a given file", + "-d|--debug Be more verbose", + NULL +}; + +int cmd_inspect_dump_csum(int argc, char **argv) +{ + struct btrfs_super_block *sb; + u8 super_block_data[BTRFS_SUPER_INFO_SIZE] = { 0 }; + char *filename; + char *device; + int fd; + int devfd; + int ret; + + optind = 0; + + sb = (struct btrfs_super_block *)super_block_data; + + while (1) { + static const struct option longopts[] = { + { "debug", no_argument, NULL, 'd' }, + { NULL, 0, NULL, 0 } + }; + + int opt = getopt_long(argc, argv, "d", longopts, NULL); + if (opt < 0) + break; + + switch (opt) { + case 'd': + debug = true; + break; + default: + usage(cmd_inspect_dump_csum_usage); + } + } + + if (check_argc_exact(argc - optind, 2)) + usage(cmd_inspect_dump_csum_usage); + + filename = argv[optind]; + device = argv[optind + 1]; + + fd = open(filename, O_RDONLY); + if (fd < 0) { + error("cannot open file %s:%m", filename); + return -errno; + } + + devfd = open(device, O_RDONLY); + if (devfd < 0) { + ret = -errno; + goto out_close; + } + load_sb(devfd, btrfs_sb_offset(0), sb, BTRFS_SUPER_INFO_SIZE); + close(devfd); + + if (btrfs_super_magic(sb) != BTRFS_MAGIC) { + ret = -EINVAL; + error("bad magic on superblock on %s", device); + goto out_close; + } + + ret = btrfs_get_extent_csum(fd,sb); + if (ret) + error("checsum lookup for file %s failed", filename); +out_close: + close(fd); + return ret; +} diff --git a/cmds-inspect.c b/cmds-inspect.c index efea0331b7aa..c20decbf6fac 100644 --- a/cmds-inspect.c +++ b/cmds-inspect.c @@ -654,6 +654,8 @@ const struct cmd_group inspect_cmd_group = { cmd_inspect_dump_super_usage, NULL, 0 }, { "tree-stats", cmd_inspect_tree_stats, cmd_inspect_tree_stats_usage, NULL, 0 }, + { "dump-csum", cmd_inspect_dump_csum, + cmd_inspect_dump_csum_usage, NULL, 0 }, NULL_CMD_STRUCT } }; diff --git a/commands.h b/commands.h index 76991f2b28d5..698ae532b2b8 100644 --- a/commands.h +++ b/commands.h @@ -92,6 +92,7 @@ extern const char * const cmd_rescue_usage[]; extern const char * const cmd_inspect_dump_super_usage[]; extern const char * const cmd_inspect_dump_tree_usage[]; extern const char * const cmd_inspect_tree_stats_usage[]; +extern const char * const cmd_inspect_dump_csum_usage[]; extern const char * const cmd_filesystem_du_usage[]; extern const char * const cmd_filesystem_usage_usage[]; @@ -108,6 +109,7 @@ int cmd_super_recover(int argc, char **argv); int cmd_inspect(int argc, char **argv); int cmd_inspect_dump_super(int argc, char **argv); int cmd_inspect_dump_tree(int argc, char **argv); +int cmd_inspect_dump_csum(int argc, char **argv); int cmd_inspect_tree_stats(int argc, char **argv); int cmd_property(int argc, char **argv); int cmd_send(int argc, char **argv); From patchwork Wed Apr 24 14:47:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Thumshirn X-Patchwork-Id: 10914977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10A2C1708 for ; Wed, 24 Apr 2019 14:50:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0051C289FC for ; Wed, 24 Apr 2019 14:50:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E7D342882C; Wed, 24 Apr 2019 14:50:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9131B2882C for ; Wed, 24 Apr 2019 14:50:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731041AbfDXOuJ (ORCPT ); Wed, 24 Apr 2019 10:50:09 -0400 Received: from mx2.suse.de ([195.135.220.15]:51704 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731903AbfDXOsE (ORCPT ); Wed, 24 Apr 2019 10:48:04 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id DB0B7AE32 for ; Wed, 24 Apr 2019 14:48:02 +0000 (UTC) From: Johannes Thumshirn To: David Sterba Cc: Linux BTRFS Mailinglist , Johannes Thumshirn Subject: [PATCH v2 3/3] btrfs-progs: completion: wire-up dump-csum Date: Wed, 24 Apr 2019 16:47:54 +0200 Message-Id: <20190424144754.4612-4-jthumshirn@suse.de> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190424144754.4612-1-jthumshirn@suse.de> References: <20190424144754.4612-1-jthumshirn@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add the new 'dump-csum' command to inspect-internal. Signed-off-by: Johannes Thumshirn --- btrfs-completion | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs-completion b/btrfs-completion index 6ae57d1b752b..a381036886f0 100644 --- a/btrfs-completion +++ b/btrfs-completion @@ -29,7 +29,7 @@ _btrfs() commands_device='scan add delete remove ready stats usage' commands_scrub='start cancel resume status' commands_rescue='chunk-recover super-recover zero-log' - commands_inspect_internal='inode-resolve logical-resolve subvolid-resolve rootid min-dev-size dump-tree dump-super tree-stats' + commands_inspect_internal='inode-resolve logical-resolve subvolid-resolve rootid min-dev-size dump-tree dump-super tree-stats dump-csum' commands_property='get set list' commands_quota='enable disable rescan' commands_qgroup='assign remove create destroy show limit' @@ -128,7 +128,7 @@ _btrfs() _btrfs_mnts return 0 ;; - dump-tree|dump-super|rootid|inode-resolve) + dump-tree|dump-super|rootid|inode-resolve|dump-csum) _filedir return 0 ;;