| Message ID | alpine.LRH.2.11.1603150104370.19675@mail.ewheeler.net (mailing list archive) |
|---|---|
| State | Rejected, archived |
| Headers | show |
On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote: > Hi Joe, > > Please review the patch below when you have a moment. I am interested in > your feedback, and also interested in having this functionality merged > upstream. This was written against thin-provisioning-tools.git tag > v0.5.6. > > We use thin_dump on live dm-thin metadata snapshots all the time. In our > case, we want to dump the XML for new (snapshot) volumes instead of > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 > minutes instead of ~5 seconds for a single volume with --device-id. I'm interested in extracting the mappings of a particular device, too. In fact, I've implemented this (along with extracting the mappings in binary format) and have only recently opened a PR: https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems that we've replicated some work here, I'm not sure whether we're supposed to send patches to this list or open PR on github. -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Tue, 15 Mar 2016, Thanos Makatos wrote: > On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote: > > Hi Joe, > > > > Please review the patch below when you have a moment. I am interested in > > your feedback, and also interested in having this functionality merged > > upstream. This was written against thin-provisioning-tools.git tag > > v0.5.6. > > > > We use thin_dump on live dm-thin metadata snapshots all the time. In our > > case, we want to dump the XML for new (snapshot) volumes instead of > > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 > > minutes instead of ~5 seconds for a single volume with --device-id. > > I'm interested in extracting the mappings of a particular device, too. > In fact, I've implemented this (along with extracting the mappings in > binary format) and have only recently opened a PR: > https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems > that we've replicated some work here, I'm not sure whether we're > supposed to send patches to this list or open PR on github. Interesting, it is neat to see a binary format too. I think we need to come up with a consistent way to extend attributes being passed down into mapping_tree_emitter::visit() by way of thin_provisioning::metadata_dump(). What is preferred? I noticed that caching::metadata_dump() has the same prototype, so do those need to remain compatable calls? Should we countinue to extend the arguments to metadata_dump() as Thanos and I have done, or, should we pop up the mte call to the caller and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let the caller configure the mte with setter functions or constructor calls? I'm open to whatever---just making sure whatever we do stays clean and scales for the future. -- Eric Wheeler -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
2016-03-16 1:51 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>: > On Tue, 15 Mar 2016, Thanos Makatos wrote: > >> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote: >> > Hi Joe, >> > >> > We use thin_dump on live dm-thin metadata snapshots all the time. In our >> > case, we want to dump the XML for new (snapshot) volumes instead of >> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 >> > minutes instead of ~5 seconds for a single volume with --device-id. >> >> I'm interested in extracting the mappings of a particular device, too. >> In fact, I've implemented this (along with extracting the mappings in >> binary format) and have only recently opened a PR: >> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems >> that we've replicated some work here, I'm not sure whether we're >> supposed to send patches to this list or open PR on github. > > Interesting, it is neat to see a binary format too. > > I think we need to come up with a consistent way to extend attributes > being passed down into mapping_tree_emitter::visit() by way of > thin_provisioning::metadata_dump(). What is preferred? > > I noticed that caching::metadata_dump() has the same prototype, so do > those need to remain compatable calls? > > Should we countinue to extend the arguments to metadata_dump() > as Thanos and I have done, or, should we pop up the mte call to the caller > and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let > the caller configure the mte with setter functions or constructor calls? Hi, You are not alone, I did the same feature and used it for a long time. My solution is similar to Thanos, I override thin_provisioning::metadata_dump() to accept the additional dev_id: void metadata_dump(metadata::ptr md, emitter::ptr e); void metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id); Instead of passing dev_id to mapping_tree_emitter, I pass the dd_map with only one device, then let the mapping_tree_emitter to traverse the bottom-level tree directly: void thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id) { uint64_t key[1] = {dev_id}; dev_tree::maybe_value single_mapping_tree_root = md->mappings_top_level_->lookup(key); device_tree::maybe_value details = md->details_->lookup(key); // ... error handling details_extractor de; de.visit(dev_id, *details); // de contains only one device e->begin_superblock(...); { mapping_tree_emitter mte(md, e, de.get_details(), mapping_damage_policy(repair)); std::vector<uint64_t> path(1, dev_id); // btree_detail::btree_path is std::vector<uint64_t> mte.visit(path, *single_mapping_tree_root); } e->end_superblock(); } But this looks still a bit messy: I should setup the visitors and trees carefully to do what I want. I also wrote some debugging feature for thin_dump. I can push the code if you have interest. 1. thin_dump -b <metadata_dev> : Dump the device_details tree only. Do not emit the data mappings. (This could be replace by the new thin_ls) output (no mappings): <superblock ... > <device ... > </device> <device ... > </device> </superblock> 2. thin_dump -r2 <metadata_dev>: dump the data mapping tree despite the device_details tree was broken Ming-Hung Tsai -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Wed, 16 Mar 2016, Ming-Hung Tsai wrote: > 2016-03-16 1:51 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>: > > On Tue, 15 Mar 2016, Thanos Makatos wrote: > > > >> On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote: > >> > Hi Joe, > >> > > >> > We use thin_dump on live dm-thin metadata snapshots all the time. In our > >> > case, we want to dump the XML for new (snapshot) volumes instead of > >> > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 > >> > minutes instead of ~5 seconds for a single volume with --device-id. > >> > >> I'm interested in extracting the mappings of a particular device, too. > >> In fact, I've implemented this (along with extracting the mappings in > >> binary format) and have only recently opened a PR: > >> https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems > >> that we've replicated some work here, I'm not sure whether we're > >> supposed to send patches to this list or open PR on github. > > > > Interesting, it is neat to see a binary format too. > > > > I think we need to come up with a consistent way to extend attributes > > being passed down into mapping_tree_emitter::visit() by way of > > thin_provisioning::metadata_dump(). What is preferred? > > > > I noticed that caching::metadata_dump() has the same prototype, so do > > those need to remain compatable calls? > > > > Should we countinue to extend the arguments to metadata_dump() > > as Thanos and I have done, or, should we pop up the mte call to the caller > > and pass a pointer to thin_provisioning::metadata_dump(..., *mte) and let > > the caller configure the mte with setter functions or constructor calls? > > Hi, > > You are not alone, I did the same feature and used it for a long time. > My solution is similar to Thanos, I override thin_provisioning::metadata_dump() > to accept the additional dev_id: > > void metadata_dump(metadata::ptr md, emitter::ptr e); > void metadata_dump(metadata::ptr md, emitter::ptr e, uint64_t dev_id); That makes for 3 different implementations, we've all modified metadata_dump: original: ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair); Ming-Hung: void metadata_dump(metadata::ptr md, emitter::ptr e, [bool repair?] + uint64_t dev_id); Thanos: ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, + const block_address *dev_id) Me: ::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, + bool skip_mappings, + block_address dev_id) -- Eric Wheeler > > Instead of passing dev_id to mapping_tree_emitter, I pass the dd_map > with only one device, > then let the mapping_tree_emitter to traverse the bottom-level tree directly: > > void > thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, > uint64_t dev_id) > { > uint64_t key[1] = {dev_id}; > dev_tree::maybe_value single_mapping_tree_root = > md->mappings_top_level_->lookup(key); > device_tree::maybe_value details = md->details_->lookup(key); > > // ... error handling > > details_extractor de; > de.visit(dev_id, *details); // de contains only one device > > e->begin_superblock(...); > { > mapping_tree_emitter mte(md, e, de.get_details(), > mapping_damage_policy(repair)); > std::vector<uint64_t> path(1, dev_id); // > btree_detail::btree_path is std::vector<uint64_t> > mte.visit(path, *single_mapping_tree_root); > } > e->end_superblock(); > } > > But this looks still a bit messy: I should setup the visitors and > trees carefully to do what I want. > > I also wrote some debugging feature for thin_dump. I can push the code > if you have interest. > > 1. thin_dump -b <metadata_dev> : Dump the device_details tree only. Do > not emit the data mappings. > (This could be replace by the new thin_ls) > > output (no mappings): > <superblock ... > > <device ... > > </device> > <device ... > > </device> > </superblock> > > 2. thin_dump -r2 <metadata_dev>: dump the data mapping tree despite > the device_details tree was broken > > > > Ming-Hung Tsai > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel > -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Tue, Mar 15, 2016 at 10:59:15AM +0000, Thanos Makatos wrote: > On 15 March 2016 at 01:45, Eric Wheeler <dm-devel@lists.ewheeler.net> wrote: > > Hi Joe, > > > > Please review the patch below when you have a moment. I am interested in > > your feedback, and also interested in having this functionality merged > > upstream. This was written against thin-provisioning-tools.git tag > > v0.5.6. > > > > We use thin_dump on live dm-thin metadata snapshots all the time. In our > > case, we want to dump the XML for new (snapshot) volumes instead of > > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 > > minutes instead of ~5 seconds for a single volume with --device-id. > > I'm interested in extracting the mappings of a particular device, too. > In fact, I've implemented this (along with extracting the mappings in > binary format) and have only recently opened a PR: > https://github.com/jthornber/thin-provisioning-tools/pull/49. It seems > that we've replicated some work here, I'm not sure whether we're > supposed to send patches to this list or open PR on github. I've not forgotten you patches, just haven't got round to them yet. -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
If you're skipping the mappings does the new thin_ls provide enough information for you? - Joe On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote: > Hi Joe, > > Please review the patch below when you have a moment. I am interested in > your feedback, and also interested in having this functionality merged > upstream. This was written against thin-provisioning-tools.git tag > v0.5.6. > > We use thin_dump on live dm-thin metadata snapshots all the time. In our > case, we want to dump the XML for new (snapshot) volumes instead of > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 > minutes instead of ~5 seconds for a single volume with --device-id. > > I started by adding --device-id to pass the `block_address dev_id` and > skip the call to emit_mappings() within mapping_tree_emitter::visit() > unless the dev_id matches --device-id as passed to thin_dump. > > It is sometimes nice to list all of the device supers without waiting for > emit_mappings() so I added the --skip-mappings option. This allows you to > get the dev_id without reading the mappings: > thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta > > Like `bool repair`, I added skip_mappings and dev_id as arguments to > thin_provisioning::metadata_dump() and passed them down to > mapping_tree_emitter with new members set_skip_mappings() and set_dev_id() > to set private attributes (default values are assigned in > metadata_dumper.h in case of backward compatible calls). > > We work with the device metadata in a simplified format, and without > superblock information: > > origin_offset:length:tdata_offset (hence, "o:L:d" or "old") > > Therefore, the output --format 'old' was added. I added the format 'null' > as well for benchmarking purposes so the ostream needn't write large > volumes of text to /dev/null. > > Benchmarks of the various formats on our sample metadata snapshot on an > idle machine: > > for i in xml human_readable old null; do > time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null > done > > real user sys > xml 29:01.27 25:35.87 01:54.49 > human 27:39.81 24:12.90 01:52.39 > old 27:07.71 23:40.64 01:52.97 > null 23:39.38 21:17.22 00:49.04 > > I have this as a branch in bitbucket if you prefer to see it online: > > https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id > or > git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id > > > -- > Eric Wheeler > > > diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc > index 191acb5..7df5d85 100644 > --- a/thin-provisioning/thin_dump.cc > +++ b/thin-provisioning/thin_dump.cc > @@ -22,6 +22,8 @@ > #include <libgen.h> > > #include "human_readable_format.h" > +#include "old_format.h" > +#include "null_format.h" > #include "metadata_dumper.h" > #include "metadata.h" > #include "xml_format.h" > @@ -36,6 +38,8 @@ using namespace thin_provisioning; > struct flags { > bool find_metadata_snap; > bool repair; > + block_address dev_id; > + bool skip_mappings; > }; > > namespace { > @@ -49,12 +53,16 @@ namespace { > e = create_xml_emitter(out); > else if (format == "human_readable") > e = create_human_readable_emitter(out); > + else if (format == "old") > + e = create_old_emitter(out); > + else if (format == "null") > + e = create_null_emitter(out); > else { > cerr << "unknown format '" << format << "'" << endl; > exit(1); > } > > - metadata_dump(md, e, flags.repair); > + metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id); > > } catch (std::exception &e) { > cerr << e.what() << endl; > @@ -77,10 +85,12 @@ namespace { > out << "Usage: " << cmd << " [options] {device|file}" << endl > << "Options:" << endl > << " {-h|--help}" << endl > - << " {-f|--format} {xml|human_readable}" << endl > + << " {-f|--format} {xml|human_readable|old|null}" << endl > << " {-r|--repair}" << endl > << " {-m|--metadata-snap} [block#]" << endl > << " {-o <xml file>}" << endl > + << " {-d device_id}" << endl > + << " {-s|--skip-mappings}" << endl > << " {-V|--version}" << endl; > } > } > @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv) > { > int c; > char const *output = NULL; > - const char shortopts[] = "hm::o:f:rV"; > + const char shortopts[] = "hm::o:sd:f:rV"; > char *end_ptr; > string format = "xml"; > block_address metadata_snap = 0; > struct flags flags; > - flags.find_metadata_snap = flags.repair = false; > + flags.find_metadata_snap = flags.repair = flags.skip_mappings = false; > + flags.dev_id = -1; > > const struct option longopts[] = { > { "help", no_argument, NULL, 'h'}, > { "metadata-snap", optional_argument, NULL, 'm' }, > { "output", required_argument, NULL, 'o'}, > + { "skip-mappings", no_argument, NULL, 's'}, > + { "device-id", required_argument, NULL, 'd'}, > { "format", required_argument, NULL, 'f' }, > { "repair", no_argument, NULL, 'r'}, > { "version", no_argument, NULL, 'V'}, > @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv) > flags.repair = true; > break; > > + case 's': > + flags.skip_mappings = true; > + break; > + > + case 'd': > + flags.dev_id = strtoull(optarg, &end_ptr, 10); > + if (end_ptr == optarg) { > + cerr << "couldn't parse <device_id>" << endl; > + usage(cerr, basename(argv[0])); > + return 1; > + } > + break; > + > case 'm': > if (optarg) { > metadata_snap = strtoull(optarg, &end_ptr, 10); > @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv) > } > } > > + if (format != "old" && flags.dev_id < 0) { > + cerr << "Output format 'old' must specify --device-id" << endl; > + return 1; > + } > + > + > if (argc == optind) { > cerr << "No input file provided." << endl; > usage(cerr, basename(argv[0])); > diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h > index c96d22e..9d9814e 100644 > --- a/thin-provisioning/metadata_dumper.h > +++ b/thin-provisioning/metadata_dumper.h > @@ -28,7 +28,7 @@ namespace thin_provisioning { > // Set the @repair flag if your metadata is corrupt, and you'd like > // the dumper to do it's best to recover info. If not set, any > // corruption encountered will cause an exception to be thrown. > - void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair); > + void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1); > } > > //---------------------------------------------------------------- > diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc > index db656ee..55cbae8 100644 > --- a/thin-provisioning/metadata_dumper.cc > +++ b/thin-provisioning/metadata_dumper.cc > @@ -171,23 +171,31 @@ namespace { > dd_(dd), > repair_(repair), > damage_policy_(damage_policy) { > + dev_id_ = -1; > + skip_mappings_ = false; > } > > + void set_dev_id(block_address dev_id) { dev_id_ = dev_id; } > + void set_skip_mappings(bool skip) { skip_mappings_ = skip; } > + > void visit(btree_path const &path, block_address tree_root) { > block_address dev_id = path[0]; > > dd_map::const_iterator it = dd_.find(path[0]); > if (it != dd_.end()) { > device_tree_detail::device_details const &d = it->second; > - e_->begin_device(dev_id, > - d.mapped_blocks_, > - d.transaction_id_, > - d.creation_time_, > - d.snapshotted_time_); > + if (dev_id_ == -1UL || dev_id == dev_id_) { > + e_->begin_device(dev_id, > + d.mapped_blocks_, > + d.transaction_id_, > + d.creation_time_, > + d.snapshotted_time_); > > - emit_mappings(tree_root); > + if (!skip_mappings_) > + emit_mappings(tree_root); > > - e_->end_device(); > + e_->end_device(); > + } > > } else if (!repair_) { > ostringstream msg; > @@ -209,6 +217,8 @@ namespace { > emitter::ptr e_; > dd_map const &dd_; > bool repair_; > + block_address dev_id_; > + bool skip_mappings_; > mapping_tree_detail::damage_visitor::ptr damage_policy_; > }; > } > @@ -216,7 +226,7 @@ namespace { > //---------------------------------------------------------------- > > void > -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair) > +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id) > { > details_extractor de; > device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair)); > @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair) > { > mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair)); > mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair)); > + > + mte.set_skip_mappings(skip_mappings); > + > + if (dev_id >= 0) > + mte.set_dev_id(dev_id); > walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy); > } > > diff --git a/man8/thin_dump.8 b/man8/thin_dump.8 > index 7a9f785..9e0bf9e 100644 > --- a/man8/thin_dump.8 > +++ b/man8/thin_dump.8 > @@ -26,9 +26,19 @@ in order to put it back onto a metadata > > This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used. > > -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP". > +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP". > Print output in XML or human readable format. > - > +.sp > +The > +.B old > +format requires the --device-id option and emits in the > +following format, one record per line: > +.B offset:length:tdata_offset > +(hence, "o:L:d") with units in bytes. > +.sp > +The > +.B null > +format emits nothing and only walks the tree. > .IP "\fB\-r, \-\-repair\fP". > Repair the metadata whilst dumping it. > > @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#. > See the thin provisioning target documentation on how to create or release > a metadata snapshot and retrieve the block number from the kernel. > > +.IP "\fB\-d, \-\-skip\-mappings\fP". > +Skip emission of the mappings. This outputs nothing if format is > +either of > +.B old > +or > +.B null > +. > + > +.IP "\fB\-d, \-\-device\-id\fP". > +Specify the device_id to be dumped. > + > .IP "\fB\-h, \-\-help\fP". > Print help and exit. > > diff --git a/Makefile.in b/Makefile.in > index e67b300..078bb53 100644 > --- a/Makefile.in > +++ b/Makefile.in > @@ -73,6 +73,8 @@ SOURCE=\ > persistent-data/validators.cc \ > thin-provisioning/device_tree.cc \ > thin-provisioning/human_readable_format.cc \ > + thin-provisioning/old_format.cc \ > + thin-provisioning/null_format.cc \ > thin-provisioning/mapping_tree.cc \ > thin-provisioning/metadata.cc \ > thin-provisioning/metadata_checker.cc \ > diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h > new file mode 100644 > index 0000000..dba69e9 > --- /dev/null > +++ b/thin-provisioning/old_format.h > @@ -0,0 +1,34 @@ > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > +// > +// This file is part of the thin-provisioning-tools source. > +// > +// thin-provisioning-tools is free software: you can redistribute it > +// and/or modify it under the terms of the GNU General Public License > +// as published by the Free Software Foundation, either version 3 of > +// the License, or (at your option) any later version. > +// > +// thin-provisioning-tools is distributed in the hope that it will be > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +// GNU General Public License for more details. > +// > +// You should have received a copy of the GNU General Public License along > +// with thin-provisioning-tools. If not, see > +// <http://www.gnu.org/licenses/>. > + > +#ifndef OLD_FORMAT_H > +#define OLD_FORMAT_H > + > +#include "emitter.h" > + > +#include <iosfwd> > + > +//---------------------------------------------------------------- > + > +namespace thin_provisioning { > + emitter::ptr create_old_emitter(std::ostream &out); > +} > + > +//---------------------------------------------------------------- > + > +#endif > diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc > new file mode 100644 > index 0000000..52056c8 > --- /dev/null > +++ b/thin-provisioning/old_format.cc > @@ -0,0 +1,103 @@ > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > +// > +// This file is part of the thin-provisioning-tools source. > +// > +// thin-provisioning-tools is free software: you can redistribute it > +// and/or modify it under the terms of the GNU General Public License > +// as published by the Free Software Foundation, either version 3 of > +// the License, or (at your option) any later version. > +// > +// thin-provisioning-tools is distributed in the hope that it will be > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +// GNU General Public License for more details. > +// > +// You should have received a copy of the GNU General Public License along > +// with thin-provisioning-tools. If not, see > +// <http://www.gnu.org/licenses/>. > + > +#include "old_format.h" > + > +#include <iostream> > +#include <boost/format.hpp> > + > +using namespace std; > +using namespace thin_provisioning; > + > +//---------------------------------------------------------------- > + > +namespace { > + template <typename T> > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { > + if (maybe) > + out << *maybe; > + > + return out; > + } > + > + class old_emitter : public emitter { > + public: > + old_emitter(ostream &out) > + : out_(out) { > + } > + > + void begin_superblock(string const &uuid, > + uint64_t time, > + uint64_t trans_id, > + uint32_t data_block_size, > + uint64_t nr_data_blocks, > + boost::optional<uint64_t> metadata_snap) { > + data_block_size_ = data_block_size; > + } > + > + void end_superblock() { > + } > + > + void begin_device(uint32_t dev_id, > + uint64_t mapped_blocks, > + uint64_t trans_id, > + uint64_t creation_time, > + uint64_t snap_time) { > + } > + > + void end_device() { > + } > + > + void begin_named_mapping(string const &name) { > + } > + > + void end_named_mapping() { > + } > + > + void identifier(string const &name) { > + } > + > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { > + out_ << (data_block_size_ << 9)*origin_begin > + << ":" << (data_block_size_ << 9)*len > + << ":" << (data_block_size_ << 9)*data_begin > + << endl; > + } > + > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { > + out_ << (data_block_size_ << 9)*origin_block > + << ":" << (data_block_size_ << 9) > + << ":" << (data_block_size_ << 9)*data_block > + << endl; > + } > + > + private: > + ostream &out_; > + uint64_t data_block_size_; > + }; > +} > + > +//---------------------------------------------------------------- > + > +thin_provisioning::emitter::ptr > +thin_provisioning::create_old_emitter(ostream &out) > +{ > + return emitter::ptr(new old_emitter(out)); > +} > + > +//---------------------------------------------------------------- > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h > new file mode 100644 > index 0000000..2688762 > --- /dev/null > +++ b/thin-provisioning/null_format.h > @@ -0,0 +1,34 @@ > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > +// > +// This file is part of the thin-provisioning-tools source. > +// > +// thin-provisioning-tools is free software: you can redistribute it > +// and/or modify it under the terms of the GNU General Public License > +// as published by the Free Software Foundation, either version 3 of > +// the License, or (at your option) any later version. > +// > +// thin-provisioning-tools is distributed in the hope that it will be > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +// GNU General Public License for more details. > +// > +// You should have received a copy of the GNU General Public License along > +// with thin-provisioning-tools. If not, see > +// <http://www.gnu.org/licenses/>. > + > +#ifndef NULL_FORMAT_H > +#define NULL_FORMAT_H > + > +#include "emitter.h" > + > +#include <iosfwd> > + > +//---------------------------------------------------------------- > + > +namespace thin_provisioning { > + emitter::ptr create_null_emitter(std::ostream &out); > +} > + > +//---------------------------------------------------------------- > + > +#endif > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc > new file mode 100644 > index 0000000..b1baa7d > --- /dev/null > +++ b/thin-provisioning/null_format.cc > @@ -0,0 +1,92 @@ > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > +// > +// This file is part of the thin-provisioning-tools source. > +// > +// thin-provisioning-tools is free software: you can redistribute it > +// and/or modify it under the terms of the GNU General Public License > +// as published by the Free Software Foundation, either version 3 of > +// the License, or (at your option) any later version. > +// > +// thin-provisioning-tools is distributed in the hope that it will be > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +// GNU General Public License for more details. > +// > +// You should have received a copy of the GNU General Public License along > +// with thin-provisioning-tools. If not, see > +// <http://www.gnu.org/licenses/>. > + > +#include "null_format.h" > + > +#include <iostream> > + > +using namespace std; > +using namespace thin_provisioning; > + > +//---------------------------------------------------------------- > + > +namespace { > + template <typename T> > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { > + if (maybe) > + out << *maybe; > + > + return out; > + } > + > + class null_emitter : public emitter { > + public: > + null_emitter(ostream &out) > + : out_(out) { > + } > + > + void begin_superblock(string const &uuid, > + uint64_t time, > + uint64_t trans_id, > + uint32_t data_block_size, > + uint64_t nr_data_blocks, > + boost::optional<uint64_t> metadata_snap) { > + } > + > + void end_superblock() { > + } > + > + void begin_device(uint32_t dev_id, > + uint64_t mapped_blocks, > + uint64_t trans_id, > + uint64_t creation_time, > + uint64_t snap_time) { > + } > + > + void end_device() { > + } > + > + void begin_named_mapping(string const &name) { > + } > + > + void end_named_mapping() { > + } > + > + void identifier(string const &name) { > + } > + > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { > + } > + > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { > + } > + > + private: > + ostream &out_; > + }; > +} > + > +//---------------------------------------------------------------- > + > +thin_provisioning::emitter::ptr > +thin_provisioning::create_null_emitter(ostream &out) > +{ > + return emitter::ptr(new null_emitter(out)); > +} > + > +//---------------------------------------------------------------- > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h > new file mode 100644 > index 0000000..2688762 > --- /dev/null > +++ b/thin-provisioning/null_format.h > @@ -0,0 +1,34 @@ > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > +// > +// This file is part of the thin-provisioning-tools source. > +// > +// thin-provisioning-tools is free software: you can redistribute it > +// and/or modify it under the terms of the GNU General Public License > +// as published by the Free Software Foundation, either version 3 of > +// the License, or (at your option) any later version. > +// > +// thin-provisioning-tools is distributed in the hope that it will be > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +// GNU General Public License for more details. > +// > +// You should have received a copy of the GNU General Public License along > +// with thin-provisioning-tools. If not, see > +// <http://www.gnu.org/licenses/>. > + > +#ifndef NULL_FORMAT_H > +#define NULL_FORMAT_H > + > +#include "emitter.h" > + > +#include <iosfwd> > + > +//---------------------------------------------------------------- > + > +namespace thin_provisioning { > + emitter::ptr create_null_emitter(std::ostream &out); > +} > + > +//---------------------------------------------------------------- > + > +#endif > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc > new file mode 100644 > index 0000000..b1baa7d > --- /dev/null > +++ b/thin-provisioning/null_format.cc > @@ -0,0 +1,92 @@ > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > +// > +// This file is part of the thin-provisioning-tools source. > +// > +// thin-provisioning-tools is free software: you can redistribute it > +// and/or modify it under the terms of the GNU General Public License > +// as published by the Free Software Foundation, either version 3 of > +// the License, or (at your option) any later version. > +// > +// thin-provisioning-tools is distributed in the hope that it will be > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +// GNU General Public License for more details. > +// > +// You should have received a copy of the GNU General Public License along > +// with thin-provisioning-tools. If not, see > +// <http://www.gnu.org/licenses/>. > + > +#include "null_format.h" > + > +#include <iostream> > + > +using namespace std; > +using namespace thin_provisioning; > + > +//---------------------------------------------------------------- > + > +namespace { > + template <typename T> > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { > + if (maybe) > + out << *maybe; > + > + return out; > + } > + > + class null_emitter : public emitter { > + public: > + null_emitter(ostream &out) > + : out_(out) { > + } > + > + void begin_superblock(string const &uuid, > + uint64_t time, > + uint64_t trans_id, > + uint32_t data_block_size, > + uint64_t nr_data_blocks, > + boost::optional<uint64_t> metadata_snap) { > + } > + > + void end_superblock() { > + } > + > + void begin_device(uint32_t dev_id, > + uint64_t mapped_blocks, > + uint64_t trans_id, > + uint64_t creation_time, > + uint64_t snap_time) { > + } > + > + void end_device() { > + } > + > + void begin_named_mapping(string const &name) { > + } > + > + void end_named_mapping() { > + } > + > + void identifier(string const &name) { > + } > + > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { > + } > + > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { > + } > + > + private: > + ostream &out_; > + }; > +} > + > +//---------------------------------------------------------------- > + > +thin_provisioning::emitter::ptr > +thin_provisioning::create_null_emitter(ostream &out) > +{ > + return emitter::ptr(new null_emitter(out)); > +} > + > +//---------------------------------------------------------------- > -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Thu, 17 Mar 2016, Joe Thornber wrote: > If you're skipping the mappings does the new thin_ls provide enough > information for you? Our primary need is the ability to specify --device-id in the o:L:d output format (old_format.cc) with mappings. Our nightly dumps now take 50 seconds instead of 40 minutes since we only dump the dev_id's that we need. As for thin_ls, we're still in the 0.5.x branch, I couldn't get 0.6.x to build on el7: # git checkout v0.6.2-rc6 Previous HEAD position was e46bdfd... v0.6.1 HEAD is now at afb71d8... bump version # ./configure [...] # make [CXX] base/application.cc g++: error: @LFS_FLAGS@: No such file or directory make: *** [base/application.o] Error 1 Am I missing something trivial here? -- Eric Wheeler > > - Joe > > On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote: > > Hi Joe, > > > > Please review the patch below when you have a moment. I am interested in > > your feedback, and also interested in having this functionality merged > > upstream. This was written against thin-provisioning-tools.git tag > > v0.5.6. > > > > We use thin_dump on live dm-thin metadata snapshots all the time. In our > > case, we want to dump the XML for new (snapshot) volumes instead of > > dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 > > minutes instead of ~5 seconds for a single volume with --device-id. > > > > I started by adding --device-id to pass the `block_address dev_id` and > > skip the call to emit_mappings() within mapping_tree_emitter::visit() > > unless the dev_id matches --device-id as passed to thin_dump. > > > > It is sometimes nice to list all of the device supers without waiting for > > emit_mappings() so I added the --skip-mappings option. This allows you to > > get the dev_id without reading the mappings: > > thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta > > > > Like `bool repair`, I added skip_mappings and dev_id as arguments to > > thin_provisioning::metadata_dump() and passed them down to > > mapping_tree_emitter with new members set_skip_mappings() and set_dev_id() > > to set private attributes (default values are assigned in > > metadata_dumper.h in case of backward compatible calls). > > > > We work with the device metadata in a simplified format, and without > > superblock information: > > > > origin_offset:length:tdata_offset (hence, "o:L:d" or "old") > > > > Therefore, the output --format 'old' was added. I added the format 'null' > > as well for benchmarking purposes so the ostream needn't write large > > volumes of text to /dev/null. > > > > Benchmarks of the various formats on our sample metadata snapshot on an > > idle machine: > > > > for i in xml human_readable old null; do > > time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null > > done > > > > real user sys > > xml 29:01.27 25:35.87 01:54.49 > > human 27:39.81 24:12.90 01:52.39 > > old 27:07.71 23:40.64 01:52.97 > > null 23:39.38 21:17.22 00:49.04 > > > > I have this as a branch in bitbucket if you prefer to see it online: > > > > https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id > > or > > git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git v0.5.6-device-id > > > > > > -- > > Eric Wheeler > > > > > > diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc > > index 191acb5..7df5d85 100644 > > --- a/thin-provisioning/thin_dump.cc > > +++ b/thin-provisioning/thin_dump.cc > > @@ -22,6 +22,8 @@ > > #include <libgen.h> > > > > #include "human_readable_format.h" > > +#include "old_format.h" > > +#include "null_format.h" > > #include "metadata_dumper.h" > > #include "metadata.h" > > #include "xml_format.h" > > @@ -36,6 +38,8 @@ using namespace thin_provisioning; > > struct flags { > > bool find_metadata_snap; > > bool repair; > > + block_address dev_id; > > + bool skip_mappings; > > }; > > > > namespace { > > @@ -49,12 +53,16 @@ namespace { > > e = create_xml_emitter(out); > > else if (format == "human_readable") > > e = create_human_readable_emitter(out); > > + else if (format == "old") > > + e = create_old_emitter(out); > > + else if (format == "null") > > + e = create_null_emitter(out); > > else { > > cerr << "unknown format '" << format << "'" << endl; > > exit(1); > > } > > > > - metadata_dump(md, e, flags.repair); > > + metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id); > > > > } catch (std::exception &e) { > > cerr << e.what() << endl; > > @@ -77,10 +85,12 @@ namespace { > > out << "Usage: " << cmd << " [options] {device|file}" << endl > > << "Options:" << endl > > << " {-h|--help}" << endl > > - << " {-f|--format} {xml|human_readable}" << endl > > + << " {-f|--format} {xml|human_readable|old|null}" << endl > > << " {-r|--repair}" << endl > > << " {-m|--metadata-snap} [block#]" << endl > > << " {-o <xml file>}" << endl > > + << " {-d device_id}" << endl > > + << " {-s|--skip-mappings}" << endl > > << " {-V|--version}" << endl; > > } > > } > > @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv) > > { > > int c; > > char const *output = NULL; > > - const char shortopts[] = "hm::o:f:rV"; > > + const char shortopts[] = "hm::o:sd:f:rV"; > > char *end_ptr; > > string format = "xml"; > > block_address metadata_snap = 0; > > struct flags flags; > > - flags.find_metadata_snap = flags.repair = false; > > + flags.find_metadata_snap = flags.repair = flags.skip_mappings = false; > > + flags.dev_id = -1; > > > > const struct option longopts[] = { > > { "help", no_argument, NULL, 'h'}, > > { "metadata-snap", optional_argument, NULL, 'm' }, > > { "output", required_argument, NULL, 'o'}, > > + { "skip-mappings", no_argument, NULL, 's'}, > > + { "device-id", required_argument, NULL, 'd'}, > > { "format", required_argument, NULL, 'f' }, > > { "repair", no_argument, NULL, 'r'}, > > { "version", no_argument, NULL, 'V'}, > > @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv) > > flags.repair = true; > > break; > > > > + case 's': > > + flags.skip_mappings = true; > > + break; > > + > > + case 'd': > > + flags.dev_id = strtoull(optarg, &end_ptr, 10); > > + if (end_ptr == optarg) { > > + cerr << "couldn't parse <device_id>" << endl; > > + usage(cerr, basename(argv[0])); > > + return 1; > > + } > > + break; > > + > > case 'm': > > if (optarg) { > > metadata_snap = strtoull(optarg, &end_ptr, 10); > > @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv) > > } > > } > > > > + if (format != "old" && flags.dev_id < 0) { > > + cerr << "Output format 'old' must specify --device-id" << endl; > > + return 1; > > + } > > + > > + > > if (argc == optind) { > > cerr << "No input file provided." << endl; > > usage(cerr, basename(argv[0])); > > diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h > > index c96d22e..9d9814e 100644 > > --- a/thin-provisioning/metadata_dumper.h > > +++ b/thin-provisioning/metadata_dumper.h > > @@ -28,7 +28,7 @@ namespace thin_provisioning { > > // Set the @repair flag if your metadata is corrupt, and you'd like > > // the dumper to do it's best to recover info. If not set, any > > // corruption encountered will cause an exception to be thrown. > > - void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair); > > + void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1); > > } > > > > //---------------------------------------------------------------- > > diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc > > index db656ee..55cbae8 100644 > > --- a/thin-provisioning/metadata_dumper.cc > > +++ b/thin-provisioning/metadata_dumper.cc > > @@ -171,23 +171,31 @@ namespace { > > dd_(dd), > > repair_(repair), > > damage_policy_(damage_policy) { > > + dev_id_ = -1; > > + skip_mappings_ = false; > > } > > > > + void set_dev_id(block_address dev_id) { dev_id_ = dev_id; } > > + void set_skip_mappings(bool skip) { skip_mappings_ = skip; } > > + > > void visit(btree_path const &path, block_address tree_root) { > > block_address dev_id = path[0]; > > > > dd_map::const_iterator it = dd_.find(path[0]); > > if (it != dd_.end()) { > > device_tree_detail::device_details const &d = it->second; > > - e_->begin_device(dev_id, > > - d.mapped_blocks_, > > - d.transaction_id_, > > - d.creation_time_, > > - d.snapshotted_time_); > > + if (dev_id_ == -1UL || dev_id == dev_id_) { > > + e_->begin_device(dev_id, > > + d.mapped_blocks_, > > + d.transaction_id_, > > + d.creation_time_, > > + d.snapshotted_time_); > > > > - emit_mappings(tree_root); > > + if (!skip_mappings_) > > + emit_mappings(tree_root); > > > > - e_->end_device(); > > + e_->end_device(); > > + } > > > > } else if (!repair_) { > > ostringstream msg; > > @@ -209,6 +217,8 @@ namespace { > > emitter::ptr e_; > > dd_map const &dd_; > > bool repair_; > > + block_address dev_id_; > > + bool skip_mappings_; > > mapping_tree_detail::damage_visitor::ptr damage_policy_; > > }; > > } > > @@ -216,7 +226,7 @@ namespace { > > //---------------------------------------------------------------- > > > > void > > -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair) > > +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id) > > { > > details_extractor de; > > device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair)); > > @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair) > > { > > mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair)); > > mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair)); > > + > > + mte.set_skip_mappings(skip_mappings); > > + > > + if (dev_id >= 0) > > + mte.set_dev_id(dev_id); > > walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy); > > } > > > > diff --git a/man8/thin_dump.8 b/man8/thin_dump.8 > > index 7a9f785..9e0bf9e 100644 > > --- a/man8/thin_dump.8 > > +++ b/man8/thin_dump.8 > > @@ -26,9 +26,19 @@ in order to put it back onto a metadata > > > > This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used. > > > > -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP". > > +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP". > > Print output in XML or human readable format. > > - > > +.sp > > +The > > +.B old > > +format requires the --device-id option and emits in the > > +following format, one record per line: > > +.B offset:length:tdata_offset > > +(hence, "o:L:d") with units in bytes. > > +.sp > > +The > > +.B null > > +format emits nothing and only walks the tree. > > .IP "\fB\-r, \-\-repair\fP". > > Repair the metadata whilst dumping it. > > > > @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#. > > See the thin provisioning target documentation on how to create or release > > a metadata snapshot and retrieve the block number from the kernel. > > > > +.IP "\fB\-d, \-\-skip\-mappings\fP". > > +Skip emission of the mappings. This outputs nothing if format is > > +either of > > +.B old > > +or > > +.B null > > +. > > + > > +.IP "\fB\-d, \-\-device\-id\fP". > > +Specify the device_id to be dumped. > > + > > .IP "\fB\-h, \-\-help\fP". > > Print help and exit. > > > > diff --git a/Makefile.in b/Makefile.in > > index e67b300..078bb53 100644 > > --- a/Makefile.in > > +++ b/Makefile.in > > @@ -73,6 +73,8 @@ SOURCE=\ > > persistent-data/validators.cc \ > > thin-provisioning/device_tree.cc \ > > thin-provisioning/human_readable_format.cc \ > > + thin-provisioning/old_format.cc \ > > + thin-provisioning/null_format.cc \ > > thin-provisioning/mapping_tree.cc \ > > thin-provisioning/metadata.cc \ > > thin-provisioning/metadata_checker.cc \ > > diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h > > new file mode 100644 > > index 0000000..dba69e9 > > --- /dev/null > > +++ b/thin-provisioning/old_format.h > > @@ -0,0 +1,34 @@ > > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > > +// > > +// This file is part of the thin-provisioning-tools source. > > +// > > +// thin-provisioning-tools is free software: you can redistribute it > > +// and/or modify it under the terms of the GNU General Public License > > +// as published by the Free Software Foundation, either version 3 of > > +// the License, or (at your option) any later version. > > +// > > +// thin-provisioning-tools is distributed in the hope that it will be > > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +// GNU General Public License for more details. > > +// > > +// You should have received a copy of the GNU General Public License along > > +// with thin-provisioning-tools. If not, see > > +// <http://www.gnu.org/licenses/>. > > + > > +#ifndef OLD_FORMAT_H > > +#define OLD_FORMAT_H > > + > > +#include "emitter.h" > > + > > +#include <iosfwd> > > + > > +//---------------------------------------------------------------- > > + > > +namespace thin_provisioning { > > + emitter::ptr create_old_emitter(std::ostream &out); > > +} > > + > > +//---------------------------------------------------------------- > > + > > +#endif > > diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc > > new file mode 100644 > > index 0000000..52056c8 > > --- /dev/null > > +++ b/thin-provisioning/old_format.cc > > @@ -0,0 +1,103 @@ > > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > > +// > > +// This file is part of the thin-provisioning-tools source. > > +// > > +// thin-provisioning-tools is free software: you can redistribute it > > +// and/or modify it under the terms of the GNU General Public License > > +// as published by the Free Software Foundation, either version 3 of > > +// the License, or (at your option) any later version. > > +// > > +// thin-provisioning-tools is distributed in the hope that it will be > > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +// GNU General Public License for more details. > > +// > > +// You should have received a copy of the GNU General Public License along > > +// with thin-provisioning-tools. If not, see > > +// <http://www.gnu.org/licenses/>. > > + > > +#include "old_format.h" > > + > > +#include <iostream> > > +#include <boost/format.hpp> > > + > > +using namespace std; > > +using namespace thin_provisioning; > > + > > +//---------------------------------------------------------------- > > + > > +namespace { > > + template <typename T> > > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { > > + if (maybe) > > + out << *maybe; > > + > > + return out; > > + } > > + > > + class old_emitter : public emitter { > > + public: > > + old_emitter(ostream &out) > > + : out_(out) { > > + } > > + > > + void begin_superblock(string const &uuid, > > + uint64_t time, > > + uint64_t trans_id, > > + uint32_t data_block_size, > > + uint64_t nr_data_blocks, > > + boost::optional<uint64_t> metadata_snap) { > > + data_block_size_ = data_block_size; > > + } > > + > > + void end_superblock() { > > + } > > + > > + void begin_device(uint32_t dev_id, > > + uint64_t mapped_blocks, > > + uint64_t trans_id, > > + uint64_t creation_time, > > + uint64_t snap_time) { > > + } > > + > > + void end_device() { > > + } > > + > > + void begin_named_mapping(string const &name) { > > + } > > + > > + void end_named_mapping() { > > + } > > + > > + void identifier(string const &name) { > > + } > > + > > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { > > + out_ << (data_block_size_ << 9)*origin_begin > > + << ":" << (data_block_size_ << 9)*len > > + << ":" << (data_block_size_ << 9)*data_begin > > + << endl; > > + } > > + > > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { > > + out_ << (data_block_size_ << 9)*origin_block > > + << ":" << (data_block_size_ << 9) > > + << ":" << (data_block_size_ << 9)*data_block > > + << endl; > > + } > > + > > + private: > > + ostream &out_; > > + uint64_t data_block_size_; > > + }; > > +} > > + > > +//---------------------------------------------------------------- > > + > > +thin_provisioning::emitter::ptr > > +thin_provisioning::create_old_emitter(ostream &out) > > +{ > > + return emitter::ptr(new old_emitter(out)); > > +} > > + > > +//---------------------------------------------------------------- > > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h > > new file mode 100644 > > index 0000000..2688762 > > --- /dev/null > > +++ b/thin-provisioning/null_format.h > > @@ -0,0 +1,34 @@ > > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > > +// > > +// This file is part of the thin-provisioning-tools source. > > +// > > +// thin-provisioning-tools is free software: you can redistribute it > > +// and/or modify it under the terms of the GNU General Public License > > +// as published by the Free Software Foundation, either version 3 of > > +// the License, or (at your option) any later version. > > +// > > +// thin-provisioning-tools is distributed in the hope that it will be > > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +// GNU General Public License for more details. > > +// > > +// You should have received a copy of the GNU General Public License along > > +// with thin-provisioning-tools. If not, see > > +// <http://www.gnu.org/licenses/>. > > + > > +#ifndef NULL_FORMAT_H > > +#define NULL_FORMAT_H > > + > > +#include "emitter.h" > > + > > +#include <iosfwd> > > + > > +//---------------------------------------------------------------- > > + > > +namespace thin_provisioning { > > + emitter::ptr create_null_emitter(std::ostream &out); > > +} > > + > > +//---------------------------------------------------------------- > > + > > +#endif > > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc > > new file mode 100644 > > index 0000000..b1baa7d > > --- /dev/null > > +++ b/thin-provisioning/null_format.cc > > @@ -0,0 +1,92 @@ > > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > > +// > > +// This file is part of the thin-provisioning-tools source. > > +// > > +// thin-provisioning-tools is free software: you can redistribute it > > +// and/or modify it under the terms of the GNU General Public License > > +// as published by the Free Software Foundation, either version 3 of > > +// the License, or (at your option) any later version. > > +// > > +// thin-provisioning-tools is distributed in the hope that it will be > > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +// GNU General Public License for more details. > > +// > > +// You should have received a copy of the GNU General Public License along > > +// with thin-provisioning-tools. If not, see > > +// <http://www.gnu.org/licenses/>. > > + > > +#include "null_format.h" > > + > > +#include <iostream> > > + > > +using namespace std; > > +using namespace thin_provisioning; > > + > > +//---------------------------------------------------------------- > > + > > +namespace { > > + template <typename T> > > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { > > + if (maybe) > > + out << *maybe; > > + > > + return out; > > + } > > + > > + class null_emitter : public emitter { > > + public: > > + null_emitter(ostream &out) > > + : out_(out) { > > + } > > + > > + void begin_superblock(string const &uuid, > > + uint64_t time, > > + uint64_t trans_id, > > + uint32_t data_block_size, > > + uint64_t nr_data_blocks, > > + boost::optional<uint64_t> metadata_snap) { > > + } > > + > > + void end_superblock() { > > + } > > + > > + void begin_device(uint32_t dev_id, > > + uint64_t mapped_blocks, > > + uint64_t trans_id, > > + uint64_t creation_time, > > + uint64_t snap_time) { > > + } > > + > > + void end_device() { > > + } > > + > > + void begin_named_mapping(string const &name) { > > + } > > + > > + void end_named_mapping() { > > + } > > + > > + void identifier(string const &name) { > > + } > > + > > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { > > + } > > + > > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { > > + } > > + > > + private: > > + ostream &out_; > > + }; > > +} > > + > > +//---------------------------------------------------------------- > > + > > +thin_provisioning::emitter::ptr > > +thin_provisioning::create_null_emitter(ostream &out) > > +{ > > + return emitter::ptr(new null_emitter(out)); > > +} > > + > > +//---------------------------------------------------------------- > > diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h > > new file mode 100644 > > index 0000000..2688762 > > --- /dev/null > > +++ b/thin-provisioning/null_format.h > > @@ -0,0 +1,34 @@ > > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > > +// > > +// This file is part of the thin-provisioning-tools source. > > +// > > +// thin-provisioning-tools is free software: you can redistribute it > > +// and/or modify it under the terms of the GNU General Public License > > +// as published by the Free Software Foundation, either version 3 of > > +// the License, or (at your option) any later version. > > +// > > +// thin-provisioning-tools is distributed in the hope that it will be > > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +// GNU General Public License for more details. > > +// > > +// You should have received a copy of the GNU General Public License along > > +// with thin-provisioning-tools. If not, see > > +// <http://www.gnu.org/licenses/>. > > + > > +#ifndef NULL_FORMAT_H > > +#define NULL_FORMAT_H > > + > > +#include "emitter.h" > > + > > +#include <iosfwd> > > + > > +//---------------------------------------------------------------- > > + > > +namespace thin_provisioning { > > + emitter::ptr create_null_emitter(std::ostream &out); > > +} > > + > > +//---------------------------------------------------------------- > > + > > +#endif > > diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc > > new file mode 100644 > > index 0000000..b1baa7d > > --- /dev/null > > +++ b/thin-provisioning/null_format.cc > > @@ -0,0 +1,92 @@ > > +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. > > +// > > +// This file is part of the thin-provisioning-tools source. > > +// > > +// thin-provisioning-tools is free software: you can redistribute it > > +// and/or modify it under the terms of the GNU General Public License > > +// as published by the Free Software Foundation, either version 3 of > > +// the License, or (at your option) any later version. > > +// > > +// thin-provisioning-tools is distributed in the hope that it will be > > +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty > > +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > +// GNU General Public License for more details. > > +// > > +// You should have received a copy of the GNU General Public License along > > +// with thin-provisioning-tools. If not, see > > +// <http://www.gnu.org/licenses/>. > > + > > +#include "null_format.h" > > + > > +#include <iostream> > > + > > +using namespace std; > > +using namespace thin_provisioning; > > + > > +//---------------------------------------------------------------- > > + > > +namespace { > > + template <typename T> > > + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { > > + if (maybe) > > + out << *maybe; > > + > > + return out; > > + } > > + > > + class null_emitter : public emitter { > > + public: > > + null_emitter(ostream &out) > > + : out_(out) { > > + } > > + > > + void begin_superblock(string const &uuid, > > + uint64_t time, > > + uint64_t trans_id, > > + uint32_t data_block_size, > > + uint64_t nr_data_blocks, > > + boost::optional<uint64_t> metadata_snap) { > > + } > > + > > + void end_superblock() { > > + } > > + > > + void begin_device(uint32_t dev_id, > > + uint64_t mapped_blocks, > > + uint64_t trans_id, > > + uint64_t creation_time, > > + uint64_t snap_time) { > > + } > > + > > + void end_device() { > > + } > > + > > + void begin_named_mapping(string const &name) { > > + } > > + > > + void end_named_mapping() { > > + } > > + > > + void identifier(string const &name) { > > + } > > + > > + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { > > + } > > + > > + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { > > + } > > + > > + private: > > + ostream &out_; > > + }; > > +} > > + > > +//---------------------------------------------------------------- > > + > > +thin_provisioning::emitter::ptr > > +thin_provisioning::create_null_emitter(ostream &out) > > +{ > > + return emitter::ptr(new null_emitter(out)); > > +} > > + > > +//---------------------------------------------------------------- > > > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel > -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>: > > On Thu, 17 Mar 2016, Joe Thornber wrote: > >> If you're skipping the mappings does the new thin_ls provide enough >> information for you? > > Our primary need is the ability to specify --device-id in the o:L:d output > format (old_format.cc) with mappings. Our nightly dumps now take 50 > seconds instead of 40 minutes since we only dump the dev_id's that we > need. > > As for thin_ls, we're still in the 0.5.x branch, I couldn't get 0.6.x to > build on el7: > > # git checkout v0.6.2-rc6 > Previous HEAD position was e46bdfd... v0.6.1 > HEAD is now at afb71d8... bump version > > # ./configure > [...] > > # make > [CXX] base/application.cc > g++: error: @LFS_FLAGS@: No such file or directory > make: *** [base/application.o] Error 1 > > > Am I missing something trivial here? Did you run autoconf to regenerate the configure script ? v0.6.x has some changes in the configure script. # make distclean # autoconf # ./configure # make Ming-Hung Tsai -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>: > > On Thu, 17 Mar 2016, Joe Thornber wrote: > >> If you're skipping the mappings does the new thin_ls provide enough >> information for you? > > Our primary need is the ability to specify --device-id in the o:L:d output > format (old_format.cc) with mappings. Our nightly dumps now take 50 > seconds instead of 40 minutes since we only dump the dev_id's that we > need. > I have similar requirement. I did this by my own compact_emitter, which is a simplified version of "human_readable_emitter", without showing the arrows and parentheses: output format: begin superblock=>uuid:"",time:<time>,transaction:<tid>,flags:<flags>,version:<version>,data_block_size:<bs>,nr_data_blocks:<nr_blocks> device=>dev_id:<dev_id>,mapped_blocks:<mapped_blocks>,transaction:<tid>,creation_time:<creation_time>,snap_time:<snap_time> <origin_begin>,<data_begin>,<len>,<time> <origin_begin>,<data_begin>,1,<time> ... end superblock e.g., begin superblock=>uuid:"",time:0,transaction:1,flags:0,version:1,data_block_size:2048,nr_data_blocks:5657156 device=>dev_id:10000,mapped_blocks:3864041,transaction:0,creation_time:0,snap_time:0 1000,2000,100,0 1100,3000,200,0 ... end superblock The <time> field is preserved for snapshot comparison. Maybe we could discuss on creating a flexible emitter that could customize the output format: - Mapping units: sectors or blocks - Emit the superblock and device tags or not. I think that an end mark might be better for network transmission, to determine whether the dump was ended or not. - Other consideration: should the customized output compatible to thin_restore ? Ming-Hung Tsai -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
2016-03-18 6:45 GMT+08:00 Eric Wheeler <dm-devel@lists.ewheeler.net>: > > On Thu, 17 Mar 2016, Joe Thornber wrote: > >> If you're skipping the mappings does the new thin_ls provide enough >> information for you? > > Our primary need is the ability to specify --device-id in the o:L:d output > format (old_format.cc) with mappings. Our nightly dumps now take 50 > seconds instead of 40 minutes since we only dump the dev_id's that we > need. I think that the --device-id option is required for thin_dump, but the necessity of --skip-mappings could be discussed. If thin_ls supports --device-id, then thin_dump might not require --skip-mappings. I put my code here for reference (based on v0.7-devel) https://github.com/mingnus/thin-provisioning-tools/commits/v0.7-thin_dump-enhancement I borrow the option name "--skip-mappings" and function name "set_skip_mappings" from Eric, since that they sounds more clear. Build method: # autoconf; ./configure --enable-dev-tools; make My use cases: # thin_dump --device-id=<dev_id> --format=<f> [--skip-mappings] Ming-Hung Tsai -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote: > Hi Joe, > > Please review the patch below when you have a moment. I am interested in > your feedback, and also interested in having this functionality merged > upstream. This was written against thin-provisioning-tools.git tag > v0.5.6. These changes are now on the v0.7-devel branch: i) --dev-id to filter thin_dump output on device id, multiple selections allowed. ii) --skip-mappings iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own. eg, thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda The contrib/ directory is for third party contributions. Use 'make contrib' to build it. I've put your emitter and tmakatos' emitter in there already. I hope that covers everything you need. - Joe -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Thu, 24 Mar 2016, Joe Thornber wrote: > On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote: > > Hi Joe, > > > > Please review the patch below when you have a moment. I am interested in > > your feedback, and also interested in having this functionality merged > > upstream. This was written against thin-provisioning-tools.git tag > > v0.5.6. > > These changes are now on the v0.7-devel branch: > > i) --dev-id to filter thin_dump output on device id, multiple selections allowed. > ii) --skip-mappings > iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own. > > eg, > > thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda > > > The contrib/ directory is for third party contributions. Use 'make > contrib' to build it. I've put your emitter and tmakatos' emitter in > there already. > > I hope that covers everything you need. I think that will work nicely. Thank you! -Eric > > - Joe > -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Thu, 24 Mar 2016, Joe Thornber wrote: > On Tue, Mar 15, 2016 at 01:45:15AM +0000, Eric Wheeler wrote: > > Hi Joe, > > > > Please review the patch below when you have a moment. I am interested in > > your feedback, and also interested in having this functionality merged > > upstream. This was written against thin-provisioning-tools.git tag > > v0.5.6. > > These changes are now on the v0.7-devel branch: > > i) --dev-id to filter thin_dump output on device id, multiple selections allowed. > ii) --skip-mappings > iii) thin_dump now takes a format called 'custom' that let's you use a plugin emitter of your own. > > eg, > > thin_dump --format custom=contrib/ewheeler_emitter.so /dev/sda > > > The contrib/ directory is for third party contributions. Use 'make > contrib' to build it. I've put your emitter and tmakatos' emitter in > there already. It took me a bit to figure out that I needed --enable-testing to get contrib/ to build. You might remove the --enable-testing requirement within the Makefile if contrib will remain as an optional Makefile target that doesn't usually build with `make`. Also FYI: I noticed that `-ldl' needed added to to the g++ linking line: /usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:15: undefined reference to `dlopen' /usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:19: undefined reference to `dlerror' /usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:20: undefined reference to `dlsym' /usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:22: undefined reference to `dlerror' /usr/src/thin-provisioning-tools/thin-provisioning/shared_library_emitter.cc:17: undefined reference to `dlerror' So I added the -ldl as below: g++ -g -Wall -fPIC -fno-strict-aliasing -std=c++11 -O8 -DSTRERROR_R_CHAR_P -o bin/pdata_tools base/application.o base/base64.o base/disk_units.o base/endian_utils.o base/error_state.o base/error_string.o base/grid_layout.o base/progress_monitor.o base/rolling_hash.o base/xml_utils.o block-cache/block_cache.o caching/cache_check.o caching/cache_dump.o caching/cache_metadata_size.o caching/cache_repair.o caching/cache_restore.o caching/commands.o caching/hint_array.o caching/mapping_array.o caching/metadata.o caching/metadata_dump.o caching/restore_emitter.o caching/superblock.o caching/xml_format.o era/commands.o era/era_array.o era/era_check.o era/era_detail.o era/era_dump.o era/era_invalidate.o era/era_restore.o era/metadata.o era/metadata_dump.o era/restore_emitter.o era/superblock.o era/writeset_tree.o era/xml_format.o main.o persistent-data/checksum.o persistent-data/data-structures/bitset.o persistent-data/data-structures/bloom_filter.o persistent-data/data-structures/btree.o persistent-data/error_set.o persistent-data/file_utils.o persistent-data/hex_dump.o persistent-data/space-maps/careful_alloc.o persistent-data/space-maps/disk.o persistent-data/space-maps/recursive.o persistent-data/space_map.o persistent-data/transaction_manager.o persistent-data/validators.o thin-provisioning/commands.o thin-provisioning/cache_stream.o thin-provisioning/chunk_stream.o thin-provisioning/device_tree.o thin-provisioning/fixed_chunk_stream.o thin-provisioning/human_readable_format.o thin-provisioning/mapping_tree.o thin-provisioning/metadata.o thin-provisioning/metadata_checker.o thin-provisioning/metadata_counter.o thin-provisioning/metadata_dumper.o thin-provisioning/pool_stream.o thin-provisioning/restore_emitter.o thin-provisioning/rmap_visitor.o thin-provisioning/superblock.o thin-provisioning/shared_library_emitter.o thin-provisioning/thin_check.o thin-provisioning/thin_delta.o thin-provisioning/thin_dump.o thin-provisioning/thin_ls.o thin-provisioning/thin_metadata_size.o thin-provisioning/thin_pool.o thin-provisioning/thin_repair.o thin-provisioning/thin_restore.o thin-provisioning/thin_rmap.o thin-provisioning/thin_trim.o thin-provisioning/xml_format.o -laio -lexpat -lstdc++ -ldl ^^^^ ADDED > > I hope that covers everything you need. > > - Joe > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel > -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc index 191acb5..7df5d85 100644 --- a/thin-provisioning/thin_dump.cc +++ b/thin-provisioning/thin_dump.cc @@ -22,6 +22,8 @@ #include <libgen.h> #include "human_readable_format.h" +#include "old_format.h" +#include "null_format.h" #include "metadata_dumper.h" #include "metadata.h" #include "xml_format.h" @@ -36,6 +38,8 @@ using namespace thin_provisioning; struct flags { bool find_metadata_snap; bool repair; + block_address dev_id; + bool skip_mappings; }; namespace { @@ -49,12 +53,16 @@ namespace { e = create_xml_emitter(out); else if (format == "human_readable") e = create_human_readable_emitter(out); + else if (format == "old") + e = create_old_emitter(out); + else if (format == "null") + e = create_null_emitter(out); else { cerr << "unknown format '" << format << "'" << endl; exit(1); } - metadata_dump(md, e, flags.repair); + metadata_dump(md, e, flags.repair, flags.skip_mappings, flags.dev_id); } catch (std::exception &e) { cerr << e.what() << endl; @@ -77,10 +85,12 @@ namespace { out << "Usage: " << cmd << " [options] {device|file}" << endl << "Options:" << endl << " {-h|--help}" << endl - << " {-f|--format} {xml|human_readable}" << endl + << " {-f|--format} {xml|human_readable|old|null}" << endl << " {-r|--repair}" << endl << " {-m|--metadata-snap} [block#]" << endl << " {-o <xml file>}" << endl + << " {-d device_id}" << endl + << " {-s|--skip-mappings}" << endl << " {-V|--version}" << endl; } } @@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv) { int c; char const *output = NULL; - const char shortopts[] = "hm::o:f:rV"; + const char shortopts[] = "hm::o:sd:f:rV"; char *end_ptr; string format = "xml"; block_address metadata_snap = 0; struct flags flags; - flags.find_metadata_snap = flags.repair = false; + flags.find_metadata_snap = flags.repair = flags.skip_mappings = false; + flags.dev_id = -1; const struct option longopts[] = { { "help", no_argument, NULL, 'h'}, { "metadata-snap", optional_argument, NULL, 'm' }, { "output", required_argument, NULL, 'o'}, + { "skip-mappings", no_argument, NULL, 's'}, + { "device-id", required_argument, NULL, 'd'}, { "format", required_argument, NULL, 'f' }, { "repair", no_argument, NULL, 'r'}, { "version", no_argument, NULL, 'V'}, @@ -120,6 +133,19 @@ int thin_dump_main(int argc, char **argv) flags.repair = true; break; + case 's': + flags.skip_mappings = true; + break; + + case 'd': + flags.dev_id = strtoull(optarg, &end_ptr, 10); + if (end_ptr == optarg) { + cerr << "couldn't parse <device_id>" << endl; + usage(cerr, basename(argv[0])); + return 1; + } + break; + case 'm': if (optarg) { metadata_snap = strtoull(optarg, &end_ptr, 10); @@ -147,6 +173,12 @@ int thin_dump_main(int argc, char **argv) } } + if (format != "old" && flags.dev_id < 0) { + cerr << "Output format 'old' must specify --device-id" << endl; + return 1; + } + + if (argc == optind) { cerr << "No input file provided." << endl; usage(cerr, basename(argv[0])); diff --git a/thin-provisioning/metadata_dumper.h b/thin-provisioning/metadata_dumper.h index c96d22e..9d9814e 100644 --- a/thin-provisioning/metadata_dumper.h +++ b/thin-provisioning/metadata_dumper.h @@ -28,7 +28,7 @@ namespace thin_provisioning { // Set the @repair flag if your metadata is corrupt, and you'd like // the dumper to do it's best to recover info. If not set, any // corruption encountered will cause an exception to be thrown. - void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair); + void metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings = false, block_address dev_id = -1); } //---------------------------------------------------------------- diff --git a/thin-provisioning/metadata_dumper.cc b/thin-provisioning/metadata_dumper.cc index db656ee..55cbae8 100644 --- a/thin-provisioning/metadata_dumper.cc +++ b/thin-provisioning/metadata_dumper.cc @@ -171,23 +171,31 @@ namespace { dd_(dd), repair_(repair), damage_policy_(damage_policy) { + dev_id_ = -1; + skip_mappings_ = false; } + void set_dev_id(block_address dev_id) { dev_id_ = dev_id; } + void set_skip_mappings(bool skip) { skip_mappings_ = skip; } + void visit(btree_path const &path, block_address tree_root) { block_address dev_id = path[0]; dd_map::const_iterator it = dd_.find(path[0]); if (it != dd_.end()) { device_tree_detail::device_details const &d = it->second; - e_->begin_device(dev_id, - d.mapped_blocks_, - d.transaction_id_, - d.creation_time_, - d.snapshotted_time_); + if (dev_id_ == -1UL || dev_id == dev_id_) { + e_->begin_device(dev_id, + d.mapped_blocks_, + d.transaction_id_, + d.creation_time_, + d.snapshotted_time_); - emit_mappings(tree_root); + if (!skip_mappings_) + emit_mappings(tree_root); - e_->end_device(); + e_->end_device(); + } } else if (!repair_) { ostringstream msg; @@ -209,6 +217,8 @@ namespace { emitter::ptr e_; dd_map const &dd_; bool repair_; + block_address dev_id_; + bool skip_mappings_; mapping_tree_detail::damage_visitor::ptr damage_policy_; }; } @@ -216,7 +226,7 @@ namespace { //---------------------------------------------------------------- void -thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair) +thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair, bool skip_mappings, block_address dev_id) { details_extractor de; device_tree_detail::damage_visitor::ptr dd_policy(details_damage_policy(repair)); @@ -231,6 +241,11 @@ thin_provisioning::metadata_dump(metadata::ptr md, emitter::ptr e, bool repair) { mapping_tree_detail::damage_visitor::ptr md_policy(mapping_damage_policy(repair)); mapping_tree_emitter mte(md, e, de.get_details(), repair, mapping_damage_policy(repair)); + + mte.set_skip_mappings(skip_mappings); + + if (dev_id >= 0) + mte.set_dev_id(dev_id); walk_mapping_tree(*md->mappings_top_level_, mte, *md_policy); } diff --git a/man8/thin_dump.8 b/man8/thin_dump.8 index 7a9f785..9e0bf9e 100644 --- a/man8/thin_dump.8 +++ b/man8/thin_dump.8 @@ -26,9 +26,19 @@ in order to put it back onto a metadata This tool cannot be run on live metadata unless the \fB\-\-metadata\-snap\fP option is used. -.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable}\fP". +.IP "\fB\-f, \-\-format\fP \fI{xml|human_readable|old|null}\fP". Print output in XML or human readable format. - +.sp +The +.B old +format requires the --device-id option and emits in the +following format, one record per line: +.B offset:length:tdata_offset +(hence, "o:L:d") with units in bytes. +.sp +The +.B null +format emits nothing and only walks the tree. .IP "\fB\-r, \-\-repair\fP". Repair the metadata whilst dumping it. @@ -39,6 +49,17 @@ the thin provisioning device-mapper target, else try the one at block#. See the thin provisioning target documentation on how to create or release a metadata snapshot and retrieve the block number from the kernel. +.IP "\fB\-d, \-\-skip\-mappings\fP". +Skip emission of the mappings. This outputs nothing if format is +either of +.B old +or +.B null +. + +.IP "\fB\-d, \-\-device\-id\fP". +Specify the device_id to be dumped. + .IP "\fB\-h, \-\-help\fP". Print help and exit. diff --git a/Makefile.in b/Makefile.in index e67b300..078bb53 100644 --- a/Makefile.in +++ b/Makefile.in @@ -73,6 +73,8 @@ SOURCE=\ persistent-data/validators.cc \ thin-provisioning/device_tree.cc \ thin-provisioning/human_readable_format.cc \ + thin-provisioning/old_format.cc \ + thin-provisioning/null_format.cc \ thin-provisioning/mapping_tree.cc \ thin-provisioning/metadata.cc \ thin-provisioning/metadata_checker.cc \ diff --git a/thin-provisioning/old_format.h b/thin-provisioning/old_format.h new file mode 100644 index 0000000..dba69e9 --- /dev/null +++ b/thin-provisioning/old_format.h @@ -0,0 +1,34 @@ +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. +// +// This file is part of the thin-provisioning-tools source. +// +// thin-provisioning-tools is free software: you can redistribute it +// and/or modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation, either version 3 of +// the License, or (at your option) any later version. +// +// thin-provisioning-tools is distributed in the hope that it will be +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with thin-provisioning-tools. If not, see +// <http://www.gnu.org/licenses/>. + +#ifndef OLD_FORMAT_H +#define OLD_FORMAT_H + +#include "emitter.h" + +#include <iosfwd> + +//---------------------------------------------------------------- + +namespace thin_provisioning { + emitter::ptr create_old_emitter(std::ostream &out); +} + +//---------------------------------------------------------------- + +#endif diff --git a/thin-provisioning/old_format.cc b/thin-provisioning/old_format.cc new file mode 100644 index 0000000..52056c8 --- /dev/null +++ b/thin-provisioning/old_format.cc @@ -0,0 +1,103 @@ +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. +// +// This file is part of the thin-provisioning-tools source. +// +// thin-provisioning-tools is free software: you can redistribute it +// and/or modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation, either version 3 of +// the License, or (at your option) any later version. +// +// thin-provisioning-tools is distributed in the hope that it will be +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with thin-provisioning-tools. If not, see +// <http://www.gnu.org/licenses/>. + +#include "old_format.h" + +#include <iostream> +#include <boost/format.hpp> + +using namespace std; +using namespace thin_provisioning; + +//---------------------------------------------------------------- + +namespace { + template <typename T> + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { + if (maybe) + out << *maybe; + + return out; + } + + class old_emitter : public emitter { + public: + old_emitter(ostream &out) + : out_(out) { + } + + void begin_superblock(string const &uuid, + uint64_t time, + uint64_t trans_id, + uint32_t data_block_size, + uint64_t nr_data_blocks, + boost::optional<uint64_t> metadata_snap) { + data_block_size_ = data_block_size; + } + + void end_superblock() { + } + + void begin_device(uint32_t dev_id, + uint64_t mapped_blocks, + uint64_t trans_id, + uint64_t creation_time, + uint64_t snap_time) { + } + + void end_device() { + } + + void begin_named_mapping(string const &name) { + } + + void end_named_mapping() { + } + + void identifier(string const &name) { + } + + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { + out_ << (data_block_size_ << 9)*origin_begin + << ":" << (data_block_size_ << 9)*len + << ":" << (data_block_size_ << 9)*data_begin + << endl; + } + + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { + out_ << (data_block_size_ << 9)*origin_block + << ":" << (data_block_size_ << 9) + << ":" << (data_block_size_ << 9)*data_block + << endl; + } + + private: + ostream &out_; + uint64_t data_block_size_; + }; +} + +//---------------------------------------------------------------- + +thin_provisioning::emitter::ptr +thin_provisioning::create_old_emitter(ostream &out) +{ + return emitter::ptr(new old_emitter(out)); +} + +//---------------------------------------------------------------- diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h new file mode 100644 index 0000000..2688762 --- /dev/null +++ b/thin-provisioning/null_format.h @@ -0,0 +1,34 @@ +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. +// +// This file is part of the thin-provisioning-tools source. +// +// thin-provisioning-tools is free software: you can redistribute it +// and/or modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation, either version 3 of +// the License, or (at your option) any later version. +// +// thin-provisioning-tools is distributed in the hope that it will be +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with thin-provisioning-tools. If not, see +// <http://www.gnu.org/licenses/>. + +#ifndef NULL_FORMAT_H +#define NULL_FORMAT_H + +#include "emitter.h" + +#include <iosfwd> + +//---------------------------------------------------------------- + +namespace thin_provisioning { + emitter::ptr create_null_emitter(std::ostream &out); +} + +//---------------------------------------------------------------- + +#endif diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc new file mode 100644 index 0000000..b1baa7d --- /dev/null +++ b/thin-provisioning/null_format.cc @@ -0,0 +1,92 @@ +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. +// +// This file is part of the thin-provisioning-tools source. +// +// thin-provisioning-tools is free software: you can redistribute it +// and/or modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation, either version 3 of +// the License, or (at your option) any later version. +// +// thin-provisioning-tools is distributed in the hope that it will be +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with thin-provisioning-tools. If not, see +// <http://www.gnu.org/licenses/>. + +#include "null_format.h" + +#include <iostream> + +using namespace std; +using namespace thin_provisioning; + +//---------------------------------------------------------------- + +namespace { + template <typename T> + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { + if (maybe) + out << *maybe; + + return out; + } + + class null_emitter : public emitter { + public: + null_emitter(ostream &out) + : out_(out) { + } + + void begin_superblock(string const &uuid, + uint64_t time, + uint64_t trans_id, + uint32_t data_block_size, + uint64_t nr_data_blocks, + boost::optional<uint64_t> metadata_snap) { + } + + void end_superblock() { + } + + void begin_device(uint32_t dev_id, + uint64_t mapped_blocks, + uint64_t trans_id, + uint64_t creation_time, + uint64_t snap_time) { + } + + void end_device() { + } + + void begin_named_mapping(string const &name) { + } + + void end_named_mapping() { + } + + void identifier(string const &name) { + } + + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { + } + + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { + } + + private: + ostream &out_; + }; +} + +//---------------------------------------------------------------- + +thin_provisioning::emitter::ptr +thin_provisioning::create_null_emitter(ostream &out) +{ + return emitter::ptr(new null_emitter(out)); +} + +//---------------------------------------------------------------- diff --git a/thin-provisioning/null_format.h b/thin-provisioning/null_format.h new file mode 100644 index 0000000..2688762 --- /dev/null +++ b/thin-provisioning/null_format.h @@ -0,0 +1,34 @@ +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. +// +// This file is part of the thin-provisioning-tools source. +// +// thin-provisioning-tools is free software: you can redistribute it +// and/or modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation, either version 3 of +// the License, or (at your option) any later version. +// +// thin-provisioning-tools is distributed in the hope that it will be +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with thin-provisioning-tools. If not, see +// <http://www.gnu.org/licenses/>. + +#ifndef NULL_FORMAT_H +#define NULL_FORMAT_H + +#include "emitter.h" + +#include <iosfwd> + +//---------------------------------------------------------------- + +namespace thin_provisioning { + emitter::ptr create_null_emitter(std::ostream &out); +} + +//---------------------------------------------------------------- + +#endif diff --git a/thin-provisioning/null_format.cc b/thin-provisioning/null_format.cc new file mode 100644 index 0000000..b1baa7d --- /dev/null +++ b/thin-provisioning/null_format.cc @@ -0,0 +1,92 @@ +// Copyright (C) 2011 Red Hat, Inc. All rights reserved. +// +// This file is part of the thin-provisioning-tools source. +// +// thin-provisioning-tools is free software: you can redistribute it +// and/or modify it under the terms of the GNU General Public License +// as published by the Free Software Foundation, either version 3 of +// the License, or (at your option) any later version. +// +// thin-provisioning-tools is distributed in the hope that it will be +// useful, but WITHOUT ANY WARRANTY; without even the implied warranty +// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with thin-provisioning-tools. If not, see +// <http://www.gnu.org/licenses/>. + +#include "null_format.h" + +#include <iostream> + +using namespace std; +using namespace thin_provisioning; + +//---------------------------------------------------------------- + +namespace { + template <typename T> + std::ostream &operator << (ostream &out, boost::optional<T> const &maybe) { + if (maybe) + out << *maybe; + + return out; + } + + class null_emitter : public emitter { + public: + null_emitter(ostream &out) + : out_(out) { + } + + void begin_superblock(string const &uuid, + uint64_t time, + uint64_t trans_id, + uint32_t data_block_size, + uint64_t nr_data_blocks, + boost::optional<uint64_t> metadata_snap) { + } + + void end_superblock() { + } + + void begin_device(uint32_t dev_id, + uint64_t mapped_blocks, + uint64_t trans_id, + uint64_t creation_time, + uint64_t snap_time) { + } + + void end_device() { + } + + void begin_named_mapping(string const &name) { + } + + void end_named_mapping() { + } + + void identifier(string const &name) { + } + + void range_map(uint64_t origin_begin, uint64_t data_begin, uint32_t time, uint64_t len) { + } + + void single_map(uint64_t origin_block, uint64_t data_block, uint32_t time) { + } + + private: + ostream &out_; + }; +} + +//---------------------------------------------------------------- + +thin_provisioning::emitter::ptr +thin_provisioning::create_null_emitter(ostream &out) +{ + return emitter::ptr(new null_emitter(out)); +} + +//----------------------------------------------------------------