From patchwork Thu Feb 15 19:04:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 10223431 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D7F63602CB for ; Thu, 15 Feb 2018 19:06:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C78D52948F for ; Thu, 15 Feb 2018 19:06:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BA1D829485; Thu, 15 Feb 2018 19:06:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B505529485 for ; Thu, 15 Feb 2018 19:06:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1167159AbeBOTGp (ORCPT ); Thu, 15 Feb 2018 14:06:45 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:37697 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1167153AbeBOTFj (ORCPT ); Thu, 15 Feb 2018 14:05:39 -0500 Received: by mail-pl0-f65.google.com with SMTP id ay8so354236plb.4 for ; Thu, 15 Feb 2018 11:05:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=ANA7y1iiTbiTWj+cA2FxUvC8XEkSz340ujJT1syF/f4=; b=ZPDTTqRhalC5nIHHpEjthbDNUHONLIQOSQmpYShWM7oXeynsPGdsze7I5z3mE9+cjC F5KOZg6JTLG6xg/PLYquYvkbslxFLMi7z19ecls2JUB1Z6Jc1BGl0U+QCWdJYZBWqrPE uGxnpYtCpN7AlzqTRNc+wNCnDU4FzJG6zXtHFc/S2SIEaZnc078xpL8UBuVDOn4+zJw5 jHQoiI/iMA++r6YTPZt1kFejKeaqPCx+17QSApGHx1smyvpTfmHJQM9SNL1BfAOC0zlo 0tXGFx2OtQaWyD5NfUEahLiCUIUL00TiKenDGxsnlRaaYkO2HkySVoAMPohNk/pp7eu6 Vj2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=ANA7y1iiTbiTWj+cA2FxUvC8XEkSz340ujJT1syF/f4=; b=K7B3PnwemJWh50zmynDhPeb/3C3KW5NkzOnRMoFZrah7XCq9HnkqtvwRbCQZdvaBvP OYYgLYhIXlWyKxcgj+Vqk5Hmr6qtzF153UhqjJd3wrqXOj0KKMOjz49qKC74Q6PSt0c4 4NvZaoH4K4nAwDGYn2/Giu5wrhKSGn0EDSOoI8C+RUlmvcK03ciCdbzUXKngglO4Ezpa JNAo5WjwYQEOjV+hP/+tkN54MuVDKcbpWPri7CUYXyI4icEvqR8eWOpspuw4b9Yk73vP ZqhqteTz2n+tIouCvDmclAgYYtK8OGNimp2sF1+AftfoztNfHBN9xWuLOkrVsEPp/tWS Ip7A== X-Gm-Message-State: APf1xPB/F/vwQLHl/R4FqQJgc2msTGiq4nB9pO6B0K4dy6KU+n1EoWF+ /E64THdvTePAslzrCCmS9wBJFxB6pyk= X-Google-Smtp-Source: AH8x227nKo9VD1py86/nr+VsfZysUhR+xsEztct5SU+zY5U7GBpX1YuPdXe/dkYQjk1lZ/oCabAJFg== X-Received: by 2002:a17:902:50e:: with SMTP id 14-v6mr3419810plf.360.1518721538677; Thu, 15 Feb 2018 11:05:38 -0800 (PST) Received: from vader.thefacebook.com ([2620:10d:c090:200::6:4a19]) by smtp.gmail.com with ESMTPSA id p1sm40467428pgr.44.2018.02.15.11.05.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Feb 2018 11:05:38 -0800 (PST) From: Omar Sandoval To: linux-btrfs@vger.kernel.org Cc: kernel-team@fb.com Subject: [PATCH v2 12/27] libbtrfsutil: add btrfs_util_create_snapshot() Date: Thu, 15 Feb 2018 11:04:57 -0800 Message-Id: <3796fbef686c9c4dd083aefbeaff24e394e083df.1518720598.git.osandov@fb.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: References: In-Reply-To: References: Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Omar Sandoval Thanks to subvolume iterators, we can also implement recursive snapshot fairly easily. Signed-off-by: Omar Sandoval --- libbtrfsutil/btrfsutil.h | 59 ++++++++++ libbtrfsutil/python/btrfsutilpy.h | 1 + libbtrfsutil/python/module.c | 11 ++ libbtrfsutil/python/subvolume.c | 49 ++++++++ libbtrfsutil/python/tests/test_qgroup.py | 10 ++ libbtrfsutil/python/tests/test_subvolume.py | 55 +++++++++ libbtrfsutil/subvolume.c | 167 ++++++++++++++++++++++++++++ 7 files changed, 352 insertions(+) diff --git a/libbtrfsutil/btrfsutil.h b/libbtrfsutil/btrfsutil.h index 7d3a87f6..04fe6666 100644 --- a/libbtrfsutil/btrfsutil.h +++ b/libbtrfsutil/btrfsutil.h @@ -345,6 +345,65 @@ enum btrfs_util_error btrfs_util_create_subvolume_fd(int parent_fd, uint64_t *async_transid, struct btrfs_util_qgroup_inherit *qgroup_inherit); +/** + * BTRFS_UTIL_CREATE_SNAPSHOT_RECURSIVE - Also snapshot subvolumes beneath the + * source subvolume onto the same location on the new snapshot. + * + * Note that this is currently implemented in userspace non-atomically. Because + * it modifies the newly-created snapshot, it cannot be combined with + * %BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY. It requires appropriate privilege + * (CAP_SYS_ADMIN). + */ +#define BTRFS_UTIL_CREATE_SNAPSHOT_RECURSIVE (1 << 0) +/** + * BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY - Create a read-only snapshot. + */ +#define BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY (1 << 1) +#define BTRFS_UTIL_CREATE_SNAPSHOT_MASK ((1 << 2) - 1) + +/** + * btrfs_util_create_snapshot() - Create a new snapshot from a source subvolume + * path. + * @source: Path of the existing subvolume to snapshot. + * @path: Where to create the snapshot. + * @flags: Bitmask of BTRFS_UTIL_CREATE_SNAPSHOT_* flags. + * @async_transid: See btrfs_util_create_subvolume(). If + * %BTRFS_UTIL_CREATE_SNAPSHOT_RECURSIVE was in @flags, then this will contain + * the largest transaction ID of all created subvolumes. + * @qgroup_inherit: See btrfs_util_create_subvolume(). + * + * Return: %BTRFS_UTIL_OK on success, non-zero error code on failure. + */ +enum btrfs_util_error btrfs_util_create_snapshot(const char *source, + const char *path, int flags, + uint64_t *async_transid, + struct btrfs_util_qgroup_inherit *qgroup_inherit); + +/** + * btrfs_util_create_snapshot_fd() - See btrfs_util_create_snapshot(). + */ +enum btrfs_util_error btrfs_util_create_snapshot_fd(int fd, const char *path, + int flags, + uint64_t *async_transid, + struct btrfs_util_qgroup_inherit *qgroup_inherit); + +/** + * btrfs_util_create_snapshot_fd2() - Create a new snapshot from a source + * subvolume file descriptor and a target parent file descriptor and name. + * @fd: File descriptor of the existing subvolume to snapshot. + * @parent_fd: File descriptor of the parent directory where the snapshot should + * be created. + * @name: Name of the snapshot to create. + * @flags: See btrfs_util_create_snapshot(). + * @async_transid: See btrfs_util_create_snapshot(). + * @qgroup_inherit: See btrfs_util_create_snapshot(). + */ +enum btrfs_util_error btrfs_util_create_snapshot_fd2(int fd, int parent_fd, + const char *name, + int flags, + uint64_t *async_transid, + struct btrfs_util_qgroup_inherit *qgroup_inherit); + struct btrfs_util_subvolume_iterator; /** diff --git a/libbtrfsutil/python/btrfsutilpy.h b/libbtrfsutil/python/btrfsutilpy.h index a9c15219..d552e416 100644 --- a/libbtrfsutil/python/btrfsutilpy.h +++ b/libbtrfsutil/python/btrfsutilpy.h @@ -70,6 +70,7 @@ PyObject *set_subvolume_read_only(PyObject *self, PyObject *args, PyObject *kwds PyObject *get_default_subvolume(PyObject *self, PyObject *args, PyObject *kwds); PyObject *set_default_subvolume(PyObject *self, PyObject *args, PyObject *kwds); PyObject *create_subvolume(PyObject *self, PyObject *args, PyObject *kwds); +PyObject *create_snapshot(PyObject *self, PyObject *args, PyObject *kwds); void add_module_constants(PyObject *m); diff --git a/libbtrfsutil/python/module.c b/libbtrfsutil/python/module.c index daf0747f..d8f797cb 100644 --- a/libbtrfsutil/python/module.c +++ b/libbtrfsutil/python/module.c @@ -195,6 +195,17 @@ static PyMethodDef btrfsutil_methods[] = { "path -- string, bytes, or path-like object\n" "async -- create the subvolume without waiting for it to commit to\n" "disk and return the transaction ID"}, + {"create_snapshot", (PyCFunction)create_snapshot, + METH_VARARGS | METH_KEYWORDS, + "create_snapshot(source, path, recursive=False, read_only=False, async=False)\n\n" + "Create a new snapshot.\n\n" + "Arguments:\n" + "source -- string, bytes, path-like object, or open file descriptor\n" + "path -- string, bytes, or path-like object\n" + "recursive -- also snapshot child subvolumes\n" + "read_only -- create a read-only snapshot\n" + "async -- create the subvolume without waiting for it to commit to\n" + "disk and return the transaction ID"}, {}, }; diff --git a/libbtrfsutil/python/subvolume.c b/libbtrfsutil/python/subvolume.c index 6c384583..a158ade7 100644 --- a/libbtrfsutil/python/subvolume.c +++ b/libbtrfsutil/python/subvolume.c @@ -349,6 +349,55 @@ PyObject *create_subvolume(PyObject *self, PyObject *args, PyObject *kwds) Py_RETURN_NONE; } +PyObject *create_snapshot(PyObject *self, PyObject *args, PyObject *kwds) +{ + static char *keywords[] = { + "source", "path", "recursive", "read_only", "async", + "qgroup_inherit", NULL, + }; + struct path_arg src = {.allow_fd = true}, dst = {.allow_fd = false}; + enum btrfs_util_error err; + int recursive = 0, read_only = 0, async = 0; + int flags = 0; + QgroupInherit *inherit = NULL; + uint64_t transid; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O&O&|pppO!:create_snapshot", + keywords, &path_converter, &src, + &path_converter, &dst, &recursive, + &read_only, &async, + &QgroupInherit_type, &inherit)) + return NULL; + + if (recursive) + flags |= BTRFS_UTIL_CREATE_SNAPSHOT_RECURSIVE; + if (read_only) + flags |= BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY; + + if (src.path) { + err = btrfs_util_create_snapshot(src.path, dst.path, flags, + async ? &transid : NULL, + inherit ? inherit->inherit : NULL); + } else { + err = btrfs_util_create_snapshot_fd(src.fd, dst.path, flags, + async ? &transid : NULL, + inherit ? inherit->inherit : NULL); + } + if (err) { + SetFromBtrfsUtilErrorWithPaths(err, &src, &dst); + path_cleanup(&src); + path_cleanup(&dst); + return NULL; + } + + path_cleanup(&src); + path_cleanup(&dst); + if (async) + return PyLong_FromUnsignedLongLong(transid); + else + Py_RETURN_NONE; +} + typedef struct { PyObject_HEAD struct btrfs_util_subvolume_iterator *iter; diff --git a/libbtrfsutil/python/tests/test_qgroup.py b/libbtrfsutil/python/tests/test_qgroup.py index 19e6b05a..74fc46b6 100644 --- a/libbtrfsutil/python/tests/test_qgroup.py +++ b/libbtrfsutil/python/tests/test_qgroup.py @@ -31,6 +31,16 @@ class TestQgroup(BtrfsTestCase): btrfsutil.create_subvolume(subvol, qgroup_inherit=inherit) + def test_snapshot_inherit(self): + subvol = os.path.join(self.mountpoint, 'subvol') + snapshot = os.path.join(self.mountpoint, 'snapshot') + + inherit = btrfsutil.QgroupInherit() + inherit.add_group(5) + + btrfsutil.create_subvolume(subvol) + btrfsutil.create_snapshot(subvol, snapshot, qgroup_inherit=inherit) + class TestQgroupInherit(unittest.TestCase): def test_new(self): diff --git a/libbtrfsutil/python/tests/test_subvolume.py b/libbtrfsutil/python/tests/test_subvolume.py index b43beca7..2951154e 100644 --- a/libbtrfsutil/python/tests/test_subvolume.py +++ b/libbtrfsutil/python/tests/test_subvolume.py @@ -129,6 +129,13 @@ class TestSubvolume(BtrfsTestCase): self.assertEqual(info.stime, 0) self.assertEqual(info.rtime, 0) + subvol_uuid = info.uuid + snapshot = os.path.join(self.mountpoint, 'snapshot') + btrfsutil.create_snapshot(subvol, snapshot) + + info = btrfsutil.subvolume_info(snapshot) + self.assertEqual(info.parent_uuid, subvol_uuid) + # TODO: test received_uuid, stransid, rtransid, stime, and rtime for arg in self.path_or_fd(self.mountpoint): @@ -215,6 +222,54 @@ class TestSubvolume(BtrfsTestCase): self.assertTrue(os.WIFEXITED(wstatus)) self.assertEqual(os.WEXITSTATUS(wstatus), 0) + def test_create_snapshot(self): + subvol = os.path.join(self.mountpoint, 'subvol') + + btrfsutil.create_subvolume(subvol) + os.mkdir(os.path.join(subvol, 'dir')) + + for i, arg in enumerate(self.path_or_fd(subvol)): + with self.subTest(type=type(arg)): + snapshots_dir = os.path.join(self.mountpoint, 'snapshots{}'.format(i)) + os.mkdir(snapshots_dir) + snapshot = os.path.join(snapshots_dir, 'snapshot') + + btrfsutil.create_snapshot(subvol, snapshot + '1') + self.assertTrue(btrfsutil.is_subvolume(snapshot + '1')) + self.assertTrue(os.path.exists(os.path.join(snapshot + '1', 'dir'))) + + btrfsutil.create_snapshot(subvol, (snapshot + '2').encode()) + self.assertTrue(btrfsutil.is_subvolume(snapshot + '2')) + self.assertTrue(os.path.exists(os.path.join(snapshot + '2', 'dir'))) + + if HAVE_PATH_LIKE: + btrfsutil.create_snapshot(subvol, PurePath(snapshot + '3')) + self.assertTrue(btrfsutil.is_subvolume(snapshot + '3')) + self.assertTrue(os.path.exists(os.path.join(snapshot + '3', 'dir'))) + + nested_subvol = os.path.join(subvol, 'nested') + more_nested_subvol = os.path.join(nested_subvol, 'more_nested') + btrfsutil.create_subvolume(nested_subvol) + btrfsutil.create_subvolume(more_nested_subvol) + os.mkdir(os.path.join(more_nested_subvol, 'nested_dir')) + + snapshot = os.path.join(self.mountpoint, 'snapshot') + + btrfsutil.create_snapshot(subvol, snapshot + '1') + # Dummy subvolume. + self.assertEqual(os.stat(os.path.join(snapshot + '1', 'nested')).st_ino, 2) + self.assertFalse(os.path.exists(os.path.join(snapshot + '1', 'nested', 'more_nested'))) + + btrfsutil.create_snapshot(subvol, snapshot + '2', recursive=True) + self.assertTrue(os.path.exists(os.path.join(snapshot + '2', 'nested/more_nested/nested_dir'))) + + transid = btrfsutil.create_snapshot(subvol, snapshot + '3', recursive=True, async=True) + self.assertTrue(os.path.exists(os.path.join(snapshot + '3', 'nested/more_nested/nested_dir'))) + self.assertGreater(transid, 0) + + btrfsutil.create_snapshot(subvol, snapshot + '4', read_only=True) + self.assertTrue(btrfsutil.get_subvolume_read_only(snapshot + '4')) + def test_subvolume_iterator(self): pwd = os.getcwd() try: diff --git a/libbtrfsutil/subvolume.c b/libbtrfsutil/subvolume.c index 5744f89f..ca3d43c8 100644 --- a/libbtrfsutil/subvolume.c +++ b/libbtrfsutil/subvolume.c @@ -836,6 +836,173 @@ out_iter: return err; } +static enum btrfs_util_error snapshot_subvolume_children(int fd, int parent_fd, + const char *name, + uint64_t *async_transid) +{ + struct btrfs_util_subvolume_iterator *iter; + enum btrfs_util_error err; + int dstfd; + + dstfd = openat(parent_fd, name, O_RDONLY); + if (dstfd == -1) + return BTRFS_UTIL_ERROR_OPEN_FAILED; + + err = btrfs_util_create_subvolume_iterator_fd(fd, 0, 0, &iter); + if (err) + goto out; + + for (;;) { + char child_name[BTRFS_SUBVOL_NAME_MAX + 1]; + char *child_path; + int child_fd, new_parent_fd; + uint64_t tmp_transid; + + err = btrfs_util_subvolume_iterator_next(iter, &child_path, + NULL); + if (err) { + if (err == BTRFS_UTIL_ERROR_STOP_ITERATION) + err = BTRFS_UTIL_OK; + break; + } + + /* Remove the placeholder directory. */ + if (unlinkat(dstfd, child_path, AT_REMOVEDIR) == -1) { + free(child_path); + err = BTRFS_UTIL_ERROR_RMDIR_FAILED; + break; + } + + child_fd = openat(fd, child_path, O_RDONLY); + if (child_fd == -1) { + free(child_path); + err = BTRFS_UTIL_ERROR_OPEN_FAILED; + break; + } + + err = openat_parent_and_name(dstfd, child_path, child_name, + sizeof(child_name), + &new_parent_fd); + free(child_path); + if (err) { + SAVE_ERRNO_AND_CLOSE(child_fd); + break; + } + + err = btrfs_util_create_snapshot_fd2(child_fd, new_parent_fd, + child_name, 0, + async_transid ? &tmp_transid : NULL, + NULL); + SAVE_ERRNO_AND_CLOSE(child_fd); + SAVE_ERRNO_AND_CLOSE(new_parent_fd); + if (err) + break; + if (async_transid && tmp_transid > *async_transid) + *async_transid = tmp_transid; + } + + btrfs_util_destroy_subvolume_iterator(iter); +out: + SAVE_ERRNO_AND_CLOSE(dstfd); + return err; +} + +PUBLIC enum btrfs_util_error btrfs_util_create_snapshot(const char *source, + const char *path, + int flags, + uint64_t *async_transid, + struct btrfs_util_qgroup_inherit *qgroup_inherit) +{ + enum btrfs_util_error err; + int fd; + + fd = open(source, O_RDONLY); + if (fd == -1) + return BTRFS_UTIL_ERROR_OPEN_FAILED; + + err = btrfs_util_create_snapshot_fd(fd, path, flags, async_transid, + qgroup_inherit); + SAVE_ERRNO_AND_CLOSE(fd); + return err; +} + +PUBLIC enum btrfs_util_error btrfs_util_create_snapshot_fd(int fd, + const char *path, + int flags, + uint64_t *async_transid, + struct btrfs_util_qgroup_inherit *qgroup_inherit) +{ + char name[BTRFS_SUBVOL_NAME_MAX + 1]; + enum btrfs_util_error err; + int parent_fd; + + err = openat_parent_and_name(AT_FDCWD, path, name, sizeof(name), + &parent_fd); + if (err) + return err; + + err = btrfs_util_create_snapshot_fd2(fd, parent_fd, name, flags, + async_transid, qgroup_inherit); + SAVE_ERRNO_AND_CLOSE(parent_fd); + return err; +} + +PUBLIC enum btrfs_util_error btrfs_util_create_snapshot_fd2(int fd, + int parent_fd, + const char *name, + int flags, + uint64_t *async_transid, + struct btrfs_util_qgroup_inherit *qgroup_inherit) +{ + struct btrfs_ioctl_vol_args_v2 args = {.fd = fd}; + enum btrfs_util_error err; + size_t len; + int ret; + + if ((flags & ~BTRFS_UTIL_CREATE_SNAPSHOT_MASK) || + ((flags & BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY) && + (flags & BTRFS_UTIL_CREATE_SNAPSHOT_RECURSIVE))) { + errno = EINVAL; + return BTRFS_UTIL_ERROR_INVALID_ARGUMENT; + } + + if (flags & BTRFS_UTIL_CREATE_SNAPSHOT_READ_ONLY) + args.flags |= BTRFS_SUBVOL_RDONLY; + if (async_transid) + args.flags |= BTRFS_SUBVOL_CREATE_ASYNC; + if (qgroup_inherit) { + args.flags |= BTRFS_SUBVOL_QGROUP_INHERIT; + args.qgroup_inherit = (struct btrfs_qgroup_inherit *)qgroup_inherit; + args.size = (sizeof(*args.qgroup_inherit) + + args.qgroup_inherit->num_qgroups * + sizeof(args.qgroup_inherit->qgroups[0])); + } + + len = strlen(name); + if (len >= sizeof(args.name)) { + errno = ENAMETOOLONG; + return BTRFS_UTIL_ERROR_INVALID_ARGUMENT; + } + memcpy(args.name, name, len); + args.name[len] = '\0'; + + ret = ioctl(parent_fd, BTRFS_IOC_SNAP_CREATE_V2, &args); + if (ret == -1) + return BTRFS_UTIL_ERROR_SUBVOL_CREATE_FAILED; + + if (async_transid) + *async_transid = args.transid; + + if (flags & BTRFS_UTIL_CREATE_SNAPSHOT_RECURSIVE) { + err = snapshot_subvolume_children(fd, parent_fd, name, + async_transid); + if (err) + return err; + } + + return BTRFS_UTIL_OK; +} + PUBLIC void btrfs_util_destroy_subvolume_iterator(struct btrfs_util_subvolume_iterator *iter) { if (iter) {