From patchwork Thu Mar 16 01:49:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 9626907 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4DB2E60244 for ; Thu, 16 Mar 2017 01:50:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3FE95285FC for ; Thu, 16 Mar 2017 01:50:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 34CEA28629; Thu, 16 Mar 2017 01:50:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 534FA285FC for ; Thu, 16 Mar 2017 01:50:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751222AbdCPBuL (ORCPT ); Wed, 15 Mar 2017 21:50:11 -0400 Received: from cn.fujitsu.com ([59.151.112.132]:54022 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1751151AbdCPBuK (ORCPT ); Wed, 15 Mar 2017 21:50:10 -0400 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="16628557" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 16 Mar 2017 09:50:03 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 46282477AE9E for ; Thu, 16 Mar 2017 09:50:01 +0800 (CST) Received: from localhost.localdomain (10.167.226.34) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 16 Mar 2017 09:50:00 +0800 From: Qu Wenruo To: Subject: [PATCH v10 1/5] btrfs-progs: Basic framework for dedupe-inband command group Date: Thu, 16 Mar 2017 09:49:52 +0800 Message-ID: <20170316014956.19916-2-quwenruo@cn.fujitsu.com> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20170316014956.19916-1-quwenruo@cn.fujitsu.com> References: <20170316014956.19916-1-quwenruo@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.34] X-yoursite-MailScanner-ID: 46282477AE9E.AFD7C X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: quwenruo@cn.fujitsu.com Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add basic ioctl header and command group framework for later use. Alone with basic man page doc. Signed-off-by: Qu Wenruo --- Documentation/Makefile.in | 1 + Documentation/btrfs-dedupe-inband.asciidoc | 40 +++++++++++++++++++++++++ Documentation/btrfs.asciidoc | 4 +++ Makefile | 2 +- btrfs.c | 2 ++ cmds-dedupe-ib.c | 48 ++++++++++++++++++++++++++++++ commands.h | 2 ++ dedupe-ib.h | 41 +++++++++++++++++++++++++ ioctl.h | 36 ++++++++++++++++++++++ 9 files changed, 175 insertions(+), 1 deletion(-) create mode 100644 Documentation/btrfs-dedupe-inband.asciidoc create mode 100644 cmds-dedupe-ib.c create mode 100644 dedupe-ib.h diff --git a/Documentation/Makefile.in b/Documentation/Makefile.in index 539c6b55..f175ae1e 100644 --- a/Documentation/Makefile.in +++ b/Documentation/Makefile.in @@ -28,6 +28,7 @@ MAN8_TXT += btrfs-qgroup.asciidoc MAN8_TXT += btrfs-replace.asciidoc MAN8_TXT += btrfs-restore.asciidoc MAN8_TXT += btrfs-property.asciidoc +MAN8_TXT += btrfs-dedupe-inband.asciidoc # Category 5 manual page MAN5_TXT += btrfs-man5.asciidoc diff --git a/Documentation/btrfs-dedupe-inband.asciidoc b/Documentation/btrfs-dedupe-inband.asciidoc new file mode 100644 index 00000000..9ee2bc75 --- /dev/null +++ b/Documentation/btrfs-dedupe-inband.asciidoc @@ -0,0 +1,40 @@ +btrfs-dedupe(8) +============== + +NAME +---- +btrfs-dedupe-inband - manage in-band (write time) de-duplication of a btrfs +filesystem + +SYNOPSIS +-------- +*btrfs dedupe-inband* + +DESCRIPTION +----------- +*btrfs dedupe-inband* is used to enable/disable or show current in-band de-duplication +status of a btrfs filesystem. + +Kernel support for in-band de-duplication starts from 4.8. + +WARNING: In-band de-duplication is still an experimental feautre of btrfs, +use with caution. + +SUBCOMMAND +---------- +Nothing yet + +EXIT STATUS +----------- +*btrfs dedupe-inband* returns a zero exit status if it succeeds. Non zero is +returned in case of failure. + +AVAILABILITY +------------ +*btrfs* is part of btrfs-progs. +Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for +further details. + +SEE ALSO +-------- +`mkfs.btrfs`(8), diff --git a/Documentation/btrfs.asciidoc b/Documentation/btrfs.asciidoc index 100a6adf..64fc0d2c 100644 --- a/Documentation/btrfs.asciidoc +++ b/Documentation/btrfs.asciidoc @@ -50,6 +50,10 @@ COMMANDS Do off-line check on a btrfs filesystem. + See `btrfs-check`(8) for details. +*dedupe*:: + Control btrfs in-band(write time) de-duplication. + + See `btrfs-dedupe`(8) for details. + *device*:: Manage devices managed by btrfs, including add/delete/scan and so on. + diff --git a/Makefile b/Makefile index 67fbc483..24445493 100644 --- a/Makefile +++ b/Makefile @@ -102,7 +102,7 @@ cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \ cmds-restore.o cmds-rescue.o chunk-recover.o super-recover.o \ cmds-property.o cmds-fi-usage.o cmds-inspect-dump-tree.o \ cmds-inspect-dump-super.o cmds-inspect-tree-stats.o cmds-fi-du.o \ - mkfs/common.o + mkfs/common.o cmds-dedupe-ib.o libbtrfs_objects = send-stream.o send-utils.o kernel-lib/rbtree.o btrfs-list.o \ kernel-lib/crc32c.o \ uuid-tree.o utils-lib.o rbtree-utils.o diff --git a/btrfs.c b/btrfs.c index 9214ae6e..1f055d75 100644 --- a/btrfs.c +++ b/btrfs.c @@ -201,6 +201,8 @@ static const struct cmd_group btrfs_cmd_group = { { "quota", cmd_quota, NULL, "a_cmd_group, 0 }, { "qgroup", cmd_qgroup, NULL, &qgroup_cmd_group, 0 }, { "replace", cmd_replace, NULL, &replace_cmd_group, 0 }, + { "dedupe-inband", cmd_dedupe_ib, NULL, &dedupe_ib_cmd_group, + 0 }, { "help", cmd_help, cmd_help_usage, NULL, 0 }, { "version", cmd_version, cmd_version_usage, NULL, 0 }, NULL_CMD_STRUCT diff --git a/cmds-dedupe-ib.c b/cmds-dedupe-ib.c new file mode 100644 index 00000000..f4d31386 --- /dev/null +++ b/cmds-dedupe-ib.c @@ -0,0 +1,48 @@ +/* + * Copyright (C) 2017 Fujitsu. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public + * License v2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; if not, write to the + * Free Software Foundation, Inc., 59 Temple Place - Suite 330, + * Boston, MA 021110-1307, USA. + */ + +#include +#include +#include + +#include "ctree.h" +#include "ioctl.h" + +#include "commands.h" +#include "utils.h" +#include "kerncompat.h" +#include "dedupe-ib.h" + +static const char * const dedupe_ib_cmd_group_usage[] = { + "btrfs dedupe-inband [options] ", + NULL +}; + +static const char dedupe_ib_cmd_group_info[] = +"manage inband(write time) de-duplication"; + +const struct cmd_group dedupe_ib_cmd_group = { + dedupe_ib_cmd_group_usage, dedupe_ib_cmd_group_info, { + NULL_CMD_STRUCT + } +}; + +int cmd_dedupe_ib(int argc, char **argv) +{ + return handle_command_group(&dedupe_ib_cmd_group, argc, argv); +} diff --git a/commands.h b/commands.h index 01bf387e..0cba5dcb 100644 --- a/commands.h +++ b/commands.h @@ -81,6 +81,7 @@ extern const struct cmd_group quota_cmd_group; extern const struct cmd_group qgroup_cmd_group; extern const struct cmd_group replace_cmd_group; extern const struct cmd_group rescue_cmd_group; +extern const struct cmd_group dedupe_ib_cmd_group; extern const char * const cmd_send_usage[]; extern const char * const cmd_receive_usage[]; @@ -110,5 +111,6 @@ int cmd_select_super(int argc, char **argv); int cmd_dump_super(int argc, char **argv); int cmd_debug_tree(int argc, char **argv); int cmd_rescue(int argc, char **argv); +int cmd_dedupe_ib(int argc, char **argv); #endif diff --git a/dedupe-ib.h b/dedupe-ib.h new file mode 100644 index 00000000..97a42c6f --- /dev/null +++ b/dedupe-ib.h @@ -0,0 +1,41 @@ +/* + * Copyright (C) 2017 Fujitsu. All rights reserved. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public + * License v2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public + * License along with this program; if not, write to the + * Free Software Foundation, Inc., 59 Temple Place - Suite 330, + * Boston, MA 021110-1307, USA. + */ + +#ifndef __BTRFS_DEDUPE__ +#define __BTRFS_DEDUPE__ + +/* + * Dedup storage backend + * On disk is persist storage but overhead is large + * In memory is fast but will lose all its hash on umount + */ +#define BTRFS_DEDUPE_BACKEND_INMEMORY 0 +#define BTRFS_DEDUPE_BACKEND_LAST 1 + +/* Dedup block size limit and default value */ +#define BTRFS_DEDUPE_BLOCKSIZE_MAX (8 * 1024 * 1024) +#define BTRFS_DEDUPE_BLOCKSIZE_MIN (16 * 1024) +#define BTRFS_DEDUPE_BLOCKSIZE_DEFAULT (128 * 1024) + +/* Default dedupe limit on number of hash */ +#define BTRFS_DEDUPE_LIMIT_NR_DEFAULT (32 * 1024) + +/* Hash algorithm, only support SHA256 yet */ +#define BTRFS_DEDUPE_HASH_SHA256 0 + +#endif diff --git a/ioctl.h b/ioctl.h index 709e996f..8268c035 100644 --- a/ioctl.h +++ b/ioctl.h @@ -559,6 +559,42 @@ struct btrfs_ioctl_qgroup_create_args { }; BUILD_ASSERT(sizeof(struct btrfs_ioctl_qgroup_create_args) == 16); +/* + * de-duplication control modes + * For re-config, enable will handle it + */ +#define BTRFS_DEDUPE_CTL_ENABLE 1 +#define BTRFS_DEDUPE_CTL_DISABLE 2 +#define BTRFS_DEDUPE_CTL_STATUS 3 +#define BTRFS_DEDUPE_CTL_RECONF 4 +#define BTRFS_DEDUPE_CTL_LAST 5 + +/* + * Allow enable command to be executed on dedupe enabled fs. + * Make dedupe_enable ioctl to be stateless. + * + * Or only dedup_reconf ioctl can be executed on dedupe enabled fs + */ +#define BTRFS_DEDUPE_FLAG_FORCE (1 << 0) +/* + * This structure is used for dedupe enable/disable/configure + * and status ioctl. + * Reserved range should be set to 0xff. + */ +struct btrfs_ioctl_dedupe_args { + __u16 cmd; /* In: command */ + __u64 blocksize; /* In/Out: blocksize */ + __u64 limit_nr; /* In/Out: limit nr for inmem backend */ + __u64 limit_mem; /* In/Out: limit mem for inmem backend */ + __u64 current_nr; /* Out: current hash nr */ + __u16 backend; /* In/Out: current backend */ + __u16 hash_algo; /* In/Out: hash algorithm */ + u8 status; /* Out: enabled or disabled */ + u8 flags; /* In: special flags for ioctl */ + u8 __unused[472]; /* Pad to 512 bytes */ +}; +BUILD_ASSERT(sizeof(btrfs_ioctl_dedupe_args) == 512); + struct btrfs_ioctl_timespec { __u64 sec; __u32 nsec;