Message ID | 9bfbf3b43d2c2663d2e3f196810288fd83c0b52e.1659031503.git.boris@bur.io (mailing list archive) |
---|---|
State | Superseded |
Headers | show |
Series | [v2] btrfs: send: add support for fs-verity | expand |
Hi Boris, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on kdave/for-next] [also build test WARNING on next-20220728] [cannot apply to fscrypt/fsverity linus/master v5.19-rc8] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next config: i386-randconfig-s002 (https://download.01.org/0day-ci/archive/20220731/202207311836.xrPNTFhA-lkp@intel.com/config) compiler: gcc-11 (Debian 11.3.0-3) 11.3.0 reproduce: # apt-get install sparse # sparse version: v0.6.4-39-gce1a6720-dirty # https://github.com/intel-lab-lkp/linux/commit/cd0224725d17f6e9ebabdddeea5bc5743a9250ae git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch --no-tags linux-review Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 git checkout cd0224725d17f6e9ebabdddeea5bc5743a9250ae # save the config file mkdir build_dir && cp config build_dir/.config make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=i386 SHELL=/bin/bash fs/btrfs/ If you fix the issue, kindly add following tag where applicable Reported-by: kernel test robot <lkp@intel.com> sparse warnings: (new ones prefixed by >>) >> fs/btrfs/send.c:4906:9: sparse: sparse: incorrect type in argument 4 (different base types) @@ expected int len @@ got restricted __le32 [usertype] sig_size @@ fs/btrfs/send.c:4906:9: sparse: expected int len fs/btrfs/send.c:4906:9: sparse: got restricted __le32 [usertype] sig_size vim +4906 fs/btrfs/send.c 4892 4893 static int send_verity(struct send_ctx *sctx, struct fs_path *path, 4894 struct fsverity_descriptor *desc) 4895 { 4896 int ret; 4897 4898 ret = begin_cmd(sctx, BTRFS_SEND_C_ENABLE_VERITY); 4899 if (ret < 0) 4900 goto out; 4901 4902 TLV_PUT_PATH(sctx, BTRFS_SEND_A_PATH, path); 4903 TLV_PUT_U8(sctx, BTRFS_SEND_A_VERITY_ALGORITHM, desc->hash_algorithm); 4904 TLV_PUT_U32(sctx, BTRFS_SEND_A_VERITY_BLOCK_SIZE, 1U << desc->log_blocksize); 4905 TLV_PUT(sctx, BTRFS_SEND_A_VERITY_SALT_DATA, desc->salt, desc->salt_size); > 4906 TLV_PUT(sctx, BTRFS_SEND_A_VERITY_SIG_DATA, desc->signature, desc->sig_size); 4907 4908 ret = send_cmd(sctx); 4909 4910 tlv_put_failure: 4911 out: 4912 return ret; 4913 } 4914
Hi Boris, Thank you for the patch! Yet something to improve: [auto build test ERROR on kdave/for-next] [also build test ERROR on next-20220728] [cannot apply to fscrypt/fsverity linus/master v5.19-rc8] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next config: arc-randconfig-r025-20220731 (https://download.01.org/0day-ci/archive/20220731/202207312226.d2uCDX53-lkp@intel.com/config) compiler: arceb-elf-gcc (GCC) 12.1.0 reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # https://github.com/intel-lab-lkp/linux/commit/cd0224725d17f6e9ebabdddeea5bc5743a9250ae git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch --no-tags linux-review Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 git checkout cd0224725d17f6e9ebabdddeea5bc5743a9250ae # save the config file mkdir build_dir && cp config build_dir/.config COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash fs/ If you fix the issue, kindly add following tag where applicable Reported-by: kernel test robot <lkp@intel.com> All errors (new ones prefixed by >>): fs/btrfs/send.c: In function 'process_new_verity': >> fs/btrfs/send.c:4926:28: error: 'struct super_block' has no member named 's_vop'; did you mean 's_op'? 4926 | ret = fs_info->sb->s_vop->get_verity_descriptor(inode, NULL, 0); | ^~~~~ | s_op fs/btrfs/send.c:4942:28: error: 'struct super_block' has no member named 's_vop'; did you mean 's_op'? 4942 | ret = fs_info->sb->s_vop->get_verity_descriptor(inode, sctx->verity_descriptor, ret); | ^~~~~ | s_op vim +4926 fs/btrfs/send.c 4914 4915 static int process_new_verity(struct send_ctx *sctx) 4916 { 4917 int ret = 0; 4918 struct btrfs_fs_info *fs_info = sctx->send_root->fs_info; 4919 struct inode *inode; 4920 struct fs_path *p; 4921 4922 inode = btrfs_iget(fs_info->sb, sctx->cur_ino, sctx->send_root); 4923 if (IS_ERR(inode)) 4924 return PTR_ERR(inode); 4925 > 4926 ret = fs_info->sb->s_vop->get_verity_descriptor(inode, NULL, 0); 4927 if (ret < 0) 4928 goto iput; 4929 4930 if (ret > FS_VERITY_MAX_DESCRIPTOR_SIZE) { 4931 ret = -EMSGSIZE; 4932 goto iput; 4933 } 4934 if (!sctx->verity_descriptor) { 4935 sctx->verity_descriptor = kvmalloc(FS_VERITY_MAX_DESCRIPTOR_SIZE, GFP_KERNEL); 4936 if (!sctx->verity_descriptor) { 4937 ret = -ENOMEM; 4938 goto iput; 4939 } 4940 } 4941 4942 ret = fs_info->sb->s_vop->get_verity_descriptor(inode, sctx->verity_descriptor, ret); 4943 if (ret < 0) 4944 goto iput; 4945 4946 p = fs_path_alloc(); 4947 if (!p) { 4948 ret = -ENOMEM; 4949 goto iput; 4950 } 4951 ret = get_cur_path(sctx, sctx->cur_ino, sctx->cur_inode_gen, p); 4952 if (ret < 0) 4953 goto free_path; 4954 4955 ret = send_verity(sctx, p, sctx->verity_descriptor); 4956 if (ret < 0) 4957 goto free_path; 4958 4959 free_path: 4960 fs_path_free(p); 4961 iput: 4962 iput(inode); 4963 return ret; 4964 } 4965
On Sun, Jul 31, 2022 at 10:19:28PM +0800, kernel test robot wrote: > Hi Boris, > > Thank you for the patch! Yet something to improve: > > [auto build test ERROR on kdave/for-next] > [also build test ERROR on next-20220728] > [cannot apply to fscrypt/fsverity linus/master v5.19-rc8] > [If your patch is applied to the wrong git tree, kindly drop us a note. > And when submitting patch, we suggest to use '--base' as documented in > https://git-scm.com/docs/git-format-patch#_base_tree_information] > > url: https://github.com/intel-lab-lkp/linux/commits/Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 > base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next > config: arc-randconfig-r025-20220731 (https://download.01.org/0day-ci/archive/20220731/202207312226.d2uCDX53-lkp@intel.com/config) > compiler: arceb-elf-gcc (GCC) 12.1.0 > reproduce (this is a W=1 build): > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross > chmod +x ~/bin/make.cross > # https://github.com/intel-lab-lkp/linux/commit/cd0224725d17f6e9ebabdddeea5bc5743a9250ae > git remote add linux-review https://github.com/intel-lab-lkp/linux > git fetch --no-tags linux-review Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 > git checkout cd0224725d17f6e9ebabdddeea5bc5743a9250ae > # save the config file > mkdir build_dir && cp config build_dir/.config > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash fs/ > > If you fix the issue, kindly add following tag where applicable > Reported-by: kernel test robot <lkp@intel.com> > > All errors (new ones prefixed by >>): > > fs/btrfs/send.c: In function 'process_new_verity': > >> fs/btrfs/send.c:4926:28: error: 'struct super_block' has no member named 's_vop'; did you mean 's_op'? > 4926 | ret = fs_info->sb->s_vop->get_verity_descriptor(inode, NULL, 0); > | ^~~~~ > | s_op > fs/btrfs/send.c:4942:28: error: 'struct super_block' has no member named 's_vop'; did you mean 's_op'? > 4942 | ret = fs_info->sb->s_vop->get_verity_descriptor(inode, sctx->verity_descriptor, ret); > | ^~~~~ > | s_op > > > vim +4926 fs/btrfs/send.c > > 4914 > 4915 static int process_new_verity(struct send_ctx *sctx) > 4916 { > 4917 int ret = 0; > 4918 struct btrfs_fs_info *fs_info = sctx->send_root->fs_info; > 4919 struct inode *inode; > 4920 struct fs_path *p; > 4921 > 4922 inode = btrfs_iget(fs_info->sb, sctx->cur_ino, sctx->send_root); > 4923 if (IS_ERR(inode)) > 4924 return PTR_ERR(inode); > 4925 > > 4926 ret = fs_info->sb->s_vop->get_verity_descriptor(inode, NULL, 0); sb::s_vop is under ifdef CONFIG_FS_VERITY so you'll need to ifdef the verity callbacks. > 4927 if (ret < 0) > 4928 goto iput; > 4929 > 4930 if (ret > FS_VERITY_MAX_DESCRIPTOR_SIZE) { > 4931 ret = -EMSGSIZE; > 4932 goto iput; > 4933 } > 4934 if (!sctx->verity_descriptor) { > 4935 sctx->verity_descriptor = kvmalloc(FS_VERITY_MAX_DESCRIPTOR_SIZE, GFP_KERNEL); > 4936 if (!sctx->verity_descriptor) { > 4937 ret = -ENOMEM; > 4938 goto iput; > 4939 } > 4940 } > 4941 > 4942 ret = fs_info->sb->s_vop->get_verity_descriptor(inode, sctx->verity_descriptor, ret); > 4943 if (ret < 0) > 4944 goto iput; > 4945 > 4946 p = fs_path_alloc(); > 4947 if (!p) { > 4948 ret = -ENOMEM; > 4949 goto iput; > 4950 } > 4951 ret = get_cur_path(sctx, sctx->cur_ino, sctx->cur_inode_gen, p); > 4952 if (ret < 0) > 4953 goto free_path; > 4954 > 4955 ret = send_verity(sctx, p, sctx->verity_descriptor); > 4956 if (ret < 0) > 4957 goto free_path; > 4958 > 4959 free_path: > 4960 fs_path_free(p); > 4961 iput: > 4962 iput(inode); > 4963 return ret; > 4964 } > 4965 > > -- > 0-DAY CI Kernel Test Service > https://01.org/lkp
On Sun, Jul 31, 2022 at 06:35:07PM +0800, kernel test robot wrote: > Hi Boris, > > Thank you for the patch! Perhaps something to improve: > > [auto build test WARNING on kdave/for-next] > [also build test WARNING on next-20220728] > [cannot apply to fscrypt/fsverity linus/master v5.19-rc8] > [If your patch is applied to the wrong git tree, kindly drop us a note. > And when submitting patch, we suggest to use '--base' as documented in > https://git-scm.com/docs/git-format-patch#_base_tree_information] > > url: https://github.com/intel-lab-lkp/linux/commits/Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 > base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next > config: i386-randconfig-s002 (https://download.01.org/0day-ci/archive/20220731/202207311836.xrPNTFhA-lkp@intel.com/config) > compiler: gcc-11 (Debian 11.3.0-3) 11.3.0 > reproduce: > # apt-get install sparse > # sparse version: v0.6.4-39-gce1a6720-dirty > # https://github.com/intel-lab-lkp/linux/commit/cd0224725d17f6e9ebabdddeea5bc5743a9250ae > git remote add linux-review https://github.com/intel-lab-lkp/linux > git fetch --no-tags linux-review Boris-Burkov/btrfs-send-add-support-for-fs-verity/20220729-021228 > git checkout cd0224725d17f6e9ebabdddeea5bc5743a9250ae > # save the config file > mkdir build_dir && cp config build_dir/.config > make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' O=build_dir ARCH=i386 SHELL=/bin/bash fs/btrfs/ > > If you fix the issue, kindly add following tag where applicable > Reported-by: kernel test robot <lkp@intel.com> > > sparse warnings: (new ones prefixed by >>) > >> fs/btrfs/send.c:4906:9: sparse: sparse: incorrect type in argument 4 (different base types) @@ expected int len @@ got restricted __le32 [usertype] sig_size @@ > fs/btrfs/send.c:4906:9: sparse: expected int len > fs/btrfs/send.c:4906:9: sparse: got restricted __le32 [usertype] sig_size The types are compatible, should this use the get_unaligned_le32 or what's the supposed fix?
On Thu, Jul 28, 2022 at 11:11:42AM -0700, Boris Burkov wrote: > --- a/fs/btrfs/send.h > +++ b/fs/btrfs/send.h > @@ -92,8 +92,11 @@ enum btrfs_send_cmd { > BTRFS_SEND_C_ENCODED_WRITE = 25, > BTRFS_SEND_C_MAX_V2 = 25, > > + /* Version 3 */ > + BTRFS_SEND_C_ENABLE_VERITY = 26, Regarding the name, same name for ioctl and command is a good idea, for potential future verity extensions we could add more and having plain 'VERITY' would not be ideal. The other ioctl is 'measure', so I thought of something like consistency check during the stream, "verify this file before continuing". The command names are either imperative or descriptive, so alternative name could be VERITY_RECORD or VERITY_DESC that matches the data stored, the BTRFS_VERITY_DESC_ITEM_KEY .
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index e7671afcee4f..d922b56c3b21 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -15,6 +15,7 @@ #include <linux/string.h> #include <linux/compat.h> #include <linux/crc32c.h> +#include <linux/fsverity.h> #include "send.h" #include "ctree.h" @@ -127,6 +128,8 @@ struct send_ctx { bool cur_inode_new_gen; bool cur_inode_deleted; bool ignore_cur_inode; + bool cur_inode_needs_verity; + void *verity_descriptor; u64 send_progress; @@ -624,6 +627,7 @@ static int tlv_put(struct send_ctx *sctx, u16 attr, const void *data, int len) return tlv_put(sctx, attr, &__tmp, sizeof(__tmp)); \ } +TLV_PUT_DEFINE_INT(8) TLV_PUT_DEFINE_INT(32) TLV_PUT_DEFINE_INT(64) @@ -4886,6 +4890,79 @@ static int process_all_new_xattrs(struct send_ctx *sctx) return ret; } +static int send_verity(struct send_ctx *sctx, struct fs_path *path, + struct fsverity_descriptor *desc) +{ + int ret; + + ret = begin_cmd(sctx, BTRFS_SEND_C_ENABLE_VERITY); + if (ret < 0) + goto out; + + TLV_PUT_PATH(sctx, BTRFS_SEND_A_PATH, path); + TLV_PUT_U8(sctx, BTRFS_SEND_A_VERITY_ALGORITHM, desc->hash_algorithm); + TLV_PUT_U32(sctx, BTRFS_SEND_A_VERITY_BLOCK_SIZE, 1U << desc->log_blocksize); + TLV_PUT(sctx, BTRFS_SEND_A_VERITY_SALT_DATA, desc->salt, desc->salt_size); + TLV_PUT(sctx, BTRFS_SEND_A_VERITY_SIG_DATA, desc->signature, desc->sig_size); + + ret = send_cmd(sctx); + +tlv_put_failure: +out: + return ret; +} + +static int process_new_verity(struct send_ctx *sctx) +{ + int ret = 0; + struct btrfs_fs_info *fs_info = sctx->send_root->fs_info; + struct inode *inode; + struct fs_path *p; + + inode = btrfs_iget(fs_info->sb, sctx->cur_ino, sctx->send_root); + if (IS_ERR(inode)) + return PTR_ERR(inode); + + ret = fs_info->sb->s_vop->get_verity_descriptor(inode, NULL, 0); + if (ret < 0) + goto iput; + + if (ret > FS_VERITY_MAX_DESCRIPTOR_SIZE) { + ret = -EMSGSIZE; + goto iput; + } + if (!sctx->verity_descriptor) { + sctx->verity_descriptor = kvmalloc(FS_VERITY_MAX_DESCRIPTOR_SIZE, GFP_KERNEL); + if (!sctx->verity_descriptor) { + ret = -ENOMEM; + goto iput; + } + } + + ret = fs_info->sb->s_vop->get_verity_descriptor(inode, sctx->verity_descriptor, ret); + if (ret < 0) + goto iput; + + p = fs_path_alloc(); + if (!p) { + ret = -ENOMEM; + goto iput; + } + ret = get_cur_path(sctx, sctx->cur_ino, sctx->cur_inode_gen, p); + if (ret < 0) + goto free_path; + + ret = send_verity(sctx, p, sctx->verity_descriptor); + if (ret < 0) + goto free_path; + +free_path: + fs_path_free(p); +iput: + iput(inode); + return ret; +} + static inline u64 max_send_read_size(const struct send_ctx *sctx) { return sctx->send_max_size - SZ_16K; @@ -6377,6 +6454,11 @@ static int finish_inode_if_needed(struct send_ctx *sctx, int at_end) if (ret < 0) goto out; } + if (sctx->cur_inode_needs_verity) { + ret = process_new_verity(sctx); + if (ret < 0) + goto out; + } ret = send_capabilities(sctx); if (ret < 0) @@ -6785,6 +6867,18 @@ static int changed_extent(struct send_ctx *sctx, return ret; } +static int changed_verity(struct send_ctx *sctx, + enum btrfs_compare_tree_result result) +{ + int ret = 0; + + if (!sctx->cur_inode_new_gen && !sctx->cur_inode_deleted) { + if (result == BTRFS_COMPARE_TREE_NEW) + sctx->cur_inode_needs_verity = true; + } + return ret; +} + static int dir_changed(struct send_ctx *sctx, u64 dir) { u64 orig_gen, new_gen; @@ -6939,6 +7033,9 @@ static int changed_cb(struct btrfs_path *left_path, ret = changed_xattr(sctx, result); else if (key->type == BTRFS_EXTENT_DATA_KEY) ret = changed_extent(sctx, result); + else if (key->type == BTRFS_VERITY_DESC_ITEM_KEY && + key->offset == 0) + ret = changed_verity(sctx, result); } out: @@ -8036,6 +8133,8 @@ long btrfs_ioctl_send(struct inode *inode, struct btrfs_ioctl_send_args *arg) kvfree(sctx->clone_roots); kfree(sctx->send_buf_pages); kvfree(sctx->send_buf); + if (sctx->verity_descriptor) + kvfree(sctx->verity_descriptor); name_cache_free(sctx); diff --git a/fs/btrfs/send.h b/fs/btrfs/send.h index 4bb4e6a638cb..0a4537775e0c 100644 --- a/fs/btrfs/send.h +++ b/fs/btrfs/send.h @@ -92,8 +92,11 @@ enum btrfs_send_cmd { BTRFS_SEND_C_ENCODED_WRITE = 25, BTRFS_SEND_C_MAX_V2 = 25, + /* Version 3 */ + BTRFS_SEND_C_ENABLE_VERITY = 26, + BTRFS_SEND_C_MAX_V3 = 26, /* End */ - BTRFS_SEND_C_MAX = 25, + BTRFS_SEND_C_MAX = 26, }; /* attributes in send stream */ @@ -160,8 +163,14 @@ enum { BTRFS_SEND_A_ENCRYPTION = 31, BTRFS_SEND_A_MAX_V2 = 31, - /* End */ - BTRFS_SEND_A_MAX = 31, + /* Version 3 */ + BTRFS_SEND_A_VERITY_ALGORITHM = 32, + BTRFS_SEND_A_VERITY_BLOCK_SIZE = 33, + BTRFS_SEND_A_VERITY_SALT_DATA = 34, + BTRFS_SEND_A_VERITY_SIG_DATA = 35, + BTRFS_SEND_A_MAX_V3 = 35, + + __BTRFS_SEND_A_MAX = 35, }; long btrfs_ioctl_send(struct inode *inode, struct btrfs_ioctl_send_args *arg); diff --git a/fs/verity/fsverity_private.h b/fs/verity/fsverity_private.h index 629785c95007..dbe1ce5b450a 100644 --- a/fs/verity/fsverity_private.h +++ b/fs/verity/fsverity_private.h @@ -70,8 +70,6 @@ struct fsverity_info { const struct inode *inode; }; -/* Arbitrary limit to bound the kmalloc() size. Can be changed. */ -#define FS_VERITY_MAX_DESCRIPTOR_SIZE 16384 #define FS_VERITY_MAX_SIGNATURE_SIZE (FS_VERITY_MAX_DESCRIPTOR_SIZE - \ sizeof(struct fsverity_descriptor)) diff --git a/include/linux/fsverity.h b/include/linux/fsverity.h index 7af030fa3c36..40f14e5fed9d 100644 --- a/include/linux/fsverity.h +++ b/include/linux/fsverity.h @@ -22,6 +22,9 @@ */ #define FS_VERITY_MAX_DIGEST_SIZE SHA512_DIGEST_SIZE +/* Arbitrary limit to bound the kmalloc() size. Can be changed. */ +#define FS_VERITY_MAX_DESCRIPTOR_SIZE 16384 + /* Verity operations for filesystems */ struct fsverity_operations {
Preserve the fs-verity status of a btrfs file across send/recv. There is no facility for installing the Merkle tree contents directly on the receiving filesystem, so we package up the parameters used to enable verity found in the verity descriptor. This gives the receive side enough information to properly enable verity again. Note that this means that receive will have to re-compute the whole Merkle tree, similar to how compression worked before encoded_write. Since the file becomes read-only after verity is enabled, it is important that verity is added to the send stream after any file writes. Therefore, when we process a verity item, merely note that it happened, then actually create the command in the send stream during 'finish_inode_if_needed'. This also creates V3 of the send stream format, without any format changes besides adding the new commands and attributes. Signed-off-by: Boris Burkov <boris@bur.io> -- Changes in v2: - Allocate 16K with kvmalloc and keep it around till the end of send instead of re-allocating on each file with fs-verity. - Use unsigned literal for bitshift. --- fs/btrfs/send.c | 99 ++++++++++++++++++++++++++++++++++++ fs/btrfs/send.h | 15 ++++-- fs/verity/fsverity_private.h | 2 - include/linux/fsverity.h | 3 ++ 4 files changed, 114 insertions(+), 5 deletions(-)