From patchwork Fri Jan 22 20:46:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12040491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E00CC433DB for ; Fri, 22 Jan 2021 20:52:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4E1F523B08 for ; Fri, 22 Jan 2021 20:52:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730228AbhAVUwq (ORCPT ); Fri, 22 Jan 2021 15:52:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730783AbhAVUvS (ORCPT ); Fri, 22 Jan 2021 15:51:18 -0500 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E011C061A2E for ; Fri, 22 Jan 2021 12:47:47 -0800 (PST) Received: by mail-pg1-x52c.google.com with SMTP id 15so4605557pgx.7 for ; Fri, 22 Jan 2021 12:47:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jsZgIcSUOs4ZF+x9oZeNSohaDc9IzmjXBuR8mId+lVo=; b=Uj7VyxfrqBaR6EfVwAVle4XnvXB/BzhgIYK7thaYm3kgYXqFPLvt9QZ2CMSlz8h634 iF2qabPlAsDPzvgT/uZ+8xJS6ejB/hjnglGwHzbQffNtyHuHlW0Zqhr3IqfEfpLmUoih nDZVn23jV5f6uSf4hah6CqfvT6pnPAgb5IzNMJNzAh3H9r8NX+oV9Y04xxch18LvOHGM N73y/Doum+kGlBNlnqH7JbXLhP6ZhyEQsX5uHtHrHL8elPlBF4JKPkaulh5TFK6GKwi0 jDmOdkpqr2sMLptGgVJAjDXVNmilgfoxDCqeKGCjuEQSWtOS7GbvSzO0LMczannWc0+R 9NOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jsZgIcSUOs4ZF+x9oZeNSohaDc9IzmjXBuR8mId+lVo=; b=nv4innYSW0zFdpR3G/iKYyhSO+RGfRjot8qUoen6lLTpilGqOoghcZdZU3NP6Lvt0V BUzrrnPGspYXsPoiyvsbq6GT9upSwUVBuEoIYuhf70gnd7RW1Bl/ovVPwMAxNWd5KS1e PUSdNBHRWsjt5FLqL7hNT0X3WK+qmp77M0oX7Z471fcvXFWVogXaawwkBztEHda2o3Y4 4NbZ6agKg9hdun9LKArSY6hIhOT+SuU7PFNd6xEXNPtFfNGIUsW2tuNBDjKWt1O38GwG 3JWNMdHWRAFCPfnmWUA+f8Ti3jr0AtLgUK2HxmPbzXHjfT8apE8lcE7TUXcT7T63HbTw Nh4g== X-Gm-Message-State: AOAM532Wt2nUw+UbCP9FgxnBV9cq3QoJuYaiMHLO2mD6QmWsVtriaXva GF6uLAMkPorCMG+RPLsA2zd6QcSHQdBxdQ== X-Google-Smtp-Source: ABdhPJxanmXvXDv6AucgzHFRVoxE//s5+VubgqFvbkDMSK4HlwyegUAlajrb1bKcLf4Avm7bzbCN6w== X-Received: by 2002:aa7:8bce:0:b029:1b8:f395:87a with SMTP id s14-20020aa78bce0000b02901b8f395087amr6789477pfd.36.1611348466357; Fri, 22 Jan 2021 12:47:46 -0800 (PST) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:ea88]) by smtp.gmail.com with ESMTPSA id j18sm4092900pfc.99.2021.01.22.12.47.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 12:47:45 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org, Al Viro , Christoph Hellwig Cc: Dave Chinner , Jann Horn , Amir Goldstein , Aleksa Sarai , linux-api@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v7 11/15] btrfs: add send stream v2 definitions Date: Fri, 22 Jan 2021 12:46:58 -0800 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Omar Sandoval This adds the definitions of the new commands for send stream version 2 and their respective attributes: fallocate, FS_IOC_SETFLAGS (a.k.a. chattr), and encoded writes. It also documents two changes to the send stream format in v2: the receiver shouldn't assume a maximum command size, and the DATA attribute is encoded differently to allow for writes larger than 64k. These will be implemented in subsequent changes, and then the ioctl will accept the new flags. Reviewed-by: Josef Bacik Signed-off-by: Omar Sandoval --- fs/btrfs/send.c | 2 +- fs/btrfs/send.h | 30 +++++++++++++++++++++++++++++- include/uapi/linux/btrfs.h | 13 +++++++++++++ 3 files changed, 43 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 3bcbf2bcb869..d07570588a16 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -7267,7 +7267,7 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg) sctx->clone_roots_cnt = arg->clone_sources_count; - sctx->send_max_size = BTRFS_SEND_BUF_SIZE; + sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1; sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL); if (!sctx->send_buf) { ret = -ENOMEM; diff --git a/fs/btrfs/send.h b/fs/btrfs/send.h index de91488b7cd0..9f4f7b96b1eb 100644 --- a/fs/btrfs/send.h +++ b/fs/btrfs/send.h @@ -12,7 +12,11 @@ #define BTRFS_SEND_STREAM_MAGIC "btrfs-stream" #define BTRFS_SEND_STREAM_VERSION 1 -#define BTRFS_SEND_BUF_SIZE SZ_64K +/* + * In send stream v1, no command is larger than 64k. In send stream v2, no limit + * should be assumed. + */ +#define BTRFS_SEND_BUF_SIZE_V1 SZ_64K enum btrfs_tlv_type { BTRFS_TLV_U8, @@ -76,6 +80,13 @@ enum btrfs_send_cmd { BTRFS_SEND_C_END, BTRFS_SEND_C_UPDATE_EXTENT, + + /* The following commands were added in send stream v2. */ + + BTRFS_SEND_C_FALLOCATE, + BTRFS_SEND_C_SETFLAGS, + BTRFS_SEND_C_ENCODED_WRITE, + __BTRFS_SEND_C_MAX, }; #define BTRFS_SEND_C_MAX (__BTRFS_SEND_C_MAX - 1) @@ -106,6 +117,11 @@ enum { BTRFS_SEND_A_PATH_LINK, BTRFS_SEND_A_FILE_OFFSET, + /* + * In send stream v2, this attribute is special: it must be the last + * attribute in a command, its header contains only the type, and its + * length is implicitly the remaining length of the command. + */ BTRFS_SEND_A_DATA, BTRFS_SEND_A_CLONE_UUID, @@ -114,6 +130,18 @@ enum { BTRFS_SEND_A_CLONE_OFFSET, BTRFS_SEND_A_CLONE_LEN, + /* The following attributes were added in send stream v2. */ + + BTRFS_SEND_A_FALLOCATE_MODE, + + BTRFS_SEND_A_SETFLAGS_FLAGS, + + BTRFS_SEND_A_UNENCODED_FILE_LEN, + BTRFS_SEND_A_UNENCODED_LEN, + BTRFS_SEND_A_UNENCODED_OFFSET, + BTRFS_SEND_A_COMPRESSION, + BTRFS_SEND_A_ENCRYPTION, + __BTRFS_SEND_A_MAX, }; #define BTRFS_SEND_A_MAX (__BTRFS_SEND_A_MAX - 1) diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index 5df73001aad4..93aa0932234e 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -770,6 +770,19 @@ struct btrfs_ioctl_received_subvol_args { */ #define BTRFS_SEND_FLAG_OMIT_END_CMD 0x4 +/* + * Use version 2 of the send stream, which adds new commands and supports larger + * writes. + */ +#define BTRFS_SEND_FLAG_STREAM_V2 0x8 + +/* + * Send compressed data using the ENCODED_WRITE command instead of decompressing + * the data and sending it with the WRITE command. This requires + * BTRFS_SEND_FLAG_STREAM_V2. + */ +#define BTRFS_SEND_FLAG_COMPRESSED 0x10 + #define BTRFS_SEND_FLAG_MASK \ (BTRFS_SEND_FLAG_NO_FILE_DATA | \ BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \ From patchwork Fri Jan 22 20:46:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12040487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70CCFC433DB for ; Fri, 22 Jan 2021 20:52:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3FCB923B08 for ; Fri, 22 Jan 2021 20:52:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730829AbhAVUwO (ORCPT ); Fri, 22 Jan 2021 15:52:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730140AbhAVUv2 (ORCPT ); Fri, 22 Jan 2021 15:51:28 -0500 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FB2EC061A32 for ; Fri, 22 Jan 2021 12:47:50 -0800 (PST) Received: by mail-pg1-x536.google.com with SMTP id i5so4637011pgo.1 for ; Fri, 22 Jan 2021 12:47:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ghXEwRnY4PzM061MuP1pb0fdrgOGpzAMsrN3mJnwO7s=; b=HmKnFQK/jJswgAFkT3ZJROdh4Z3YOkNDeYIlJscI+HpQrxnHekiCujkaBybSZNHIzw RGuBL1jy59/TNYcdp1CYLHfRoKSIWOOx2pRnuM2WNcC2XgrjEMf6z/YlHQhSqWeCy9lM TiFHc6p9dHvK2IF22pQv1BXpzMD1f1gznY+nZ/2P+Y18J2dOQk1w20FS6tgajzVPtK3O 8tw0Vu6yC7W9sCTnV9rA305AokqC43te5I5dbMRjsf4Bn/QhiYUlcrdm+E/4nTnDzIHo QvbNSq6nFfct+U0SKCfrAgp77UoozLJRct+jEEcFN8wtSqO/s5jLogjhhlExg02oUP0o BciA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ghXEwRnY4PzM061MuP1pb0fdrgOGpzAMsrN3mJnwO7s=; b=Nlg0IGLbX7vhqxJfXnxeIux6sZK1VtApYQzu/tLbMrv0uAGtQvMJW6fj5465wQ5fAl SHdRqRxZ+DZt05f4U0sNyui/6Q69wLb/yPq0TI1G5vvhhzA25ByD5+prWqLbPVPcMLB7 IGrpSQcO65opIEEktKW9B24SpJ7YmnQQSi48zzR8THnvv2LgkZs+tJ0n/Z4cSEuDXzoK Nts+Xz1Go7bK9mL75rT444usNaPQWD2u5hbdWN5gRkjVopuQY2z6E5LwH6UwMASXT5bp uuDQ3xtzcUcNO9COhDUKi6T6/4KOiijEGLdzq4DbXcKdzh3vTgqpgBS0jfxBYcjMkQhl OmHA== X-Gm-Message-State: AOAM530YQof8IdQq8wNaSaB30XNPQeI+n+qElwriaCefwCE17f2F+75v /O+N31VwXNWuKptJU5fqXMI816qagbwfQQ== X-Google-Smtp-Source: ABdhPJwd4LKDpGluZiqDxWsIEyKvGh7+x2LDFrwLw+VW2djYzxaDCyreVtnZQmD7XMSynIiDfLMzrA== X-Received: by 2002:a62:7b90:0:b029:1be:9e89:1db5 with SMTP id w138-20020a627b900000b02901be9e891db5mr452126pfc.35.1611348468907; Fri, 22 Jan 2021 12:47:48 -0800 (PST) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:ea88]) by smtp.gmail.com with ESMTPSA id j18sm4092900pfc.99.2021.01.22.12.47.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 12:47:47 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org, Al Viro , Christoph Hellwig Cc: Dave Chinner , Jann Horn , Amir Goldstein , Aleksa Sarai , linux-api@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v7 12/15] btrfs: send: write larger chunks when using stream v2 Date: Fri, 22 Jan 2021 12:46:59 -0800 Message-Id: <984c54fe2526fe8a6fad01ae7c28a0efccb2c99d.1611346574.git.osandov@fb.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Omar Sandoval The length field of the send stream TLV header is 16 bits. This means that the maximum amount of data that can be sent for one write is 64k minus one. However, encoded writes must be able to send the maximum compressed extent (128k) in one command. To support this, send stream version 2 encodes the DATA attribute differently: it has no length field, and the length is implicitly up to the end of containing command (which has a 32-bit length field). Although this is necessary for encoded writes, normal writes can benefit from it, too. For v2, let's bump up the send buffer to the maximum compressed extent size plus 16k for the other metadata (144k total). Since this will most likely be vmalloc'd (and always will be after the next commit), we round it up to the next page since we might as well use the rest of the page on systems with >16k pages. Signed-off-by: Omar Sandoval --- fs/btrfs/send.c | 34 ++++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index d07570588a16..98948568017c 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -4915,14 +4915,27 @@ static inline u64 max_send_read_size(const struct send_ctx *sctx) static int put_data_header(struct send_ctx *sctx, u32 len) { - struct btrfs_tlv_header *hdr; + if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) { + /* + * In v2, the data attribute header doesn't include a length; it + * is implicitly to the end of the command. + */ + if (sctx->send_max_size - sctx->send_size < 2 + len) + return -EOVERFLOW; + put_unaligned_le16(BTRFS_SEND_A_DATA, + sctx->send_buf + sctx->send_size); + sctx->send_size += 2; + } else { + struct btrfs_tlv_header *hdr; - if (sctx->send_max_size - sctx->send_size < sizeof(*hdr) + len) - return -EOVERFLOW; - hdr = (struct btrfs_tlv_header *)(sctx->send_buf + sctx->send_size); - put_unaligned_le16(BTRFS_SEND_A_DATA, &hdr->tlv_type); - put_unaligned_le16(len, &hdr->tlv_len); - sctx->send_size += sizeof(*hdr); + if (sctx->send_max_size - sctx->send_size < sizeof(*hdr) + len) + return -EOVERFLOW; + hdr = (struct btrfs_tlv_header *)(sctx->send_buf + + sctx->send_size); + put_unaligned_le16(BTRFS_SEND_A_DATA, &hdr->tlv_type); + put_unaligned_le16(len, &hdr->tlv_len); + sctx->send_size += sizeof(*hdr); + } return 0; } @@ -7267,7 +7280,12 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg) sctx->clone_roots_cnt = arg->clone_sources_count; - sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1; + if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) { + sctx->send_max_size = ALIGN(SZ_16K + BTRFS_MAX_COMPRESSED, + PAGE_SIZE); + } else { + sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1; + } sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL); if (!sctx->send_buf) { ret = -ENOMEM; From patchwork Fri Jan 22 20:47:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12040615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DEFAC433E0 for ; Fri, 22 Jan 2021 21:11:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4EC5B23A7A for ; Fri, 22 Jan 2021 21:11:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729543AbhAVVLv (ORCPT ); Fri, 22 Jan 2021 16:11:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730487AbhAVUwC (ORCPT ); Fri, 22 Jan 2021 15:52:02 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF8FEC061BD1 for ; Fri, 22 Jan 2021 12:47:52 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id m6so4614539pfm.6 for ; Fri, 22 Jan 2021 12:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q4vux4IYhSQov6A9+C6Myiq5tzHRUo+ynvL8O+spikA=; b=gvca7qDrqZ0Wa1VjQTiJmUBdws/Y0o9mE18jDgN7CDCZaAAUCsb/qFM1f3i1E31Srk ofFEvh30Fe8DM4awZja1cgieHfWVVM/xeKKW2g2psoWHbPzzjCbNHsKb19SHaqiypIxH OKd+8cLL/r4v6pP5i0wnNjTQeN7Izl/Wm4qsgc4wXc9aPgnk7vUARGtzFN+4gq2psJUn yTQ7hp72UNpC9JUQcs7EGSPbMvpKS+isi+/jaAkFR2P75mqqF79Oih6vjVFgAhynJouv 9DUz9kaHdo7eERNVHsTwPD+/q9jT/xLgbEaE0h2uBJUFCkH9lqFssUqXWdV7YMwHfuN4 OSvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q4vux4IYhSQov6A9+C6Myiq5tzHRUo+ynvL8O+spikA=; b=Hepx1PYsYk28PBK4TVjicc0ySDv6L8+g68JbI3rNkQMxPfQYuOqNnHRJkpZFHtW89T pcmgAMUrUrbnA+IndQlCaNSNFiUqSyqMO/BS3pVasnePCko38eCtAKS2W1UBdAiyxiCn Ot2IEN+/dI+kDLrkOFQc5LGrXCZAmInQeb72lQg5Doru/te3R+n1yTqkMty0zhHuiKCU S+5Dm3bGixQ2pWBwLTVO4w36L4ZPdJMnRb0SFpnnVZm47GsFn7cHgOtYNGxjyjOQ7/Wa sFP7EBV7mzb5VuD4G1/qONTHtgc9LFF/VEiWURmDFD6s5sgMy3539y+mn4S6Nxjoguqw Pqww== X-Gm-Message-State: AOAM5326Es0rD1EzgpXtVKlSlfqSpeFCpY95ilgD9d49pOjcTeISxWVR FC6eYIbRKMfSYqWSYrPn0tl7DNuLYPNzDQ== X-Google-Smtp-Source: ABdhPJxi2I6deG2+BmNH0RU39zwTIM5ZJrtytwUwCnMqIqiCxMNYjOOKUVZ+Bbz/HQuEmXpcyrQGKQ== X-Received: by 2002:a63:b550:: with SMTP id u16mr6365796pgo.448.1611348471960; Fri, 22 Jan 2021 12:47:51 -0800 (PST) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:ea88]) by smtp.gmail.com with ESMTPSA id j18sm4092900pfc.99.2021.01.22.12.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 12:47:50 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org, Al Viro , Christoph Hellwig Cc: Dave Chinner , Jann Horn , Amir Goldstein , Aleksa Sarai , linux-api@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v7 13/15] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2 Date: Fri, 22 Jan 2021 12:47:00 -0800 Message-Id: <1f0903a7762e9bf7381584251d0d22c0ae684402.1611346574.git.osandov@fb.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Omar Sandoval For encoded writes, we need the raw pages for reading compressed data directly via a bio. So, replace kvmalloc() with vmap() so we have access to the raw pages. 144k is large enough that it usually gets allocated with vmalloc(), anyways. Signed-off-by: Omar Sandoval --- fs/btrfs/send.c | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 98948568017c..25b1a60a568c 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -81,6 +81,7 @@ struct send_ctx { char *send_buf; u32 send_size; u32 send_max_size; + struct page **send_buf_pages; u64 total_send_size; u64 cmd_send_size[BTRFS_SEND_C_MAX + 1]; u64 flags; /* 'flags' member of btrfs_ioctl_send_args is u64 */ @@ -7203,6 +7204,7 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg) struct btrfs_root *clone_root; struct send_ctx *sctx = NULL; u32 i; + u32 send_buf_num_pages = 0; u64 *clone_sources_tmp = NULL; int clone_sources_to_rollback = 0; size_t alloc_size; @@ -7283,10 +7285,28 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg) if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) { sctx->send_max_size = ALIGN(SZ_16K + BTRFS_MAX_COMPRESSED, PAGE_SIZE); + send_buf_num_pages = sctx->send_max_size >> PAGE_SHIFT; + sctx->send_buf_pages = kcalloc(send_buf_num_pages, + sizeof(*sctx->send_buf_pages), + GFP_KERNEL); + if (!sctx->send_buf_pages) { + send_buf_num_pages = 0; + ret = -ENOMEM; + goto out; + } + for (i = 0; i < send_buf_num_pages; i++) { + sctx->send_buf_pages[i] = alloc_page(GFP_KERNEL); + if (!sctx->send_buf_pages[i]) { + ret = -ENOMEM; + goto out; + } + } + sctx->send_buf = vmap(sctx->send_buf_pages, send_buf_num_pages, + VM_MAP, PAGE_KERNEL); } else { sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1; + sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL); } - sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL); if (!sctx->send_buf) { ret = -ENOMEM; goto out; @@ -7495,7 +7515,16 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg) fput(sctx->send_filp); kvfree(sctx->clone_roots); - kvfree(sctx->send_buf); + if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) { + vunmap(sctx->send_buf); + for (i = 0; i < send_buf_num_pages; i++) { + if (sctx->send_buf_pages[i]) + __free_page(sctx->send_buf_pages[i]); + } + kfree(sctx->send_buf_pages); + } else { + kvfree(sctx->send_buf); + } name_cache_free(sctx); From patchwork Fri Jan 22 20:47:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12040613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 943BAC433DB for ; Fri, 22 Jan 2021 21:11:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6316D23A7A for ; Fri, 22 Jan 2021 21:11:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730318AbhAVVLm (ORCPT ); Fri, 22 Jan 2021 16:11:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730735AbhAVUwK (ORCPT ); Fri, 22 Jan 2021 15:52:10 -0500 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCD97C061D7D for ; Fri, 22 Jan 2021 12:47:55 -0800 (PST) Received: by mail-pj1-x1033.google.com with SMTP id md11so4596845pjb.0 for ; Fri, 22 Jan 2021 12:47:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=umRYvP60XGN4PA/ksAi2n9O9qWnIiTU1aTpnFhsBKiw=; b=0jCP6b35XTcffh56d3e7ntHh+SGn9uGkrLpqVlAaJMshuvc/LdLFRGanTyAFds1yEv eoc5bQI6aaGe15loGSnRMhbmv5gZAOh2pd/LpJ/m91aLkr4jP+mjkl7uT0JL/6T+P6mM 7qXZB7eu9No735BHPEkSj/x3qrBqah0gk6p7LWqBf3nePyohIOX+I/z7m5WioKGUQyrt vb6l50cXPzhXAXVrwLSq2BepMMcmjfP7xw8K4BzatEqTMVfpA3CLdgKSuZrxB/hrlA1O S5o45CE8+P5VdOnmEYewSZvVU8Q1VpAqShOB99G7Wb4AmRX0gQ+f33g/bJZbkg0kYkNa +l+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=umRYvP60XGN4PA/ksAi2n9O9qWnIiTU1aTpnFhsBKiw=; b=cHVtuxlc+j6JEBlLWSSf6ccx2tX/KDqBXcLVRVN8WBkY7HoX+PWAnQn2uS23cIBjnA kQ4lIgG1qaPXFbZp2sMiZSJrdRchz7z3HyQjhTzUrzp5lpfqHVHrXzIWFuqfMxeCY7Wm WdytEMrVRBdixuaW62yQZIVXvjWJegkhMnVlkZoWx4Qn08+Ok5kEohgZQOH2PpqhB9Ad 1Qz4wG3CcLre1yv7Ibf1f5zu1GwxXJMUJDTfUJ2+r9kcPNuWLirzWO4gByBr3jprYOtB +x5yBpo7TONlVkTLoj/dzpR4VpwvrV0wBqC/lIcc4E3p/mTFWIrSvKWYqYwe08UPnb1Q /3aQ== X-Gm-Message-State: AOAM532y6GlPDs7RqhZnu8jBHfbNYDD5U7Gs4HqMdv0oFdEWghGf+tkT fNlh7PM/2moBES2rtT3f7ZWcaQIPPJqZAw== X-Google-Smtp-Source: ABdhPJxP6YDr449euHflBUANEA8+njyrPlWeZwrhM+2Z8jYL6hE0YksQPlDYhXhh6S6fsw3FMxG6Qg== X-Received: by 2002:a17:90a:ae13:: with SMTP id t19mr7646868pjq.52.1611348474580; Fri, 22 Jan 2021 12:47:54 -0800 (PST) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:ea88]) by smtp.gmail.com with ESMTPSA id j18sm4092900pfc.99.2021.01.22.12.47.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 12:47:53 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org, Al Viro , Christoph Hellwig Cc: Dave Chinner , Jann Horn , Amir Goldstein , Aleksa Sarai , linux-api@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v7 14/15] btrfs: send: send compressed extents with encoded writes Date: Fri, 22 Jan 2021 12:47:01 -0800 Message-Id: <3822ff837813fc6afcf6d37a45accf2cb41a0923.1611346574.git.osandov@fb.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Omar Sandoval Now that all of the pieces are in place, we can use the ENCODED_WRITE command to send compressed extents when appropriate. Signed-off-by: Omar Sandoval --- fs/btrfs/ctree.h | 4 + fs/btrfs/inode.c | 6 +- fs/btrfs/send.c | 230 +++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 220 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 33a08ab5cb0e..402ffdce81ac 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3157,6 +3157,10 @@ int btrfs_run_delalloc_range(struct btrfs_inode *inode, struct page *locked_page int btrfs_writepage_cow_fixup(struct page *page, u64 start, u64 end); void btrfs_writepage_endio_finish_ordered(struct page *page, u64 start, u64 end, int uptodate); +int encoded_iov_compression_from_btrfs(unsigned int compress_type); +int btrfs_encoded_read_regular_fill_pages(struct inode *inode, u64 offset, + u64 disk_io_size, + struct page **pages); ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter); ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, struct encoded_iov *encoded); diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index c2fe76f57bf5..3c1c8879a9e1 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -9984,7 +9984,7 @@ void btrfs_set_range_writeback(struct extent_io_tree *tree, u64 start, u64 end) } } -static int encoded_iov_compression_from_btrfs(unsigned int compress_type) +int encoded_iov_compression_from_btrfs(unsigned int compress_type) { switch (compress_type) { case BTRFS_COMPRESS_NONE: @@ -10190,8 +10190,8 @@ static void btrfs_encoded_read_endio(struct bio *bio) bio_put(bio); } -static int btrfs_encoded_read_regular_fill_pages(struct inode *inode, u64 offset, - u64 disk_io_size, struct page **pages) +int btrfs_encoded_read_regular_fill_pages(struct inode *inode, u64 offset, + u64 disk_io_size, struct page **pages) { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); struct btrfs_encoded_read_private priv = { diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 25b1a60a568c..7516eba701af 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -595,6 +595,7 @@ static int tlv_put(struct send_ctx *sctx, u16 attr, const void *data, int len) return tlv_put(sctx, attr, &__tmp, sizeof(__tmp)); \ } +TLV_PUT_DEFINE_INT(32) TLV_PUT_DEFINE_INT(64) static int tlv_put_string(struct send_ctx *sctx, u16 attr, @@ -5213,16 +5214,211 @@ static int send_hole(struct send_ctx *sctx, u64 end) return ret; } -static int send_extent_data(struct send_ctx *sctx, - const u64 offset, - const u64 len) +static int send_encoded_inline_extent(struct send_ctx *sctx, + struct btrfs_path *path, u64 offset, + u64 len) { + struct btrfs_root *root = sctx->send_root; + struct btrfs_fs_info *fs_info = root->fs_info; + struct inode *inode; + struct fs_path *p; + struct extent_buffer *leaf = path->nodes[0]; + struct btrfs_key key; + struct btrfs_file_extent_item *ei; + u64 ram_bytes; + size_t inline_size; + int ret; + + inode = btrfs_iget(fs_info->sb, sctx->cur_ino, root); + if (IS_ERR(inode)) + return PTR_ERR(inode); + + p = fs_path_alloc(); + if (!p) { + ret = -ENOMEM; + goto out; + } + + ret = begin_cmd(sctx, BTRFS_SEND_C_ENCODED_WRITE); + if (ret < 0) + goto out; + + ret = get_cur_path(sctx, sctx->cur_ino, sctx->cur_inode_gen, p); + if (ret < 0) + goto out; + + btrfs_item_key_to_cpu(leaf, &key, path->slots[0]); + ei = btrfs_item_ptr(leaf, path->slots[0], + struct btrfs_file_extent_item); + ram_bytes = btrfs_file_extent_ram_bytes(leaf, ei); + inline_size = btrfs_file_extent_inline_item_len(leaf, + btrfs_item_nr(path->slots[0])); + + TLV_PUT_PATH(sctx, BTRFS_SEND_A_PATH, p); + TLV_PUT_U64(sctx, BTRFS_SEND_A_FILE_OFFSET, offset); + TLV_PUT_U64(sctx, BTRFS_SEND_A_UNENCODED_FILE_LEN, + min(key.offset + ram_bytes - offset, len)); + TLV_PUT_U64(sctx, BTRFS_SEND_A_UNENCODED_LEN, ram_bytes); + TLV_PUT_U64(sctx, BTRFS_SEND_A_UNENCODED_OFFSET, offset - key.offset); + ret = encoded_iov_compression_from_btrfs( + btrfs_file_extent_compression(leaf, ei)); + if (ret < 0) + goto out; + TLV_PUT_U32(sctx, BTRFS_SEND_A_COMPRESSION, ret); + TLV_PUT_U32(sctx, BTRFS_SEND_A_ENCRYPTION, 0); + + ret = put_data_header(sctx, inline_size); + if (ret < 0) + goto out; + read_extent_buffer(leaf, sctx->send_buf + sctx->send_size, + btrfs_file_extent_inline_start(ei), inline_size); + sctx->send_size += inline_size; + + ret = send_cmd(sctx); + +tlv_put_failure: +out: + fs_path_free(p); + iput(inode); + return ret; +} + +static int send_encoded_extent(struct send_ctx *sctx, struct btrfs_path *path, + u64 offset, u64 len) +{ + struct btrfs_root *root = sctx->send_root; + struct btrfs_fs_info *fs_info = root->fs_info; + struct inode *inode; + struct fs_path *p; + struct extent_buffer *leaf = path->nodes[0]; + struct btrfs_key key; + struct btrfs_file_extent_item *ei; + u64 block_start; + u64 block_len; + u32 data_offset; + struct btrfs_cmd_header *hdr; + u32 crc; + int ret; + + inode = btrfs_iget(fs_info->sb, sctx->cur_ino, root); + if (IS_ERR(inode)) + return PTR_ERR(inode); + + p = fs_path_alloc(); + if (!p) { + ret = -ENOMEM; + goto out; + } + + ret = begin_cmd(sctx, BTRFS_SEND_C_ENCODED_WRITE); + if (ret < 0) + goto out; + + ret = get_cur_path(sctx, sctx->cur_ino, sctx->cur_inode_gen, p); + if (ret < 0) + goto out; + + btrfs_item_key_to_cpu(leaf, &key, path->slots[0]); + ei = btrfs_item_ptr(leaf, path->slots[0], + struct btrfs_file_extent_item); + block_start = btrfs_file_extent_disk_bytenr(leaf, ei); + block_len = btrfs_file_extent_disk_num_bytes(leaf, ei); + + TLV_PUT_PATH(sctx, BTRFS_SEND_A_PATH, p); + TLV_PUT_U64(sctx, BTRFS_SEND_A_FILE_OFFSET, offset); + TLV_PUT_U64(sctx, BTRFS_SEND_A_UNENCODED_FILE_LEN, + min(key.offset + btrfs_file_extent_num_bytes(leaf, ei) - offset, + len)); + TLV_PUT_U64(sctx, BTRFS_SEND_A_UNENCODED_LEN, + btrfs_file_extent_ram_bytes(leaf, ei)); + TLV_PUT_U64(sctx, BTRFS_SEND_A_UNENCODED_OFFSET, + offset - key.offset + btrfs_file_extent_offset(leaf, ei)); + ret = encoded_iov_compression_from_btrfs( + btrfs_file_extent_compression(leaf, ei)); + if (ret < 0) + goto out; + TLV_PUT_U32(sctx, BTRFS_SEND_A_COMPRESSION, ret); + TLV_PUT_U32(sctx, BTRFS_SEND_A_ENCRYPTION, 0); + + ret = put_data_header(sctx, block_len); + if (ret < 0) + goto out; + + data_offset = ALIGN(sctx->send_size, PAGE_SIZE); + if (data_offset > sctx->send_max_size || + sctx->send_max_size - data_offset < block_len) { + ret = -EOVERFLOW; + goto out; + } + + ret = btrfs_encoded_read_regular_fill_pages(inode, block_start, + block_len, + sctx->send_buf_pages + + (data_offset >> PAGE_SHIFT)); + if (ret) + goto out; + + hdr = (struct btrfs_cmd_header *)sctx->send_buf; + hdr->len = cpu_to_le32(sctx->send_size + block_len - sizeof(*hdr)); + hdr->crc = 0; + crc = btrfs_crc32c(0, sctx->send_buf, sctx->send_size); + crc = btrfs_crc32c(crc, sctx->send_buf + data_offset, block_len); + hdr->crc = cpu_to_le32(crc); + + ret = write_buf(sctx->send_filp, sctx->send_buf, sctx->send_size, + &sctx->send_off); + if (!ret) { + ret = write_buf(sctx->send_filp, sctx->send_buf + data_offset, + block_len, &sctx->send_off); + } + sctx->total_send_size += sctx->send_size + block_len; + sctx->cmd_send_size[le16_to_cpu(hdr->cmd)] += + sctx->send_size + block_len; + sctx->send_size = 0; + +tlv_put_failure: +out: + fs_path_free(p); + iput(inode); + return ret; +} + +static int send_extent_data(struct send_ctx *sctx, struct btrfs_path *path, + const u64 offset, const u64 len) +{ + struct extent_buffer *leaf = path->nodes[0]; + struct btrfs_file_extent_item *ei; u64 read_size = max_send_read_size(sctx); u64 sent = 0; if (sctx->flags & BTRFS_SEND_FLAG_NO_FILE_DATA) return send_update_extent(sctx, offset, len); + ei = btrfs_item_ptr(leaf, path->slots[0], + struct btrfs_file_extent_item); + if ((sctx->flags & BTRFS_SEND_FLAG_COMPRESSED) && + btrfs_file_extent_compression(leaf, ei) != BTRFS_COMPRESS_NONE) { + bool is_inline = (btrfs_file_extent_type(leaf, ei) == + BTRFS_FILE_EXTENT_INLINE); + + /* + * Send the compressed extent unless the compressed data is + * larger than the decompressed data. This can happen if we're + * not sending the entire extent, either because it has been + * partially overwritten/truncated or because this is a part of + * the extent that we couldn't clone in clone_range(). + */ + if (is_inline && + btrfs_file_extent_inline_item_len(leaf, + btrfs_item_nr(path->slots[0])) <= len) { + return send_encoded_inline_extent(sctx, path, offset, + len); + } else if (!is_inline && + btrfs_file_extent_disk_num_bytes(leaf, ei) <= len) { + return send_encoded_extent(sctx, path, offset, len); + } + } + while (sent < len) { u64 size = min(len - sent, read_size); int ret; @@ -5293,12 +5489,9 @@ static int send_capabilities(struct send_ctx *sctx) return ret; } -static int clone_range(struct send_ctx *sctx, - struct clone_root *clone_root, - const u64 disk_byte, - u64 data_offset, - u64 offset, - u64 len) +static int clone_range(struct send_ctx *sctx, struct btrfs_path *dst_path, + struct clone_root *clone_root, const u64 disk_byte, + u64 data_offset, u64 offset, u64 len) { struct btrfs_path *path; struct btrfs_key key; @@ -5322,7 +5515,7 @@ static int clone_range(struct send_ctx *sctx, */ if (clone_root->offset == 0 && len == sctx->send_root->fs_info->sectorsize) - return send_extent_data(sctx, offset, len); + return send_extent_data(sctx, dst_path, offset, len); path = alloc_path_for_send(); if (!path) @@ -5419,7 +5612,8 @@ static int clone_range(struct send_ctx *sctx, if (hole_len > len) hole_len = len; - ret = send_extent_data(sctx, offset, hole_len); + ret = send_extent_data(sctx, dst_path, offset, + hole_len); if (ret < 0) goto out; @@ -5492,14 +5686,16 @@ static int clone_range(struct send_ctx *sctx, if (ret < 0) goto out; } - ret = send_extent_data(sctx, offset + slen, + ret = send_extent_data(sctx, dst_path, + offset + slen, clone_len - slen); } else { ret = send_clone(sctx, offset, clone_len, clone_root); } } else { - ret = send_extent_data(sctx, offset, clone_len); + ret = send_extent_data(sctx, dst_path, offset, + clone_len); } if (ret < 0) @@ -5531,7 +5727,7 @@ static int clone_range(struct send_ctx *sctx, } if (len > 0) - ret = send_extent_data(sctx, offset, len); + ret = send_extent_data(sctx, dst_path, offset, len); else ret = 0; out: @@ -5562,10 +5758,10 @@ static int send_write_or_clone(struct send_ctx *sctx, struct btrfs_file_extent_item); disk_byte = btrfs_file_extent_disk_bytenr(path->nodes[0], ei); data_offset = btrfs_file_extent_offset(path->nodes[0], ei); - ret = clone_range(sctx, clone_root, disk_byte, data_offset, - offset, end - offset); + ret = clone_range(sctx, path, clone_root, disk_byte, + data_offset, offset, end - offset); } else { - ret = send_extent_data(sctx, offset, end - offset); + ret = send_extent_data(sctx, path, offset, end - offset); } sctx->cur_inode_next_write_offset = end; return ret; From patchwork Fri Jan 22 20:47:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12040577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1E3AC433E6 for ; Fri, 22 Jan 2021 21:07:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB10B23B42 for ; Fri, 22 Jan 2021 21:07:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730265AbhAVUxs (ORCPT ); Fri, 22 Jan 2021 15:53:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730791AbhAVUwN (ORCPT ); Fri, 22 Jan 2021 15:52:13 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D91C6C0698C0 for ; Fri, 22 Jan 2021 12:47:57 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id g3so3988872plp.2 for ; Fri, 22 Jan 2021 12:47:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iHS144DxLbBMpm965BQLT7zCwBiEuqsfigSMglTrs0I=; b=eEe3KzBJ5cBvVQTFc/Fat4nX2EbBqXLkGs10fIv31CgraFSVLOvQ4nDPq7BCySkz87 +pp73OUQ0pHOxptR0zCs1Ef541bOtFyuwSZ6uzkySCys3RldTYa1HYZf7Rop363g3Q7T dIzGL0Xz0ok4Eq5uMjVoXJRvDyFPe5aanvR9G5n/yOGVcK8Pm4pheyn/1q0O462g/nRn kRUzvAgZMD0GclNO8FZHNBKn5tWfonGJr0qTKbH6vxjx0bDQzqVHvDCRyjmlM3uLX2nA yTsM14xJB+21a6R8QmR9UJXiDMJw7hHZN4lbzkHhggT63+nXNOKu7e0sDE5B/9dM4fU7 jHqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iHS144DxLbBMpm965BQLT7zCwBiEuqsfigSMglTrs0I=; b=Nm+dNQpJ0XfzcGw21I3Wot7S3jmHs+ZSRiMBfDKJizW0qT8ydpO7b65rL0BhZT9lEy 7W9IvBOJyeY9Ss9ivuwD4BlkWUlNfAHvkAKCpIWbyDGPU/P4Rr91Q1IJiCkI1lwTRRAI UGdflgwvpDntamw5AHjSbh5s8qHB77rBCr+od8tACbAT4RJsFHyd3VgKUEnrHL+2m7j2 KlTSMqWcp5Lvwcw557ergBvqRrxidXrCW7DlUu+i5RvzKGN/J6J3ARY534V3KFCDLG4B Bm0pxVfNkpIPHr30FjwHreWOmfVa8Ua0f78C9GZ/Wk6ezZ0MwL+M9bdd7FgPx4Hib8m+ ZB7A== X-Gm-Message-State: AOAM531NX+FWJUoa+xW41oLf5YdAsc8sKstp9NHbqSln++Sdl/BSAWmO B9DafS5p31E9RPQe3EPECp7IugLirXKP2g== X-Google-Smtp-Source: ABdhPJw0CDTLukWVGH2ygrhTfT70j0x9Uu65xoNtFS+yDtbeNYP2KeSbSmofvhiz+rOQCKfzxeNKBA== X-Received: by 2002:a17:902:c284:b029:df:c0d8:6b7 with SMTP id i4-20020a170902c284b02900dfc0d806b7mr6420291pld.34.1611348476863; Fri, 22 Jan 2021 12:47:56 -0800 (PST) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:ea88]) by smtp.gmail.com with ESMTPSA id j18sm4092900pfc.99.2021.01.22.12.47.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 12:47:55 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org, Al Viro , Christoph Hellwig Cc: Dave Chinner , Jann Horn , Amir Goldstein , Aleksa Sarai , linux-api@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v7 15/15] btrfs: send: enable support for stream v2 and compressed writes Date: Fri, 22 Jan 2021 12:47:02 -0800 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Omar Sandoval Now that the new support is implemented, allow the ioctl to accept the flags and update the version in sysfs. Signed-off-by: Omar Sandoval --- fs/btrfs/send.c | 10 +++++++++- fs/btrfs/send.h | 2 +- include/uapi/linux/btrfs.h | 4 +++- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 7516eba701af..cb824d1271fa 100644 --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -671,7 +671,10 @@ static int send_header(struct send_ctx *sctx) struct btrfs_stream_header hdr; strcpy(hdr.magic, BTRFS_SEND_STREAM_MAGIC); - hdr.version = cpu_to_le32(BTRFS_SEND_STREAM_VERSION); + if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) + hdr.version = cpu_to_le32(2); + else + hdr.version = cpu_to_le32(1); return write_buf(sctx->send_filp, &hdr, sizeof(hdr), &sctx->send_off); @@ -7446,6 +7449,11 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg) ret = -EINVAL; goto out; } + if ((arg->flags & BTRFS_SEND_FLAG_COMPRESSED) && + !(arg->flags & BTRFS_SEND_FLAG_STREAM_V2)) { + ret = -EINVAL; + goto out; + } sctx = kzalloc(sizeof(struct send_ctx), GFP_KERNEL); if (!sctx) { diff --git a/fs/btrfs/send.h b/fs/btrfs/send.h index 9f4f7b96b1eb..9c83e14a43b2 100644 --- a/fs/btrfs/send.h +++ b/fs/btrfs/send.h @@ -10,7 +10,7 @@ #include "ctree.h" #define BTRFS_SEND_STREAM_MAGIC "btrfs-stream" -#define BTRFS_SEND_STREAM_VERSION 1 +#define BTRFS_SEND_STREAM_VERSION 2 /* * In send stream v1, no command is larger than 64k. In send stream v2, no limit diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index 93aa0932234e..b12a9a1a106c 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -786,7 +786,9 @@ struct btrfs_ioctl_received_subvol_args { #define BTRFS_SEND_FLAG_MASK \ (BTRFS_SEND_FLAG_NO_FILE_DATA | \ BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \ - BTRFS_SEND_FLAG_OMIT_END_CMD) + BTRFS_SEND_FLAG_OMIT_END_CMD | \ + BTRFS_SEND_FLAG_STREAM_V2 | \ + BTRFS_SEND_FLAG_COMPRESSED) struct btrfs_ioctl_send_args { __s64 send_fd; /* in */