From patchwork Tue Aug 17 21:06:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12442047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBD85C43214 for ; Tue, 17 Aug 2021 21:07:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CAC1260FE6 for ; Tue, 17 Aug 2021 21:07:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235044AbhHQVIT (ORCPT ); Tue, 17 Aug 2021 17:08:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234881AbhHQVIP (ORCPT ); Tue, 17 Aug 2021 17:08:15 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8A47C0613A3 for ; Tue, 17 Aug 2021 14:07:36 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id m24-20020a17090a7f98b0290178b1a81700so864341pjl.4 for ; Tue, 17 Aug 2021 14:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xIa62ruoOEIIoBpJ21XTPbvc/+VgQCZWkH1izv0jlfw=; b=OYFBqtUPkOX9V6qgzaZ5e7TRdqQiGUjX7hBq8U/1e2UBFKRXQpbrJQO8GgtAniERDV A9qGF0VKanhnZimw3q1CY4IBDikGXGtwZ3pgQXa/aVN82D5x2E5qFFlqylIFeiLiXVeW 0py1Qja5lEG9rBqyAPzfFb5WMDFWX7N2dnXTRkNNqV7iYNv9C6grmpEp5Hrg1sZE2ReY ZGYtI2P4ESAO6Y+bzxbQxnJtZmukLlY/mJMlbBjtRDwqIPgTmWV/ZSzCyiqqCKQ4V72U uPFBVJmmPhWd0131zfLK3F2iLiq+Yay6oPB9848Kaot+/fqR/mbL/b2WZZ3uxhy/8Xar PvUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xIa62ruoOEIIoBpJ21XTPbvc/+VgQCZWkH1izv0jlfw=; b=QApy9d7wiv1s8xl5w8HkvBcFCPym3A41R7H3oyeq+KQenzqelbu61fD4Q9hUPJ268Y X6j5V7PlRdQNTTghe4MXuxwivYS2KfnjT9apUIgz5MhDbnPdxqJNwSW/Fcbg22KNwhFP kAF8NQq0dCbByjBx+TKT+NQf6wtIZ6qDQH8PuYhD93T50RZ/o89UlFCj4NSIFRyJwfUS HpBQFwoKNqJAYaVbNBlx1Q5MwjGtvzZnXu1oTnqinK/TFz1WbsnPjbdYS6Mqzm5CtKZb o4uqhV5zr5ZNIKOCHc4VdHEZ1mK8EhbGQQdUjni/lUqwMouM3dyWAmV7Q6CaKqHhbhgL LjzQ== X-Gm-Message-State: AOAM532NU+IJ/7UMsSpMtWrxNXfCxvLDmFsTAHfio16gAQgAxHhHLprz IrHjrtC4dPAPGG4FKiqJGfUlCDzxP7ydkw== X-Google-Smtp-Source: ABdhPJykc7o8Iroj31v5Fqt+t6qtHMxN2M49HO4EkV1Tg+WtkinsQIfZYEWF5Ww4oAGGd3fXWd4gXA== X-Received: by 2002:a63:5b01:: with SMTP id p1mr5296769pgb.250.1629234456092; Tue, 17 Aug 2021 14:07:36 -0700 (PDT) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:df70]) by smtp.gmail.com with ESMTPSA id c9sm4205194pgq.58.2021.08.17.14.07.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 14:07:35 -0700 (PDT) From: Omar Sandoval To: linux-btrfs@vger.kernel.org Cc: kernel-team@fb.com, linux-fsdevel@vger.kernel.org, Al Viro , Linus Torvalds , linux-api@vger.kernel.org Subject: [PATCH v10 01/10] btrfs-progs: receive: support v2 send stream larger tlv_len Date: Tue, 17 Aug 2021 14:06:47 -0700 Message-Id: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Boris Burkov An encoded extent can be up to 128K in length, which exceeds the largest value expressible by the current send stream format's 16 bit tlv_len field. Since encoded writes cannot be split into multiple writes by btrfs send, the send stream format must change to accommodate encoded writes. Supporting this changed format requires retooling how we store the commands we have processed. Since we can no longer use btrfs_tlv_header to describe every attribute, we define a new struct btrfs_send_attribute which has a 32 bit length field, and use that to store the attribute information needed for receive processing. This is transparent to users of the various TLV_GET macros. Signed-off-by: Boris Burkov --- common/send-stream.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/common/send-stream.c b/common/send-stream.c index a0c52f79..cd5aa311 100644 --- a/common/send-stream.c +++ b/common/send-stream.c @@ -24,13 +24,23 @@ #include "crypto/crc32c.h" #include "common/utils.h" +struct btrfs_send_attribute { + u16 tlv_type; + /* + * Note: in btrfs_tlv_header, this is __le16, but we need 32 bits for + * attributes with file data as of version 2 of the send stream format + */ + u32 tlv_len; + char *data; +}; + struct btrfs_send_stream { char read_buf[BTRFS_SEND_BUF_SIZE]; int fd; int cmd; struct btrfs_cmd_header *cmd_hdr; - struct btrfs_tlv_header *cmd_attrs[BTRFS_SEND_A_MAX + 1]; + struct btrfs_send_attribute cmd_attrs[BTRFS_SEND_A_MAX + 1]; u32 version; /* @@ -152,6 +162,7 @@ static int read_cmd(struct btrfs_send_stream *sctx) struct btrfs_tlv_header *tlv_hdr; u16 tlv_type; u16 tlv_len; + struct btrfs_send_attribute *send_attr; tlv_hdr = (struct btrfs_tlv_header *)data; tlv_type = le16_to_cpu(tlv_hdr->tlv_type); @@ -164,10 +175,15 @@ static int read_cmd(struct btrfs_send_stream *sctx) goto out; } - sctx->cmd_attrs[tlv_type] = tlv_hdr; + send_attr = &sctx->cmd_attrs[tlv_type]; + send_attr->tlv_type = tlv_type; + send_attr->tlv_len = tlv_len; + pos += sizeof(*tlv_hdr); + data += sizeof(*tlv_hdr); - data += sizeof(*tlv_hdr) + tlv_len; - pos += sizeof(*tlv_hdr) + tlv_len; + send_attr->data = data; + pos += send_attr->tlv_len; + data += send_attr->tlv_len; } sctx->cmd = cmd; @@ -180,7 +196,7 @@ out: static int tlv_get(struct btrfs_send_stream *sctx, int attr, void **data, int *len) { int ret; - struct btrfs_tlv_header *hdr; + struct btrfs_send_attribute *send_attr; if (attr <= 0 || attr > BTRFS_SEND_A_MAX) { error("invalid attribute requested, attr = %d", attr); @@ -188,15 +204,15 @@ static int tlv_get(struct btrfs_send_stream *sctx, int attr, void **data, int *l goto out; } - hdr = sctx->cmd_attrs[attr]; - if (!hdr) { + send_attr = &sctx->cmd_attrs[attr]; + if (!send_attr->data) { error("attribute %d requested but not present", attr); ret = -ENOENT; goto out; } - *len = le16_to_cpu(hdr->tlv_len); - *data = hdr + 1; + *len = send_attr->tlv_len; + *data = send_attr->data; ret = 0;