From patchwork Tue Aug 17 21:06:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 12442033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F41B5C432BE for ; Tue, 17 Aug 2021 21:07:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DC83261029 for ; Tue, 17 Aug 2021 21:07:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234364AbhHQVIA (ORCPT ); Tue, 17 Aug 2021 17:08:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233836AbhHQVH5 (ORCPT ); Tue, 17 Aug 2021 17:07:57 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63827C06179A for ; Tue, 17 Aug 2021 14:07:24 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id n12so498931plf.4 for ; Tue, 17 Aug 2021 14:07:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L0xQZa3NHRv/YJBHwdMlr3YdmFg51MKltcZezbvVR+A=; b=F7nxRaeK17B8wZHPWrLoLECFPBXswXfhoC6QQqy6hQewLDDini3L0R/KHqp6XetxhF mcqJXWF7ul7VNSExTh3aXSOyu/S2boc0fWtNv087a7xOce+F5f4tnVbZiBV21OA1lHbA I5pxVyRdVI5Xt5FUTSHEb+cNmG7I6odbe6BKbH1PyA8iKbyQpdZO1P87NFsA4F7UGfuY ZVzF/ctXRbWhzbCpo1Nmz5AbUd1Eose8ibospkHcYzF8hGi1lyN9ejm83k07Ei+1tirV xFcWio41Aqfz0vBIP5BbcKBv4lmhgk9IPaZkSu81JdqOiIk2nOikPlg1fLXkjUDJroN9 BzrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L0xQZa3NHRv/YJBHwdMlr3YdmFg51MKltcZezbvVR+A=; b=ZCuEJoTMeNms7cUWDoTOvkOAofN5O/9gdamAj9hPbLMqsCpJv/9TbqdeEPxPiJjG1j dv8su7g0e5k0tYk3AzKQEHaASNfajPTtv9QIKv2zm+oYmf1lyxUneoLi9kILzLW1JPbd ImyFO/bkC4M2nzikuMcdNBZo3TyCoLSrHblRSt+Pi6/A2jqBtyjP31TD6dLfmwIGwjt7 BJ2WT2yrVOnqZKn4QNEJX+g0vK4Kg8CbwsdFmJhlV0Qky2tDBXLrpfOo8yW4Pr/Ary5o DWc0FF34kkSIYgMOBrP7aXrQM9Z6gPHNdDrzfaB0BjicGWChweOWGix6wz/8E+4DqScd wkNg== X-Gm-Message-State: AOAM531kcXkgO1PGNoK8oYYzRTzbvOLSeCJfz3d8VzT8B25y7P58xqlH t6nFKVushjhwolLWLs/X4EGGxz5BXVOE0w== X-Google-Smtp-Source: ABdhPJyR650nd7twpnZ9V46Hc/h8s6f6l4P8YpbGlGdt+hLsID4ZgAx3OAFvnQ2tpXSkrBSArOgAog== X-Received: by 2002:a05:6a00:248a:b029:3e0:9be4:963f with SMTP id c10-20020a056a00248ab02903e09be4963fmr5578748pfv.29.1629234443600; Tue, 17 Aug 2021 14:07:23 -0700 (PDT) Received: from relinquished.tfbnw.net ([2620:10d:c090:400::5:df70]) by smtp.gmail.com with ESMTPSA id c9sm4205194pgq.58.2021.08.17.14.07.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Aug 2021 14:07:23 -0700 (PDT) From: Omar Sandoval To: linux-btrfs@vger.kernel.org Cc: kernel-team@fb.com, linux-fsdevel@vger.kernel.org, Al Viro , Linus Torvalds , linux-api@vger.kernel.org Subject: [PATCH v10 07/14] btrfs: add definitions + documentation for encoded I/O ioctls Date: Tue, 17 Aug 2021 14:06:39 -0700 Message-Id: <9bd601f8c5494342d8c7d8aaa86aa815c2118173.1629234193.git.osandov@fb.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org From: Omar Sandoval In order to allow sending and receiving compressed data without decompressing it, we need an interface to write pre-compressed data directly to the filesystem and the matching interface to read compressed data without decompressing it. This adds the definitions for ioctls to do that and detailed explanations of how to use them. Signed-off-by: Omar Sandoval --- include/uapi/linux/btrfs.h | 132 +++++++++++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index d7d3cfead056..95da52955894 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -861,6 +861,134 @@ struct btrfs_ioctl_get_subvol_rootref_args { __u8 align[7]; }; +/* + * Data and metadata for an encoded read or write. + * + * Encoded I/O bypasses any encoding automatically done by the filesystem (e.g., + * compression). This can be used to read the compressed contents of a file or + * write pre-compressed data directly to a file. + * + * BTRFS_IOC_ENCODED_READ and BTRFS_IOC_ENCODED_WRITE are essentially + * preadv/pwritev with additional metadata about how the data is encoded and the + * size of the unencoded data. + * + * BTRFS_IOC_ENCODED_READ fills the given iovecs with the encoded data, fills + * the metadata fields, and returns the size of the encoded data. It reads one + * extent per call. It can also read data which is not encoded. + * + * BTRFS_IOC_ENCODED_WRITE uses the metadata fields, writes the encoded data + * from the iovecs, and returns the size of the encoded data. Note that the + * encoded data is not validated when it is written; if it is not valid (e.g., + * it cannot be decompressed), then a subsequent read may return an error. + * + * Since the filesystem page cache contains decoded data, encoded I/O bypasses + * the page cache. Encoded I/O requires CAP_SYS_ADMIN. + */ +struct btrfs_ioctl_encoded_io_args { + /* Input parameters for both reads and writes. */ + + /* + * iovecs containing encoded data. + * + * For reads, if the size of the encoded data is larger than the sum of + * iov[n].iov_len for 0 <= n < iovcnt, then the ioctl fails with + * ENOBUFS. + * + * For writes, the size of the encoded data is the sum of iov[n].iov_len + * for 0 <= n < iovcnt. This must be less than 128 KiB (this limit may + * increase in the future). This must also be less than or equal to + * unencoded_len. + */ + const struct iovec __user *iov; + /* Number of iovecs. */ + unsigned long iovcnt; + /* + * Offset in file. + * + * For writes, must be aligned to the sector size of the filesystem. + */ + __s64 offset; + /* Currently must be zero. */ + __u64 flags; + + /* + * For reads, the following members are filled in with the metadata for + * the encoded data. + * For writes, the following members must be set to the metadata for the + * encoded data. + */ + + /* + * Length of the data in the file. + * + * Must be less than or equal to unencoded_len - unencoded_offset. For + * writes, must be aligned to the sector size of the filesystem unless + * the data ends at or beyond the current end of the file. + */ + __u64 len; + /* + * Length of the unencoded (i.e., decrypted and decompressed) data. + * + * For writes, must be no more than 128 KiB (this limit may increase in + * the future). If the unencoded data is actually longer than + * unencoded_len, then it is truncated; if it is shorter, then it is + * extended with zeroes. + */ + __u64 unencoded_len; + /* + * Offset from the first byte of the unencoded data to the first byte of + * logical data in the file. + * + * Must be less than unencoded_len. + */ + __u64 unencoded_offset; + /* + * BTRFS_ENCODED_IO_COMPRESSION_* type. + * + * For writes, must not be BTRFS_ENCODED_IO_COMPRESSION_NONE. + */ + __u32 compression; + /* Currently always BTRFS_ENCODED_IO_ENCRYPTION_NONE. */ + __u32 encryption; + /* + * Reserved for future expansion. + * + * For reads, always returned as zero. Users should check for non-zero + * bytes. If there are any, then the kernel has a newer version of this + * structure with additional information that the user definition is + * missing. + * + * For writes, must be zeroed. + */ + __u8 reserved[32]; +}; + +/* Data is not compressed. */ +#define BTRFS_ENCODED_IO_COMPRESSION_NONE 0 +/* Data is compressed as a single zlib stream. */ +#define BTRFS_ENCODED_IO_COMPRESSION_ZLIB 1 +/* + * Data is compressed as a single zstd frame with the windowLog compression + * parameter set to no more than 17. + */ +#define BTRFS_ENCODED_IO_COMPRESSION_ZSTD 2 +/* + * Data is compressed page by page (using the page size indicated by the name of + * the constant) with LZO1X and wrapped in the format documented in + * fs/btrfs/lzo.c. For writes, the compression page size must match the + * filesystem page size. + */ +#define BTRFS_ENCODED_IO_COMPRESSION_LZO_4K 3 +#define BTRFS_ENCODED_IO_COMPRESSION_LZO_8K 4 +#define BTRFS_ENCODED_IO_COMPRESSION_LZO_16K 5 +#define BTRFS_ENCODED_IO_COMPRESSION_LZO_32K 6 +#define BTRFS_ENCODED_IO_COMPRESSION_LZO_64K 7 +#define BTRFS_ENCODED_IO_COMPRESSION_TYPES 8 + +/* Data is not encrypted. */ +#define BTRFS_ENCODED_IO_ENCRYPTION_NONE 0 +#define BTRFS_ENCODED_IO_ENCRYPTION_TYPES 1 + /* Error codes as returned by the kernel */ enum btrfs_err_code { BTRFS_ERROR_DEV_RAID1_MIN_NOT_MET = 1, @@ -989,5 +1117,9 @@ enum btrfs_err_code { struct btrfs_ioctl_ino_lookup_user_args) #define BTRFS_IOC_SNAP_DESTROY_V2 _IOW(BTRFS_IOCTL_MAGIC, 63, \ struct btrfs_ioctl_vol_args_v2) +#define BTRFS_IOC_ENCODED_READ _IOR(BTRFS_IOCTL_MAGIC, 64, \ + struct btrfs_ioctl_encoded_io_args) +#define BTRFS_IOC_ENCODED_WRITE _IOW(BTRFS_IOCTL_MAGIC, 64, \ + struct btrfs_ioctl_encoded_io_args) #endif /* _UAPI_LINUX_BTRFS_H */