[v4,0/9] fs: interface for directly reading/writing compressed data

Message

Omar Sandoval Feb. 28, 2020, 11:13 p.m. UTC

From: Omar Sandoval <[e-mail]>

Hello,

This series adds an API for reading compressed data on a filesystem
without decompressing it as well as support for writing compressed data
directly to the filesystem. As with the previous submissions, I've
included a man page patch describing the API. I have test cases
(including fsstress support) and example programs which I'll send up
once the interface is more or less settled [1].

The main use-case is Btrfs send/receive: currently, when sending data
from one compressed filesystem to another, the sending side decompresses
the data and the receiving side recompresses it before writing it out.
This is wasteful and can be avoided if we can just send and write
compressed extents. The send part will be implemented in a separate
series, as this API can stand alone.

I'm fairly happy with the UAPI and VFS interface now. I'd love for Al
and/or Christoph to take a look at that part. The Btrfs side is mostly
there, just missing read repair.

Patches 1-3 add the VFS support. Patches 4-7 are Btrfs prep patches.
Patch 8 adds Btrfs encoded read support and patch 9 adds Btrfs encoded
write support.

Changes from v3:

- Rebase on v5.6-rc3.
- Disallow extents with a file length greater than the unencoded
  length.
- Drop Btrfs cleanups and fixes that have already been merged.
- Add Nikolay's reviewed-bys.

Please share any comments on the API or implementation. Thanks!

1: https://github.com/osandov/xfstests/tree/rwf-encoded
2: https://lore.kernel.org/linux-btrfs/[e-mail]/

Omar Sandoval (9):
  iov_iter: add copy_struct_from_iter()
  fs: add O_ALLOW_ENCODED open flag
  fs: add RWF_ENCODED for reading/writing compressed data
  btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio()
  btrfs: add ram_bytes and offset to btrfs_ordered_extent
  btrfs: support different disk extent size for delalloc
  btrfs: optionally extend i_size in cow_file_range_inline()
  btrfs: implement RWF_ENCODED reads
  btrfs: implement RWF_ENCODED writes

 Documentation/filesystems/encoded_io.rst |  74 ++
 Documentation/filesystems/index.rst      |   1 +
 arch/alpha/include/uapi/asm/fcntl.h      |   1 +
 arch/parisc/include/uapi/asm/fcntl.h     |   1 +
 arch/sparc/include/uapi/asm/fcntl.h      |   1 +
 fs/btrfs/compression.c                   |  12 +-
 fs/btrfs/compression.h                   |   6 +-
 fs/btrfs/ctree.h                         |   9 +-
 fs/btrfs/delalloc-space.c                |  38 +-
 fs/btrfs/delalloc-space.h                |   4 +-
 fs/btrfs/file-item.c                     |  35 +-
 fs/btrfs/file.c                          |  55 +-
 fs/btrfs/inode.c                         | 873 ++++++++++++++++++++---
 fs/btrfs/ordered-data.c                  |  77 +-
 fs/btrfs/ordered-data.h                  |  18 +-
 fs/btrfs/relocation.c                    |   4 +-
 fs/fcntl.c                               |  10 +-
 fs/namei.c                               |   4 +
 include/linux/fcntl.h                    |   2 +-
 include/linux/fs.h                       |  16 +
 include/linux/uio.h                      |   2 +
 include/uapi/asm-generic/fcntl.h         |   4 +
 include/uapi/linux/fs.h                  |  33 +-
 lib/iov_iter.c                           |  82 +++
 mm/filemap.c                             | 166 ++++-
 25 files changed, 1306 insertions(+), 222 deletions(-)
 create mode 100644 Documentation/filesystems/encoded_io.rst