[GIT,PULL] afs: Preparation for fscache overhaul

Pull-request

git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-netfs-lib-20210426

Message

David Howells April 27, 2021, 12:13 a.m. UTC

Hi Linus,

Here's a set of patches for the AFS filesystem for 5.13 to begin the
process of overhauling the use of the fscache API by AFS and the
introduction of support for features such as Transparent Huge Pages (THPs).

 (1) Add some support for THPs, including using core VM helper functions to
     find details of pages.

 (2) Use the ITER_XARRAY I/O iterator to mediate access to the pagecache as
     this handles THPs and doesn't require allocation of large bvec arrays.

 (3) Delegate address_space read/pre-write I/O methods for AFS to the netfs
     helper library.  A method is provided to the library that allows it to
     issue a read against the server.

     This includes a change in use for PG_fscache (it now indicates a DIO
     write in progress from the marked page), so a number of waits need to
     be deployed for it.

 (4) Split the core AFS writeback function to make it easier to modify in
     future patches to handle writing to the cache.  [This might feasibly
     make more sense moved out into my fscache-iter branch].

I've tested these with "xfstests -g quick" against an AFS volume (xfstests
needs patching to make it work).  With this, AFS without a cache passes all
expected xfstests; with a cache, there's an extra failure, but that's also
there before these patches.  Fixing that probably requires a greater
overhaul (as can be found on my fscache-iter branch, but that's for a later
time).

Thanks should go to Marc Dionne and Jeff Altman of AuriStor for exercising
the patches in their test farm also.


Changes
=======

These patches are dependent on the netfs-lib branch and have been posted in
association with them.  The changes relevant to these patches are:

ver #6:
      Split the afs patches out into their own branch.

ver #5:
      Fixed some review comments from Matthew Wilcox:

      - Better names for wrangling functions for PG_private_2 and
        PG_fscache wrangling functions[3].  Came up with
        {set,end,wait_for}_page_private_2() and aliased these for fscache.

      Moved the taking of/dropping a page ref for the PG_private_2 flag
      into the set and end functions.

ver #4:
      Rebased to v5.12-rc2 and added a bunch of references into individual
      commits.

ver #3:
      Adjusted the functions that unlock and wait for PG_fscache according
      to Linus's suggestion[1].

      Hold a ref on a page when PG_fscache is set as per Linus's
      suggestion[2].

Link: https://lore.kernel.org/r/CAHk-=wh+2gbF7XEjYc=HV9w_2uVzVf7vs60BPz0gFA=[e-mail]/ [1]
Link: https://lore.kernel.org/r/CAHk-=wjgA-74ddehziVk=[e-mail]/ [2]
Link: https://lore.kernel.org/r/[e-mail]/ [3]

References
==========

These patches have been published for review before, firstly as part of a
larger set:

Link: https://lore.kernel.org/r/[e-mail]/

Link: https://lore.kernel.org/r/[e-mail]/
Link: https://lore.kernel.org/r/[e-mail]/
Link: https://lore.kernel.org/r/[e-mail]/

Link: https://lore.kernel.org/r/[e-mail]/

Then as a cut-down set:

Link: https://lore.kernel.org/r/[e-mail]/ # v1
Link: https://lore.kernel.org/r/[e-mail]/ # v2
Link: https://lore.kernel.org/r/[e-mail]/ # v3
Link: https://lore.kernel.org/r/[e-mail]/ # v4
Link: https://lore.kernel.org/r/[e-mail]/ # v5
Link: https://lore.kernel.org/r/[e-mail]/ # v6
Link: https://lore.kernel.org/r/[e-mail] # v7

David
---
The following changes since commit 26aaeffcafe6cbb7c3978fa6ed7555122f8c9f8c:

  fscache, cachefiles: Add alternate API to use kiocb for read/write to cache (2021-04-23 10:14:32 +0100)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-netfs-lib-20210426

for you to fetch changes up to 3003bbd0697b659944237f3459489cb596ba196c:

  afs: Use the netfs_write_begin() helper (2021-04-23 10:17:28 +0100)

----------------------------------------------------------------
AFS: Use the new netfs lib

----------------------------------------------------------------
David Howells (14):
      afs: Disable use of the fscache I/O routines
      afs: Pass page into dirty region helpers to provide THP size
      afs: Print the operation debug_id when logging an unexpected data version
      afs: Move key to afs_read struct
      afs: Don't truncate iter during data fetch
      afs: Log remote unmarshalling errors
      afs: Set up the iov_iter before calling afs_extract_data()
      afs: Use ITER_XARRAY for writing
      afs: Wait on PG_fscache before modifying/releasing a page
      afs: Extract writeback extension into its own function
      afs: Prepare for use of THPs
      afs: Use the fs operation ops to handle FetchData completion
      afs: Use new netfs lib read helper API
      afs: Use the netfs_write_begin() helper

 fs/afs/Kconfig             |   1 +
 fs/afs/dir.c               | 225 +++++++++++-----
 fs/afs/file.c              | 483 +++++++++------------------------
 fs/afs/fs_operation.c      |   4 +-
 fs/afs/fsclient.c          | 108 +++-----
 fs/afs/inode.c             |   7 +-
 fs/afs/internal.h          |  59 ++--
 fs/afs/rxrpc.c             | 150 ++++-------
 fs/afs/write.c             | 657 +++++++++++++++++++++++----------------------
 fs/afs/yfsclient.c         |  82 ++----
 include/net/af_rxrpc.h     |   2 +-
 include/trace/events/afs.h |  74 +++--
 net/rxrpc/recvmsg.c        |   9 +-
 13 files changed, 805 insertions(+), 1056 deletions(-)

Comments

[e-mail] April 27, 2021, 8:32 p.m. UTC | #1

The pull request you sent on Tue, 27 Apr 2021 01:13:49 +0100:

> git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-netfs-lib-20210426

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/fafe1e39ed213221c0bce6b0b31669334368dc97

Thank you!