[v2,62/67] nfs: Convert to new fscache volume/cookie API

Commit Message

David Howells Dec. 9, 2021, 5:09 p.m. UTC

From: Dave Wysochanski <[e-mail]>

Change the nfs filesystem to support fscache's indexing rewrite and
reenable caching in nfs.

The following changes have been made:

 (1) The fscache_netfs struct is no more, and there's no need to register
     the filesystem as a whole.

 (2) The session cookie is now an fscache_volume cookie, allocated with
     fscache_acquire_volume().  That takes three parameters: a string
     representing the "volume" in the index, a string naming the cache to
     use (or NULL) and a u64 that conveys coherency metadata for the
     volume.

     For nfs, I've made it render the volume name string as:

        "nfs,<ver>,<family>,<address>,<port>,<fsidH>,<fsidL>*<,param>[,<uniq>]"

 (3) The fscache_cookie_def is no more and needed information is passed
     directly to fscache_acquire_cookie().  The cache no longer calls back
     into the filesystem, but rather metadata changes are indicated at
     other times.

     fscache_acquire_cookie() is passed the same keying and coherency
     information as before.

 (4) fscache_enable/disable_cookie() have been removed.

     Call fscache_use_cookie() and fscache_unuse_cookie() when a file is
     opened or closed to prevent a cache file from being culled and to keep
     resources to hand that are needed to do I/O.

     Unuse the cookie when a file is opened for writing.  This is gated by
     the NFS_INO_FSCACHE flag on the nfs_inode.

     A better way might be to invalidate it with FSCACHE_INVAL_DIO_WRITE
     which will keep it unused until all open files are closed.

 (5) fscache_invalidate() now needs to be given uptodate auxiliary data and
     a file size.  It also takes a flag to indicate if this was due to a
     DIO write.

 (6) Call nfs_fscache_invalidate() with FSCACHE_INVAL_DIO_WRITE on a file
     to which a DIO write is made.

 (7) Call fscache_note_page_release() from nfs_release_page().

 (8) Use a killable wait in nfs_vm_page_mkwrite() when waiting for
     PG_fscache to be cleared.

 (9) The functions to read and write data to/from the cache are stubbed out
     pending a conversion to use netfslib.

Changes
=======
ver #2:
 - Use gfpflags_allow_blocking() rather than using flag directly.
 - fscache_acquire_volume() now returns errors.
 - Remove NFS_INO_FSCACHE as it's no longer used.
 - Need to unuse a cookie on file-release, not inode-clear.

Signed-off-by: Dave Wysochanski <[e-mail]>
Co-developed-by: David Howells <[e-mail]>
Signed-off-by: David Howells <[e-mail]>
cc: Trond Myklebust <[e-mail]>
cc: Anna Schumaker <[e-mail]>
cc: [e-mail]
cc: [e-mail]
Link: https://lore.kernel.org/r/[e-mail]/ # v1
---

 fs/nfs/Kconfig            |    2 
 fs/nfs/Makefile           |    2 
 fs/nfs/client.c           |    4 
 fs/nfs/direct.c           |    2 
 fs/nfs/file.c             |   13 +
 fs/nfs/fscache-index.c    |  140 ---------------
 fs/nfs/fscache.c          |  434 +++++++++++----------------------------------
 fs/nfs/fscache.h          |  126 ++++---------
 fs/nfs/inode.c            |   11 -
 fs/nfs/nfstrace.h         |    1 
 fs/nfs/super.c            |   28 ++-
 fs/nfs/write.c            |    1 
 include/linux/nfs_fs.h    |    1 
 include/linux/nfs_fs_sb.h |    9 -
 14 files changed, 171 insertions(+), 603 deletions(-)
 delete mode 100644 fs/nfs/fscache-index.c

Comments

David Wysochanski Dec. 11, 2021, 1:22 p.m. UTC | #1

On Thu, Dec 9, 2021 at 12:10 PM David Howells <[e-mail]> wrote:
>
> From: Dave Wysochanski <[e-mail]>
>
> Change the nfs filesystem to support fscache's indexing rewrite and
> reenable caching in nfs.
>
> The following changes have been made:
>
>  (1) The fscache_netfs struct is no more, and there's no need to register
>      the filesystem as a whole.
>
>  (2) The session cookie is now an fscache_volume cookie, allocated with
>      fscache_acquire_volume().  That takes three parameters: a string
>      representing the "volume" in the index, a string naming the cache to
>      use (or NULL) and a u64 that conveys coherency metadata for the
>      volume.
>
>      For nfs, I've made it render the volume name string as:
>
>         "nfs,<ver>,<family>,<address>,<port>,<fsidH>,<fsidL>*<,param>[,<uniq>]"
>
>  (3) The fscache_cookie_def is no more and needed information is passed
>      directly to fscache_acquire_cookie().  The cache no longer calls back
>      into the filesystem, but rather metadata changes are indicated at
>      other times.
>
>      fscache_acquire_cookie() is passed the same keying and coherency
>      information as before.
>
>  (4) fscache_enable/disable_cookie() have been removed.
>
>      Call fscache_use_cookie() and fscache_unuse_cookie() when a file is
>      opened or closed to prevent a cache file from being culled and to keep
>      resources to hand that are needed to do I/O.
>
>      Unuse the cookie when a file is opened for writing.  This is gated by
>      the NFS_INO_FSCACHE flag on the nfs_inode.
>
>      A better way might be to invalidate it with FSCACHE_INVAL_DIO_WRITE
>      which will keep it unused until all open files are closed.
>

It looks like the comment doesn't match what was actually done inside
nfs_fscache_open_file().  Is the code right and the comment just out of date?

I'm getting that kasan UAF firing periodically in this code path, and
so it looks
related to this change,though I don't have great info on it so far and
it's hard to
reproduce.


>  (5) fscache_invalidate() now needs to be given uptodate auxiliary data and
>      a file size.  It also takes a flag to indicate if this was due to a
>      DIO write.
>
>  (6) Call nfs_fscache_invalidate() with FSCACHE_INVAL_DIO_WRITE on a file
>      to which a DIO write is made.
>
>  (7) Call fscache_note_page_release() from nfs_release_page().
>
>  (8) Use a killable wait in nfs_vm_page_mkwrite() when waiting for
>      PG_fscache to be cleared.
>
>  (9) The functions to read and write data to/from the cache are stubbed out
>      pending a conversion to use netfslib.
>
> Changes
> =======
> ver #2:
>  - Use gfpflags_allow_blocking() rather than using flag directly.
>  - fscache_acquire_volume() now returns errors.
>  - Remove NFS_INO_FSCACHE as it's no longer used.
>  - Need to unuse a cookie on file-release, not inode-clear.
>
> Signed-off-by: Dave Wysochanski <[e-mail]>
> Co-developed-by: David Howells <[e-mail]>
> Signed-off-by: David Howells <[e-mail]>
> cc: Trond Myklebust <[e-mail]>
> cc: Anna Schumaker <[e-mail]>
> cc: [e-mail]
> cc: [e-mail]
> Link: https://lore.kernel.org/r/[e-mail]/ # v1
> ---
>
>  fs/nfs/Kconfig            |    2
>  fs/nfs/Makefile           |    2
>  fs/nfs/client.c           |    4
>  fs/nfs/direct.c           |    2
>  fs/nfs/file.c             |   13 +
>  fs/nfs/fscache-index.c    |  140 ---------------
>  fs/nfs/fscache.c          |  434 +++++++++++----------------------------------
>  fs/nfs/fscache.h          |  126 ++++---------
>  fs/nfs/inode.c            |   11 -
>  fs/nfs/nfstrace.h         |    1
>  fs/nfs/super.c            |   28 ++-
>  fs/nfs/write.c            |    1
>  include/linux/nfs_fs.h    |    1
>  include/linux/nfs_fs_sb.h |    9 -
>  14 files changed, 171 insertions(+), 603 deletions(-)
>  delete mode 100644 fs/nfs/fscache-index.c
>
> diff --git a/fs/nfs/Kconfig b/fs/nfs/Kconfig
> index bdc11b89eac5..14a72224b657 100644
> --- a/fs/nfs/Kconfig
> +++ b/fs/nfs/Kconfig
> @@ -170,7 +170,7 @@ config ROOT_NFS
>
>  config NFS_FSCACHE
>         bool "Provide NFS client caching support"
> -       depends on NFS_FS=m && FSCACHE_OLD_API || NFS_FS=y && FSCACHE_OLD_API=y
> +       depends on NFS_FS=m && FSCACHE || NFS_FS=y && FSCACHE=y
>         help
>           Say Y here if you want NFS data to be cached locally on disc through
>           the general filesystem cache manager
> diff --git a/fs/nfs/Makefile b/fs/nfs/Makefile
> index 22d11fdc6deb..5f6db37f461e 100644
> --- a/fs/nfs/Makefile
> +++ b/fs/nfs/Makefile
> @@ -12,7 +12,7 @@ nfs-y                         := client.o dir.o file.o getroot.o inode.o super.o \
>                            export.o sysfs.o fs_context.o
>  nfs-$(CONFIG_ROOT_NFS) += nfsroot.o
>  nfs-$(CONFIG_SYSCTL)   += sysctl.o
> -nfs-$(CONFIG_NFS_FSCACHE) += fscache.o fscache-index.o
> +nfs-$(CONFIG_NFS_FSCACHE) += fscache.o
>
>  obj-$(CONFIG_NFS_V2) += nfsv2.o
>  nfsv2-y := nfs2super.o proc.o nfs2xdr.o
> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
> index 1e4dc1ab9312..8d8b85b5a641 100644
> --- a/fs/nfs/client.c
> +++ b/fs/nfs/client.c
> @@ -183,8 +183,6 @@ struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
>         clp->cl_net = get_net(cl_init->net);
>
>         clp->cl_principal = "*";
> -       nfs_fscache_get_client_cookie(clp);
> -
>         return clp;
>
>  error_cleanup:
> @@ -238,8 +236,6 @@ static void pnfs_init_server(struct nfs_server *server)
>   */
>  void nfs_free_client(struct nfs_client *clp)
>  {
> -       nfs_fscache_release_client_cookie(clp);
> -
>         /* -EIO all pending I/O */
>         if (!IS_ERR(clp->cl_rpcclient))
>                 rpc_shutdown_client(clp->cl_rpcclient);
> diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
> index 9cff8709c80a..eabfdab543c8 100644
> --- a/fs/nfs/direct.c
> +++ b/fs/nfs/direct.c
> @@ -59,6 +59,7 @@
>  #include "internal.h"
>  #include "iostat.h"
>  #include "pnfs.h"
> +#include "fscache.h"
>
>  #define NFSDBG_FACILITY                NFSDBG_VFS
>
> @@ -959,6 +960,7 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter)
>         } else {
>                 result = requested;
>         }
> +       nfs_fscache_invalidate(inode, FSCACHE_INVAL_DIO_WRITE);
>  out_release:
>         nfs_direct_req_release(dreq);
>  out:
> diff --git a/fs/nfs/file.c b/fs/nfs/file.c
> index 24e7dccce355..76d76acbc594 100644
> --- a/fs/nfs/file.c
> +++ b/fs/nfs/file.c
> @@ -84,6 +84,7 @@ nfs_file_release(struct inode *inode, struct file *filp)
>
>         nfs_inc_stats(inode, NFSIOS_VFSRELEASE);
>         nfs_file_clear_open_context(filp);
> +       nfs_fscache_release_file(inode, filp);
>         return 0;
>  }
>  EXPORT_SYMBOL_GPL(nfs_file_release);
> @@ -415,8 +416,7 @@ static void nfs_invalidate_page(struct page *page, unsigned int offset,
>                 return;
>         /* Cancel any unstarted writes on this page */
>         nfs_wb_page_cancel(page_file_mapping(page)->host, page);
> -
> -       nfs_fscache_invalidate_page(page, page->mapping->host);
> +       wait_on_page_fscache(page);
>  }
>
>  /*
> @@ -475,12 +475,11 @@ static void nfs_check_dirty_writeback(struct page *page,
>  static int nfs_launder_page(struct page *page)
>  {
>         struct inode *inode = page_file_mapping(page)->host;
> -       struct nfs_inode *nfsi = NFS_I(inode);
>
>         dfprintk(PAGECACHE, "NFS: launder_page(%ld, %llu)\n",
>                 inode->i_ino, (long long)page_offset(page));
>
> -       nfs_fscache_wait_on_page_write(nfsi, page);
> +       wait_on_page_fscache(page);
>         return nfs_wb_page(inode, page);
>  }
>
> @@ -555,7 +554,11 @@ static vm_fault_t nfs_vm_page_mkwrite(struct vm_fault *vmf)
>         sb_start_pagefault(inode->i_sb);
>
>         /* make sure the cache has finished storing the page */
> -       nfs_fscache_wait_on_page_write(NFS_I(inode), page);
> +       if (PageFsCache(page) &&
> +           wait_on_page_fscache_killable(vmf->page) < 0) {
> +               ret = VM_FAULT_RETRY;
> +               goto out;
> +       }
>
>         wait_on_bit_action(&NFS_I(inode)->flags, NFS_INO_INVALIDATING,
>                         nfs_wait_bit_killable, TASK_KILLABLE);
> diff --git a/fs/nfs/fscache-index.c b/fs/nfs/fscache-index.c
> deleted file mode 100644
> index 573b1da9342c..000000000000
> --- a/fs/nfs/fscache-index.c
> +++ /dev/null
> @@ -1,140 +0,0 @@
> -// SPDX-License-Identifier: GPL-2.0-or-later
> -/* NFS FS-Cache index structure definition
> - *
> - * Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
> - * Written by David Howells ([e-mail])
> - */
> -
> -#include <linux/init.h>
> -#include <linux/kernel.h>
> -#include <linux/sched.h>
> -#include <linux/mm.h>
> -#include <linux/nfs_fs.h>
> -#include <linux/nfs_fs_sb.h>
> -#include <linux/in6.h>
> -#include <linux/iversion.h>
> -
> -#include "internal.h"
> -#include "fscache.h"
> -
> -#define NFSDBG_FACILITY                NFSDBG_FSCACHE
> -
> -/*
> - * Define the NFS filesystem for FS-Cache.  Upon registration FS-Cache sticks
> - * the cookie for the top-level index object for NFS into here.  The top-level
> - * index can than have other cache objects inserted into it.
> - */
> -struct fscache_netfs nfs_fscache_netfs = {
> -       .name           = "nfs",
> -       .version        = 0,
> -};
> -
> -/*
> - * Register NFS for caching
> - */
> -int nfs_fscache_register(void)
> -{
> -       return fscache_register_netfs(&nfs_fscache_netfs);
> -}
> -
> -/*
> - * Unregister NFS for caching
> - */
> -void nfs_fscache_unregister(void)
> -{
> -       fscache_unregister_netfs(&nfs_fscache_netfs);
> -}
> -
> -/*
> - * Define the server object for FS-Cache.  This is used to describe a server
> - * object to fscache_acquire_cookie().  It is keyed by the NFS protocol and
> - * server address parameters.
> - */
> -const struct fscache_cookie_def nfs_fscache_server_index_def = {
> -       .name           = "NFS.server",
> -       .type           = FSCACHE_COOKIE_TYPE_INDEX,
> -};
> -
> -/*
> - * Define the superblock object for FS-Cache.  This is used to describe a
> - * superblock object to fscache_acquire_cookie().  It is keyed by all the NFS
> - * parameters that might cause a separate superblock.
> - */
> -const struct fscache_cookie_def nfs_fscache_super_index_def = {
> -       .name           = "NFS.super",
> -       .type           = FSCACHE_COOKIE_TYPE_INDEX,
> -};
> -
> -/*
> - * Consult the netfs about the state of an object
> - * - This function can be absent if the index carries no state data
> - * - The netfs data from the cookie being used as the target is
> - *   presented, as is the auxiliary data
> - */
> -static
> -enum fscache_checkaux nfs_fscache_inode_check_aux(void *cookie_netfs_data,
> -                                                 const void *data,
> -                                                 uint16_t datalen,
> -                                                 loff_t object_size)
> -{
> -       struct nfs_fscache_inode_auxdata auxdata;
> -       struct nfs_inode *nfsi = cookie_netfs_data;
> -
> -       if (datalen != sizeof(auxdata))
> -               return FSCACHE_CHECKAUX_OBSOLETE;
> -
> -       memset(&auxdata, 0, sizeof(auxdata));
> -       auxdata.mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
> -       auxdata.mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
> -       auxdata.ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
> -       auxdata.ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
> -
> -       if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
> -               auxdata.change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
> -
> -       if (memcmp(data, &auxdata, datalen) != 0)
> -               return FSCACHE_CHECKAUX_OBSOLETE;
> -
> -       return FSCACHE_CHECKAUX_OKAY;
> -}
> -
> -/*
> - * Get an extra reference on a read context.
> - * - This function can be absent if the completion function doesn't require a
> - *   context.
> - * - The read context is passed back to NFS in the event that a data read on the
> - *   cache fails with EIO - in which case the server must be contacted to
> - *   retrieve the data, which requires the read context for security.
> - */
> -static void nfs_fh_get_context(void *cookie_netfs_data, void *context)
> -{
> -       get_nfs_open_context(context);
> -}
> -
> -/*
> - * Release an extra reference on a read context.
> - * - This function can be absent if the completion function doesn't require a
> - *   context.
> - */
> -static void nfs_fh_put_context(void *cookie_netfs_data, void *context)
> -{
> -       if (context)
> -               put_nfs_open_context(context);
> -}
> -
> -/*
> - * Define the inode object for FS-Cache.  This is used to describe an inode
> - * object to fscache_acquire_cookie().  It is keyed by the NFS file handle for
> - * an inode.
> - *
> - * Coherency is managed by comparing the copies of i_size, i_mtime and i_ctime
> - * held in the cache auxiliary data for the data storage object with those in
> - * the inode struct in memory.
> - */
> -const struct fscache_cookie_def nfs_fscache_inode_object_def = {
> -       .name           = "NFS.fh",
> -       .type           = FSCACHE_COOKIE_TYPE_DATAFILE,
> -       .check_aux      = nfs_fscache_inode_check_aux,
> -       .get_context    = nfs_fh_get_context,
> -       .put_context    = nfs_fh_put_context,
> -};
> diff --git a/fs/nfs/fscache.c b/fs/nfs/fscache.c
> index d743629e05e1..d10e50ab0b3d 100644
> --- a/fs/nfs/fscache.c
> +++ b/fs/nfs/fscache.c
> @@ -22,24 +22,18 @@
>
>  #define NFSDBG_FACILITY                NFSDBG_FSCACHE
>
> -static struct rb_root nfs_fscache_keys = RB_ROOT;
> -static DEFINE_SPINLOCK(nfs_fscache_keys_lock);
> +#define NFS_MAX_KEY_LEN 1000
>
> -/*
> - * Layout of the key for an NFS server cache object.
> - */
> -struct nfs_server_key {
> -       struct {
> -               uint16_t        nfsversion;             /* NFS protocol version */
> -               uint32_t        minorversion;           /* NFSv4 minor version */
> -               uint16_t        family;                 /* address family */
> -               __be16          port;                   /* IP port */
> -       } hdr;
> -       union {
> -               struct in_addr  ipv4_addr;      /* IPv4 address */
> -               struct in6_addr ipv6_addr;      /* IPv6 address */
> -       };
> -} __packed;
> +static bool nfs_append_int(char *key, int *_len, unsigned long long x)
> +{
> +       if (*_len > NFS_MAX_KEY_LEN)
> +               return false;
> +       if (x == 0)
> +               key[(*_len)++] = ',';
> +       else
> +               *_len += sprintf(key + *_len, ",%llx", x);
> +       return true;
> +}
>
>  /*
>   * Get the per-client index cookie for an NFS client if the appropriate mount
> @@ -47,160 +41,108 @@ struct nfs_server_key {
>   * - We always try and get an index cookie for the client, but get filehandle
>   *   cookies on a per-superblock basis, depending on the mount flags
>   */
> -void nfs_fscache_get_client_cookie(struct nfs_client *clp)
> +static bool nfs_fscache_get_client_key(struct nfs_client *clp,
> +                                      char *key, int *_len)
>  {
>         const struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *) &clp->cl_addr;
>         const struct sockaddr_in *sin = (struct sockaddr_in *) &clp->cl_addr;
> -       struct nfs_server_key key;
> -       uint16_t len = sizeof(key.hdr);
>
> -       memset(&key, 0, sizeof(key));
> -       key.hdr.nfsversion = clp->rpc_ops->version;
> -       key.hdr.minorversion = clp->cl_minorversion;
> -       key.hdr.family = clp->cl_addr.ss_family;
> +       *_len += snprintf(key + *_len, NFS_MAX_KEY_LEN - *_len,
> +                         ",%u.%u,%x",
> +                         clp->rpc_ops->version,
> +                         clp->cl_minorversion,
> +                         clp->cl_addr.ss_family);
>
>         switch (clp->cl_addr.ss_family) {
>         case AF_INET:
> -               key.hdr.port = sin->sin_port;
> -               key.ipv4_addr = sin->sin_addr;
> -               len += sizeof(key.ipv4_addr);
> -               break;
> +               if (!nfs_append_int(key, _len, sin->sin_port) ||
> +                   !nfs_append_int(key, _len, sin->sin_addr.s_addr))
> +                       return false;
> +               return true;
>
>         case AF_INET6:
> -               key.hdr.port = sin6->sin6_port;
> -               key.ipv6_addr = sin6->sin6_addr;
> -               len += sizeof(key.ipv6_addr);
> -               break;
> +               if (!nfs_append_int(key, _len, sin6->sin6_port) ||
> +                   !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[0]) ||
> +                   !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[1]) ||
> +                   !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[2]) ||
> +                   !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[3]))
> +                       return false;
> +               return true;
>
>         default:
>                 printk(KERN_WARNING "NFS: Unknown network family '%d'\n",
>                        clp->cl_addr.ss_family);
> -               clp->fscache = NULL;
> -               return;
> +               return false;
>         }
> -
> -       /* create a cache index for looking up filehandles */
> -       clp->fscache = fscache_acquire_cookie(nfs_fscache_netfs.primary_index,
> -                                             &nfs_fscache_server_index_def,
> -                                             &key, len,
> -                                             NULL, 0,
> -                                             clp, 0, true);
> -       dfprintk(FSCACHE, "NFS: get client cookie (0x%p/0x%p)\n",
> -                clp, clp->fscache);
> -}
> -
> -/*
> - * Dispose of a per-client cookie
> - */
> -void nfs_fscache_release_client_cookie(struct nfs_client *clp)
> -{
> -       dfprintk(FSCACHE, "NFS: releasing client cookie (0x%p/0x%p)\n",
> -                clp, clp->fscache);
> -
> -       fscache_relinquish_cookie(clp->fscache, NULL, false);
> -       clp->fscache = NULL;
>  }
>
>  /*
> - * Get the cache cookie for an NFS superblock.  We have to handle
> - * uniquification here because the cache doesn't do it for us.
> + * Get the cache cookie for an NFS superblock.
>   *
>   * The default uniquifier is just an empty string, but it may be overridden
>   * either by the 'fsc=xxx' option to mount, or by inheriting it from the parent
>   * superblock across an automount point of some nature.
>   */
> -void nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int ulen)
> +int nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int ulen)
>  {
> -       struct nfs_fscache_key *key, *xkey;
> +       struct fscache_volume *vcookie;
>         struct nfs_server *nfss = NFS_SB(sb);
> -       struct rb_node **p, *parent;
> -       int diff;
> +       unsigned int len = 3;
> +       char *key;
>
> -       nfss->fscache_key = NULL;
> -       nfss->fscache = NULL;
> -       if (!uniq) {
> -               uniq = "";
> -               ulen = 1;
> +       if (uniq) {
> +               nfss->fscache_uniq = kmemdup_nul(uniq, ulen, GFP_KERNEL);
> +               if (!nfss->fscache_uniq)
> +                       return -ENOMEM;
>         }
>
> -       key = kzalloc(sizeof(*key) + ulen, GFP_KERNEL);
> +       key = kmalloc(NFS_MAX_KEY_LEN + 24, GFP_KERNEL);
>         if (!key)
> -               return;
> -
> -       key->nfs_client = nfss->nfs_client;
> -       key->key.super.s_flags = sb->s_flags & NFS_SB_MASK;
> -       key->key.nfs_server.flags = nfss->flags;
> -       key->key.nfs_server.rsize = nfss->rsize;
> -       key->key.nfs_server.wsize = nfss->wsize;
> -       key->key.nfs_server.acregmin = nfss->acregmin;
> -       key->key.nfs_server.acregmax = nfss->acregmax;
> -       key->key.nfs_server.acdirmin = nfss->acdirmin;
> -       key->key.nfs_server.acdirmax = nfss->acdirmax;
> -       key->key.nfs_server.fsid = nfss->fsid;
> -       key->key.rpc_auth.au_flavor = nfss->client->cl_auth->au_flavor;
> -
> -       key->key.uniq_len = ulen;
> -       memcpy(key->key.uniquifier, uniq, ulen);
> -
> -       spin_lock(&nfs_fscache_keys_lock);
> -       p = &nfs_fscache_keys.rb_node;
> -       parent = NULL;
> -       while (*p) {
> -               parent = *p;
> -               xkey = rb_entry(parent, struct nfs_fscache_key, node);
> -
> -               if (key->nfs_client < xkey->nfs_client)
> -                       goto go_left;
> -               if (key->nfs_client > xkey->nfs_client)
> -                       goto go_right;
> -
> -               diff = memcmp(&key->key, &xkey->key, sizeof(key->key));
> -               if (diff < 0)
> -                       goto go_left;
> -               if (diff > 0)
> -                       goto go_right;
> -
> -               if (key->key.uniq_len == 0)
> -                       goto non_unique;
> -               diff = memcmp(key->key.uniquifier,
> -                             xkey->key.uniquifier,
> -                             key->key.uniq_len);
> -               if (diff < 0)
> -                       goto go_left;
> -               if (diff > 0)
> -                       goto go_right;
> -               goto non_unique;
> -
> -       go_left:
> -               p = &(*p)->rb_left;
> -               continue;
> -       go_right:
> -               p = &(*p)->rb_right;
> +               return -ENOMEM;
> +
> +       memcpy(key, "nfs", 3);
> +       if (!nfs_fscache_get_client_key(nfss->nfs_client, key, &len) ||
> +           !nfs_append_int(key, &len, nfss->fsid.major) ||
> +           !nfs_append_int(key, &len, nfss->fsid.minor) ||
> +           !nfs_append_int(key, &len, sb->s_flags & NFS_SB_MASK) ||
> +           !nfs_append_int(key, &len, nfss->flags) ||
> +           !nfs_append_int(key, &len, nfss->rsize) ||
> +           !nfs_append_int(key, &len, nfss->wsize) ||
> +           !nfs_append_int(key, &len, nfss->acregmin) ||
> +           !nfs_append_int(key, &len, nfss->acregmax) ||
> +           !nfs_append_int(key, &len, nfss->acdirmin) ||
> +           !nfs_append_int(key, &len, nfss->acdirmax) ||
> +           !nfs_append_int(key, &len, nfss->client->cl_auth->au_flavor))
> +               goto out;
> +
> +       if (ulen > 0) {
> +               if (ulen > NFS_MAX_KEY_LEN - len)
> +                       goto out;
> +               key[len++] = ',';
> +               memcpy(key + len, uniq, ulen);
> +               len += ulen;
>         }
> -
> -       rb_link_node(&key->node, parent, p);
> -       rb_insert_color(&key->node, &nfs_fscache_keys);
> -       spin_unlock(&nfs_fscache_keys_lock);
> -       nfss->fscache_key = key;
> +       key[len] = 0;
>
>         /* create a cache index for looking up filehandles */
> -       nfss->fscache = fscache_acquire_cookie(nfss->nfs_client->fscache,
> -                                              &nfs_fscache_super_index_def,
> -                                              &key->key,
> -                                              sizeof(key->key) + ulen,
> -                                              NULL, 0,
> -                                              nfss, 0, true);
> +       vcookie = fscache_acquire_volume(key,
> +                                        NULL, /* preferred_cache */
> +                                        0 /* coherency_data */);
>         dfprintk(FSCACHE, "NFS: get superblock cookie (0x%p/0x%p)\n",
> -                nfss, nfss->fscache);
> -       return;
> +                nfss, vcookie);
> +       if (IS_ERR(vcookie)) {
> +               if (vcookie != ERR_PTR(-EBUSY)) {
> +                       kfree(key);
> +                       return PTR_ERR(vcookie);
> +               }
> +               pr_err("NFS: Cache volume key already in use (%s)\n", key);
> +               vcookie = NULL;
> +       }
> +       nfss->fscache = vcookie;
>
> -non_unique:
> -       spin_unlock(&nfs_fscache_keys_lock);
> +out:
>         kfree(key);
> -       nfss->fscache_key = NULL;
> -       nfss->fscache = NULL;
> -       printk(KERN_WARNING "NFS:"
> -              " Cache request denied due to non-unique superblock keys\n");
> +       return 0;
>  }
>
>  /*
> @@ -213,29 +155,9 @@ void nfs_fscache_release_super_cookie(struct super_block *sb)
>         dfprintk(FSCACHE, "NFS: releasing superblock cookie (0x%p/0x%p)\n",
>                  nfss, nfss->fscache);
>
> -       fscache_relinquish_cookie(nfss->fscache, NULL, false);
> +       fscache_relinquish_volume(nfss->fscache, 0, false);
>         nfss->fscache = NULL;
> -
> -       if (nfss->fscache_key) {
> -               spin_lock(&nfs_fscache_keys_lock);
> -               rb_erase(&nfss->fscache_key->node, &nfs_fscache_keys);
> -               spin_unlock(&nfs_fscache_keys_lock);
> -               kfree(nfss->fscache_key);
> -               nfss->fscache_key = NULL;
> -       }
> -}
> -
> -static void nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
> -                                 struct nfs_inode *nfsi)
> -{
> -       memset(auxdata, 0, sizeof(*auxdata));
> -       auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
> -       auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
> -       auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
> -       auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
> -
> -       if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
> -               auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
> +       kfree(nfss->fscache_uniq);
>  }
>
>  /*
> @@ -254,10 +176,12 @@ void nfs_fscache_init_inode(struct inode *inode)
>         nfs_fscache_update_auxdata(&auxdata, nfsi);
>
>         nfsi->fscache = fscache_acquire_cookie(NFS_SB(inode->i_sb)->fscache,
> -                                              &nfs_fscache_inode_object_def,
> -                                              nfsi->fh.data, nfsi->fh.size,
> -                                              &auxdata, sizeof(auxdata),
> -                                              nfsi, nfsi->vfs_inode.i_size, false);
> +                                              0,
> +                                              nfsi->fh.data, /* index_key */
> +                                              nfsi->fh.size,
> +                                              &auxdata,      /* aux_data */
> +                                              sizeof(auxdata),
> +                                              i_size_read(&nfsi->vfs_inode));
>  }
>
>  /*
> @@ -265,24 +189,15 @@ void nfs_fscache_init_inode(struct inode *inode)
>   */
>  void nfs_fscache_clear_inode(struct inode *inode)
>  {
> -       struct nfs_fscache_inode_auxdata auxdata;
>         struct nfs_inode *nfsi = NFS_I(inode);
>         struct fscache_cookie *cookie = nfs_i_fscache(inode);
>
>         dfprintk(FSCACHE, "NFS: clear cookie (0x%p/0x%p)\n", nfsi, cookie);
>
> -       nfs_fscache_update_auxdata(&auxdata, nfsi);
> -       fscache_relinquish_cookie(cookie, &auxdata, false);
> +       fscache_relinquish_cookie(cookie, false);
>         nfsi->fscache = NULL;
>  }
>
> -static bool nfs_fscache_can_enable(void *data)
> -{
> -       struct inode *inode = data;
> -
> -       return !inode_is_open_for_write(inode);
> -}
> -
>  /*
>   * Enable or disable caching for a file that is being opened as appropriate.
>   * The cookie is allocated when the inode is initialised, but is not enabled at
> @@ -307,93 +222,31 @@ void nfs_fscache_open_file(struct inode *inode, struct file *filp)
>         struct nfs_fscache_inode_auxdata auxdata;
>         struct nfs_inode *nfsi = NFS_I(inode);
>         struct fscache_cookie *cookie = nfs_i_fscache(inode);
> +       bool open_for_write = inode_is_open_for_write(inode);
>
>         if (!fscache_cookie_valid(cookie))
>                 return;
>
> -       nfs_fscache_update_auxdata(&auxdata, nfsi);
> -
> -       if (inode_is_open_for_write(inode)) {
> +       fscache_use_cookie(cookie, open_for_write);
> +       if (open_for_write) {
>                 dfprintk(FSCACHE, "NFS: nfsi 0x%p disabling cache\n", nfsi);
> -               clear_bit(NFS_INO_FSCACHE, &nfsi->flags);
> -               fscache_disable_cookie(cookie, &auxdata, true);
> -               fscache_uncache_all_inode_pages(cookie, inode);
> -       } else {
> -               dfprintk(FSCACHE, "NFS: nfsi 0x%p enabling cache\n", nfsi);
> -               fscache_enable_cookie(cookie, &auxdata, nfsi->vfs_inode.i_size,
> -                                     nfs_fscache_can_enable, inode);
> -               if (fscache_cookie_enabled(cookie))
> -                       set_bit(NFS_INO_FSCACHE, &NFS_I(inode)->flags);
> +               nfs_fscache_update_auxdata(&auxdata, nfsi);
> +               fscache_invalidate(cookie, &auxdata, i_size_read(inode),
> +                                  FSCACHE_INVAL_DIO_WRITE);
>         }
>  }
>  EXPORT_SYMBOL_GPL(nfs_fscache_open_file);
>
> -/*
> - * Release the caching state associated with a page, if the page isn't busy
> - * interacting with the cache.
> - * - Returns true (can release page) or false (page busy).
> - */
> -int nfs_fscache_release_page(struct page *page, gfp_t gfp)
> -{
> -       if (PageFsCache(page)) {
> -               struct fscache_cookie *cookie = nfs_i_fscache(page->mapping->host);
> -
> -               BUG_ON(!cookie);
> -               dfprintk(FSCACHE, "NFS: fscache releasepage (0x%p/0x%p/0x%p)\n",
> -                        cookie, page, NFS_I(page->mapping->host));
> -
> -               if (!fscache_maybe_release_page(cookie, page, gfp))
> -                       return 0;
> -
> -               nfs_inc_fscache_stats(page->mapping->host,
> -                                     NFSIOS_FSCACHE_PAGES_UNCACHED);
> -       }
> -
> -       return 1;
> -}
> -
> -/*
> - * Release the caching state associated with a page if undergoing complete page
> - * invalidation.
> - */
> -void __nfs_fscache_invalidate_page(struct page *page, struct inode *inode)
> +void nfs_fscache_release_file(struct inode *inode, struct file *filp)
>  {
> +       struct nfs_fscache_inode_auxdata auxdata;
> +       struct nfs_inode *nfsi = NFS_I(inode);
>         struct fscache_cookie *cookie = nfs_i_fscache(inode);
>
> -       BUG_ON(!cookie);
> -
> -       dfprintk(FSCACHE, "NFS: fscache invalidatepage (0x%p/0x%p/0x%p)\n",
> -                cookie, page, NFS_I(inode));
> -
> -       fscache_wait_on_page_write(cookie, page);
> -
> -       BUG_ON(!PageLocked(page));
> -       fscache_uncache_page(cookie, page);
> -       nfs_inc_fscache_stats(page->mapping->host,
> -                             NFSIOS_FSCACHE_PAGES_UNCACHED);
> -}
> -
> -/*
> - * Handle completion of a page being read from the cache.
> - * - Called in process (keventd) context.
> - */
> -static void nfs_readpage_from_fscache_complete(struct page *page,
> -                                              void *context,
> -                                              int error)
> -{
> -       dfprintk(FSCACHE,
> -                "NFS: readpage_from_fscache_complete (0x%p/0x%p/%d)\n",
> -                page, context, error);
> -
> -       /*
> -        * If the read completes with an error, mark the page with PG_checked,
> -        * unlock the page, and let the VM reissue the readpage.
> -        */
> -       if (!error)
> -               SetPageUptodate(page);
> -       else
> -               SetPageChecked(page);
> -       unlock_page(page);
> +       if (fscache_cookie_valid(cookie)) {
> +               nfs_fscache_update_auxdata(&auxdata, nfsi);
> +               fscache_unuse_cookie(cookie, &auxdata, NULL);
> +       }
>  }
>
>  /*
> @@ -402,8 +255,6 @@ static void nfs_readpage_from_fscache_complete(struct page *page,
>  int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,
>                                 struct inode *inode, struct page *page)
>  {
> -       int ret;
> -
>         dfprintk(FSCACHE,
>                  "NFS: readpage_from_fscache(fsc:%p/p:%p(i:%lx f:%lx)/0x%p)\n",
>                  nfs_i_fscache(inode), page, page->index, page->flags, inode);
> @@ -413,31 +264,7 @@ int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,
>                 return 1;
>         }
>
> -       ret = fscache_read_or_alloc_page(nfs_i_fscache(inode),
> -                                        page,
> -                                        nfs_readpage_from_fscache_complete,
> -                                        ctx,
> -                                        GFP_KERNEL);
> -
> -       switch (ret) {
> -       case 0: /* read BIO submitted (page in fscache) */
> -               dfprintk(FSCACHE,
> -                        "NFS:    readpage_from_fscache: BIO submitted\n");
> -               nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK);
> -               return ret;
> -
> -       case -ENOBUFS: /* inode not in cache */
> -       case -ENODATA: /* page not in cache */
> -               nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
> -               dfprintk(FSCACHE,
> -                        "NFS:    readpage_from_fscache %d\n", ret);
> -               return 1;
> -
> -       default:
> -               dfprintk(FSCACHE, "NFS:    readpage_from_fscache %d\n", ret);
> -               nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
> -       }
> -       return ret;
> +       return -ENOBUFS; // TODO: Use netfslib
>  }
>
>  /*
> @@ -449,45 +276,10 @@ int __nfs_readpages_from_fscache(struct nfs_open_context *ctx,
>                                  struct list_head *pages,
>                                  unsigned *nr_pages)
>  {
> -       unsigned npages = *nr_pages;
> -       int ret;
> -
>         dfprintk(FSCACHE, "NFS: nfs_getpages_from_fscache (0x%p/%u/0x%p)\n",
> -                nfs_i_fscache(inode), npages, inode);
> -
> -       ret = fscache_read_or_alloc_pages(nfs_i_fscache(inode),
> -                                         mapping, pages, nr_pages,
> -                                         nfs_readpage_from_fscache_complete,
> -                                         ctx,
> -                                         mapping_gfp_mask(mapping));
> -       if (*nr_pages < npages)
> -               nfs_add_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK,
> -                                     npages);
> -       if (*nr_pages > 0)
> -               nfs_add_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL,
> -                                     *nr_pages);
> -
> -       switch (ret) {
> -       case 0: /* read submitted to the cache for all pages */
> -               BUG_ON(!list_empty(pages));
> -               BUG_ON(*nr_pages != 0);
> -               dfprintk(FSCACHE,
> -                        "NFS: nfs_getpages_from_fscache: submitted\n");
> -
> -               return ret;
> -
> -       case -ENOBUFS: /* some pages aren't cached and can't be */
> -       case -ENODATA: /* some pages aren't cached */
> -               dfprintk(FSCACHE,
> -                        "NFS: nfs_getpages_from_fscache: no page: %d\n", ret);
> -               return 1;
> +                nfs_i_fscache(inode), *nr_pages, inode);
>
> -       default:
> -               dfprintk(FSCACHE,
> -                        "NFS: nfs_getpages_from_fscache: ret  %d\n", ret);
> -       }
> -
> -       return ret;
> +       return -ENOBUFS; // TODO: Use netfslib
>  }
>
>  /*
> @@ -496,25 +288,9 @@ int __nfs_readpages_from_fscache(struct nfs_open_context *ctx,
>   */
>  void __nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
>  {
> -       int ret;
> -
>         dfprintk(FSCACHE,
>                  "NFS: readpage_to_fscache(fsc:%p/p:%p(i:%lx f:%lx)/%d)\n",
>                  nfs_i_fscache(inode), page, page->index, page->flags, sync);
>
> -       ret = fscache_write_page(nfs_i_fscache(inode), page,
> -                                inode->i_size, GFP_KERNEL);
> -       dfprintk(FSCACHE,
> -                "NFS:     readpage_to_fscache: p:%p(i:%lu f:%lx) ret %d\n",
> -                page, page->index, page->flags, ret);
> -
> -       if (ret != 0) {
> -               fscache_uncache_page(nfs_i_fscache(inode), page);
> -               nfs_inc_fscache_stats(inode,
> -                                     NFSIOS_FSCACHE_PAGES_WRITTEN_FAIL);
> -               nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_UNCACHED);
> -       } else {
> -               nfs_inc_fscache_stats(inode,
> -                                     NFSIOS_FSCACHE_PAGES_WRITTEN_OK);
> -       }
> +       return; // TODO: Use netfslib
>  }
> diff --git a/fs/nfs/fscache.h b/fs/nfs/fscache.h
> index 6754c8607230..26b6fb1cfd58 100644
> --- a/fs/nfs/fscache.h
> +++ b/fs/nfs/fscache.h
> @@ -12,46 +12,10 @@
>  #include <linux/nfs_mount.h>
>  #include <linux/nfs4_mount.h>
>  #include <linux/fscache.h>
> +#include <linux/iversion.h>
>
>  #ifdef CONFIG_NFS_FSCACHE
>
> -/*
> - * set of NFS FS-Cache objects that form a superblock key
> - */
> -struct nfs_fscache_key {
> -       struct rb_node          node;
> -       struct nfs_client       *nfs_client;    /* the server */
> -
> -       /* the elements of the unique key - as used by nfs_compare_super() and
> -        * nfs_compare_mount_options() to distinguish superblocks */
> -       struct {
> -               struct {
> -                       unsigned long   s_flags;        /* various flags
> -                                                        * (& NFS_MS_MASK) */
> -               } super;
> -
> -               struct {
> -                       struct nfs_fsid fsid;
> -                       int             flags;
> -                       unsigned int    rsize;          /* read size */
> -                       unsigned int    wsize;          /* write size */
> -                       unsigned int    acregmin;       /* attr cache timeouts */
> -                       unsigned int    acregmax;
> -                       unsigned int    acdirmin;
> -                       unsigned int    acdirmax;
> -               } nfs_server;
> -
> -               struct {
> -                       rpc_authflavor_t au_flavor;
> -               } rpc_auth;
> -
> -               /* uniquifier - can be used if nfs_server.flags includes
> -                * NFS_MOUNT_UNSHARED  */
> -               u8 uniq_len;
> -               char uniquifier[0];
> -       } key;
> -};
> -
>  /*
>   * Definition of the auxiliary data attached to NFS inode storage objects
>   * within the cache.
> @@ -69,32 +33,18 @@ struct nfs_fscache_inode_auxdata {
>         u64     change_attr;
>  };
>
> -/*
> - * fscache-index.c
> - */
> -extern struct fscache_netfs nfs_fscache_netfs;
> -extern const struct fscache_cookie_def nfs_fscache_server_index_def;
> -extern const struct fscache_cookie_def nfs_fscache_super_index_def;
> -extern const struct fscache_cookie_def nfs_fscache_inode_object_def;
> -
> -extern int nfs_fscache_register(void);
> -extern void nfs_fscache_unregister(void);
> -
>  /*
>   * fscache.c
>   */
> -extern void nfs_fscache_get_client_cookie(struct nfs_client *);
> -extern void nfs_fscache_release_client_cookie(struct nfs_client *);
> -
> -extern void nfs_fscache_get_super_cookie(struct super_block *, const char *, int);
> +extern int nfs_fscache_get_super_cookie(struct super_block *, const char *, int);
>  extern void nfs_fscache_release_super_cookie(struct super_block *);
>
>  extern void nfs_fscache_init_inode(struct inode *);
>  extern void nfs_fscache_clear_inode(struct inode *);
>  extern void nfs_fscache_open_file(struct inode *, struct file *);
> +extern void nfs_fscache_release_file(struct inode *, struct file *);
>
>  extern void __nfs_fscache_invalidate_page(struct page *, struct inode *);
> -extern int nfs_fscache_release_page(struct page *, gfp_t);
>
>  extern int __nfs_readpage_from_fscache(struct nfs_open_context *,
>                                        struct inode *, struct page *);
> @@ -103,25 +53,17 @@ extern int __nfs_readpages_from_fscache(struct nfs_open_context *,
>                                         struct list_head *, unsigned *);
>  extern void __nfs_readpage_to_fscache(struct inode *, struct page *, int);
>
> -/*
> - * wait for a page to complete writing to the cache
> - */
> -static inline void nfs_fscache_wait_on_page_write(struct nfs_inode *nfsi,
> -                                                 struct page *page)
> -{
> -       if (PageFsCache(page))
> -               fscache_wait_on_page_write(nfsi->fscache, page);
> -}
> -
> -/*
> - * release the caching state associated with a page if undergoing complete page
> - * invalidation
> - */
> -static inline void nfs_fscache_invalidate_page(struct page *page,
> -                                              struct inode *inode)
> +static inline int nfs_fscache_release_page(struct page *page, gfp_t gfp)
>  {
> -       if (PageFsCache(page))
> -               __nfs_fscache_invalidate_page(page, inode);
> +       if (PageFsCache(page)) {
> +               if (!gfpflags_allow_blocking(gfp) || !(gfp & __GFP_FS))
> +                       return false;
> +               wait_on_page_fscache(page);
> +               fscache_note_page_release(nfs_i_fscache(page->mapping->host));
> +               nfs_inc_fscache_stats(page->mapping->host,
> +                                     NFSIOS_FSCACHE_PAGES_UNCACHED);
> +       }
> +       return true;
>  }
>
>  /*
> @@ -163,20 +105,32 @@ static inline void nfs_readpage_to_fscache(struct inode *inode,
>                 __nfs_readpage_to_fscache(inode, page, sync);
>  }
>
> -/*
> - * Invalidate the contents of fscache for this inode.  This will not sleep.
> - */
> -static inline void nfs_fscache_invalidate(struct inode *inode)
> +static inline void nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
> +                                             struct nfs_inode *nfsi)
>  {
> -       fscache_invalidate(NFS_I(inode)->fscache);
> +       memset(auxdata, 0, sizeof(*auxdata));
> +       auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
> +       auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
> +       auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
> +       auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
> +
> +       if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
> +               auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
>  }
>
>  /*
> - * Wait for an object to finish being invalidated.
> + * Invalidate the contents of fscache for this inode.  This will not sleep.
>   */
> -static inline void nfs_fscache_wait_on_invalidate(struct inode *inode)
> +static inline void nfs_fscache_invalidate(struct inode *inode, int flags)
>  {
> -       fscache_wait_on_invalidate(NFS_I(inode)->fscache);
> +       struct nfs_fscache_inode_auxdata auxdata;
> +       struct nfs_inode *nfsi = NFS_I(inode);
> +
> +       if (nfsi->fscache) {
> +               nfs_fscache_update_auxdata(&auxdata, nfsi);
> +               fscache_invalidate(nfsi->fscache, &auxdata,
> +                                  i_size_read(&nfsi->vfs_inode), flags);
> +       }
>  }
>
>  /*
> @@ -190,12 +144,6 @@ static inline const char *nfs_server_fscache_state(struct nfs_server *server)
>  }
>
>  #else /* CONFIG_NFS_FSCACHE */
> -static inline int nfs_fscache_register(void) { return 0; }
> -static inline void nfs_fscache_unregister(void) {}
> -
> -static inline void nfs_fscache_get_client_cookie(struct nfs_client *clp) {}
> -static inline void nfs_fscache_release_client_cookie(struct nfs_client *clp) {}
> -
>  static inline void nfs_fscache_release_super_cookie(struct super_block *sb) {}
>
>  static inline void nfs_fscache_init_inode(struct inode *inode) {}
> @@ -207,11 +155,6 @@ static inline int nfs_fscache_release_page(struct page *page, gfp_t gfp)
>  {
>         return 1; /* True: may release page */
>  }
> -static inline void nfs_fscache_invalidate_page(struct page *page,
> -                                              struct inode *inode) {}
> -static inline void nfs_fscache_wait_on_page_write(struct nfs_inode *nfsi,
> -                                                 struct page *page) {}
> -
>  static inline int nfs_readpage_from_fscache(struct nfs_open_context *ctx,
>                                             struct inode *inode,
>                                             struct page *page)
> @@ -230,8 +173,7 @@ static inline void nfs_readpage_to_fscache(struct inode *inode,
>                                            struct page *page, int sync) {}
>
>
> -static inline void nfs_fscache_invalidate(struct inode *inode) {}
> -static inline void nfs_fscache_wait_on_invalidate(struct inode *inode) {}
> +static inline void nfs_fscache_invalidate(struct inode *inode, int flags) {}
>
>  static inline const char *nfs_server_fscache_state(struct nfs_server *server)
>  {
> diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
> index fda530d5e764..a918c3a834b6 100644
> --- a/fs/nfs/inode.c
> +++ b/fs/nfs/inode.c
> @@ -209,7 +209,7 @@ void nfs_set_cache_invalid(struct inode *inode, unsigned long flags)
>         if (!nfs_has_xattr_cache(nfsi))
>                 flags &= ~NFS_INO_INVALID_XATTR;
>         if (flags & NFS_INO_INVALID_DATA)
> -               nfs_fscache_invalidate(inode);
> +               nfs_fscache_invalidate(inode, 0);
>         flags &= ~(NFS_INO_REVAL_PAGECACHE | NFS_INO_REVAL_FORCED);
>
>         nfsi->cache_validity |= flags;
> @@ -1289,6 +1289,7 @@ static int nfs_invalidate_mapping(struct inode *inode, struct address_space *map
>  {
>         int ret;
>
> +       nfs_fscache_invalidate(inode, 0);
>         if (mapping->nrpages != 0) {
>                 if (S_ISREG(inode->i_mode)) {
>                         ret = nfs_sync_mapping(mapping);
> @@ -1300,7 +1301,6 @@ static int nfs_invalidate_mapping(struct inode *inode, struct address_space *map
>                         return ret;
>         }
>         nfs_inc_stats(inode, NFSIOS_DATAINVALIDATE);
> -       nfs_fscache_wait_on_invalidate(inode);
>
>         dfprintk(PAGECACHE, "NFS: (%s/%Lu) data cache invalidated\n",
>                         inode->i_sb->s_id,
> @@ -2374,10 +2374,6 @@ static int __init init_nfs_fs(void)
>         if (err < 0)
>                 goto out9;
>
> -       err = nfs_fscache_register();
> -       if (err < 0)
> -               goto out8;
> -
>         err = nfsiod_start();
>         if (err)
>                 goto out7;
> @@ -2429,8 +2425,6 @@ static int __init init_nfs_fs(void)
>  out6:
>         nfsiod_stop();
>  out7:
> -       nfs_fscache_unregister();
> -out8:
>         unregister_pernet_subsys(&nfs_net_ops);
>  out9:
>         nfs_sysfs_exit();
> @@ -2445,7 +2439,6 @@ static void __exit exit_nfs_fs(void)
>         nfs_destroy_readpagecache();
>         nfs_destroy_inodecache();
>         nfs_destroy_nfspagecache();
> -       nfs_fscache_unregister();
>         unregister_pernet_subsys(&nfs_net_ops);
>         rpc_proc_unregister(&init_net, "nfs");
>         unregister_nfs_fs();
> diff --git a/fs/nfs/nfstrace.h b/fs/nfs/nfstrace.h
> index b3aee261801e..317ce27bdc4b 100644
> --- a/fs/nfs/nfstrace.h
> +++ b/fs/nfs/nfstrace.h
> @@ -42,7 +42,6 @@
>                         { BIT(NFS_INO_ACL_LRU_SET), "ACL_LRU_SET" }, \
>                         { BIT(NFS_INO_INVALIDATING), "INVALIDATING" }, \
>                         { BIT(NFS_INO_FSCACHE), "FSCACHE" }, \
> -                       { BIT(NFS_INO_FSCACHE_LOCK), "FSCACHE_LOCK" }, \
>                         { BIT(NFS_INO_LAYOUTCOMMIT), "NEED_LAYOUTCOMMIT" }, \
>                         { BIT(NFS_INO_LAYOUTCOMMITTING), "LAYOUTCOMMIT" }, \
>                         { BIT(NFS_INO_LAYOUTSTATS), "LAYOUTSTATS" }, \
> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
> index 3aced401735c..6ab5eeb000dc 100644
> --- a/fs/nfs/super.c
> +++ b/fs/nfs/super.c
> @@ -1204,42 +1204,42 @@ static int nfs_compare_super(struct super_block *sb, struct fs_context *fc)
>  }
>
>  #ifdef CONFIG_NFS_FSCACHE
> -static void nfs_get_cache_cookie(struct super_block *sb,
> -                                struct nfs_fs_context *ctx)
> +static int nfs_get_cache_cookie(struct super_block *sb,
> +                               struct nfs_fs_context *ctx)
>  {
>         struct nfs_server *nfss = NFS_SB(sb);
>         char *uniq = NULL;
>         int ulen = 0;
>
> -       nfss->fscache_key = NULL;
>         nfss->fscache = NULL;
>
>         if (!ctx)
> -               return;
> +               return 0;
>
>         if (ctx->clone_data.sb) {
>                 struct nfs_server *mnt_s = NFS_SB(ctx->clone_data.sb);
>                 if (!(mnt_s->options & NFS_OPTION_FSCACHE))
> -                       return;
> -               if (mnt_s->fscache_key) {
> -                       uniq = mnt_s->fscache_key->key.uniquifier;
> -                       ulen = mnt_s->fscache_key->key.uniq_len;
> +                       return 0;
> +               if (mnt_s->fscache_uniq) {
> +                       uniq = mnt_s->fscache_uniq;
> +                       ulen = strlen(uniq);
>                 }
>         } else {
>                 if (!(ctx->options & NFS_OPTION_FSCACHE))
> -                       return;
> +                       return 0;
>                 if (ctx->fscache_uniq) {
>                         uniq = ctx->fscache_uniq;
>                         ulen = strlen(ctx->fscache_uniq);
>                 }
>         }
>
> -       nfs_fscache_get_super_cookie(sb, uniq, ulen);
> +       return nfs_fscache_get_super_cookie(sb, uniq, ulen);
>  }
>  #else
> -static void nfs_get_cache_cookie(struct super_block *sb,
> -                                struct nfs_fs_context *ctx)
> +static int nfs_get_cache_cookie(struct super_block *sb,
> +                               struct nfs_fs_context *ctx)
>  {
> +       return 0;
>  }
>  #endif
>
> @@ -1299,7 +1299,9 @@ int nfs_get_tree_common(struct fs_context *fc)
>                         s->s_blocksize_bits = bsize;
>                         s->s_blocksize = 1U << bsize;
>                 }
> -               nfs_get_cache_cookie(s, ctx);
> +               error = nfs_get_cache_cookie(s, ctx);
> +               if (error < 0)
> +                       goto error_splat_super;
>         }
>
>         error = nfs_get_root(s, fc);
> diff --git a/fs/nfs/write.c b/fs/nfs/write.c
> index 9b7619ce17a7..2b322170372a 100644
> --- a/fs/nfs/write.c
> +++ b/fs/nfs/write.c
> @@ -294,6 +294,7 @@ static void nfs_grow_file(struct page *page, unsigned int offset, unsigned int c
>         nfs_inc_stats(inode, NFSIOS_EXTENDWRITE);
>  out:
>         spin_unlock(&inode->i_lock);
> +       nfs_fscache_invalidate(inode, 0);
>  }
>
>  /* A writeback failed: mark the page as bad, and invalidate the page cache */
> diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
> index 05f249f20f55..00835bacd236 100644
> --- a/include/linux/nfs_fs.h
> +++ b/include/linux/nfs_fs.h
> @@ -275,7 +275,6 @@ struct nfs4_copy_state {
>  #define NFS_INO_ACL_LRU_SET    (2)             /* Inode is on the LRU list */
>  #define NFS_INO_INVALIDATING   (3)             /* inode is being invalidated */
>  #define NFS_INO_FSCACHE                (5)             /* inode can be cached by FS-Cache */
> -#define NFS_INO_FSCACHE_LOCK   (6)             /* FS-Cache cookie management lock */
>  #define NFS_INO_FORCE_READDIR  (7)             /* force readdirplus */
>  #define NFS_INO_LAYOUTCOMMIT   (9)             /* layoutcommit required */
>  #define NFS_INO_LAYOUTCOMMITTING (10)          /* layoutcommit inflight */
> diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
> index 2a9acbfe00f0..77b2dba27bbb 100644
> --- a/include/linux/nfs_fs_sb.h
> +++ b/include/linux/nfs_fs_sb.h
> @@ -120,11 +120,6 @@ struct nfs_client {
>          * This is used to generate the mv0 callback address.
>          */
>         char                    cl_ipaddr[48];
> -
> -#ifdef CONFIG_NFS_FSCACHE
> -       struct fscache_cookie   *fscache;       /* client index cache cookie */
> -#endif
> -
>         struct net              *cl_net;
>         struct list_head        pending_cb_stateids;
>  };
> @@ -194,8 +189,8 @@ struct nfs_server {
>         struct nfs_auth_info    auth_info;      /* parsed auth flavors */
>
>  #ifdef CONFIG_NFS_FSCACHE
> -       struct nfs_fscache_key  *fscache_key;   /* unique key for superblock */
> -       struct fscache_cookie   *fscache;       /* superblock cookie */
> +       struct fscache_volume   *fscache;       /* superblock cookie */
> +       char                    *fscache_uniq;  /* Uniquifier (or NULL) */
>  #endif
>
>         u32                     pnfs_blksize;   /* layout_blksize attr */
>
>

David Howells Dec. 11, 2021, 1:37 p.m. UTC | #2

David Wysochanski <[e-mail]> wrote:

> >  (4) fscache_enable/disable_cookie() have been removed.
> >
> >      Call fscache_use_cookie() and fscache_unuse_cookie() when a file is
> >      opened or closed to prevent a cache file from being culled and to keep
> >      resources to hand that are needed to do I/O.
> >
> >      Unuse the cookie when a file is opened for writing.  This is gated by
> >      the NFS_INO_FSCACHE flag on the nfs_inode.
> >
> >      A better way might be to invalidate it with FSCACHE_INVAL_DIO_WRITE
> >      which will keep it unused until all open files are closed.
> >
> 
> It looks like the comment doesn't match what was actually done inside
> nfs_fscache_open_file().  Is the code right and the comment just out of date?

The comment is out of date.  NFS_INO_FSCACHE isn't used now.

> I'm getting that kasan UAF firing periodically in this code path, and so it
> looks related to this change,though I don't have great info on it so far and
> it's hard to reproduce.

Can you copy the kasan UAF text into a reply?

David

Patch

diff --git a/fs/nfs/Kconfig b/fs/nfs/Kconfig
index bdc11b89eac5..14a72224b657 100644
--- a/fs/nfs/Kconfig
+++ b/fs/nfs/Kconfig
@@ -170,7 +170,7 @@  config ROOT_NFS
 
 config NFS_FSCACHE
 	bool "Provide NFS client caching support"
-	depends on NFS_FS=m && FSCACHE_OLD_API || NFS_FS=y && FSCACHE_OLD_API=y
+	depends on NFS_FS=m && FSCACHE || NFS_FS=y && FSCACHE=y
 	help
 	  Say Y here if you want NFS data to be cached locally on disc through
 	  the general filesystem cache manager
diff --git a/fs/nfs/Makefile b/fs/nfs/Makefile
index 22d11fdc6deb..5f6db37f461e 100644
--- a/fs/nfs/Makefile
+++ b/fs/nfs/Makefile
@@ -12,7 +12,7 @@  nfs-y 			:= client.o dir.o file.o getroot.o inode.o super.o \
 			   export.o sysfs.o fs_context.o
 nfs-$(CONFIG_ROOT_NFS)	+= nfsroot.o
 nfs-$(CONFIG_SYSCTL)	+= sysctl.o
-nfs-$(CONFIG_NFS_FSCACHE) += fscache.o fscache-index.o
+nfs-$(CONFIG_NFS_FSCACHE) += fscache.o
 
 obj-$(CONFIG_NFS_V2) += nfsv2.o
 nfsv2-y := nfs2super.o proc.o nfs2xdr.o
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 1e4dc1ab9312..8d8b85b5a641 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -183,8 +183,6 @@  struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_init)
 	clp->cl_net = get_net(cl_init->net);
 
 	clp->cl_principal = "*";
-	nfs_fscache_get_client_cookie(clp);
-
 	return clp;
 
 error_cleanup:
@@ -238,8 +236,6 @@  static void pnfs_init_server(struct nfs_server *server)
  */
 void nfs_free_client(struct nfs_client *clp)
 {
-	nfs_fscache_release_client_cookie(clp);
-
 	/* -EIO all pending I/O */
 	if (!IS_ERR(clp->cl_rpcclient))
 		rpc_shutdown_client(clp->cl_rpcclient);
diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
index 9cff8709c80a..eabfdab543c8 100644
--- a/fs/nfs/direct.c
+++ b/fs/nfs/direct.c
@@ -59,6 +59,7 @@ 
 #include "internal.h"
 #include "iostat.h"
 #include "pnfs.h"
+#include "fscache.h"
 
 #define NFSDBG_FACILITY		NFSDBG_VFS
 
@@ -959,6 +960,7 @@  ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter)
 	} else {
 		result = requested;
 	}
+	nfs_fscache_invalidate(inode, FSCACHE_INVAL_DIO_WRITE);
 out_release:
 	nfs_direct_req_release(dreq);
 out:
diff --git a/fs/nfs/file.c b/fs/nfs/file.c
index 24e7dccce355..76d76acbc594 100644
--- a/fs/nfs/file.c
+++ b/fs/nfs/file.c
@@ -84,6 +84,7 @@  nfs_file_release(struct inode *inode, struct file *filp)
 
 	nfs_inc_stats(inode, NFSIOS_VFSRELEASE);
 	nfs_file_clear_open_context(filp);
+	nfs_fscache_release_file(inode, filp);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(nfs_file_release);
@@ -415,8 +416,7 @@  static void nfs_invalidate_page(struct page *page, unsigned int offset,
 		return;
 	/* Cancel any unstarted writes on this page */
 	nfs_wb_page_cancel(page_file_mapping(page)->host, page);
-
-	nfs_fscache_invalidate_page(page, page->mapping->host);
+	wait_on_page_fscache(page);
 }
 
 /*
@@ -475,12 +475,11 @@  static void nfs_check_dirty_writeback(struct page *page,
 static int nfs_launder_page(struct page *page)
 {
 	struct inode *inode = page_file_mapping(page)->host;
-	struct nfs_inode *nfsi = NFS_I(inode);
 
 	dfprintk(PAGECACHE, "NFS: launder_page(%ld, %llu)\n",
 		inode->i_ino, (long long)page_offset(page));
 
-	nfs_fscache_wait_on_page_write(nfsi, page);
+	wait_on_page_fscache(page);
 	return nfs_wb_page(inode, page);
 }
 
@@ -555,7 +554,11 @@  static vm_fault_t nfs_vm_page_mkwrite(struct vm_fault *vmf)
 	sb_start_pagefault(inode->i_sb);
 
 	/* make sure the cache has finished storing the page */
-	nfs_fscache_wait_on_page_write(NFS_I(inode), page);
+	if (PageFsCache(page) &&
+	    wait_on_page_fscache_killable(vmf->page) < 0) {
+		ret = VM_FAULT_RETRY;
+		goto out;
+	}
 
 	wait_on_bit_action(&NFS_I(inode)->flags, NFS_INO_INVALIDATING,
 			nfs_wait_bit_killable, TASK_KILLABLE);
diff --git a/fs/nfs/fscache-index.c b/fs/nfs/fscache-index.c
deleted file mode 100644
index 573b1da9342c..000000000000
--- a/fs/nfs/fscache-index.c
+++ /dev/null
@@ -1,140 +0,0 @@ 
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* NFS FS-Cache index structure definition
- *
- * Copyright (C) 2008 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells ([e-mail])
- */
-
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/sched.h>
-#include <linux/mm.h>
-#include <linux/nfs_fs.h>
-#include <linux/nfs_fs_sb.h>
-#include <linux/in6.h>
-#include <linux/iversion.h>
-
-#include "internal.h"
-#include "fscache.h"
-
-#define NFSDBG_FACILITY		NFSDBG_FSCACHE
-
-/*
- * Define the NFS filesystem for FS-Cache.  Upon registration FS-Cache sticks
- * the cookie for the top-level index object for NFS into here.  The top-level
- * index can than have other cache objects inserted into it.
- */
-struct fscache_netfs nfs_fscache_netfs = {
-	.name		= "nfs",
-	.version	= 0,
-};
-
-/*
- * Register NFS for caching
- */
-int nfs_fscache_register(void)
-{
-	return fscache_register_netfs(&nfs_fscache_netfs);
-}
-
-/*
- * Unregister NFS for caching
- */
-void nfs_fscache_unregister(void)
-{
-	fscache_unregister_netfs(&nfs_fscache_netfs);
-}
-
-/*
- * Define the server object for FS-Cache.  This is used to describe a server
- * object to fscache_acquire_cookie().  It is keyed by the NFS protocol and
- * server address parameters.
- */
-const struct fscache_cookie_def nfs_fscache_server_index_def = {
-	.name		= "NFS.server",
-	.type 		= FSCACHE_COOKIE_TYPE_INDEX,
-};
-
-/*
- * Define the superblock object for FS-Cache.  This is used to describe a
- * superblock object to fscache_acquire_cookie().  It is keyed by all the NFS
- * parameters that might cause a separate superblock.
- */
-const struct fscache_cookie_def nfs_fscache_super_index_def = {
-	.name		= "NFS.super",
-	.type 		= FSCACHE_COOKIE_TYPE_INDEX,
-};
-
-/*
- * Consult the netfs about the state of an object
- * - This function can be absent if the index carries no state data
- * - The netfs data from the cookie being used as the target is
- *   presented, as is the auxiliary data
- */
-static
-enum fscache_checkaux nfs_fscache_inode_check_aux(void *cookie_netfs_data,
-						  const void *data,
-						  uint16_t datalen,
-						  loff_t object_size)
-{
-	struct nfs_fscache_inode_auxdata auxdata;
-	struct nfs_inode *nfsi = cookie_netfs_data;
-
-	if (datalen != sizeof(auxdata))
-		return FSCACHE_CHECKAUX_OBSOLETE;
-
-	memset(&auxdata, 0, sizeof(auxdata));
-	auxdata.mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
-	auxdata.mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
-	auxdata.ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
-	auxdata.ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
-
-	if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
-		auxdata.change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
-
-	if (memcmp(data, &auxdata, datalen) != 0)
-		return FSCACHE_CHECKAUX_OBSOLETE;
-
-	return FSCACHE_CHECKAUX_OKAY;
-}
-
-/*
- * Get an extra reference on a read context.
- * - This function can be absent if the completion function doesn't require a
- *   context.
- * - The read context is passed back to NFS in the event that a data read on the
- *   cache fails with EIO - in which case the server must be contacted to
- *   retrieve the data, which requires the read context for security.
- */
-static void nfs_fh_get_context(void *cookie_netfs_data, void *context)
-{
-	get_nfs_open_context(context);
-}
-
-/*
- * Release an extra reference on a read context.
- * - This function can be absent if the completion function doesn't require a
- *   context.
- */
-static void nfs_fh_put_context(void *cookie_netfs_data, void *context)
-{
-	if (context)
-		put_nfs_open_context(context);
-}
-
-/*
- * Define the inode object for FS-Cache.  This is used to describe an inode
- * object to fscache_acquire_cookie().  It is keyed by the NFS file handle for
- * an inode.
- *
- * Coherency is managed by comparing the copies of i_size, i_mtime and i_ctime
- * held in the cache auxiliary data for the data storage object with those in
- * the inode struct in memory.
- */
-const struct fscache_cookie_def nfs_fscache_inode_object_def = {
-	.name		= "NFS.fh",
-	.type		= FSCACHE_COOKIE_TYPE_DATAFILE,
-	.check_aux	= nfs_fscache_inode_check_aux,
-	.get_context	= nfs_fh_get_context,
-	.put_context	= nfs_fh_put_context,
-};
diff --git a/fs/nfs/fscache.c b/fs/nfs/fscache.c
index d743629e05e1..d10e50ab0b3d 100644
--- a/fs/nfs/fscache.c
+++ b/fs/nfs/fscache.c
@@ -22,24 +22,18 @@ 
 
 #define NFSDBG_FACILITY		NFSDBG_FSCACHE
 
-static struct rb_root nfs_fscache_keys = RB_ROOT;
-static DEFINE_SPINLOCK(nfs_fscache_keys_lock);
+#define NFS_MAX_KEY_LEN 1000
 
-/*
- * Layout of the key for an NFS server cache object.
- */
-struct nfs_server_key {
-	struct {
-		uint16_t	nfsversion;		/* NFS protocol version */
-		uint32_t	minorversion;		/* NFSv4 minor version */
-		uint16_t	family;			/* address family */
-		__be16		port;			/* IP port */
-	} hdr;
-	union {
-		struct in_addr	ipv4_addr;	/* IPv4 address */
-		struct in6_addr ipv6_addr;	/* IPv6 address */
-	};
-} __packed;
+static bool nfs_append_int(char *key, int *_len, unsigned long long x)
+{
+	if (*_len > NFS_MAX_KEY_LEN)
+		return false;
+	if (x == 0)
+		key[(*_len)++] = ',';
+	else
+		*_len += sprintf(key + *_len, ",%llx", x);
+	return true;
+}
 
 /*
  * Get the per-client index cookie for an NFS client if the appropriate mount
@@ -47,160 +41,108 @@  struct nfs_server_key {
  * - We always try and get an index cookie for the client, but get filehandle
  *   cookies on a per-superblock basis, depending on the mount flags
  */
-void nfs_fscache_get_client_cookie(struct nfs_client *clp)
+static bool nfs_fscache_get_client_key(struct nfs_client *clp,
+				       char *key, int *_len)
 {
 	const struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *) &clp->cl_addr;
 	const struct sockaddr_in *sin = (struct sockaddr_in *) &clp->cl_addr;
-	struct nfs_server_key key;
-	uint16_t len = sizeof(key.hdr);
 
-	memset(&key, 0, sizeof(key));
-	key.hdr.nfsversion = clp->rpc_ops->version;
-	key.hdr.minorversion = clp->cl_minorversion;
-	key.hdr.family = clp->cl_addr.ss_family;
+	*_len += snprintf(key + *_len, NFS_MAX_KEY_LEN - *_len,
+			  ",%u.%u,%x",
+			  clp->rpc_ops->version,
+			  clp->cl_minorversion,
+			  clp->cl_addr.ss_family);
 
 	switch (clp->cl_addr.ss_family) {
 	case AF_INET:
-		key.hdr.port = sin->sin_port;
-		key.ipv4_addr = sin->sin_addr;
-		len += sizeof(key.ipv4_addr);
-		break;
+		if (!nfs_append_int(key, _len, sin->sin_port) ||
+		    !nfs_append_int(key, _len, sin->sin_addr.s_addr))
+			return false;
+		return true;
 
 	case AF_INET6:
-		key.hdr.port = sin6->sin6_port;
-		key.ipv6_addr = sin6->sin6_addr;
-		len += sizeof(key.ipv6_addr);
-		break;
+		if (!nfs_append_int(key, _len, sin6->sin6_port) ||
+		    !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[0]) ||
+		    !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[1]) ||
+		    !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[2]) ||
+		    !nfs_append_int(key, _len, sin6->sin6_addr.s6_addr32[3]))
+			return false;
+		return true;
 
 	default:
 		printk(KERN_WARNING "NFS: Unknown network family '%d'\n",
 		       clp->cl_addr.ss_family);
-		clp->fscache = NULL;
-		return;
+		return false;
 	}
-
-	/* create a cache index for looking up filehandles */
-	clp->fscache = fscache_acquire_cookie(nfs_fscache_netfs.primary_index,
-					      &nfs_fscache_server_index_def,
-					      &key, len,
-					      NULL, 0,
-					      clp, 0, true);
-	dfprintk(FSCACHE, "NFS: get client cookie (0x%p/0x%p)\n",
-		 clp, clp->fscache);
-}
-
-/*
- * Dispose of a per-client cookie
- */
-void nfs_fscache_release_client_cookie(struct nfs_client *clp)
-{
-	dfprintk(FSCACHE, "NFS: releasing client cookie (0x%p/0x%p)\n",
-		 clp, clp->fscache);
-
-	fscache_relinquish_cookie(clp->fscache, NULL, false);
-	clp->fscache = NULL;
 }
 
 /*
- * Get the cache cookie for an NFS superblock.  We have to handle
- * uniquification here because the cache doesn't do it for us.
+ * Get the cache cookie for an NFS superblock.
  *
  * The default uniquifier is just an empty string, but it may be overridden
  * either by the 'fsc=xxx' option to mount, or by inheriting it from the parent
  * superblock across an automount point of some nature.
  */
-void nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int ulen)
+int nfs_fscache_get_super_cookie(struct super_block *sb, const char *uniq, int ulen)
 {
-	struct nfs_fscache_key *key, *xkey;
+	struct fscache_volume *vcookie;
 	struct nfs_server *nfss = NFS_SB(sb);
-	struct rb_node **p, *parent;
-	int diff;
+	unsigned int len = 3;
+	char *key;
 
-	nfss->fscache_key = NULL;
-	nfss->fscache = NULL;
-	if (!uniq) {
-		uniq = "";
-		ulen = 1;
+	if (uniq) {
+		nfss->fscache_uniq = kmemdup_nul(uniq, ulen, GFP_KERNEL);
+		if (!nfss->fscache_uniq)
+			return -ENOMEM;
 	}
 
-	key = kzalloc(sizeof(*key) + ulen, GFP_KERNEL);
+	key = kmalloc(NFS_MAX_KEY_LEN + 24, GFP_KERNEL);
 	if (!key)
-		return;
-
-	key->nfs_client = nfss->nfs_client;
-	key->key.super.s_flags = sb->s_flags & NFS_SB_MASK;
-	key->key.nfs_server.flags = nfss->flags;
-	key->key.nfs_server.rsize = nfss->rsize;
-	key->key.nfs_server.wsize = nfss->wsize;
-	key->key.nfs_server.acregmin = nfss->acregmin;
-	key->key.nfs_server.acregmax = nfss->acregmax;
-	key->key.nfs_server.acdirmin = nfss->acdirmin;
-	key->key.nfs_server.acdirmax = nfss->acdirmax;
-	key->key.nfs_server.fsid = nfss->fsid;
-	key->key.rpc_auth.au_flavor = nfss->client->cl_auth->au_flavor;
-
-	key->key.uniq_len = ulen;
-	memcpy(key->key.uniquifier, uniq, ulen);
-
-	spin_lock(&nfs_fscache_keys_lock);
-	p = &nfs_fscache_keys.rb_node;
-	parent = NULL;
-	while (*p) {
-		parent = *p;
-		xkey = rb_entry(parent, struct nfs_fscache_key, node);
-
-		if (key->nfs_client < xkey->nfs_client)
-			goto go_left;
-		if (key->nfs_client > xkey->nfs_client)
-			goto go_right;
-
-		diff = memcmp(&key->key, &xkey->key, sizeof(key->key));
-		if (diff < 0)
-			goto go_left;
-		if (diff > 0)
-			goto go_right;
-
-		if (key->key.uniq_len == 0)
-			goto non_unique;
-		diff = memcmp(key->key.uniquifier,
-			      xkey->key.uniquifier,
-			      key->key.uniq_len);
-		if (diff < 0)
-			goto go_left;
-		if (diff > 0)
-			goto go_right;
-		goto non_unique;
-
-	go_left:
-		p = &(*p)->rb_left;
-		continue;
-	go_right:
-		p = &(*p)->rb_right;
+		return -ENOMEM;
+
+	memcpy(key, "nfs", 3);
+	if (!nfs_fscache_get_client_key(nfss->nfs_client, key, &len) ||
+	    !nfs_append_int(key, &len, nfss->fsid.major) ||
+	    !nfs_append_int(key, &len, nfss->fsid.minor) ||
+	    !nfs_append_int(key, &len, sb->s_flags & NFS_SB_MASK) ||
+	    !nfs_append_int(key, &len, nfss->flags) ||
+	    !nfs_append_int(key, &len, nfss->rsize) ||
+	    !nfs_append_int(key, &len, nfss->wsize) ||
+	    !nfs_append_int(key, &len, nfss->acregmin) ||
+	    !nfs_append_int(key, &len, nfss->acregmax) ||
+	    !nfs_append_int(key, &len, nfss->acdirmin) ||
+	    !nfs_append_int(key, &len, nfss->acdirmax) ||
+	    !nfs_append_int(key, &len, nfss->client->cl_auth->au_flavor))
+		goto out;
+
+	if (ulen > 0) {
+		if (ulen > NFS_MAX_KEY_LEN - len)
+			goto out;
+		key[len++] = ',';
+		memcpy(key + len, uniq, ulen);
+		len += ulen;
 	}
-
-	rb_link_node(&key->node, parent, p);
-	rb_insert_color(&key->node, &nfs_fscache_keys);
-	spin_unlock(&nfs_fscache_keys_lock);
-	nfss->fscache_key = key;
+	key[len] = 0;
 
 	/* create a cache index for looking up filehandles */
-	nfss->fscache = fscache_acquire_cookie(nfss->nfs_client->fscache,
-					       &nfs_fscache_super_index_def,
-					       &key->key,
-					       sizeof(key->key) + ulen,
-					       NULL, 0,
-					       nfss, 0, true);
+	vcookie = fscache_acquire_volume(key,
+					 NULL, /* preferred_cache */
+					 0 /* coherency_data */);
 	dfprintk(FSCACHE, "NFS: get superblock cookie (0x%p/0x%p)\n",
-		 nfss, nfss->fscache);
-	return;
+		 nfss, vcookie);
+	if (IS_ERR(vcookie)) {
+		if (vcookie != ERR_PTR(-EBUSY)) {
+			kfree(key);
+			return PTR_ERR(vcookie);
+		}
+		pr_err("NFS: Cache volume key already in use (%s)\n", key);
+		vcookie = NULL;
+	}
+	nfss->fscache = vcookie;
 
-non_unique:
-	spin_unlock(&nfs_fscache_keys_lock);
+out:
 	kfree(key);
-	nfss->fscache_key = NULL;
-	nfss->fscache = NULL;
-	printk(KERN_WARNING "NFS:"
-	       " Cache request denied due to non-unique superblock keys\n");
+	return 0;
 }
 
 /*
@@ -213,29 +155,9 @@  void nfs_fscache_release_super_cookie(struct super_block *sb)
 	dfprintk(FSCACHE, "NFS: releasing superblock cookie (0x%p/0x%p)\n",
 		 nfss, nfss->fscache);
 
-	fscache_relinquish_cookie(nfss->fscache, NULL, false);
+	fscache_relinquish_volume(nfss->fscache, 0, false);
 	nfss->fscache = NULL;
-
-	if (nfss->fscache_key) {
-		spin_lock(&nfs_fscache_keys_lock);
-		rb_erase(&nfss->fscache_key->node, &nfs_fscache_keys);
-		spin_unlock(&nfs_fscache_keys_lock);
-		kfree(nfss->fscache_key);
-		nfss->fscache_key = NULL;
-	}
-}
-
-static void nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
-				  struct nfs_inode *nfsi)
-{
-	memset(auxdata, 0, sizeof(*auxdata));
-	auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
-	auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
-	auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
-	auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
-
-	if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
-		auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
+	kfree(nfss->fscache_uniq);
 }
 
 /*
@@ -254,10 +176,12 @@  void nfs_fscache_init_inode(struct inode *inode)
 	nfs_fscache_update_auxdata(&auxdata, nfsi);
 
 	nfsi->fscache = fscache_acquire_cookie(NFS_SB(inode->i_sb)->fscache,
-					       &nfs_fscache_inode_object_def,
-					       nfsi->fh.data, nfsi->fh.size,
-					       &auxdata, sizeof(auxdata),
-					       nfsi, nfsi->vfs_inode.i_size, false);
+					       0,
+					       nfsi->fh.data, /* index_key */
+					       nfsi->fh.size,
+					       &auxdata,      /* aux_data */
+					       sizeof(auxdata),
+					       i_size_read(&nfsi->vfs_inode));
 }
 
 /*
@@ -265,24 +189,15 @@  void nfs_fscache_init_inode(struct inode *inode)
  */
 void nfs_fscache_clear_inode(struct inode *inode)
 {
-	struct nfs_fscache_inode_auxdata auxdata;
 	struct nfs_inode *nfsi = NFS_I(inode);
 	struct fscache_cookie *cookie = nfs_i_fscache(inode);
 
 	dfprintk(FSCACHE, "NFS: clear cookie (0x%p/0x%p)\n", nfsi, cookie);
 
-	nfs_fscache_update_auxdata(&auxdata, nfsi);
-	fscache_relinquish_cookie(cookie, &auxdata, false);
+	fscache_relinquish_cookie(cookie, false);
 	nfsi->fscache = NULL;
 }
 
-static bool nfs_fscache_can_enable(void *data)
-{
-	struct inode *inode = data;
-
-	return !inode_is_open_for_write(inode);
-}
-
 /*
  * Enable or disable caching for a file that is being opened as appropriate.
  * The cookie is allocated when the inode is initialised, but is not enabled at
@@ -307,93 +222,31 @@  void nfs_fscache_open_file(struct inode *inode, struct file *filp)
 	struct nfs_fscache_inode_auxdata auxdata;
 	struct nfs_inode *nfsi = NFS_I(inode);
 	struct fscache_cookie *cookie = nfs_i_fscache(inode);
+	bool open_for_write = inode_is_open_for_write(inode);
 
 	if (!fscache_cookie_valid(cookie))
 		return;
 
-	nfs_fscache_update_auxdata(&auxdata, nfsi);
-
-	if (inode_is_open_for_write(inode)) {
+	fscache_use_cookie(cookie, open_for_write);
+	if (open_for_write) {
 		dfprintk(FSCACHE, "NFS: nfsi 0x%p disabling cache\n", nfsi);
-		clear_bit(NFS_INO_FSCACHE, &nfsi->flags);
-		fscache_disable_cookie(cookie, &auxdata, true);
-		fscache_uncache_all_inode_pages(cookie, inode);
-	} else {
-		dfprintk(FSCACHE, "NFS: nfsi 0x%p enabling cache\n", nfsi);
-		fscache_enable_cookie(cookie, &auxdata, nfsi->vfs_inode.i_size,
-				      nfs_fscache_can_enable, inode);
-		if (fscache_cookie_enabled(cookie))
-			set_bit(NFS_INO_FSCACHE, &NFS_I(inode)->flags);
+		nfs_fscache_update_auxdata(&auxdata, nfsi);
+		fscache_invalidate(cookie, &auxdata, i_size_read(inode),
+				   FSCACHE_INVAL_DIO_WRITE);
 	}
 }
 EXPORT_SYMBOL_GPL(nfs_fscache_open_file);
 
-/*
- * Release the caching state associated with a page, if the page isn't busy
- * interacting with the cache.
- * - Returns true (can release page) or false (page busy).
- */
-int nfs_fscache_release_page(struct page *page, gfp_t gfp)
-{
-	if (PageFsCache(page)) {
-		struct fscache_cookie *cookie = nfs_i_fscache(page->mapping->host);
-
-		BUG_ON(!cookie);
-		dfprintk(FSCACHE, "NFS: fscache releasepage (0x%p/0x%p/0x%p)\n",
-			 cookie, page, NFS_I(page->mapping->host));
-
-		if (!fscache_maybe_release_page(cookie, page, gfp))
-			return 0;
-
-		nfs_inc_fscache_stats(page->mapping->host,
-				      NFSIOS_FSCACHE_PAGES_UNCACHED);
-	}
-
-	return 1;
-}
-
-/*
- * Release the caching state associated with a page if undergoing complete page
- * invalidation.
- */
-void __nfs_fscache_invalidate_page(struct page *page, struct inode *inode)
+void nfs_fscache_release_file(struct inode *inode, struct file *filp)
 {
+	struct nfs_fscache_inode_auxdata auxdata;
+	struct nfs_inode *nfsi = NFS_I(inode);
 	struct fscache_cookie *cookie = nfs_i_fscache(inode);
 
-	BUG_ON(!cookie);
-
-	dfprintk(FSCACHE, "NFS: fscache invalidatepage (0x%p/0x%p/0x%p)\n",
-		 cookie, page, NFS_I(inode));
-
-	fscache_wait_on_page_write(cookie, page);
-
-	BUG_ON(!PageLocked(page));
-	fscache_uncache_page(cookie, page);
-	nfs_inc_fscache_stats(page->mapping->host,
-			      NFSIOS_FSCACHE_PAGES_UNCACHED);
-}
-
-/*
- * Handle completion of a page being read from the cache.
- * - Called in process (keventd) context.
- */
-static void nfs_readpage_from_fscache_complete(struct page *page,
-					       void *context,
-					       int error)
-{
-	dfprintk(FSCACHE,
-		 "NFS: readpage_from_fscache_complete (0x%p/0x%p/%d)\n",
-		 page, context, error);
-
-	/*
-	 * If the read completes with an error, mark the page with PG_checked,
-	 * unlock the page, and let the VM reissue the readpage.
-	 */
-	if (!error)
-		SetPageUptodate(page);
-	else
-		SetPageChecked(page);
-	unlock_page(page);
+	if (fscache_cookie_valid(cookie)) {
+		nfs_fscache_update_auxdata(&auxdata, nfsi);
+		fscache_unuse_cookie(cookie, &auxdata, NULL);
+	}
 }
 
 /*
@@ -402,8 +255,6 @@  static void nfs_readpage_from_fscache_complete(struct page *page,
 int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,
 				struct inode *inode, struct page *page)
 {
-	int ret;
-
 	dfprintk(FSCACHE,
 		 "NFS: readpage_from_fscache(fsc:%p/p:%p(i:%lx f:%lx)/0x%p)\n",
 		 nfs_i_fscache(inode), page, page->index, page->flags, inode);
@@ -413,31 +264,7 @@  int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,
 		return 1;
 	}
 
-	ret = fscache_read_or_alloc_page(nfs_i_fscache(inode),
-					 page,
-					 nfs_readpage_from_fscache_complete,
-					 ctx,
-					 GFP_KERNEL);
-
-	switch (ret) {
-	case 0: /* read BIO submitted (page in fscache) */
-		dfprintk(FSCACHE,
-			 "NFS:    readpage_from_fscache: BIO submitted\n");
-		nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK);
-		return ret;
-
-	case -ENOBUFS: /* inode not in cache */
-	case -ENODATA: /* page not in cache */
-		nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
-		dfprintk(FSCACHE,
-			 "NFS:    readpage_from_fscache %d\n", ret);
-		return 1;
-
-	default:
-		dfprintk(FSCACHE, "NFS:    readpage_from_fscache %d\n", ret);
-		nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL);
-	}
-	return ret;
+	return -ENOBUFS; // TODO: Use netfslib
 }
 
 /*
@@ -449,45 +276,10 @@  int __nfs_readpages_from_fscache(struct nfs_open_context *ctx,
 				 struct list_head *pages,
 				 unsigned *nr_pages)
 {
-	unsigned npages = *nr_pages;
-	int ret;
-
 	dfprintk(FSCACHE, "NFS: nfs_getpages_from_fscache (0x%p/%u/0x%p)\n",
-		 nfs_i_fscache(inode), npages, inode);
-
-	ret = fscache_read_or_alloc_pages(nfs_i_fscache(inode),
-					  mapping, pages, nr_pages,
-					  nfs_readpage_from_fscache_complete,
-					  ctx,
-					  mapping_gfp_mask(mapping));
-	if (*nr_pages < npages)
-		nfs_add_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_OK,
-				      npages);
-	if (*nr_pages > 0)
-		nfs_add_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_READ_FAIL,
-				      *nr_pages);
-
-	switch (ret) {
-	case 0: /* read submitted to the cache for all pages */
-		BUG_ON(!list_empty(pages));
-		BUG_ON(*nr_pages != 0);
-		dfprintk(FSCACHE,
-			 "NFS: nfs_getpages_from_fscache: submitted\n");
-
-		return ret;
-
-	case -ENOBUFS: /* some pages aren't cached and can't be */
-	case -ENODATA: /* some pages aren't cached */
-		dfprintk(FSCACHE,
-			 "NFS: nfs_getpages_from_fscache: no page: %d\n", ret);
-		return 1;
+		 nfs_i_fscache(inode), *nr_pages, inode);
 
-	default:
-		dfprintk(FSCACHE,
-			 "NFS: nfs_getpages_from_fscache: ret  %d\n", ret);
-	}
-
-	return ret;
+	return -ENOBUFS; // TODO: Use netfslib
 }
 
 /*
@@ -496,25 +288,9 @@  int __nfs_readpages_from_fscache(struct nfs_open_context *ctx,
  */
 void __nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)
 {
-	int ret;
-
 	dfprintk(FSCACHE,
 		 "NFS: readpage_to_fscache(fsc:%p/p:%p(i:%lx f:%lx)/%d)\n",
 		 nfs_i_fscache(inode), page, page->index, page->flags, sync);
 
-	ret = fscache_write_page(nfs_i_fscache(inode), page,
-				 inode->i_size, GFP_KERNEL);
-	dfprintk(FSCACHE,
-		 "NFS:     readpage_to_fscache: p:%p(i:%lu f:%lx) ret %d\n",
-		 page, page->index, page->flags, ret);
-
-	if (ret != 0) {
-		fscache_uncache_page(nfs_i_fscache(inode), page);
-		nfs_inc_fscache_stats(inode,
-				      NFSIOS_FSCACHE_PAGES_WRITTEN_FAIL);
-		nfs_inc_fscache_stats(inode, NFSIOS_FSCACHE_PAGES_UNCACHED);
-	} else {
-		nfs_inc_fscache_stats(inode,
-				      NFSIOS_FSCACHE_PAGES_WRITTEN_OK);
-	}
+	return; // TODO: Use netfslib
 }
diff --git a/fs/nfs/fscache.h b/fs/nfs/fscache.h
index 6754c8607230..26b6fb1cfd58 100644
--- a/fs/nfs/fscache.h
+++ b/fs/nfs/fscache.h
@@ -12,46 +12,10 @@ 
 #include <linux/nfs_mount.h>
 #include <linux/nfs4_mount.h>
 #include <linux/fscache.h>
+#include <linux/iversion.h>
 
 #ifdef CONFIG_NFS_FSCACHE
 
-/*
- * set of NFS FS-Cache objects that form a superblock key
- */
-struct nfs_fscache_key {
-	struct rb_node		node;
-	struct nfs_client	*nfs_client;	/* the server */
-
-	/* the elements of the unique key - as used by nfs_compare_super() and
-	 * nfs_compare_mount_options() to distinguish superblocks */
-	struct {
-		struct {
-			unsigned long	s_flags;	/* various flags
-							 * (& NFS_MS_MASK) */
-		} super;
-
-		struct {
-			struct nfs_fsid fsid;
-			int		flags;
-			unsigned int	rsize;		/* read size */
-			unsigned int	wsize;		/* write size */
-			unsigned int	acregmin;	/* attr cache timeouts */
-			unsigned int	acregmax;
-			unsigned int	acdirmin;
-			unsigned int	acdirmax;
-		} nfs_server;
-
-		struct {
-			rpc_authflavor_t au_flavor;
-		} rpc_auth;
-
-		/* uniquifier - can be used if nfs_server.flags includes
-		 * NFS_MOUNT_UNSHARED  */
-		u8 uniq_len;
-		char uniquifier[0];
-	} key;
-};
-
 /*
  * Definition of the auxiliary data attached to NFS inode storage objects
  * within the cache.
@@ -69,32 +33,18 @@  struct nfs_fscache_inode_auxdata {
 	u64	change_attr;
 };
 
-/*
- * fscache-index.c
- */
-extern struct fscache_netfs nfs_fscache_netfs;
-extern const struct fscache_cookie_def nfs_fscache_server_index_def;
-extern const struct fscache_cookie_def nfs_fscache_super_index_def;
-extern const struct fscache_cookie_def nfs_fscache_inode_object_def;
-
-extern int nfs_fscache_register(void);
-extern void nfs_fscache_unregister(void);
-
 /*
  * fscache.c
  */
-extern void nfs_fscache_get_client_cookie(struct nfs_client *);
-extern void nfs_fscache_release_client_cookie(struct nfs_client *);
-
-extern void nfs_fscache_get_super_cookie(struct super_block *, const char *, int);
+extern int nfs_fscache_get_super_cookie(struct super_block *, const char *, int);
 extern void nfs_fscache_release_super_cookie(struct super_block *);
 
 extern void nfs_fscache_init_inode(struct inode *);
 extern void nfs_fscache_clear_inode(struct inode *);
 extern void nfs_fscache_open_file(struct inode *, struct file *);
+extern void nfs_fscache_release_file(struct inode *, struct file *);
 
 extern void __nfs_fscache_invalidate_page(struct page *, struct inode *);
-extern int nfs_fscache_release_page(struct page *, gfp_t);
 
 extern int __nfs_readpage_from_fscache(struct nfs_open_context *,
 				       struct inode *, struct page *);
@@ -103,25 +53,17 @@  extern int __nfs_readpages_from_fscache(struct nfs_open_context *,
 					struct list_head *, unsigned *);
 extern void __nfs_readpage_to_fscache(struct inode *, struct page *, int);
 
-/*
- * wait for a page to complete writing to the cache
- */
-static inline void nfs_fscache_wait_on_page_write(struct nfs_inode *nfsi,
-						  struct page *page)
-{
-	if (PageFsCache(page))
-		fscache_wait_on_page_write(nfsi->fscache, page);
-}
-
-/*
- * release the caching state associated with a page if undergoing complete page
- * invalidation
- */
-static inline void nfs_fscache_invalidate_page(struct page *page,
-					       struct inode *inode)
+static inline int nfs_fscache_release_page(struct page *page, gfp_t gfp)
 {
-	if (PageFsCache(page))
-		__nfs_fscache_invalidate_page(page, inode);
+	if (PageFsCache(page)) {
+		if (!gfpflags_allow_blocking(gfp) || !(gfp & __GFP_FS))
+			return false;
+		wait_on_page_fscache(page);
+		fscache_note_page_release(nfs_i_fscache(page->mapping->host));
+		nfs_inc_fscache_stats(page->mapping->host,
+				      NFSIOS_FSCACHE_PAGES_UNCACHED);
+	}
+	return true;
 }
 
 /*
@@ -163,20 +105,32 @@  static inline void nfs_readpage_to_fscache(struct inode *inode,
 		__nfs_readpage_to_fscache(inode, page, sync);
 }
 
-/*
- * Invalidate the contents of fscache for this inode.  This will not sleep.
- */
-static inline void nfs_fscache_invalidate(struct inode *inode)
+static inline void nfs_fscache_update_auxdata(struct nfs_fscache_inode_auxdata *auxdata,
+					      struct nfs_inode *nfsi)
 {
-	fscache_invalidate(NFS_I(inode)->fscache);
+	memset(auxdata, 0, sizeof(*auxdata));
+	auxdata->mtime_sec  = nfsi->vfs_inode.i_mtime.tv_sec;
+	auxdata->mtime_nsec = nfsi->vfs_inode.i_mtime.tv_nsec;
+	auxdata->ctime_sec  = nfsi->vfs_inode.i_ctime.tv_sec;
+	auxdata->ctime_nsec = nfsi->vfs_inode.i_ctime.tv_nsec;
+
+	if (NFS_SERVER(&nfsi->vfs_inode)->nfs_client->rpc_ops->version == 4)
+		auxdata->change_attr = inode_peek_iversion_raw(&nfsi->vfs_inode);
 }
 
 /*
- * Wait for an object to finish being invalidated.
+ * Invalidate the contents of fscache for this inode.  This will not sleep.
  */
-static inline void nfs_fscache_wait_on_invalidate(struct inode *inode)
+static inline void nfs_fscache_invalidate(struct inode *inode, int flags)
 {
-	fscache_wait_on_invalidate(NFS_I(inode)->fscache);
+	struct nfs_fscache_inode_auxdata auxdata;
+	struct nfs_inode *nfsi = NFS_I(inode);
+
+	if (nfsi->fscache) {
+		nfs_fscache_update_auxdata(&auxdata, nfsi);
+		fscache_invalidate(nfsi->fscache, &auxdata,
+				   i_size_read(&nfsi->vfs_inode), flags);
+	}
 }
 
 /*
@@ -190,12 +144,6 @@  static inline const char *nfs_server_fscache_state(struct nfs_server *server)
 }
 
 #else /* CONFIG_NFS_FSCACHE */
-static inline int nfs_fscache_register(void) { return 0; }
-static inline void nfs_fscache_unregister(void) {}
-
-static inline void nfs_fscache_get_client_cookie(struct nfs_client *clp) {}
-static inline void nfs_fscache_release_client_cookie(struct nfs_client *clp) {}
-
 static inline void nfs_fscache_release_super_cookie(struct super_block *sb) {}
 
 static inline void nfs_fscache_init_inode(struct inode *inode) {}
@@ -207,11 +155,6 @@  static inline int nfs_fscache_release_page(struct page *page, gfp_t gfp)
 {
 	return 1; /* True: may release page */
 }
-static inline void nfs_fscache_invalidate_page(struct page *page,
-					       struct inode *inode) {}
-static inline void nfs_fscache_wait_on_page_write(struct nfs_inode *nfsi,
-						  struct page *page) {}
-
 static inline int nfs_readpage_from_fscache(struct nfs_open_context *ctx,
 					    struct inode *inode,
 					    struct page *page)
@@ -230,8 +173,7 @@  static inline void nfs_readpage_to_fscache(struct inode *inode,
 					   struct page *page, int sync) {}
 
 
-static inline void nfs_fscache_invalidate(struct inode *inode) {}
-static inline void nfs_fscache_wait_on_invalidate(struct inode *inode) {}
+static inline void nfs_fscache_invalidate(struct inode *inode, int flags) {}
 
 static inline const char *nfs_server_fscache_state(struct nfs_server *server)
 {
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index fda530d5e764..a918c3a834b6 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -209,7 +209,7 @@  void nfs_set_cache_invalid(struct inode *inode, unsigned long flags)
 	if (!nfs_has_xattr_cache(nfsi))
 		flags &= ~NFS_INO_INVALID_XATTR;
 	if (flags & NFS_INO_INVALID_DATA)
-		nfs_fscache_invalidate(inode);
+		nfs_fscache_invalidate(inode, 0);
 	flags &= ~(NFS_INO_REVAL_PAGECACHE | NFS_INO_REVAL_FORCED);
 
 	nfsi->cache_validity |= flags;
@@ -1289,6 +1289,7 @@  static int nfs_invalidate_mapping(struct inode *inode, struct address_space *map
 {
 	int ret;
 
+	nfs_fscache_invalidate(inode, 0);
 	if (mapping->nrpages != 0) {
 		if (S_ISREG(inode->i_mode)) {
 			ret = nfs_sync_mapping(mapping);
@@ -1300,7 +1301,6 @@  static int nfs_invalidate_mapping(struct inode *inode, struct address_space *map
 			return ret;
 	}
 	nfs_inc_stats(inode, NFSIOS_DATAINVALIDATE);
-	nfs_fscache_wait_on_invalidate(inode);
 
 	dfprintk(PAGECACHE, "NFS: (%s/%Lu) data cache invalidated\n",
 			inode->i_sb->s_id,
@@ -2374,10 +2374,6 @@  static int __init init_nfs_fs(void)
 	if (err < 0)
 		goto out9;
 
-	err = nfs_fscache_register();
-	if (err < 0)
-		goto out8;
-
 	err = nfsiod_start();
 	if (err)
 		goto out7;
@@ -2429,8 +2425,6 @@  static int __init init_nfs_fs(void)
 out6:
 	nfsiod_stop();
 out7:
-	nfs_fscache_unregister();
-out8:
 	unregister_pernet_subsys(&nfs_net_ops);
 out9:
 	nfs_sysfs_exit();
@@ -2445,7 +2439,6 @@  static void __exit exit_nfs_fs(void)
 	nfs_destroy_readpagecache();
 	nfs_destroy_inodecache();
 	nfs_destroy_nfspagecache();
-	nfs_fscache_unregister();
 	unregister_pernet_subsys(&nfs_net_ops);
 	rpc_proc_unregister(&init_net, "nfs");
 	unregister_nfs_fs();
diff --git a/fs/nfs/nfstrace.h b/fs/nfs/nfstrace.h
index b3aee261801e..317ce27bdc4b 100644
--- a/fs/nfs/nfstrace.h
+++ b/fs/nfs/nfstrace.h
@@ -42,7 +42,6 @@ 
 			{ BIT(NFS_INO_ACL_LRU_SET), "ACL_LRU_SET" }, \
 			{ BIT(NFS_INO_INVALIDATING), "INVALIDATING" }, \
 			{ BIT(NFS_INO_FSCACHE), "FSCACHE" }, \
-			{ BIT(NFS_INO_FSCACHE_LOCK), "FSCACHE_LOCK" }, \
 			{ BIT(NFS_INO_LAYOUTCOMMIT), "NEED_LAYOUTCOMMIT" }, \
 			{ BIT(NFS_INO_LAYOUTCOMMITTING), "LAYOUTCOMMIT" }, \
 			{ BIT(NFS_INO_LAYOUTSTATS), "LAYOUTSTATS" }, \
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index 3aced401735c..6ab5eeb000dc 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1204,42 +1204,42 @@  static int nfs_compare_super(struct super_block *sb, struct fs_context *fc)
 }
 
 #ifdef CONFIG_NFS_FSCACHE
-static void nfs_get_cache_cookie(struct super_block *sb,
-				 struct nfs_fs_context *ctx)
+static int nfs_get_cache_cookie(struct super_block *sb,
+				struct nfs_fs_context *ctx)
 {
 	struct nfs_server *nfss = NFS_SB(sb);
 	char *uniq = NULL;
 	int ulen = 0;
 
-	nfss->fscache_key = NULL;
 	nfss->fscache = NULL;
 
 	if (!ctx)
-		return;
+		return 0;
 
 	if (ctx->clone_data.sb) {
 		struct nfs_server *mnt_s = NFS_SB(ctx->clone_data.sb);
 		if (!(mnt_s->options & NFS_OPTION_FSCACHE))
-			return;
-		if (mnt_s->fscache_key) {
-			uniq = mnt_s->fscache_key->key.uniquifier;
-			ulen = mnt_s->fscache_key->key.uniq_len;
+			return 0;
+		if (mnt_s->fscache_uniq) {
+			uniq = mnt_s->fscache_uniq;
+			ulen = strlen(uniq);
 		}
 	} else {
 		if (!(ctx->options & NFS_OPTION_FSCACHE))
-			return;
+			return 0;
 		if (ctx->fscache_uniq) {
 			uniq = ctx->fscache_uniq;
 			ulen = strlen(ctx->fscache_uniq);
 		}
 	}
 
-	nfs_fscache_get_super_cookie(sb, uniq, ulen);
+	return nfs_fscache_get_super_cookie(sb, uniq, ulen);
 }
 #else
-static void nfs_get_cache_cookie(struct super_block *sb,
-				 struct nfs_fs_context *ctx)
+static int nfs_get_cache_cookie(struct super_block *sb,
+				struct nfs_fs_context *ctx)
 {
+	return 0;
 }
 #endif
 
@@ -1299,7 +1299,9 @@  int nfs_get_tree_common(struct fs_context *fc)
 			s->s_blocksize_bits = bsize;
 			s->s_blocksize = 1U << bsize;
 		}
-		nfs_get_cache_cookie(s, ctx);
+		error = nfs_get_cache_cookie(s, ctx);
+		if (error < 0)
+			goto error_splat_super;
 	}
 
 	error = nfs_get_root(s, fc);
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 9b7619ce17a7..2b322170372a 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -294,6 +294,7 @@  static void nfs_grow_file(struct page *page, unsigned int offset, unsigned int c
 	nfs_inc_stats(inode, NFSIOS_EXTENDWRITE);
 out:
 	spin_unlock(&inode->i_lock);
+	nfs_fscache_invalidate(inode, 0);
 }
 
 /* A writeback failed: mark the page as bad, and invalidate the page cache */
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index 05f249f20f55..00835bacd236 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
@@ -275,7 +275,6 @@  struct nfs4_copy_state {
 #define NFS_INO_ACL_LRU_SET	(2)		/* Inode is on the LRU list */
 #define NFS_INO_INVALIDATING	(3)		/* inode is being invalidated */
 #define NFS_INO_FSCACHE		(5)		/* inode can be cached by FS-Cache */
-#define NFS_INO_FSCACHE_LOCK	(6)		/* FS-Cache cookie management lock */
 #define NFS_INO_FORCE_READDIR	(7)		/* force readdirplus */
 #define NFS_INO_LAYOUTCOMMIT	(9)		/* layoutcommit required */
 #define NFS_INO_LAYOUTCOMMITTING (10)		/* layoutcommit inflight */
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 2a9acbfe00f0..77b2dba27bbb 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -120,11 +120,6 @@  struct nfs_client {
 	 * This is used to generate the mv0 callback address.
 	 */
 	char			cl_ipaddr[48];
-
-#ifdef CONFIG_NFS_FSCACHE
-	struct fscache_cookie	*fscache;	/* client index cache cookie */
-#endif
-
 	struct net		*cl_net;
 	struct list_head	pending_cb_stateids;
 };
@@ -194,8 +189,8 @@  struct nfs_server {
 	struct nfs_auth_info	auth_info;	/* parsed auth flavors */
 
 #ifdef CONFIG_NFS_FSCACHE
-	struct nfs_fscache_key	*fscache_key;	/* unique key for superblock */
-	struct fscache_cookie	*fscache;	/* superblock cookie */
+	struct fscache_volume	*fscache;	/* superblock cookie */
+	char			*fscache_uniq;	/* Uniquifier (or NULL) */
 #endif
 
 	u32			pnfs_blksize;	/* layout_blksize attr */