| Message ID | 20220223211305.296816-11-trondmy@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Readdir improvements | expand |
Hi Trond, On Wed, Feb 23, 2022 at 8:25 PM <trondmy@kernel.org> wrote: > > From: Trond Myklebust <trond.myklebust@hammerspace.com> > > When reading a very large directory, we want to try to keep the page > cache up to date if doing so is inexpensive. With the change to allow > readdir to continue reading even when the cache is incomplete, we no > longer need to fall back to uncached readdir in order to scale to large > directories. > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> As of this patch, cthon tests are passing again. Anna Anna > --- > fs/nfs/dir.c | 23 +++-------------------- > 1 file changed, 3 insertions(+), 20 deletions(-) > > diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c > index 9b0f13b52dbf..982b5dbe30d7 100644 > --- a/fs/nfs/dir.c > +++ b/fs/nfs/dir.c > @@ -986,28 +986,11 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) > return res; > } > > -static bool nfs_readdir_dont_search_cache(struct nfs_readdir_descriptor *desc) > -{ > - struct address_space *mapping = desc->file->f_mapping; > - struct inode *dir = file_inode(desc->file); > - unsigned int dtsize = NFS_SERVER(dir)->dtsize; > - loff_t size = i_size_read(dir); > - > - /* > - * Default to uncached readdir if the page cache is empty, and > - * we're looking for a non-zero cookie in a large directory. > - */ > - return desc->dir_cookie != 0 && mapping->nrpages == 0 && size > dtsize; > -} > - > /* Search for desc->dir_cookie from the beginning of the page cache */ > static int readdir_search_pagecache(struct nfs_readdir_descriptor *desc) > { > int res; > > - if (nfs_readdir_dont_search_cache(desc)) > - return -EBADCOOKIE; > - > do { > if (desc->page_index == 0) { > desc->current_index = 0; > @@ -1262,10 +1245,10 @@ static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence) > } > if (offset != filp->f_pos) { > filp->f_pos = offset; > - if (!nfs_readdir_use_cookie(filp)) { > + dir_ctx->page_index = 0; > + if (!nfs_readdir_use_cookie(filp)) > dir_ctx->dir_cookie = 0; > - dir_ctx->page_index = 0; > - } else > + else > dir_ctx->dir_cookie = offset; > if (offset == 0) > memset(dir_ctx->verf, 0, sizeof(dir_ctx->verf)); > -- > 2.35.1 >
On Thu, 2022-02-24 at 11:55 -0500, Anna Schumaker wrote: > Hi Trond, > > On Wed, Feb 23, 2022 at 8:25 PM <trondmy@kernel.org> wrote: > > > > From: Trond Myklebust <trond.myklebust@hammerspace.com> > > > > When reading a very large directory, we want to try to keep the > > page > > cache up to date if doing so is inexpensive. With the change to > > allow > > readdir to continue reading even when the cache is incomplete, we > > no > > longer need to fall back to uncached readdir in order to scale to > > large > > directories. > > > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com> > > As of this patch, cthon tests are passing again. > I'm going to push out a v8 patchset. I'm just waiting for a few more comments, etc. Anyhow, I wonder if the fact that we're not initialising the verifier in nfs_opendir() is part of the problem here. I've added a patch to do this. I also found an issue with the nfs2/nfs3_decode_dirent() return values (and have a fix for that), however I'm assuming that is not the problem you're seeing, since you were reporting issues with NFSv4, which is unaffected.
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 9b0f13b52dbf..982b5dbe30d7 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -986,28 +986,11 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) return res; } -static bool nfs_readdir_dont_search_cache(struct nfs_readdir_descriptor *desc) -{ - struct address_space *mapping = desc->file->f_mapping; - struct inode *dir = file_inode(desc->file); - unsigned int dtsize = NFS_SERVER(dir)->dtsize; - loff_t size = i_size_read(dir); - - /* - * Default to uncached readdir if the page cache is empty, and - * we're looking for a non-zero cookie in a large directory. - */ - return desc->dir_cookie != 0 && mapping->nrpages == 0 && size > dtsize; -} - /* Search for desc->dir_cookie from the beginning of the page cache */ static int readdir_search_pagecache(struct nfs_readdir_descriptor *desc) { int res; - if (nfs_readdir_dont_search_cache(desc)) - return -EBADCOOKIE; - do { if (desc->page_index == 0) { desc->current_index = 0; @@ -1262,10 +1245,10 @@ static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence) } if (offset != filp->f_pos) { filp->f_pos = offset; - if (!nfs_readdir_use_cookie(filp)) { + dir_ctx->page_index = 0; + if (!nfs_readdir_use_cookie(filp)) dir_ctx->dir_cookie = 0; - dir_ctx->page_index = 0; - } else + else dir_ctx->dir_cookie = offset; if (offset == 0) memset(dir_ctx->verf, 0, sizeof(dir_ctx->verf));