From patchwork Wed Feb 23 13:40:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4970FC4332F for ; Wed, 23 Feb 2022 13:41:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241003AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237639AbiBWNlH (ORCPT ); Wed, 23 Feb 2022 08:41:07 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id AE59AAC05D for ; Wed, 23 Feb 2022 05:40:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623638; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dRQpV3tuiAuRESH2Zw0RYAHVDBG1KtyuQE/uQ/zmGug=; b=DKhRqsECglyhAXtfx2rKE3JJ3oQpiNvvHMPgWl28EE3sAJCrP0CNblUUNU0mDL6O+8Ib+A Z49lRq9Rjdc71L6FzP/y0DqURpEkGYCi8ndksN/EPGcPKnQnJA5AssmM3przdwGimuGvGC PVCDGIpvatCk6EQ/aOm19gYjviDYd2I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-482-TuJApbJgMw63qrgP8U-j-A-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: TuJApbJgMw63qrgP8U-j-A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A01AD801AAD for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 52876106A7B2 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id E948B10C30F0; Wed, 23 Feb 2022 08:40:35 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 1/8] NFS: save the directory's change attribute on pagecache pages Date: Wed, 23 Feb 2022 08:40:28 -0500 Message-Id: In-Reply-To: <20220221160851.15508-14-trondmy@kernel.org> References: <20220221160851.15508-14-trondmy@kernel.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org After a pagecache page has been filled with entries, set PagePrivate and the directory's change attribute on the page. This will help us perform per-page invalidations in a later patch. Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 9f48c75dbf4c..79bdcedc0cad 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -54,6 +54,9 @@ static int nfs_closedir(struct inode *, struct file *); static int nfs_readdir(struct file *, struct dir_context *); static int nfs_fsync_dir(struct file *, loff_t, loff_t, int); static loff_t nfs_llseek_dir(struct file *, loff_t, int); +static void nfs_readdir_invalidatepage(struct page *, + unsigned int, unsigned int); +static int nfs_readdir_clear_page(struct page*, gfp_t); static void nfs_readdir_clear_array(struct page*); const struct file_operations nfs_dir_operations = { @@ -66,6 +69,8 @@ const struct file_operations nfs_dir_operations = { }; const struct address_space_operations nfs_dir_aops = { + .invalidatepage = nfs_readdir_invalidatepage, + .releasepage = nfs_readdir_clear_page, .freepage = nfs_readdir_clear_array, }; @@ -212,6 +217,27 @@ static void nfs_readdir_page_init_array(struct page *page, u64 last_cookie) array->last_cookie = last_cookie; array->cookies_are_ordered = 1; kunmap_atomic(array); + set_page_private(page, 0); +} + +static int +nfs_readdir_clear_page(struct page *page, gfp_t gfp_mask) +{ + detach_page_private(page); + return 1; +} + +static void +nfs_readdir_invalidatepage(struct page *page, unsigned int offset, + unsigned int length) +{ + nfs_readdir_clear_page(page, GFP_KERNEL); +} + +static void +nfs_readdir_set_page_verifier(struct page *page, unsigned long verf) +{ + attach_page_private(page, (void *)verf); } /* @@ -794,6 +820,8 @@ static int nfs_readdir_page_filler(struct nfs_readdir_descriptor *desc, if (status != -ENOSPC) continue; + nfs_readdir_set_page_verifier(page, desc->dir_verifier); + if (page->mapping != mapping) { if (!--narrays) break; @@ -822,10 +850,13 @@ static int nfs_readdir_page_filler(struct nfs_readdir_descriptor *desc, case -EBADCOOKIE: if (entry->eof) { nfs_readdir_page_set_eof(page); + nfs_readdir_set_page_verifier(page, desc->dir_verifier); status = 0; } break; case -ENOSPC: + nfs_readdir_set_page_verifier(page, desc->dir_verifier); + fallthrough; case -EAGAIN: status = 0; break; From patchwork Wed Feb 23 13:40:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756944 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B61CC433EF for ; Wed, 23 Feb 2022 13:41:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240999AbiBWNlJ (ORCPT ); Wed, 23 Feb 2022 08:41:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241001AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C8B10AC065 for ; Wed, 23 Feb 2022 05:40:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=orXhI6SXyypSX1JBBjrp4e1ep+yrNv2FjfbJmeW8+58=; b=DeipjoVR7sdRq+f+XsQNVrqbNECHO49ZxueF5LSiOEZ6J1p9iDMr/43w1+f8lwpOlygNDN CIbywqTsZIUJM5RFsKKh+p246PVVXwQilTJNd92ZakkQefSprSD07vj9mQ2nI2tziQC4A8 6Dlmg6oa/pXuELMY5u0yDClMpFS5nsM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-875VWFbrNNSQn9ye5xlMxQ-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: 875VWFbrNNSQn9ye5xlMxQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9F72C1854E26 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 569CF2B4D2 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id EC83010C30F1; Wed, 23 Feb 2022 08:40:35 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 2/8] NFSv4: Send GETATTR with READDIR Date: Wed, 23 Feb 2022 08:40:29 -0500 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org For each batch of entries, track whether the directory has changed. We can use this information to better manage the cache when reading long directories. Signed-off-by: Benjamin Coddington --- fs/nfs/nfs42proc.c | 2 +- fs/nfs/nfs4proc.c | 29 +++++++++++++++++++++-------- fs/nfs/nfs4xdr.c | 6 ++++++ include/linux/nfs_fs_sb.h | 5 +++++ include/linux/nfs_xdr.h | 2 ++ 5 files changed, 35 insertions(+), 9 deletions(-) diff --git a/fs/nfs/nfs42proc.c b/fs/nfs/nfs42proc.c index 882bf84484ac..3ab54228b2ed 100644 --- a/fs/nfs/nfs42proc.c +++ b/fs/nfs/nfs42proc.c @@ -1082,7 +1082,7 @@ static int _nfs42_proc_clone(struct rpc_message *msg, struct file *src_f, if (!res.dst_fattr) return -ENOMEM; - nfs4_bitmask_set(dst_bitmask, server->cache_consistency_bitmask, + nfs4_bitmask_set(dst_bitmask, server->cache_consistency_bitmask_nl, dst_inode, NFS_INO_INVALID_BLOCKS); status = nfs4_call_sync(server->client, server, msg, diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 73a9b6de666c..45285447c077 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -3665,7 +3665,7 @@ static void nfs4_close_prepare(struct rpc_task *task, void *data) /* Close-to-open cache consistency revalidation */ if (!nfs4_have_delegation(inode, FMODE_READ)) { nfs4_bitmask_set(calldata->arg.bitmask_store, - server->cache_consistency_bitmask, + server->cache_consistency_bitmask_nl, inode, 0); calldata->arg.bitmask = calldata->arg.bitmask_store; } else @@ -3905,7 +3905,12 @@ static int _nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *f memcpy(server->cache_consistency_bitmask, res.attr_bitmask, sizeof(server->cache_consistency_bitmask)); server->cache_consistency_bitmask[0] &= FATTR4_WORD0_CHANGE|FATTR4_WORD0_SIZE; server->cache_consistency_bitmask[1] &= FATTR4_WORD1_TIME_METADATA|FATTR4_WORD1_TIME_MODIFY; - server->cache_consistency_bitmask[2] = 0; + server->cache_consistency_bitmask[2] = res.attr_bitmask[2] & FATTR4_WORD2_SECURITY_LABEL; + + memcpy(server->cache_consistency_bitmask_nl, server->cache_consistency_bitmask, sizeof(server->cache_consistency_bitmask)); + server->cache_consistency_bitmask_nl[2] = 0; + + /* Avoid a regression due to buggy server */ for (i = 0; i < ARRAY_SIZE(res.exclcreat_bitmask); i++) @@ -4576,7 +4581,7 @@ static int _nfs4_proc_access(struct inode *inode, struct nfs_access_entry *entry res.fattr = nfs_alloc_fattr(); if (res.fattr == NULL) return -ENOMEM; - args.bitmask = server->cache_consistency_bitmask; + args.bitmask = server->cache_consistency_bitmask_nl; } status = nfs4_call_sync(server->client, server, &msg, &args.seq_args, &res.seq_res, 0); if (!status) { @@ -5098,14 +5103,19 @@ static int _nfs4_proc_readdir(struct nfs_readdir_arg *nr_arg, .rpc_resp = &res, .rpc_cred = nr_arg->cred, }; - int status; + int status = -ENOMEM; dprintk("%s: dentry = %pd2, cookie = %llu\n", __func__, nr_arg->dentry, (unsigned long long)nr_arg->cookie); if (!(server->caps & NFS_CAP_SECURITY_LABEL)) - args.bitmask = server->attr_bitmask_nl; + args.bitmask = server->cache_consistency_bitmask_nl; else - args.bitmask = server->attr_bitmask; + args.bitmask = server->cache_consistency_bitmask; + + res.dir_attr = nfs_alloc_fattr(); + if (res.dir_attr == NULL) + goto out; + res.server = server; nfs4_setup_readdir(nr_arg->cookie, nr_arg->verf, nr_arg->dentry, &args); res.pgbase = args.pgbase; @@ -5118,6 +5128,9 @@ static int _nfs4_proc_readdir(struct nfs_readdir_arg *nr_arg, nfs_invalidate_atime(dir); + nfs_refresh_inode(dir, res.dir_attr); + nfs_free_fattr(res.dir_attr); +out: dprintk("%s: returns %d\n", __func__, status); return status; } @@ -5583,7 +5596,7 @@ static void nfs4_proc_write_setup(struct nfs_pgio_header *hdr, hdr->res.fattr = NULL; } else { nfs4_bitmask_set(hdr->args.bitmask_store, - server->cache_consistency_bitmask, + server->cache_consistency_bitmask_nl, hdr->inode, NFS_INO_INVALID_BLOCKS); hdr->args.bitmask = hdr->args.bitmask_store; } @@ -6622,7 +6635,7 @@ static int _nfs4_proc_delegreturn(struct inode *inode, const struct cred *cred, data->args.fhandle = &data->fh; data->args.stateid = &data->stateid; nfs4_bitmask_set(data->args.bitmask_store, - server->cache_consistency_bitmask, inode, 0); + server->cache_consistency_bitmask_nl, inode, 0); data->args.bitmask = data->args.bitmask_store; nfs_copy_fh(&data->fh, NFS_FH(inode)); nfs4_stateid_copy(&data->stateid, stateid); diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c index b7780b97dc4d..1cd0d49ef992 100644 --- a/fs/nfs/nfs4xdr.c +++ b/fs/nfs/nfs4xdr.c @@ -469,10 +469,12 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req, #define NFS4_enc_readdir_sz (compound_encode_hdr_maxsz + \ encode_sequence_maxsz + \ encode_putfh_maxsz + \ + encode_getattr_maxsz + \ encode_readdir_maxsz) #define NFS4_dec_readdir_sz (compound_decode_hdr_maxsz + \ decode_sequence_maxsz + \ decode_putfh_maxsz + \ + decode_getattr_maxsz + \ decode_readdir_maxsz) #define NFS4_enc_write_sz (compound_encode_hdr_maxsz + \ encode_sequence_maxsz + \ @@ -2529,6 +2531,7 @@ static void nfs4_xdr_enc_readdir(struct rpc_rqst *req, struct xdr_stream *xdr, encode_compound_hdr(xdr, req, &hdr); encode_sequence(xdr, &args->seq_args, &hdr); encode_putfh(xdr, args->fh, &hdr); + encode_getfattr(xdr, args->bitmask, &hdr); encode_readdir(xdr, args, req, &hdr); rpc_prepare_reply_pages(req, args->pages, args->pgbase, @@ -6769,6 +6772,9 @@ static int nfs4_xdr_dec_readdir(struct rpc_rqst *rqstp, struct xdr_stream *xdr, if (status) goto out; status = decode_putfh(xdr); + if (status) + goto out; + status = decode_getfattr(xdr, res->dir_attr, res->server); if (status) goto out; status = decode_readdir(xdr, rqstp, res); diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h index ca0959e51e81..04bc827e4367 100644 --- a/include/linux/nfs_fs_sb.h +++ b/include/linux/nfs_fs_sb.h @@ -213,6 +213,11 @@ struct nfs_server { of change attribute, size, ctime and mtime attributes supported by the server */ + u32 cache_consistency_bitmask_nl[3]; + /* V4 bitmask representing the subset + of change attribute, size, ctime + and mtime attributes supported by + the server excluding label support */ u32 acl_bitmask; /* V4 bitmask representing the ACEs that are supported on this filesystem */ diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 728cb0c1f0b6..fbb8b7695c30 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1139,6 +1139,8 @@ struct nfs4_readdir_res { struct nfs4_sequence_res seq_res; nfs4_verifier verifier; unsigned int pgbase; + struct nfs_fattr *dir_attr; + const struct nfs_server *server; }; struct nfs4_readlink { From patchwork Wed Feb 23 13:40:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB68CC433F5 for ; Wed, 23 Feb 2022 13:42:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240976AbiBWNlK (ORCPT ); Wed, 23 Feb 2022 08:41:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238752AbiBWNlH (ORCPT ); Wed, 23 Feb 2022 08:41:07 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C7D94AC05F for ; Wed, 23 Feb 2022 05:40:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=boVjQySP0bow3lHCS1C6zSDqVgfR35bAsnNUPhH4nmY=; b=Xg72O65iTYYitB0Wjy1KnF9unj6bjhTpvoAZHar1llrP5sX0e9QyOjEL870E3z2/0GRi03 TBIdQ4OaZE7Es0KOcs1s20VFDcOEV9E5+4i0/wMHvaOm6pl64FWn472Q9bUF7cGYF0LTtX 4Mi3GwqYbHXxGWlF9qDdlpfclyjoMOE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-dwOF8fY8P_SryZCTWCz-gw-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: dwOF8fY8P_SryZCTWCz-gw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A01A01091DA1 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 528B67C03E for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id F1C5D10C30F7; Wed, 23 Feb 2022 08:40:35 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 3/8] NFS: Add a struct to track readdir pagecache location Date: Wed, 23 Feb 2022 08:40:30 -0500 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Directory entries in the NFS readdir pagecache are referenced by their cookie value and offset. By defining a structure to group these values, we'll simplify changes to validate pagecache pages in patches that follow. Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 46 ++++++++++++++++++++---------------------- include/linux/nfs_fs.h | 6 ++++++ 2 files changed, 28 insertions(+), 24 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 79bdcedc0cad..009187c0ae0f 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -158,11 +158,10 @@ struct nfs_readdir_descriptor { struct file *file; struct page *page; struct dir_context *ctx; - pgoff_t page_index; pgoff_t page_index_max; u64 dir_cookie; - u64 last_cookie; u64 dup_cookie; + struct nfs_dir_page_cursor pgc; loff_t current_index; loff_t prev_index; @@ -172,7 +171,6 @@ struct nfs_readdir_descriptor { unsigned long gencount; unsigned long attr_gencount; unsigned int page_fill_misses; - unsigned int cache_entry_index; unsigned int buffer_fills; unsigned int dtsize; signed char duped; @@ -457,7 +455,7 @@ static int nfs_readdir_search_for_pos(struct nfs_cache_array *array, index = (unsigned int)diff; desc->dir_cookie = array->array[index].cookie; - desc->cache_entry_index = index; + desc->pgc.entry_index = index; return 0; out_eof: desc->eof = true; @@ -526,7 +524,7 @@ static int nfs_readdir_search_for_cookie(struct nfs_cache_array *array, else desc->ctx->pos = new_pos; desc->prev_index = new_pos; - desc->cache_entry_index = i; + desc->pgc.entry_index = i; return 0; } } @@ -553,9 +551,9 @@ static int nfs_readdir_search_array(struct nfs_readdir_descriptor *desc) status = nfs_readdir_search_for_cookie(array, desc); if (status == -EAGAIN) { - desc->last_cookie = array->last_cookie; + desc->pgc.index_cookie = array->last_cookie; desc->current_index += array->size; - desc->page_index++; + desc->pgc.page_index++; } kunmap_atomic(array); return status; @@ -968,8 +966,8 @@ static struct page * nfs_readdir_page_get_cached(struct nfs_readdir_descriptor *desc) { return nfs_readdir_page_get_locked(desc->file->f_mapping, - desc->page_index, - desc->last_cookie); + desc->pgc.page_index, + desc->pgc.index_cookie); } #define NFS_READDIR_PAGE_FILL_MISS_MAX 5 @@ -1001,10 +999,10 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) if (nfs_readdir_page_needs_filling(desc->page)) { if (!nfs_readdir_may_fill_pagecache(desc)) return -EBADCOOKIE; - desc->page_index_max = desc->page_index; + desc->page_index_max = desc->pgc.page_index; trace_nfs_readdir_cache_fill(desc->file, nfsi->cookieverf, - desc->last_cookie, - desc->page_index, desc->dtsize); + desc->pgc.index_cookie, + desc->pgc.page_index, desc->dtsize); res = nfs_readdir_xdr_to_array(desc, nfsi->cookieverf, verf, &desc->page, 1); if (res < 0) { @@ -1012,7 +1010,7 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) trace_nfs_readdir_cache_fill_done(inode, res); if (res == -EBADCOOKIE || res == -ENOTSYNC) { invalidate_inode_pages2(desc->file->f_mapping); - desc->page_index = 0; + desc->pgc.page_index = 0; return -EAGAIN; } return res; @@ -1020,7 +1018,7 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) /* * Set the cookie verifier if the page cache was empty */ - if (desc->page_index == 0) + if (desc->pgc.page_index == 0) memcpy(nfsi->cookieverf, verf, sizeof(nfsi->cookieverf)); desc->page_fill_misses++; @@ -1040,10 +1038,10 @@ static int readdir_search_pagecache(struct nfs_readdir_descriptor *desc) int res; do { - if (desc->page_index == 0) { + if (desc->pgc.page_index == 0) { desc->current_index = 0; desc->prev_index = 0; - desc->last_cookie = 0; + desc->pgc.index_cookie = 0; } res = find_and_lock_cache_page(desc); } while (res == -EAGAIN); @@ -1061,7 +1059,7 @@ static void nfs_do_filldir(struct nfs_readdir_descriptor *desc, unsigned int i = 0; array = kmap(desc->page); - for (i = desc->cache_entry_index; i < array->size; i++) { + for (i = desc->pgc.entry_index; i < array->size; i++) { struct nfs_cache_array_entry *ent; ent = &array->array[i]; @@ -1119,13 +1117,13 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) if (!arrays[0]) goto out; - desc->page_index = 0; - desc->cache_entry_index = 0; - desc->last_cookie = desc->dir_cookie; + desc->pgc.page_index = 0; + desc->pgc.entry_index = 0; + desc->pgc.index_cookie = desc->dir_cookie; desc->duped = 0; desc->page_index_max = 0; - trace_nfs_readdir_uncached(desc->file, desc->verf, desc->last_cookie, + trace_nfs_readdir_uncached(desc->file, desc->verf, desc->pgc.index_cookie, -1, desc->dtsize); status = nfs_readdir_xdr_to_array(desc, desc->verf, verf, arrays, sz); @@ -1258,7 +1256,7 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) } if (res == -ETOOSMALL && desc->plus) { nfs_zap_caches(inode); - desc->page_index = 0; + desc->pgc.page_index = 0; desc->plus = false; desc->eof = false; continue; @@ -1271,7 +1269,7 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) if (desc->eob || desc->eof) break; /* Grow the dtsize if we have to go back for more pages */ - if (desc->page_index == desc->page_index_max) + if (desc->pgc.page_index == desc->page_index_max) nfs_grow_dtsize(desc); } while (!desc->eob && !desc->eof); @@ -1280,7 +1278,7 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) dir_ctx->dup_cookie = desc->dup_cookie; dir_ctx->duped = desc->duped; dir_ctx->attr_gencount = desc->attr_gencount; - dir_ctx->page_index = desc->page_index; + dir_ctx->page_index = desc->pgc.page_index; dir_ctx->page_fill_misses = desc->page_fill_misses; dir_ctx->eof = desc->eof; dir_ctx->dtsize = desc->dtsize; diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h index 0a5425a58bbd..2f5ded282477 100644 --- a/include/linux/nfs_fs.h +++ b/include/linux/nfs_fs.h @@ -99,6 +99,12 @@ struct nfs_open_context { struct rcu_head rcu_head; }; +struct nfs_dir_page_cursor { + __u64 index_cookie; + pgoff_t page_index; + unsigned int entry_index; +}; + struct nfs_open_dir_context { struct list_head list; atomic_t cache_hits; From patchwork Wed Feb 23 13:40:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B4B1C433F5 for ; Wed, 23 Feb 2022 13:41:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241000AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240999AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2E371AC06A for ; Wed, 23 Feb 2022 05:40:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vha02VFW5Iw1IQcPMRwnb3PRTXDYtnVPVn1kQ09IAtM=; b=Ugj1zYDB7Mp3Uh92fOw/rPXjRb45FWDE5GyYixdCShtDWkL4j6l0acheV2QzJTIVMdsEPU tk+CZcCDq4dBkWH3uYNQE7s262vgxYzp3z041b/jolkOWS1m5F1ZCFMKC3Rv4XIss/Plz8 kFbVp1t8IVuKWCEtK/RrFDhWOLIt9S4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-407-k1icQD9nM-aRhaCh4ULf2A-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: k1icQD9nM-aRhaCh4ULf2A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A1B88800425 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 605EA7A537 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id 01DA310C3101; Wed, 23 Feb 2022 08:40:36 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 4/8] NFS: Keep the readdir pagecache cursor updated Date: Wed, 23 Feb 2022 08:40:31 -0500 Message-Id: <5479c8c5be9cf3f387edac54f170461f8f7b89e2.1645623510.git.bcodding@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Whenever we successfully locate our dir_cookie within the pagecache, or finish emitting entries to userspace, update the pagecache cursor. These updates provide marker points to validate pagecache pages in a future patch. Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 009187c0ae0f..2b1a0c1cdce4 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -154,6 +154,10 @@ struct nfs_cache_array { struct nfs_cache_array_entry array[]; }; +static const int cache_entries_per_page = + (PAGE_SIZE - sizeof(struct nfs_cache_array)) / + sizeof(struct nfs_cache_array_entry); + struct nfs_readdir_descriptor { struct file *file; struct page *page; @@ -282,6 +286,21 @@ static bool nfs_readdir_array_is_full(struct nfs_cache_array *array) return array->page_full; } +static void nfs_readdir_set_cursor(struct nfs_readdir_descriptor *desc, int index) +{ + desc->pgc.entry_index = index; + desc->pgc.index_cookie = desc->dir_cookie; +} + +static void nfs_readdir_cursor_next(struct nfs_dir_page_cursor *pgc, u64 cookie) +{ + pgc->index_cookie = cookie; + if (++pgc->entry_index == cache_entries_per_page) { + pgc->entry_index = 0; + pgc->page_index++; + } +} + /* * the caller is responsible for freeing qstr.name * when called by nfs_readdir_add_to_array, the strings will be freed in @@ -455,7 +474,7 @@ static int nfs_readdir_search_for_pos(struct nfs_cache_array *array, index = (unsigned int)diff; desc->dir_cookie = array->array[index].cookie; - desc->pgc.entry_index = index; + nfs_readdir_set_cursor(desc, index); return 0; out_eof: desc->eof = true; @@ -524,7 +543,7 @@ static int nfs_readdir_search_for_cookie(struct nfs_cache_array *array, else desc->ctx->pos = new_pos; desc->prev_index = new_pos; - desc->pgc.entry_index = i; + nfs_readdir_set_cursor(desc, i); return 0; } } @@ -551,9 +570,9 @@ static int nfs_readdir_search_array(struct nfs_readdir_descriptor *desc) status = nfs_readdir_search_for_cookie(array, desc); if (status == -EAGAIN) { - desc->pgc.index_cookie = array->last_cookie; + desc->pgc.entry_index = array->size - 1; + nfs_readdir_cursor_next(&desc->pgc, array->last_cookie); desc->current_index += array->size; - desc->pgc.page_index++; } kunmap_atomic(array); return status; @@ -1084,6 +1103,8 @@ static void nfs_do_filldir(struct nfs_readdir_descriptor *desc, desc->eof = !desc->eob; kunmap(desc->page); + desc->pgc.entry_index = i-1; + nfs_readdir_cursor_next(&desc->pgc, desc->dir_cookie); dfprintk(DIRCACHE, "NFS: nfs_do_filldir() filling ended @ cookie %llu\n", (unsigned long long)desc->dir_cookie); } @@ -1118,8 +1139,7 @@ static int uncached_readdir(struct nfs_readdir_descriptor *desc) goto out; desc->pgc.page_index = 0; - desc->pgc.entry_index = 0; - desc->pgc.index_cookie = desc->dir_cookie; + nfs_readdir_set_cursor(desc, 0); desc->duped = 0; desc->page_index_max = 0; From patchwork Wed Feb 23 13:40:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72FD9C43217 for ; Wed, 23 Feb 2022 13:41:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241004AbiBWNlJ (ORCPT ); Wed, 23 Feb 2022 08:41:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240996AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C8C46AC066 for ; Wed, 23 Feb 2022 05:40:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zswve9h+bHX8YWfrXdD3Pg2yvfY2GXiIjZiSYIOdGgE=; b=Flmzlh+sqWjCNOWQoAlcGXDCgLVWJb9ZHwZ7RZYTVanloMWXejeltzDsM7lgX0xL7ITakF +lvJgiW07ysSL7ZZ3WXfV2GQuXShiF0WYfBwWfcA3JtD/hSXznMWTZPiTBH/OethE2cdx9 utQNkWnz6SUEO204eivKMwQh/AeFTgw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-Kn1-SXN2POWDkQB5HA-OBA-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: Kn1-SXN2POWDkQB5HA-OBA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A0A5D1854E27 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60944832B9 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id 05E6D10C3106; Wed, 23 Feb 2022 08:40:36 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 5/8] NFS: readdir per-page cache validation Date: Wed, 23 Feb 2022 08:40:32 -0500 Message-Id: In-Reply-To: <5479c8c5be9cf3f387edac54f170461f8f7b89e2.1645623510.git.bcodding@redhat.com> References: <5479c8c5be9cf3f387edac54f170461f8f7b89e2.1645623510.git.bcodding@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The current implementation of the readdir page cache requires that all pages contain entries ordered such that the cookie references lead to the first entry as represented by cookie 0. The invalidation of the cache truncates either the entire cache or every page beyond a known good page. A process that wants to emit directory entries near the end of a directory must first fill in any entries missing in the cache near the beginning of the directory in order that the entries decoded from READDIR XDR are appropriately page-aligned for any readers thay may come later (and for some error handling). However, if we're careful to check the alignment of directory entries on each page when the page is read, then it should be permissable to allow "disconnected" filling of the pagecache. Rather than requiring pagecache data to always be positionally aligned, we can instead validate that each page is properly aligned to the reading process' directory context. If it doesn't match our alignment, we'll refresh the entries in the page so that it does. This patch implements a check for validity for each page as it is obtained from the pagecache. A page is valid if it was filled within the client's current version of the directory and if the entries are aligned with the current reader's directory context. Invalid pages are re-filled by READDIR operations before being used to emit entries for the current reader. Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 68 ++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 55 insertions(+), 13 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 2b1a0c1cdce4..ba75a9593dae 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -219,7 +219,9 @@ static void nfs_readdir_page_init_array(struct page *page, u64 last_cookie) array->last_cookie = last_cookie; array->cookies_are_ordered = 1; kunmap_atomic(array); - set_page_private(page, 0); + if (page->mapping) + set_page_private(page, nfs_save_change_attribute(page->mapping->host)); + SetPageUptodate(page); } static int @@ -256,6 +258,15 @@ void nfs_readdir_clear_array(struct page *page) kfree(array->array[i].name); nfs_readdir_array_init(array); kunmap_atomic(array); + ClearPageUptodate(page); +} + +static void +nfs_readdir_recycle_page(struct page *page, u64 last_cookie) +{ + nfs_readdir_clear_array(page); + nfs_readdir_invalidatepage(page, 0, 0); + nfs_readdir_page_init_array(page, last_cookie); } static struct page * @@ -372,18 +383,47 @@ int nfs_readdir_add_to_array(struct nfs_entry *entry, struct page *page) return ret; } +static bool +nfs_readdir_page_valid(struct page *page, unsigned int entry_index, u64 index_cookie) +{ + bool ret = false; + struct nfs_cache_array *array; + + if (page_private(page) != nfs_save_change_attribute(page->mapping->host)) + goto out; + + ret = true; + array = kmap_atomic(page); + + if (array->size == 0 && array->last_cookie == index_cookie) + goto out_unmap; + + if (array->size > entry_index && + array->array[entry_index].cookie == index_cookie) + goto out_unmap; + + ret = false; +out_unmap: + kunmap_atomic(array); +out: + return ret; +} + static struct page *nfs_readdir_page_get_locked(struct address_space *mapping, - pgoff_t index, u64 last_cookie) + struct nfs_dir_page_cursor *pgc) { struct page *page; - page = grab_cache_page(mapping, index); - if (page && !PageUptodate(page)) { - nfs_readdir_page_init_array(page, last_cookie); - if (invalidate_inode_pages2_range(mapping, index + 1, -1) < 0) - nfs_zap_mapping(mapping->host, mapping); - SetPageUptodate(page); - } + page = grab_cache_page(mapping, pgc->page_index); + + if (!page) + return page; + + if (!PageUptodate(page)) + nfs_readdir_page_init_array(page, pgc->index_cookie); + + if (!nfs_readdir_page_valid(page, pgc->entry_index, pgc->index_cookie)) + nfs_readdir_recycle_page(page, pgc->index_cookie); return page; } @@ -429,8 +469,12 @@ static struct page *nfs_readdir_page_get_next(struct address_space *mapping, pgoff_t index, u64 cookie) { struct page *page; + struct nfs_dir_page_cursor pgc = { + .page_index = index, + .index_cookie = cookie, + }; - page = nfs_readdir_page_get_locked(mapping, index, cookie); + page = nfs_readdir_page_get_locked(mapping, &pgc); if (page) { if (nfs_readdir_page_last_cookie(page) == cookie) return page; @@ -984,9 +1028,7 @@ nfs_readdir_page_unlock_and_put_cached(struct nfs_readdir_descriptor *desc) static struct page * nfs_readdir_page_get_cached(struct nfs_readdir_descriptor *desc) { - return nfs_readdir_page_get_locked(desc->file->f_mapping, - desc->pgc.page_index, - desc->pgc.index_cookie); + return nfs_readdir_page_get_locked(desc->file->f_mapping, &desc->pgc); } #define NFS_READDIR_PAGE_FILL_MISS_MAX 5 From patchwork Wed Feb 23 13:40:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C17FFC433EF for ; Wed, 23 Feb 2022 13:42:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238752AbiBWNlL (ORCPT ); Wed, 23 Feb 2022 08:41:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241002AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 6E501AC06C for ; Wed, 23 Feb 2022 05:40:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R7pcIYBVZDzX8FW4mHNWhR8zvOrbLMd2XRXa9PZHwAQ=; b=e7KfF7Zf4/6BGrGagZP0XLNeAQqLY0L61Rf1mui6v6Uy8kagtlZc8HIlK/XY9HpD3Nrapa 28anrXXFbwOv3LllLFEJJJs1eJdB8d32+hTSaY3jn1OLZSK0xR+WiY8B/QCqZoViUjnQIw Geu7F5rb+Wfuf6/7XNXizkmDuADZq0Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-391-H4tNlG6AMXiold9NcqtIkQ-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: H4tNlG6AMXiold9NcqtIkQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0628F804312 for ; Wed, 23 Feb 2022 13:40:37 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD660106F97E for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id 0C96010C3108; Wed, 23 Feb 2022 08:40:36 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 6/8] NFS: stash the readdir pagecache cursor on the open directory context Date: Wed, 23 Feb 2022 08:40:33 -0500 Message-Id: <19ef38cda6b0eb6548c65c2bff7a4d4dd1baa122.1645623510.git.bcodding@redhat.com> In-Reply-To: References: <5479c8c5be9cf3f387edac54f170461f8f7b89e2.1645623510.git.bcodding@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Now that we have per-page cache validation, we can allow filling of the pagecache from arbitrary offsets. Store the pagecache cursor on the open directory context so it can be used to validate our entries the next time we enter nfs_readdir() without having to fill the pagecache from the beginning. The open_directory_context's dir_cookie and index_cookie will store the same value between calls to nfs_readdir(), so we can save a little room in the struct by dropping dir_cookie. Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 13 +++++++------ include/linux/nfs_fs.h | 2 +- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index ba75a9593dae..4f4a139f3181 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -395,7 +395,8 @@ nfs_readdir_page_valid(struct page *page, unsigned int entry_index, u64 index_co ret = true; array = kmap_atomic(page); - if (array->size == 0 && array->last_cookie == index_cookie) + if ((array->size == 0 || array->size == entry_index) + && array->last_cookie == index_cookie) goto out_unmap; if (array->size > entry_index && @@ -1102,7 +1103,6 @@ static int readdir_search_pagecache(struct nfs_readdir_descriptor *desc) if (desc->pgc.page_index == 0) { desc->current_index = 0; desc->prev_index = 0; - desc->pgc.index_cookie = 0; } res = find_and_lock_cache_page(desc); } while (res == -EAGAIN); @@ -1279,7 +1279,8 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) desc->page_index_max = -1; spin_lock(&file->f_lock); - desc->dir_cookie = dir_ctx->dir_cookie; + desc->dir_cookie = dir_ctx->pgc.index_cookie; + desc->pgc = dir_ctx->pgc; desc->dup_cookie = dir_ctx->dup_cookie; desc->duped = dir_ctx->duped; page_index = dir_ctx->page_index; @@ -1336,7 +1337,7 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) } while (!desc->eob && !desc->eof); spin_lock(&file->f_lock); - dir_ctx->dir_cookie = desc->dir_cookie; + dir_ctx->pgc = desc->pgc; dir_ctx->dup_cookie = desc->dup_cookie; dir_ctx->duped = desc->duped; dir_ctx->attr_gencount = desc->attr_gencount; @@ -1382,9 +1383,9 @@ static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence) if (offset != filp->f_pos) { filp->f_pos = offset; if (nfs_readdir_use_cookie(filp)) - dir_ctx->dir_cookie = offset; + dir_ctx->pgc.index_cookie = offset; else - dir_ctx->dir_cookie = 0; + dir_ctx->pgc.index_cookie = 0; dir_ctx->page_fill_misses = 0; if (offset == 0) memset(dir_ctx->verf, 0, sizeof(dir_ctx->verf)); diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h index 2f5ded282477..aaeaad4006a4 100644 --- a/include/linux/nfs_fs.h +++ b/include/linux/nfs_fs.h @@ -111,7 +111,7 @@ struct nfs_open_dir_context { atomic_t cache_misses; unsigned long attr_gencount; __be32 verf[NFS_DIR_VERIFIER_SIZE]; - __u64 dir_cookie; + struct nfs_dir_page_cursor pgc; __u64 dup_cookie; pgoff_t page_index; unsigned int page_fill_misses; From patchwork Wed Feb 23 13:40:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F041C433EF for ; Wed, 23 Feb 2022 13:41:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237639AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241000AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0FD7BAC069 for ; Wed, 23 Feb 2022 05:40:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FAtltKz/f+X4fOyD4r1GJP5vrXhW8/0cnZDFjVmWsxY=; b=ixOYNi2VQ8s7KSV1+1IVltaPWdI5HpyghBlQ9BIQBFjYG3UK3Y/TW1fqq3NaKFOzw3RJH5 e+iNoJJ6xyR/hN5ng1giTX9KSxVJEcrV2McoFL4t6uWBaD9+fsohpRFv0U1IuxwcK63BAW K3lgCjUfCeNljLpRq6zUoOuoSaYx5vQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-370-h-K_Ed_lNAi0r5YKjkOvVQ-1; Wed, 23 Feb 2022 08:40:37 -0500 X-MC-Unique: h-K_Ed_lNAi0r5YKjkOvVQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2239C10247A7 for ; Wed, 23 Feb 2022 13:40:37 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id E972E84979 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id 10FD810C3110; Wed, 23 Feb 2022 08:40:36 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 7/8] NFS: Support headless readdir pagecache pages Date: Wed, 23 Feb 2022 08:40:34 -0500 Message-Id: <71835b457fb123f8e4d51ea9fb586e46016562ff.1645623510.git.bcodding@redhat.com> In-Reply-To: <19ef38cda6b0eb6548c65c2bff7a4d4dd1baa122.1645623510.git.bcodding@redhat.com> References: <5479c8c5be9cf3f387edac54f170461f8f7b89e2.1645623510.git.bcodding@redhat.com> <19ef38cda6b0eb6548c65c2bff7a4d4dd1baa122.1645623510.git.bcodding@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org It is now possible that a reader will resume a directory listing after an invalidation and fill the rest of the pages with the offset left over from the last partially-filled page. These pages will then be recycled and refilled by the next reader since their alignment is incorrect. Add an index to the nfs_cache_array that will indicate where the next entry should be filled. This allows partially-filled pages to have the best alignment possible. They are more likely to be useful to readers that follow. This optimization targets the case when there are multiple processes listing the directory simultaneously. Often the processes will collect and block on the same page waiting for a READDIR call to fill the pagecache. If the pagecache is invalidated, a partially-filled page will usually result. This partially-filled page can immediately be used by all processes to emit entries rather than having to discard and refill it for every process. The addition of another integer to struct nfs_cache_array increases its size to 24 bytes. We do not lose the original capacity of 127 entries per page. Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 47 ++++++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 19 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 4f4a139f3181..a570f14633ab 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -147,6 +147,7 @@ struct nfs_cache_array_entry { struct nfs_cache_array { u64 last_cookie; + unsigned int index; unsigned int size; unsigned char page_full : 1, page_is_eof : 1, @@ -210,13 +211,15 @@ static void nfs_readdir_array_init(struct nfs_cache_array *array) memset(array, 0, sizeof(struct nfs_cache_array)); } -static void nfs_readdir_page_init_array(struct page *page, u64 last_cookie) +static void +nfs_readdir_page_init_array(struct page *page, struct nfs_dir_page_cursor *pgc) { struct nfs_cache_array *array; array = kmap_atomic(page); nfs_readdir_array_init(array); - array->last_cookie = last_cookie; + array->last_cookie = pgc->index_cookie; + array->index = pgc->entry_index; array->cookies_are_ordered = 1; kunmap_atomic(array); if (page->mapping) @@ -254,7 +257,7 @@ void nfs_readdir_clear_array(struct page *page) int i; array = kmap_atomic(page); - for (i = 0; i < array->size; i++) + for (i = array->index - array->size; i < array->size; i++) kfree(array->array[i].name); nfs_readdir_array_init(array); kunmap_atomic(array); @@ -262,19 +265,20 @@ void nfs_readdir_clear_array(struct page *page) } static void -nfs_readdir_recycle_page(struct page *page, u64 last_cookie) +nfs_readdir_recycle_page(struct page *page, struct nfs_dir_page_cursor *pgc) { nfs_readdir_clear_array(page); nfs_readdir_invalidatepage(page, 0, 0); - nfs_readdir_page_init_array(page, last_cookie); + nfs_readdir_page_init_array(page, pgc); } static struct page * nfs_readdir_page_array_alloc(u64 last_cookie, gfp_t gfp_flags) { struct page *page = alloc_page(gfp_flags); + struct nfs_dir_page_cursor pgc = { .index_cookie = last_cookie }; if (page) - nfs_readdir_page_init_array(page, last_cookie); + nfs_readdir_page_init_array(page, &pgc); return page; } @@ -339,7 +343,7 @@ static int nfs_readdir_array_can_expand(struct nfs_cache_array *array) if (array->page_full) return -ENOSPC; - cache_entry = &array->array[array->size + 1]; + cache_entry = &array->array[array->index + 1]; if ((char *)cache_entry - (char *)array > PAGE_SIZE) { array->page_full = 1; return -ENOSPC; @@ -366,7 +370,7 @@ int nfs_readdir_add_to_array(struct nfs_entry *entry, struct page *page) goto out; } - cache_entry = &array->array[array->size]; + cache_entry = &array->array[array->index]; cache_entry->cookie = entry->prev_cookie; cache_entry->ino = entry->ino; cache_entry->d_type = entry->d_type; @@ -375,6 +379,7 @@ int nfs_readdir_add_to_array(struct nfs_entry *entry, struct page *page) array->last_cookie = entry->cookie; if (array->last_cookie <= cache_entry->cookie) array->cookies_are_ordered = 0; + array->index++; array->size++; if (entry->eof != 0) nfs_readdir_array_set_eof(array); @@ -395,13 +400,15 @@ nfs_readdir_page_valid(struct page *page, unsigned int entry_index, u64 index_co ret = true; array = kmap_atomic(page); - if ((array->size == 0 || array->size == entry_index) - && array->last_cookie == index_cookie) - goto out_unmap; + if (entry_index >= array->index - array->size) { + if ((array->size == 0 || array->size == entry_index) + && array->last_cookie == index_cookie) + goto out_unmap; - if (array->size > entry_index && - array->array[entry_index].cookie == index_cookie) - goto out_unmap; + if (array->size > entry_index && + array->array[entry_index].cookie == index_cookie) + goto out_unmap; + } ret = false; out_unmap: @@ -421,10 +428,10 @@ static struct page *nfs_readdir_page_get_locked(struct address_space *mapping, return page; if (!PageUptodate(page)) - nfs_readdir_page_init_array(page, pgc->index_cookie); + nfs_readdir_page_init_array(page, pgc); if (!nfs_readdir_page_valid(page, pgc->entry_index, pgc->index_cookie)) - nfs_readdir_recycle_page(page, pgc->index_cookie); + nfs_readdir_recycle_page(page, pgc); return page; } @@ -544,7 +551,7 @@ static bool nfs_readdir_array_cookie_in_range(struct nfs_cache_array *array, /* Optimisation for monotonically increasing cookies */ if (cookie >= array->last_cookie) return false; - if (array->size && cookie < array->array[0].cookie) + if (array->size && cookie < array->array[array->index - array->size].cookie) return false; return true; } @@ -559,7 +566,7 @@ static int nfs_readdir_search_for_cookie(struct nfs_cache_array *array, if (!nfs_readdir_array_cookie_in_range(array, desc->dir_cookie)) goto check_eof; - for (i = 0; i < array->size; i++) { + for (i = array->index - array->size; i < array->index; i++) { if (array->array[i].cookie == desc->dir_cookie) { struct nfs_inode *nfsi = NFS_I(file_inode(desc->file)); @@ -1120,7 +1127,7 @@ static void nfs_do_filldir(struct nfs_readdir_descriptor *desc, unsigned int i = 0; array = kmap(desc->page); - for (i = desc->pgc.entry_index; i < array->size; i++) { + for (i = desc->pgc.entry_index; i < array->index; i++) { struct nfs_cache_array_entry *ent; ent = &array->array[i]; @@ -1387,6 +1394,8 @@ static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence) else dir_ctx->pgc.index_cookie = 0; dir_ctx->page_fill_misses = 0; + dir_ctx->pgc.page_index = 0; + dir_ctx->pgc.entry_index = 0; if (offset == 0) memset(dir_ctx->verf, 0, sizeof(dir_ctx->verf)); dir_ctx->duped = 0; From patchwork Wed Feb 23 13:40:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Coddington X-Patchwork-Id: 12756946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E036C433FE for ; Wed, 23 Feb 2022 13:41:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240996AbiBWNlJ (ORCPT ); Wed, 23 Feb 2022 08:41:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240976AbiBWNlI (ORCPT ); Wed, 23 Feb 2022 08:41:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2E3A0AC06B for ; Wed, 23 Feb 2022 05:40:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1645623639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fjTKGeI8OtqK7SBy1Z/zaXir5SneRCC72cv4gdc2uQI=; b=E6ETOLeCMF8dBIcby89h12Oks5Xhl7Urlq9952Q+Eh0WuklwQbCe6FL+dpFnnhAnGQiCdu lrSRhsj/sJ/ofOCu0LP2xOPnLnVA5h/YoUMitQjirnQ5sHQ4qHjRrGIhqdOt9p3BdEUiGi Sn2McQ7wqs4RlLort180ycXbRu+/rT4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-391-vdH1e-aVNYqmdLOxdHuEBw-1; Wed, 23 Feb 2022 08:40:38 -0500 X-MC-Unique: vdH1e-aVNYqmdLOxdHuEBw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D717FC80 for ; Wed, 23 Feb 2022 13:40:37 +0000 (UTC) Received: from bcodding.csb (ovpn-64-2.rdu2.redhat.com [10.10.64.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9F381073023 for ; Wed, 23 Feb 2022 13:40:36 +0000 (UTC) Received: by bcodding.csb (Postfix, from userid 24008) id 1622A10C3115; Wed, 23 Feb 2022 08:40:36 -0500 (EST) From: Benjamin Coddington To: linux-nfs@vger.kernel.org Subject: [PATCH v3 8/8] NFS: Revalidate the directory pagecache on every nfs_readdir() Date: Wed, 23 Feb 2022 08:40:35 -0500 Message-Id: In-Reply-To: <71835b457fb123f8e4d51ea9fb586e46016562ff.1645623510.git.bcodding@redhat.com> References: <5479c8c5be9cf3f387edac54f170461f8f7b89e2.1645623510.git.bcodding@redhat.com> <19ef38cda6b0eb6548c65c2bff7a4d4dd1baa122.1645623510.git.bcodding@redhat.com> <71835b457fb123f8e4d51ea9fb586e46016562ff.1645623510.git.bcodding@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Since there's no longer a significant performance penalty for dropping the pagecache, and because we want to ensure that the directory's change attribute is accurate before validating pagecache pages, revalidate the directory's mapping on every call to nfs_readdir(). Signed-off-by: Benjamin Coddington --- fs/nfs/dir.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index a570f14633ab..565ff26e1b52 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -1271,11 +1271,9 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx) * to either find the entry with the appropriate number or * revalidate the cookie. */ - if (ctx->pos == 0 || nfs_attribute_cache_expired(inode)) { - res = nfs_revalidate_mapping(inode, file->f_mapping); - if (res < 0) - goto out; - } + res = nfs_revalidate_mapping(inode, file->f_mapping); + if (res < 0) + goto out; res = -ENOMEM; desc = kzalloc(sizeof(*desc), GFP_KERNEL);