From patchwork Tue Mar 24 23:47:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456707 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81C3613A4 for ; Tue, 24 Mar 2020 23:49:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 636292074D for ; Tue, 24 Mar 2020 23:49:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093779; bh=6mPriu0gpOtKj59zFY2FBD+rje0XT523KPDFTm2vWOw=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=MKtv6OJs0+AWUAa4zpaLnumN/ST/S7e+Dd/v+EpCAuCpfa3csWVuSp2zgAoIHN4ow uaRNOxi9IOnY+Jn89ilEde/6WtS3ZWOHJzrvhWPKf1gs5WNzxHNLaxZ0hv0Ib9Jj/P xwER5thQ45y0fOc/vrFaJERG40JJRJyVMsR/CaBs= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727028AbgCXXti (ORCPT ); Tue, 24 Mar 2020 19:49:38 -0400 Received: from mail.kernel.org ([198.145.29.99]:35348 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726956AbgCXXti (ORCPT ); Tue, 24 Mar 2020 19:49:38 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 062E32072E for ; Tue, 24 Mar 2020 23:49:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093778; bh=6mPriu0gpOtKj59zFY2FBD+rje0XT523KPDFTm2vWOw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=lOFZqQN/UbZcC6GLbF45TDNvE31v66asbcpKyT946fyF8dA1INSokCtvQ+S9IJrdx Ni3OCVQkgvMzfWWqJbVpH8Up738Z8FH3opEmZ4XU35b5Mhy9woRukDDBHhzqWxL/uB z4gPNro5C0+0S530PMGL7N7/NBbR2v8nZPz26NBM= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 01/22] pNFS/flexfiles: Simplify allocation of the mirror array Date: Tue, 24 Mar 2020 19:47:07 -0400 Message-Id: <20200324234728.8997-2-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-1-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Just allocate the array at the end of the layout segment structure, instead of allocating it as a separate array of pointers. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 21 +++++---------------- fs/nfs/flexfilelayout/flexfilelayout.h | 2 +- 2 files changed, 6 insertions(+), 17 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index e7d8ae4d0cc5..19728206e9c6 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -248,18 +248,10 @@ static void ff_layout_put_mirror(struct nfs4_ff_layout_mirror *mirror) static void ff_layout_free_mirror_array(struct nfs4_ff_layout_segment *fls) { - int i; + u32 i; - if (fls->mirror_array) { - for (i = 0; i < fls->mirror_array_cnt; i++) { - /* normally mirror_ds is freed in - * .free_deviceid_node but we still do it here - * for .alloc_lseg error path */ - ff_layout_put_mirror(fls->mirror_array[i]); - } - kfree(fls->mirror_array); - fls->mirror_array = NULL; - } + for (i = 0; i < fls->mirror_array_cnt; i++) + ff_layout_put_mirror(fls->mirror_array[i]); } static int ff_layout_check_layout(struct nfs4_layoutget_res *lgr) @@ -400,16 +392,13 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, goto out_err_free; rc = -ENOMEM; - fls = kzalloc(sizeof(*fls), gfp_flags); + fls = kzalloc(struct_size(fls, mirror_array, mirror_array_cnt), + gfp_flags); if (!fls) goto out_err_free; fls->mirror_array_cnt = mirror_array_cnt; fls->stripe_unit = stripe_unit; - fls->mirror_array = kcalloc(fls->mirror_array_cnt, - sizeof(fls->mirror_array[0]), gfp_flags); - if (fls->mirror_array == NULL) - goto out_err_free; for (i = 0; i < fls->mirror_array_cnt; i++) { struct nfs4_ff_layout_mirror *mirror; diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h index 2f369966abf7..354a031c69b1 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.h +++ b/fs/nfs/flexfilelayout/flexfilelayout.h @@ -99,7 +99,7 @@ struct nfs4_ff_layout_segment { u64 stripe_unit; u32 flags; u32 mirror_array_cnt; - struct nfs4_ff_layout_mirror **mirror_array; + struct nfs4_ff_layout_mirror *mirror_array[]; }; struct nfs4_flexfile_layout { From patchwork Tue Mar 24 23:47:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456711 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A28A17D4 for ; Tue, 24 Mar 2020 23:49:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0D0DA2072E for ; Tue, 24 Mar 2020 23:49:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093780; bh=Ihi7Dit/Q8Thk1RIhBwaphJ7XIANwUrtk7jMQv7GX8s=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=v7WA1wTDkroN+kZKdoYEyAxJEmfPgfcuNmIgAncf3TMdmCeM4wp8wP2SpgUoym3Ux GA/cK/VD+BMohvjTE4tn9K+C1D9yNbjj66BJMRMU9te0VP9hQyEOj7aUxEhW/arWI7 Bo3XK6jiguvLYUceEL43/7A8Z3i7H8NYutcT4GWA= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727070AbgCXXtj (ORCPT ); Tue, 24 Mar 2020 19:49:39 -0400 Received: from mail.kernel.org ([198.145.29.99]:35354 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726956AbgCXXtj (ORCPT ); Tue, 24 Mar 2020 19:49:39 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 7217B2073C for ; Tue, 24 Mar 2020 23:49:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093778; bh=Ihi7Dit/Q8Thk1RIhBwaphJ7XIANwUrtk7jMQv7GX8s=; h=From:To:Subject:Date:In-Reply-To:References:From; b=q+/R74IuJ5X5JwoHI0b7oblsDydVWvdn+Zl3ZqEcV81EpjGm7/mcJW+OZizVIgiDs IyOQ91utNCPgRAOwbxhzZ01284m2NfK1xizA+iNASze+ixp0bzsLf3U2ee3SjRSLVU 230/m2G5QPCSgq5oL/uqjmes0Fr3d17KqZAbf7mQ= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 02/22] NFS/pNFS: Refactor pnfs_generic_commit_pagelist() Date: Tue, 24 Mar 2020 19:47:08 -0400 Message-Id: <20200324234728.8997-3-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-2-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Refactor pnfs_generic_commit_pagelist() to simplify the conversion to layout segment based commit lists. Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 164 +++++++++++++++++++--------------------------- fs/nfs/write.c | 13 ++-- 2 files changed, 76 insertions(+), 101 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 8b37e7f8e789..3d0942541618 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -156,103 +156,86 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); -static void pnfs_generic_retry_commit(struct nfs_commit_info *cinfo, int idx) +static struct pnfs_layout_segment * +pnfs_bucket_get_committing(struct list_head *head, + struct pnfs_commit_bucket *bucket, + struct nfs_commit_info *cinfo) { - struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - struct pnfs_commit_bucket *bucket; struct pnfs_layout_segment *freeme; struct list_head *pos; + + list_for_each(pos, &bucket->committing) + cinfo->ds->ncommitting--; + list_splice_init(&bucket->committing, head); + freeme = bucket->clseg; + bucket->clseg = NULL; + return freeme; +} + +static struct nfs_commit_data * +pnfs_bucket_fetch_commitdata(struct pnfs_commit_bucket *bucket, + struct nfs_commit_info *cinfo) +{ + struct nfs_commit_data *data = nfs_commitdata_alloc(false); + + if (!data) + return NULL; + data->lseg = pnfs_bucket_get_committing(&data->pages, bucket, cinfo); + return data; +} + +static void pnfs_generic_retry_commit(struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + struct nfs_commit_info *cinfo, + unsigned int idx) +{ + struct pnfs_commit_bucket *bucket; + struct pnfs_layout_segment *freeme; LIST_HEAD(pages); - int i; - mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); - for (i = idx; i < fl_cinfo->nbuckets; i++) { - bucket = &fl_cinfo->buckets[i]; + for (bucket = buckets; idx < nbuckets; bucket++, idx++) { if (list_empty(&bucket->committing)) continue; - freeme = bucket->clseg; - bucket->clseg = NULL; - list_for_each(pos, &bucket->committing) - cinfo->ds->ncommitting--; - list_splice_init(&bucket->committing, &pages); + mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); + freeme = pnfs_bucket_get_committing(&pages, bucket, cinfo); mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); - nfs_retry_commit(&pages, freeme, cinfo, i); + nfs_retry_commit(&pages, freeme, cinfo, idx); pnfs_put_lseg(freeme); - mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); } - mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); } static unsigned int -pnfs_generic_alloc_ds_commits(struct nfs_commit_info *cinfo, - struct list_head *list) +pnfs_bucket_alloc_ds_commits(struct list_head *list, + struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + struct nfs_commit_info *cinfo) { - struct pnfs_ds_commit_info *fl_cinfo; struct pnfs_commit_bucket *bucket; struct nfs_commit_data *data; - int i; + unsigned int i; unsigned int nreq = 0; - fl_cinfo = cinfo->ds; - bucket = fl_cinfo->buckets; - for (i = 0; i < fl_cinfo->nbuckets; i++, bucket++) { + for (i = 0, bucket = buckets; i < nbuckets; i++, bucket++) { if (list_empty(&bucket->committing)) continue; - data = nfs_commitdata_alloc(false); - if (!data) - break; - data->ds_commit_index = i; - list_add(&data->pages, list); - nreq++; + mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); + if (!list_empty(&bucket->committing)) { + data = pnfs_bucket_fetch_commitdata(bucket, cinfo); + if (!data) + goto out_error; + data->ds_commit_index = i; + list_add_tail(&data->list, list); + atomic_inc(&cinfo->mds->rpcs_out); + nreq++; + } + mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); } - - /* Clean up on error */ - pnfs_generic_retry_commit(cinfo, i); return nreq; -} - -static inline -void pnfs_fetch_commit_bucket_list(struct list_head *pages, - struct nfs_commit_data *data, - struct nfs_commit_info *cinfo) -{ - struct pnfs_commit_bucket *bucket; - struct list_head *pos; - - bucket = &cinfo->ds->buckets[data->ds_commit_index]; - mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); - list_for_each(pos, &bucket->committing) - cinfo->ds->ncommitting--; - list_splice_init(&bucket->committing, pages); - data->lseg = bucket->clseg; - bucket->clseg = NULL; +out_error: mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); - -} - -/* Helper function for pnfs_generic_commit_pagelist to catch an empty - * page list. This can happen when two commits race. - * - * This must be called instead of nfs_init_commit - call one or the other, but - * not both! - */ -static bool -pnfs_generic_commit_cancel_empty_pagelist(struct list_head *pages, - struct nfs_commit_data *data, - struct nfs_commit_info *cinfo) -{ - if (list_empty(pages)) { - if (atomic_dec_and_test(&cinfo->mds->rpcs_out)) - wake_up_var(&cinfo->mds->rpcs_out); - /* don't call nfs_commitdata_release - it tries to put - * the open_context which is not acquired until nfs_init_commit - * which has not been called on @data */ - WARN_ON_ONCE(data->context); - nfs_commit_free(data); - return true; - } - - return false; + /* Clean up on error */ + pnfs_generic_retry_commit(buckets, nbuckets, cinfo, i); + return nreq; } /* This follows nfs_commit_list pretty closely */ @@ -262,6 +245,7 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, int (*initiate_commit)(struct nfs_commit_data *data, int how)) { + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; struct nfs_commit_data *data, *tmp; LIST_HEAD(list); unsigned int nreq = 0; @@ -269,40 +253,26 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, if (!list_empty(mds_pages)) { data = nfs_commitdata_alloc(true); data->ds_commit_index = -1; - list_add(&data->pages, &list); + list_splice_init(mds_pages, &data->pages); + list_add_tail(&data->list, &list); + atomic_inc(&cinfo->mds->rpcs_out); nreq++; } - nreq += pnfs_generic_alloc_ds_commits(cinfo, &list); - + nreq += pnfs_bucket_alloc_ds_commits(&list, fl_cinfo->buckets, + fl_cinfo->nbuckets, cinfo); if (nreq == 0) goto out; - atomic_add(nreq, &cinfo->mds->rpcs_out); - - list_for_each_entry_safe(data, tmp, &list, pages) { - list_del_init(&data->pages); + list_for_each_entry_safe(data, tmp, &list, list) { + list_del(&data->list); if (data->ds_commit_index < 0) { - /* another commit raced with us */ - if (pnfs_generic_commit_cancel_empty_pagelist(mds_pages, - data, cinfo)) - continue; - - nfs_init_commit(data, mds_pages, NULL, cinfo); + nfs_init_commit(data, NULL, NULL, cinfo); nfs_initiate_commit(NFS_CLIENT(inode), data, NFS_PROTO(data->inode), data->mds_ops, how, 0); } else { - LIST_HEAD(pages); - - pnfs_fetch_commit_bucket_list(&pages, data, cinfo); - - /* another commit raced with us */ - if (pnfs_generic_commit_cancel_empty_pagelist(&pages, - data, cinfo)) - continue; - - nfs_init_commit(data, &pages, data->lseg, cinfo); + nfs_init_commit(data, NULL, data->lseg, cinfo); initiate_commit(data, how); } } diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 5544ee6cfda8..1f8108f5a041 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -1746,14 +1746,19 @@ void nfs_init_commit(struct nfs_commit_data *data, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo) { - struct nfs_page *first = nfs_list_entry(head->next); - struct nfs_open_context *ctx = nfs_req_openctx(first); - struct inode *inode = d_inode(ctx->dentry); + struct nfs_page *first; + struct nfs_open_context *ctx; + struct inode *inode; /* Set up the RPC argument and reply structs * NB: take care not to mess about with data->commit et al. */ - list_splice_init(head, &data->pages); + if (head) + list_splice_init(head, &data->pages); + + first = nfs_list_entry(data->pages.next); + ctx = nfs_req_openctx(first); + inode = d_inode(ctx->dentry); data->inode = inode; data->cred = ctx->cred; From patchwork Tue Mar 24 23:47:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456713 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 714C317EF for ; Tue, 24 Mar 2020 23:49:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 547BC2073E for ; Tue, 24 Mar 2020 23:49:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093780; bh=ogmst/gQoqoMT45GiaXzThYRPK322Tm06zT3I5HCDIU=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=JM4j1jmlvW+kk+Q9aP5C+VrT0RZtPdtNQaryy4bq3GG6lzyWO0WrgFzVUi6vUQ4RC SEzyTfmo0jMo2Hb/DNbosYwmNJ1cKERifh8LtRLBfcYKtoIyN5pTOdTvcVVnZuprrI GhgNiyaVpMnm9/7MDEWEecHitXfxWdT+snDE+HLQ= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727100AbgCXXtj (ORCPT ); Tue, 24 Mar 2020 19:49:39 -0400 Received: from mail.kernel.org ([198.145.29.99]:35356 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726212AbgCXXtj (ORCPT ); Tue, 24 Mar 2020 19:49:39 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E54192072E for ; Tue, 24 Mar 2020 23:49:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093779; bh=ogmst/gQoqoMT45GiaXzThYRPK322Tm06zT3I5HCDIU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=qcEXbJDYflBSRi3nTdpLGgO9fx34TKxfu2pXL5TabPlRyV7hHbnnRt+ivcHsNYM3F 7mhjiZNCF+J4do1mdIz7uo6yCi5kmwPd7fhxig/ZSO8lgNpYNnwQYrIFUgw8VFJztN s5j9T44Y5YIOW6T9eTxAwRFog9S06Ldi09/bY8mk= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 03/22] pNFS: Add a helper to allocate the array of buckets Date: Tue, 24 Mar 2020 19:47:09 -0400 Message-Id: <20200324234728.8997-4-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-3-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.h | 3 +++ fs/nfs/pnfs_nfs.c | 31 +++++++++++++++++++++++++++++++ include/linux/nfs_xdr.h | 15 ++++++++++++--- 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 7bfb6970134a..f6b1099aa151 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -366,6 +366,9 @@ bool nfs4_test_deviceid_unavailable(struct nfs4_deviceid_node *node); void nfs4_deviceid_purge_client(const struct nfs_client *); /* pnfs_nfs.c */ +struct pnfs_commit_array *pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags); +void pnfs_free_commit_array(struct pnfs_commit_array *p); + void pnfs_generic_clear_request_commit(struct nfs_page *req, struct nfs_commit_info *cinfo); void pnfs_generic_commit_release(void *calldata); diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 3d0942541618..c8518ce3a4ef 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -87,6 +87,37 @@ pnfs_generic_clear_request_commit(struct nfs_page *req, } EXPORT_SYMBOL_GPL(pnfs_generic_clear_request_commit); +struct pnfs_commit_array * +pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags) +{ + struct pnfs_commit_array *p; + struct pnfs_commit_bucket *b; + + p = kmalloc(struct_size(p, buckets, n), gfp_flags); + if (!p) + return NULL; + p->nbuckets = n; + INIT_LIST_HEAD(&p->cinfo_list); + INIT_LIST_HEAD(&p->lseg_list); + p->lseg = NULL; + for (b = &p->buckets[0]; n != 0; b++, n--) { + INIT_LIST_HEAD(&b->written); + INIT_LIST_HEAD(&b->committing); + b->wlseg = NULL; + b->clseg = NULL; + b->direct_verf.committed = NFS_INVALID_STABLE_HOW; + } + return p; +} +EXPORT_SYMBOL_GPL(pnfs_alloc_commit_array); + +void +pnfs_free_commit_array(struct pnfs_commit_array *p) +{ + kfree_rcu(p, rcu); +} +EXPORT_SYMBOL_GPL(pnfs_free_commit_array); + static int pnfs_generic_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, struct nfs_commit_info *cinfo, diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 94c77ed55ce1..e91c917c9c1c 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1270,10 +1270,19 @@ struct pnfs_commit_bucket { struct nfs_writeverf direct_verf; }; +struct pnfs_commit_array { + struct list_head cinfo_list; + struct list_head lseg_list; + struct pnfs_layout_segment *lseg; + struct rcu_head rcu; + unsigned int nbuckets; + struct pnfs_commit_bucket buckets[]; +}; + struct pnfs_ds_commit_info { - int nwritten; - int ncommitting; - int nbuckets; + unsigned int nwritten; + unsigned int ncommitting; + unsigned int nbuckets; struct pnfs_commit_bucket *buckets; }; From patchwork Tue Mar 24 23:47:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456715 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A89213A4 for ; Tue, 24 Mar 2020 23:49:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1D7322073C for ; Tue, 24 Mar 2020 23:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093781; bh=xHWBf0eqI5OhgRxmiUyN/XIxypyT+fzgs4uxipL4d7Y=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=wjzOpSvL+OvJJ8BN4Pin+Y41iC2Z3E7xCXEq68pKgygobH9KopZnT/t2nwK0ZjXsp fC2QtGqI0TAEm+IuwMkb1UfTIszpYADX8rc+E/PsqDhbZurC2XBG1Np+QF2DuhtnRb jC5JjM8v/bXDpjvF9CWBEP4NrD3uIJulaEGmhPx8= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727101AbgCXXtk (ORCPT ); Tue, 24 Mar 2020 19:49:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:35354 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726956AbgCXXtj (ORCPT ); Tue, 24 Mar 2020 19:49:39 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 625172073E for ; Tue, 24 Mar 2020 23:49:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093779; bh=xHWBf0eqI5OhgRxmiUyN/XIxypyT+fzgs4uxipL4d7Y=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Jbx+ah7W+O601HYTNqH4S5MJ1k8vw3KYCdIgckPC1CxfoCRfteaH4dP+sUsdDPSoo WMvoVNpD8+Rp78HjJY0u/PRNY8YYYe6dK3HmGX8n+dXeHhfqvGX5qfYbyrBVCP26Qw GILG0F/vMSeO2Uyu/o07LUsikK885DqxigPJhYmQ= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 04/22] NFSv4/pnfs: Support a list of commit arrays in struct pnfs_ds_commit_info Date: Tue, 24 Mar 2020 19:47:10 -0400 Message-Id: <20200324234728.8997-5-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-4-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust When we have multiple layout segments with different lists of mirrored data, we need to track the commits on a per layout segment basis. This patch adds a list to support this tracking in struct pnfs_ds_commit_info. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 1 + fs/nfs/filelayout/filelayout.c | 5 ++++- fs/nfs/flexfilelayout/flexfilelayout.c | 1 + include/linux/nfs_xdr.h | 1 + 4 files changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index ade2435551c8..45d3ad1ad4df 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -305,6 +305,7 @@ static inline struct nfs_direct_req *nfs_direct_req_alloc(void) kref_get(&dreq->kref); init_completion(&dreq->completion); INIT_LIST_HEAD(&dreq->mds_cinfo.list); + INIT_LIST_HEAD(&dreq->ds_cinfo.commits); dreq->verf.committed = NFS_INVALID_STABLE_HOW; /* not set yet */ INIT_WORK(&dreq->work, nfs_direct_write_schedule_work); spin_lock_init(&dreq->lock); diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index bd234394a87c..7bd02efbe19a 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1140,7 +1140,10 @@ filelayout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) struct nfs4_filelayout *flo; flo = kzalloc(sizeof(*flo), gfp_flags); - return flo != NULL ? &flo->generic_hdr : NULL; + if (flo == NULL) + return NULL; + INIT_LIST_HEAD(&flo->commit_info.commits); + return &flo->generic_hdr; } static void diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 19728206e9c6..c7cccdd746e4 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -48,6 +48,7 @@ ff_layout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) ffl = kzalloc(sizeof(*ffl), gfp_flags); if (ffl) { + INIT_LIST_HEAD(&ffl->commit_info.commits); INIT_LIST_HEAD(&ffl->error_list); INIT_LIST_HEAD(&ffl->mirrors); ffl->last_report_time = ktime_get(); diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index e91c917c9c1c..9946787eda72 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1280,6 +1280,7 @@ struct pnfs_commit_array { }; struct pnfs_ds_commit_info { + struct list_head commits; unsigned int nwritten; unsigned int ncommitting; unsigned int nbuckets; From patchwork Tue Mar 24 23:47:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456717 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE2AA17D4 for ; Tue, 24 Mar 2020 23:49:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A104A2072E for ; Tue, 24 Mar 2020 23:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093781; bh=asZPu00uS55iLq65hnYce26g4HNvsBAKVZEYJOk+hl8=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=kwK1rmOuzvVaqDENeZ/5/cn2nWaevPZqcpiIBO/sw6lNw0AbUeK6r7ir7Fjlt2YFc AF0IxcQz7lEfYiWByz8NuXGcwR7EKPthLHXlyltNU4r0nuLmYTMogWfYZtTLlSVGVO BSMp+6WlPEyIBISbhOf6YGcAeaDDINPTYGI81Y9s= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727104AbgCXXtl (ORCPT ); Tue, 24 Mar 2020 19:49:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:35382 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726212AbgCXXtk (ORCPT ); Tue, 24 Mar 2020 19:49:40 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D19552073C for ; Tue, 24 Mar 2020 23:49:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093780; bh=asZPu00uS55iLq65hnYce26g4HNvsBAKVZEYJOk+hl8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=ivo0NeoGuYpZiPYNbCxYu5JLfqWEsN8rfx1QgLLA+lLwbG9yUJ8yr/VGA9H3V6xT2 yI1ZZqZNPyqQoIUgcbrfbDrpsbP4gL0kVE7qBsyU/NxRLDcH+TnrdAXxjC+nqk11K9 /3pw0N8rmgiLXuvWuJz1+UjrCLDfEmfvcUNx5RNo= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 05/22] NFSv4/pNFS: Scan the full list of commit arrays when committing Date: Tue, 24 Mar 2020 19:47:11 -0400 Message-Id: <20200324234728.8997-6-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-5-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to pnfs_generic_scan_commit_lists() Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 52 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 40 insertions(+), 12 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index c8518ce3a4ef..81fd85e66fd9 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -118,10 +118,14 @@ pnfs_free_commit_array(struct pnfs_commit_array *p) } EXPORT_SYMBOL_GPL(pnfs_free_commit_array); +/* + * Locks the nfs_page requests for commit and moves them to + * @bucket->committing. + */ static int -pnfs_generic_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, - struct nfs_commit_info *cinfo, - int max) +pnfs_bucket_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, + struct nfs_commit_info *cinfo, + int max) { struct list_head *src = &bucket->written; struct list_head *dst = &bucket->committing; @@ -142,20 +146,44 @@ pnfs_generic_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, return ret; } +static int pnfs_bucket_scan_array(struct nfs_commit_info *cinfo, + struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + int max) +{ + unsigned int i; + int rv = 0, cnt; + + for (i = 0; i < nbuckets && max != 0; i++) { + cnt = pnfs_bucket_scan_ds_commit_list(&buckets[i], cinfo, max); + rv += cnt; + max -= cnt; + } + return rv; +} + /* Move reqs from written to committing lists, returning count * of number moved. */ -int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, - int max) +int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) { - int i, rv = 0, cnt; - - lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); - for (i = 0; i < cinfo->ds->nbuckets && max != 0; i++) { - cnt = pnfs_generic_scan_ds_commit_list(&cinfo->ds->buckets[i], - cinfo, max); - max -= cnt; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + struct pnfs_commit_array *array; + int rv = 0, cnt; + + cnt = pnfs_bucket_scan_array(cinfo, fl_cinfo->buckets, + fl_cinfo->nbuckets, max); + rv += cnt; + max -= cnt; + if (!max) + return rv; + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + cnt = pnfs_bucket_scan_array(cinfo, array->buckets, + array->nbuckets, max); rv += cnt; + max -= cnt; + if (!max) + break; } return rv; } From patchwork Tue Mar 24 23:47:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456719 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4046613A4 for ; Tue, 24 Mar 2020 23:49:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2370B2072E for ; Tue, 24 Mar 2020 23:49:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093782; bh=ksX41en9zXztZkK7yJUP1w30ssz69SdnHLr2M+BntI0=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=jL2MzgwUhtqNSuDbNambRKfvwQS9ubCN07E/bBdK9Qcr9bngJQHFzKO2CbjwYVYqw jKKuOcLpJEAM2cbhKpI3cy7YF3v2VAQomdquZWYMiLEhFPCsMwItyuF8vBMwGJ4IdT bu2/nmyGp1lIp5C8MtJNjP5tQCbo8JKzhc9zS2WI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727118AbgCXXtl (ORCPT ); Tue, 24 Mar 2020 19:49:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:35392 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726956AbgCXXtk (ORCPT ); Tue, 24 Mar 2020 19:49:40 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 520FD20719 for ; Tue, 24 Mar 2020 23:49:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093780; bh=ksX41en9zXztZkK7yJUP1w30ssz69SdnHLr2M+BntI0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=nHR5kdL8jDBM7VYzDqNshOvXF4f15UZQ0JEv2mf9SPbss2Vu+2411rYgt210QA+Qy 2mXzklU2ioEnvzUv2IS/3nuCcEgQLepGffoXVbItpfBYQwsjGgJ2DlV1oVcRwSySja H1yujwZf/4hPwDBY6w0lCpz69ts22gMp33hp85G0= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 06/22] pNFS: Support per-layout segment commits in pnfs_generic_recover_commit_reqs() Date: Tue, 24 Mar 2020 19:47:12 -0400 Message-Id: <20200324234728.8997-7-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-6-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to pnfs_generic_recover_commit_reqs(). Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 41 +++++++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 81fd85e66fd9..f16bd6d0e830 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -189,22 +189,23 @@ int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) } EXPORT_SYMBOL_GPL(pnfs_generic_scan_commit_lists); -/* Pull everything off the committing lists and dump into @dst. */ -void pnfs_generic_recover_commit_reqs(struct list_head *dst, - struct nfs_commit_info *cinfo) +static unsigned int +pnfs_bucket_recover_commit_reqs(struct list_head *dst, + struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + struct nfs_commit_info *cinfo) { struct pnfs_commit_bucket *b; struct pnfs_layout_segment *freeme; - int nwritten; - int i; + unsigned int nwritten, ret = 0; + unsigned int i; - lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); restart: - for (i = 0, b = cinfo->ds->buckets; i < cinfo->ds->nbuckets; i++, b++) { + for (i = 0, b = buckets; i < nbuckets; i++, b++) { nwritten = nfs_scan_commit_list(&b->written, dst, cinfo, 0); if (!nwritten) continue; - cinfo->ds->nwritten -= nwritten; + ret += nwritten; if (list_empty(&b->written)) { freeme = b->wlseg; b->wlseg = NULL; @@ -212,6 +213,30 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, goto restart; } } + return ret; +} + +/* Pull everything off the committing lists and dump into @dst. */ +void pnfs_generic_recover_commit_reqs(struct list_head *dst, + struct nfs_commit_info *cinfo) +{ + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + struct pnfs_commit_array *array; + unsigned int nwritten; + + lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); + nwritten = pnfs_bucket_recover_commit_reqs(dst, + fl_cinfo->buckets, + fl_cinfo->nbuckets, + cinfo); + fl_cinfo->nwritten -= nwritten; + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + nwritten = pnfs_bucket_recover_commit_reqs(dst, + array->buckets, + array->nbuckets, + cinfo); + fl_cinfo->nwritten -= nwritten; + } } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); From patchwork Tue Mar 24 23:47:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456721 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48A6113A4 for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1FF7A2073E for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093783; bh=7EnajCoY61wyaI04/xGUaouttgnSwwfGTe62JcRBxjE=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=CT/Oxrtxnu/faXIkGApFmVWs1pQPr/sqCQa9ZVBWETWs+f71yxyrDQJYP9FMz/Ba8 9e5HafI1RoFrrHa3bUx/IjaCqHAdHgu1OFgA3jlNWfhObrxPqDvcZ2eLjHoXpboPAB yvZj6acW8v0ZH/1hghkXd3x66Rma2JlJ5KIf1W+4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727126AbgCXXtm (ORCPT ); Tue, 24 Mar 2020 19:49:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:35394 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726212AbgCXXtl (ORCPT ); Tue, 24 Mar 2020 19:49:41 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C2D8F2072E for ; Tue, 24 Mar 2020 23:49:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093781; bh=7EnajCoY61wyaI04/xGUaouttgnSwwfGTe62JcRBxjE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=dQani27y7k6IiSGhhX0m61Hq93Oy1jTHlLTgyqq+7b31On8zh8ycHEXA8XNUU+9Ep jDXnW8Zt183CRRtbkZVVhErVN1jIcXrUXDpGS5yeZB0eSGp9O4mT/Gdxz2M95BQlSq pwqGt/KdQ5qEm7aiWX2jDLahzziU0gqr/2q5WDxM= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 07/22] pNFS: Support per-layout segment commits in pnfs_generic_commit_pagelist() Date: Tue, 24 Mar 2020 19:47:13 -0400 Message-Id: <20200324234728.8997-8-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-7-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to pnfs_generic_commit_pagelist(). Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index f16bd6d0e830..f895a28b1e26 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -322,6 +322,20 @@ pnfs_bucket_alloc_ds_commits(struct list_head *list, return nreq; } +static unsigned int +pnfs_alloc_ds_commits_list(struct list_head *list, + struct pnfs_ds_commit_info *fl_cinfo, + struct nfs_commit_info *cinfo) +{ + struct pnfs_commit_array *array; + unsigned int ret = 0; + + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) + ret += pnfs_bucket_alloc_ds_commits(list, array->buckets, + array->nbuckets, cinfo); + return ret; +} + /* This follows nfs_commit_list pretty closely */ int pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, @@ -345,6 +359,8 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, nreq += pnfs_bucket_alloc_ds_commits(&list, fl_cinfo->buckets, fl_cinfo->nbuckets, cinfo); + + nreq += pnfs_alloc_ds_commits_list(&list, fl_cinfo, cinfo); if (nreq == 0) goto out; From patchwork Tue Mar 24 23:47:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456723 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B2A617D4 for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6EC852073C for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093783; bh=p37VSKP4Th+3RkSkshW4MOTSZ2uTQKxP63nhdF5s+3Q=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=WqZGqywaBmaFWuTu5iegguQnSw1DA76+p8rPhmZrM9Y6Hu/hNLUVulzRCrGxazWEt FennXaB8cX6t4d/Rk9W4IAhMBIcCUv21qe1EJ9g9TJobQdPmxr/DeM4cHYtxAGD/pA W9MO2aMMwxb5qEkxc7w9bJS90nKMGKE8Y507gzoM= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726212AbgCXXtm (ORCPT ); Tue, 24 Mar 2020 19:49:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:35404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726956AbgCXXtm (ORCPT ); Tue, 24 Mar 2020 19:49:42 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3FBA02073E for ; Tue, 24 Mar 2020 23:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093781; bh=p37VSKP4Th+3RkSkshW4MOTSZ2uTQKxP63nhdF5s+3Q=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Mzx3ImHOYcykLi0DNgACkNqNFs/6qvL9ceQIdVMhqQj+A9qcu5F3Nd9xyIkRn8Iqx OxMLsp0dQvp8C9eKaFiSywbdfGTLxpebT6A2NJ00cqyvVo0JvZCggtDcrHg8/mhHL1 8qKzVVvmpZ8LWPBxxJEBc8WWwdka9juZSKQlyxeg= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 08/22] NFS/pNFS: Allow O_DIRECT to release the DS commitinfo Date: Tue, 24 Mar 2020 19:47:14 -0400 Message-Id: <20200324234728.8997-9-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-8-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add a pNFS callback to allow the O_DIRECT code to release the DS commitinfo when freeing the dreq. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 1 + fs/nfs/pnfs.h | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 45d3ad1ad4df..226d92761ff1 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -317,6 +317,7 @@ static void nfs_direct_req_free(struct kref *kref) { struct nfs_direct_req *dreq = container_of(kref, struct nfs_direct_req, kref); + pnfs_release_ds_info(&dreq->ds_cinfo, dreq->inode); nfs_free_pnfs_ds_cinfo(&dreq->ds_cinfo); if (dreq->l_ctx != NULL) nfs_put_lock_context(dreq->l_ctx); diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index f6b1099aa151..4ff974631206 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -149,6 +149,8 @@ struct pnfs_layoutdriver_type { const struct nfs_pageio_ops *pg_write_ops; struct pnfs_ds_commit_info *(*get_ds_info) (struct inode *inode); + void (*release_ds_info)(struct pnfs_ds_commit_info *, + struct inode *inode); void (*mark_request_commit) (struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, @@ -462,6 +464,15 @@ pnfs_get_ds_info(struct inode *inode) return ld->get_ds_info(inode); } +static inline void +pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ + struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + + if (ld != NULL && ld->release_ds_info != NULL) + ld->release_ds_info(fl_cinfo, inode); +} + static inline void pnfs_generic_mark_devid_invalid(struct nfs4_deviceid_node *node) { @@ -759,6 +770,11 @@ pnfs_get_ds_info(struct inode *inode) return NULL; } +static inline void +pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ +} + static inline bool pnfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, u32 ds_commit_idx) From patchwork Tue Mar 24 23:47:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456725 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3B19174A for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 96C982076A for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093783; bh=v9VwoEXJBfQvyNH7B9/gXNGHypJmMHzVnkvCqQDbhVI=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=uWNzTZ3dG1O3hRrs0uR5dG5w3AIeF9GtLO+VUtEx/S7o2xBeYo7yjTZ6aO6SKp8QW CwwWGidYuRXUCgVIyF2p0I+yIm9vP7YtLUqNa0Qg6qZ+G8uIss+nNpMjuLk+4hUlFe Q4S1lgZ2/paI48XF7BTsU2rb+mnXmfmbq6jjE66o= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726956AbgCXXtm (ORCPT ); Tue, 24 Mar 2020 19:49:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:35394 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727121AbgCXXtm (ORCPT ); Tue, 24 Mar 2020 19:49:42 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AE82F2073C for ; Tue, 24 Mar 2020 23:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093781; bh=v9VwoEXJBfQvyNH7B9/gXNGHypJmMHzVnkvCqQDbhVI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=UYDtUy4pkgK9lPhOhETb/Hz/3DlEJx7fsJKKmgemthHztITZ5Ii1B/W77Nq35N09p nw2G2yIo6onVGo32ATBIX8E7J477ZBPGhlp0gFrwzNL50OYXetcgqjgQQrZfYZTrqR aV21xp9YL6HUCXrUtAbw/bxqyeyscCx3II2MbdOM= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 09/22] NFS: commit errors should be fatal Date: Tue, 24 Mar 2020 19:47:15 -0400 Message-Id: <20200324234728.8997-10-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-9-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Fix the O_DIRECT code to avoid retries if the COMMIT fails with a fatal error. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 226d92761ff1..b58381b3edbc 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -94,6 +94,7 @@ struct nfs_direct_req { #define NFS_ODIRECT_RESCHED_WRITES (2) /* write verification failed */ /* for read */ #define NFS_ODIRECT_SHOULD_DIRTY (3) /* dirty user-space page after read */ +#define NFS_ODIRECT_DONE INT_MAX /* write verification failed */ struct nfs_writeverf verf; /* unstable write verifier */ }; @@ -678,8 +679,17 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) struct nfs_page *req; int status = data->task.tk_status; + if (status < 0) { + /* Errors in commit are fatal */ + dreq->error = status; + dreq->max_count = 0; + dreq->count = 0; + dreq->flags = NFS_ODIRECT_DONE; + } else if (dreq->flags == NFS_ODIRECT_DONE) + status = dreq->error; + nfs_init_cinfo_from_dreq(&cinfo, dreq); - if (status < 0 || nfs_direct_cmp_commit_data_verf(dreq, data)) + if (nfs_direct_cmp_commit_data_verf(dreq, data)) dreq->flags = NFS_ODIRECT_RESCHED_WRITES; while (!list_empty(&data->pages)) { @@ -708,7 +718,8 @@ static void nfs_direct_resched_write(struct nfs_commit_info *cinfo, struct nfs_direct_req *dreq = cinfo->dreq; spin_lock(&dreq->lock); - dreq->flags = NFS_ODIRECT_RESCHED_WRITES; + if (dreq->flags != NFS_ODIRECT_DONE) + dreq->flags = NFS_ODIRECT_RESCHED_WRITES; spin_unlock(&dreq->lock); nfs_mark_request_commit(req, NULL, cinfo, 0); } @@ -731,6 +742,22 @@ static void nfs_direct_commit_schedule(struct nfs_direct_req *dreq) nfs_direct_write_reschedule(dreq); } +static void nfs_direct_write_clear_reqs(struct nfs_direct_req *dreq) +{ + struct nfs_commit_info cinfo; + struct nfs_page *req; + LIST_HEAD(reqs); + + nfs_init_cinfo_from_dreq(&cinfo, dreq); + nfs_direct_write_scan_commit_list(dreq->inode, &reqs, &cinfo); + + while (!list_empty(&reqs)) { + req = nfs_list_entry(reqs.next); + nfs_list_remove_request(req); + nfs_unlock_and_release_request(req); + } +} + static void nfs_direct_write_schedule_work(struct work_struct *work) { struct nfs_direct_req *dreq = container_of(work, struct nfs_direct_req, work); @@ -745,6 +772,7 @@ static void nfs_direct_write_schedule_work(struct work_struct *work) nfs_direct_write_reschedule(dreq); break; default: + nfs_direct_write_clear_reqs(dreq); nfs_zap_mapping(dreq->inode, dreq->inode->i_mapping); nfs_direct_complete(dreq); } From patchwork Tue Mar 24 23:47:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456729 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 565F813A4 for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3080B2073E for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093785; bh=54+SJxB4rrF3g6/CzkafuNmAUSnVvMIQXgiS3bfQ0Xs=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=aUD26MaFsy+Vnz3aIYPm5eRY5AJeXHP1JA7U7fRdz6IRK5HG+gQ20li1zESRm9/D4 mhIG71nYSPA2OpAmf56BNg0BZWdQ4RguKyaJauaOJzIjPzZNcX9Y2qqzYAUMPls2oJ UbKhz8n+yneBH7/mGpbT8SROGKEUuQ85OocU3/Ag= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727149AbgCXXto (ORCPT ); Tue, 24 Mar 2020 19:49:44 -0400 Received: from mail.kernel.org ([198.145.29.99]:35404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727134AbgCXXtn (ORCPT ); Tue, 24 Mar 2020 19:49:43 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 25FA52074D for ; Tue, 24 Mar 2020 23:49:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093782; bh=54+SJxB4rrF3g6/CzkafuNmAUSnVvMIQXgiS3bfQ0Xs=; h=From:To:Subject:Date:In-Reply-To:References:From; b=GIyF9pivyN9sd4psLtkktK1aaiaClvjC678IC+8gEyD+Ncr2jsrLzjqV9JIsNIFpB 6+My0IKwtmV5EeZOO7gzNn36RpolzuEWcVXXdf7yxfxr/b0wftu4ca/IPFnlSfZ4e/ PmDfAfUYtWcsH2rJeNqBQqz7kRMvfcssMKympvHI= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 10/22] NFS: Fix O_DIRECT commit verifier handling Date: Tue, 24 Mar 2020 19:47:16 -0400 Message-Id: <20200324234728.8997-11-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-10-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Instead of trying to save the commit verifiers and checking them against previous writes, adopt the same strategy as for buffered writes, of just checking the verifiers at commit time. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 135 +++++----------------------------------------- fs/nfs/internal.h | 8 +++ fs/nfs/write.c | 3 +- 3 files changed, 22 insertions(+), 124 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index b58381b3edbc..0f970ef2f026 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -95,7 +95,6 @@ struct nfs_direct_req { /* for read */ #define NFS_ODIRECT_SHOULD_DIRTY (3) /* dirty user-space page after read */ #define NFS_ODIRECT_DONE INT_MAX /* write verification failed */ - struct nfs_writeverf verf; /* unstable write verifier */ }; static const struct nfs_pgio_completion_ops nfs_direct_write_completion_ops; @@ -152,106 +151,6 @@ nfs_direct_count_bytes(struct nfs_direct_req *dreq, dreq->count = dreq_len; } -/* - * nfs_direct_select_verf - select the right verifier - * @dreq - direct request possibly spanning multiple servers - * @ds_clp - nfs_client of data server or NULL if MDS / non-pnfs - * @commit_idx - commit bucket index for the DS - * - * returns the correct verifier to use given the role of the server - */ -static struct nfs_writeverf * -nfs_direct_select_verf(struct nfs_direct_req *dreq, - struct nfs_client *ds_clp, - int commit_idx) -{ - struct nfs_writeverf *verfp = &dreq->verf; - -#ifdef CONFIG_NFS_V4_1 - /* - * pNFS is in use, use the DS verf except commit_through_mds is set - * for layout segment where nbuckets is zero. - */ - if (ds_clp && dreq->ds_cinfo.nbuckets > 0) { - if (commit_idx >= 0 && commit_idx < dreq->ds_cinfo.nbuckets) - verfp = &dreq->ds_cinfo.buckets[commit_idx].direct_verf; - else - WARN_ON_ONCE(1); - } -#endif - return verfp; -} - - -/* - * nfs_direct_set_hdr_verf - set the write/commit verifier - * @dreq - direct request possibly spanning multiple servers - * @hdr - pageio header to validate against previously seen verfs - * - * Set the server's (MDS or DS) "seen" verifier - */ -static void nfs_direct_set_hdr_verf(struct nfs_direct_req *dreq, - struct nfs_pgio_header *hdr) -{ - struct nfs_writeverf *verfp; - - verfp = nfs_direct_select_verf(dreq, hdr->ds_clp, hdr->ds_commit_idx); - WARN_ON_ONCE(verfp->committed >= 0); - memcpy(verfp, &hdr->verf, sizeof(struct nfs_writeverf)); - WARN_ON_ONCE(verfp->committed < 0); -} - -static int nfs_direct_cmp_verf(const struct nfs_writeverf *v1, - const struct nfs_writeverf *v2) -{ - return nfs_write_verifier_cmp(&v1->verifier, &v2->verifier); -} - -/* - * nfs_direct_cmp_hdr_verf - compare verifier for pgio header - * @dreq - direct request possibly spanning multiple servers - * @hdr - pageio header to validate against previously seen verf - * - * set the server's "seen" verf if not initialized. - * returns result of comparison between @hdr->verf and the "seen" - * verf of the server used by @hdr (DS or MDS) - */ -static int nfs_direct_set_or_cmp_hdr_verf(struct nfs_direct_req *dreq, - struct nfs_pgio_header *hdr) -{ - struct nfs_writeverf *verfp; - - verfp = nfs_direct_select_verf(dreq, hdr->ds_clp, hdr->ds_commit_idx); - if (verfp->committed < 0) { - nfs_direct_set_hdr_verf(dreq, hdr); - return 0; - } - return nfs_direct_cmp_verf(verfp, &hdr->verf); -} - -/* - * nfs_direct_cmp_commit_data_verf - compare verifier for commit data - * @dreq - direct request possibly spanning multiple servers - * @data - commit data to validate against previously seen verf - * - * returns result of comparison between @data->verf and the verf of - * the server used by @data (DS or MDS) - */ -static int nfs_direct_cmp_commit_data_verf(struct nfs_direct_req *dreq, - struct nfs_commit_data *data) -{ - struct nfs_writeverf *verfp; - - verfp = nfs_direct_select_verf(dreq, data->ds_clp, - data->ds_commit_index); - - /* verifier not set so always fail */ - if (verfp->committed < 0 || data->res.verf->committed <= NFS_UNSTABLE) - return 1; - - return nfs_direct_cmp_verf(verfp, data->res.verf); -} - /** * nfs_direct_IO - NFS address space operation for direct I/O * @iocb: target I/O control block @@ -307,7 +206,6 @@ static inline struct nfs_direct_req *nfs_direct_req_alloc(void) init_completion(&dreq->completion); INIT_LIST_HEAD(&dreq->mds_cinfo.list); INIT_LIST_HEAD(&dreq->ds_cinfo.commits); - dreq->verf.committed = NFS_INVALID_STABLE_HOW; /* not set yet */ INIT_WORK(&dreq->work, nfs_direct_write_schedule_work); spin_lock_init(&dreq->lock); @@ -637,7 +535,6 @@ static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq) dreq->max_count = 0; list_for_each_entry(req, &reqs, wb_list) dreq->max_count += req->wb_bytes; - dreq->verf.committed = NFS_INVALID_STABLE_HOW; nfs_clear_pnfs_ds_commit_verifiers(&dreq->ds_cinfo); get_dreq(dreq); @@ -674,6 +571,7 @@ static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq) static void nfs_direct_commit_complete(struct nfs_commit_data *data) { + const struct nfs_writeverf *verf = data->res.verf; struct nfs_direct_req *dreq = data->dreq; struct nfs_commit_info cinfo; struct nfs_page *req; @@ -689,21 +587,19 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) status = dreq->error; nfs_init_cinfo_from_dreq(&cinfo, dreq); - if (nfs_direct_cmp_commit_data_verf(dreq, data)) - dreq->flags = NFS_ODIRECT_RESCHED_WRITES; while (!list_empty(&data->pages)) { req = nfs_list_entry(data->pages.next); nfs_list_remove_request(req); - if (dreq->flags == NFS_ODIRECT_RESCHED_WRITES) { + if (status >= 0 && !nfs_write_match_verf(verf, req)) { + dreq->flags = NFS_ODIRECT_RESCHED_WRITES; /* * Despite the reboot, the write was successful, * so reset wb_nio. */ req->wb_nio = 0; - /* Note the rewrite will go through mds */ nfs_mark_request_commit(req, NULL, &cinfo, 0); - } else + } else /* Error or match */ nfs_release_request(req); nfs_unlock_and_release_request(req); } @@ -799,20 +695,15 @@ static void nfs_direct_write_completion(struct nfs_pgio_header *hdr) } nfs_direct_count_bytes(dreq, hdr); - if (hdr->good_bytes != 0) { - if (nfs_write_need_commit(hdr)) { - if (dreq->flags == NFS_ODIRECT_RESCHED_WRITES) - request_commit = true; - else if (dreq->flags == 0) { - nfs_direct_set_hdr_verf(dreq, hdr); - request_commit = true; - dreq->flags = NFS_ODIRECT_DO_COMMIT; - } else if (dreq->flags == NFS_ODIRECT_DO_COMMIT) { - request_commit = true; - if (nfs_direct_set_or_cmp_hdr_verf(dreq, hdr)) - dreq->flags = - NFS_ODIRECT_RESCHED_WRITES; - } + if (hdr->good_bytes != 0 && nfs_write_need_commit(hdr)) { + switch (dreq->flags) { + case 0: + dreq->flags = NFS_ODIRECT_DO_COMMIT; + request_commit = true; + break; + case NFS_ODIRECT_RESCHED_WRITES: + case NFS_ODIRECT_DO_COMMIT: + request_commit = true; } } spin_unlock(&dreq->lock); diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 3b6fa9edc9b5..6542411c020f 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -544,6 +544,14 @@ nfs_write_verifier_cmp(const struct nfs_write_verifier *v1, return memcmp(v1->data, v2->data, sizeof(v1->data)); } +static inline bool +nfs_write_match_verf(const struct nfs_writeverf *verf, + struct nfs_page *req) +{ + return verf->committed > NFS_UNSTABLE && + !nfs_write_verifier_cmp(&req->wb_verf, &verf->verifier); +} + /* unlink.c */ extern struct rpc_task * nfs_async_rename(struct inode *old_dir, struct inode *new_dir, diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 1f8108f5a041..03b7f64f7c4f 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -1874,8 +1874,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data) /* Okay, COMMIT succeeded, apparently. Check the verifier * returned by the server against all stored verfs. */ - if (verf->committed > NFS_UNSTABLE && - !nfs_write_verifier_cmp(&req->wb_verf, &verf->verifier)) { + if (nfs_write_match_verf(verf, req)) { /* We have a match */ if (req->wb_page) nfs_inode_remove_request(req); From patchwork Tue Mar 24 23:47:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456727 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC53D174A for ; Tue, 24 Mar 2020 23:49:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 92F2E2073E for ; Tue, 24 Mar 2020 23:49:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093784; bh=9TfW3eJyhlN+A0d2XUhAtxTSyb8hkL/qjLtKnnStKMQ=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=Z/ysZdxu/5zNCZ4kChQHMqUYjWk/W2M0tg8sGdfdVinkZvgt8ygwScACfViI1hG7B 2L5Iiq3O6ku+TQ3DlU7Rg50qu+c9X59rxeRVBS4mnlFE2+Zac3No/FlT4OI2CJxyEv 71hHFqTMufT0nHPhwRMDBNhUrxXapL+CRynC/1n4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727144AbgCXXtn (ORCPT ); Tue, 24 Mar 2020 19:49:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:35394 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727135AbgCXXtn (ORCPT ); Tue, 24 Mar 2020 19:49:43 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9A9C920719 for ; Tue, 24 Mar 2020 23:49:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093782; bh=9TfW3eJyhlN+A0d2XUhAtxTSyb8hkL/qjLtKnnStKMQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=q4Rza0JzomfBzJ3eoxHDfR07M9Y25chOHx4A5kjYQ+FjT59INNakePyljNkf8dZpW q65wTD9VhbdiR1DrGv/NEsrMqgrMJYBWnnYSx/4TAPMeRqFzLzPMZVXsxvnAQeq+Y7 cGhIEfon7I7b7UJCpa4UWtjGVDKHDWgwntMKlkwc= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 11/22] NFS/pNFS: Support commit arrays in nfs_clear_pnfs_ds_commit_verifiers() Date: Tue, 24 Mar 2020 19:47:17 -0400 Message-Id: <20200324234728.8997-12-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-11-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to nfs_clear_pnfs_ds_commit_verifiers(). Signed-off-by: Trond Myklebust --- fs/nfs/internal.h | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 6542411c020f..4a1adad3740f 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -517,13 +517,26 @@ int nfs_filemap_write_and_wait_range(struct address_space *mapping, loff_t lstart, loff_t lend); #ifdef CONFIG_NFS_V4_1 +static inline void +pnfs_bucket_clear_pnfs_ds_commit_verifiers(struct pnfs_commit_bucket *buckets, + unsigned int nbuckets) +{ + unsigned int i; + + for (i = 0; i < nbuckets; i++) + buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW; +} static inline void nfs_clear_pnfs_ds_commit_verifiers(struct pnfs_ds_commit_info *cinfo) { - int i; + struct pnfs_commit_array *array; + + pnfs_bucket_clear_pnfs_ds_commit_verifiers(cinfo->buckets, + cinfo->nbuckets); - for (i = 0; i < cinfo->nbuckets; i++) - cinfo->buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW; + list_for_each_entry(array, &cinfo->commits, cinfo_list) + pnfs_bucket_clear_pnfs_ds_commit_verifiers(array->buckets, + array->nbuckets); } #else static inline From patchwork Tue Mar 24 23:47:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456735 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A9D5174A for ; Tue, 24 Mar 2020 23:49:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 553DB2074D for ; Tue, 24 Mar 2020 23:49:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093786; bh=B1Ui4potbHONcfYQIKEAnWl8MumgXWyp0og6Om1tDXc=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=egPHDefS7CCrRZF+q7vMv8olnA1RC8xlo3KI5jmOicpGWoRYZqAAvV03agPcoVol+ UTwwSnS+xf7z1ImlZ4gB0uE8/7VOwuDgXWIMVhPtLC600SWg+e5/XwOoC4fOX5RGy7 Z7SZztWic2t9vpFCSj97bXnhw20SflTps7KdGorM= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727135AbgCXXtp (ORCPT ); Tue, 24 Mar 2020 19:49:45 -0400 Received: from mail.kernel.org ([198.145.29.99]:35418 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727121AbgCXXto (ORCPT ); Tue, 24 Mar 2020 19:49:44 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0EEC02072E for ; Tue, 24 Mar 2020 23:49:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093783; bh=B1Ui4potbHONcfYQIKEAnWl8MumgXWyp0og6Om1tDXc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Qbu1Zn32z0/23AJ1E7XVC0SA1INTVLTm+NgPE762hYzlMtyHXP0jy3dKfTfHBys1E Ma3P/TO3/dHkfCEQHDXN878RuO0o/eWrkxfgUec/yRNLcTIJHe8hnD5nCSsTJJpcIr bx0dEc3t0WOon0xYhkndLf3pBoDsTb11SkIvBCPY= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 12/22] pNFS: Add infrastructure for cleaning up per-layout commit structures Date: Tue, 24 Mar 2020 19:47:18 -0400 Message-Id: <20200324234728.8997-13-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-12-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Ensure that both the file and flexfiles layout types clean up when freeing the layout segments. Signed-off-by: Trond Myklebust --- fs/nfs/filelayout/filelayout.c | 16 +++++ fs/nfs/flexfilelayout/flexfilelayout.c | 11 ++++ fs/nfs/internal.h | 4 +- fs/nfs/pnfs.c | 1 + fs/nfs/pnfs.h | 4 ++ fs/nfs/pnfs_nfs.c | 88 +++++++++++++++++++++++++- include/linux/nfs_xdr.h | 1 + 7 files changed, 121 insertions(+), 4 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 7bd02efbe19a..1cb043838b14 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -750,11 +750,16 @@ filelayout_free_lseg(struct pnfs_layout_segment *lseg) /* This assumes a single RW lseg */ if (lseg->pls_range.iomode == IOMODE_RW) { struct nfs4_filelayout *flo; + struct inode *inode; flo = FILELAYOUT_FROM_HDR(lseg->pls_layout); + inode = flo->generic_hdr.plh_inode; + spin_lock(&inode->i_lock); flo->commit_info.nbuckets = 0; kfree(flo->commit_info.buckets); flo->commit_info.buckets = NULL; + pnfs_generic_ds_cinfo_release_lseg(&flo->commit_info, lseg); + spin_unlock(&inode->i_lock); } _filelayout_free_lseg(fl); } @@ -1163,6 +1168,16 @@ filelayout_get_ds_info(struct inode *inode) return &FILELAYOUT_FROM_HDR(layout)->commit_info; } +static void +filelayout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct inode *inode) +{ + spin_lock(&inode->i_lock); + pnfs_generic_ds_cinfo_destroy(fl_cinfo); + spin_unlock(&inode->i_lock); +} + + static struct pnfs_layoutdriver_type filelayout_type = { .id = LAYOUT_NFSV4_1_FILES, .name = "LAYOUT_NFSV4_1_FILES", @@ -1176,6 +1191,7 @@ static struct pnfs_layoutdriver_type filelayout_type = { .pg_read_ops = &filelayout_pg_read_ops, .pg_write_ops = &filelayout_pg_write_ops, .get_ds_info = &filelayout_get_ds_info, + .release_ds_info = filelayout_release_ds_info, .mark_request_commit = filelayout_mark_request_commit, .clear_request_commit = pnfs_generic_clear_request_commit, .scan_commit_lists = pnfs_generic_scan_commit_lists, diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index c7cccdd746e4..3ddd8080f76d 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -580,6 +580,7 @@ ff_layout_free_lseg(struct pnfs_layout_segment *lseg) kfree(ffl->commit_info.buckets); ffl->commit_info.buckets = NULL; } + pnfs_generic_ds_cinfo_release_lseg(&ffl->commit_info, lseg); spin_unlock(&inode->i_lock); } _ff_layout_free_lseg(fls); @@ -2003,6 +2004,15 @@ ff_layout_get_ds_info(struct inode *inode) return &FF_LAYOUT_FROM_HDR(layout)->commit_info; } +static void +ff_layout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct inode *inode) +{ + spin_lock(&inode->i_lock); + pnfs_generic_ds_cinfo_destroy(fl_cinfo); + spin_unlock(&inode->i_lock); +} + static void ff_layout_free_deviceid_node(struct nfs4_deviceid_node *d) { @@ -2503,6 +2513,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_read_ops = &ff_layout_pg_read_ops, .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, + .release_ds_info = ff_layout_release_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, .mark_request_commit = pnfs_layout_mark_request_commit, .clear_request_commit = pnfs_generic_clear_request_commit, diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 4a1adad3740f..683146a51599 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -534,9 +534,11 @@ void nfs_clear_pnfs_ds_commit_verifiers(struct pnfs_ds_commit_info *cinfo) pnfs_bucket_clear_pnfs_ds_commit_verifiers(cinfo->buckets, cinfo->nbuckets); - list_for_each_entry(array, &cinfo->commits, cinfo_list) + rcu_read_lock(); + list_for_each_entry_rcu(array, &cinfo->commits, cinfo_list) pnfs_bucket_clear_pnfs_ds_commit_verifiers(array->buckets, array->nbuckets); + rcu_read_unlock(); } #else static inline diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 6b25117fca5f..eba18f137fb0 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -506,6 +506,7 @@ pnfs_init_lseg(struct pnfs_layout_hdr *lo, struct pnfs_layout_segment *lseg, { INIT_LIST_HEAD(&lseg->pls_list); INIT_LIST_HEAD(&lseg->pls_lc_list); + INIT_LIST_HEAD(&lseg->pls_commits); refcount_set(&lseg->pls_refcount, 1); set_bit(NFS_LSEG_VALID, &lseg->pls_flags); lseg->pls_layout = lo; diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 4ff974631206..9742ab184960 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -66,6 +66,7 @@ struct nfs4_pnfs_ds { struct pnfs_layout_segment { struct list_head pls_list; struct list_head pls_lc_list; + struct list_head pls_commits; struct pnfs_layout_range pls_range; refcount_t pls_refcount; u32 pls_seq; @@ -370,6 +371,9 @@ void nfs4_deviceid_purge_client(const struct nfs_client *); /* pnfs_nfs.c */ struct pnfs_commit_array *pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags); void pnfs_free_commit_array(struct pnfs_commit_array *p); +void pnfs_generic_ds_cinfo_release_lseg(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg); +void pnfs_generic_ds_cinfo_destroy(struct pnfs_ds_commit_info *fl_cinfo); void pnfs_generic_clear_request_commit(struct nfs_page *req, struct nfs_commit_info *cinfo); diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index f895a28b1e26..edad251a6a48 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -118,6 +118,67 @@ pnfs_free_commit_array(struct pnfs_commit_array *p) } EXPORT_SYMBOL_GPL(pnfs_free_commit_array); +static void +pnfs_release_commit_array_locked(struct pnfs_commit_array *array) +{ + list_del_rcu(&array->cinfo_list); + list_del(&array->lseg_list); + pnfs_free_commit_array(array); +} + +static void +pnfs_put_commit_array_locked(struct pnfs_commit_array *array) +{ + if (refcount_dec_and_test(&array->refcount)) + pnfs_release_commit_array_locked(array); +} + +static void +pnfs_put_commit_array(struct pnfs_commit_array *array, struct inode *inode) +{ + if (refcount_dec_and_lock(&array->refcount, &inode->i_lock)) { + pnfs_release_commit_array_locked(array); + spin_unlock(&inode->i_lock); + } +} + +static struct pnfs_commit_array * +pnfs_get_commit_array(struct pnfs_commit_array *array) +{ + if (refcount_inc_not_zero(&array->refcount)) + return array; + return NULL; +} + +static void +pnfs_remove_and_free_commit_array(struct pnfs_commit_array *array) +{ + array->lseg = NULL; + list_del_init(&array->lseg_list); + pnfs_put_commit_array_locked(array); +} + +void +pnfs_generic_ds_cinfo_release_lseg(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array, *tmp; + + list_for_each_entry_safe(array, tmp, &lseg->pls_commits, lseg_list) + pnfs_remove_and_free_commit_array(array); +} +EXPORT_SYMBOL_GPL(pnfs_generic_ds_cinfo_release_lseg); + +void +pnfs_generic_ds_cinfo_destroy(struct pnfs_ds_commit_info *fl_cinfo) +{ + struct pnfs_commit_array *array, *tmp; + + list_for_each_entry_safe(array, tmp, &fl_cinfo->commits, cinfo_list) + pnfs_remove_and_free_commit_array(array); +} +EXPORT_SYMBOL_GPL(pnfs_generic_ds_cinfo_destroy); + /* * Locks the nfs_page requests for commit and moves them to * @bucket->committing. @@ -177,14 +238,21 @@ int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) max -= cnt; if (!max) return rv; - list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + rcu_read_lock(); + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (!array->lseg || !pnfs_get_commit_array(array)) + continue; + rcu_read_unlock(); cnt = pnfs_bucket_scan_array(cinfo, array->buckets, array->nbuckets, max); + rcu_read_lock(); + pnfs_put_commit_array(array, cinfo->inode); rv += cnt; max -= cnt; if (!max) break; } + rcu_read_unlock(); return rv; } EXPORT_SYMBOL_GPL(pnfs_generic_scan_commit_lists); @@ -230,13 +298,20 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, fl_cinfo->nbuckets, cinfo); fl_cinfo->nwritten -= nwritten; - list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + rcu_read_lock(); + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (!array->lseg || !pnfs_get_commit_array(array)) + continue; + rcu_read_unlock(); nwritten = pnfs_bucket_recover_commit_reqs(dst, array->buckets, array->nbuckets, cinfo); + rcu_read_lock(); + pnfs_put_commit_array(array, cinfo->inode); fl_cinfo->nwritten -= nwritten; } + rcu_read_unlock(); } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); @@ -330,9 +405,16 @@ pnfs_alloc_ds_commits_list(struct list_head *list, struct pnfs_commit_array *array; unsigned int ret = 0; - list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) + rcu_read_lock(); + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (!array->lseg || !pnfs_get_commit_array(array)) + continue; + rcu_read_unlock(); ret += pnfs_bucket_alloc_ds_commits(list, array->buckets, array->nbuckets, cinfo); + rcu_read_lock(); + pnfs_put_commit_array(array, cinfo->inode); + } return ret; } diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 9946787eda72..33be2ee2a248 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1275,6 +1275,7 @@ struct pnfs_commit_array { struct list_head lseg_list; struct pnfs_layout_segment *lseg; struct rcu_head rcu; + refcount_t refcount; unsigned int nbuckets; struct pnfs_commit_bucket buckets[]; }; From patchwork Tue Mar 24 23:47:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456733 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DEB4E17D4 for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C1B582076A for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093785; bh=hhzKpP8c2GTIW7gqxW/CRy3p6bi3jL+mDtXSFO5Fn1s=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=MvDtGtSm54Lsj+xyKAEwpexSfVNm7Qj+lS6iEHhiiBFLnss2HHlo80+xgjg7e8S9F 5LtuBkLhosiJ0zJ45PSZ73pzT2FHPqmDxcAZaB0N10bsjCxytI5VXG36GFmMFDC02S WIyPstfnLupnivRhoROn+aMGQ2iFIhhEeofyG464= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727134AbgCXXto (ORCPT ); Tue, 24 Mar 2020 19:49:44 -0400 Received: from mail.kernel.org ([198.145.29.99]:35394 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727148AbgCXXto (ORCPT ); Tue, 24 Mar 2020 19:49:44 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 840972073E for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093783; bh=hhzKpP8c2GTIW7gqxW/CRy3p6bi3jL+mDtXSFO5Fn1s=; h=From:To:Subject:Date:In-Reply-To:References:From; b=h82gNB+b7NmlSIZWrH/0wJ4ANBUmvR0F71+j+49HX/Q7pYInYDqxEe8srEBloXNs2 wzdoC9coRIaaI9KkbU6Ids2W+5rGH4IV9zJjBaJuo9NUncpqs3btbtO80bVx+BFu4r K3vS488Nh9bBlOo2I6KE5o2Li4CaD6ryI0RUUFj0= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 13/22] pNFS: Enable per-layout segment commit structures Date: Tue, 24 Mar 2020 19:47:19 -0400 Message-Id: <20200324234728.8997-14-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-13-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Enable adding and lookup of per-layout segment commits in filelayout and flexfilelayout. Signed-off-by: Trond Myklebust --- fs/nfs/filelayout/filelayout.c | 21 +++++++ fs/nfs/flexfilelayout/flexfilelayout.c | 19 +++++++ fs/nfs/pnfs.h | 6 ++ fs/nfs/pnfs_nfs.c | 77 ++++++++++++++++++++++++-- 4 files changed, 117 insertions(+), 6 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 1cb043838b14..90c660aad337 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1168,6 +1168,26 @@ filelayout_get_ds_info(struct inode *inode) return &FILELAYOUT_FROM_HDR(layout)->commit_info; } +static void +filelayout_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct nfs4_filelayout_segment *fl = FILELAYOUT_LSEG(lseg); + struct inode *inode = lseg->pls_layout->plh_inode; + struct pnfs_commit_array *array, *new; + unsigned int size = (fl->stripe_type == STRIPE_SPARSE) ? + fl->dsaddr->ds_num : fl->dsaddr->stripe_count; + + new = pnfs_alloc_commit_array(size, GFP_NOIO); + if (new) { + spin_lock(&inode->i_lock); + array = pnfs_add_commit_array(fl_cinfo, new, lseg); + spin_unlock(&inode->i_lock); + if (array != new) + pnfs_free_commit_array(new); + } +} + static void filelayout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) @@ -1191,6 +1211,7 @@ static struct pnfs_layoutdriver_type filelayout_type = { .pg_read_ops = &filelayout_pg_read_ops, .pg_write_ops = &filelayout_pg_write_ops, .get_ds_info = &filelayout_get_ds_info, + .setup_ds_info = filelayout_setup_ds_info, .release_ds_info = filelayout_release_ds_info, .mark_request_commit = filelayout_mark_request_commit, .clear_request_commit = pnfs_generic_clear_request_commit, diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 3ddd8080f76d..e4db8510ef5f 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -2004,6 +2004,24 @@ ff_layout_get_ds_info(struct inode *inode) return &FF_LAYOUT_FROM_HDR(layout)->commit_info; } +static void +ff_layout_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct nfs4_ff_layout_segment *flseg = FF_LAYOUT_LSEG(lseg); + struct inode *inode = lseg->pls_layout->plh_inode; + struct pnfs_commit_array *array, *new; + + new = pnfs_alloc_commit_array(flseg->mirror_array_cnt, GFP_NOIO); + if (new) { + spin_lock(&inode->i_lock); + array = pnfs_add_commit_array(fl_cinfo, new, lseg); + spin_unlock(&inode->i_lock); + if (array != new) + pnfs_free_commit_array(new); + } +} + static void ff_layout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) @@ -2513,6 +2531,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_read_ops = &ff_layout_pg_read_ops, .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, + .setup_ds_info = ff_layout_setup_ds_info, .release_ds_info = ff_layout_release_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, .mark_request_commit = pnfs_layout_mark_request_commit, diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 9742ab184960..434503875f7f 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -150,6 +150,8 @@ struct pnfs_layoutdriver_type { const struct nfs_pageio_ops *pg_write_ops; struct pnfs_ds_commit_info *(*get_ds_info) (struct inode *inode); + void (*setup_ds_info)(struct pnfs_ds_commit_info *, + struct pnfs_layout_segment *); void (*release_ds_info)(struct pnfs_ds_commit_info *, struct inode *inode); void (*mark_request_commit) (struct nfs_page *req, @@ -371,6 +373,10 @@ void nfs4_deviceid_purge_client(const struct nfs_client *); /* pnfs_nfs.c */ struct pnfs_commit_array *pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags); void pnfs_free_commit_array(struct pnfs_commit_array *p); +struct pnfs_commit_array *pnfs_add_commit_array(struct pnfs_ds_commit_info *, + struct pnfs_commit_array *, + struct pnfs_layout_segment *); + void pnfs_generic_ds_cinfo_release_lseg(struct pnfs_ds_commit_info *fl_cinfo, struct pnfs_layout_segment *lseg); void pnfs_generic_ds_cinfo_destroy(struct pnfs_ds_commit_info *fl_cinfo); diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index edad251a6a48..5b426a090ee3 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -118,6 +118,66 @@ pnfs_free_commit_array(struct pnfs_commit_array *p) } EXPORT_SYMBOL_GPL(pnfs_free_commit_array); +static struct pnfs_commit_array * +pnfs_find_commit_array_by_lseg(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array; + + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (array->lseg == lseg) + return array; + } + return NULL; +} + +struct pnfs_commit_array * +pnfs_add_commit_array(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_commit_array *new, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array; + + array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); + if (array) + return array; + new->lseg = lseg; + refcount_set(&new->refcount, 1); + list_add_rcu(&new->cinfo_list, &fl_cinfo->commits); + list_add(&new->lseg_list, &lseg->pls_commits); + return new; +} +EXPORT_SYMBOL_GPL(pnfs_add_commit_array); + +static void +pnfs_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct inode *inode = lseg->pls_layout->plh_inode; + struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + + if (ld->setup_ds_info != NULL) + ld->setup_ds_info(fl_cinfo, lseg); +} + +static struct pnfs_commit_array * +pnfs_lookup_commit_array(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array; + + rcu_read_lock(); + array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); + if (!array) { + rcu_read_unlock(); + pnfs_setup_ds_info(fl_cinfo, lseg); + rcu_read_lock(); + array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); + } + rcu_read_unlock(); + return array; +} + static void pnfs_release_commit_array_locked(struct pnfs_commit_array *array) { @@ -1082,17 +1142,18 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, u32 ds_commit_idx) { struct list_head *list; + struct pnfs_commit_array *array; struct pnfs_commit_bucket *buckets; mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); - buckets = cinfo->ds->buckets; + array = pnfs_lookup_commit_array(cinfo->ds, lseg); + if (!array) + goto out_resched; + buckets = array->buckets; list = &buckets[ds_commit_idx].written; if (list_empty(list)) { - if (!pnfs_is_valid_lseg(lseg)) { - mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); - cinfo->completion_ops->resched_write(cinfo, req); - return; - } + if (!pnfs_is_valid_lseg(lseg)) + goto out_resched; /* Non-empty buckets hold a reference on the lseg. That ref * is normally transferred to the COMMIT call and released * there. It could also be released if the last req is pulled @@ -1108,6 +1169,10 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, nfs_request_add_commit_list_locked(req, list, cinfo); mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); nfs_mark_page_unstable(req->wb_page, cinfo); + return; +out_resched: + mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); + cinfo->completion_ops->resched_write(cinfo, req); } EXPORT_SYMBOL_GPL(pnfs_layout_mark_request_commit); From patchwork Tue Mar 24 23:47:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456731 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6FA3174A for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9A6152074D for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093785; bh=MndP8/Rbk2/z0nAKJxncvbhzUO1YBScofE5SoxyUJ/4=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=nX/fLmFen/9T/LQdhwZTu6J36e+GezQoS7jUI1eZ+34oFCYYZ97l+nWhO2D0nDjbJ XnkkCy6XP4OAYwORx36hicDK1ZADVWjdZCBIxLfoI+4s5OzHkpivcWZypo6mDIBJ2n ahwUor+KGmyC8wGA/BcQU0fBdOpSlolJf7XyA89I= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727151AbgCXXto (ORCPT ); Tue, 24 Mar 2020 19:49:44 -0400 Received: from mail.kernel.org ([198.145.29.99]:35404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727135AbgCXXto (ORCPT ); Tue, 24 Mar 2020 19:49:44 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 02AE820719 for ; Tue, 24 Mar 2020 23:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093784; bh=MndP8/Rbk2/z0nAKJxncvbhzUO1YBScofE5SoxyUJ/4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=1bdxFS89vsuLQJ/IcJEZZUJsOhlYlDAaOStrg5CT2ruhniTBpW3BkvTQNEkMrmKjN 75bgHXQqE0ys8waW5S7O/5v4OCHfEKCZHTw2J5cenJ4Tk5V6O0LmocLv3V7OGlf8LF rJMiIw3+no3DbHYlrqWSu1D50DD8QwJAkMhgRQMk= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 14/22] NFS/pNFS: Add a helper pnfs_generic_search_commit_reqs() Date: Tue, 24 Mar 2020 19:47:20 -0400 Message-Id: <20200324234728.8997-15-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-14-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Lift filelayout_search_commit_reqs() into the generic pnfs/nfs code, and add support for commit arrays. Signed-off-by: Trond Myklebust --- fs/nfs/filelayout/filelayout.c | 32 +-------------------- fs/nfs/pnfs.h | 2 ++ fs/nfs/pnfs_nfs.c | 51 ++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 31 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 90c660aad337..05c97b6ba15d 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1083,36 +1083,6 @@ static int filelayout_initiate_commit(struct nfs_commit_data *data, int how) return -EAGAIN; } -/* filelayout_search_commit_reqs - Search lists in @cinfo for the head reqest - * for @page - * @cinfo - commit info for current inode - * @page - page to search for matching head request - * - * Returns a the head request if one is found, otherwise returns NULL. - */ -static struct nfs_page * -filelayout_search_commit_reqs(struct nfs_commit_info *cinfo, struct page *page) -{ - struct nfs_page *freq, *t; - struct pnfs_commit_bucket *b; - int i; - - /* Linearly search the commit lists for each bucket until a matching - * request is found */ - for (i = 0, b = cinfo->ds->buckets; i < cinfo->ds->nbuckets; i++, b++) { - list_for_each_entry_safe(freq, t, &b->written, wb_list) { - if (freq->wb_page == page) - return freq->wb_head; - } - list_for_each_entry_safe(freq, t, &b->committing, wb_list) { - if (freq->wb_page == page) - return freq->wb_head; - } - } - - return NULL; -} - static int filelayout_commit_pagelist(struct inode *inode, struct list_head *mds_pages, int how, struct nfs_commit_info *cinfo) @@ -1217,7 +1187,7 @@ static struct pnfs_layoutdriver_type filelayout_type = { .clear_request_commit = pnfs_generic_clear_request_commit, .scan_commit_lists = pnfs_generic_scan_commit_lists, .recover_commit_reqs = pnfs_generic_recover_commit_reqs, - .search_commit_reqs = filelayout_search_commit_reqs, + .search_commit_reqs = pnfs_generic_search_commit_reqs, .commit_pagelist = filelayout_commit_pagelist, .read_pagelist = filelayout_read_pagelist, .write_pagelist = filelayout_write_pagelist, diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 434503875f7f..20acd5108ddf 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -388,6 +388,8 @@ void pnfs_generic_prepare_to_resend_writes(struct nfs_commit_data *data); void pnfs_generic_rw_release(void *data); void pnfs_generic_recover_commit_reqs(struct list_head *dst, struct nfs_commit_info *cinfo); +struct nfs_page *pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo, + struct page *page); int pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, int how, diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 5b426a090ee3..9b55919e64ac 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -375,6 +375,57 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); +static struct nfs_page * +pnfs_bucket_search_commit_reqs(struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, struct page *page) +{ + struct nfs_page *req; + struct pnfs_commit_bucket *b; + unsigned int i; + + /* Linearly search the commit lists for each bucket until a matching + * request is found */ + for (i = 0, b = buckets; i < nbuckets; i++, b++) { + list_for_each_entry(req, &b->written, wb_list) { + if (req->wb_page == page) + return req->wb_head; + } + list_for_each_entry(req, &b->committing, wb_list) { + if (req->wb_page == page) + return req->wb_head; + } + } + return NULL; +} + +/* pnfs_generic_search_commit_reqs - Search lists in @cinfo for the head reqest + * for @page + * @cinfo - commit info for current inode + * @page - page to search for matching head request + * + * Returns a the head request if one is found, otherwise returns NULL. + */ +struct nfs_page * +pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo, struct page *page) +{ + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + struct pnfs_commit_array *array; + struct nfs_page *req; + + req = pnfs_bucket_search_commit_reqs(fl_cinfo->buckets, + fl_cinfo->nbuckets, page); + if (req) + return req; + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + req = pnfs_bucket_search_commit_reqs(array->buckets, + array->nbuckets, page); + if (req) + return req; + } + return NULL; +} +EXPORT_SYMBOL_GPL(pnfs_generic_search_commit_reqs); + static struct pnfs_layout_segment * pnfs_bucket_get_committing(struct list_head *head, struct pnfs_commit_bucket *bucket, From patchwork Tue Mar 24 23:47:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456737 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 895EA13A4 for ; Tue, 24 Mar 2020 23:49:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 63A612073E for ; Tue, 24 Mar 2020 23:49:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093787; bh=b2erwZpS9u5QnDIOFBwu7m0SzDEOCF4A88kdglAtMZE=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=sjE9bReZvaTIEx7nFvpiFnJLZjDfU98GYmKep+R5iAjDFeYMhmEHrDkMAdO7ulZ/o h3ye14EilYgdiKzObDpOigdI7zf+o+pq5PrFRxy69Ld8u7B2raOoPCAWvuW/kMDB+a vehesXnK4AB/m626VNNQ48Tlhfs3TDDvjFO+fgws= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727175AbgCXXtq (ORCPT ); Tue, 24 Mar 2020 19:49:46 -0400 Received: from mail.kernel.org ([198.145.29.99]:35432 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727148AbgCXXtq (ORCPT ); Tue, 24 Mar 2020 19:49:46 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 730CF2073C for ; Tue, 24 Mar 2020 23:49:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093784; bh=b2erwZpS9u5QnDIOFBwu7m0SzDEOCF4A88kdglAtMZE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=oO7mItZo0g1l9SLLdiX/4iuiYyowaUty02tXcBfp1zFICeqa4JDiIk1orO6kQfS4D 0sQ2rtd20Dn7SZd7O6YhpG3Rfk+zRhIt4100AKDXGeldVVwV8KLOR+h6ElIzwaxl8y zRI94dQZXVA4FE2BurHIyIsXMSemu+PDHgJwjFJE= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 15/22] NFS: Remove bucket array from struct pnfs_ds_commit_info Date: Tue, 24 Mar 2020 19:47:21 -0400 Message-Id: <20200324234728.8997-16-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-15-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Remove the unused bucket array in struct pnfs_ds_commit_info. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 1 - fs/nfs/filelayout/filelayout.c | 75 +------------------------ fs/nfs/flexfilelayout/flexfilelayout.c | 76 -------------------------- fs/nfs/internal.h | 3 - fs/nfs/pnfs_nfs.c | 18 ------ include/linux/nfs_xdr.h | 13 ----- 6 files changed, 1 insertion(+), 185 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 0f970ef2f026..a76c2160f5e5 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -217,7 +217,6 @@ static void nfs_direct_req_free(struct kref *kref) struct nfs_direct_req *dreq = container_of(kref, struct nfs_direct_req, kref); pnfs_release_ds_info(&dreq->ds_cinfo, dreq->inode); - nfs_free_pnfs_ds_cinfo(&dreq->ds_cinfo); if (dreq->l_ctx != NULL) nfs_put_lock_context(dreq->l_ctx); if (dreq->ctx != NULL) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 05c97b6ba15d..0bb4f0e0a4ba 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -755,72 +755,12 @@ filelayout_free_lseg(struct pnfs_layout_segment *lseg) flo = FILELAYOUT_FROM_HDR(lseg->pls_layout); inode = flo->generic_hdr.plh_inode; spin_lock(&inode->i_lock); - flo->commit_info.nbuckets = 0; - kfree(flo->commit_info.buckets); - flo->commit_info.buckets = NULL; pnfs_generic_ds_cinfo_release_lseg(&flo->commit_info, lseg); spin_unlock(&inode->i_lock); } _filelayout_free_lseg(fl); } -static int -filelayout_alloc_commit_info(struct pnfs_layout_segment *lseg, - struct nfs_commit_info *cinfo, - gfp_t gfp_flags) -{ - struct nfs4_filelayout_segment *fl = FILELAYOUT_LSEG(lseg); - struct pnfs_commit_bucket *buckets; - int size, i; - - if (fl->commit_through_mds) - return 0; - - size = (fl->stripe_type == STRIPE_SPARSE) ? - fl->dsaddr->ds_num : fl->dsaddr->stripe_count; - - if (cinfo->ds->nbuckets >= size) { - /* This assumes there is only one IOMODE_RW lseg. What - * we really want to do is have a layout_hdr level - * dictionary of keys, each - * associated with a struct list_head, populated by calls - * to filelayout_write_pagelist(). - * */ - return 0; - } - - buckets = kcalloc(size, sizeof(struct pnfs_commit_bucket), - gfp_flags); - if (!buckets) - return -ENOMEM; - for (i = 0; i < size; i++) { - INIT_LIST_HEAD(&buckets[i].written); - INIT_LIST_HEAD(&buckets[i].committing); - /* mark direct verifier as unset */ - buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW; - } - - spin_lock(&cinfo->inode->i_lock); - if (cinfo->ds->nbuckets >= size) - goto out; - for (i = 0; i < cinfo->ds->nbuckets; i++) { - list_splice(&cinfo->ds->buckets[i].written, - &buckets[i].written); - list_splice(&cinfo->ds->buckets[i].committing, - &buckets[i].committing); - buckets[i].direct_verf.committed = - cinfo->ds->buckets[i].direct_verf.committed; - buckets[i].wlseg = cinfo->ds->buckets[i].wlseg; - buckets[i].clseg = cinfo->ds->buckets[i].clseg; - } - swap(cinfo->ds->buckets, buckets); - cinfo->ds->nbuckets = size; -out: - spin_unlock(&cinfo->inode->i_lock); - kfree(buckets); - return 0; -} - static struct pnfs_layout_segment * filelayout_alloc_lseg(struct pnfs_layout_hdr *layoutid, struct nfs4_layoutget_res *lgr, @@ -943,9 +883,6 @@ static void filelayout_pg_init_write(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) { - struct nfs_commit_info cinfo; - int status; - pnfs_generic_pg_check_layout(pgio); if (!pgio->pg_lseg) { pgio->pg_lseg = fl_pnfs_update_layout(pgio->pg_inode, @@ -964,17 +901,7 @@ filelayout_pg_init_write(struct nfs_pageio_descriptor *pgio, /* If no lseg, fall back to write through mds */ if (pgio->pg_lseg == NULL) - goto out_mds; - nfs_init_cinfo(&cinfo, pgio->pg_inode, pgio->pg_dreq); - status = filelayout_alloc_commit_info(pgio->pg_lseg, &cinfo, GFP_NOFS); - if (status < 0) { - pnfs_put_lseg(pgio->pg_lseg); - pgio->pg_lseg = NULL; - goto out_mds; - } - return; -out_mds: - nfs_pageio_reset_write_mds(pgio); + nfs_pageio_reset_write_mds(pgio); } static const struct nfs_pageio_ops filelayout_pg_read_ops = { diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index e4db8510ef5f..8fed7bae70b3 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -550,17 +550,6 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, goto out_free_page; } -static bool ff_layout_has_rw_segments(struct pnfs_layout_hdr *layout) -{ - struct pnfs_layout_segment *lseg; - - list_for_each_entry(lseg, &layout->plh_segs, pls_list) - if (lseg->pls_range.iomode == IOMODE_RW) - return true; - - return false; -} - static void ff_layout_free_lseg(struct pnfs_layout_segment *lseg) { @@ -575,24 +564,12 @@ ff_layout_free_lseg(struct pnfs_layout_segment *lseg) ffl = FF_LAYOUT_FROM_HDR(lseg->pls_layout); inode = ffl->generic_hdr.plh_inode; spin_lock(&inode->i_lock); - if (!ff_layout_has_rw_segments(lseg->pls_layout)) { - ffl->commit_info.nbuckets = 0; - kfree(ffl->commit_info.buckets); - ffl->commit_info.buckets = NULL; - } pnfs_generic_ds_cinfo_release_lseg(&ffl->commit_info, lseg); spin_unlock(&inode->i_lock); } _ff_layout_free_lseg(fls); } -/* Return 1 until we have multiple lsegs support */ -static int -ff_layout_get_lseg_count(struct nfs4_ff_layout_segment *fls) -{ - return 1; -} - static void nfs4_ff_start_busy_timer(struct nfs4_ff_busy_timer *timer, ktime_t now) { @@ -737,52 +714,6 @@ nfs4_ff_layout_stat_io_end_write(struct rpc_task *task, spin_unlock(&mirror->lock); } -static int -ff_layout_alloc_commit_info(struct pnfs_layout_segment *lseg, - struct nfs_commit_info *cinfo, - gfp_t gfp_flags) -{ - struct nfs4_ff_layout_segment *fls = FF_LAYOUT_LSEG(lseg); - struct pnfs_commit_bucket *buckets; - int size; - - if (cinfo->ds->nbuckets != 0) { - /* This assumes there is only one RW lseg per file. - * To support multiple lseg per file, we need to - * change struct pnfs_commit_bucket to allow dynamic - * increasing nbuckets. - */ - return 0; - } - - size = ff_layout_get_lseg_count(fls) * FF_LAYOUT_MIRROR_COUNT(lseg); - - buckets = kcalloc(size, sizeof(struct pnfs_commit_bucket), - gfp_flags); - if (!buckets) - return -ENOMEM; - else { - int i; - - spin_lock(&cinfo->inode->i_lock); - if (cinfo->ds->nbuckets != 0) - kfree(buckets); - else { - cinfo->ds->buckets = buckets; - cinfo->ds->nbuckets = size; - for (i = 0; i < size; i++) { - INIT_LIST_HEAD(&buckets[i].written); - INIT_LIST_HEAD(&buckets[i].committing); - /* mark direct verifier as unset */ - buckets[i].direct_verf.committed = - NFS_INVALID_STABLE_HOW; - } - } - spin_unlock(&cinfo->inode->i_lock); - return 0; - } -} - static void ff_layout_mark_ds_unreachable(struct pnfs_layout_segment *lseg, int idx) { @@ -944,10 +875,8 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, { struct nfs4_ff_layout_mirror *mirror; struct nfs_pgio_mirror *pgm; - struct nfs_commit_info cinfo; struct nfs4_pnfs_ds *ds; int i; - int status; retry: pnfs_generic_pg_check_layout(pgio); @@ -969,11 +898,6 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, if (pgio->pg_lseg == NULL) goto out_mds; - nfs_init_cinfo(&cinfo, pgio->pg_inode, pgio->pg_dreq); - status = ff_layout_alloc_commit_info(pgio->pg_lseg, &cinfo, GFP_NOFS); - if (status < 0) - goto out_mds; - /* Use a direct mapping of ds_idx to pgio mirror_idx */ if (WARN_ON_ONCE(pgio->pg_mirror_count != FF_LAYOUT_MIRROR_COUNT(pgio->pg_lseg))) diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 683146a51599..78f317fac940 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -531,9 +531,6 @@ void nfs_clear_pnfs_ds_commit_verifiers(struct pnfs_ds_commit_info *cinfo) { struct pnfs_commit_array *array; - pnfs_bucket_clear_pnfs_ds_commit_verifiers(cinfo->buckets, - cinfo->nbuckets); - rcu_read_lock(); list_for_each_entry_rcu(array, &cinfo->commits, cinfo_list) pnfs_bucket_clear_pnfs_ds_commit_verifiers(array->buckets, diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 9b55919e64ac..20f12f3cbe38 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -292,12 +292,6 @@ int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) struct pnfs_commit_array *array; int rv = 0, cnt; - cnt = pnfs_bucket_scan_array(cinfo, fl_cinfo->buckets, - fl_cinfo->nbuckets, max); - rv += cnt; - max -= cnt; - if (!max) - return rv; rcu_read_lock(); list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { if (!array->lseg || !pnfs_get_commit_array(array)) @@ -353,11 +347,6 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, unsigned int nwritten; lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); - nwritten = pnfs_bucket_recover_commit_reqs(dst, - fl_cinfo->buckets, - fl_cinfo->nbuckets, - cinfo); - fl_cinfo->nwritten -= nwritten; rcu_read_lock(); list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { if (!array->lseg || !pnfs_get_commit_array(array)) @@ -412,10 +401,6 @@ pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo, struct page *page struct pnfs_commit_array *array; struct nfs_page *req; - req = pnfs_bucket_search_commit_reqs(fl_cinfo->buckets, - fl_cinfo->nbuckets, page); - if (req) - return req; list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { req = pnfs_bucket_search_commit_reqs(array->buckets, array->nbuckets, page); @@ -550,9 +535,6 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, nreq++; } - nreq += pnfs_bucket_alloc_ds_commits(&list, fl_cinfo->buckets, - fl_cinfo->nbuckets, cinfo); - nreq += pnfs_alloc_ds_commits_list(&list, fl_cinfo, cinfo); if (nreq == 0) goto out; diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 33be2ee2a248..2903597ec88c 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1284,8 +1284,6 @@ struct pnfs_ds_commit_info { struct list_head commits; unsigned int nwritten; unsigned int ncommitting; - unsigned int nbuckets; - struct pnfs_commit_bucket *buckets; }; struct nfs41_state_protection { @@ -1396,22 +1394,11 @@ struct nfs41_free_stateid_res { unsigned int status; }; -static inline void -nfs_free_pnfs_ds_cinfo(struct pnfs_ds_commit_info *cinfo) -{ - kfree(cinfo->buckets); -} - #else struct pnfs_ds_commit_info { }; -static inline void -nfs_free_pnfs_ds_cinfo(struct pnfs_ds_commit_info *cinfo) -{ -} - #endif /* CONFIG_NFS_V4_1 */ #ifdef CONFIG_NFS_V4_2 From patchwork Tue Mar 24 23:47:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456747 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4A6013A4 for ; Tue, 24 Mar 2020 23:49:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B72E02072E for ; Tue, 24 Mar 2020 23:49:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093789; bh=aWR7vToA2x0Cp+5ialQ0dpPC0W2mWnnR9XGPg+c4XTw=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=awO0NZcokjUZt7yjnhXfBfSoQg1kA5MAdDD9Z6QQE7oOhovWi1UjAJqh58OQC8TgW Bl19crt4UEaXYCQCCQXxUZ09IjUkkeT7EDWemqxo2N50Y9guOa/PQQxRn//fdUuk62 7AhgkgpqlJaCTkBRneGhAHMNv6f2xcbnake0pO68= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727167AbgCXXtr (ORCPT ); Tue, 24 Mar 2020 19:49:47 -0400 Received: from mail.kernel.org ([198.145.29.99]:35438 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727163AbgCXXtq (ORCPT ); Tue, 24 Mar 2020 19:49:46 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EF1DE20719 for ; Tue, 24 Mar 2020 23:49:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093785; bh=aWR7vToA2x0Cp+5ialQ0dpPC0W2mWnnR9XGPg+c4XTw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=n7qnTgVLmgKeEXNjbFHojvm6A3TjXwpkK7MTwCY0dEEQjaaReSRMkMFAMQU/6GVs7 k2o2x2WYqwXdKS/hXd7Q8h1eFDVrn9IjwQpIztRZO0lOzMtzODSWV39BwjLxf4uB+N 2Jp9BbETg/HUy5qSFoijmfO0Uu/iJekto/NvXarA= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 16/22] NFS/pNFS: Clean up pNFS commit operations Date: Tue, 24 Mar 2020 19:47:22 -0400 Message-Id: <20200324234728.8997-17-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-16-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Move the pNFS commit related operations into a separate structure that can be carried by the pnfs_ds_commit_info. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 6 +- fs/nfs/filelayout/filelayout.c | 20 +++-- fs/nfs/flexfilelayout/flexfilelayout.c | 19 +++-- fs/nfs/pnfs.h | 111 ++++++++++++++++--------- fs/nfs/pnfs_nfs.c | 13 +-- include/linux/nfs_xdr.h | 1 + 6 files changed, 99 insertions(+), 71 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index a76c2160f5e5..d5b6760589a1 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -511,10 +511,7 @@ nfs_direct_write_scan_commit_list(struct inode *inode, struct nfs_commit_info *cinfo) { mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); -#ifdef CONFIG_NFS_V4_1 - if (cinfo->ds != NULL && cinfo->ds->nwritten != 0) - NFS_SERVER(inode)->pnfs_curr_ld->recover_commit_reqs(list, cinfo); -#endif + pnfs_recover_commit_reqs(list, cinfo); nfs_scan_commit_list(&cinfo->mds->list, list, cinfo, 0); mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); } @@ -917,6 +914,7 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter) dreq->l_ctx = l_ctx; if (!is_sync_kiocb(iocb)) dreq->iocb = iocb; + pnfs_init_ds_commit_info(&dreq->ds_cinfo, inode); nfs_start_io_direct(inode); diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 0bb4f0e0a4ba..ef4881540c64 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -49,6 +49,7 @@ MODULE_AUTHOR("Dean Hildebrand "); MODULE_DESCRIPTION("The NFSv4 file layout driver"); #define FILELAYOUT_POLL_RETRY_MAX (15*HZ) +static const struct pnfs_commit_ops filelayout_commit_ops; static loff_t filelayout_get_dense_offset(struct nfs4_filelayout_segment *flseg, @@ -1045,6 +1046,7 @@ filelayout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) if (flo == NULL) return NULL; INIT_LIST_HEAD(&flo->commit_info.commits); + flo->commit_info.ops = &filelayout_commit_ops; return &flo->generic_hdr; } @@ -1094,6 +1096,16 @@ filelayout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, spin_unlock(&inode->i_lock); } +static const struct pnfs_commit_ops filelayout_commit_ops = { + .setup_ds_info = filelayout_setup_ds_info, + .release_ds_info = filelayout_release_ds_info, + .mark_request_commit = filelayout_mark_request_commit, + .clear_request_commit = pnfs_generic_clear_request_commit, + .scan_commit_lists = pnfs_generic_scan_commit_lists, + .recover_commit_reqs = pnfs_generic_recover_commit_reqs, + .search_commit_reqs = pnfs_generic_search_commit_reqs, + .commit_pagelist = filelayout_commit_pagelist, +}; static struct pnfs_layoutdriver_type filelayout_type = { .id = LAYOUT_NFSV4_1_FILES, @@ -1108,14 +1120,6 @@ static struct pnfs_layoutdriver_type filelayout_type = { .pg_read_ops = &filelayout_pg_read_ops, .pg_write_ops = &filelayout_pg_write_ops, .get_ds_info = &filelayout_get_ds_info, - .setup_ds_info = filelayout_setup_ds_info, - .release_ds_info = filelayout_release_ds_info, - .mark_request_commit = filelayout_mark_request_commit, - .clear_request_commit = pnfs_generic_clear_request_commit, - .scan_commit_lists = pnfs_generic_scan_commit_lists, - .recover_commit_reqs = pnfs_generic_recover_commit_reqs, - .search_commit_reqs = pnfs_generic_search_commit_reqs, - .commit_pagelist = filelayout_commit_pagelist, .read_pagelist = filelayout_read_pagelist, .write_pagelist = filelayout_write_pagelist, .alloc_deviceid_node = filelayout_alloc_deviceid_node, diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 8fed7bae70b3..cdd0c87b03dc 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -32,6 +32,7 @@ static unsigned short io_maxretrans; +static const struct pnfs_commit_ops ff_layout_commit_ops; static void ff_layout_read_record_layoutstats_done(struct rpc_task *task, struct nfs_pgio_header *hdr); static int ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo, @@ -52,6 +53,7 @@ ff_layout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) INIT_LIST_HEAD(&ffl->error_list); INIT_LIST_HEAD(&ffl->mirrors); ffl->last_report_time = ktime_get(); + ffl->commit_info.ops = &ff_layout_commit_ops; return &ffl->generic_hdr; } else return NULL; @@ -2440,6 +2442,16 @@ ff_layout_set_layoutdriver(struct nfs_server *server, return 0; } +static const struct pnfs_commit_ops ff_layout_commit_ops = { + .setup_ds_info = ff_layout_setup_ds_info, + .release_ds_info = ff_layout_release_ds_info, + .mark_request_commit = pnfs_layout_mark_request_commit, + .clear_request_commit = pnfs_generic_clear_request_commit, + .scan_commit_lists = pnfs_generic_scan_commit_lists, + .recover_commit_reqs = pnfs_generic_recover_commit_reqs, + .commit_pagelist = ff_layout_commit_pagelist, +}; + static struct pnfs_layoutdriver_type flexfilelayout_type = { .id = LAYOUT_FLEX_FILES, .name = "LAYOUT_FLEX_FILES", @@ -2455,14 +2467,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_read_ops = &ff_layout_pg_read_ops, .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, - .setup_ds_info = ff_layout_setup_ds_info, - .release_ds_info = ff_layout_release_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, - .mark_request_commit = pnfs_layout_mark_request_commit, - .clear_request_commit = pnfs_generic_clear_request_commit, - .scan_commit_lists = pnfs_generic_scan_commit_lists, - .recover_commit_reqs = pnfs_generic_recover_commit_reqs, - .commit_pagelist = ff_layout_commit_pagelist, .read_pagelist = ff_layout_read_pagelist, .write_pagelist = ff_layout_write_pagelist, .alloc_deviceid_node = ff_layout_alloc_deviceid_node, diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 20acd5108ddf..52a5d8aaaf46 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -150,26 +150,6 @@ struct pnfs_layoutdriver_type { const struct nfs_pageio_ops *pg_write_ops; struct pnfs_ds_commit_info *(*get_ds_info) (struct inode *inode); - void (*setup_ds_info)(struct pnfs_ds_commit_info *, - struct pnfs_layout_segment *); - void (*release_ds_info)(struct pnfs_ds_commit_info *, - struct inode *inode); - void (*mark_request_commit) (struct nfs_page *req, - struct pnfs_layout_segment *lseg, - struct nfs_commit_info *cinfo, - u32 ds_commit_idx); - void (*clear_request_commit) (struct nfs_page *req, - struct nfs_commit_info *cinfo); - int (*scan_commit_lists) (struct nfs_commit_info *cinfo, - int max); - void (*recover_commit_reqs) (struct list_head *list, - struct nfs_commit_info *cinfo); - struct nfs_page * (*search_commit_reqs)(struct nfs_commit_info *cinfo, - struct page *page); - int (*commit_pagelist)(struct inode *inode, - struct list_head *mds_pages, - int how, - struct nfs_commit_info *cinfo); int (*sync)(struct inode *inode, bool datasync); @@ -192,6 +172,29 @@ struct pnfs_layoutdriver_type { int (*prepare_layoutstats) (struct nfs42_layoutstat_args *args); }; +struct pnfs_commit_ops { + void (*setup_ds_info)(struct pnfs_ds_commit_info *, + struct pnfs_layout_segment *); + void (*release_ds_info)(struct pnfs_ds_commit_info *, + struct inode *inode); + int (*commit_pagelist)(struct inode *inode, + struct list_head *mds_pages, + int how, + struct nfs_commit_info *cinfo); + void (*mark_request_commit) (struct nfs_page *req, + struct pnfs_layout_segment *lseg, + struct nfs_commit_info *cinfo, + u32 ds_commit_idx); + void (*clear_request_commit) (struct nfs_page *req, + struct nfs_commit_info *cinfo); + int (*scan_commit_lists) (struct nfs_commit_info *cinfo, + int max); + void (*recover_commit_reqs) (struct list_head *list, + struct nfs_commit_info *cinfo); + struct nfs_page * (*search_commit_reqs)(struct nfs_commit_info *cinfo, + struct page *page); +}; + struct pnfs_layout_hdr { refcount_t plh_refcount; atomic_t plh_outstanding; /* number of RPCs out */ @@ -461,9 +464,11 @@ static inline int pnfs_commit_list(struct inode *inode, struct list_head *mds_pages, int how, struct nfs_commit_info *cinfo) { - if (cinfo->ds == NULL || cinfo->ds->ncommitting == 0) + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + + if (fl_cinfo == NULL || fl_cinfo->ncommitting == 0) return PNFS_NOT_ATTEMPTED; - return NFS_SERVER(inode)->pnfs_curr_ld->commit_pagelist(inode, mds_pages, how, cinfo); + return fl_cinfo->ops->commit_pagelist(inode, mds_pages, how, cinfo); } static inline struct pnfs_ds_commit_info * @@ -477,12 +482,19 @@ pnfs_get_ds_info(struct inode *inode) } static inline void -pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) { - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *inode_cinfo = pnfs_get_ds_info(inode); + + INIT_LIST_HEAD(&fl_cinfo->commits); + fl_cinfo->ops = (inode_cinfo != NULL) ? inode_cinfo->ops : NULL; +} - if (ld != NULL && ld->release_ds_info != NULL) - ld->release_ds_info(fl_cinfo, inode); +static inline void +pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ + if (fl_cinfo->ops != NULL && fl_cinfo->ops->release_ds_info != NULL) + fl_cinfo->ops->release_ds_info(fl_cinfo, inode); } static inline void @@ -495,24 +507,22 @@ static inline bool pnfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, u32 ds_commit_idx) { - struct inode *inode = d_inode(nfs_req_openctx(req)->dentry); - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - if (lseg == NULL || ld->mark_request_commit == NULL) + if (!lseg || !fl_cinfo->ops->mark_request_commit) return false; - ld->mark_request_commit(req, lseg, cinfo, ds_commit_idx); + fl_cinfo->ops->mark_request_commit(req, lseg, cinfo, ds_commit_idx); return true; } static inline bool pnfs_clear_request_commit(struct nfs_page *req, struct nfs_commit_info *cinfo) { - struct inode *inode = d_inode(nfs_req_openctx(req)->dentry); - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - if (ld == NULL || ld->clear_request_commit == NULL) + if (!fl_cinfo->ops || !fl_cinfo->ops->clear_request_commit) return false; - ld->clear_request_commit(req, cinfo); + fl_cinfo->ops->clear_request_commit(req, cinfo); return true; } @@ -520,21 +530,31 @@ static inline int pnfs_scan_commit_lists(struct inode *inode, struct nfs_commit_info *cinfo, int max) { - if (cinfo->ds == NULL || cinfo->ds->nwritten == 0) + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + + if (!fl_cinfo || fl_cinfo->nwritten == 0) return 0; - else - return NFS_SERVER(inode)->pnfs_curr_ld->scan_commit_lists(cinfo, max); + return fl_cinfo->ops->scan_commit_lists(cinfo, max); +} + +static inline void +pnfs_recover_commit_reqs(struct list_head *head, struct nfs_commit_info *cinfo) +{ + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + + if (fl_cinfo && fl_cinfo->nwritten != 0) + fl_cinfo->ops->recover_commit_reqs(head, cinfo); } static inline struct nfs_page * pnfs_search_commit_reqs(struct inode *inode, struct nfs_commit_info *cinfo, struct page *page) { - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - if (ld == NULL || ld->search_commit_reqs == NULL) + if (!fl_cinfo->ops || !fl_cinfo->ops->search_commit_reqs) return NULL; - return ld->search_commit_reqs(cinfo, page); + return fl_cinfo->ops->search_commit_reqs(cinfo, page); } /* Should the pNFS client commit and return the layout upon a setattr */ @@ -782,6 +802,12 @@ pnfs_get_ds_info(struct inode *inode) return NULL; } +static inline void +pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ + INIT_LIST_HEAD(&fl_cinfo->commits); +} + static inline void pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) { @@ -807,6 +833,11 @@ pnfs_scan_commit_lists(struct inode *inode, struct nfs_commit_info *cinfo, return 0; } +static inline void +pnfs_recover_commit_reqs(struct list_head *head, struct nfs_commit_info *cinfo) +{ +} + static inline struct nfs_page * pnfs_search_commit_reqs(struct inode *inode, struct nfs_commit_info *cinfo, struct page *page) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 20f12f3cbe38..06df2e6663dc 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -149,17 +149,6 @@ pnfs_add_commit_array(struct pnfs_ds_commit_info *fl_cinfo, } EXPORT_SYMBOL_GPL(pnfs_add_commit_array); -static void -pnfs_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, - struct pnfs_layout_segment *lseg) -{ - struct inode *inode = lseg->pls_layout->plh_inode; - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; - - if (ld->setup_ds_info != NULL) - ld->setup_ds_info(fl_cinfo, lseg); -} - static struct pnfs_commit_array * pnfs_lookup_commit_array(struct pnfs_ds_commit_info *fl_cinfo, struct pnfs_layout_segment *lseg) @@ -170,7 +159,7 @@ pnfs_lookup_commit_array(struct pnfs_ds_commit_info *fl_cinfo, array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); if (!array) { rcu_read_unlock(); - pnfs_setup_ds_info(fl_cinfo, lseg); + fl_cinfo->ops->setup_ds_info(fl_cinfo, lseg); rcu_read_lock(); array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); } diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 2903597ec88c..adbbeae9ce5b 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1284,6 +1284,7 @@ struct pnfs_ds_commit_info { struct list_head commits; unsigned int nwritten; unsigned int ncommitting; + const struct pnfs_commit_ops *ops; }; struct nfs41_state_protection { From patchwork Tue Mar 24 23:47:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6898B174A for ; Tue, 24 Mar 2020 23:49:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4B7892072E for ; Tue, 24 Mar 2020 23:49:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093788; bh=k5rZj86dcDGk+1q5xIdpBhWggFIGK8n7JGoeppfbZaI=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=rDNzPWU+C9rZHDo2ZzDNnXtkMwfk7+9rUehXWmvpq66OXoOWNrYuZxymygOos6LtX wb/k8tRVlO1xJVpCTz7IPQS8/FJNKdqqf4DWxzZomgjSvI1FCWJnowNagWIuTQ19e8 FC6ZElso7IwXCS4rzT97Qil6xJ9wWfrxlcajgEVc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727148AbgCXXtq (ORCPT ); Tue, 24 Mar 2020 19:49:46 -0400 Received: from mail.kernel.org ([198.145.29.99]:35446 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727121AbgCXXtq (ORCPT ); Tue, 24 Mar 2020 19:49:46 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6D0FC2072E for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093785; bh=k5rZj86dcDGk+1q5xIdpBhWggFIGK8n7JGoeppfbZaI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=FrubboOuLzCmgm420McS92k2Q6APd+PdNliGKPbTMzJYc2+nCBVqV/1KI42mHoOe0 +k3eKQ3VEFJjKZlTiy09dkvGZagjgvT8z2JzSzyEyV9BDCSyRm6+aLKHwfDWdnJ41R sk4QV26TQ4JI7rzgArh8cstokfbqTFrExmY5q+so= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 17/22] NFS/pNFS: Simplify bucket layout segment reference counting Date: Tue, 24 Mar 2020 19:47:23 -0400 Message-Id: <20200324234728.8997-18-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-17-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> <20200324234728.8997-17-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 39 ++++++++++++++++++++------------------- include/linux/nfs_xdr.h | 3 +-- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 06df2e6663dc..abf16fc98346 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -59,6 +59,17 @@ void pnfs_generic_commit_release(void *calldata) } EXPORT_SYMBOL_GPL(pnfs_generic_commit_release); +static struct pnfs_layout_segment * +pnfs_free_bucket_lseg(struct pnfs_commit_bucket *bucket) +{ + if (list_empty(&bucket->committing) && list_empty(&bucket->written)) { + struct pnfs_layout_segment *freeme = bucket->lseg; + bucket->lseg = NULL; + return freeme; + } + return NULL; +} + /* The generic layer is about to remove the req from the commit list. * If this will make the bucket empty, it will need to put the lseg reference. * Note this must be called holding nfsi->commit_mutex @@ -78,8 +89,7 @@ pnfs_generic_clear_request_commit(struct nfs_page *req, bucket = list_first_entry(&req->wb_list, struct pnfs_commit_bucket, written); - freeme = bucket->wlseg; - bucket->wlseg = NULL; + freeme = pnfs_free_bucket_lseg(bucket); } out: nfs_request_remove_commit_list(req, cinfo); @@ -103,8 +113,7 @@ pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags) for (b = &p->buckets[0]; n != 0; b++, n--) { INIT_LIST_HEAD(&b->written); INIT_LIST_HEAD(&b->committing); - b->wlseg = NULL; - b->clseg = NULL; + b->lseg = NULL; b->direct_verf.committed = NFS_INVALID_STABLE_HOW; } return p; @@ -246,12 +255,6 @@ pnfs_bucket_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, if (ret) { cinfo->ds->nwritten -= ret; cinfo->ds->ncommitting += ret; - if (bucket->clseg == NULL) - bucket->clseg = pnfs_get_lseg(bucket->wlseg); - if (list_empty(src)) { - pnfs_put_lseg(bucket->wlseg); - bucket->wlseg = NULL; - } } return ret; } @@ -317,9 +320,8 @@ pnfs_bucket_recover_commit_reqs(struct list_head *dst, if (!nwritten) continue; ret += nwritten; - if (list_empty(&b->written)) { - freeme = b->wlseg; - b->wlseg = NULL; + freeme = pnfs_free_bucket_lseg(b); + if (freeme) { pnfs_put_lseg(freeme); goto restart; } @@ -405,15 +407,12 @@ pnfs_bucket_get_committing(struct list_head *head, struct pnfs_commit_bucket *bucket, struct nfs_commit_info *cinfo) { - struct pnfs_layout_segment *freeme; struct list_head *pos; list_for_each(pos, &bucket->committing) cinfo->ds->ncommitting--; list_splice_init(&bucket->committing, head); - freeme = bucket->clseg; - bucket->clseg = NULL; - return freeme; + return pnfs_free_bucket_lseg(bucket); } static struct nfs_commit_data * @@ -425,6 +424,8 @@ pnfs_bucket_fetch_commitdata(struct pnfs_commit_bucket *bucket, if (!data) return NULL; data->lseg = pnfs_bucket_get_committing(&data->pages, bucket, cinfo); + if (!data->lseg) + data->lseg = pnfs_get_lseg(bucket->lseg); return data; } @@ -1182,8 +1183,8 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, * off due to a rewrite, in which case it will be done in * pnfs_common_clear_request_commit */ - WARN_ON_ONCE(buckets[ds_commit_idx].wlseg != NULL); - buckets[ds_commit_idx].wlseg = pnfs_get_lseg(lseg); + if (!buckets[ds_commit_idx].lseg) + buckets[ds_commit_idx].lseg = pnfs_get_lseg(lseg); } set_bit(PG_COMMIT_TO_DS, &req->wb_flags); cinfo->ds->nwritten++; diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index adbbeae9ce5b..7bbb1f6fc1b1 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1265,8 +1265,7 @@ struct nfstime4 { struct pnfs_commit_bucket { struct list_head written; struct list_head committing; - struct pnfs_layout_segment *wlseg; - struct pnfs_layout_segment *clseg; + struct pnfs_layout_segment *lseg; struct nfs_writeverf direct_verf; }; From patchwork Tue Mar 24 23:47:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456739 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01A3417D4 for ; Tue, 24 Mar 2020 23:49:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D8EDC2073E for ; Tue, 24 Mar 2020 23:49:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093787; bh=+SFShDxnoX+gVXJT4uPxAaGlDjkIaMd8lZ0ElJY57m0=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=qxYz5TIqal4ZXWo2LSvuyIKbrVipdJfUNBiKzja9R9Qhu3MNvNaZfOwbCh03ujkhp nvAnk/r8sHceGXYZLBC8Ke5nMyJWTD9TUEBBY6Te/RvNMWpiJb3OllzOKPWBAbf1GA JbS4lbfJzWCxZIxCkGSA4i+q4LWo4tfZItPciHg4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727121AbgCXXtr (ORCPT ); Tue, 24 Mar 2020 19:49:47 -0400 Received: from mail.kernel.org ([198.145.29.99]:35462 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727167AbgCXXtq (ORCPT ); Tue, 24 Mar 2020 19:49:46 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E05902073E for ; Tue, 24 Mar 2020 23:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093786; bh=+SFShDxnoX+gVXJT4uPxAaGlDjkIaMd8lZ0ElJY57m0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=OpBHSN/kZ5Bg4wZo6mCV6f9AWK2PoBru3RYU/unOgi0FeH50KabqqE/7ZqikPMAJg e/6phyKn1ggSUBVGN4orlM5Ijv5lGdCae92LCWE8sRmi+aOd5bVWlicLVF5+OBM2Qm m9PXE8HlFSXMIqrY6lpXVzuP3KsopDlN1+hBAbdM= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 18/22] NFS/pNFS: Fix pnfs_layout_mark_request_commit() invalid layout segment handling Date: Tue, 24 Mar 2020 19:47:24 -0400 Message-Id: <20200324234728.8997-19-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-18-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> <20200324234728.8997-17-trondmy@kernel.org> <20200324234728.8997-18-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Fix up pnfs_layout_mark_request_commit() to alway reschedule the write if the layout segment is invalid. Also minor cleanup. Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index abf16fc98346..25f135572fc8 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -1166,26 +1166,22 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, { struct list_head *list; struct pnfs_commit_array *array; - struct pnfs_commit_bucket *buckets; + struct pnfs_commit_bucket *bucket; mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); array = pnfs_lookup_commit_array(cinfo->ds, lseg); - if (!array) + if (!array || !pnfs_is_valid_lseg(lseg)) goto out_resched; - buckets = array->buckets; - list = &buckets[ds_commit_idx].written; - if (list_empty(list)) { - if (!pnfs_is_valid_lseg(lseg)) - goto out_resched; - /* Non-empty buckets hold a reference on the lseg. That ref - * is normally transferred to the COMMIT call and released - * there. It could also be released if the last req is pulled - * off due to a rewrite, in which case it will be done in - * pnfs_common_clear_request_commit - */ - if (!buckets[ds_commit_idx].lseg) - buckets[ds_commit_idx].lseg = pnfs_get_lseg(lseg); - } + bucket = &array->buckets[ds_commit_idx]; + list = &bucket->written; + /* Non-empty buckets hold a reference on the lseg. That ref + * is normally transferred to the COMMIT call and released + * there. It could also be released if the last req is pulled + * off due to a rewrite, in which case it will be done in + * pnfs_common_clear_request_commit + */ + if (!bucket->lseg) + bucket->lseg = pnfs_get_lseg(lseg); set_bit(PG_COMMIT_TO_DS, &req->wb_flags); cinfo->ds->nwritten++; From patchwork Tue Mar 24 23:47:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456743 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC12113A4 for ; Tue, 24 Mar 2020 23:49:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BEE802072E for ; Tue, 24 Mar 2020 23:49:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093788; bh=xt+b6i6Dbj54XKSB1pNam2L2unBJ6u/n1S6L5zmv1Rw=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=b4dm8zu2A7obgtkZqu6aygsQF98146FQHDqAbZLtxT3YC6drGHiOCT7qXT5g39VVT 4+hLpuzg53ggd1PDjpzO2uD27ICMjmDBXmkhLNB9JwcohWn9T8QN0QaQrFpMhzE3LL aCxR51nRWOKUgcn3NSFDtmrcDNrROXnro3c4Q2LI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727163AbgCXXtr (ORCPT ); Tue, 24 Mar 2020 19:49:47 -0400 Received: from mail.kernel.org ([198.145.29.99]:35470 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727176AbgCXXtr (ORCPT ); Tue, 24 Mar 2020 19:49:47 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5E89E2076A for ; Tue, 24 Mar 2020 23:49:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093786; bh=xt+b6i6Dbj54XKSB1pNam2L2unBJ6u/n1S6L5zmv1Rw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=LbdEaGSOAIbmMAW1rFSxMTwDSjD6zH7p7s5eq6AL5BGIzNG7eKH/YH4tffLxGYJ5a HrwqRqFgUQaA0b5oAlYqeLvq50W/pq66dInzQ4ajyKQ8ESrRqC61278IsohP01EqG9 pdj67aVohg193jXsPyxq+882Y3sd5PC3F6QrP7Sw= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 19/22] pNFS/flexfile: Don't merge layout segments if the mirrors don't match Date: Tue, 24 Mar 2020 19:47:25 -0400 Message-Id: <20200324234728.8997-20-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-19-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> <20200324234728.8997-17-trondmy@kernel.org> <20200324234728.8997-18-trondmy@kernel.org> <20200324234728.8997-19-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Check that the number of mirrors, and the mirror information matches before deciding to merge layout segments in pNFS/flexfiles. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index cdd0c87b03dc..3cc2d3165a11 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -283,6 +283,23 @@ static void _ff_layout_free_lseg(struct nfs4_ff_layout_segment *fls) } } +static bool +ff_lseg_match_mirrors(struct pnfs_layout_segment *l1, + struct pnfs_layout_segment *l2) +{ + const struct nfs4_ff_layout_segment *fl1 = FF_LAYOUT_LSEG(l1); + const struct nfs4_ff_layout_segment *fl2 = FF_LAYOUT_LSEG(l1); + u32 i; + + if (fl1->mirror_array_cnt != fl2->mirror_array_cnt) + return false; + for (i = 0; i < fl1->mirror_array_cnt; i++) { + if (fl1->mirror_array[i] != fl2->mirror_array[i]) + return false; + } + return true; +} + static bool ff_lseg_range_is_after(const struct pnfs_layout_range *l1, const struct pnfs_layout_range *l2) @@ -318,6 +335,8 @@ ff_lseg_merge(struct pnfs_layout_segment *new, new->pls_range.length); if (new_end < old->pls_range.offset) return false; + if (!ff_lseg_match_mirrors(new, old)) + return false; /* Mergeable: copy info from 'old' to 'new' */ if (new_end < old_end) From patchwork Tue Mar 24 23:47:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456745 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D396174A for ; Tue, 24 Mar 2020 23:49:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 305682072E for ; Tue, 24 Mar 2020 23:49:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093789; bh=gtNIgmXlu2AIGfLM52JvOCgTcrhxeBVuK7HZPWsafRc=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=bWimzsug+aGHS7gUcqws3gE2+G1qnYD2XO7cL1OSWhTdmGcjCrqJwEr5AwwIL+6fq VhMBCDusI/5OB7TV68s1fe2x6gJm6s/J/BTxTZEOmpKLb9GmeEPDL6PQ/C1SW/Q2CJ OFlW7msR4kQUQWlvYebGbzfWan0Nc+CTkv7KiQR0= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727176AbgCXXts (ORCPT ); Tue, 24 Mar 2020 19:49:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:35478 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727179AbgCXXtr (ORCPT ); Tue, 24 Mar 2020 19:49:47 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C70462072E for ; Tue, 24 Mar 2020 23:49:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093787; bh=gtNIgmXlu2AIGfLM52JvOCgTcrhxeBVuK7HZPWsafRc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=HbAFa1a/1coXfrp84cOAnRH5wH8H6JiF0E9dp8YWVUVUgjNI1rIht3HHGVvwJAD+7 1N94piLOLwQH+f9P7VU11lZRh8tIAmO2QcMwfAvjySLmiih6MD8rrVd/X92pU2uhfu 4cgJO2QP19AigwY5viRg/HpGND6VS31L/N7kkAk4= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 20/22] pNFS/flexfiles: Check the layout segment range before doing I/O Date: Tue, 24 Mar 2020 19:47:26 -0400 Message-Id: <20200324234728.8997-21-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-20-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> <20200324234728.8997-17-trondmy@kernel.org> <20200324234728.8997-18-trondmy@kernel.org> <20200324234728.8997-19-trondmy@kernel.org> <20200324234728.8997-20-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust When starting to read or write with a layout segment, check that the range matches our request. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 12 ++++++++++-- fs/nfs/pnfs.c | 3 ++- fs/nfs/pnfs.h | 1 + 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 3cc2d3165a11..a816e304e876 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -830,6 +830,14 @@ ff_layout_pg_get_read(struct nfs_pageio_descriptor *pgio, } } +static void +ff_layout_pg_check_layout(struct nfs_pageio_descriptor *pgio, + struct nfs_page *req) +{ + pnfs_generic_pg_check_layout(pgio); + pnfs_generic_pg_check_range(pgio, req); +} + static void ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) @@ -840,7 +848,7 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, int ds_idx; retry: - pnfs_generic_pg_check_layout(pgio); + ff_layout_pg_check_layout(pgio, req); /* Use full layout for now */ if (!pgio->pg_lseg) { ff_layout_pg_get_read(pgio, req, false); @@ -900,7 +908,7 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, int i; retry: - pnfs_generic_pg_check_layout(pgio); + ff_layout_pg_check_layout(pgio, req); if (!pgio->pg_lseg) { pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req), diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index eba18f137fb0..6fcf26b16816 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2611,7 +2611,7 @@ EXPORT_SYMBOL_GPL(pnfs_generic_pg_check_layout); * Check for any intersection between the request and the pgio->pg_lseg, * and if none, put this pgio->pg_lseg away. */ -static void +void pnfs_generic_pg_check_range(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) { if (pgio->pg_lseg && !pnfs_lseg_request_intersecting(pgio->pg_lseg, req)) { @@ -2619,6 +2619,7 @@ pnfs_generic_pg_check_range(struct nfs_pageio_descriptor *pgio, struct nfs_page pgio->pg_lseg = NULL; } } +EXPORT_SYMBOL_GPL(pnfs_generic_pg_check_range); void pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 52a5d8aaaf46..ff2a0c53e05f 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -252,6 +252,7 @@ void pnfs_put_lseg(struct pnfs_layout_segment *lseg); void set_pnfs_layoutdriver(struct nfs_server *, const struct nfs_fh *, struct nfs_fsinfo *); void unset_pnfs_layoutdriver(struct nfs_server *); void pnfs_generic_pg_check_layout(struct nfs_pageio_descriptor *pgio); +void pnfs_generic_pg_check_range(struct nfs_pageio_descriptor *pgio, struct nfs_page *req); void pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *, struct nfs_page *); int pnfs_generic_pg_readpages(struct nfs_pageio_descriptor *desc); void pnfs_generic_pg_init_write(struct nfs_pageio_descriptor *pgio, From patchwork Tue Mar 24 23:47:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FCF713A4 for ; Tue, 24 Mar 2020 23:49:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 82FA720719 for ; Tue, 24 Mar 2020 23:49:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093790; bh=+JUSw23D99QxkWO2AZ4kTb67ZenkEhA5XM7/9tnCej4=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=IAeoYA0frY5tE0GABYUvaz9UFEZIxmi2WaExPBXs1xNrXKShd46U6iDifElCydu8h zCKl1v+f3npfg04MLYK1E3hK+s/ZN4woN88Ak2G0pvde9IN0ek62rsimcVC1UZtfE5 LsTXVcWsavRk60xroacaXDSy97lm7elLHG6vlhIM= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727179AbgCXXts (ORCPT ); Tue, 24 Mar 2020 19:49:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:35470 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727183AbgCXXts (ORCPT ); Tue, 24 Mar 2020 19:49:48 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 40A3F20719 for ; Tue, 24 Mar 2020 23:49:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093787; bh=+JUSw23D99QxkWO2AZ4kTb67ZenkEhA5XM7/9tnCej4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=0FRLWQudMHiFefssvIl9lJlZgZVLay0buRs9jv9hK1lD8sJYj+cF3Hz8b6jHqXHgz Nz/Uq89PDsjIhEzMMtziEglaAJrPgTNovbjL9rZINHymgasHjOTWGi3VE7zCLYlim2 5wdujgh/ECVtWMmTay+xBfyMQ9G+IOYaPKgczFFU= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 21/22] pNFS/flexfiles: remove requirement for whole file layouts Date: Tue, 24 Mar 2020 19:47:27 -0400 Message-Id: <20200324234728.8997-22-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-21-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> <20200324234728.8997-17-trondmy@kernel.org> <20200324234728.8997-18-trondmy@kernel.org> <20200324234728.8997-19-trondmy@kernel.org> <20200324234728.8997-20-trondmy@kernel.org> <20200324234728.8997-21-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Remove the requirement that the server always sends whole file layouts. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index a816e304e876..8ad26cf786de 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -257,24 +257,6 @@ static void ff_layout_free_mirror_array(struct nfs4_ff_layout_segment *fls) ff_layout_put_mirror(fls->mirror_array[i]); } -static int ff_layout_check_layout(struct nfs4_layoutget_res *lgr) -{ - int ret = 0; - - dprintk("--> %s\n", __func__); - - /* FIXME: remove this check when layout segment support is added */ - if (lgr->range.offset != 0 || - lgr->range.length != NFS4_MAX_UINT64) { - dprintk("%s Only whole file layouts supported. Use MDS i/o\n", - __func__); - ret = -EINVAL; - } - - dprintk("--> %s returns %d\n", __func__, ret); - return ret; -} - static void _ff_layout_free_lseg(struct nfs4_ff_layout_segment *fls) { if (fls) { @@ -556,9 +538,6 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, out_sort_mirrors: ff_layout_sort_mirrors(fls); - rc = ff_layout_check_layout(lgr); - if (rc) - goto out_err_free; ret = &fls->generic_hdr; dprintk("<-- %s (success)\n", __func__); out_free_page: From patchwork Tue Mar 24 23:47:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11456751 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC74913A4 for ; Tue, 24 Mar 2020 23:49:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AE6AF2072E for ; Tue, 24 Mar 2020 23:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093797; bh=ar0wkMwCe6ZGIRTs9sssTCqmlezrmEtiKGCGz+0IHWM=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=LpTMRG7TtfEE2JaT+TsmfIrMR6PznG7YovE6r4RUNijRLDaUyIu/qgO8FdHSbv/aJ 8DdTESAhxX57YThokcrsgcall7uDjc36ge0yXAzc75DheEbsS2mH1LpoyrV9s8bKRF 3KqegUsk6OsoUaZaZzmqeJpvwTaZnV9pzr3Z0bks= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727183AbgCXXtt (ORCPT ); Tue, 24 Mar 2020 19:49:49 -0400 Received: from mail.kernel.org ([198.145.29.99]:35478 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727188AbgCXXts (ORCPT ); Tue, 24 Mar 2020 19:49:48 -0400 Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net [68.40.189.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AD6462073C for ; Tue, 24 Mar 2020 23:49:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585093787; bh=ar0wkMwCe6ZGIRTs9sssTCqmlezrmEtiKGCGz+0IHWM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=uIb37uBCmB9bWUx6rR1njArMLp0EfY37Ifa2x81z6M6MsjEcD5Sqx3+4a3R32M6S8 H6+xJVuE/4xQl7qZrL+N+DGwBsiQ0C4CdIzoXHLRJ01U0qc6v8sdiOUiVeCeqwr5Ac B1EXC/b8cmUosmoigZMs4/xthoJOuD8xe/Ac9NK0= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 22/22] pNFS/flexfiles: Specify the layout segment range in LAYOUTGET Date: Tue, 24 Mar 2020 19:47:28 -0400 Message-Id: <20200324234728.8997-23-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200324234728.8997-22-trondmy@kernel.org> References: <20200324234728.8997-1-trondmy@kernel.org> <20200324234728.8997-2-trondmy@kernel.org> <20200324234728.8997-3-trondmy@kernel.org> <20200324234728.8997-4-trondmy@kernel.org> <20200324234728.8997-5-trondmy@kernel.org> <20200324234728.8997-6-trondmy@kernel.org> <20200324234728.8997-7-trondmy@kernel.org> <20200324234728.8997-8-trondmy@kernel.org> <20200324234728.8997-9-trondmy@kernel.org> <20200324234728.8997-10-trondmy@kernel.org> <20200324234728.8997-11-trondmy@kernel.org> <20200324234728.8997-12-trondmy@kernel.org> <20200324234728.8997-13-trondmy@kernel.org> <20200324234728.8997-14-trondmy@kernel.org> <20200324234728.8997-15-trondmy@kernel.org> <20200324234728.8997-16-trondmy@kernel.org> <20200324234728.8997-17-trondmy@kernel.org> <20200324234728.8997-18-trondmy@kernel.org> <20200324234728.8997-19-trondmy@kernel.org> <20200324234728.8997-20-trondmy@kernel.org> <20200324234728.8997-21-trondmy@kernel.org> <20200324234728.8997-22-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Move from requesting only full file layout segments, to requesting layout segments that match our I/O size. This means the server is still free to return a full file layout, but we will no longer error out if it does not. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 8ad26cf786de..b2a7c711e911 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -798,8 +798,8 @@ ff_layout_pg_get_read(struct nfs_pageio_descriptor *pgio, pnfs_put_lseg(pgio->pg_lseg); pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req), - 0, - NFS4_MAX_UINT64, + req_offset(req), + req->wb_bytes, IOMODE_READ, strict_iomode, GFP_KERNEL); @@ -891,8 +891,8 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, if (!pgio->pg_lseg) { pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req), - 0, - NFS4_MAX_UINT64, + req_offset(req), + req->wb_bytes, IOMODE_RW, false, GFP_NOFS);