From patchwork Wed Sep 16 16:49:19 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Tao X-Patchwork-Id: 7192671 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C3B96BEEC1 for ; Wed, 16 Sep 2015 08:50:33 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 9AEB220852 for ; Wed, 16 Sep 2015 08:50:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6E47320851 for ; Wed, 16 Sep 2015 08:50:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752007AbbIPIu3 (ORCPT ); Wed, 16 Sep 2015 04:50:29 -0400 Received: from mail-pa0-f53.google.com ([209.85.220.53]:36859 "EHLO mail-pa0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751985AbbIPIu1 (ORCPT ); Wed, 16 Sep 2015 04:50:27 -0400 Received: by padhk3 with SMTP id hk3so203262455pad.3 for ; Wed, 16 Sep 2015 01:50:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=8LbTqOabCk1p1FgJjJy6LWy+2rhxCt8IBojhsbfooSg=; b=LF5wP+wr7422bkZZ0Sl5KN9uRq0Hm5lnX5cOlEwPLAdWiB/cZ3DJ6HYrzyTdTMsy1O 11G+pwnRm6lWcayIUXJwZshMv7gX125vPyw7Y9VmrsmwXc0pIZJgN/oojlY0ucoOw2Th NCgMtQu4itS7Mi+toIsRnbt74pexpqk1MAZRn25yf9/IJb8klyMtA957rRntJG6HiuJT N9N+8eN4vTuKUg4cLOEu1waQihMTa5847p+Ty/mJds3URWf7ffpgEookC2xwQp7u2SMo PxaOAGGZcNdW+wVZgDTNLVvL1usfUzO+T7nORaiwuMpArGXvODwoIWtmjAQE6Y27963E aqVQ== X-Gm-Message-State: ALoCoQkK3gktzK9UDZlWzUNq/P/fghCsT/oFueU4gQ59zw0ZPo45p6WBw9PGMaCeSZAaalGmcTk/ X-Received: by 10.68.194.73 with SMTP id hu9mr58867860pbc.146.1442393426446; Wed, 16 Sep 2015 01:50:26 -0700 (PDT) Received: from lear.localdomain (ec2-54-65-164-9.ap-northeast-1.compute.amazonaws.com. [54.65.164.9]) by smtp.gmail.com with ESMTPSA id ox2sm26418934pbb.87.2015.09.16.01.50.22 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2015 01:50:25 -0700 (PDT) From: Peng Tao To: linux-nfs@vger.kernel.org Cc: Trond Myklebust , Peng Tao Subject: [PATCH] NFS41: make close wait for layoutreturn Date: Thu, 17 Sep 2015 00:49:19 +0800 Message-Id: <1442422159-9621-1-git-send-email-tao.peng@primarydata.com> X-Mailer: git-send-email 1.8.3.1 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-5.0 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_06_12, RCVD_IN_DNSWL_HI,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If we send a layoutreturn asynchronously before close, the close might reach server first and layoutreturn would fail with BADSTATEID because there is nothing keeping the layout stateid alive. Also do not pretend sending layoutreturn if we are not. Signed-off-by: Peng Tao --- fs/nfs/nfs4proc.c | 17 +++++++++++++++++ fs/nfs/pnfs.c | 32 ++++++++++++++++++++++---------- fs/nfs/pnfs.h | 7 +++++++ 3 files changed, 46 insertions(+), 10 deletions(-) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 693b903..05f2da4 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -2645,6 +2645,15 @@ out: return err; } +static bool +nfs4_wait_on_layoutreturn(struct inode *inode, struct rpc_task *task) +{ + if (!nfs_have_layout(inode)) + return false; + + return pnfs_wait_on_layoutreturn(inode, task); +} + struct nfs4_closedata { struct inode *inode; struct nfs4_state *state; @@ -2763,6 +2772,11 @@ static void nfs4_close_prepare(struct rpc_task *task, void *data) goto out_no_action; } + if (nfs4_wait_on_layoutreturn(inode, task)) { + nfs_release_seqid(calldata->arg.seqid); + goto out_wait; + } + if (calldata->arg.fmode == 0) task->tk_msg.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_CLOSE]; if (calldata->roc) @@ -5308,6 +5322,9 @@ static void nfs4_delegreturn_prepare(struct rpc_task *task, void *data) d_data = (struct nfs4_delegreturndata *)data; + if (nfs4_wait_on_layoutreturn(d_data->inode, task)) + return; + if (d_data->roc) pnfs_roc_get_barrier(d_data->inode, &d_data->roc_barrier); diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index ba12464..84e87bf 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -1104,19 +1104,11 @@ bool pnfs_roc(struct inode *ino) mark_lseg_invalid(lseg, &tmp_list); found = true; } - /* pnfs_prepare_layoutreturn() grabs lo ref and it will be put - * in pnfs_roc_release(). We don't really send a layoutreturn but - * still want others to view us like we are sending one! - * - * If pnfs_prepare_layoutreturn() fails, it means someone else is doing - * LAYOUTRETURN, so we proceed like there are no layouts to return. - * - * ROC in three conditions: + /* ROC in two conditions: * 1. there are ROC lsegs * 2. we don't send layoutreturn - * 3. no others are sending layoutreturn */ - if (found && !layoutreturn && pnfs_prepare_layoutreturn(lo)) + if (found && !layoutreturn) roc = true; out_noroc: @@ -1172,6 +1164,26 @@ void pnfs_roc_get_barrier(struct inode *ino, u32 *barrier) spin_unlock(&ino->i_lock); } +bool pnfs_wait_on_layoutreturn(struct inode *ino, struct rpc_task *task) +{ + struct nfs_inode *nfsi = NFS_I(ino); + struct pnfs_layout_hdr *lo; + bool sleep = false; + + /* we might not have grabbed lo reference. so need to check under + * i_lock */ + spin_lock(&ino->i_lock); + lo = nfsi->layout; + if (lo && test_bit(NFS_LAYOUT_RETURN, &lo->plh_flags)) + sleep = true; + spin_unlock(&ino->i_lock); + + if (sleep) + rpc_sleep_on(&NFS_SERVER(ino)->roc_rpcwaitq, task, NULL); + + return sleep; +} + /* * Compare two layout segments for sorting into layout cache. * We want to preferentially return RW over RO layouts, so ensure those diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 78c9351..d1990e9 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -270,6 +270,7 @@ bool pnfs_roc(struct inode *ino); void pnfs_roc_release(struct inode *ino); void pnfs_roc_set_barrier(struct inode *ino, u32 barrier); void pnfs_roc_get_barrier(struct inode *ino, u32 *barrier); +bool pnfs_wait_on_layoutreturn(struct inode *ino, struct rpc_task *task); void pnfs_set_layoutcommit(struct inode *, struct pnfs_layout_segment *, loff_t); void pnfs_cleanup_layoutcommit(struct nfs4_layoutcommit_data *data); int pnfs_layoutcommit_inode(struct inode *inode, bool sync); @@ -639,6 +640,12 @@ pnfs_roc_get_barrier(struct inode *ino, u32 *barrier) { } +static inline bool +pnfs_wait_on_layoutreturn(struct inode *ino, struct rpc_task *task) +{ + return false; +} + static inline void set_pnfs_layoutdriver(struct nfs_server *s, const struct nfs_fh *mntfh, u32 id) {