From patchwork Wed Aug 23 21:33:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363167 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5FD3200A5 for ; Wed, 23 Aug 2023 21:34:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o7rtjBWasTboMR+96WnOdhlx9MGDmW1KFfPhMK83oC8=; b=C+L+GwVgZ4+7ugRKRKbNpZt0jr3kb4Xaf0hAzWkq5AAhaA/1HqMgNkgnW+eZ38MTkWiQtU NteqscfJYDjksKz2Fg/HWzm6sMtSjH/DEJtzrmIGKbgTtMcVD6OMY3VK+fE8FouXLn4zzN wI0ILlg6iPml2z5D4QhxBZXkuvEW6Yg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-290-o-UhRl3lMMqwYDy5EHcM4w-1; Wed, 23 Aug 2023 17:33:54 -0400 X-MC-Unique: o-UhRl3lMMqwYDy5EHcM4w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 119CC856DED; Wed, 23 Aug 2023 21:33:54 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9DCE400E427; Wed, 23 Aug 2023 21:33:53 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 1/7] lockd: introduce safe async lock op Date: Wed, 23 Aug 2023 17:33:46 -0400 Message-Id: <20230823213352.1971009-2-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch reverts mostly commit 40595cdc93ed ("nfs: block notification on fs with its own ->lock") and introduces an EXPORT_OP_SAFE_ASYNC_LOCK export flag to signal that the "own ->lock" implementation supports async lock requests. The only main user is DLM that is used by GFS2 and OCFS2 filesystem. Those implement their own lock() implementation and return FILE_LOCK_DEFERRED as return value. Since commit 40595cdc93ed ("nfs: block notification on fs with its own ->lock") the DLM implementation were never updated. This patch should prepare for DLM to set the EXPORT_OP_SAFE_ASYNC_LOCK export flag and update the DLM plock implementation regarding to it. Acked-by: Jeff Layton Signed-off-by: Alexander Aring --- fs/lockd/svclock.c | 5 ++--- fs/nfsd/nfs4state.c | 13 ++++++++++--- include/linux/exportfs.h | 8 ++++++++ 3 files changed, 20 insertions(+), 6 deletions(-) diff --git a/fs/lockd/svclock.c b/fs/lockd/svclock.c index c43ccdf28ed9..6e3b230e8317 100644 --- a/fs/lockd/svclock.c +++ b/fs/lockd/svclock.c @@ -470,9 +470,7 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, struct nlm_host *host, struct nlm_lock *lock, int wait, struct nlm_cookie *cookie, int reclaim) { -#if IS_ENABLED(CONFIG_SUNRPC_DEBUG) struct inode *inode = nlmsvc_file_inode(file); -#endif struct nlm_block *block = NULL; int error; int mode; @@ -486,7 +484,8 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, (long long)lock->fl.fl_end, wait); - if (nlmsvc_file_file(file)->f_op->lock) { + if (!export_op_support_safe_async_lock(inode->i_sb->s_export_op, + nlmsvc_file_file(file)->f_op)) { async_block = wait; wait = 0; } diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 3aefbad4cc09..14ca06424ff1 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -7430,6 +7430,7 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, struct nfsd4_blocked_lock *nbl = NULL; struct file_lock *file_lock = NULL; struct file_lock *conflock = NULL; + struct super_block *sb; __be32 status = 0; int lkflg; int err; @@ -7451,6 +7452,7 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, dprintk("NFSD: nfsd4_lock: permission denied!\n"); return status; } + sb = cstate->current_fh.fh_dentry->d_sb; if (lock->lk_is_new) { if (nfsd4_has_session(cstate)) @@ -7502,7 +7504,9 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, fp = lock_stp->st_stid.sc_file; switch (lock->lk_type) { case NFS4_READW_LT: - if (nfsd4_has_session(cstate)) + if (nfsd4_has_session(cstate) || + export_op_support_safe_async_lock(sb->s_export_op, + nf->nf_file->f_op)) fl_flags |= FL_SLEEP; fallthrough; case NFS4_READ_LT: @@ -7514,7 +7518,9 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, fl_type = F_RDLCK; break; case NFS4_WRITEW_LT: - if (nfsd4_has_session(cstate)) + if (nfsd4_has_session(cstate) || + export_op_support_safe_async_lock(sb->s_export_op, + nf->nf_file->f_op)) fl_flags |= FL_SLEEP; fallthrough; case NFS4_WRITE_LT: @@ -7542,7 +7548,8 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, * for file locks), so don't attempt blocking lock notifications * on those filesystems: */ - if (nf->nf_file->f_op->lock) + if (!export_op_support_safe_async_lock(sb->s_export_op, + nf->nf_file->f_op)) fl_flags &= ~FL_SLEEP; nbl = find_or_allocate_block(lock_sop, &fp->fi_fhandle, nn); diff --git a/include/linux/exportfs.h b/include/linux/exportfs.h index 11fbd0ee1370..10358a93cdc1 100644 --- a/include/linux/exportfs.h +++ b/include/linux/exportfs.h @@ -3,6 +3,7 @@ #define LINUX_EXPORTFS_H 1 #include +#include struct dentry; struct iattr; @@ -224,9 +225,16 @@ struct export_operations { atomic attribute updates */ #define EXPORT_OP_FLUSH_ON_CLOSE (0x20) /* fs flushes file data on close */ +#define EXPORT_OP_SAFE_ASYNC_LOCK (0x40) /* fs can do async lock request */ unsigned long flags; }; +static inline bool export_op_support_safe_async_lock(const struct export_operations *export_ops, + const struct file_operations *f_op) +{ + return (export_ops->flags & EXPORT_OP_SAFE_ASYNC_LOCK) || !f_op->lock; +} + extern int exportfs_encode_inode_fh(struct inode *inode, struct fid *fid, int *max_len, struct inode *parent, int flags); From patchwork Wed Aug 23 21:33:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363165 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 917DB200A6 for ; Wed, 23 Aug 2023 21:33:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=htB9ruBs/M3gzRN+MQhu18/VcxVtJfis4P4SxexSrGE=; b=CdMtoViBXQGxJGyivZTarQK5xwiVcTWZqt4j4dmUwIx8c8Tvb1KMc7BZo1IJRgSpnnFftO 8s472aUuoj2Co2Azqyj1UU1fKwY6x6NYS92gnlKLviiQVfB7OZ6UKe1rpL2r2yKYTW2lY/ x7tZwqT5u6Cid5nqh1dmukkS+xN0gqU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-454-j9OrqPG9M-uKAhTJKw2f4g-1; Wed, 23 Aug 2023 17:33:54 -0400 X-MC-Unique: j9OrqPG9M-uKAhTJKw2f4g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 60E94185A78B; Wed, 23 Aug 2023 21:33:54 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1971640C6F4C; Wed, 23 Aug 2023 21:33:54 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 2/7] lockd: don't call vfs_lock_file() for pending requests Date: Wed, 23 Aug 2023 17:33:47 -0400 Message-Id: <20230823213352.1971009-3-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch returns nlm_lck_blocked in nlmsvc_lock() when an asynchronous lock request is pending. During testing I ran into the case with the side-effects that lockd is waiting for only one lm_grant() callback because it's already part of the nlm_blocked list. If another asynchronous for the same nlm_block is triggered two lm_grant() callbacks will occur but lockd was only waiting for one. To avoid any change of existing users this handling will only being made when export_op_support_safe_async_lock() returns true. Signed-off-by: Alexander Aring Reviewed-by: Jeff Layton --- fs/lockd/svclock.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/fs/lockd/svclock.c b/fs/lockd/svclock.c index 6e3b230e8317..aa4174fbaf5b 100644 --- a/fs/lockd/svclock.c +++ b/fs/lockd/svclock.c @@ -531,6 +531,23 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, goto out; } + spin_lock(&nlm_blocked_lock); + /* + * If this is a lock request for an already pending + * lock request we return nlm_lck_blocked without calling + * vfs_lock_file() again. Otherwise we have two pending + * requests on the underlaying ->lock() implementation but + * only one nlm_block to being granted by lm_grant(). + */ + if (export_op_support_safe_async_lock(inode->i_sb->s_export_op, + nlmsvc_file_file(file)->f_op) && + !list_empty(&block->b_list)) { + spin_unlock(&nlm_blocked_lock); + ret = nlm_lck_blocked; + goto out; + } + spin_unlock(&nlm_blocked_lock); + if (!wait) lock->fl.fl_flags &= ~FL_SLEEP; mode = lock_to_openmode(&lock->fl); @@ -543,13 +560,6 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, ret = nlm_granted; goto out; case -EAGAIN: - /* - * If this is a blocking request for an - * already pending lock request then we need - * to put it back on lockd's block list - */ - if (wait) - break; ret = async_block ? nlm_lck_blocked : nlm_lck_denied; goto out; case FILE_LOCK_DEFERRED: From patchwork Wed Aug 23 21:33:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363163 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 800D8200A5 for ; Wed, 23 Aug 2023 21:33:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pJjn0CQwyqlhlZhAb4qX8Pfq4Wy5CGCCkFzUT+vOXgU=; b=UYSt0ayfY1cgUX8sSHmR2OX+VLmtkWVks1Og3/VzW/GYt7mNzlrU/0mrr28PPnqyI/IBFz +HWrMR7FyK+Blcs7FqLsAhQvoTe4uCUCZtzxRhk5SdkuR6KTD/KFQCiDkod0IJVdRvrMqG rN4Nsp+7zehgu81b/2j9Eb54AEC+OYA= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-271-DqDih_iXOruReN3ZKC48iQ-1; Wed, 23 Aug 2023 17:33:55 -0400 X-MC-Unique: DqDih_iXOruReN3ZKC48iQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B28311C05ED0; Wed, 23 Aug 2023 21:33:54 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6915D400E02F; Wed, 23 Aug 2023 21:33:54 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 3/7] lockd: fix race in async lock request handling Date: Wed, 23 Aug 2023 17:33:48 -0400 Message-Id: <20230823213352.1971009-4-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch fixes a race in async lock request handling between adding the relevant struct nlm_block to nlm_blocked list after the request was sent by vfs_lock_file() and nlmsvc_grant_deferred() does a lookup of the nlm_block in the nlm_blocked list. It could be that the async request is completed before the nlm_block was added to the list. This would end in a -ENOENT and a kernel log message of "lockd: grant for unknown block". To solve this issue we add the nlm_block before the vfs_lock_file() call to be sure it has been added when a possible nlmsvc_grant_deferred() is called. If the vfs_lock_file() results in an case when it wouldn't be added to nlm_blocked list, the nlm_block struct will be removed from this list again. The introducing of the new B_PENDING_CALLBACK nlm_block flag will handle async lock requests on a pending lock requests as a retry on the caller level to hit the -EAGAIN case. Signed-off-by: Alexander Aring Reviewed-by: Jeff Layton --- fs/lockd/svclock.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fs/lockd/svclock.c b/fs/lockd/svclock.c index aa4174fbaf5b..3b158446203b 100644 --- a/fs/lockd/svclock.c +++ b/fs/lockd/svclock.c @@ -546,6 +546,9 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, ret = nlm_lck_blocked; goto out; } + + /* Append to list of blocked */ + nlmsvc_insert_block_locked(block, NLM_NEVER); spin_unlock(&nlm_blocked_lock); if (!wait) @@ -557,9 +560,12 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, dprintk("lockd: vfs_lock_file returned %d\n", error); switch (error) { case 0: + nlmsvc_remove_block(block); ret = nlm_granted; goto out; case -EAGAIN: + if (!wait) + nlmsvc_remove_block(block); ret = async_block ? nlm_lck_blocked : nlm_lck_denied; goto out; case FILE_LOCK_DEFERRED: @@ -570,17 +576,16 @@ nlmsvc_lock(struct svc_rqst *rqstp, struct nlm_file *file, ret = nlmsvc_defer_lock_rqst(rqstp, block); goto out; case -EDEADLK: + nlmsvc_remove_block(block); ret = nlm_deadlock; goto out; default: /* includes ENOLCK */ + nlmsvc_remove_block(block); ret = nlm_lck_denied_nolocks; goto out; } ret = nlm_lck_blocked; - - /* Append to list of blocked */ - nlmsvc_insert_block(block, NLM_NEVER); out: mutex_unlock(&file->f_mutex); nlmsvc_release_block(block); From patchwork Wed Aug 23 21:33:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363162 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EBB6200A4 for ; Wed, 23 Aug 2023 21:33:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a6joEGKWnnjVY5UbKUjhUBMyK3LpAKMDMx1lRCkXV9w=; b=R0RGaXjlFGN0xZ9u1wbaDeJBkdRh/FYhHC2BtE0IZR34OuwVI9TWGqFakssSf+oXAipDX1 OZRiKjX4s9fQhrRv0wl7K7hYpc7tkZDhBDwhXKSis4pKjSmb9vuvY1KNhCeiIzL5oepq8g 8I3R9x+Zq/scHuUjGKOqMSyMinuHaIc= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-502-LFaibqAEPq-WPmG8rfnF6Q-1; Wed, 23 Aug 2023 17:33:56 -0400 X-MC-Unique: LFaibqAEPq-WPmG8rfnF6Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0E3D11C05122; Wed, 23 Aug 2023 21:33:55 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA62040C6F4C; Wed, 23 Aug 2023 21:33:54 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 4/7] lockd: add doc to enable EXPORT_OP_SAFE_ASYNC_LOCK Date: Wed, 23 Aug 2023 17:33:49 -0400 Message-Id: <20230823213352.1971009-5-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch adds a note to enable EXPORT_OP_SAFE_ASYNC_LOCK for asynchronous lock request handling. Signed-off-by: Alexander Aring Reviewed-by: Jeff Layton --- fs/locks.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/fs/locks.c b/fs/locks.c index df8b26a42524..edee02d1ca93 100644 --- a/fs/locks.c +++ b/fs/locks.c @@ -2255,11 +2255,13 @@ int fcntl_getlk(struct file *filp, unsigned int cmd, struct flock *flock) * To avoid blocking kernel daemons, such as lockd, that need to acquire POSIX * locks, the ->lock() interface may return asynchronously, before the lock has * been granted or denied by the underlying filesystem, if (and only if) - * lm_grant is set. Callers expecting ->lock() to return asynchronously - * will only use F_SETLK, not F_SETLKW; they will set FL_SLEEP if (and only if) - * the request is for a blocking lock. When ->lock() does return asynchronously, - * it must return FILE_LOCK_DEFERRED, and call ->lm_grant() when the lock - * request completes. + * lm_grant is set. Additionally EXPORT_OP_SAFE_ASYNC_LOCK in export_operations + * flags need to be set. + * + * Callers expecting ->lock() to return asynchronously will only use F_SETLK, + * not F_SETLKW; they will set FL_SLEEP if (and only if) the request is for a + * blocking lock. When ->lock() does return asynchronously, it must return + * FILE_LOCK_DEFERRED, and call ->lm_grant() when the lock request completes. * If the request is for non-blocking lock the file system should return * FILE_LOCK_DEFERRED then try to get the lock and call the callback routine * with the result. If the request timed out the callback routine will return a From patchwork Wed Aug 23 21:33:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363164 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A3C81FB35 for ; Wed, 23 Aug 2023 21:33:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fr4SUxX9SlUeT5S24Dcxh6kDffrcH+Vs48c2xCXZptc=; b=K1fqA/+rfpHhk5LS92wtBGdvV7lUe9EUmHX4oCxN9ipyera9n5IJfIRBbbFrI1mI4IrMHy uI8uIH4uAN3dVC49wpZlygVlhWlTsCB65HEiNAnn2HuNsMjcQ/zmGnTiYd+IErzI2ElJCS T6wzQjnQmHYrm7fvncx24BtRBGF13n8= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-311-Bd61JCYyOD-XDkO5x-v2EA-1; Wed, 23 Aug 2023 17:33:56 -0400 X-MC-Unique: Bd61JCYyOD-XDkO5x-v2EA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 772B73C11A0F; Wed, 23 Aug 2023 21:33:55 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 16360400E02F; Wed, 23 Aug 2023 21:33:55 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 5/7] dlm: use fl_owner from lockd Date: Wed, 23 Aug 2023 17:33:50 -0400 Message-Id: <20230823213352.1971009-6-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch is changing the fl_owner value in case of an nfs lock request to not be the pid of lockd. Instead this patch changes it to be the owner value that nfs is giving us. Currently there exists proved problems with this behaviour. One nfsd server was created to export a gfs2 filesystem mount. Two nfs clients doing a nfs mount of this export. Those two clients should conflict each other operating on the same nfs file. A small test program was written: int main(int argc, const char *argv[]) { struct flock fl = { .l_type = F_WRLCK, .l_whence = SEEK_SET, .l_start = 1L, .l_len = 1L, }; int fd; fd = open("filename", O_RDWR | O_CREAT, 0700); printf("try to lock...\n"); fcntl(fd, F_SETLKW, &fl); printf("locked!\n"); getc(stdin); return 0; } Running on both clients at the same time and don't interrupting by pressing any key. It will show that both clients are able to acquire the lock which shouldn't be the case. The issue is here that the fl_owner value is the same and the lock context of both clients should be separated. This patch lets lockd define how to deal with lock contexts and chose hopefully the right fl_owner value. A test after this patch was made and the locks conflicts each other which should be the case. Acked-by: Jeff Layton Signed-off-by: Alexander Aring --- fs/dlm/plock.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/fs/dlm/plock.c b/fs/dlm/plock.c index 00e1d802a81c..0094fa4004cc 100644 --- a/fs/dlm/plock.c +++ b/fs/dlm/plock.c @@ -145,6 +145,7 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file, op->info.number = number; op->info.start = fl->fl_start; op->info.end = fl->fl_end; + op->info.owner = (__u64)(long)fl->fl_owner; /* async handling */ if (fl->fl_lmops && fl->fl_lmops->lm_grant) { op_data = kzalloc(sizeof(*op_data), GFP_NOFS); @@ -154,9 +155,6 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file, goto out; } - /* fl_owner is lockd which doesn't distinguish - processes on the nfs client */ - op->info.owner = (__u64) fl->fl_pid; op_data->callback = fl->fl_lmops->lm_grant; locks_init_lock(&op_data->flc); locks_copy_lock(&op_data->flc, fl); @@ -168,8 +166,6 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file, send_op(op); rv = FILE_LOCK_DEFERRED; goto out; - } else { - op->info.owner = (__u64)(long) fl->fl_owner; } send_op(op); @@ -326,10 +322,7 @@ int dlm_posix_unlock(dlm_lockspace_t *lockspace, u64 number, struct file *file, op->info.number = number; op->info.start = fl->fl_start; op->info.end = fl->fl_end; - if (fl->fl_lmops && fl->fl_lmops->lm_grant) - op->info.owner = (__u64) fl->fl_pid; - else - op->info.owner = (__u64)(long) fl->fl_owner; + op->info.owner = (__u64)(long)fl->fl_owner; if (fl->fl_flags & FL_CLOSE) { op->info.flags |= DLM_PLOCK_FL_CLOSE; @@ -389,7 +382,7 @@ int dlm_posix_cancel(dlm_lockspace_t *lockspace, u64 number, struct file *file, info.number = number; info.start = fl->fl_start; info.end = fl->fl_end; - info.owner = (__u64)fl->fl_pid; + info.owner = (__u64)(long)fl->fl_owner; rv = do_lock_cancel(&info); switch (rv) { @@ -450,10 +443,7 @@ int dlm_posix_get(dlm_lockspace_t *lockspace, u64 number, struct file *file, op->info.number = number; op->info.start = fl->fl_start; op->info.end = fl->fl_end; - if (fl->fl_lmops && fl->fl_lmops->lm_grant) - op->info.owner = (__u64) fl->fl_pid; - else - op->info.owner = (__u64)(long) fl->fl_owner; + op->info.owner = (__u64)(long)fl->fl_owner; send_op(op); wait_event(recv_wq, (op->done != 0)); From patchwork Wed Aug 23 21:33:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363168 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE5191FB35 for ; Wed, 23 Aug 2023 21:34:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MHYX4a6Baf9t0hYlusUokWnG9KN4LxtXmYn0dFm/ZDM=; b=EDSGm0ZMxL/Y/ARssEh0hbvWESPADuerKrmUzdKaO0PI5EP7lwlYGphBljVrnYOGt9G9wR odMAWzQbXtAxbgPjkwLiCGYMNbjsnY4xcACtpTzH+kjjVZStLCZSUr7R9OYi1mnzcTfTOI zXK9mulQI9P1ayMG1djvG3jKccjUP8k= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-299-7W_F9yi9MmS-MNHeCgyAGw-1; Wed, 23 Aug 2023 17:33:56 -0400 X-MC-Unique: 7W_F9yi9MmS-MNHeCgyAGw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C690A856DED; Wed, 23 Aug 2023 21:33:55 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E5BF400E02F; Wed, 23 Aug 2023 21:33:55 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 6/7] dlm: use FL_SLEEP to determine blocking vs non-blocking Date: Wed, 23 Aug 2023 17:33:51 -0400 Message-Id: <20230823213352.1971009-7-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch uses the FL_SLEEP flag in struct file_lock to determine if the lock request is a blocking or non-blocking request. Before dlm was using IS_SETLKW() was being used which is not usable for lock requests coming from lockd when EXPORT_OP_SAFE_ASYNC_LOCK inside the export flags is set. Signed-off-by: Alexander Aring Reviewed-by: Jeff Layton --- fs/dlm/plock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/dlm/plock.c b/fs/dlm/plock.c index 0094fa4004cc..0c6ed5eeb840 100644 --- a/fs/dlm/plock.c +++ b/fs/dlm/plock.c @@ -140,7 +140,7 @@ int dlm_posix_lock(dlm_lockspace_t *lockspace, u64 number, struct file *file, op->info.optype = DLM_PLOCK_OP_LOCK; op->info.pid = fl->fl_pid; op->info.ex = (fl->fl_type == F_WRLCK); - op->info.wait = IS_SETLKW(cmd); + op->info.wait = !!(fl->fl_flags & FL_SLEEP); op->info.fsid = ls->ls_global_id; op->info.number = number; op->info.start = fl->fl_start; From patchwork Wed Aug 23 21:33:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Aring X-Patchwork-Id: 13363166 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77311200A4 for ; Wed, 23 Aug 2023 21:34:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692826439; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pArjaDFSJm/PaSO+le75amYj6A0iIy9B3F5/c/CEF10=; b=did84/pVtboPXdyPVNxk26lN6oe3orypI5wy86ymE9wTmxB+09s7ig7JRiYIEDTP8xP2JS JWmgUbyUio1YgXVeMXOqtuQSsTGuRE+p0uhzpR+/77zCfe3ScZDKWx+41TR2TZZIHWeJZG T6KFo+7ASF2iIivZq/9e1Ck5hN3YRlo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-hSjMRL5wMv-Dm62OZlRExg-1; Wed, 23 Aug 2023 17:33:56 -0400 X-MC-Unique: hSjMRL5wMv-Dm62OZlRExg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 226F8101A528; Wed, 23 Aug 2023 21:33:56 +0000 (UTC) Received: from fs-i40c-03.fs.lab.eng.bos.redhat.com (fs-i40c-03.fs.lab.eng.bos.redhat.com [10.16.224.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE4EF40C6F4E; Wed, 23 Aug 2023 21:33:55 +0000 (UTC) From: Alexander Aring To: linux-nfs@vger.kernel.org Cc: cluster-devel@redhat.com, ocfs2-devel@lists.linux.dev, linux-fsdevel@vger.kernel.org, teigland@redhat.com, rpeterso@redhat.com, agruenba@redhat.com, trond.myklebust@hammerspace.com, anna@kernel.org, chuck.lever@oracle.com, jlayton@kernel.org Subject: [PATCH 7/7] dlm: implement EXPORT_OP_SAFE_ASYNC_LOCK Date: Wed, 23 Aug 2023 17:33:52 -0400 Message-Id: <20230823213352.1971009-8-aahringo@redhat.com> In-Reply-To: <20230823213352.1971009-1-aahringo@redhat.com> References: <20230823213352.1971009-1-aahringo@redhat.com> Precedence: bulk X-Mailing-List: ocfs2-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 This patch is activating the EXPORT_OP_SAFE_ASYNC_LOCK export flag to signal lockd that both filesystems are able to handle async lock requests. The cluster filesystems gfs2 and ocfs2 will redirect their lock requests to DLMs plock implementation that can handle async lock requests. Signed-off-by: Alexander Aring --- fs/gfs2/export.c | 1 + fs/ocfs2/export.c | 1 + 2 files changed, 2 insertions(+) diff --git a/fs/gfs2/export.c b/fs/gfs2/export.c index cf40895233f5..36bc43b9d141 100644 --- a/fs/gfs2/export.c +++ b/fs/gfs2/export.c @@ -192,5 +192,6 @@ const struct export_operations gfs2_export_ops = { .fh_to_parent = gfs2_fh_to_parent, .get_name = gfs2_get_name, .get_parent = gfs2_get_parent, + .flags = EXPORT_OP_SAFE_ASYNC_LOCK, }; diff --git a/fs/ocfs2/export.c b/fs/ocfs2/export.c index eaa8c80ace3c..8a1169e01dd9 100644 --- a/fs/ocfs2/export.c +++ b/fs/ocfs2/export.c @@ -280,4 +280,5 @@ const struct export_operations ocfs2_export_ops = { .fh_to_dentry = ocfs2_fh_to_dentry, .fh_to_parent = ocfs2_fh_to_parent, .get_parent = ocfs2_get_parent, + .flags = EXPORT_OP_SAFE_ASYNC_LOCK, };