From patchwork Thu Feb 20 16:47:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13984281 X-Patchwork-Delegate: cel@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 767BD213E66; Thu, 20 Feb 2025 16:47:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070049; cv=none; b=Z9wMPE8C9yPQ0Dm3JGrKwWMnlxwwQ9MEjsXlTpQsE0cbr8dTjxizR67n4ElHO1xn5zAykXmNF4GAHxEPl7HNuEakKKA0QAkScSKFSK1bz89sZwn/BGPaXkd8Pcs42HkmZ1LklYzdwZdappL9YeXQ7okDLbF21whk2SAoCBY0MQM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070049; c=relaxed/simple; bh=cM51zXKQgB7chVb7HCS+hb5p+nJh2B05fSYFZWc6QqA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RBPO1ybkULX9KvfjcTRjXNN8n/YSn2cczQPNt8fGfeYRwtgWE+jiSAEABtQdcIsw+s8Wbvk6P6endR+iDkqBKDOMkDt2g05Y9BoP2aOzifpkKRdPyBj8pDZ02WnKb7lmiag6k1MS+r2SS2FRr5o086geok0MQrVmUnaHforwHys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Jr9g/zM0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Jr9g/zM0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EB9EC4CEDD; Thu, 20 Feb 2025 16:47:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740070048; bh=cM51zXKQgB7chVb7HCS+hb5p+nJh2B05fSYFZWc6QqA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Jr9g/zM0r5mTY0Wmbw4q7Rc92zwe+qqD8Vmi/QJ4ZBqtYtx28tgqtIPJjR9bwrIDV sXAYsfslIu+Pt8zskVDGAsYseQVDJ1C5dVX55jME9bwNXXWBFfze/cC7nGejbx1tvL 2hNgunvZFe80OhUzt1G2JVmcaN1ba2toS8OE0N2J19mOrDMzoL+avlt2Xq/WDGopZ+ l7HfKNpm6NJZmTTtG6+dVyBVPfEzWTt2BrIDgGVHLn7kv7SNdqzB3ieXsvFdeuVP8M sTEf3tA5uPn4oKobLvD/l9gRpgIpTofdIB1ZNyt0/HDXJQy+/yn2A7BOnQCv4k8IZw DPx6jrRONVayg== From: Jeff Layton Date: Thu, 20 Feb 2025 11:47:13 -0500 Subject: [PATCH v2 1/5] nfsd: prevent callback tasks running concurrently Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250220-nfsd-callback-v2-1-6a57f46e1c3a@kernel.org> References: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> In-Reply-To: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: Li Lingfeng , linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6190; i=jlayton@kernel.org; h=from:subject:message-id; bh=cM51zXKQgB7chVb7HCS+hb5p+nJh2B05fSYFZWc6QqA=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnt1yetRj0At3wEXKZLmmVcWst+WHAgQSRfg5o8 AR6RO/tNcuJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ7dcngAKCRAADmhBGVaC FZqoEACRp1d+ELs8oaCUJgU7jqBtu6TQoZdOe/Rd3/aW8EdGN3Iv5TQfIjJ8dFfUxyUKNjgqINf IpjJ7P+fPMt1ENWd08OHcUUXG+URVHaEdqgDIuJvANmiiuzXyCG2vhmT112ZF0lXwHAUB79ad+t j0XoMqv3Ze0vL4rpImvo3kvDBi5sqRnRCpXVywZg+v9Qal2h0f72KntvN0D0/SEsIthM5Pc/4ST lk3UNKpIVAmv0F13Sa/zVI/h7xJo46SXa61EsbMxj0I3YyoyCGqW9atxIG0jJ/PMQx11S4S9qMs 4XBrSEiBbKj7CjygYGw9yLmV55pWILKc0a6zgbj4EqJuK0TYjYB9fMqOMHRD7re8ii66dfkQawT NeeIs2TCHzhLjQW1bSmLZ7QVjMaZeAdwuGyinNMDxmLTsraqICScNliJ+BK4Pz+Qb+Dn4IISt/l ReRVzCU1y4Z2rrXZXbbnjTEur17okuKCv0H5By0K+mBDGVwR7V3tdfsVFJemp74m3NoWfODUzNA qdxLb0CjFAWE5Eo8yRGYQ/Jj3wHiZrKb+Q7I0epItmw8iJDd7RRvDzxykt7DNJqABqsnzd5N3lg 3cOynuq0JxHrj7arX1Vygtw7hg0N8Bc9FhSzxwkf0RcdRpZyxqpdhScBGmeNUa3B6PxGZKeCYed zTGw4IGMuXfV4BA== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 The nfsd4_callback workqueue jobs exist to queue backchannel RPCs to rpciod. Because they run in different workqueue contexts, the rpc_task can run concurrently with the workqueue job itself, should it become requeued. This is problematic as there is no locking when accessing the fields in the nfsd4_callback. Add a new unsigned long to nfsd4_callback and declare a new NFSD4_CALLBACK_RUNNING flag to be set in it. When attempting to run a workqueue job, do a test_and_set_bit() on that flag first, and don't queue the workqueue job if it returns true. Clear NFSD4_CALLBACK_RUNNING in nfsd41_destroy_cb(). This also gives us a more reliable mechanism for handling queueing failures in codepaths where we have to take references under spinlocks. We can now do the test_and_set_bit on NFSD4_CALLBACK_RUNNING first, and only take references to the objects if that returns false. Most of the nfsd4_run_cb() callers are converted to use this new flag or the nfsd4_try_run_cb() wrapper. The main exception is the callback channel probe, which has its own synchronization. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 2 ++ fs/nfsd/nfs4layouts.c | 7 ++++--- fs/nfsd/nfs4proc.c | 2 +- fs/nfsd/nfs4state.c | 14 ++++++++++---- fs/nfsd/state.h | 9 +++++++++ 5 files changed, 26 insertions(+), 8 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index ae4b7b6df47ff054db197bd2f6083905e4149bee..1f26c811e5f73c2e745ee68d0b6e668d1dd7c704 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1312,6 +1312,7 @@ static void nfsd41_destroy_cb(struct nfsd4_callback *cb) trace_nfsd_cb_destroy(clp, cb); nfsd41_cb_release_slot(cb); + clear_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags); if (cb->cb_ops && cb->cb_ops->release) cb->cb_ops->release(cb); nfsd41_cb_inflight_end(clp); @@ -1632,6 +1633,7 @@ void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp, cb->cb_msg.rpc_proc = &nfs4_cb_procedures[op]; cb->cb_msg.rpc_argp = cb; cb->cb_msg.rpc_resp = cb; + cb->cb_flags = 0; cb->cb_ops = ops; INIT_WORK(&cb->cb_work, nfsd4_run_cb_work); cb->cb_status = 0; diff --git a/fs/nfsd/nfs4layouts.c b/fs/nfsd/nfs4layouts.c index fbfddd3c4c943c34aa3991328eacb5759e311cc4..290271ac424540e4405a5fd0eacc8db9f47603cd 100644 --- a/fs/nfsd/nfs4layouts.c +++ b/fs/nfsd/nfs4layouts.c @@ -344,9 +344,10 @@ nfsd4_recall_file_layout(struct nfs4_layout_stateid *ls) atomic_inc(&ls->ls_stid.sc_file->fi_lo_recalls); trace_nfsd_layout_recall(&ls->ls_stid.sc_stateid); - refcount_inc(&ls->ls_stid.sc_count); - nfsd4_run_cb(&ls->ls_recall); - + if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ls->ls_recall.cb_flags)) { + refcount_inc(&ls->ls_stid.sc_count); + nfsd4_run_cb(&ls->ls_recall); + } out_unlock: spin_unlock(&ls->ls_lock); } diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index f6e06c779d09dacdcea81fb3b4135bf600f6cc63..b397246dae7b7e8c2a0ba436bb3813213cfe4fa2 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -1847,7 +1847,7 @@ static void nfsd4_send_cb_offload(struct nfsd4_copy *copy) NFSPROC4_CLNT_CB_OFFLOAD); trace_nfsd_cb_offload(copy->cp_clp, &cbo->co_res.cb_stateid, &cbo->co_fh, copy->cp_count, copy->nfserr); - nfsd4_run_cb(&cbo->co_cb); + nfsd4_try_run_cb(&cbo->co_cb); } /** diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index e806fd97cca972559d1929d37c1a24e760d9d6d8..fabcd979c40695ebcc795cfd2d8a035b7d589a37 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -3232,8 +3232,10 @@ static void nfs4_cb_getattr(struct nfs4_cb_fattr *ncf) /* set to proper status when nfsd4_cb_getattr_done runs */ ncf->ncf_cb_status = NFS4ERR_IO; - refcount_inc(&dp->dl_stid.sc_count); - nfsd4_run_cb(&ncf->ncf_getattr); + if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ncf->ncf_getattr.cb_flags)) { + refcount_inc(&dp->dl_stid.sc_count); + nfsd4_run_cb(&ncf->ncf_getattr); + } } static struct nfs4_client *create_client(struct xdr_netobj name, @@ -5422,6 +5424,10 @@ static const struct nfsd4_callback_ops nfsd4_cb_recall_ops = { static void nfsd_break_one_deleg(struct nfs4_delegation *dp) { bool queued; + + if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags)) + return; + /* * We're assuming the state code never drops its reference * without first removing the lease. Since we're in this lease @@ -6910,7 +6916,7 @@ deleg_reaper(struct nfsd_net *nn) clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) | BIT(RCA4_TYPE_MASK_WDATA_DLG); trace_nfsd_cb_recall_any(clp->cl_ra); - nfsd4_run_cb(&clp->cl_ra->ra_cb); + nfsd4_try_run_cb(&clp->cl_ra->ra_cb); } } @@ -7839,7 +7845,7 @@ nfsd4_lm_notify(struct file_lock *fl) if (queue) { trace_nfsd_cb_notify_lock(lo, nbl); - nfsd4_run_cb(&nbl->nbl_cb); + nfsd4_try_run_cb(&nbl->nbl_cb); } } diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 74d2d7b42676d907bec9159b927aeed223d668c3..e6af969a03f26b1feb6768af5baa4887bf5fa5e9 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -67,6 +67,8 @@ typedef struct { struct nfsd4_callback { struct nfs4_client *cb_clp; struct rpc_message cb_msg; +#define NFSD4_CALLBACK_RUNNING BIT(0) // Callback is running + unsigned long cb_flags; const struct nfsd4_callback_ops *cb_ops; struct work_struct cb_work; int cb_seq_status; @@ -780,6 +782,13 @@ extern void nfsd4_change_callback(struct nfs4_client *clp, struct nfs4_cb_conn * extern void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp, const struct nfsd4_callback_ops *ops, enum nfsd4_cb_op op); extern bool nfsd4_run_cb(struct nfsd4_callback *cb); + +static inline void nfsd4_try_run_cb(struct nfsd4_callback *cb) +{ + if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags)) + WARN_ON_ONCE(!nfsd4_run_cb(cb)); +} + extern void nfsd4_shutdown_callback(struct nfs4_client *); extern void nfsd4_shutdown_copy(struct nfs4_client *clp); void nfsd4_async_copy_reaper(struct nfsd_net *nn); From patchwork Thu Feb 20 16:47:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13984282 X-Patchwork-Delegate: cel@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A17D213E8A; Thu, 20 Feb 2025 16:47:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070050; cv=none; b=b5P4fIBf3Ta0h+cESDR2M03dkyURXcLszLOnBIpj2NISjRCZwAjGsF7oqnpuoijS0r05qKqAjB5dqKYiHHXzl+cbjkdaZ7m2X1GOg2VbHXr9nQDhTd3zFTYOB0NrcGTwpYvtUFrsjd5dcLFtVa2pWxnMQmGeEpTJefclgVl3qhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070050; c=relaxed/simple; bh=10fJE8iIzBywIv//IGMFScNX8ddz9jj63jeGVuq8/u4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=NtXpgg1qhTEEQEEfw255AuBc6JJf1WZEOQUQHAaCVovwqhYTvxnFPTzWGz3Q03e4WqaM+IkiNiFugozSeZKT7CQu6EIQaUeE5njiiR/VRcCEWaQbF40vqZVUz5PtO5UVY67IC6GZ0obFD2ABnCUQZnsTuXoBoS06hvVhJSInxE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E1pzxPOu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E1pzxPOu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F117C4CEE6; Thu, 20 Feb 2025 16:47:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740070049; bh=10fJE8iIzBywIv//IGMFScNX8ddz9jj63jeGVuq8/u4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=E1pzxPOuFdSwAa2ZQEHEE5yCTIwJdGRb0qzHdxQ8k5BQnkXsObCNMgutrE0e7JW5F ZwDSn29AY1WQarvkve1ZGCR5Fun7P2GnxVDLIbUY/9V+TdzDYnpaST/5OCikaQDNaG OpJND+sWmvh5+m6zt42W5IIgDSr5LrNQklPOt5Zoun4syRXmMcvqfO/oYV1wJiVhZR gYdjg5nAyG3RiHBpnt9W51u3DMff40I2jqDwJgllsDBGFVPesAykrvs9WV+6tJwIqo uNw8y0NzNTOwGywEYC8UskriuTqEuWQlRcm+b7Ni+2WAE64VJ0QDcUeWyWCTDBJJDz e1hWj2/0CqNGA== From: Jeff Layton Date: Thu, 20 Feb 2025 11:47:14 -0500 Subject: [PATCH v2 2/5] nfsd: eliminate cl_ra_cblist and NFSD4_CLIENT_CB_RECALL_ANY Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250220-nfsd-callback-v2-2-6a57f46e1c3a@kernel.org> References: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> In-Reply-To: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: Li Lingfeng , linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3279; i=jlayton@kernel.org; h=from:subject:message-id; bh=10fJE8iIzBywIv//IGMFScNX8ddz9jj63jeGVuq8/u4=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnt1yekrI7C3/QhJ1L5NFsw+HJvZ7tYgcDOk5I+ RHPhLMU/SCJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ7dcngAKCRAADmhBGVaC FV8ZEACXg5ohCqw9b0jqRibXpw14FkhFPoUKKhrOnBIB9GfOVYx3F6Ky9+S2LfZoKuHcTrEZlVj 9mZCneEw/BBPnGb071d6nKpY8UgDM2Xp/VDhJyn6TnvG+/6XF1wE5AqUSIlYvmeWx8Aa1cLKWxP /O6eTKEWjg8X+7oVua4kawwcltsReDAIf2bNM4RbYd0Sj+wxntULKsxq30bi12dLITPDwigaBxe o2OwiMpiifjhTD/dlQkcheNXZ5lJhYgOXTYCVeNGsdv/4hCnlaKGD0PFr9oJ/Eu8ILkgw49J+oM ELqex0+8l1pwg4/ttGLygVdq8OohWaATc1QlJ9TAfESXTHEPtsHSyp0gqUjbXbou9/K/4EcD12K FQd4+dYTwP0RkgLksNcMskPhykmDTgctStKH5i5/y4z6GWYiutlLYlkuT0pmX/nPzTS3Daq3I58 xJBQ8qOA+mDOpJgbmcFtW1WFbVkeQTbXiPzXaJ7zcjHylWi44qJa859xLdFEIzW/7oFqQGj7tzP 8z+sRSWQyQVk5L/f51xCb7+gWBMJisx409YJCzNQaFJxFxu5OdTg7nE51c+AyvHUOsEBgnv7nvT hvzmXweG2Efq8OuA+EQJG9PgVjs3UoWrYJS1od3NG1PbQR4e4av3SHZ7fPCXNwwI5fZsR4hQy/t GujlUpmAdEokjYw== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 deleg_reaper() will walk the client_lru list and put any suitable entries onto "cblist" using the cl_ra_cblist pointer. It then walks the objects outside the spinlock and queues callbacks for them. None of the operations that deleg_reaper() does outside the nn->client_lock are blocking operations. Just queue their workqueue jobs under the nn->client_lock instead. Also, the NFSD4_CLIENT_CB_RECALL_ANY and NFSD4_CALLBACK_RUNNING flags serve an identical purpose now. Drop the NFSD4_CLIENT_CB_RECALL_ANY flag and just use the one in the callback. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4state.c | 16 +++------------- fs/nfsd/state.h | 2 -- 2 files changed, 3 insertions(+), 15 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index fabcd979c40695ebcc795cfd2d8a035b7d589a37..422439a46ffd03926524b8463cfdabfb866281b3 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -3175,7 +3175,6 @@ nfsd4_cb_recall_any_release(struct nfsd4_callback *cb) { struct nfs4_client *clp = cb->cb_clp; - clear_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags); drop_client(clp); } @@ -6881,7 +6880,6 @@ deleg_reaper(struct nfsd_net *nn) { struct list_head *pos, *next; struct nfs4_client *clp; - LIST_HEAD(cblist); spin_lock(&nn->client_lock); list_for_each_safe(pos, next, &nn->client_lru) { @@ -6893,31 +6891,23 @@ deleg_reaper(struct nfsd_net *nn) continue; if (atomic_read(&clp->cl_delegs_in_recall)) continue; - if (test_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags)) + if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags)) continue; if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5) continue; if (clp->cl_cb_state != NFSD4_CB_UP) continue; - list_add(&clp->cl_ra_cblist, &cblist); /* release in nfsd4_cb_recall_any_release */ kref_get(&clp->cl_nfsdfs.cl_ref); - set_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags); clp->cl_ra_time = ktime_get_boottime_seconds(); - } - spin_unlock(&nn->client_lock); - - while (!list_empty(&cblist)) { - clp = list_first_entry(&cblist, struct nfs4_client, - cl_ra_cblist); - list_del_init(&clp->cl_ra_cblist); clp->cl_ra->ra_keep = 0; clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) | BIT(RCA4_TYPE_MASK_WDATA_DLG); trace_nfsd_cb_recall_any(clp->cl_ra); - nfsd4_try_run_cb(&clp->cl_ra->ra_cb); + nfsd4_run_cb(&clp->cl_ra->ra_cb); } + spin_unlock(&nn->client_lock); } static void diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index e6af969a03f26b1feb6768af5baa4887bf5fa5e9..6967925f4b438b24c2abd5fcd4f3bd6664a81862 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -454,7 +454,6 @@ struct nfs4_client { #define NFSD4_CLIENT_UPCALL_LOCK (5) /* upcall serialization */ #define NFSD4_CLIENT_CB_FLAG_MASK (1 << NFSD4_CLIENT_CB_UPDATE | \ 1 << NFSD4_CLIENT_CB_KILL) -#define NFSD4_CLIENT_CB_RECALL_ANY (6) unsigned long cl_flags; struct workqueue_struct *cl_callback_wq; @@ -500,7 +499,6 @@ struct nfs4_client { struct nfsd4_cb_recall_any *cl_ra; time64_t cl_ra_time; - struct list_head cl_ra_cblist; }; /* struct nfs4_client_reset From patchwork Thu Feb 20 16:47:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13984283 X-Patchwork-Delegate: cel@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D88F212D86; Thu, 20 Feb 2025 16:47:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070051; cv=none; b=t3rum16trFcAZQOy6m9SygafLeHePB6EYMM1nW2i5BLbwIm5VYJtTIGmw/GcvKsd0uZfADkOUFHai6/HUfahGRH/r8Y8SRbIMD1kdZJbpdlZD5rU/r2964LnUT90b9FmY3kk5ck0CGXs8t8bJVSsL/wXFO/DuRtEw4ff4VyB42o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070051; c=relaxed/simple; bh=ila+U1RbL04csr1V3SrKR23UcsiSK+mTiCSxKDF7PPI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KLAFFbO/RKiI4l+YnRjE8XvYX4QNVUkFmfRaCo0hE/TcejUQHU0ny+ce3LyOWlbUQlgQfHZAsS/CbMjE2ECb6jzJ8aGWTV10u0zdXjEbyPaeuaSuaMEJIueJE5L4oxTuYezxdvq6JDVaWZlQ68a0yW3p7GxUOSTGftFCBEXXt/o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UGuNJaaK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UGuNJaaK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2DC7BC4CEE2; Thu, 20 Feb 2025 16:47:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740070050; bh=ila+U1RbL04csr1V3SrKR23UcsiSK+mTiCSxKDF7PPI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=UGuNJaaKioadX1AWSd5yBlw7i/yf3OaYPOk//FjtLu+mMazVM4CvQ4WSEV2Er1tdp IXMLtEUn6kNCI5LVJNzJi4NEYKQa5ZSdms9teKIkfY1YBl+tHkfyr4VmWG8qBheS1I UvoA+yCxD0HTau+HzYEa11+jekvYrr5HTosUpPQ8VxC4SMweuQBaINOa+7Fve56C1m kKFo+A4dtRyrlpeqW2sbGQDlzmAQL5awsC3Nlu7h4A1V3p+zReBivWiA7VkP7iSnis 5eZOCzgum+/rMLfpnUtesg3Z6wzPrNPhF+kE4y5+cxrS0nWRP9jkZCvKuT9OGqA3sT OOmm+VHkFRoVw== From: Jeff Layton Date: Thu, 20 Feb 2025 11:47:15 -0500 Subject: [PATCH v2 3/5] nfsd: replace CB_GETATTR_BUSY with NFSD4_CALLBACK_RUNNING Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250220-nfsd-callback-v2-3-6a57f46e1c3a@kernel.org> References: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> In-Reply-To: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: Li Lingfeng , linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4425; i=jlayton@kernel.org; h=from:subject:message-id; bh=ila+U1RbL04csr1V3SrKR23UcsiSK+mTiCSxKDF7PPI=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnt1yekIeVzU2mnGiQR4Cik3t8bX9ez2J0ynAvy ZFawIB3RSKJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ7dcngAKCRAADmhBGVaC Ffj7EAC4zNF4gP+kE8zUapXIUfRNiHlDsCX/WMLQPqbjPCK2fhBkPBGQNAvIRRu0ZEg2oszaFH/ o4ftFqE524FdteYwynzxtxmGjf06fO1lu+S5YGUjcMp1Q5TcqJDHzm1+AkVXwj4zyq7EMd8/X+l Du5exjMwRafGhiWcIwRMt0yLhLLduFSFdGgc37Q648eQPHVf1gw4sbIUHtrexKgca8wi5gP2DkG 3kRJgm4Qg68bGMvqmLEWOG3wQl1MGKAxodPU4oclOVwhx/HcKeiCLf8fxR9pGZ2E5uK8iWq5iD0 JNuFEyNkcBOQIzbYhATbofUNS1F8RWZy9F7QWHHpPPjUYqZedARR2oe5/NE79CtWz7oRbFv5KoG NDMcOww2x0Gl+LEU6Ec8Olr93+rP/UZYhgEQvN/kKUMGFTOwJ+8xRF0QtsdK8P/l9hZ4+OrbG4M 3NS0GdSFOCb5va2m/8AfSeCjeX4C1PB0dUNdhXX19YktOhRrZh7l3dnM2IOkUhGH/tw7+TXhY5V NDnHdgnIi3LnIGEEi6kI35IDhljjWVXLLQYZkiPC5QgMz/4JQ/DKq7wbNq09yp2YVA6dcu0fg7Z LAJ7mxvHP2lk64XH4LxnV2QQBo+Iqy7l+16xaRMFmVxPn80bcJ7lMPsJib7gdZlRoYfRmhDqEeK dbWMog3MZnmiAbg== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 These flags serve essentially the same purpose and get set and cleared at the same time. Drop CB_GETATTR_BUSY and just use NFSD4_CALLBACK_RUNNING instead. For this to work, we must use clear_and_wake_up_bit(), but doing that on for other types of callbacks is wasteful. Declare a new NFSD4_CALLBACK_WAKE flag in cb_flags to indicate that wake_up is needed, and only set that for CB_GETATTRs. Also, make the wait use a TASK_UNINTERRUPTIBLE sleep. This is done in the context of an nfsd thread, and it should never need to deal with signals. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 6 +++++- fs/nfsd/nfs4state.c | 17 +++++++++-------- fs/nfsd/state.h | 5 +---- 3 files changed, 15 insertions(+), 13 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 1f26c811e5f73c2e745ee68d0b6e668d1dd7c704..bb2cb0d1b7885bf5f6be39221cb3dd95518326e5 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1312,7 +1312,11 @@ static void nfsd41_destroy_cb(struct nfsd4_callback *cb) trace_nfsd_cb_destroy(clp, cb); nfsd41_cb_release_slot(cb); - clear_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags); + if (test_bit(NFSD4_CALLBACK_WAKE, &cb->cb_flags)) + clear_and_wake_up_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags); + else + clear_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags); + if (cb->cb_ops && cb->cb_ops->release) cb->cb_ops->release(cb); nfsd41_cb_inflight_end(clp); diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 422439a46ffd03926524b8463cfdabfb866281b3..b68bc3f12a8c29904e6bdd2b5a1e4ed6418104e3 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -3205,7 +3205,6 @@ nfsd4_cb_getattr_release(struct nfsd4_callback *cb) struct nfs4_delegation *dp = container_of(ncf, struct nfs4_delegation, dl_cb_fattr); - clear_and_wake_up_bit(CB_GETATTR_BUSY, &ncf->ncf_cb_flags); nfs4_put_stid(&dp->dl_stid); } @@ -3226,15 +3225,17 @@ static void nfs4_cb_getattr(struct nfs4_cb_fattr *ncf) struct nfs4_delegation *dp = container_of(ncf, struct nfs4_delegation, dl_cb_fattr); - if (test_and_set_bit(CB_GETATTR_BUSY, &ncf->ncf_cb_flags)) + if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ncf->ncf_getattr.cb_flags)) return; + /* set to proper status when nfsd4_cb_getattr_done runs */ ncf->ncf_cb_status = NFS4ERR_IO; - if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ncf->ncf_getattr.cb_flags)) { - refcount_inc(&dp->dl_stid.sc_count); - nfsd4_run_cb(&ncf->ncf_getattr); - } + /* ensure that wake_bit is done when RUNNING is cleared */ + set_bit(NFSD4_CALLBACK_WAKE, &ncf->ncf_getattr.cb_flags); + + refcount_inc(&dp->dl_stid.sc_count); + nfsd4_run_cb(&ncf->ncf_getattr); } static struct nfs4_client *create_client(struct xdr_netobj name, @@ -9210,8 +9211,8 @@ nfsd4_deleg_getattr_conflict(struct svc_rqst *rqstp, struct dentry *dentry, nfs4_cb_getattr(&dp->dl_cb_fattr); spin_unlock(&ctx->flc_lock); - wait_on_bit_timeout(&ncf->ncf_cb_flags, CB_GETATTR_BUSY, - TASK_INTERRUPTIBLE, NFSD_CB_GETATTR_TIMEOUT); + wait_on_bit_timeout(&ncf->ncf_getattr.cb_flags, NFSD4_CALLBACK_RUNNING, + TASK_UNINTERRUPTIBLE, NFSD_CB_GETATTR_TIMEOUT); if (ncf->ncf_cb_status) { /* Recall delegation only if client didn't respond */ status = nfserrno(nfsd_open_break_lease(inode, NFSD_MAY_READ)); diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 6967925f4b438b24c2abd5fcd4f3bd6664a81862..b7d6be45ad022194e90cdf8d19c2879be08143ff 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -68,6 +68,7 @@ struct nfsd4_callback { struct nfs4_client *cb_clp; struct rpc_message cb_msg; #define NFSD4_CALLBACK_RUNNING BIT(0) // Callback is running +#define NFSD4_CALLBACK_WAKE BIT(1) // must wake_bit when clearing RUNNING unsigned long cb_flags; const struct nfsd4_callback_ops *cb_ops; struct work_struct cb_work; @@ -164,15 +165,11 @@ struct nfs4_cb_fattr { struct timespec64 ncf_cb_mtime; struct timespec64 ncf_cb_atime; - unsigned long ncf_cb_flags; bool ncf_file_modified; u64 ncf_initial_cinfo; u64 ncf_cur_fsize; }; -/* bits for ncf_cb_flags */ -#define CB_GETATTR_BUSY 0 - /* * Represents a delegation stateid. The nfs4_client holds references to these * and they are put when it is being destroyed or when the delegation is From patchwork Thu Feb 20 16:47:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13984284 X-Patchwork-Delegate: cel@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A165214237; Thu, 20 Feb 2025 16:47:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070052; cv=none; b=NWo0ieNfz4JI/NE7j1tKbCyvMtMPrS+pZiWgFFV5fBTRdaKoKG0aUC0IYzElWt5yy61N6Yj1O8rugopw+LUZZq++UToupcNyAU01ZpbNoB5ZgrYNDDFonPttIlR+gQX4KkROGSQmQrQO+MCtfKo/huZ8c13J7SUC3kY1YeXVv0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070052; c=relaxed/simple; bh=g5VLUIvdnQKh6dDCL6AEV2BEFOeKTlGBSLsSlyN/ZSQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SeetMe+vwNShvR9+eP8QXnorcxe7jftMVcvxn46EPmNcYyAaYJ6X1o2yWYF1IAzjyz3kKJgSjS+e5EbbZyIVqOAzuO2ThAPycp8DnHpMbTGtATQwNgQQE41Lb48cweVrJd9FaVjNN6t0Lz6Wy1YAhGWlqa6ADjyDc9Rgq3+bDhc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=L15M5AtL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L15M5AtL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2CEC9C4CED1; Thu, 20 Feb 2025 16:47:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740070051; bh=g5VLUIvdnQKh6dDCL6AEV2BEFOeKTlGBSLsSlyN/ZSQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=L15M5AtLAWtSYujJtZVFQl0dnEOB/gMYz0iurV9xWO3/p14/cuiKygLOHFWbrgyWO 8foFJdonCwM4+V1X74eEU7UM90B3/9eElQp3Yk9FLCdCuy6k/jbqmJQnQ7AYAKX5Vx HPBKiXq2LL4SIXiyP/0QynekvcN7gKDj9zQEEvXDBe/HQxCqs35uNHpKyiahufZKAr As8JzT2rxHfgIVH4n4EXMRgi3onjg5lsoaSEKEYS1AVAfsjSyAb/ewCEUsOVj9a4Rs kbY8oxht/hfKqZz84g8cKaUpwS+U9A97A5lENGc5Q9ruccJIXz8QC8IjhLXitZJgjB H1dklkOxfZJqg== From: Jeff Layton Date: Thu, 20 Feb 2025 11:47:16 -0500 Subject: [PATCH v2 4/5] nfsd: move cb_need_restart flag into cb_flags Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250220-nfsd-callback-v2-4-6a57f46e1c3a@kernel.org> References: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> In-Reply-To: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: Li Lingfeng , linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3206; i=jlayton@kernel.org; h=from:subject:message-id; bh=g5VLUIvdnQKh6dDCL6AEV2BEFOeKTlGBSLsSlyN/ZSQ=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnt1yeATJEvgxP1v06nxL7BrouhyUSv1lBo+nUQ Kv8iTKFDT6JAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ7dcngAKCRAADmhBGVaC FcW0EADKpBsE7Uyy3HmcwVrxvoenvpRkdzjKCe0tVb84OEa3cE9oFV4mf5keAZivpyNv6xc56TQ xEoN3/dDojs7MGuFOhpLwtBTybLAf8AnmO98EofJV3V/FIiPKMbjCoo64+CF1+pshU4WDpjCNdl Q6Bcpw0i5aXioM5p+d9W2Cz0V07ZQvDOqjsVbwGBbv/G2nNTx/iV8SC0JzygLwgqPHxJ6bspPVp /2t61/7xALgeJKzoWZfULAx1OZ+ERgfldw3OAw3Ot2Tv3O/dMc4p5GFlHgLp9Bt3c1HHg/uq/Kq pOedamL11gSIkHAADdUP2Ad1nDuLEQIoOyxYff8IOy8038CyfbYn1q89Lc2ntc5dyl7dDC8ki01 GxRJTOknfUmIIsRdpUek15sNa0As4bPn/xuNLS2CEilBaDCtE6bzX9VDml8xg73LZc4Gx/9A+9R vZ/YaC7tj8vomTWBNQg+xV4LAa1RkjhJvYi238hyyHVi6OTo09DMaG+LEb27HpI6J9/teIGEX3g DaU33Usxr4prixy5KaPuCsG9mXwHorkURCFLWH+mSkRzECmFlBZ9nkolS8/uw1JUzImPSFmwbas F/S1eJtk7+iX2CjAkPXbelwXYW4FnDe/z34lTaphumiCkH6oek4ZydRyQ21MBpO7x5ljQ3ecvZJ K/Xf6HJcuJCn3rw== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 Since there is now a cb_flags word, use a new NFSD4_CALLBACK_REQUEUE flag in that instead of a separate boolean. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 10 ++++------ fs/nfsd/state.h | 2 +- fs/nfsd/trace.h | 2 +- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index bb2cb0d1b7885bf5f6be39221cb3dd95518326e5..018533bb83a3ca6fda46e072eebca6d2583c393a 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1071,7 +1071,7 @@ static void nfsd4_requeue_cb(struct rpc_task *task, struct nfsd4_callback *cb) if (!test_bit(NFSD4_CLIENT_CB_KILL, &clp->cl_flags)) { trace_nfsd_cb_restart(clp, cb); task->tk_status = 0; - cb->cb_need_restart = true; + set_bit(NFSD4_CALLBACK_REQUEUE, &cb->cb_flags); } } @@ -1479,7 +1479,7 @@ static void nfsd4_cb_release(void *calldata) trace_nfsd_cb_rpc_release(cb->cb_clp); - if (cb->cb_need_restart) + if (test_bit(NFSD4_CALLBACK_REQUEUE, &cb->cb_flags)) nfsd4_queue_cb(cb); else nfsd41_destroy_cb(cb); @@ -1618,12 +1618,11 @@ nfsd4_run_cb_work(struct work_struct *work) return; } - if (cb->cb_need_restart) { - cb->cb_need_restart = false; - } else { + if (!test_and_clear_bit(NFSD4_CALLBACK_REQUEUE, &cb->cb_flags)) { if (cb->cb_ops && cb->cb_ops->prepare) cb->cb_ops->prepare(cb); } + cb->cb_msg.rpc_cred = clp->cl_cb_cred; flags = clp->cl_minorversion ? RPC_TASK_NOCONNECT : RPC_TASK_SOFTCONN; rpc_call_async(clnt, &cb->cb_msg, RPC_TASK_SOFT | flags, @@ -1641,7 +1640,6 @@ void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp, cb->cb_ops = ops; INIT_WORK(&cb->cb_work, nfsd4_run_cb_work); cb->cb_status = 0; - cb->cb_need_restart = false; cb->cb_held_slot = -1; } diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index b7d6be45ad022194e90cdf8d19c2879be08143ff..bdcbee1d328a2f68da2e7c733f50f8eee5775be7 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -69,13 +69,13 @@ struct nfsd4_callback { struct rpc_message cb_msg; #define NFSD4_CALLBACK_RUNNING BIT(0) // Callback is running #define NFSD4_CALLBACK_WAKE BIT(1) // must wake_bit when clearing RUNNING +#define NFSD4_CALLBACK_REQUEUE BIT(2) // requeue callback instead of destroying unsigned long cb_flags; const struct nfsd4_callback_ops *cb_ops; struct work_struct cb_work; int cb_seq_status; int cb_status; int cb_held_slot; - bool cb_need_restart; }; struct nfsd4_callback_ops { diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index 49bbd26ffcdb36173047569b8d4b41efdec4880b..f26a42a37f452486e9683d7778016adf923f2995 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -1613,7 +1613,7 @@ DECLARE_EVENT_CLASS(nfsd_cb_lifetime_class, __entry->cl_id = clp->cl_clientid.cl_id; __entry->cb = cb; __entry->opcode = cb->cb_ops ? cb->cb_ops->opcode : _CB_NULL; - __entry->need_restart = cb->cb_need_restart; + __entry->need_restart = test_bit(NFSD4_CALLBACK_REQUEUE, &cb->cb_flags); __assign_sockaddr(addr, &clp->cl_cb_conn.cb_addr, clp->cl_cb_conn.cb_addrlen) ), From patchwork Thu Feb 20 16:47:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13984285 X-Patchwork-Delegate: cel@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A457214803; Thu, 20 Feb 2025 16:47:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070053; cv=none; b=naVSYB2PViR79fHMzss2O6bPa4SZjHrxw+bDyBcPrGm3WTO9+ikk0HXtFdB3+Sd7/VzRoVrMnSBg5ipqWXDG7JcWQrELMBMiYeT4Bf77RhgP++QGZKOCUmLi1QAMZant3wJJRWIfq33915BA4ZCYbeh2W9nh6MjuOwtMC5O/iDY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740070053; c=relaxed/simple; bh=Wft6oG6T8/oEeeQa1GWtph/ItnypOf1z1wIgrP2JdHE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=prJ9APZxbBCNaGc1aoC5sgqzXZEHW8mN8WVe9UZN5Rbtziuu2wHRbRvdNYOjSyceeqJWUCBkoyXbubwsaK1AYxLrdKM59hZ11nDfTUUxXjPcYIXTh1Te2jrevMkqfyHrs/6atK+ltjDKJB34zXC3Ge98mECTUXYM/Yz1En/n4OU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CFmuObO6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CFmuObO6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BC4AC4CEEB; Thu, 20 Feb 2025 16:47:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740070052; bh=Wft6oG6T8/oEeeQa1GWtph/ItnypOf1z1wIgrP2JdHE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=CFmuObO6Km08jTaUQ2diKRG0ueul0lafRhF/tjow4faEK1PLv14LVh7EsRgKAm8dO ol4z33DhmRWzF085w4Mrw2eywo7gv3MMdrFK3nw/s1dfCtaBz5N06bArA7N8B+GGUN hVrHwet3TtXKd8R2Nlp+fivB+AiKr3s+r6I2J2mKdOE/CB8OwFwKvfaJHhTFEHFAjD Y9FUFjJrj6S02qsCCkSc5/vI0w3OXl4u6YgUtSOAQzxcJdyjR09idFvt07JlEN/FiT 56G/I73HOPlILcMMv5A4jOXZPuEcjMT8vv5VRUxhGJlwJQFsbUnrQvKBmgqVEYmvDt /m2+bnxb5YcIw== From: Jeff Layton Date: Thu, 20 Feb 2025 11:47:17 -0500 Subject: [PATCH v2 5/5] nfsd: handle errors from rpc_call_async() Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250220-nfsd-callback-v2-5-6a57f46e1c3a@kernel.org> References: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> In-Reply-To: <20250220-nfsd-callback-v2-0-6a57f46e1c3a@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: Li Lingfeng , linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1442; i=jlayton@kernel.org; h=from:subject:message-id; bh=Wft6oG6T8/oEeeQa1GWtph/ItnypOf1z1wIgrP2JdHE=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnt1yemp+lyVjGK9VZRdurO3X1oCmfc3OCc3xy6 LEbJOQvmLGJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ7dcngAKCRAADmhBGVaC FR2kD/9ltlTi8jLn1usSf6osbYMROoo7/bZ8zaop6ikyJu59DSN3apuyGSdR3vl+qr+Eh/9OzVT sbKxufKda2W/ciSdGVmW38zZ/3BlvObGMttGPQK93y3k0lqAz3qLYTUWm7VEH7yPb/IrA2aoz09 HIHstn0JajBNQrm44YJoozhFXIakGfLK7AkWjNqfleufqIzooJZhJRpCOmjde7jR8+1gP8gKHjn t3tAzOPEtKU8mSY38BZ1nDvKuQeUa1sfQe42E13Sm7/EQz/g70sY9mm1bqur+mb2OtY3vyzkRAr u0Shk6Ex+HD0RK9+LPDhBcv+ZKr8j7fSltLzVu8Yz0BCkeAhWbpu/5D3NZJ7Aqur0I58eA6a2aT 1p5p93JQkEaockCG/brCffe9ES++fAkWFRd3LTIqFGMhKwZxtqGhUGXE3oQoMuS+YBHrTdf8bmo Lqn9qx8W1rf12vxrJzgvRLGzJxrkvZ9/tF7r9QhDNXfuyuXo63qi2cCYe6aOb+N2joMSO0sQ4AI 1HrtkPb0lBvqCKlhaKR0iCojM9PNinsfopgnuCkiWFpP+HkJipmfSjObF4uE7R4dsNleMh1StAZ PReoR1bjDQQr3w0uiiE1VcLsReT/Wg43mNLjAaqNqnv3FqnVLQ9yMOeGWMLzglkg+WI1mHA2gRv BkjihJtmT8jZdZw== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 It's possible for rpc_call_async() to fail (mainly due to memory allocation failure). If it does, there isn't much recourse other than to requeue the callback and try again later. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 018533bb83a3ca6fda46e072eebca6d2583c393a..3f21de06154e3f80818c85a92d6044058339697f 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1592,7 +1592,7 @@ nfsd4_run_cb_work(struct work_struct *work) container_of(work, struct nfsd4_callback, cb_work); struct nfs4_client *clp = cb->cb_clp; struct rpc_clnt *clnt; - int flags; + int flags, ret; trace_nfsd_cb_start(clp); @@ -1625,8 +1625,12 @@ nfsd4_run_cb_work(struct work_struct *work) cb->cb_msg.rpc_cred = clp->cl_cb_cred; flags = clp->cl_minorversion ? RPC_TASK_NOCONNECT : RPC_TASK_SOFTCONN; - rpc_call_async(clnt, &cb->cb_msg, RPC_TASK_SOFT | flags, - cb->cb_ops ? &nfsd4_cb_ops : &nfsd4_cb_probe_ops, cb); + ret = rpc_call_async(clnt, &cb->cb_msg, RPC_TASK_SOFT | flags, + cb->cb_ops ? &nfsd4_cb_ops : &nfsd4_cb_probe_ops, cb); + if (ret != 0) { + set_bit(NFSD4_CALLBACK_REQUEUE, &cb->cb_flags); + nfsd4_queue_cb(cb); + } } void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp,