From patchwork Wed Jan 29 23:20:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13954233 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E5121B425D; Wed, 29 Jan 2025 23:20:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192855; cv=none; b=DnmAv6sNdYKqF4xhRGuBp3k5wo7vavM/n9wUfQHWt+ZuJvPPF0YbxqB24lAETp4eOTLru5M/U7FFD8bt0fFaDxAAZyBSxfWLQj5Q2DG54tEIxEASK0HHRPsPiYvT4KgHBawO9SM59lyerXMBmcp9dhb1YW3++YNPIOq0NenGsKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192855; c=relaxed/simple; bh=0WRw457uiiqAmupT0nL4dUWX82jE82sEr8yyHXSOa5Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QiYKghXIhsmNsbhDyMOVo8ch6m0QovdZwRRRAHxdVrPV0dMtENjLRbIl5xkhTjTG9EM+XZZePuzDpNCP04/TScTLNn4zsc6Zkni3VCmFgxI8amVokZouh2f6GFL//gKHusWiCRDnL6NLclLY4a2jRWMZ2vJ7x1iIah9laICd9eM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FQK6gTEV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FQK6gTEV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42719C4CEE2; Wed, 29 Jan 2025 23:20:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738192855; bh=0WRw457uiiqAmupT0nL4dUWX82jE82sEr8yyHXSOa5Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FQK6gTEVU2MzeHqG11SvEw9fMcJ35Gm/a+2Tt5wJEa8zp5hUpcPv7yuXVWhVOuwex 4jq1VjslaV8HS/JpsWZLqDtBtWWvOeGfJJu0qTlXP4xfSJRDAXYKtg4PHbNgpTJehV 5zlxZl6gXCs+bGSOhuWl0kkRUILEetBw+8Xi/KJUeyzDJwoQbKOjr4ZwHdHmJcf6vA fp2JEAVWB+WqNisOmMsntCQpyvBtFT5XvxZac62arxHIPeSyRHPWEN5iVIlQGNdsgX 1+4W1uJuhjA4lxJqQEWDTV8vLrT5L6/aEPuDco7bqqm2/E6L6hs4vANx9k9m/K0Mtq tB0fgg3vNkvYQ== From: Jeff Layton Date: Wed, 29 Jan 2025 18:20:41 -0500 Subject: [PATCH v3 1/6] nfsd: add routines to get/put session references for callbacks Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250129-nfsd-6-14-v3-1-506e71e39e6b@kernel.org> References: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> In-Reply-To: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3008; i=jlayton@kernel.org; h=from:subject:message-id; bh=0WRw457uiiqAmupT0nL4dUWX82jE82sEr8yyHXSOa5Y=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnmrfTHIiyNSUmAARPmUcM92GB9fipXDf1fGeYp bf+O7FoKieJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ5q30wAKCRAADmhBGVaC FZJkD/9rR03U9XMS47NGkYcrXwmn4W9VYNopM4F9zLq4rExrJ8SSeKjw+PJOQa6p8s5uSLCibgW dy35k/JlTmuZnq1IcaCJ4nL1o4kOq9/n+8jzMFCNtu4sVrEiWCQ/K1TnE40NE3wpAoxUJGtwFUZ O5A/E8PcgyzVp1Ep2IRK5usT9XJC+vxTj1EghOSQVMVdez0HNKW1E+RKFef+6U6S3guDj/rswNb GW+1WdxaJVCVoxPBRQiPhw72BQRHmfGEJ3BZJecl3RlOGj1bEczUs8C0sYAiY4Q8gR+Frxb3172 UxezeN7idCAUhmWnzJfqu7lWIl7juYwOKgD2SsrugYl9AdGqc+IfXFWu9N6zk3a3F8pJKQhoucU DHZP4QZopuOQ6W5d0+pHbyNGFe2duo78K9qmjXelGlE3hfujvnEZMOLfz4a3t8TwhT8KyCVr4TX TZv/HITs8kMQE+g527ERoqZS7nXThQnr6zabMxptHATcYV907Z+FVWMz5zYMJfReRbQ/AtHfQzO x8ImQ/aeFUSYCE2A8OGmoEaQ8BZhenNNfISl3bzHdVAFTnUxqFzKYuHRibBY7HDw/uTPox6CDBc yOv6l7+Galf9AQP8wica8JtqOBEQKI+3kXxDsRzUtKjd+1z1hatX2noZlQXkeC2A53ibmsHvwd7 vdgaofYlQY77iPA== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 The existing session reference counting is too heavyweight for callbacks. There is an atomic refcount in nfsd4_session (se_ref), but the existing functions take a client reference alongside it, and require the nn->client_lock. This is unnecessary for callbacks as they are already owned by the client. Add new nfsd4_cb_get_session() and nfsd4_cb_put_session() calls that take and put a session reference on behalf of a callback. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4state.c | 34 ++++++++++++++++++++++++++++++++-- fs/nfsd/state.h | 2 ++ 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index cc819b8e8acdf5dcfe44c5bae45c6233f7b695e9..db68fd579ff0454153537817ee3cca71303654b4 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -234,6 +234,37 @@ static void put_client_renew(struct nfs4_client *clp) spin_unlock(&nn->client_lock); } +/** + * nfsd4_cb_get_session - get a session reference for a callback + * @ses: session of which to get a reference + * + * Callbacks are different than client-driven RPCs. The caller doesn't + * need a reference to the nfs4_client, and doesn't want to renew the + * lease when putting the reference. Returns true if a session was + * acquired, or false otherwise (which indicates that the session is + * dead). + */ +bool nfsd4_cb_get_session(struct nfsd4_session *ses) +{ + if (is_session_dead(ses)) + return false; + return atomic_inc_not_zero(&ses->se_ref); +} + +/** + * nfsd4_cb_put_session - put a session reference for a callback + * @ses: session of which to put a reference + * + * Callbacks are different than client-driven RPCs. The caller doesn't + * need a reference to the nfs4_client, and doesn't want to renew the + * lease when putting the reference. + */ +void nfsd4_cb_put_session(struct nfsd4_session *ses) +{ + if (ses && atomic_dec_and_test(&ses->se_ref) && is_session_dead(ses)) + free_session(ses); +} + static __be32 nfsd4_get_session_locked(struct nfsd4_session *ses) { __be32 status; @@ -254,8 +285,7 @@ static void nfsd4_put_session_locked(struct nfsd4_session *ses) lockdep_assert_held(&nn->client_lock); - if (atomic_dec_and_test(&ses->se_ref) && is_session_dead(ses)) - free_session(ses); + nfsd4_cb_put_session(ses); put_client_renew_locked(clp); } diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 74d2d7b42676d907bec9159b927aeed223d668c3..79d985d2a656e1a5b22a6a9c88f309515725e847 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -753,6 +753,8 @@ struct nfsd4_compound_state; struct nfsd_net; struct nfsd4_copy; +bool nfsd4_cb_get_session(struct nfsd4_session *ses); +void nfsd4_cb_put_session(struct nfsd4_session *ses); extern __be32 nfs4_preprocess_stateid_op(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, struct svc_fh *fhp, stateid_t *stateid, int flags, struct nfsd_file **filp, From patchwork Wed Jan 29 23:20:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13954235 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20DAE1E3DD6; Wed, 29 Jan 2025 23:20:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192856; cv=none; b=jtj5JruxaSnSTui52tRX7DgCU+Tow0MymYunhFDTh72FZ3vZGE3D1IYyI1SBnF+m/oQfmEJL4xRqfV9jvt3KpwonVN7lD7rA6L6e7BDx53VRJpcNJQUY817di11Jho+Cng/1sLfHR2y87cFB6HyAlfhuPHbrjpKHf6Y/csDWdWY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192856; c=relaxed/simple; bh=OiL8q1LfHA/I9AsXcKVH8K322R8abnjFCLN6FLdRUSQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OB9ACMbRBkYLfEC2mCMrNHrDZLaBmjb1VYlQ2UfEKmnnO/Y6jxFfgyI6AAHRaMRO5dR8e4AQNdsf6li63qh3tXK7s5l1WhPlzq0RQvL6TZ/wjdoDBn45YQgMcZK/XZP3R66EfeXU3VLWFtANqpW7ZnQurWpILz+Vg6T2D+cPtaQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=s242wHhM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="s242wHhM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C04BC4CEE4; Wed, 29 Jan 2025 23:20:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738192856; bh=OiL8q1LfHA/I9AsXcKVH8K322R8abnjFCLN6FLdRUSQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=s242wHhMFUWyBxKvDwZgymd8KV25vZ8jIx8+rxHstzeS10W1GYl7JTc2STN/S3cRi sS9TsIOasm3gPRuuOInY8uvZv1n1a2JEi8QJKInftS+RmY/+yF8n1qs/dj/zWb6GL4 kvXMpytt1h3qGbjPgxZOUYIlzb+2NycDO9heF5Ktel5cXADGjCQBKNYOr9wqy/9wUB cdfpsUB9wlc9Bdu/ln1JGXySyMtleSBbJkM41oEMJ/vFlMuYPy/Z5xckd5wEDACNf3 DKcWsvTflFWB1DVb3Adm8GlN0O3wzrS42XwHyxrbEuaQ73Fp5HkdrUnmHOGlJzLczi 90Fl56FmaXXEA== From: Jeff Layton Date: Wed, 29 Jan 2025 18:20:42 -0500 Subject: [PATCH v3 2/6] nfsd: make clp->cl_cb_session be an RCU managed pointer Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250129-nfsd-6-14-v3-2-506e71e39e6b@kernel.org> References: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> In-Reply-To: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5132; i=jlayton@kernel.org; h=from:subject:message-id; bh=OiL8q1LfHA/I9AsXcKVH8K322R8abnjFCLN6FLdRUSQ=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnmrfU6rKdmT6h/T/xIiITDkf7glkLWYjyWRRJE 9wamiqtBgWJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ5q31AAKCRAADmhBGVaC FecvEACHwdFbOL7MO/krhGsetbMdrgJ4vtbGthNke+p87/8psLO7Jc4OpKmU19FZHINQBxcB3ub qVvmp/OziCGmGrjXCBX/45GTVr4OF47FXoNmqD1VZoGZhlyGnfcFL1GI2bJDR2btEXNmDMOocSj VAyLxNbM1tD4oiBMcNiueDJdw9Gi1AjYUZz0/+k+e5qPzOmZLwPJxoAsmfVgCMPVGUJl73XCuyy RLaIM6Js/ua7RhTTPE+wX9RBiq8VTAunM7l5lsxwOvmHytPU8AIBNCvo0qIWrCKTSgvzVpsrofn 0+x9UukF+bawrzNCNys6y/GUM9kIzr84ypDdXJsC6mE/iUuNDAhrvDZ3txfmS6kXQSFpliu67h+ hITLhAcrw/WjqhfyDPniukecsGUD89SZVFvMJIbWltMNyRcJaMMvprbpW/Ob8M2h/C0Mrpm5l2f RvfDW6uraYVuwgHalTn6XrAVpq4zX7FbLmaeJk1Xdch5hr1ZMiT1+fb5uKHRIkI5MMDlNbmhgbd 49o94MJtrBFzums7Uqpbbe8BYBKsiVTprJx6Iq1vFpHHyHIWw3YSDfvql7mmxIjzN5MKCoa7SgD NKSGUjl2fBUfYGsnrzLYT/qMppmUG+weKtObRYp5ui5LVXBEw5ou72sd/jhuaYR9Y1Dec3oZ8HT D4dNRO9rv26Igtw== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 Currently, this is just a pointer to the most recent session, but there is no guarantee that the session is still valid and in memory. It's possible for this pointer go NULL or replaced. First, embed a struct rcu in nfsd4_session and free it via free_rcu. Ensure that when clp->cl_cb_session pointer is changed, that it is done via RCU-safe methods. This will allow callbacks to access the cl_cb_session pointer safely via RCU. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 21 ++++++++++++++++++--- fs/nfsd/nfs4state.c | 11 +++++++++-- fs/nfsd/state.h | 3 ++- 3 files changed, 29 insertions(+), 6 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 50e468bdb8d4838b5217346dcc2bd0fec1765c1a..e55bf66a33d6efb56d2f75f0a49a60307e3807ac 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1122,6 +1122,7 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c }; struct rpc_clnt *client; const struct cred *cred; + int ret; if (clp->cl_minorversion == 0) { if (!clp->cl_cred.cr_principal && @@ -1137,7 +1138,9 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c } else { if (!conn->cb_xprt || !ses) return -EINVAL; - clp->cl_cb_session = ses; + if (!nfsd4_cb_get_session(ses)) + return -EINVAL; + rcu_assign_pointer(clp->cl_cb_session, ses); args.bc_xprt = conn->cb_xprt; args.prognumber = clp->cl_cb_session->se_cb_prog; args.protocol = conn->cb_xprt->xpt_class->xcl_ident | @@ -1148,13 +1151,15 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c client = rpc_create(&args); if (IS_ERR(client)) { trace_nfsd_cb_setup_err(clp, PTR_ERR(client)); - return PTR_ERR(client); + ret = PTR_ERR(client); + goto out_put_ses; } cred = get_backchannel_cred(clp, client, ses); if (!cred) { trace_nfsd_cb_setup_err(clp, -ENOMEM); rpc_shutdown_client(client); - return -ENOMEM; + ret = -ENOMEM; + goto out_put_ses; } if (clp->cl_minorversion != 0) @@ -1166,6 +1171,12 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c args.authflavor); rcu_read_unlock(); return 0; +out_put_ses: + if (clp->cl_minorversion != 0) { + rcu_assign_pointer(clp->cl_cb_session, NULL); + nfsd4_cb_put_session(ses); + } + return ret; } static void nfsd4_mark_cb_state(struct nfs4_client *clp, int newstate) @@ -1529,11 +1540,15 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb) * kill the old client: */ if (clp->cl_cb_client) { + struct nfsd4_session *ses; + trace_nfsd_cb_bc_shutdown(clp, cb); rpc_shutdown_client(clp->cl_cb_client); clp->cl_cb_client = NULL; put_cred(clp->cl_cb_cred); clp->cl_cb_cred = NULL; + ses = rcu_replace_pointer(clp->cl_cb_session, NULL, true); + nfsd4_cb_put_session(ses); } if (clp->cl_cb_conn.cb_xprt) { svc_xprt_put(clp->cl_cb_conn.cb_xprt); diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index db68fd579ff0454153537817ee3cca71303654b4..77b0338c50d7f75fe6d03659b2ed3188976debac 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -2182,7 +2182,7 @@ static void __free_session(struct nfsd4_session *ses) { free_session_slots(ses, 0); xa_destroy(&ses->se_slots); - kfree(ses); + kfree_rcu(ses, se_rcu); } static void free_session(struct nfsd4_session *ses) @@ -3285,7 +3285,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name, clp->cl_time = ktime_get_boottime_seconds(); copy_verf(clp, verf); memcpy(&clp->cl_addr, sa, sizeof(struct sockaddr_storage)); - clp->cl_cb_session = NULL; + rcu_assign_pointer(clp->cl_cb_session, NULL); clp->net = net; clp->cl_nfsd_dentry = nfsd_client_mkdir( nn, &clp->cl_nfsdfs, @@ -4253,6 +4253,13 @@ nfsd4_destroy_session(struct svc_rqst *r, struct nfsd4_compound_state *cstate, status = nfserr_wrong_cred; if (!nfsd4_mach_creds_match(ses->se_client, r)) goto out_put_session; + + /* + * Is this session the backchannel session? Count an extra + * reference if so. + */ + if (ses == rcu_access_pointer(ses->se_client->cl_cb_session)) + ref_held_by_me++; status = mark_session_dead_locked(ses, 1 + ref_held_by_me); if (status) goto out_put_session; diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 79d985d2a656e1a5b22a6a9c88f309515725e847..0faa367c9fa3280fa4a8a982f974804bb89f2035 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -354,6 +354,7 @@ struct nfsd4_session { u16 se_slot_gen; bool se_dead; u32 se_target_maxslots; + struct rcu_head se_rcu; }; /* formatted contents of nfs4_sessionid */ @@ -465,7 +466,7 @@ struct nfs4_client { #define NFSD4_CB_FAULT 3 int cl_cb_state; struct nfsd4_callback cl_cb_null; - struct nfsd4_session *cl_cb_session; + struct nfsd4_session __rcu *cl_cb_session; /* for all client information that callback code might need: */ spinlock_t cl_lock; From patchwork Wed Jan 29 23:20:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13954236 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1615B1E47CA; Wed, 29 Jan 2025 23:20:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192857; cv=none; b=WzQ5miC/aSIa1Up55dCVW+kyircGq6mk2GLPxmIflqyxLK0+dw9ijwOiufaYjXfF/ag3KauW2jtRl6UOnfIV7GYEtaAIzfwH0U0baBw2QChe8nUDPBzUc4K7WxAD5WC/jmi+PQ9yGBovsniBtTXbw5hJCpu1WXeokwf2QsAE1FQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192857; c=relaxed/simple; bh=yQ+thEBAqsSgndfez3KwbQk4kYB4c0YOyYwqV2evsgk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tS2whnru8fEydlXV1fkJtJTOhD0UiLLdilNr7KPH4LTheGuiXSK1OwqAPsHcpi5E3V0XWSF8GHYNInpTT8F8REp7wKBiZc2C1lrHkpQ7AF/XaNCdcIk7p07qy4j8L0aGRsxLXXO9b6UPnkEVGYkwgF06LjUYXkIR35Kv+3piqAs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XgPKWZZF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XgPKWZZF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36004C4CED1; Wed, 29 Jan 2025 23:20:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738192856; bh=yQ+thEBAqsSgndfez3KwbQk4kYB4c0YOyYwqV2evsgk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XgPKWZZFHCCf70nsoqKggpaiJSsB3KuG3svIVGrabB9CQUdRl9DQaiZVsx028qL1j LMVInpqAJovk/Cd8C9GqlLQucALYDAcAdaNJYYJMk6HEyhS8DZ/+RhDRzO7j5o8YCX hfcT2esakQKgcpspG+VangBxHKTwTtIIPHCWuVj7IVfKS8HPwQ/8iS/uqx3UTtcpWX 2gO8/wGyat65E8PW8QufaTQAc3lhcx8Oxf0W0envuprX30KHZnhcJTciiTSYiQIS1w XYpuS6X2QuGaSYML/jsXWJfg8aiRlSc2JKO8x9qh8zCtvnRuGG7iUWvdufcrhfsIt9 lHK6BKsMuGS/A== From: Jeff Layton Date: Wed, 29 Jan 2025 18:20:43 -0500 Subject: [PATCH v3 3/6] nfsd: add a cb_ses pointer to nfsd4_callback and use it instead of clp->cb_cb_session Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250129-nfsd-6-14-v3-3-506e71e39e6b@kernel.org> References: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> In-Reply-To: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=7124; i=jlayton@kernel.org; h=from:subject:message-id; bh=yQ+thEBAqsSgndfez3KwbQk4kYB4c0YOyYwqV2evsgk=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnmrfUhuAxs1o8Iou6bEixCslvz95wtP0VQ+ngC /pb4MkfNLCJAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ5q31AAKCRAADmhBGVaC FaMyD/sHWNUs1UNFM+v57UebnsthnWP1ay5AEOHNriLUvNrWJO93B3wepZ8MLu4xt3KLzkb2358 SyIHyL3ZJHqc81O+m2BnSILjW6Q1oUJRKH2dAFPOnMfJpEIhLmGO1ho0BoRaAC4OMYQ47oqknYz hmkJhx33TIvLqxiSFz8nA84kIKJOkIR4QJi3K0BF+0hy8MO4dp180bBp/bGKwZ5gq4ot1b288hK 2xtRu96hhC7LuByIiR+kLWK3zW5OUDGn1pHHuxCatMQ4i1fJuQ6dwcviIQNTZ4BQnzu0NI5rELD HzokmweFEcs/mdswru6KzlGs8IxW5+nGqQj1abxivZ+dUel+KfGKMgMZxpzHsIfxhom8PnDhBXW dhWwcf5Wb5eqLWB5QMruODgK3jFX7gD6LmzeLj9+GPPbjIQYd5R6M0e3LxoA4FGQVPAwdZrOP/V zhmCVJhZ6wmU6UOJ3C5g8/w01rlP0Oz/8r6YMTW2TPA7BPfbqpbB0p3fp9hJzDkkPJv+Mdupy0D Y0liPZQkDCYm6mydjNunAjbKof4e78rap+bkbiYPuasl1mVzd6rKuyTY/KD8xKfNXOytf8TVABm YBxvO0150BJWXSfh/dsPj5VMlNx0QlWDCzs8C2qblisne3nTLEMQ1+pfxCfEtIQA8tPSVwl+dFh ewdzjRspGR4L0yw== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 Once a callback workqueue job has completed, the cl_cb_session could change to a completely different session, if there is more than one callback in flight at a time. Have the callback hold its own cb reference to the session, and fix the slot acquisition routines to get/put a session reference. This ensures that the call and reply handling are working with the same session. In the event that rpc_prepare can't get a session reference, allow the rpc_task to sleep until the session reference changes. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 67 +++++++++++++++++++++++++++++++++++++++++--------- fs/nfsd/state.h | 1 + fs/nfsd/trace.h | 6 ++--- 3 files changed, 58 insertions(+), 16 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index e55bf66a33d6efb56d2f75f0a49a60307e3807ac..9f4838a6d9c668cdf66a77793f63c064586f2b22 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -435,7 +435,7 @@ static void encode_cb_sequence4args(struct xdr_stream *xdr, const struct nfsd4_callback *cb, struct nfs4_cb_compound_hdr *hdr) { - struct nfsd4_session *session = cb->cb_clp->cl_cb_session; + struct nfsd4_session *session = cb->cb_ses; __be32 *p; if (hdr->minorversion == 0) @@ -503,7 +503,7 @@ static void update_cb_slot_table(struct nfsd4_session *ses, u32 target) static int decode_cb_sequence4resok(struct xdr_stream *xdr, struct nfsd4_callback *cb) { - struct nfsd4_session *session = cb->cb_clp->cl_cb_session; + struct nfsd4_session *session = cb->cb_ses; int status = -ESERVERFAULT; __be32 *p; u32 seqid, slotid, target; @@ -1142,7 +1142,7 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c return -EINVAL; rcu_assign_pointer(clp->cl_cb_session, ses); args.bc_xprt = conn->cb_xprt; - args.prognumber = clp->cl_cb_session->se_cb_prog; + args.prognumber = ses->se_cb_prog; args.protocol = conn->cb_xprt->xpt_class->xcl_ident | XPRT_TRANSPORT_BC; args.authflavor = ses->se_cb_sec.flavor; @@ -1161,9 +1161,10 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c ret = -ENOMEM; goto out_put_ses; } - - if (clp->cl_minorversion != 0) + if (clp->cl_minorversion != 0) { clp->cl_cb_conn.cb_xprt = conn->cb_xprt; + rpc_wake_up(&clp->cl_cb_waitq); + } clp->cl_cb_client = client; clp->cl_cb_cred = cred; rcu_read_lock(); @@ -1252,6 +1253,34 @@ void nfsd4_change_callback(struct nfs4_client *clp, struct nfs4_cb_conn *conn) spin_unlock(&clp->cl_lock); } +static bool grab_cb_ses(struct nfsd4_callback *cb) +{ + struct nfs4_client *clp = cb->cb_clp; + struct nfsd4_session *ses; + bool ret = false; + + if (cb->cb_ses) + return true; + + rcu_read_lock(); + ses = rcu_dereference(clp->cl_cb_session); + if (ses && nfsd4_cb_get_session(ses)) { + cb->cb_ses = ses; + ret = true; + } + rcu_read_unlock(); + + return ret; +} + +static void put_cb_ses(struct nfsd4_callback *cb) +{ + if (cb->cb_ses) { + nfsd4_cb_put_session(cb->cb_ses); + cb->cb_ses = NULL; + } +} + static int grab_slot(struct nfsd4_session *ses) { int idx; @@ -1269,24 +1298,33 @@ static int grab_slot(struct nfsd4_session *ses) } /* - * There's currently a single callback channel slot. - * If the slot is available, then mark it busy. Otherwise, set the - * thread for sleeping on the callback RPC wait queue. + * Get both a session reference and a slot. */ static bool nfsd41_cb_get_slot(struct nfsd4_callback *cb, struct rpc_task *task) { struct nfs4_client *clp = cb->cb_clp; - struct nfsd4_session *ses = clp->cl_cb_session; + struct nfsd4_session *ses; + + if (!grab_cb_ses(cb)) { + rpc_sleep_on(&clp->cl_cb_waitq, task, NULL); + if (!grab_cb_ses(cb)) + return false; + rpc_wake_up_queued_task(&clp->cl_cb_waitq, task); + } if (cb->cb_held_slot >= 0) return true; + + ses = cb->cb_ses; cb->cb_held_slot = grab_slot(ses); if (cb->cb_held_slot < 0) { rpc_sleep_on(&clp->cl_cb_waitq, task, NULL); /* Race breaker */ cb->cb_held_slot = grab_slot(ses); - if (cb->cb_held_slot < 0) + if (cb->cb_held_slot < 0) { + put_cb_ses(cb); return false; + } rpc_wake_up_queued_task(&clp->cl_cb_waitq, task); } return true; @@ -1295,7 +1333,10 @@ static bool nfsd41_cb_get_slot(struct nfsd4_callback *cb, struct rpc_task *task) static void nfsd41_cb_release_slot(struct nfsd4_callback *cb) { struct nfs4_client *clp = cb->cb_clp; - struct nfsd4_session *ses = clp->cl_cb_session; + struct nfsd4_session *ses = cb->cb_ses; + + if (!ses) + return; if (cb->cb_held_slot >= 0) { spin_lock(&ses->se_lock); @@ -1304,6 +1345,7 @@ static void nfsd41_cb_release_slot(struct nfsd4_callback *cb) cb->cb_held_slot = -1; rpc_wake_up_next(&clp->cl_cb_waitq); } + put_cb_ses(cb); } static void nfsd41_destroy_cb(struct nfsd4_callback *cb) @@ -1342,7 +1384,7 @@ static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata) static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback *cb) { struct nfs4_client *clp = cb->cb_clp; - struct nfsd4_session *session = clp->cl_cb_session; + struct nfsd4_session *session = cb->cb_ses; bool ret = true; if (!clp->cl_minorversion) { @@ -1629,6 +1671,7 @@ void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp, const struct nfsd4_callback_ops *ops, enum nfsd4_cb_op op) { cb->cb_clp = clp; + cb->cb_ses = NULL; cb->cb_msg.rpc_proc = &nfs4_cb_procedures[op]; cb->cb_msg.rpc_argp = cb; cb->cb_msg.rpc_resp = cb; diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 0faa367c9fa3280fa4a8a982f974804bb89f2035..56fe34d8dd90344404512114113c00a027aeb6a4 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -66,6 +66,7 @@ typedef struct { struct nfsd4_callback { struct nfs4_client *cb_clp; + struct nfsd4_session *cb_ses; struct rpc_message cb_msg; const struct nfsd4_callback_ops *cb_ops; struct work_struct cb_work; diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index ad2c0c432d08705bcebf00f7309f19267afcae03..fff665bac3b252387f92139b5f868cf1b034d1c9 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -1644,8 +1644,7 @@ TRACE_EVENT(nfsd_cb_seq_status, __field(int, seq_status) ), TP_fast_assign( - const struct nfs4_client *clp = cb->cb_clp; - const struct nfsd4_session *session = clp->cl_cb_session; + const struct nfsd4_session *session = cb->cb_ses; const struct nfsd4_sessionid *sid = (struct nfsd4_sessionid *)&session->se_sessionid; @@ -1684,8 +1683,7 @@ TRACE_EVENT(nfsd_cb_free_slot, __field(u32, slot_seqno) ), TP_fast_assign( - const struct nfs4_client *clp = cb->cb_clp; - const struct nfsd4_session *session = clp->cl_cb_session; + const struct nfsd4_session *session = cb->cb_ses; const struct nfsd4_sessionid *sid = (struct nfsd4_sessionid *)&session->se_sessionid; From patchwork Wed Jan 29 23:20:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13954237 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62BF21E7640; Wed, 29 Jan 2025 23:20:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192858; cv=none; b=C99sWkMDO9AuCjHqzbAdhBg8bxAL9UflygYcMqC5HvTps/j/WRcnfuuAYO4/zei07I6eoPnjON52lv/PnLsbmq9e4G6Oah+USfxlzqlgdfuO/OYi2nrxBcvO5Rg/HEVyNBK9vizkssQB/i4F731c9axFjyg3k+SUpABBTjzIQv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192858; c=relaxed/simple; bh=QhwaF3Zpw2Ay8YmeGNGfXktwX3M8UrnOUWFD85Hkp/0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BtRjswEWhQw66oQ4p+ot/BC4GAsk+dVrNjbGA59TMxI67O0GYTtUMW/uUqDBr3HPELZrjYHcAxZtDd+q9tiZuOvi75FFKel/hkhThTCPh6LNUCMiTb9uItpo8HU0JORAoQff5j0isJ41fI1XroZFU09w0C9EC9XiQhExi2kwpt4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZkCZcX80; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZkCZcX80" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2FB96C4CED3; Wed, 29 Jan 2025 23:20:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738192857; bh=QhwaF3Zpw2Ay8YmeGNGfXktwX3M8UrnOUWFD85Hkp/0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ZkCZcX80nuvAsTLretbvNn4+TDFmewSTb38E51fuVVpt0B3q9HGE3SG90RUmDP8jj 2RxHNoaDD7Rh86VGwdRv01XcgCOSXuU4G3lcHop/W2EcWvu4QRJ+MOsAu5Emg9LyRV tihxmP50XtpVNM8aYfbmYd/t8ArwK1NaxZNbrDO/07APj9IknNHG3A2ty+6BUv2hfP 2pyC1yMzwof2A+yTylA9Y/ISjdOw4cQydLDBYGThKf74FvqDxquysiUgI1gh52+Dq+ cRjLELsPeS0i2+KyjNr/b1/PR7oNE4Hx7famQmZdrVjjIiRP8DeR91xhMNfMFLMQGw Oa8RYMMmt4nAw== From: Jeff Layton Date: Wed, 29 Jan 2025 18:20:44 -0500 Subject: [PATCH v3 4/6] nfsd: overhaul CB_SEQUENCE error handling Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250129-nfsd-6-14-v3-4-506e71e39e6b@kernel.org> References: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> In-Reply-To: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5263; i=jlayton@kernel.org; h=from:subject:message-id; bh=QhwaF3Zpw2Ay8YmeGNGfXktwX3M8UrnOUWFD85Hkp/0=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnmrfUO58CW3cQ0DoQ5miTyBByZVB35/TTJyuh0 2Sge0l7Fg6JAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ5q31AAKCRAADmhBGVaC FYP0D/9RI8DDcxgXCx8EGXbsAsFoO5pCo0tIDsuWBfbPwGo0gjfUGTSokvOwc+Ls3LHcwcJzsZ2 alv0NZjQ9dYEZPHyhbb5PwuWSHjNDATGB3XJpvNhflX0aTr69ViPwC6rwlP5pAOrgT5ZCrOGEVv Ecbag9YkmWbYsUMBbbAhQEcfjNy91s+BijMwlke8i0LU2FNqjGrTgH1OmEGOFiE0Ft4JL3PSOR2 SthsI+JIuzdNuWsAf2VrgoTBTFr0YpUqPESFrVHfIJYcluFWM9PeQzAAHcLglGgxpCgGIV8Hsl3 SJJAu86xLMtG0Trovm03KJ+GdKpeIa2evCgbYOVz2U4Jd13Q2E0yUny/hfWk1Ij4qEjpMDVEXQ8 IdWHk+37HB+aKAgO7B9kVPAZ1vySNvrDkXXjZMq1+9JvBmgyqjtoiU2iumQQLOIrpNRZDgnjPaj ollkoTuZAcfCtTc3x/V/RAWAX00n7KBzdSPib9twSROtn7igEEKQjMBoGOCNdlFpd8xmayIheQI FXFo+4S9WbxjuSl1L7Zuwzv39vSbkAWiDl34CfKs7AIiwm0AhcCXoIccWbv9dHKY4KlqeYD0LGu TsZiuNNV9oDBRt+vk9fmymVk74WZ10M7hHAEXEQQoYr7vTkdIsLTEU2MrWq/rk3uCzIia9B66b2 ghwV6O0oBOh6TxQ== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 Some of the existing error handling in nfsd4_cb_sequence_done is incorrect. This patch first does some general cleanup and then reworks some of the error handling cases. There is only one case where we want to proceed with processing the rest of the CB_COMPOUND, and that's when the cb_seq_status is 0. Make the default return value be false, and only set it to true in that case. Now that there is a clear record of the session used to handle the CB_COMPOUND, we can take changes to the cl_cb_session into account when reattempting a call. When restarting the RPC (but not the entire callback), test RPC_SIGNALLED(). If it has been, then fall through to restart the callback instead of just restarting the RPC. Whenever restarting the entire callback, release the slot and session, in case there have been changes in the interim. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 85 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 61 insertions(+), 24 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 9f4838a6d9c668cdf66a77793f63c064586f2b22..109e8f872eaf2227ee30a1df3df86ab1ad88c384 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1381,11 +1381,16 @@ static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata) rpc_call_start(task); } +static bool nfsd4_cb_session_changed(struct nfsd4_callback *cb) +{ + return cb->cb_ses != rcu_access_pointer(cb->cb_clp->cl_cb_session); +} + static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback *cb) { struct nfs4_client *clp = cb->cb_clp; struct nfsd4_session *session = cb->cb_ses; - bool ret = true; + bool ret = false; if (!clp->cl_minorversion) { /* @@ -1398,13 +1403,13 @@ static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback * handle that case here. */ if (RPC_SIGNALLED(task)) - goto need_restart; + goto requeue; return true; } if (cb->cb_held_slot < 0) - goto need_restart; + goto requeue; /* This is the operation status code for CB_SEQUENCE */ trace_nfsd_cb_seq_status(task, cb); @@ -1418,11 +1423,15 @@ static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback * (sequence ID, cached reply) MUST NOT change. */ ++session->se_cb_seq_nr[cb->cb_held_slot]; + ret = true; break; case -ESERVERFAULT: - ++session->se_cb_seq_nr[cb->cb_held_slot]; + /* + * Call succeeded, but CB_SEQUENCE reply failed sanity checks. + * The client has gone insane. Mark the BC faulty, since there + * isn't much else we can do. + */ nfsd4_mark_cb_fault(cb->cb_clp); - ret = false; break; case 1: /* @@ -1433,39 +1442,67 @@ static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback */ fallthrough; case -NFS4ERR_BADSESSION: - nfsd4_mark_cb_fault(cb->cb_clp); - ret = false; - goto need_restart; + /* + * If the session hasn't changed, mark it faulty. Restart + * the call. + */ + if (!nfsd4_cb_session_changed(cb)) + nfsd4_mark_cb_fault(cb->cb_clp); + goto requeue; case -NFS4ERR_DELAY: + /* + * If the rpc_clnt is being torn down, then we must restart + * the call from scratch. + */ + if (RPC_SIGNALLED(task)) + goto requeue; cb->cb_seq_status = 1; - if (!rpc_restart_call(task)) - goto out; - + rpc_restart_call(task); rpc_delay(task, 2 * HZ); return false; - case -NFS4ERR_BADSLOT: - goto retry_nowait; case -NFS4ERR_SEQ_MISORDERED: - if (session->se_cb_seq_nr[cb->cb_held_slot] != 1) { - session->se_cb_seq_nr[cb->cb_held_slot] = 1; + /* + * Reattempt once with seq_nr 1. If that fails, treat this + * like BADSLOT. + */ + if (!nfsd4_cb_session_changed(cb)) { + if (session->se_cb_seq_nr[cb->cb_held_slot] != 1) { + session->se_cb_seq_nr[cb->cb_held_slot] = 1; + goto retry_nowait; + } + } + fallthrough; + case -NFS4ERR_BADSLOT: + /* + * BADSLOT means that the client and server are out of sync + * on the BC parameters. In this case, we want to mark the + * backchannel faulty and then try it again, but _leak_ the + * slot so no one uses it. If the callback session has + * changed since then though, don't mark it. + */ + if (!nfsd4_cb_session_changed(cb)) { + nfsd4_mark_cb_fault(cb->cb_clp); + cb->cb_held_slot = -1; goto retry_nowait; } - break; + goto requeue; default: nfsd4_mark_cb_fault(cb->cb_clp); } trace_nfsd_cb_free_slot(task, cb); nfsd41_cb_release_slot(cb); - - if (RPC_SIGNALLED(task)) - goto need_restart; -out: return ret; retry_nowait: - if (rpc_restart_call_prepare(task)) - ret = false; - goto out; -need_restart: + /* + * RPC_SIGNALLED() means that the rpc_client is being torn down and + * (possibly) recreated. Restart the call completely in that case. + */ + if (!RPC_SIGNALLED(task)) { + rpc_restart_call_prepare(task); + return false; + } +requeue: + nfsd41_cb_release_slot(cb); if (!test_bit(NFSD4_CLIENT_CB_KILL, &clp->cl_flags)) { trace_nfsd_cb_restart(clp, cb); task->tk_status = 0; From patchwork Wed Jan 29 23:20:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13954238 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 768E41E9904; Wed, 29 Jan 2025 23:20:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192859; cv=none; b=Pt1ZLsZxStvn9Fh2RHhVeBd/4NmsMGikE8cjXmQDNV49StOeKy2+Xt21ubO2R7ONA7qaKmHVdnzHfQfY90IwOZG/WCCVklIknplPMW3I9389Tjsb/l4I7j0kY5/qz75IoUuIpj+6oVFjqjUFNoaxVOw7fXsqujiwXFm22OwOVfA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192859; c=relaxed/simple; bh=KLezOvtJaSEl8tcY+wBxmhbYNw75f3c7qDVJYdTgOrQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PZbAe9K1GgbqdjDTLIQJ19NaDwpjo3g7wbckUAGimc1AZWSPiCGyRWjh6CBricU66/iVekkQMVB7FN3+qYQJKmddG/p+qz+peNKWlOmXMoYm0qvWxLJFLyH9dvRDo257+CHQqqNobKNVXDa5ZeMYWvAlF2iG7ikAzI5jE5r/6ww= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hK8XqOna; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hK8XqOna" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BB2FC4CEE7; Wed, 29 Jan 2025 23:20:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738192858; bh=KLezOvtJaSEl8tcY+wBxmhbYNw75f3c7qDVJYdTgOrQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hK8XqOnaV2z8TV0lR03BzO41BXnfKInPKdVpxqGKNqOMkUw0DA75gDh8vEyPIqqWG UXf57GzoJtwWtrBhd7v6+CP9Z3B76UXPa/PDN9Fp9K+yz0AEzEtDzbUAz6b2OIdYEl oE2PFJvHGNfA6988N0DUGv2YxcEjNaPDcld5j+LYEYWbOKiZvzDCpOuLjZpP3ICHFL aiWcem7bJc+ci7vKVBelLv1/8xRn28zynhlDU8NXk5ujqd3p0/MSiwW4rozcwHgyOu N1K7cwEiikYZW6EL8elWRwCwyA43sSyV1Ts1QcAMx5wxvaW0gNiaMn0ul9JQ5zXarp bm1+pfyhoKLgA== From: Jeff Layton Date: Wed, 29 Jan 2025 18:20:45 -0500 Subject: [PATCH v3 5/6] nfsd: remove unneeded forward declaration of nfsd4_mark_cb_fault() Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250129-nfsd-6-14-v3-5-506e71e39e6b@kernel.org> References: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> In-Reply-To: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=613; i=jlayton@kernel.org; h=from:subject:message-id; bh=KLezOvtJaSEl8tcY+wBxmhbYNw75f3c7qDVJYdTgOrQ=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnmrfUkAiR05FUARY271IxLnDvsNCkLPEDdxPsb ZElzD+DaD+JAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ5q31AAKCRAADmhBGVaC FYPcD/4mpgQyet/q4HeP5RLpmvw6VAxHIy14AxUu4Uj4sAM6ed6jJmDoE/3WHS8BTGEBRTFSxvJ ShRy0dDO+eWtLKpFCiyNulrwwxaMLVj/3dgUqW0EiBJIuhlymPsw6fvxSJgkhyeEx39YCe1I5cw kNH6hdFB263DAyUFtBAN3zMsOZmA/EUtocQanNhzPUULe6vdZHhZ1zazKCLNf5FQnl0pcbLRmFo oZ6+u9Epkt94CjT7oZf/GTxq+/QgZZ8NEDctBuhCZlBzqCMRW1sSUlm7C1x3aneJKoSjdDetuzv cNag4riAEQlT3mYfTAp2mp8Nie2BPqkQEEi8CzHf2a9lT8XPD56u9XE9GGGwM71pgzbG7J3g8nS J5yVLdPzHGlvGJQ0j8XV5g16HWntijL6tCdBPc5DQ9ld4GGQINkN2aNMFsmAQ65y0zTb3/dtmQ6 2bd5zf/CT0O9COxRvFLEL8+rSwjNhhQUD9undfRIiUCZWnIIQyxX7DEnCFD5JML5O78qtcvb39B 7NZfGIVMuD37cWE39b2FDh26I+LwOaXF0BNcls2S8GsIsb985y92PObmXN1+uyC7ZQLLJlIDpwu ZizAKlXA8heUJ7XMjYYDEUpVYv7h/tvLpdb/JddOTTkJV0qki9OLBc9XHx2NZHhYGfgKn4g1OiJ cRO0pKgsymanNNA== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 This isn't needed. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 109e8f872eaf2227ee30a1df3df86ab1ad88c384..c8834454bcc7b5044de42ec8622653a44dbb0812 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -45,9 +45,6 @@ #include "nfs4xdr_gen.h" #define NFSDDBG_FACILITY NFSDDBG_PROC - -static void nfsd4_mark_cb_fault(struct nfs4_client *clp); - #define NFSPROC4_CB_NULL 0 #define NFSPROC4_CB_COMPOUND 1 From patchwork Wed Jan 29 23:20:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 13954239 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0393E1E9B0C; Wed, 29 Jan 2025 23:20:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192860; cv=none; b=pJA5qUjcxQxqlsmvE6JeHQpB/TBH6mgS7JCj4ijgUZzfrSpvNwsCRlRk9hT6qGw6ay/o2hUTnI4zjV2zoot5Yh6bD+xdJogZt562yuHYrC0XIP7d7NsI3lbXGie4zQ9r7PU2WucGn+Y6lKYm3MFalu6sPcYR3VM9kknENDYVu8M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738192860; c=relaxed/simple; bh=SrBrOxdAiSSWt1qo1usHtLL4Asei1ZG+/Lci0b5nwTQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Jl49mvusyOmjJgWCCcpFKPXM5K6KATfKLT7u+1QQOFlsgUNtZc/GzbJjEuTpT1s3DqcfY/F8ZY0bXHV3TYYNhnIOEbE+QTNWFy19KUYrwk0vtUJnSR/PQEHe4Zb8A0WpJsh3jpoPPLS/zKFC+9m0eRXLREfNSDw4yjljAETUJB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SDz1mBkE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SDz1mBkE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24E95C4CEE0; Wed, 29 Jan 2025 23:20:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738192859; bh=SrBrOxdAiSSWt1qo1usHtLL4Asei1ZG+/Lci0b5nwTQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SDz1mBkEdjN8LJnc518/r3Y4zbtqBJbQZuZv58YnDlG7985kNEpg2AG6DsxYGnScH YfMQyyKUraESCFdoUSjutGce9MS0m77v6l5tEQICPI9z9tm/9BlJXuomzYDgTp5j0u wxM1f1kwcxTyr4CRBhc60m98Q5q3ZgQoIuqu/7SVvvBSDYCzhpuiAEV3geBbPnQtz/ nvb3/Y4kpYnYd6a5GH4D3+dpewSxwm0jmpqM5CY+lv80ovOguN1r76o/x3y/1XJDe3 9/K9SpIMRxP+LR1xFmoBAp2RcU3DCeAkhf6sTqw76VLrRDAJmDbw7/DAp77llicIBv E6mQcIsilYUWQ== From: Jeff Layton Date: Wed, 29 Jan 2025 18:20:46 -0500 Subject: [PATCH v3 6/6] nfsd: lift NFSv4.0 handling out of nfsd4_cb_sequence_done() Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250129-nfsd-6-14-v3-6-506e71e39e6b@kernel.org> References: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> In-Reply-To: <20250129-nfsd-6-14-v3-0-506e71e39e6b@kernel.org> To: Chuck Lever , Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, Jeff Layton X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3342; i=jlayton@kernel.org; h=from:subject:message-id; bh=SrBrOxdAiSSWt1qo1usHtLL4Asei1ZG+/Lci0b5nwTQ=; b=owEBbQKS/ZANAwAIAQAOaEEZVoIVAcsmYgBnmrfURknfUWccL+v4OnFGNFglYkLRu+jSZBu1p wPN5iqzYd2JAjMEAAEIAB0WIQRLwNeyRHGyoYTq9dMADmhBGVaCFQUCZ5q31AAKCRAADmhBGVaC FZ8SD/oCD5oHtaaQQ2OuTW5FWaDSUgPtiHyfqzIC23g9Dq0BHrMTYepPuRrTwM2BBWnD0J+CV/W WJpAvO60ZyLh9hSeF9sV/UMqhaDGCB/VhjiZrd8HXGuRKrnXv0ROoaGMNEc5OiCnDu7jz3ua7hv h7JU1xzHGx+yjTcWEnMKt/Jhe/bfI2OTm+4VZDMmxOeIvLXIiSfreKjs4LcfA86tvagW5Q/vIV2 Wnk7ldklqW2XF7DcBG33RhVOY9kkimVn5tm11u3dDqORPR44ikgTnPKBqI5eZhVGRVzAQ+2yT/c wBvpj/Kcp7qNiivGE+IsLVCHg3xQnS9g+zgAiHwwtP6h0eEWa4DnyVnF1ZbNPhpi2BGto6s0BCJ ZtD7URMVP1t4ZifYDsXu5yf24FbAlyfKDDL9EIksyU7y/anC/n4yYQfKRpARa7vbjYMxw2Bl2O1 VCoHZZT/rez0rz+JSruM/Lc4i0jJzbIcLRgD8C38MeqiJMJEkRFoIeIyIeDPJgVQg9ZGbPrWTse 683oxsDzyQLigDgYkZAXF0/NT6YBEc43hOVoZAth1Kmkwa+9JwzZLRsrzZpB9nowKYY3E02sjsf VfvsAbPk2VAH3vcSR1VYJRjF41mdkUpyAkDjHuWJmOR6gYU95Scj5Koe5l4Bxc7le5iCxMcTFDI 7uKFsJqu7hZG2ZA== X-Developer-Key: i=jlayton@kernel.org; a=openpgp; fpr=4BC0D7B24471B2A184EAF5D3000E684119568215 It's a bit strange to call nfsd4_cb_sequence_done() on a callback with no CB_SEQUENCE. Lift the handling of restarting a call into a new helper, and move the handling of NFSv4.0 into nfsd4_cb_done(). Signed-off-by: Jeff Layton --- fs/nfsd/nfs4callback.c | 53 ++++++++++++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 25 deletions(-) diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index c8834454bcc7b5044de42ec8622653a44dbb0812..bf7248f84913ebd9e8942195b24191e494027dbf 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1383,27 +1383,22 @@ static bool nfsd4_cb_session_changed(struct nfsd4_callback *cb) return cb->cb_ses != rcu_access_pointer(cb->cb_clp->cl_cb_session); } -static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback *cb) +static void requeue_callback(struct rpc_task *task, struct nfsd4_callback *cb) { struct nfs4_client *clp = cb->cb_clp; - struct nfsd4_session *session = cb->cb_ses; - bool ret = false; - if (!clp->cl_minorversion) { - /* - * If the backchannel connection was shut down while this - * task was queued, we need to resubmit it after setting up - * a new backchannel connection. - * - * Note that if we lost our callback connection permanently - * the submission code will error out, so we don't need to - * handle that case here. - */ - if (RPC_SIGNALLED(task)) - goto requeue; - - return true; + nfsd41_cb_release_slot(cb); + if (!test_bit(NFSD4_CLIENT_CB_KILL, &clp->cl_flags)) { + trace_nfsd_cb_restart(clp, cb); + task->tk_status = 0; + cb->cb_need_restart = true; } +} + +static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback *cb) +{ + struct nfsd4_session *session = cb->cb_ses; + bool ret = false; if (cb->cb_held_slot < 0) goto requeue; @@ -1492,19 +1487,14 @@ static bool nfsd4_cb_sequence_done(struct rpc_task *task, struct nfsd4_callback retry_nowait: /* * RPC_SIGNALLED() means that the rpc_client is being torn down and - * (possibly) recreated. Restart the call completely in that case. + * (possibly) recreated. Restart the whole callback in that case. */ if (!RPC_SIGNALLED(task)) { rpc_restart_call_prepare(task); return false; } requeue: - nfsd41_cb_release_slot(cb); - if (!test_bit(NFSD4_CLIENT_CB_KILL, &clp->cl_flags)) { - trace_nfsd_cb_restart(clp, cb); - task->tk_status = 0; - cb->cb_need_restart = true; - } + requeue_callback(task, cb); return false; } @@ -1515,8 +1505,21 @@ static void nfsd4_cb_done(struct rpc_task *task, void *calldata) trace_nfsd_cb_rpc_done(clp); - if (!nfsd4_cb_sequence_done(task, cb)) + if (!clp->cl_minorversion) { + /* + * If the backchannel connection was shut down while this + * task was queued, we need to resubmit it after setting up + * a new backchannel connection. + * + * Note that if we lost our callback connection permanently + * the submission code will error out, so we don't need to + * handle that case here. + */ + if (RPC_SIGNALLED(task)) + requeue_callback(task, cb); + } else if (!nfsd4_cb_sequence_done(task, cb)) { return; + } if (cb->cb_status) { WARN_ONCE(task->tk_status,