| Message ID | 20240614141851.97723-2-cel@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | SUNRPC: Fix backchannel reply, again | expand |
On Fri, Jun 14, 2024 at 10:18:52AM -0400, cel@kernel.org wrote: > From: Chuck Lever <chuck.lever@oracle.com> > > I still see "RPC: Could not send backchannel reply error: -110" > quite often, along with slow-running tests. Debugging shows that the > backchannel is still stumbling when it has to queue a callback reply > on a busy transport. > > Note that every one of these timeouts causes a connection loss by > virtue of the xprt_conditional_disconnect() call in that arm of > call_cb_transmit_status(). > > I found that setting to_maxval is necessary to get the RPC timeout > logic to behave whenever to_exponential is not set. > > Fixes: 57331a59ac0d ("NFSv4.1: Use the nfs_client's rpc timeouts for backchannel") > Signed-off-by: Chuck Lever <chuck.lever@oracle.com> > --- > net/sunrpc/svc.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c > index 965a27806bfd..f4ddb2961042 100644 > --- a/net/sunrpc/svc.c > +++ b/net/sunrpc/svc.c > @@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp) > timeout.to_initval = req->rq_xprt->timeout->to_initval; > timeout.to_retries = req->rq_xprt->timeout->to_retries; > } > + timeout.to_maxval = timeout.to_initval; > memcpy(&req->rq_snd_buf, &rqstp->rq_res, sizeof(req->rq_snd_buf)); > task = rpc_run_bc_task(req, &timeout); > > -- > 2.45.1 > Hi - would love to see this in 6.10-rc. Is there a chance that could happen?
On Fri, 2024-06-14 at 10:18 -0400, cel@kernel.org wrote: > From: Chuck Lever <chuck.lever@oracle.com> > > I still see "RPC: Could not send backchannel reply error: -110" > quite often, along with slow-running tests. Debugging shows that the > backchannel is still stumbling when it has to queue a callback reply > on a busy transport. > > Note that every one of these timeouts causes a connection loss by > virtue of the xprt_conditional_disconnect() call in that arm of > call_cb_transmit_status(). > > I found that setting to_maxval is necessary to get the RPC timeout > logic to behave whenever to_exponential is not set. > > Fixes: 57331a59ac0d ("NFSv4.1: Use the nfs_client's rpc timeouts for > backchannel") > Signed-off-by: Chuck Lever <chuck.lever@oracle.com> > --- > net/sunrpc/svc.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c > index 965a27806bfd..f4ddb2961042 100644 > --- a/net/sunrpc/svc.c > +++ b/net/sunrpc/svc.c > @@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, > struct svc_rqst *rqstp) > timeout.to_initval = req->rq_xprt->timeout- > >to_initval; > timeout.to_retries = req->rq_xprt->timeout- > >to_retries; > } > + timeout.to_maxval = timeout.to_initval; > memcpy(&req->rq_snd_buf, &rqstp->rq_res, sizeof(req- > >rq_snd_buf)); > task = rpc_run_bc_task(req, &timeout); > Reviewed-by: Jeff Layton <jlayton@kernel.org>
On Tue, 2024-06-18 at 16:31 -0400, Chuck Lever wrote: > On Fri, Jun 14, 2024 at 10:18:52AM -0400, cel@kernel.org wrote: > > From: Chuck Lever <chuck.lever@oracle.com> > > > > I still see "RPC: Could not send backchannel reply error: -110" > > quite often, along with slow-running tests. Debugging shows that > > the > > backchannel is still stumbling when it has to queue a callback > > reply > > on a busy transport. > > > > Note that every one of these timeouts causes a connection loss by > > virtue of the xprt_conditional_disconnect() call in that arm of > > call_cb_transmit_status(). > > > > I found that setting to_maxval is necessary to get the RPC timeout > > logic to behave whenever to_exponential is not set. > > > > Fixes: 57331a59ac0d ("NFSv4.1: Use the nfs_client's rpc timeouts > > for backchannel") > > Signed-off-by: Chuck Lever <chuck.lever@oracle.com> > > --- > > net/sunrpc/svc.c | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c > > index 965a27806bfd..f4ddb2961042 100644 > > --- a/net/sunrpc/svc.c > > +++ b/net/sunrpc/svc.c > > @@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, > > struct svc_rqst *rqstp) > > timeout.to_initval = req->rq_xprt->timeout- > > >to_initval; > > timeout.to_retries = req->rq_xprt->timeout- > > >to_retries; > > } > > + timeout.to_maxval = timeout.to_initval; > > memcpy(&req->rq_snd_buf, &rqstp->rq_res, sizeof(req- > > >rq_snd_buf)); > > task = rpc_run_bc_task(req, &timeout); > > > > -- > > 2.45.1 > > > > Hi - would love to see this in 6.10-rc. Is there a chance that > could happen? Hmm... Can we please also set the remaining fields in timeout to 0? Otherwise, we're still playing roulette with what actually ends up happening in xprt_calc_majortimeo(). If to_increment happens to be large enough, we could overflow and end up with a silly small timeout value on a retry.
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 965a27806bfd..f4ddb2961042 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp) timeout.to_initval = req->rq_xprt->timeout->to_initval; timeout.to_retries = req->rq_xprt->timeout->to_retries; } + timeout.to_maxval = timeout.to_initval; memcpy(&req->rq_snd_buf, &rqstp->rq_res, sizeof(req->rq_snd_buf)); task = rpc_run_bc_task(req, &timeout);