[2] SUNRPC: Fix a slow server-side memory leak with RPC-over-TCP

Commit Message

Chuck Lever April 3, 2024, 2:36 p.m. UTC

From: Chuck Lever <[e-mail]>

Jan Schunk reports that his small NFS servers suffer from memory
exhaustion after just a few days. A bisect shows that commit
e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single
sock_sendmsg() call") is the first bad commit.

That commit assumed that sock_sendmsg() releases all the pages in
the underlying bio_vec array, but the reality is that it doesn't.
svc_xprt_release() releases the rqst's response pages, but the
record marker page fragment isn't one of those, so it is never
released.

This is a narrow fix that can be applied to stable kernels. A
more extensive fix is in the works.

Reported-by: Jan Schunk <[e-mail]>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=218671
Fixes: e18e157bb5c8 ("SUNRPC: Send RPC message on TCP with a single sock_sendmsg() call")
Cc: Alexander Duyck <[e-mail]>
Cc: Jakub Kacinski <[e-mail]>
Cc: David Howells <[e-mail]>
Signed-off-by: Chuck Lever <[e-mail]>
---
 net/sunrpc/svcsock.c |   10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

Changes since RFC:
- It's safe to release the fragment as soon as sock_sendmsg() returns
- Remove the now-stale documenting comment
- Deeper testing has been successful

Comments

David Howells April 4, 2024, 1:04 p.m. UTC | #1

Chuck Lever <[e-mail]> wrote:

> That commit assumed that sock_sendmsg() releases all the pages in
> the underlying bio_vec array, but the reality is that it doesn't.
> svc_xprt_release() releases the rqst's response pages, but the
> record marker page fragment isn't one of those, so it is never
> released.

More like the network layer will take its own refs and drop those when it is
done.  As you say, it doesn't release the caller's refs.

Reviewed-by: David Howells <[e-mail]>

Patch

diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 545017a3daa4..6b3f01beb294 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -1206,15 +1206,6 @@  static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
  * MSG_SPLICE_PAGES is used exclusively to reduce the number of
  * copy operations in this path. Therefore the caller must ensure
  * that the pages backing @xdr are unchanging.
- *
- * Note that the send is non-blocking. The caller has incremented
- * the reference count on each page backing the RPC message, and
- * the network layer will "put" these pages when transmission is
- * complete.
- *
- * This is safe for our RPC services because the memory backing
- * the head and tail components is never kmalloc'd. These always
- * come from pages in the svc_rqst::rq_pages array.
  */
 static int svc_tcp_sendmsg(struct svc_sock *svsk, struct svc_rqst *rqstp,
 			   rpc_fraghdr marker, unsigned int *sentp)
@@ -1244,6 +1235,7 @@  static int svc_tcp_sendmsg(struct svc_sock *svsk, struct svc_rqst *rqstp,
 	iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, rqstp->rq_bvec,
 		      1 + count, sizeof(marker) + rqstp->rq_res.len);
 	ret = sock_sendmsg(svsk->sk_sock, &msg);
+	page_frag_free(buf);
 	if (ret < 0)
 		return ret;
 	*sentp += ret;