[v3,net-next,0/3] net: add new socket option SO_RESERVE_MEM

Message

Wei Wang Sept. 29, 2021, 5:25 p.m. UTC

This patch series introduces a new socket option SO_RESERVE_MEM. 
This socket option provides a mechanism for users to reserve a certain
amount of memory for the socket to use. When this option is set, kernel
charges the user specified amount of memory to memcg, as well as
sk_forward_alloc. This amount of memory is not reclaimable and is
available in sk_forward_alloc for this socket.
With this socket option set, the networking stack spends less cycles
doing forward alloc and reclaim, which should lead to better system
performance, with the cost of an amount of pre-allocated and
unreclaimable memory, even under memory pressure.
With a tcp_stream test with 10 flows running on a simulated 100ms RTT
link, I can see the cycles spent in __sk_mem_raise_allocated() dropping
by ~0.02%. Not a whole lot, since we already have logic in
sk_mem_uncharge() to only reclaim 1MB when sk_forward_alloc has more
than 2MB free space. But on a system suffering memory pressure
constently, the savings should be more.

The first patch is the implementation of this socket option. The
following 2 patches change the tcp stack to make use of this reserved
memory when under memory pressure. This makes the tcp stack behavior
more flexible when under memory pressure, and provides a way for user to
control the distribution of the memory among its sockets.
With a TCP connection on a simulated 100ms RTT link, the default
throughput under memory pressure is ~500Kbps. With SO_RESERVE_MEM set to
100KB, the throughput under memory pressure goes up to ~3.5Mbps.

Change since v2:
- Added description for new field added in struct sock in patch 1
Change since v1:
- Added performance stats in cover letter and rebased

Wei Wang (3):
  net: add new socket option SO_RESERVE_MEM
  tcp: adjust sndbuf according to sk_reserved_mem
  tcp: adjust rcv_ssthresh according to sk_reserved_mem

 include/net/sock.h                | 45 +++++++++++++++++---
 include/net/tcp.h                 | 11 +++++
 include/uapi/asm-generic/socket.h |  2 +
 net/core/sock.c                   | 69 +++++++++++++++++++++++++++++++
 net/core/stream.c                 |  2 +-
 net/ipv4/af_inet.c                |  2 +-
 net/ipv4/tcp_input.c              | 26 ++++++++++--
 net/ipv4/tcp_output.c             |  3 +-
 8 files changed, 147 insertions(+), 13 deletions(-)

Comments

[e-mail] Sept. 30, 2021, 12:50 p.m. UTC | #1

Hello:

This series was applied to netdev/net-next.git (refs/heads/master):

On Wed, 29 Sep 2021 10:25:10 -0700 you wrote:
> This patch series introduces a new socket option SO_RESERVE_MEM.
> This socket option provides a mechanism for users to reserve a certain
> amount of memory for the socket to use. When this option is set, kernel
> charges the user specified amount of memory to memcg, as well as
> sk_forward_alloc. This amount of memory is not reclaimable and is
> available in sk_forward_alloc for this socket.
> With this socket option set, the networking stack spends less cycles
> doing forward alloc and reclaim, which should lead to better system
> performance, with the cost of an amount of pre-allocated and
> unreclaimable memory, even under memory pressure.
> With a tcp_stream test with 10 flows running on a simulated 100ms RTT
> link, I can see the cycles spent in __sk_mem_raise_allocated() dropping
> by ~0.02%. Not a whole lot, since we already have logic in
> sk_mem_uncharge() to only reclaim 1MB when sk_forward_alloc has more
> than 2MB free space. But on a system suffering memory pressure
> constently, the savings should be more.
> 
> [...]

Here is the summary with links:
  - [v3,net-next,1/3] net: add new socket option SO_RESERVE_MEM
    https://git.kernel.org/netdev/net-next/c/2bb2f5fb21b0
  - [v3,net-next,2/3] tcp: adjust sndbuf according to sk_reserved_mem
    https://git.kernel.org/netdev/net-next/c/ca057051cf25
  - [v3,net-next,3/3] tcp: adjust rcv_ssthresh according to sk_reserved_mem
    https://git.kernel.org/netdev/net-next/c/053f368412c9

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html