[v5,net-next,v5,0/6] Add TCP fraglist GRO support

Message

Felix Fietkau May 2, 2024, 8:44 a.m. UTC

When forwarding TCP after GRO, software segmentation is very expensive,
especially when the checksum needs to be recalculated.
One case where that's currently unavoidable is when routing packets over
PPPoE. Performance improves significantly when using fraglist GRO
implemented in the same way as for UDP.

When NETIF_F_GRO_FRAGLIST is enabled, perform a lookup for an established
socket in the same netns as the receiving device. While this may not
cover all relevant use cases in multi-netns configurations, it should be
good enough for most configurations that need this.

Here's a measurement of running 2 TCP streams through a MediaTek MT7622
device (2-core Cortex-A53), which runs NAT with flow offload enabled from
one ethernet port to PPPoE on another ethernet port + cake qdisc set to
1Gbps.

rx-gro-list off: 630 Mbit/s, CPU 35% idle
rx-gro-list on:  770 Mbit/s, CPU 40% idle

Changes since v4:
 - add likely() to prefer the non-fraglist path in check

Changes since v3:
 - optimize __tcpv4_gso_segment_csum
 - add unlikely()
 - reorder dev_net/skb_gro_network_header calls after NETIF_F_GRO_FRAGLIST
   check
 - add support for ipv6 nat
 - drop redundant pskb_may_pull check

Changes since v2:
 - create tcp_gro_header_pull helper function to pull tcp header only once
 - optimize __tcpv4_gso_segment_list_csum, drop obsolete flags check

Changes since v1:
 - revert bogus tcp flags overwrite on segmentation
 - fix kbuild issue with !CONFIG_IPV6
 - only perform socket lookup for the first skb in the GRO train

Changes since RFC:
 - split up patches
 - handle TCP flags mutations

Felix Fietkau (6):
  net: move skb_gro_receive_list from udp to core
  net: add support for segmenting TCP fraglist GSO packets
  net: add code for TCP fraglist GRO
  net: create tcp_gro_lookup helper function
  net: create tcp_gro_header_pull helper function
  net: add heuristic for enabling TCP fraglist GRO

 include/net/gro.h        |   1 +
 include/net/tcp.h        |   5 +-
 net/core/gro.c           |  27 +++++
 net/ipv4/tcp_offload.c   | 214 ++++++++++++++++++++++++++++++++-------
 net/ipv4/udp_offload.c   |  27 -----
 net/ipv6/tcpv6_offload.c | 120 +++++++++++++++++++++-
 6 files changed, 325 insertions(+), 69 deletions(-)

Comments

Willem de Bruijn May 2, 2024, 3:46 p.m. UTC | #1

Felix Fietkau wrote:
> When forwarding TCP after GRO, software segmentation is very expensive,
> especially when the checksum needs to be recalculated.
> One case where that's currently unavoidable is when routing packets over
> PPPoE. Performance improves significantly when using fraglist GRO
> implemented in the same way as for UDP.
> 
> When NETIF_F_GRO_FRAGLIST is enabled, perform a lookup for an established
> socket in the same netns as the receiving device. While this may not
> cover all relevant use cases in multi-netns configurations, it should be
> good enough for most configurations that need this.
> 
> Here's a measurement of running 2 TCP streams through a MediaTek MT7622
> device (2-core Cortex-A53), which runs NAT with flow offload enabled from
> one ethernet port to PPPoE on another ethernet port + cake qdisc set to
> 1Gbps.
> 
> rx-gro-list off: 630 Mbit/s, CPU 35% idle
> rx-gro-list on:  770 Mbit/s, CPU 40% idle
> 
> Changes since v4:
>  - add likely() to prefer the non-fraglist path in check
> 
> Changes since v3:
>  - optimize __tcpv4_gso_segment_csum
>  - add unlikely()
>  - reorder dev_net/skb_gro_network_header calls after NETIF_F_GRO_FRAGLIST
>    check
>  - add support for ipv6 nat
>  - drop redundant pskb_may_pull check
> 
> Changes since v2:
>  - create tcp_gro_header_pull helper function to pull tcp header only once
>  - optimize __tcpv4_gso_segment_list_csum, drop obsolete flags check
> 
> Changes since v1:
>  - revert bogus tcp flags overwrite on segmentation
>  - fix kbuild issue with !CONFIG_IPV6
>  - only perform socket lookup for the first skb in the GRO train
> 
> Changes since RFC:
>  - split up patches
>  - handle TCP flags mutations
> 
> Felix Fietkau (6):
>   net: move skb_gro_receive_list from udp to core
>   net: add support for segmenting TCP fraglist GSO packets
>   net: add code for TCP fraglist GRO
>   net: create tcp_gro_lookup helper function
>   net: create tcp_gro_header_pull helper function
>   net: add heuristic for enabling TCP fraglist GRO

Reviewed-by: Willem de Bruijn <[e-mail]>

[e-mail] May 6, 2024, 10:10 a.m. UTC | #2

Hello:

This series was applied to netdev/net-next.git (main)
by Paolo Abeni <[e-mail]>:

On Thu,  2 May 2024 10:44:41 +0200 you wrote:
> When forwarding TCP after GRO, software segmentation is very expensive,
> especially when the checksum needs to be recalculated.
> One case where that's currently unavoidable is when routing packets over
> PPPoE. Performance improves significantly when using fraglist GRO
> implemented in the same way as for UDP.
> 
> When NETIF_F_GRO_FRAGLIST is enabled, perform a lookup for an established
> socket in the same netns as the receiving device. While this may not
> cover all relevant use cases in multi-netns configurations, it should be
> good enough for most configurations that need this.
> 
> [...]

Here is the summary with links:
  - [v5,net-next,v5,1/6] net: move skb_gro_receive_list from udp to core
    https://git.kernel.org/netdev/net-next/c/8928756d53d5
  - [v5,net-next,v5,2/6] net: add support for segmenting TCP fraglist GSO packets
    https://git.kernel.org/netdev/net-next/c/bee88cd5bd83
  - [v5,net-next,v5,3/6] net: add code for TCP fraglist GRO
    https://git.kernel.org/netdev/net-next/c/8d95dc474f85
  - [v5,net-next,v5,4/6] net: create tcp_gro_lookup helper function
    https://git.kernel.org/netdev/net-next/c/80e85fbdf19e
  - [v5,net-next,v5,5/6] net: create tcp_gro_header_pull helper function
    https://git.kernel.org/netdev/net-next/c/7516b27c555c
  - [v5,net-next,v5,6/6] net: add heuristic for enabling TCP fraglist GRO
    https://git.kernel.org/netdev/net-next/c/c9d1d23e5239

You are awesome, thank you!