From patchwork Sun Nov 1 13:16:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 11872039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 992BFC4741F for ; Sun, 1 Nov 2020 13:16:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CA6DB2158C for ; Sun, 1 Nov 2020 13:16:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pm.me header.i=@pm.me header.b="WmM3wlI4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726563AbgKANQr (ORCPT ); Sun, 1 Nov 2020 08:16:47 -0500 Received: from mail1.protonmail.ch ([185.70.40.18]:26311 "EHLO mail1.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726252AbgKANQr (ORCPT ); Sun, 1 Nov 2020 08:16:47 -0500 Date: Sun, 01 Nov 2020 13:16:32 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1604236604; bh=W5mDsry01zdm8v+mdkIsMmMzKunu58VfAGfU4PiW1Ng=; h=Date:To:From:Cc:Reply-To:Subject:From; b=WmM3wlI4r2E6vtJMrBzwgsb84TLDSbiPDFiiofVz+5E/SSW96ktVv6bDUWFBj9pTE 2JW8RwJO260/MUUijloa82M6wVXuHZVPZqIlyUNCTeTCEBZ9BX67UB3pc5RTwS9k4h 05/Zn9+oZ0HfERWa/z3Txc7jteF+qvVeMnnSfK0kYu+aUuQlVx/dxLPeR+AGkkY1rT nO/LB29ODsLDDR1Cv7q5SOc3ohoiAWzdynp2UyWjT5WjcOUaiQiTAfnnenrZ0jGol1 OqPt0OpBxK+MuZrIjLoKsSEbntUQn7oKWYgL3xnJDak2FUZ8w+E/aSWTKfyNqy9EzM vbhiZH71127hg== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Jay Vosburgh , Veaceslav Falico , Andy Gospodarek , Jiri Pirko , Steffen Klassert , Willem de Bruijn , Alexander Lobakin , Miaohe Lin , Antoine Tenart , Mauro Carvalho Chehab , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 0/2] net: allow virtual netdevs to forward UDP L4 and fraglist GSO skbs Message-ID: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org NETIF_F_GSO_UDP_L4 and NETIF_F_GSO_FRAGLIST allow drivers to offload GSO UDP L4. This works well on simple setups, but when any logical netdev (e.g. VLAN) is present, kernel stack always performs software resegmentation which actually kills the performance. The full path in such cases is like: 1. Our NIC driver advertises a support for fraglists, GSO UDP L4, GSO fraglists. 2. User enables fraglisted GRO via Ethtool. 3. GRO subsystem receives UDP frames from driver and merges the packets into fraglisted GSO skb(s). 4. Networking stack queues it up for xmitting. 5. Virtual device like VLAN doesn't advertise a support for GSO UDP L4 and GSO fraglists, so skb_gso_check() doesn't allow to pass this skb as is to the real driver. 6. Kernel then has to form a bunch of regular UDP skbs from that one and pass it to the driver instead. This fallback is *extremely* slow for any GSO types, but especially for GSO fraglists. 7. All further processing performs with a series of plain UDP skbs, and the driver gets it one-by-one, despite that it supports UDP L4 and fraglisted GSO. That's not OK because: a) logical/virtual netdevs like VLANs, bridges etc. should pass GSO skbs as is; b) even if the final driver doesn't support such type of GSO, this software resegmenting should be performed right before it, not in the middle of processing -- I think I even saw that note somewhere in kernel documentation, and it's totally reasonable in terms of performance. Despite the fact that no mainline drivers currently supports fraglist GSO, this should and can be easily fixed by adding UDP L4 and fraglist GSO to the list of GSO types that can be passed-through the logical interfaces (NETIF_F_GSO_SOFTWARE). After this change, no resegmentation occurs (if a particular driver supports and advertises this), and the performance goes on par with e.g. 1:1 forwarding. The only logical netdevs that seem to be unaffected to this are bridge interfaces, as their code uses full NETIF_F_GSO_MASK. Tested on MIPS32 R2 router board with a WIP NIC driver in VLAN NAT: 20 Mbps baseline, 1 Gbps / link speed with this patch. Since v1 [1]: - handle bonding and team drivers as suggested by Willem de Bruijn; - reword and expand the introduction with the particular example. [1] https://lore.kernel.org/netdev/Mx3BWGop6fGORN6Cpo4mHIHz2b1bb0eLxeMG8vsijnk@cp3-web-020.plabs.ch Alexander Lobakin (2): net: add GSO UDP L4 and GSO fraglists to the list of software-backed types net: bonding, dummy, ifb, team: advertise NETIF_F_GSO_SOFTWARE drivers/net/bonding/bond_main.c | 11 +++++------ drivers/net/dummy.c | 2 +- drivers/net/ifb.c | 3 +-- drivers/net/team/team.c | 9 ++++----- include/linux/netdev_features.h | 4 ++-- 5 files changed, 13 insertions(+), 16 deletions(-)