From patchwork Fri Apr 19 15:35:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Gobert X-Patchwork-Id: 13636531 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16E1112FB34; Fri, 19 Apr 2024 15:36:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713540988; cv=none; b=BAFrqx3Vihu/jt7HY0svVm4rHkcO7I+T5/akeIhEX8L2nlVcf3gELfURtffGGm23g695iEYOKh3j2eLsytIVUXZceUUi8lDX9sh0wSTwKyEVCVqAY7sICAxKJ5CU0B4NQTRkFrWtSMP8XPNEdxVhzWb6cJXdCn21BxMo7lV7XPI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713540988; c=relaxed/simple; bh=WyEbXvtsNVWgOtX0bo1AxdKOIqpGQ7KA1yYW2GOQePw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=O0bNw6glZk+XIa4buC+88efwJtOssSUVsgT306xO0ofw6R26OrGpzbJKuUPCwif5Thb2s5Z/IYVwM9isJowGNY0r9L0W6vxzplDXiau09DLHztIZrOAmGu6Czb/ZbYyoN1+Fga97uBIe8mrRqOEgveX0etPYNTqCogyGar4DRa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BlnZ2gp7; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BlnZ2gp7" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-419ca3f3dd0so3280805e9.2; Fri, 19 Apr 2024 08:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713540985; x=1714145785; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B9n1q4eRK+xzr5UrVS/+dPXTIvrcgwvHfK/8hZOcG+0=; b=BlnZ2gp7oJgeCB2nsNU7E57fwgqEYgJyVBWpv3VdVTbGY2eGwx41EVHy7mSnQfnSg/ H6OdiXzhimeeIZSntcWn9tfyb1NP/0km7+3wKDs6iSMqr++an+0pEdpOlJkbRJzoa6CS yqM19HVyw4K8q8KwF+FdEzOyQ7hIMYYe7hXP3NlLpBPnneH1OR2lVklx/blIwVnOXsW7 o5Fuj/z2utfypaeuchdFL6vaYDl2KRbL+dQEJzOxr+uBq8qocvCHGs2vuKJNm9PydEuP +ZFyd3FanTyrQ84bid5AETY5qymaF5rd5MsZ2m8fxkHw2umxOrhRxV87LR2dseX5hyoY uG+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713540985; x=1714145785; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B9n1q4eRK+xzr5UrVS/+dPXTIvrcgwvHfK/8hZOcG+0=; b=PWB6m/N5ja0pBRAazDg1J2Jhm+TSyekKGeldfc0SBCvQK/DrIrKN/VFmnRFMCiTJXS QRW6txCLw84L4fTyhOwXd+Yi+NmzlxISTvY0LOuLczlW54tnpD+f/fqz8m176C3/OxdQ G+w5PqyqDMwYIZkKtLt7ZoYnP7KZPWlfZq/JbUk+6Fj4NKz4djNtlJMwpKwre+Rq7OGL 8DWvZClUHhwJGvvCN7EHoFt8JsHferaCXvJwUb0M+kgyMtFIXBLwS2CHFmlBo4waya7U GobNED3PFxsc9MFf0zfWODI9a/BxJi0nk0rDhA/VBFPOCkFJtDasKIE6PwYcNl64IvQG TcFg== X-Forwarded-Encrypted: i=1; AJvYcCVkTrhJxM8IYlOXUgbN/0IBbKSq6oddb1y/ZtbuSjOQbLNndmxTaK/640YTzegcJhHabdDzLVNcZr5KvtRw3yoho08sJLl4lDByIY7PedYVk9UeoJZ+ojZXaaKvSA7QKIFqJK15 X-Gm-Message-State: AOJu0Yz6DnwqpLljzG+LB8DMc08Nh2npKHEshl7gcK81BYALFPFBDgxB /E/5LsF8PMelnZWjM6ryDjZ8zqZ4tLW6jdRJNm9QQiKxbqFMbFHf X-Google-Smtp-Source: AGHT+IGkebQNQXL5Bql2j2QAwm4CyArSU2/bANJXhk1CnkRJFsC2cfBDKalHEZV2ORzogajm71nR3g== X-Received: by 2002:a05:600c:4fcf:b0:418:ad02:913e with SMTP id o15-20020a05600c4fcf00b00418ad02913emr2325251wmq.12.1713540985313; Fri, 19 Apr 2024 08:36:25 -0700 (PDT) Received: from localhost ([146.70.204.204]) by smtp.gmail.com with ESMTPSA id f11-20020a05600c4e8b00b00417ee886977sm10786939wmq.4.2024.04.19.08.36.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 08:36:25 -0700 (PDT) From: Richard Gobert To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, alexander.duyck@gmail.com, aleksander.lobakin@intel.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Richard Gobert Subject: [PATCH net v2 1/3] net: gro: add {inner_}network_offset to napi_gro_cb Date: Fri, 19 Apr 2024 17:35:40 +0200 Message-Id: <20240419153542.121087-2-richardbgobert@gmail.com> In-Reply-To: <20240419153542.121087-1-richardbgobert@gmail.com> References: <20240419153542.121087-1-richardbgobert@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org This patch adds network_offset and inner_network_offset to napi_gro_cb, and makes sure both are set correctly. In the common path there's only one write (skb_gro_reset_offset, which replaces skb_set_network_header). Signed-off-by: Richard Gobert --- drivers/net/geneve.c | 1 + drivers/net/vxlan/vxlan_core.c | 1 + include/net/gro.h | 18 ++++++++++++++++-- net/8021q/vlan_core.c | 2 ++ net/core/gro.c | 1 + net/ethernet/eth.c | 1 + net/ipv4/af_inet.c | 5 +---- net/ipv4/gre_offload.c | 1 + net/ipv6/ip6_offload.c | 8 ++++---- 9 files changed, 28 insertions(+), 10 deletions(-) diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c index 6c2835086b57..6549348cc24e 100644 --- a/drivers/net/geneve.c +++ b/drivers/net/geneve.c @@ -542,6 +542,7 @@ static struct sk_buff *geneve_gro_receive(struct sock *sk, if (!ptype) goto out; + NAPI_GRO_CB(skb)->inner_network_offset = hlen; pp = call_gro_receive(ptype->callbacks.gro_receive, head, skb); flush = 0; diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index ba319fc21957..c649a82eeca7 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -754,6 +754,7 @@ static struct sk_buff *vxlan_gpe_gro_receive(struct sock *sk, vh = vxlan_gro_prepare_receive(sk, head, skb, &grc); if (vh) { + NAPI_GRO_CB(skb)->inner_network_offset = skb_gro_offset(skb); if (!vxlan_parse_gpe_proto(vh, &protocol)) goto out; ptype = gro_find_receive_by_type(protocol); diff --git a/include/net/gro.h b/include/net/gro.h index 50f1e403dbbb..1faff23b66e8 100644 --- a/include/net/gro.h +++ b/include/net/gro.h @@ -87,6 +87,15 @@ struct napi_gro_cb { /* used to support CHECKSUM_COMPLETE for tunneling protocols */ __wsum csum; + + /* L3 offsets */ + union { + struct { + u16 network_offset; + u16 inner_network_offset; + }; + u16 network_offsets[2]; + }; }; #define NAPI_GRO_CB(skb) ((struct napi_gro_cb *)(skb)->cb) @@ -172,12 +181,17 @@ static inline void *skb_gro_header(struct sk_buff *skb, unsigned int hlen, return ptr; } +static inline int skb_gro_network_offset(const struct sk_buff *skb) +{ + return NAPI_GRO_CB(skb)->network_offsets[NAPI_GRO_CB(skb)->encap_mark]; +} + static inline void *skb_gro_network_header(const struct sk_buff *skb) { if (skb_gro_may_pull(skb, skb_gro_offset(skb))) - return skb_gro_header_fast(skb, skb_network_offset(skb)); + return skb_gro_header_fast(skb, skb_gro_network_offset(skb)); - return skb_network_header(skb); + return skb->data + skb_gro_network_offset(skb); } static inline __wsum inet_gro_compute_pseudo(const struct sk_buff *skb, diff --git a/net/8021q/vlan_core.c b/net/8021q/vlan_core.c index f00158234505..9404dd551dfd 100644 --- a/net/8021q/vlan_core.c +++ b/net/8021q/vlan_core.c @@ -478,6 +478,8 @@ static struct sk_buff *vlan_gro_receive(struct list_head *head, if (unlikely(!vhdr)) goto out; + NAPI_GRO_CB(skb)->network_offsets[NAPI_GRO_CB(skb)->encap_mark] = hlen; + type = vhdr->h_vlan_encapsulated_proto; ptype = gro_find_receive_by_type(type); diff --git a/net/core/gro.c b/net/core/gro.c index 83f35d99a682..c7901253a1a8 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -371,6 +371,7 @@ static inline void skb_gro_reset_offset(struct sk_buff *skb, u32 nhoff) const skb_frag_t *frag0; unsigned int headlen; + NAPI_GRO_CB(skb)->network_offset = 0; NAPI_GRO_CB(skb)->data_offset = 0; headlen = skb_headlen(skb); NAPI_GRO_CB(skb)->frag0 = skb->data; diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c index 2edc8b796a4e..ea589e8cde2a 100644 --- a/net/ethernet/eth.c +++ b/net/ethernet/eth.c @@ -441,6 +441,7 @@ struct sk_buff *eth_gro_receive(struct list_head *head, struct sk_buff *skb) skb_gro_pull(skb, sizeof(*eh)); skb_gro_postpull_rcsum(skb, eh, sizeof(*eh)); + NAPI_GRO_CB(skb)->inner_network_offset = hlen; pp = indirect_call_gro_receive_inet(ptype->callbacks.gro_receive, ipv6_gro_receive, inet_gro_receive, diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index 55bd72997b31..7899cbd5b263 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1568,10 +1568,6 @@ struct sk_buff *inet_gro_receive(struct list_head *head, struct sk_buff *skb) NAPI_GRO_CB(skb)->is_atomic = !!(iph->frag_off & htons(IP_DF)); NAPI_GRO_CB(skb)->flush |= flush; - skb_set_network_header(skb, off); - /* The above will be needed by the transport layer if there is one - * immediately following this IP hdr. - */ /* Note : No need to call skb_gro_postpull_rcsum() here, * as we already checked checksum over ipv4 header was 0 @@ -1597,6 +1593,7 @@ static struct sk_buff *ipip_gro_receive(struct list_head *head, } NAPI_GRO_CB(skb)->encap_mark = 1; + NAPI_GRO_CB(skb)->inner_network_offset = skb_gro_offset(skb); return inet_gro_receive(head, skb); } diff --git a/net/ipv4/gre_offload.c b/net/ipv4/gre_offload.c index 5028c72d494a..a1ff2bdf6206 100644 --- a/net/ipv4/gre_offload.c +++ b/net/ipv4/gre_offload.c @@ -224,6 +224,7 @@ static struct sk_buff *gre_gro_receive(struct list_head *head, /* Adjusted NAPI_GRO_CB(skb)->csum after skb_gro_pull()*/ skb_gro_postpull_rcsum(skb, greh, grehlen); + NAPI_GRO_CB(skb)->inner_network_offset = hlen; pp = call_gro_receive(ptype->callbacks.gro_receive, head, skb); flush = 0; diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c index b41e35af69ea..765797ca729c 100644 --- a/net/ipv6/ip6_offload.c +++ b/net/ipv6/ip6_offload.c @@ -67,7 +67,7 @@ static int ipv6_gro_pull_exthdrs(struct sk_buff *skb, int off, int proto) off += len; } - skb_gro_pull(skb, off - skb_network_offset(skb)); + skb_gro_pull(skb, off - skb_gro_network_offset(skb)); return proto; } @@ -236,8 +236,6 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head, if (unlikely(!iph)) goto out; - skb_set_network_header(skb, off); - flush += ntohs(iph->payload_len) != skb->len - hlen; proto = iph->nexthdr; @@ -259,7 +257,7 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head, NAPI_GRO_CB(skb)->proto = proto; flush--; - nlen = skb_network_header_len(skb); + nlen = skb_gro_offset(skb) - off; list_for_each_entry(p, head, list) { const struct ipv6hdr *iph2; @@ -327,6 +325,7 @@ static struct sk_buff *sit_ip6ip6_gro_receive(struct list_head *head, } NAPI_GRO_CB(skb)->encap_mark = 1; + NAPI_GRO_CB(skb)->inner_network_offset = skb_gro_offset(skb); return ipv6_gro_receive(head, skb); } @@ -342,6 +341,7 @@ static struct sk_buff *ip4ip6_gro_receive(struct list_head *head, } NAPI_GRO_CB(skb)->encap_mark = 1; + NAPI_GRO_CB(skb)->inner_network_offset = skb_gro_offset(skb); return inet_gro_receive(head, skb); } From patchwork Fri Apr 19 15:35:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Gobert X-Patchwork-Id: 13636532 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5D9377F13; Fri, 19 Apr 2024 15:36:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713540998; cv=none; b=GNSD3a9krFYvQRgojtHdiChTmCnOkRVbSUsQzL1EowbTZ15dKhlh63QA/gASehn/JhljfWB1vEdppxBoShuS/4IMB2c01TtyEqfwcab76BItwozCz/WkIej8zQRq1LjgzDaoDVuUle7UnXhWc3t5XR4ZAOwKK/Jo/jXhd9b+4Qk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713540998; c=relaxed/simple; bh=Skui0WZke+2fDXRGytHsLEi0moy7ejJAoZnm+XJZrQ0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NUIu5rvVHhOc7VotUPn0+AMk8HYnMqiXpHrFu24u0OO2fUEd1K/1beuBeheypRrB8pX2/AUpRrFfCG3jqocZq4pI0VkLlT19fYb6jUWWo1QgQcye3FLsFFIbh6D/Q3WdDEhMEkqDTtgOqlX/OnhUsytCC7S97iW678kCrxSx5ig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=g5JXUbHs; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="g5JXUbHs" Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-4155819f710so18258075e9.2; Fri, 19 Apr 2024 08:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713540995; x=1714145795; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6GonKnqV6NFhsZSk6EQZMWgxYIk8eQbiGI3oluYbyLQ=; b=g5JXUbHswa+ZfBOiRQMYptwBPiXhJy/Mmo4+D2ep5FlBNRQDUPcTiNoaWje1/AegeW U8KgeDyoX948zv7yuWOCARnRDBGTWraXGl/st0a5p8PrYNT06LY0gbfWQMr2mcc6Nd5R 4x8YkxFAyOgWJuT03mXqi9M7wfKLH024F8rMPlWtEFmslZkZfTNhPZs9NE/ZSSmM/jBB 2G8ZVAaNL5GLrbjchPhTM5QgZEP2QaIkprlv98UWRTKmckUiCV/X5hCUnHpkRktc94DR HaqrHSxYKj0LCxg9a6QT8iohHjJAIbjjk9vWeIKN9691bwdN3JaMhssAYdEIkEXvGcQi STwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713540995; x=1714145795; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6GonKnqV6NFhsZSk6EQZMWgxYIk8eQbiGI3oluYbyLQ=; b=m57aBOT+QlBXAUwOXLGHfNMjgm1X0ctUftb1goBxpuIVHcq0UbI3A3uKURCCv4fhPd NCUF3qFp6bcV00U0ko+TADET7IlSVYGoh16MSyx+f9puBwLRMiDbUpKvgmwpP9N6c1VV e7q2jPmFzV7DlfQktapf3c52PC0bsB9K8+OEEwIMnjo4vMdi/qKlAa3xDeoXyG6qHArQ 1EfKnhGeaZHqnUc5Ro/iB4wu5uGbhpsdaWJjlYPF1miS8LwYJHJ7O9p/9ve8Cy3E/In8 IC6d85T3W250BG2a3yKIjlnxdpnxFC5Qzc0N2LMK8n4GKENR1NWLOL12y5iJfdTX71L1 NRrg== X-Forwarded-Encrypted: i=1; AJvYcCUC4ihZU7bJW6UGXvl3Lt4+rxZUDRIz0A+ej1VCr1Tpr5JwfHamVV+/7rFDWGoImRU12hpL3fqbGQhu6dYO8zix70UfBS1AW2XyPC5kJoxZBJ/E1AbbrK8hjd90R1Mh8yyd4TTj X-Gm-Message-State: AOJu0YwvMbwMf0zaGCte1Vn2llvL95om1D3ioLJf2ldvItsHkSMM2ffp Ciivv0byEriwXfPT6l/pZsKKN/+Cr6HNkK989ResC+pvxrKfh6LO X-Google-Smtp-Source: AGHT+IEA2dZ2+2gTMlOMW74UYs9uqReDRdBKc/JYC4amTWwcdL6sne/ikOqx/pwpI3rlyGBm9by0aw== X-Received: by 2002:a05:600c:3b09:b0:418:969b:cb37 with SMTP id m9-20020a05600c3b0900b00418969bcb37mr1771696wms.39.1713540995081; Fri, 19 Apr 2024 08:36:35 -0700 (PDT) Received: from localhost ([146.70.204.204]) by smtp.gmail.com with ESMTPSA id g13-20020a05600c310d00b004196c93da22sm1922124wmo.2.2024.04.19.08.36.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 08:36:34 -0700 (PDT) From: Richard Gobert To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, alexander.duyck@gmail.com, aleksander.lobakin@intel.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Richard Gobert Subject: [PATCH net v2 2/3] net: gro: fix udp bad offset in socket lookup Date: Fri, 19 Apr 2024 17:35:41 +0200 Message-Id: <20240419153542.121087-3-richardbgobert@gmail.com> In-Reply-To: <20240419153542.121087-1-richardbgobert@gmail.com> References: <20240419153542.121087-1-richardbgobert@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Commits a602456 ("udp: Add GRO functions to UDP socket") and 57c67ff ("udp: additional GRO support") introduce incorrect usage of {ip,ipv6}_hdr in the complete phase of gro. The functions always return skb->network_header, which in the case of encapsulated packets at the gro complete phase, is always set to the innermost L3 of the packet. That means that calling {ip,ipv6}_hdr for skbs which completed the GRO receive phase (both in gro_list and *_gro_complete) when parsing an encapsulated packet's _outer_ L3/L4 may return an unexpected value. This incorrect usage leads to a bug in GRO's UDP socket lookup. udp{4,6}_lib_lookup_skb functions use ip_hdr/ipv6_hdr respectively. These *_hdr functions return network_header which will point to the innermost L3, resulting in the wrong offset being used in __udp{4,6}_lib_lookup with encapsulated packets. To fix this issue, network_offsets union is used inside napi_gro_cb, in which both the outer and the inner network offsets are saved. Reproduction example: Endpoint configuration example (fou + local address bind) # ip fou add port 6666 ipproto 4 # ip link add name tun1 type ipip remote 2.2.2.1 local 2.2.2.2 encap fou encap-dport 5555 encap-sport 6666 mode ipip # ip link set tun1 up # ip a add 1.1.1.2/24 dev tun1 Netperf TCP_STREAM result on net-next before patch is applied: net-next main, GRO enabled: $ netperf -H 1.1.1.2 -t TCP_STREAM -l 5 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 131072 16384 16384 5.28 2.37 net-next main, GRO disabled: $ netperf -H 1.1.1.2 -t TCP_STREAM -l 5 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 131072 16384 16384 5.01 2745.06 patch applied, GRO enabled: $ netperf -H 1.1.1.2 -t TCP_STREAM -l 5 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 131072 16384 16384 5.01 2877.38 Fixes: 57c67ff4bd92 ("udp: additional GRO support") Signed-off-by: Richard Gobert --- net/ipv4/udp.c | 3 ++- net/ipv4/udp_offload.c | 3 ++- net/ipv6/udp.c | 3 ++- net/ipv6/udp_offload.c | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index c02bf011d4a6..1399fce82b3f 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -532,7 +532,8 @@ static inline struct sock *__udp4_lib_lookup_skb(struct sk_buff *skb, struct sock *udp4_lib_lookup_skb(const struct sk_buff *skb, __be16 sport, __be16 dport) { - const struct iphdr *iph = ip_hdr(skb); + const u16 offset = NAPI_GRO_CB(skb)->network_offsets[skb->encapsulation]; + const struct iphdr *iph = (struct iphdr *)(skb->data + offset); struct net *net = dev_net(skb->dev); int iif, sdif; diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 3498dd1d0694..fd29d21d579c 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -718,7 +718,8 @@ EXPORT_SYMBOL(udp_gro_complete); INDIRECT_CALLABLE_SCOPE int udp4_gro_complete(struct sk_buff *skb, int nhoff) { - const struct iphdr *iph = ip_hdr(skb); + const u16 offset = NAPI_GRO_CB(skb)->network_offsets[skb->encapsulation]; + const struct iphdr *iph = (struct iphdr *)(skb->data + offset); struct udphdr *uh = (struct udphdr *)(skb->data + nhoff); /* do fraglist only if there is no outer UDP encap (or we already processed it) */ diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 8b1dd7f51249..f7880e306410 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -272,7 +272,8 @@ static struct sock *__udp6_lib_lookup_skb(struct sk_buff *skb, struct sock *udp6_lib_lookup_skb(const struct sk_buff *skb, __be16 sport, __be16 dport) { - const struct ipv6hdr *iph = ipv6_hdr(skb); + const u16 offset = NAPI_GRO_CB(skb)->network_offsets[skb->encapsulation]; + const struct ipv6hdr *iph = (struct ipv6hdr *)(skb->data + offset); struct net *net = dev_net(skb->dev); int iif, sdif; diff --git a/net/ipv6/udp_offload.c b/net/ipv6/udp_offload.c index bbd347de00b4..b41152dd4246 100644 --- a/net/ipv6/udp_offload.c +++ b/net/ipv6/udp_offload.c @@ -164,7 +164,8 @@ struct sk_buff *udp6_gro_receive(struct list_head *head, struct sk_buff *skb) INDIRECT_CALLABLE_SCOPE int udp6_gro_complete(struct sk_buff *skb, int nhoff) { - const struct ipv6hdr *ipv6h = ipv6_hdr(skb); + const u16 offset = NAPI_GRO_CB(skb)->network_offsets[skb->encapsulation]; + const struct ipv6hdr *ipv6h = (struct ipv6hdr *)(skb->data + offset); struct udphdr *uh = (struct udphdr *)(skb->data + nhoff); /* do fraglist only if there is no outer UDP encap (or we already processed it) */ From patchwork Fri Apr 19 15:35:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Gobert X-Patchwork-Id: 13636533 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A444912E1E7; Fri, 19 Apr 2024 15:36:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713541007; cv=none; b=AXPXSlZ5XzglfLCGa31YumU3ZqdtvPCydyYQy5tmXKXfR89gNJu3FFgW8Q2dbevV6iaSRAyd4ZhOr5+CyubJr8sKY0GUM/GMKVja4I0Ks/L/9dvZcZT+viSCmM+a1Gq5j4hOaWB70egl8F5JcvPC2kYhZB9WKybb2hRYLHwlDqY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713541007; c=relaxed/simple; bh=bXd2cF1r5wGgGn8qjmQ3gV0VuKXhxZccccWQSzrZm9Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kg3g7ZZVAskH32H/WEQbaCXJNKCxvd8f195wXQ0q9MCD6KdsP/n8ggHlM3zbR+MY8waPByRDNy0ARBNXxGVZNBhNUpCy6G0+lEKDFThT9fp6P1YDDefMw7GlNYkE054SOnS7vRPD4FzMsdgmSYb5BhE1uvonJdvzIA0Ca82AquY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hTCtZ/rD; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hTCtZ/rD" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-34a7e47d164so287928f8f.3; Fri, 19 Apr 2024 08:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713541004; x=1714145804; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dBCxnbysmOgF/ewLoKlev2MPx9AEH27+l/wXXPvFzlE=; b=hTCtZ/rDjFcmpsxKKEFGcRQ8OivwwbPt5Qk0CWkrMgqojFPLVNgxswXfPkbtdX4aVO m8vtNjLn87/eA8w0tM63PCTnVc7dSUKmgjvoUkRdjA1hbisDyQwEFVKesUrINn9VU1p4 Q6z1v3ZM3SFJZ6kS5XpN0gM1S1RwmnvXhTjYu2zG+n57MiUkmlLT9tynEdd9sgAkxD/O RWpodTSgNPnyXxvNQvtqjCeUDz4T3oWHHpm8Iz86MFyNQ/1qxLeuI8/xyNqyvS14+SDD HsyD5l8oa2dO/L3j3e6NmeFT7W1oSJSr5KInxPuvb4ZQ50sPtqwJQs4xI8avWOPIQWR9 eNSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713541004; x=1714145804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dBCxnbysmOgF/ewLoKlev2MPx9AEH27+l/wXXPvFzlE=; b=miUnJ4lEtQB4gIfY46dwapmecGN1XjuowgDKHker9qGlvc+nA/ulLhd/s3Xfa9J1KZ VFUavhfH8sXJlQmZFiQJd0fo8vkVy/IdDMc7DBdiQsInwFvE+9BEh1ObjOjHQsgWzROI e/AV7ViMXN2Q8FAcHbIe6VQ3UOQrjzczjT/Xce0vgOONrHWWjyrAvZiP1H24cRhfm7jp DlNgn8I8/Dfhn2li4+x1ntL+lOZExLDYYREk2wYWqw2GFC4RR3H5aZjJs+4vN1QjjLdR A0KwY8yyZYcDs/8qg/Mo6xQxcRWewLWDFsx65ScYBmkesJleof62rC7d7tAWumlAqTCN WjFA== X-Forwarded-Encrypted: i=1; AJvYcCWKIe16iGbmQV1EiYhjvRJHKguTVyaSl1ZZ3WB4pDmUMP7CI/UOOgQfCJTiDlUWeVzUjj4eor8d9g+YqpRGq6fz3fRILQ6J2B4n9JV5Uxly6qEA55x0aH+xUIMuKDgZAJHi73ej X-Gm-Message-State: AOJu0YyMLO/Bc/1JsYhmCia3OD2JAo3hqoGyQgRXYk8jRIf5eZVXOfF9 p0/AsLWVH2CW40y5V32d28FD3qRPt5Ai5uaoZZ2S8xgBC47eIHfm X-Google-Smtp-Source: AGHT+IEEsUl+sziq8fG2MR4CX89/a844zgARc3ht6bS6rLiL+jkcyfjhn1daKZQG4bY/FTJ6M1A14Q== X-Received: by 2002:adf:f9ce:0:b0:349:bd11:1bf1 with SMTP id w14-20020adff9ce000000b00349bd111bf1mr1690575wrr.46.1713541003830; Fri, 19 Apr 2024 08:36:43 -0700 (PDT) Received: from localhost ([146.70.204.204]) by smtp.gmail.com with ESMTPSA id u17-20020adfeb51000000b00347321735a6sm4693773wrn.66.2024.04.19.08.36.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Apr 2024 08:36:43 -0700 (PDT) From: Richard Gobert To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, alexander.duyck@gmail.com, aleksander.lobakin@intel.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Richard Gobert Subject: [PATCH net v2 3/3] net: gro: add flush check in udp_gro_receive_segment Date: Fri, 19 Apr 2024 17:35:42 +0200 Message-Id: <20240419153542.121087-4-richardbgobert@gmail.com> In-Reply-To: <20240419153542.121087-1-richardbgobert@gmail.com> References: <20240419153542.121087-1-richardbgobert@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org GRO-GSO path is supposed to be transparent and as such L3 flush checks are relevant to all UDP flows merging in GRO. This patch uses the same logic and code from tcp_gro_receive, terminating merge if flush is non zero. Fixes: 36707061d6ba ("udp: allow forwarding of plain (non-fraglisted) UDP GRO packets") Signed-off-by: Richard Gobert --- net/ipv4/udp_offload.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index fd29d21d579c..8721fe5beca2 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -471,6 +471,7 @@ static struct sk_buff *udp_gro_receive_segment(struct list_head *head, struct sk_buff *p; unsigned int ulen; int ret = 0; + int flush; /* requires non zero csum, for symmetry with GSO */ if (!uh->check) { @@ -504,13 +505,22 @@ static struct sk_buff *udp_gro_receive_segment(struct list_head *head, return p; } + flush = NAPI_GRO_CB(p)->flush; + + if (NAPI_GRO_CB(p)->flush_id != 1 || + NAPI_GRO_CB(p)->count != 1 || + !NAPI_GRO_CB(p)->is_atomic) + flush |= NAPI_GRO_CB(p)->flush_id; + else + NAPI_GRO_CB(p)->is_atomic = false; + /* Terminate the flow on len mismatch or if it grow "too much". * Under small packet flood GRO count could elsewhere grow a lot * leading to excessive truesize values. * On len mismatch merge the first packet shorter than gso_size, * otherwise complete the GRO packet. */ - if (ulen > ntohs(uh2->len)) { + if (ulen > ntohs(uh2->len) || flush) { pp = p; } else { if (NAPI_GRO_CB(skb)->is_flist) {