From patchwork Fri Mar 12 16:21:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12135227 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9DB3C43331 for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7F1A65019 for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231601AbhCLQWT (ORCPT ); Fri, 12 Mar 2021 11:22:19 -0500 Received: from mail-40136.protonmail.ch ([185.70.40.136]:28881 "EHLO mail-40136.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231855AbhCLQWB (ORCPT ); Fri, 12 Mar 2021 11:22:01 -0500 Date: Fri, 12 Mar 2021 16:21:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615566119; bh=n32rb6vRZgpCVR60KMaNApA0b+cldverLEokVhYSONc=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=BCOkWEZLvcX3pEMmjRe3WM+H1CUtp0qG16IAYlU2uFlBy20P1RWTK8knbV/lAgICi i/Sa/y+GU0WSVQrwNcaOrYWIXchtTnDdJJCg679tV35aH8yrw8OF/nuhWPoWw8ImX9 Dp44qYMC6+Wj2d/uy8vKaFKaeEgVx42d1Pb0aFEAI4Kq/l4SDMjjvfqqtHAhm8m6ZC d8aVQgiKNepR5GMgQOm+cBBIo7KwCCaMfpdXpznEgUslAaAFAkyjTiKGVyPCaZEzit Ho0dXwmNEa0YcGEkmJXCksXJGjXfD1ACHpDTy6WXVHjdFwK/shSoJBerU3lm4KuBbV +N1Ll8qlEietA== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 1/4] gro: give 'hash' variable in dev_gro_receive() a less confusing name Message-ID: <20210312162127.239795-2-alobakin@pm.me> In-Reply-To: <20210312162127.239795-1-alobakin@pm.me> References: <20210312162127.239795-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org 'hash' stores not the flow hash, but the index of the GRO bucket corresponding to it. Change its name to 'bucket' to avoid confusion while reading lines like '__set_bit(hash, &napi->gro_bitmask)'. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index 2bfdd528c7c3..adc42ba7ffd8 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5956,7 +5956,7 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb) { - u32 hash = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); + u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); struct list_head *head = &offload_base; struct packet_offload *ptype; __be16 type = skb->protocol; @@ -6024,7 +6024,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (pp) { skb_list_del_init(pp); napi_gro_complete(napi, pp); - napi->gro_hash[hash].count--; + napi->gro_hash[bucket].count--; } if (same_flow) @@ -6033,10 +6033,10 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (NAPI_GRO_CB(skb)->flush) goto normal; - if (unlikely(napi->gro_hash[hash].count >= MAX_GRO_SKBS)) { + if (unlikely(napi->gro_hash[bucket].count >= MAX_GRO_SKBS)) { gro_flush_oldest(napi, gro_head); } else { - napi->gro_hash[hash].count++; + napi->gro_hash[bucket].count++; } NAPI_GRO_CB(skb)->count = 1; NAPI_GRO_CB(skb)->age = jiffies; @@ -6050,11 +6050,11 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (grow > 0) gro_pull_from_frag0(skb, grow); ok: - if (napi->gro_hash[hash].count) { - if (!test_bit(hash, &napi->gro_bitmask)) - __set_bit(hash, &napi->gro_bitmask); - } else if (test_bit(hash, &napi->gro_bitmask)) { - __clear_bit(hash, &napi->gro_bitmask); + if (napi->gro_hash[bucket].count) { + if (!test_bit(bucket, &napi->gro_bitmask)) + __set_bit(bucket, &napi->gro_bitmask); + } else if (test_bit(bucket, &napi->gro_bitmask)) { + __clear_bit(bucket, &napi->gro_bitmask); } return ret; From patchwork Fri Mar 12 16:21:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12135219 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F519C433DB for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DEA5E6501D for ; Fri, 12 Mar 2021 16:22:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232229AbhCLQWU (ORCPT ); Fri, 12 Mar 2021 11:22:20 -0500 Received: from mail-40134.protonmail.ch ([185.70.40.134]:59365 "EHLO mail-40134.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231789AbhCLQWB (ORCPT ); Fri, 12 Mar 2021 11:22:01 -0500 Date: Fri, 12 Mar 2021 16:21:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615566119; bh=pYXdNgxXLm9jRXofMIKcS3j5jzWazEkuwrSWkfABQ9g=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=KVNUVCXhng00EaJ920Tvs0q2twphNZCSQ9ASVAuEhst+KRNaP9DVlBsl5Hy5tYvRd 5iQbqrn2C7/5qQY9GdtYv8HEBXHZ59qPjNntmZbPU2sJZJfoDKzUIeWNEe2TmEVZmS EACJ8ZLlJNr3o3VGGd1JczA0c2Gk5Tp4C5EabU6p0li5HPIbaQpOWPIxu8tLfO1eqL tpSrl2ndnfZpXgQe6CDQvzptlvJsZXzAN3GekjqnWIu1RzPjnHYDhKsGcsuk/jJRdQ EzEi4F3aVhhtLVuA/CXPYOLeDkjpWewV5HCFCCMr6W6Pj6UQA21gdVXKBeluKUCCV/ CX2ULINCyKL5Q== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 2/4] gro: don't dereference napi->gro_hash[x] multiple times in dev_gro_receive() Message-ID: <20210312162127.239795-3-alobakin@pm.me> In-Reply-To: <20210312162127.239795-1-alobakin@pm.me> References: <20210312162127.239795-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org GRO bucket index doesn't change through the entire function. Store a pointer to the corresponding bucket on stack once and use it later instead of dereferencing again and again. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index adc42ba7ffd8..ee124aecb8a2 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5957,6 +5957,7 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb) { u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); + struct gro_list *gro_list = &napi->gro_hash[bucket]; struct list_head *head = &offload_base; struct packet_offload *ptype; __be16 type = skb->protocol; @@ -6024,7 +6025,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (pp) { skb_list_del_init(pp); napi_gro_complete(napi, pp); - napi->gro_hash[bucket].count--; + gro_list->count--; } if (same_flow) @@ -6033,10 +6034,10 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (NAPI_GRO_CB(skb)->flush) goto normal; - if (unlikely(napi->gro_hash[bucket].count >= MAX_GRO_SKBS)) { + if (unlikely(gro_list->count >= MAX_GRO_SKBS)) { gro_flush_oldest(napi, gro_head); } else { - napi->gro_hash[bucket].count++; + gro_list->count++; } NAPI_GRO_CB(skb)->count = 1; NAPI_GRO_CB(skb)->age = jiffies; @@ -6050,7 +6051,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (grow > 0) gro_pull_from_frag0(skb, grow); ok: - if (napi->gro_hash[bucket].count) { + if (gro_list->count) { if (!test_bit(bucket, &napi->gro_bitmask)) __set_bit(bucket, &napi->gro_bitmask); } else if (test_bit(bucket, &napi->gro_bitmask)) { From patchwork Fri Mar 12 16:22:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12135223 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B1CAC4332E for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4721965034 for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232515AbhCLQWW (ORCPT ); Fri, 12 Mar 2021 11:22:22 -0500 Received: from mail2.protonmail.ch ([185.70.40.22]:22961 "EHLO mail2.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232246AbhCLQWL (ORCPT ); Fri, 12 Mar 2021 11:22:11 -0500 Date: Fri, 12 Mar 2021 16:22:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615566129; bh=kBDDl8otlT8o/LM6cr4LLdHLvlyC4ODcgI6jQUcxeAo=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=TPHwMLPFx8yRTAB179mgR4wMVehNyO1TW5OYr/dIAAqdgMy2JYicEKDCcO+VFjYyJ IKN4UB/I6vvxrSSOr6OlqtnK8/u+OW0HBgr7gKyN/K89KXe4axUMhWhSM+UzeSzEHU MwUckvm5CtZq5Yi52sx3dAKqidGMlCcb7nGVJJ+3llgTos3O7SXTnhuAZjSrRlGeeQ yJpiCrDpScE8Msn0QCnkgllWL7aDgXJjBCDkTjMaX1vwKpcJVB4OX+RWD9Lwh36tZY KsOUt9bwav3507PGqtPRUdUZ7/dikwJNHeqqHbSElUICxbx7+dzXQ5HP2mPX5GwKUv U7DaUwhm5Wo3Q== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 3/4] gro: simplify gro_list_prepare() Message-ID: <20210312162127.239795-4-alobakin@pm.me> In-Reply-To: <20210312162127.239795-1-alobakin@pm.me> References: <20210312162127.239795-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org gro_list_prepare() always returns &napi->gro_hash[bucket].list, without any variations. Moreover, it uses 'napi' argument only to have access to this list, and calculates the bucket index for the second time (firstly it happens at the beginning of dev_gro_receive()) to do that. Given that dev_gro_receive() already has a pointer to the needed list, just pass it as the first argument to eliminate redundant calculations, and make gro_list_prepare() return void. Also, both arguments of gro_list_prepare() can be constified since this function can only modify the skbs from the bucket list. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index ee124aecb8a2..65d9e7d9d1e8 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5858,15 +5858,13 @@ void napi_gro_flush(struct napi_struct *napi, bool flush_old) } EXPORT_SYMBOL(napi_gro_flush); -static struct list_head *gro_list_prepare(struct napi_struct *napi, - struct sk_buff *skb) +static void gro_list_prepare(const struct list_head *head, + const struct sk_buff *skb) { unsigned int maclen = skb->dev->hard_header_len; u32 hash = skb_get_hash_raw(skb); - struct list_head *head; struct sk_buff *p; - head = &napi->gro_hash[hash & (GRO_HASH_BUCKETS - 1)].list; list_for_each_entry(p, head, list) { unsigned long diffs; @@ -5892,8 +5890,6 @@ static struct list_head *gro_list_prepare(struct napi_struct *napi, maclen); NAPI_GRO_CB(p)->same_flow = !diffs; } - - return head; } static void skb_gro_reset_offset(struct sk_buff *skb) @@ -5958,10 +5954,10 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff { u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); struct gro_list *gro_list = &napi->gro_hash[bucket]; + struct list_head *gro_head = &gro_list->list; struct list_head *head = &offload_base; struct packet_offload *ptype; __be16 type = skb->protocol; - struct list_head *gro_head; struct sk_buff *pp = NULL; enum gro_result ret; int same_flow; @@ -5970,7 +5966,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (netif_elide_gro(skb->dev)) goto normal; - gro_head = gro_list_prepare(napi, skb); + gro_list_prepare(gro_head, skb); rcu_read_lock(); list_for_each_entry_rcu(ptype, head, list) { From patchwork Fri Mar 12 16:22:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12135225 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9CD8C433E9 for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7C10E65027 for ; Fri, 12 Mar 2021 16:22:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232529AbhCLQWX (ORCPT ); Fri, 12 Mar 2021 11:22:23 -0500 Received: from mail1.protonmail.ch ([185.70.40.18]:44310 "EHLO mail1.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232301AbhCLQWS (ORCPT ); Fri, 12 Mar 2021 11:22:18 -0500 Date: Fri, 12 Mar 2021 16:22:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615566136; bh=YmdVo4a5muajAW7ky6oQdDWmCsEr3lutZLcm8CzAK8g=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=Gxn0eCPzV4PGe17edQnPQpPylHh4XzMaGCPbxHxHjyfMIXAd3m7xStNFdThQ00ElP GjM50m5sXUXL4og8AOZ1rRcmmL1DRYJtgsSJndp4PwsXELfkguazt35Wh1mI1o98Ss BUpVrINRoxlZMa5aiSgZ3AX4vg25oQOxhMvcmnmK6DT2dGiXE9Y6yuiZWZ3lmKBXRj gKtpSjs9wCLhaOCwxKpT/MEpT+3m8iIhO/TBjMHxZ09YHItZJHDrqXNPuiZJ8/SCCr M/lTU/X0acg0yNutXVhGSMcXHse204uy6r3K+9wrASXIFtdb2GsvczGT2I59rfOnik 1ScsMIEqnRjVA== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net-next 4/4] gro: improve flow distribution across GRO buckets in dev_gro_receive() Message-ID: <20210312162127.239795-5-alobakin@pm.me> In-Reply-To: <20210312162127.239795-1-alobakin@pm.me> References: <20210312162127.239795-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Most of the functions that "convert" hash value into an index (when RPS is configured / XPS is not configured / etc.) set reciprocal_scale() on it. Its logics is simple, but fair enough and accounts the entire input value. On the opposite side, 'hash & (GRO_HASH_BUCKETS - 1)' expression uses only 3 least significant bits of the value, which is far from optimal (especially for XOR RSS hashers, where the hashes of two different flows may differ only by 1 bit somewhere in the middle). Use reciprocal_scale() here too to take the entire hash value into account and improve flow dispersion between GRO hash buckets. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index 65d9e7d9d1e8..bd7c9ba54623 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5952,7 +5952,7 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb) { - u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); + u32 bucket = reciprocal_scale(skb_get_hash_raw(skb), GRO_HASH_BUCKETS); struct gro_list *gro_list = &napi->gro_hash[bucket]; struct list_head *gro_head = &gro_list->list; struct list_head *head = &offload_base;