From patchwork Sat Mar 13 20:30:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12137087 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A3C5C433E6 for ; Sat, 13 Mar 2021 20:30:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E851464DF3 for ; Sat, 13 Mar 2021 20:30:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233635AbhCMUaX (ORCPT ); Sat, 13 Mar 2021 15:30:23 -0500 Received: from mail-40136.protonmail.ch ([185.70.40.136]:35942 "EHLO mail-40136.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234336AbhCMUaR (ORCPT ); Sat, 13 Mar 2021 15:30:17 -0500 Date: Sat, 13 Mar 2021 20:30:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615667415; bh=+Y41TIeJ8/xi12wFQoHTtKGc2Uu9MqV6KIC4TT+u/qU=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=AHkb80T4Amw/xvefMpgDOC2I4wSUzXCmltZvMow4B0RyA3hMPjT13nYxG/VTjNoaX VuGvRA0hlO0Tzw3Uj6s1mVJ7E1cuI7jEsdeGSSH6wmJzBvMzugjLzZlosmSW9081Xo c8JBdeIQ/WEA1aV8Qu+gADu9GpoaRuUXrm69g4NVkwxZ4FTd+NRWJNMG/0Juz3nFy6 zfMYgTDjZy0ogX8HFyWJJmnlpOjiJiDMdVgppkrNdZW6QgjI504FAm7UJsRv7lGaQG kmraSuGAhERMd2c089P3J3PvYl/kfcvUtUhz7eWFVYpx8hC+HKHHIXJxWj4zj9Ss50 Vu/E0tFXjzYbg== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 1/3] gro: simplify gro_list_prepare() Message-ID: <20210313202946.59729-2-alobakin@pm.me> In-Reply-To: <20210313202946.59729-1-alobakin@pm.me> References: <20210313202946.59729-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org gro_list_prepare() always returns &napi->gro_hash[bucket].list, without any variations. Moreover, it uses 'napi' argument only to have access to this list, and calculates the bucket index for the second time (firstly it happens at the beginning of dev_gro_receive()) to do that. Given that dev_gro_receive() already has an index to the needed list, just pass it as the first argument to eliminate redundant calculations, and make gro_list_prepare() return void. Also, both arguments of gro_list_prepare() can be constified since this function can only modify the skbs from the bucket list. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index 2bfdd528c7c3..1317e6b6758a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5858,15 +5858,13 @@ void napi_gro_flush(struct napi_struct *napi, bool flush_old) } EXPORT_SYMBOL(napi_gro_flush); -static struct list_head *gro_list_prepare(struct napi_struct *napi, - struct sk_buff *skb) +static void gro_list_prepare(const struct list_head *head, + const struct sk_buff *skb) { unsigned int maclen = skb->dev->hard_header_len; u32 hash = skb_get_hash_raw(skb); - struct list_head *head; struct sk_buff *p; - head = &napi->gro_hash[hash & (GRO_HASH_BUCKETS - 1)].list; list_for_each_entry(p, head, list) { unsigned long diffs; @@ -5892,8 +5890,6 @@ static struct list_head *gro_list_prepare(struct napi_struct *napi, maclen); NAPI_GRO_CB(p)->same_flow = !diffs; } - - return head; } static void skb_gro_reset_offset(struct sk_buff *skb) @@ -5957,10 +5953,10 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb) { u32 hash = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); + struct list_head *gro_head = &napi->gro_hash[hash].list; struct list_head *head = &offload_base; struct packet_offload *ptype; __be16 type = skb->protocol; - struct list_head *gro_head; struct sk_buff *pp = NULL; enum gro_result ret; int same_flow; @@ -5969,7 +5965,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (netif_elide_gro(skb->dev)) goto normal; - gro_head = gro_list_prepare(napi, skb); + gro_list_prepare(gro_head, skb); rcu_read_lock(); list_for_each_entry_rcu(ptype, head, list) { From patchwork Sat Mar 13 20:30:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12137089 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 594BEC433E9 for ; Sat, 13 Mar 2021 20:30:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1EE5764ECE for ; Sat, 13 Mar 2021 20:30:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234648AbhCMUaY (ORCPT ); Sat, 13 Mar 2021 15:30:24 -0500 Received: from mail-40134.protonmail.ch ([185.70.40.134]:49178 "EHLO mail-40134.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234410AbhCMUaU (ORCPT ); Sat, 13 Mar 2021 15:30:20 -0500 Date: Sat, 13 Mar 2021 20:30:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615667419; bh=DnDoi6Lmfg94w7sOkyFM7FB6VyKUOf07zHIo4MZA/Cg=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=KvqIDRGtNKVPx5uG9YerkyQl3N1CJnINMgTHxmftAcTiLHHa41/f5C0HZVjRizpT7 PmZ/s47R27kYsuSjzIaDoHpUbvxxBbZnKgrS6i8CJteVgs3zBEFJfmry8FuPQbe1FX szl8KYXjVHR5j129ss46aG/wpv6f15aUYu/EMnXejnOlzwRMqkgrJDqcSdwdb2lDPR gW1JvOcCRllwCTcWXIagb14VrpCD3izrJnCdUDQmyl0YD+eu537m1e0mOG/57hxGh/ +VDy4EZ9C8NHD0gasuG+SL4BQZXc0uuZUxSSJBsos1lUL8FMT+wGLdU/ySb/ldrjw3 APB59EOK8Uo9w== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 2/3] gro: consistentify napi->gro_hash[x] access in dev_gro_receive() Message-ID: <20210313202946.59729-3-alobakin@pm.me> In-Reply-To: <20210313202946.59729-1-alobakin@pm.me> References: <20210313202946.59729-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org GRO bucket index doesn't change through the entire function. Store a pointer to the corresponding bucket instead of its member and use it consistently through the function. It is performance-safe since &gro_list->list == gro_list. Misc: remove superfluous braces around single-line branches. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index 1317e6b6758a..b635467087f3 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5953,7 +5953,7 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb) { u32 hash = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); - struct list_head *gro_head = &napi->gro_hash[hash].list; + struct gro_list *gro_list = &napi->gro_hash[hash]; struct list_head *head = &offload_base; struct packet_offload *ptype; __be16 type = skb->protocol; @@ -5965,7 +5965,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (netif_elide_gro(skb->dev)) goto normal; - gro_list_prepare(gro_head, skb); + gro_list_prepare(&gro_list->list, skb); rcu_read_lock(); list_for_each_entry_rcu(ptype, head, list) { @@ -6001,7 +6001,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff pp = INDIRECT_CALL_INET(ptype->callbacks.gro_receive, ipv6_gro_receive, inet_gro_receive, - gro_head, skb); + &gro_list->list, skb); break; } rcu_read_unlock(); @@ -6020,7 +6020,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (pp) { skb_list_del_init(pp); napi_gro_complete(napi, pp); - napi->gro_hash[hash].count--; + gro_list->count--; } if (same_flow) @@ -6029,16 +6029,16 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (NAPI_GRO_CB(skb)->flush) goto normal; - if (unlikely(napi->gro_hash[hash].count >= MAX_GRO_SKBS)) { - gro_flush_oldest(napi, gro_head); - } else { - napi->gro_hash[hash].count++; - } + if (unlikely(gro_list->count >= MAX_GRO_SKBS)) + gro_flush_oldest(napi, &gro_list->list); + else + gro_list->count++; + NAPI_GRO_CB(skb)->count = 1; NAPI_GRO_CB(skb)->age = jiffies; NAPI_GRO_CB(skb)->last = skb; skb_shinfo(skb)->gso_size = skb_gro_len(skb); - list_add(&skb->list, gro_head); + list_add(&skb->list, &gro_list->list); ret = GRO_HELD; pull: @@ -6046,7 +6046,7 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff if (grow > 0) gro_pull_from_frag0(skb, grow); ok: - if (napi->gro_hash[hash].count) { + if (gro_list->count) { if (!test_bit(hash, &napi->gro_bitmask)) __set_bit(hash, &napi->gro_bitmask); } else if (test_bit(hash, &napi->gro_bitmask)) { From patchwork Sat Mar 13 20:30:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12137091 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08062C433DB for ; Sat, 13 Mar 2021 20:31:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BE3AC64EB6 for ; Sat, 13 Mar 2021 20:31:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234684AbhCMUay (ORCPT ); Sat, 13 Mar 2021 15:30:54 -0500 Received: from mail-40134.protonmail.ch ([185.70.40.134]:60471 "EHLO mail-40134.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234415AbhCMUaV (ORCPT ); Sat, 13 Mar 2021 15:30:21 -0500 Date: Sat, 13 Mar 2021 20:30:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1615667419; bh=h8AxKy9QmCvHGWAs8euuTGEWJfSRa96KUPzMBTCT3Bs=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=h3JPpOw+8WBMl4rj82j7AX9N7ZOtXgJ5vj21dlyj+DDHkvRWQ8qHHJGxDS8ZiXxzr qpbX63vj6ImP5t2h/5GbON5g2TkGc737ft5Sfo/mKihjSn0OHB1JVgCuhykpULm0Hc PyiMCbgZVBKbYS5atvIgKvmqgG1VSEL6Des9BRPP97y+TxVCUW/HGK2AousZqOfn7W ZRptEqAhTpyLczMeNThcUwCyRmn+kZeOManx75uLN8xFo9Nlcyzfc3RrhAv8ZDnBSV EdKVulCqWAsDcwKgDhhg6DcqvzBPLE5bDnE7FZgwh+Ei9hs5QBW1ADOcA03HMgpHId fZC+8Zv+O6y8A== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Alexander Lobakin , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH v2 net-next 3/3] gro: give 'hash' variable in dev_gro_receive() a less confusing name Message-ID: <20210313202946.59729-4-alobakin@pm.me> In-Reply-To: <20210313202946.59729-1-alobakin@pm.me> References: <20210313202946.59729-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org 'hash' stores not the flow hash, but the index of the GRO bucket corresponding to it. Change its name to 'bucket' to avoid confusion while reading lines like '__set_bit(hash, &napi->gro_bitmask)'. Signed-off-by: Alexander Lobakin --- net/core/dev.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) -- 2.30.2 diff --git a/net/core/dev.c b/net/core/dev.c index b635467087f3..5a2847a19cf2 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5952,8 +5952,8 @@ static void gro_flush_oldest(struct napi_struct *napi, struct list_head *head) static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff *skb) { - u32 hash = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); - struct gro_list *gro_list = &napi->gro_hash[hash]; + u32 bucket = skb_get_hash_raw(skb) & (GRO_HASH_BUCKETS - 1); + struct gro_list *gro_list = &napi->gro_hash[bucket]; struct list_head *head = &offload_base; struct packet_offload *ptype; __be16 type = skb->protocol; @@ -6047,10 +6047,10 @@ static enum gro_result dev_gro_receive(struct napi_struct *napi, struct sk_buff gro_pull_from_frag0(skb, grow); ok: if (gro_list->count) { - if (!test_bit(hash, &napi->gro_bitmask)) - __set_bit(hash, &napi->gro_bitmask); - } else if (test_bit(hash, &napi->gro_bitmask)) { - __clear_bit(hash, &napi->gro_bitmask); + if (!test_bit(bucket, &napi->gro_bitmask)) + __set_bit(bucket, &napi->gro_bitmask); + } else if (test_bit(bucket, &napi->gro_bitmask)) { + __clear_bit(bucket, &napi->gro_bitmask); } return ret;