From patchwork Sun Apr 18 11:42:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 12210041 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73D04C433ED for ; Sun, 18 Apr 2021 11:43:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4592C6044F for ; Sun, 18 Apr 2021 11:43:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230457AbhDRLnd (ORCPT ); Sun, 18 Apr 2021 07:43:33 -0400 Received: from mail1.protonmail.ch ([185.70.40.18]:52070 "EHLO mail1.protonmail.ch" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229574AbhDRLnb (ORCPT ); Sun, 18 Apr 2021 07:43:31 -0400 Date: Sun, 18 Apr 2021 11:42:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1618746181; bh=uFGkfd6SSEGyPUyf0IwGBqQSMt0EHtgOF4jDCcrsprA=; h=Date:To:From:Cc:Reply-To:Subject:From; b=OCDgkUQxPBc75HEKHfPWHcuo79aNHy6390QgJujnEubklCI857H1O2zOn1BXBYAud xh3SYKREtiBNzTlQwGaS1+p3pfyWSoIEa34INqo05sRCj/QPwxk249SMHpPWY2ClVC E2jKvtYiYDGeLM+uNfXiKivd0Cpl5LFaa40kh4SVZTsUD2gESceyXX5+Zz4QgRsFfm gCBir0x/0t9zRWDOqj+xdzHa1l4l4h0Cq31KpkbLLIxtPW0Ng8ATsRB8VvFyNmBrkz J+ePLL354nJR0HFLfGvTOZjxmQslqppFPtdT7wjSVC8DD1QTp/gVxIz0zyCt85tBG7 nqF4hhIve8CWA== To: "David S. Miller" , Jakub Kicinski From: Alexander Lobakin Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Eric Dumazet , Wei Wang , Cong Wang , Taehee Yoo , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , "Michael S. Tsirkin" , Alexander Lobakin , netdev@vger.kernel.org, linux-kernel@vger.kernel.org Reply-To: Alexander Lobakin Subject: [PATCH net] gro: fix napi_gro_frags() Fast GRO breakage due to IP alignment check Message-ID: <20210418114200.5839-1-alobakin@pm.me> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Commit 38ec4944b593 ("gro: ensure frag0 meets IP header alignment") did the right thing, but missed the fact that napi_gro_frags() logics calls for skb_gro_reset_offset() *before* pulling Ethernet header to the skb linear space. That said, the introduced check for frag0 address being aligned to 4 always fails for it as Ethernet header is obviously 14 bytes long, and in case with NET_IP_ALIGN its start is not aligned to 4. Fix this by adding @nhoff argument to skb_gro_reset_offset() which tells if an IP header is placed right at the start of frag0 or not. This restores Fast GRO for napi_gro_frags() that became very slow after the mentioned commit, and preserves the introduced check to avoid silent unaligned accesses. Fixes: 38ec4944b593 ("gro: ensure frag0 meets IP header alignment") Signed-off-by: Alexander Lobakin Reviewed-by: Eric Dumazet --- net/core/dev.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) -- 2.31.1 diff --git a/net/core/dev.c b/net/core/dev.c index 1f79b9aa9a3f..965d5f9b6fee 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -5914,7 +5914,7 @@ static struct list_head *gro_list_prepare(struct napi_struct *napi, return head; } -static void skb_gro_reset_offset(struct sk_buff *skb) +static void skb_gro_reset_offset(struct sk_buff *skb, u32 nhoff) { const struct skb_shared_info *pinfo = skb_shinfo(skb); const skb_frag_t *frag0 = &pinfo->frags[0]; @@ -5925,7 +5925,7 @@ static void skb_gro_reset_offset(struct sk_buff *skb) if (!skb_headlen(skb) && pinfo->nr_frags && !PageHighMem(skb_frag_page(frag0)) && - (!NET_IP_ALIGN || !(skb_frag_off(frag0) & 3))) { + (!NET_IP_ALIGN || !((skb_frag_off(frag0) + nhoff) & 3))) { NAPI_GRO_CB(skb)->frag0 = skb_frag_address(frag0); NAPI_GRO_CB(skb)->frag0_len = min_t(unsigned int, skb_frag_size(frag0), @@ -6143,7 +6143,7 @@ gro_result_t napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb) skb_mark_napi_id(skb, napi); trace_napi_gro_receive_entry(skb); - skb_gro_reset_offset(skb); + skb_gro_reset_offset(skb, 0); ret = napi_skb_finish(napi, skb, dev_gro_receive(napi, skb)); trace_napi_gro_receive_exit(ret); @@ -6232,7 +6232,7 @@ static struct sk_buff *napi_frags_skb(struct napi_struct *napi) napi->skb = NULL; skb_reset_mac_header(skb); - skb_gro_reset_offset(skb); + skb_gro_reset_offset(skb, hlen); if (unlikely(skb_gro_header_hard(skb, hlen))) { eth = skb_gro_header_slow(skb, hlen, 0);