From patchwork Wed Jul 26 12:05:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Laight X-Patchwork-Id: 13327918 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A302CA77 for ; Wed, 26 Jul 2023 12:05:13 +0000 (UTC) Received: from eu-smtp-delivery-151.mimecast.com (eu-smtp-delivery-151.mimecast.com [185.58.85.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2694AAA for ; Wed, 26 Jul 2023 05:05:11 -0700 (PDT) Received: from AcuMS.aculab.com (156.67.243.121 [156.67.243.121]) by relay.mimecast.com with ESMTP with both STARTTLS and AUTH (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id uk-mta-140-3WAb1yA1P6OELqLW_7NEXg-1; Wed, 26 Jul 2023 13:05:08 +0100 X-MC-Unique: 3WAb1yA1P6OELqLW_7NEXg-1 Received: from AcuMS.Aculab.com (10.202.163.4) by AcuMS.aculab.com (10.202.163.4) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Wed, 26 Jul 2023 13:05:07 +0100 Received: from AcuMS.Aculab.com ([::1]) by AcuMS.aculab.com ([::1]) with mapi id 15.00.1497.048; Wed, 26 Jul 2023 13:05:07 +0100 From: David Laight To: "'willemdebruijn.kernel@gmail.com'" , "'davem@davemloft.net'" , "'dsahern@kernel.org'" , 'Eric Dumazet' , "'kuba@kernel.org'" , "'pabeni@redhat.com'" , "'netdev@vger.kernel.org'" Subject: [PATCH 1/2] Move hash calculation inside udp4_lib_lookup2() Thread-Topic: [PATCH 1/2] Move hash calculation inside udp4_lib_lookup2() Thread-Index: Adm/uWROjgkE4udKTdiQAS8haeW/0g== Date: Wed, 26 Jul 2023 12:05:07 +0000 Message-ID: <5eb8631d430248999116ce8ced13e4b2@AcuMS.aculab.com> References: In-Reply-To: Accept-Language: en-GB, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.202.205.107] Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: aculab.com Content-Language: en-US X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org Pass the udptable address into udp4_lib_lookup2() instead of the hash slot. While ipv4_portaddr_hash(net, IP_ADDR_ANY, 0) is constant for each net (the port is an xor) the value isn't saved. Since the hash function doesn't get simplified when passed zero the hash might as well be computed inside udp4_lib_lookup2(). This reduces the cache footprint and allows additional checks. Signed-off-by: David Laight --- net/ipv4/udp.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index 42a96b3547c9..ad64d6c4cd99 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -439,12 +439,18 @@ static struct sock *udp4_lib_lookup2(struct net *net, __be32 saddr, __be16 sport, __be32 daddr, unsigned int hnum, int dif, int sdif, - struct udp_hslot *hslot2, + struct udp_table *udptable, struct sk_buff *skb) { + unsigned int hash2, slot2; + struct udp_hslot *hslot2; struct sock *sk, *result; int score, badness; + hash2 = ipv4_portaddr_hash(net, daddr, hnum); + slot2 = hash2 & udptable->mask; + hslot2 = &udptable->hash2[slot2]; + result = NULL; badness = 0; udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) { @@ -495,18 +501,12 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr, int sdif, struct udp_table *udptable, struct sk_buff *skb) { unsigned short hnum = ntohs(dport); - unsigned int hash2, slot2; - struct udp_hslot *hslot2; struct sock *result, *sk; - hash2 = ipv4_portaddr_hash(net, daddr, hnum); - slot2 = hash2 & udptable->mask; - hslot2 = &udptable->hash2[slot2]; - /* Lookup connected or non-wildcard socket */ result = udp4_lib_lookup2(net, saddr, sport, daddr, hnum, dif, sdif, - hslot2, skb); + udptable, skb); if (!IS_ERR_OR_NULL(result) && result->sk_state == TCP_ESTABLISHED) goto done; @@ -525,13 +525,9 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr, goto done; /* Lookup wildcard sockets */ - hash2 = ipv4_portaddr_hash(net, htonl(INADDR_ANY), hnum); - slot2 = hash2 & udptable->mask; - hslot2 = &udptable->hash2[slot2]; - result = udp4_lib_lookup2(net, saddr, sport, htonl(INADDR_ANY), hnum, dif, sdif, - hslot2, skb); + udptable, skb); done: if (IS_ERR(result)) return NULL; From patchwork Wed Jul 26 12:05:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Laight X-Patchwork-Id: 13327919 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BB0BBE5D for ; Wed, 26 Jul 2023 12:06:05 +0000 (UTC) Received: from eu-smtp-delivery-151.mimecast.com (eu-smtp-delivery-151.mimecast.com [185.58.86.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A03EAE0 for ; Wed, 26 Jul 2023 05:06:03 -0700 (PDT) Received: from AcuMS.aculab.com (156.67.243.121 [156.67.243.121]) by relay.mimecast.com with ESMTP with both STARTTLS and AUTH (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id uk-mta-65-dxeZYJvvPxaSglt4r8Eqag-1; Wed, 26 Jul 2023 13:06:01 +0100 X-MC-Unique: dxeZYJvvPxaSglt4r8Eqag-1 Received: from AcuMS.Aculab.com (10.202.163.4) by AcuMS.aculab.com (10.202.163.4) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Wed, 26 Jul 2023 13:05:59 +0100 Received: from AcuMS.Aculab.com ([::1]) by AcuMS.aculab.com ([::1]) with mapi id 15.00.1497.048; Wed, 26 Jul 2023 13:05:59 +0100 From: David Laight To: "'willemdebruijn.kernel@gmail.com'" , "'davem@davemloft.net'" , "'dsahern@kernel.org'" , 'Eric Dumazet' , "'kuba@kernel.org'" , "'pabeni@redhat.com'" , "'netdev@vger.kernel.org'" Subject: [PATCH 2/2] Rescan the hash2 list if the hash chains have got cross-linked. Thread-Topic: [PATCH 2/2] Rescan the hash2 list if the hash chains have got cross-linked. Thread-Index: Adm/uYXsrlBDnyZsQsqXScrMcg4Dhw== Date: Wed, 26 Jul 2023 12:05:59 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-GB, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.202.205.107] Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: aculab.com Content-Language: en-US X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org udp_lib_rehash() can get called at any time and will move a socket to a different hash2 chain. This can cause udp4_lib_lookup2() (processing incoming UDP) to fail to find a socket and an ICMP port unreachable be sent. Prior to ca065d0cf80fa the lookup used 'hlist_nulls' and checked that the 'end if list' marker was on the correct list. Rather than re-instate the 'nulls' list just check that the final socket is on the correct list. The cross-linking can definitely happen (see earlier issues with it looping forever because gcc cached the list head). Fixes: ca065d0cf80fa ("udp: no longer use SLAB_DESTROY_BY_RCU") Signed-off-by: David Laight --- net/ipv4/udp.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index ad64d6c4cd99..ed92ba7610b0 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -443,6 +443,7 @@ static struct sock *udp4_lib_lookup2(struct net *net, struct sk_buff *skb) { unsigned int hash2, slot2; + unsigned int hash2_rescan; struct udp_hslot *hslot2; struct sock *sk, *result; int score, badness; @@ -451,9 +452,12 @@ static struct sock *udp4_lib_lookup2(struct net *net, slot2 = hash2 & udptable->mask; hslot2 = &udptable->hash2[slot2]; +rescan: + hash2_rescan = hash2; result = NULL; badness = 0; udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) { + hash2_rescan = udp_sk(sk)->udp_portaddr_hash; score = compute_score(sk, net, saddr, sport, daddr, hnum, dif, sdif); if (score > badness) { @@ -467,6 +471,16 @@ static struct sock *udp4_lib_lookup2(struct net *net, badness = score; } } + + /* udp sockets can get moved to a different hash chain. + * If the chains have got crossed then rescan. + */ + if ((hash2_rescan & udptable->mask) != slot2) { + /* Ensure hslot2->head is reread */ + barrier(); + goto rescan; + } + return result; }