From patchwork Tue Oct 22 13:43:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13845723 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3413C19EED0 for ; Tue, 22 Oct 2024 13:44:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604653; cv=fail; b=WxvCkGpqylzdVvE2QvEnuL/xkDerS+EjTC/kY5OZvUR2T86bdQ62VwIoZR+y/1TV5lUzZvmmg1JRaR4NJ4lcvdQn0VbrmF6GAFEFVhqCL7YGYYDoHHXqb3dqrBsQWb+VAlNVtBkxCkGjSvT3ZDKy5sihNuNaFZaKr6grloeDoYw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604653; c=relaxed/simple; bh=SW7NHBWvw14tlTZSJoxluwHNg+nG2tk4ZZ+/uA+uqdM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=kJVpr97fbkpj48YFJf/jwj7WA52U0wLZnMDecJarmzE3Adq/CxxL+q2hSDWny2lvAc9d66+1Oaa3jAlp3aj5v6bQmZjeq4jH3nbnwl4+s0+CDRt7dF6dOnblRjA0zginOQNdp9wruqi+62cRntYS7+Wth56D/1X/ZjBTCf2IUTg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=YCcgnjfj; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="YCcgnjfj" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03lp2239.outbound.protection.outlook.com [104.47.51.239]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id A285E200058; Tue, 22 Oct 2024 13:44:01 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HMxQajFzNWZm6CC6Zxp9iuN+zfk2hPyB2h4MNN91yJu+jx00RVcsC9QmESjXJ+h8Vm3eOOdpypI7HtQQDzI1WQrXsFB3C2kYsHUa8mE11lkEv3GLwCWbXAmi+9BjbN4O7+oZQZ4CyBIlEAHJj9wA3nZk0caviNgFNzsyt6r90dyoi7Htvlat8uO2R8vcyuZjxthrT0d9zM2H1N4InOhseu2/zPg7c37sYUA+3z5sperrr5CXuHxkruMvfJiSpse+RtQdQX2YvlpXj9fem6zC9RHhTvra8dcZqY6wkkmRm+Wqiwu/LQizLoTJaXs+Cn7ybIwzi7sqP+Xqc+SVfzJT8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cylQm6lZRUJ/JcocrWyTx3aP0fgE/+335oaJiISgVbE=; b=QIQkNW1P57UyGC7QncjdQUNu5cKOcbBorS0LStAQMjxCLfX87h/l/O19w7C7GWcUkSSuPZY9LBwsXafihw8BlKvd9Dr38MmmCZhD+GdyzSR+O7QeYRqZ8oL7TJK3frgskmqdidxteuHXQuumqsiBLw+JEg/Ui5mTa96sP1cypPzmhhA+eh4VS+XXj3J4UFbQ8l+b3j58ZBVt4J1dSzfas1etkLQlIzdc+4oPua2FHmbCDS9CVgTLtSYZMYtNFHgx3VhbiYg0V30RruLTMnjy2vZtIX8/AROgDuTAL6dZOG8qPOPjNsCffmRDUgEfCxSBgb0KqIgusLgQFGAlyZI5FQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cylQm6lZRUJ/JcocrWyTx3aP0fgE/+335oaJiISgVbE=; b=YCcgnjfjq2Ij0Qd5Td3kaUQuV57qBZY4/1POlECBxfWf0Red+rjlBpek6NCaIncVWu77gwDAjCkY/+swsxZHZ6rWxlW65UUC3zM9SdmaktpDdoC4ZCljT2Z/WygD5etPJ57nUWlMuWbqPU7hZ9CUEHmWkIC5txkXSSlhAEIrLGM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by DU0PR08MB9077.eurprd08.prod.outlook.com (2603:10a6:10:471::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 13:43:54 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 13:43:54 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v7 1/6] neighbour: Add hlist_node to struct neighbour Date: Tue, 22 Oct 2024 13:43:36 +0000 Message-ID: <20241022134343.3354111-2-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241022134343.3354111-1-gnaaman@drivenets.com> References: <20241022134343.3354111-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0192.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::36) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|DU0PR08MB9077:EE_ X-MS-Office365-Filtering-Correlation-Id: 347707d3-9a83-4e35-53d8-08dcf29f9167 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: 0UpX7mf9mDFXoF4zbq9gqj9BLNszikoAiud//6Ja8fT1NbMrTILS1E3z1gvGCy/X/cu/mrHPE02mACGd04nt6tFJM4R04CXBaXA1ScFXdRNwlnV19Llk1WVbzksVY5qy1FPL3Zy1gpkhwdvQDHwlslSUTNNsz74o1NvY2d19qtV/ybhtnxVklzcA2T0gEp7A0hv97eGal8VBW8plDFht6eZza1SlnAbPsO1H7z9tekPaKIyL0Bb2kk2e21UenDtmJNzwM1SBhF6yqz+3o6bVExhhCy++tOrUu8eLB00IWe18XIwPZh9Mh2AtJuP7mKlE2oaOBYuv3jwuCIES7xL2dmKPvyzVnVN9+Saqb02OJeyuP/U4GSir6+T+YyMVmTNWK9hgvKGrSkLrEEXhwxuL7ExDkDKRI2d8bRk8DVcIA/D+grS2/4uV8w6PbWTNGeEuLC20sDO05uq6Hh2hmYk1MgEG8FWSf8KovlQKpBdhSSC7z7JffkwxpEnrRf0zz7N5cdEQwHND2eM2ODKUNL7OS9zXdIuf9JE7F0aEZoCEa7MUqEEqq500+JvLZRq9rVr5IoG8E05l/ywYpSezw4ObrKzfIGSNfIYTWJzkqALNMf3fD+DtplVsBQHtHIxJpu8o5ryO7dZYR2GwtLrrqVsIaQKt9WJcpoovdczRXStfg6hK3HIBKkQaIDHyDmPsplX5I9TvHLplDdGdcaB/1LnZiPj8T9hPbmhTcEjJX5SkHX8P29I66USGVwZh+oKGPLZE8xaOY/KFra4zsHGtFFErmLvyNIRc7X04Si12gvYXLdbrRxPM3NjWl64CpqcuTYY7B7UJeLQ5axi0NPdhLQs90Y74fRqcdHjnEYf3HCCCil5gKi/BDA6y2beOYrTMU/zZJu7kdxw1ot/WtL05PZuMu3ZYYNVykguch6Sg5GFsZRECUGmEZzfsf2WM9VOXrV4CambrC+IrCOTH2wWIB0BmI2z0J08szRYUe5bm8LVaIYd9ODnUyYGizzFCLTTnSrAXchNdDJMjlQyQ8xjn0th4ZQdTqrLJxeevPC0OaHmOC9byPxyT5ytG1R+HiO0sEi46w4GLOm/fdDJ/fW/03AJ8S0PvCgBEfuj2///rQI5HIPO3kKbTAlll5Etr3bcFqHt518roKAApDACtS/+2L1OOKexet7bQR7f7Yk32UjbqmTXkCs1WA+2W/JiXrNJtzI7OiJi0FVKFJJHIfRtoJEGsjytT6X90HxrLImnBNE74vEa+3hLYSMTMHCv1dDUqbceqBlhEEvQ+kNdB/yZJRg0ZLAibPhkSaN/KGZb4J/ZnYCZsCSfB+p4F1KhMjC12TPnDtw6GozQTs9vOxn1JjhhUj2oL1CXR0ziA+gae64RXlTg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: B+Gu3ZOWrez39qo2TtAiUOcbGhSkH/vDkAMGfTH8lbVTw62ETfxBQJrwi379J8HjaSK/quHTZgxIJhW/Wf6jnV3FUP7TswMrixeDNw6zvdmYTwUK1ojHBf4z9j0zU8BbreVAF2y9gngsljL6bl/Cl4Jz+GbzaEotOefAQ+sQWQlmT3tgaLBPL1B9zEdvegO40PrH2gh3bxlOjfbTKS6vasZjV1/sNqcAe4HuYR2sbsnxwOOEjFzC8Ulp31ADzOeeqx7SfQ8fqtUspo5mq0bDP/OaKRqEgsXtgvvu3TmBA4k3YXUJC5A0S2rRsGIVQEhevwmMdY7GP87orLwdi+XCmmEzmLXzM3diTCwtVEkvSmYUzb9GdSz5suyWYdJTekxbX0VZQ+frt5iOEf7LJD2AH99VhFHGUYWFnk9ir81V0nxKJnKAvo4mR/diNY+f63x1+wONZgV9+0lGf15hRZ8ggvugTzWZB3SnmERggdk5nBE09+6JhDSU1CURNpV4z6IZLxHNqG2eRJ7coEqaHn3RRbOraKHLV1+v4FfW5bAJT4ERQrEDYXSoxtRsfKrGEbKec3uh4HfcbnjuOe/2ONFhraMcPpUpAs0Zknj4wJjtDeH9rSggbUbc20exf3bEuNmkafSkNowV+VaqC+1ouxAdWT8cS4/UILFbaFDEYPr1rloGa8QgLcNIt7kSi5fDOcsy8t43TIQDLKV3FCbJiwI6JPvsscV4mcVDDIvd4nS1ITwtlsW8OptqVfgLNWr66CwY46tw5D2GtUzrFV7Kq71yqsziRjLHClp5ClaPjXNCRf0teLW9w6R5eO4zZgxoTMuMtVNhvAm5CeZLZxBU15aPsir7oCV5VP/UZ/pUyjeMUy14cHzCmxETvkW8xFjubNJRmz1amLgXed0Pf/3DO19Ava2fz/nqp6WlByk1GloowGADNndtwN2IdnrXITOFjAlNR2nWPDF/p+VFm9rKOB4d5mRUqHXcAoGfYnj8DNKG1T5JqrdC5X9O0DULfX8QFVXxQQ6Oje0eFgrbYrbwF00maTCz2EsAyzw7YOxL3BJ0aG20KlHD/H8REYQfks6GivTaPyW94q68t/pz4pQu5mK6P2xP+3p1Po8eh68hNuC9gS+BAiCskGHOlTnspKU/MiM9peSCEmS7mBhpu57WZ7H8xgxnCMSI9gvWj/W8eX+8rmDA+qqhP7W10Xvf2rBj92S38mEX44HbzFHtm/fbovHaqdZxiq84ty2YsMwGmzFo1j+fkcBLU5gcf/y4OnyKeY/B87KBgc2Nc2U10om/S3/24h2thHOvNQUVE3A2gTeYycHRiycp4GmOGD3LYj5oNSrptkg2k5MWUnN7hPKh1t/2LtfHaUJjKwijl8K6Cekl77ZyXwEbh95pnpktlAuyXzfGeYTdOoYqZudEWW+JraDGhCRfzqd27B4axcBaIk7TXvzgJ57wBcx8F+GzKgKHxbIq5tFLFbkouyk+CEcWKw6R0UiZIIc2Ie9272p3/2XNChddwb5wBwXufnnt2fncyo5T94r/o5wKSjrTS2ZBeKcoYdlmQ0Ij7wxaA4sF8QEZ1a1vlehhfzBS7c9NYIbtDpES X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ZzDVbMGjr5YW+o+5k6vilHoTScJwLnMu1vJKCqjIUUfJmcZ4fKNUscr3lGzO0OcAIcPq8z3y9T08I0mBGess7bZSAAx8b3CazwZAqbnEaSDIb+xli5rxzken8zzdCkwS4NQhLVOaP+luCI+U3AvViV5Ez4ZpjOdSnoMG0Ysc25mCCCgbYb9gVHL/BNasJxFokHYBc4jzpPO6QwIFvLrmnSr1vI74u+4c0kc0pRx0fTvdIU/DOnfoxvH7tBw/WftdbRelTBtJMhtLnYg75jpOb5+2pvMmAH3wLHtalYHMwyJnIXdN2JLl2u+/Zoyvp3g+TJkHPj7qy5MQnKOK1+serK7bdCViiN+ZUcfRKTQY4MVEfuqGdISBavo2D5b+jBkABDuyLWBw3sKjSRzLXPUmtetqJ5vIO5jVr1KrHgAgygpvZfT81vejHkzLzX1CNj9cQEOjPeUG5ULoDLMYgA88sfGDrOnxe7TiJQ7zQGVJEPHPQYT8IC2nSjqVDeJpCVURQZtAnYZl1llW/pRNu5MebXIY3m36hwjYgFhc/Oy4XeAY5uvF3pqFtZXxBxbuRnYkU0CbdpO4iK2+5Tih4AV943cHzICdj5iA/muRP8d4weoB4wOMP5iiadJ1BLh9jgUJ X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 347707d3-9a83-4e35-53d8-08dcf29f9167 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 13:43:53.4570 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4C7U8l3mJ1CftZ9m4ypZ9c5c21jf9gW6ZLJZOZaMn3x4e+zMtW5SNvL7p0rYdkIl0J74pyiZe2qPIeyRxzNqoA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9077 X-MDID: 1729604642-cJWXA_s_hfV8 X-MDID-O: eu1;ams;1729604642;cJWXA_s_hfV8;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Add a doubly-linked node to neighbours, so that they can be deleted without iterating the entire bucket they're in. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 ++ net/core/neighbour.c | 40 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 3887ed9e5026..0402447854c7 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -136,6 +136,7 @@ struct neigh_statistics { struct neighbour { struct neighbour __rcu *next; + struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -191,6 +192,7 @@ struct pneigh_entry { struct neigh_hash_table { struct neighbour __rcu **hash_buckets; + struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; struct rcu_head rcu; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 395ae1626eef..7df4cfc0ac9a 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -217,6 +217,7 @@ static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, neigh = rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock)); rcu_assign_pointer(*np, neigh); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); retval = true; } @@ -403,6 +404,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); neigh_mark_dead(n); @@ -530,27 +532,47 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { + size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); size_t size = (1 << shift) * sizeof(struct neighbour *); - struct neigh_hash_table *ret; struct neighbour __rcu **buckets; + struct hlist_head *hash_heads; + struct neigh_hash_table *ret; int i; + hash_heads = NULL; + ret = kmalloc(sizeof(*ret), GFP_ATOMIC); if (!ret) return NULL; if (size <= PAGE_SIZE) { buckets = kzalloc(size, GFP_ATOMIC); + + if (buckets) { + hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); + if (!hash_heads) + kfree(buckets); + } } else { buckets = (struct neighbour __rcu **) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); + + if (buckets) { + hash_heads = (struct hlist_head *) + __get_free_pages(GFP_ATOMIC | __GFP_ZERO, + get_order(hash_heads_size)); + kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); + if (!hash_heads) + free_pages((unsigned long)buckets, get_order(size)); + } } - if (!buckets) { + if (!buckets || !hash_heads) { kfree(ret); return NULL; } ret->hash_buckets = buckets; + ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) neigh_get_hash_rnd(&ret->hash_rnd[i]); @@ -562,8 +584,10 @@ static void neigh_hash_free_rcu(struct rcu_head *head) struct neigh_hash_table *nht = container_of(head, struct neigh_hash_table, rcu); + size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); struct neighbour __rcu **buckets = nht->hash_buckets; + struct hlist_head *hash_heads = nht->hash_heads; if (size <= PAGE_SIZE) { kfree(buckets); @@ -571,6 +595,13 @@ static void neigh_hash_free_rcu(struct rcu_head *head) kmemleak_free(buckets); free_pages((unsigned long)buckets, get_order(size)); } + + if (hash_heads_size < PAGE_SIZE) { + kfree(hash_heads); + } else { + kmemleak_free(hash_heads); + free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + } kfree(nht); } @@ -607,6 +638,8 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, new_nht->hash_buckets[hash], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(new_nht->hash_buckets[hash], n); + hlist_del_rcu(&n->hash); + hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } } @@ -717,6 +750,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, rcu_dereference_protected(nht->hash_buckets[hash_val], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(nht->hash_buckets[hash_val], n); + hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -1002,6 +1036,7 @@ static void neigh_periodic_work(struct work_struct *work) rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3131,6 +3166,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); } else np = &n->next; From patchwork Tue Oct 22 13:43:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13845721 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41E511E481 for ; Tue, 22 Oct 2024 13:44:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604651; cv=fail; b=jS0/uyXRMjyru9dK3ZLWKeZD8H0ZYd4hSa1T1kBrZCtiDPq6SQJSkH6JR/XMjHt25eGycwvZr0i8LOz0XUcXllV73pwY1cNFYCe7EiNM3tY/Dsp6M30KABsy/kKzJRugu3W2o7LzrYIp40LxqX1PNqg92NABwNxPnor7ZNY6q8A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604651; c=relaxed/simple; bh=A+5WLTVTgKnbJgd5p+xzqrEFTvxsdewGTqgillMnlVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=lLuxXGJIHokOGN38j8PcQL5FLzQtpDqd9naCGP19AuB8RZTQ+8a9V+8KTJiSKfEA6+g209HEJwW+gmB7IFTRAGt2tLEb0twMZzuMhoKc2mxRKZa9PRT2WNCESIHu6B3tFlbSSZPbNaOYdVtaclOXtX7afLKIQOmQ3wiJWnT725s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=qxqcUkl6; arc=fail smtp.client-ip=185.183.29.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="qxqcUkl6" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03lp2239.outbound.protection.outlook.com [104.47.51.239]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 02D75200058; Tue, 22 Oct 2024 13:44:00 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EbAP+jehdwIIoc/AIQB7M9ZiP9kNVEO4lhrKv7sgdYQD/RshywM7SA3G5gbG7K+6Tpc3RXY1hd6CiW8rgk+W9c5JwlIj2DOjify/zMWJ9JrbxXoNJvGl9S0DABDraZvYC/fHjK/xL/diL0n6XXa1QaYRggfZXEyHUFvsqXDlQ2++ndKisuHDXQXpPzlC02kmN+g0PVyIAXbamMZAseplK9ytEh1DwRT0aJLonMoinq5WoqrpbTtEbWeSv90x8hS/WoLW7ksKQuCxCTm6X39O38EgGdAqjo0o5Zn4R+3rrZVusfmDqeHqHN66wpcggAZhYlBIuViwKenCz9ihSya+bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S82OqHsPVGbQfGIM4S/NdJl8U8lhgnpRIP+IpeCwvVw=; b=fy/ACKewfP7sEl0cZE4T+cAE4k27kkH0B8TZrwPgTLxVnb4AjcffJGpgOFG6GbpRIXjBRXCG3vLKaXxYS9D/j60mA1yiXHXFzJH9cvrejN6rqcPX2Y7mE/Ykc6aJVMrrb/3YZ5oHAKupXJdFzyp4y5jYpvYO/OuAjpbsCZrftUs4t2tDSRGgLtZa7FMmQ2C71EbCUFCcMJj0fT1Vo/kxK9wOtZRjuZa2V8aDz+Rtdcc1vL3iW6xB6hhEGbiMncjFXH4CfHgs583qQ7wzhp0zcLapJPUy3EocjnUe3ehrQHALz++0yKNd+DLR4M91Tcw3dLQcyJQ35HlWWSPbMh4z/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S82OqHsPVGbQfGIM4S/NdJl8U8lhgnpRIP+IpeCwvVw=; b=qxqcUkl6Nl65V+4vSMkMk9cGpgcwYrmh6Wum7R0lK1rnexWFFBmbvzRM0G5RzktY7bIHYl65livMrKC1svfS4evj1z8cQUE6khWVlntF8dU4ThowG2NI1Mpzi1ffW6fEmTi8sMRJrO0irBJNNepFYCYSTbWD5GCo62mFjLhrBN0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by DU0PR08MB9077.eurprd08.prod.outlook.com (2603:10a6:10:471::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 13:43:54 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 13:43:54 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v7 2/6] neighbour: Define neigh_for_each_in_bucket Date: Tue, 22 Oct 2024 13:43:37 +0000 Message-ID: <20241022134343.3354111-3-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241022134343.3354111-1-gnaaman@drivenets.com> References: <20241022134343.3354111-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0192.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::36) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|DU0PR08MB9077:EE_ X-MS-Office365-Filtering-Correlation-Id: 79d41c71-600c-4a6c-602a-08dcf29f91fc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: LiXeIKJ7LumMcRhby78IIue/qkZAKUFqFsXBz8/BNajg1gQ8L6pHDabbQyq75/ueMCg7fF4nlDys3xJfQVsmsiXNvonLxQjZGeYOeG7S1hHJ+rrHnoe+9/sL6nWjuopzwmbd3WfCMSstpBATHINnHBGEUp2jWdF9D9kxDQ1VpnyDHYoDWLdxYnm9mKaybnfwsctPMebrTggq66ykByUA+Z01xM4VBWZCbx+ArqdD/ix4sfjazpfqADjvD0AL+76nffxxNPTi4nz6r/QWLQawxLnfbC96EHqLPsreSm94nkAj3u1xD0jMNDdJ6TelxLN1yclFbA0SoWLjmfhFy04GZ6Rq8VRfuBsGqKGWnmmdKtboX8YLa48/PwjGlJlYVLy+iKh8cc+tm6zoGWyfLo9WUDAKZIO7NyzTdrY5kxKiWgfvg0Q9/MxT/FRElNOQ/9fPo/wpXq53moZlvVeao5OP8v7MoOqAT8WyNuoNQv7DtbA3ZWmqpnfOLtc07/xisqfcu7sUQQm1g20I/iTieykIySDc5pNI/8deGiuCYsdlVgp7FqN0Li12VK95vFyScJ0aZCfwGA0MclkowuKKfuec5L88+lnnEuvW8j2Dcc0oBmrewp+tZgXwIvuoZw0NziYzwP6n2XRCdK1E/oGIPuXlxc82VYOAe36ZlT/m9NQBSe+QKCH+zhiHvIL6H1asnzrmSbt3UUQgJgbVqMXMwn+QPmZobBpGXiHZUxkDETivqynOHKSzRQkHYZKhvPdngY72RCNSPSHF21AYK5CBOWZFIxpKmPo/TJSugCHYZTRLYxtCHeL2kaStcXTeqyZSdEjvdfMPVJSb3NYlcmdKZVTfjW9+q1RGdFd2/d/4jyml5gPVK+IIwk9yq5aBUbwx+Yc5Kzz1lC2UknXmu1gKfEM+4b43gDZqFTt99c7yicdv+4t6wnDjyh4L+Vo8x34q3CJFrkVqOdiFnrNQ4HvMlDDcDggIhWKJRg59pXbUtGm5U5/HpxJZURrvQe2qNNh6g+0hIvDWl0TMxckLpGQugIpVPFPN5OAr81M9G2ekea3Oj38fDRyyXIkxWo9rMeQRbsq380pZtJJm0NExhjeh8hc7FlcEGc602KWjh2bfC6KcA/SfLhv0uozSWjIh65xm0y8asHREyGaq02yeAQyuD84Ixc6hEt7tQGNqbtfu5/xHXHQmRXXXDFOaAyLPqaqwstCm0pxoab14G9SEohjjS9ENp6oaEz8j8vRMe6v4NCYpxm3orfzzZ23r9I2kWf7uFpFr7smskQ//TfIGVYZtP89qoz94b+/U+HciyZitF8i5A+GwiVIiTp2BKJpRJb8o30K6KLaH/tQWB1IzjqfXv6NUccyJHXWOE3g8jPvv1uVRHXg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Sh5Cj8pRBH11tvKhv406hZ8T1jwUgIQEJ/YOnxxI49yIbnTwPAX/Hek5zzP7KB1eu3OehZb4lY2rh9PtKM3EeSM2714G7/YyZ3c4HwXAI2kSNeUAuDeob9PNqPMA9LlDxkOhz6a1+lVCaF9YZgXaTrRQ5F0S9nGgQY9Fu8GDAZlsq9GvpVLeyYOBPWI6Ka+HzRHh8Q2z9ha5WZ8yOWqQVMdCylOQxdIbjKGPQ37S4TONf38anDdICyQ09feZWR5DEDc6n7plHxOPNk5G4qlUM5OrFZ1qHZgaA7nCmMA2vC5IVaVqKCEI4fxgulodANVYm0ze6bSsGZIaQ98dKFIXYJCEaUyyS6R/qillu9ioaDCOQuYuSSk66BUON8X13j0pOiRTrcmueE6ErGdml82jnZ1phVsn5B4szM1xp8V8mAXLA86vv22ZZYBEOUvTVHAwlzYRttx0VVy7dpA9b9XyrgDzV9plLvqcfVtVHNhqOwEkY56xkHFm5//SSMlHgeCr8L4Vuosi7rEy7uNL9CXc+Yrj/EtWjPSot7dULS6WIqvCaMX+aL6qR0wdAXmHx4CAAuBgtEOdWORsIpNlh22puB0fa5RACGtQq5OTs7v0JXWyTK1Qz88xXdnczbyyI3xWfhEfbTGboWPPsUZaAqyZjJHNOmJphFC69obOJTSydGnl0F40h6C8AgLb3ROSU4zCmuUaK6/gCqp4G7/hr1wEEOj0t091NKJgt/OA/7+rO6DV4I3+c6L49zLiGEAMKc6gwXuuVwRPahpkhOXdyGOJ5jtQNpBcZWQ/PrVVN1F/xbhxiS9Gg1gL3/GTeTbsEk6Hdr1EnxT/4eLdM9pqyTI7BXwZ6MpOuJnpkJzVt5bvadyWuG/2E3yiyTe9skusK7AOm9xeTQXIGOTHsllM6AfOZeZBDWzK8/l//8KcYFIfVERJfJ6432fcyFS2C0ebzTWG7A2HzLc4Q/Bjx0+2094tQ6CA2rGF0rfL28q3AK6WqKwYycZ9Jn1l+9g8VWi9FDeMFokFCFCbEeFALtFv6JaoLT90hX4FczlxGAELJwkACS9HCF4W0LuP9AVSQyTvsQxYDbKGOVqOCWvwup119Dz0sWomnmHlDCZlRe/dkvfGW+zPeCGp0GAyfmviRzepw8QYFctgu7yZuLYp7mtMC1xu8TzS2gux4voZTAXFtW8wNo4h8709PpZ/7IqxCdFogusaCfHzX6O7bpoCAm9b2ikH2WHZR5Xc+lzUTqq7mIoX4Sfej3I1qjJQs1/rEXVYq4hdv9lPNNfjf9BYRtIxjbyNlaq9fnT794eV/V2tnXsvSvUgM8zC3A5GglOtPzQnG3ZFGM04rjKIHbpet5oRiV71XzW9Wj9CKy9olLiFRIj52HAFWkfrQ3Y+uXXiUoIArrpbrDjwV1zyuRrkwSM96ApINlTjLu8tFHv1xEzdC2ba29IXDEPSgmP2C/aPpp/i2ZyRwdQ9vo+6UB/p9r0DeGrN5n30NMhPL3MLPXt552GyrzFz+ZkbNe+q5utcgpBWIbwotPKZ/onNtE5lzB1JgqOaYMGO3eJ1tYzsbIlxeGAOHXuOAlWCT7YlMdg6nmtqWPQ0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: c598k69gp/3AwzVtgv/LnmwamJQy19UCq7R+pbbBkRFGalvikWFqC30MLJ7ecnKyNF6oWKnn1YITr4aEAHTl/QBfV5Ic19F4Ew3W80Om6jDl3p6WWPToAk1c5suoeXsZUf9+ptOr74t9Q/0C1lVzPgzig5hc/L8jhQ3RwoJK9u2K4J+7cfkyiQjvEBk/DxokCd1pM+4eRVIQljGJH1vnwDrl3EnWs4Y3Fc8GNevJ7Ia1gRhlMRE6ITkLAc3kO0JoDEzxGG3vhGEdeWXCz//3Xf5njvCN7Z/MM+BaGOvHxU7JJb84Wzj8PDRpVq3h3oCmIr9ATSeg2fAU9bndQXYxAcw/4/rAuU2aOrsRwCylSzHO0I64fE2WwrpNmnkKvMKg5GXukfnHAwy3ehnmuyeYveF6gfR6oBQL7mz0zEKVApDSEdsa5g4x9tnGcfVAGfVwWulgemUPcP5Y+i7Nwm0AFa19LGwkKI2po/cjveHqkQ/RDh4GSs0ywACiE6rn/wSCeahE4d+TeqYiqpsla+qlAdsAGNVh8JO0jwJ1Nij6aNMMVbjFsinPq0BIwgIvGjP1o/k2pWmOcwf0NL8zAeYPheNHNSZamkIsyPpmFMHOTzUkxH612xBjoQ4HfulExE0x X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 79d41c71-600c-4a6c-602a-08dcf29f91fc X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 13:43:54.3974 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PXasvq61ZRYSjMQkFf9ttTwX9x8urxmIwpYUx5bhqN9/TKu94WvSj26dHCDzpxKc2xq4Py5lkfJwB3+nrXWBIg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9077 X-MDID: 1729604641-BJzk2hdVjgyq X-MDID-O: eu1;ams;1729604641;BJzk2hdVjgyq;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Introduce neigh_for_each_in_bucket in neighbour.h, to help iterate over the neighbour table more succinctly. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 0402447854c7..69aaacd1419f 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -277,6 +277,10 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; +#define neigh_for_each_in_bucket(pos, head) hlist_for_each_entry(pos, head, hash) +#define neigh_for_each_in_bucket_safe(pos, tmp, head) \ + hlist_for_each_entry_safe(pos, tmp, head, hash) + static inline bool neigh_key_eq32(const struct neighbour *n, const void *pkey) { return *(const u32 *)n->primary_key == *(const u32 *)pkey; From patchwork Tue Oct 22 13:43:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13845722 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC3C519EEC7 for ; Tue, 22 Oct 2024 13:44:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604652; cv=fail; b=I8SqIyquWAjVlWqDeyScu9N4eT2JS8IINjnPhIDlb0yeKu2AslAXyOMZYpeHXcHHxkpdsdN36hs5MkBVDh6+83ul4DTMh6PiATqehHvG6m0HMBayV82slARcbKkpSDQ0cVl6DPaR9PJ44kCqZv7B7uMsLxEfPT5d4+vkE5T9d3I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604652; c=relaxed/simple; bh=8obzbqWFluULiPWIcD2Ia5X94nIIJAO7gk8WhIbeMW8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=OnU0HDzjnaQwo4EpTF6clE6sMCe0niM7g90eDrtxyhYgE7iagBW5cb1FcUNri9K/YF4yYneu71Oiqpd4/JxC/7aF6IqA18I0Tw6r6EMaO622GS0GeWGd3wt87HkqNQZErSjl3P+Q3JO9jYtMTM5w9L1MEXBjpaURWKaoI7lriGc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=VxRhYkzo; arc=fail smtp.client-ip=185.183.29.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="VxRhYkzo" Received: from dispatch1-eu1.ppe-hosted.com (ip6-localhost [127.0.0.1]) by dispatch1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 74EEA8189A for ; Tue, 22 Oct 2024 13:44:09 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id D1D8D480061; Tue, 22 Oct 2024 13:44:00 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tFb7pomGdanQpVqru7X6btUka2bSkyAbqJGzOla/bWr6QSuB+VobiONwRSQ1LqHNP6qbxwF6euqNCr+80YEOiSiZNvX5ClGwExex/QgVoH3gg0eSTmJ8DB1tq/53ZPzPBR1zI4WhH/MaThReG3O+U9lgDGwPKtX0w3typPLzTuqChxPBf3FCGSrzSDXXl9j24bdAIeLd4YDOu6bEfD+M5mNFOUBeXajj15ra2kGptH6d9C0vaWrMMYrWVqLkdmqCc5WwyhqnkN1DBJtO28iG3iwI4GR47uhAwPGB5X5+z+UHLkGaLW4GVMsGnB3OywC9aLcFmbK9RoBwCz5JdhCmdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NMuORxLc2CeQrv7llk+FMcVOzXT+Hz464psQ3Jf4gos=; b=O9ybjr86HOE6eqVvxorson8EosgljZzudOlRz56pDfsWda4oFmhYg3qlAe6TiaQEt63rm2WRxQ97XDb/rROBdb6CNRp/WJuhtxo9g+DyCaaqSZaf4G1jeO+1/IAlrKxjI5bjzFkkKz45r1/j02D3aQ0obEpuXj54pADYr1HJhv3bbPO1kZ6etZ6HsD0EhKqx4+MlbCtmvbS+EMD0mi/ja135zhNjR8vjrRZsFYP7woPdfxMH0TkSLjZ91/4P/PowZFD+O6xp0NE3HuPQtU+7FJasx39QL3ZajuA+bDf+ChYbSic0E3ylLsQqc9JB/Ul7Zkw7cqXU2BoXhWbdyCIuwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NMuORxLc2CeQrv7llk+FMcVOzXT+Hz464psQ3Jf4gos=; b=VxRhYkzoUp+Y20qG/imEVFYrShrZgNfP3kv6/9xJI8ZlZfYd7UScTg4oKPzDWNOxrlpGiVWpZuB593XvuLMc0LvgWffKM3YAPVxbKR0MT4b8J3S04YXUUoYLBopfmc3lRJmUx5fArmr+5OOBzbYChXclHfn/Iw34ZOub9yeClMU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV2PR08MB9928.eurprd08.prod.outlook.com (2603:10a6:150:be::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 13:43:55 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 13:43:55 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v7 3/6] neighbour: Convert seq_file functions to use hlist Date: Tue, 22 Oct 2024 13:43:38 +0000 Message-ID: <20241022134343.3354111-4-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241022134343.3354111-1-gnaaman@drivenets.com> References: <20241022134343.3354111-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0192.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::36) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV2PR08MB9928:EE_ X-MS-Office365-Filtering-Correlation-Id: 711e122f-8dd3-4f33-8fe8-08dcf29f928a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: bH7lZNQV8fLxT79dRdArVpmufFsBREC3l2p9cY74N80MWRlOic5fnLdVxAD3fKss5iHuZ7ymjKaaGC+9TOuBKV2JCTVttzgiLDM5WIwkJUY1FLIkXCWj5n85kRex79TTTZnZbqRFZZNZ9+gIxX+Y8sg0PrHj6YakvC1K0r2zyNegKrENq819TtLup8Mrin2XPacM+x+/HXJvwpLaddvP4E7nVVphzrSslEX0NwBlfce+p4vzLnChiF2yrBh+0/nKAFgsmOXiZnV/UW0fqDQtebnSIWP7fzXUUp/POTSd4VdVysMpzo+hmW9OrYcSuixPCKfckL202lMp3ryfbDu+YKfk948TeXi5UfZhhosW3K9aupBMcCWi0qu7gIMw3GVU2zlW8qiLRTrnEMPydlqeDpsoQyuC8UvwPRUUSXZqvsWbwaAAPSDbRanh3lG5SNAAcChTqrmU7YgbJJfkISQRKBhRqj6w+W2VP/k27ABh35VYy/GbK3gCm24Sd3iBRdxNZvWwN6gl95VV/oDmhSVSCRkL3iPj6OGZ1SMH65HhFg7nzOycNJm0oQuof/b94boMfG+XmZ9kLi0+wyXqvjziq0bZ03l2Pwm3RqSZpf82oU0nu4ajgCot0u8xZXoo/fhaplXZZgspGBnTg4jfpXeiu09tFVmymWmylsZ3S+JFRO5VYFy5OFq1YLYAd7rHwIZ5CTCI8b5eyArx6f+mphNIFlO9u8n+pyv6wxscJ1qxIPUmZNRYBza6kqE3aS5WXjyvHwwBapFv4l8t4S8MkTKqiFX0H1MXPtYhdLeTrhEIPEYdKzwmx19tcX4QiStEuxP5qcOe8oN3TU8GGJqQu2CbidqXZ93mlOWX1SVzBG7OPRZDrVFoIQkuQZtcbPOcg1jEJjF8DpvAKdgm3ZGglAtYgje+3tsH28Pa//B40kWsxGBnkinIGB2Q15pFL1Or8rhuP2AqqQnfNgxYdmk1cYEp9w5Ud99ErCXxgJmsBpfS/USKEQyNSzB6S0y1g5X6dmzUgqlkdKPzrfFjWVY/mypp0Td74DQIJTZdrYU1Wo2l4FDcGDJJVGjo43H6mX4PdKapBrAazbHIxs6rBarfwMkswOJeBU97x46E9qDfVk4eqR8pOUc/Pix92rFWUFKp8VSeMlc6y1v2ofeaSfaZ4LyxF4MCuqEHDP27m/I3UbV0i/NF4TZPe4ekvK5BUQBP0lAcGScI3KIxVzE1OjxbV+MK5ec6jaNgYQFjDaEmkNOro4BMD2T8KKkXdVwMPb6XyH3VZ+SEg7xzbgdtu0SFuDK/q2iuXLuZo6KAj2/bNizD5OhTWfO5ldaS0MWToCGhC3VeWAu+68IK60ga9A6v7fX81eyW4VjaBYzAsAGqUgI9bbE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: p2bAvvZVDQ8r1VlX2IHJ410aQvskCu+yC8ziLCuV+6VQ545Aj5gnZazlWnbSVhw/dnh/jmSW3bWKlHa0Vwgj4lGXTeuycSKrg5L2ozXoaxR7tUZXOhmB/+5aPVPLrwjyTihPcNAoUMYx9rPWVGMYCsck2h7M3LOXLOmI2yt7foAp8MLlLRUehjNhB58XlDMYRRiSD9ZRI9VSKAdxBK1Eb9UPFScsL3BbhL3QAgtmNjHA687sHeluuKTjRygitGTe8n7dzH/ZXYyyEd9sTS6iBEpmA6elDOqOJrUEwy7GLqLq9YDEzD4R03qMwo74SQWZiSP8tetC8MAIT+00joPhCz5d9fc2WzX+38gOqcQvQZMwaFe4I7LNPl1n3CF6/bKib3u601dYyxMGAfFR+DwzN3fxWz4vdcamKf7VyR5YI5N/zoKu06Vuba3TrlM/VXIx4dZcmfhVALTWz5EHqvAfehd1A62Aee1/h66IHR/jJLP4NusSCaihvz0sG2lUG57K6BJtRd49g/gXCHsSmBp11OBFkH9lbPDzgqdPIMIXyDCS5JLs5XvrVT6jTwwrFrboTTwzKgFGP9kwBNRiamVV3RLq08YP7IA+98drX4I4yILchhux3eSXYrzMUDMfIiEiZG7yJbh1YBP081uFJ/ZPUDhpXsxMx4RHzKDdKWI0KNBm/5kgyxijl7wJ3l0bTjN/gCDIIiO/TFVSpF21Yb4pwvS6SvoBWwTxfmRcUjSSxtdfEsYcCGaQmU7I6vM9BDbdopItX85rt0CG2ALnEE0Z/z/9n0m0CnfTqS90IJwyfJL7yarak0lxx7MXl6mS8vStoXyiXJsDtA7N4VzLXt1FnBiFSD4CEtIP1qft3+HCWamvY7RnI3khGwZ2V7O06OAm1qz/y1HzqfgVXErUWEtBQmiLdkft5QQlyDq4DkAhYVGWpoOEOHiTq4jYb/dUlkogaSI9kiH5j1l70l3rgw3alJdUxKe49o0XmsGPHHx6X6k51x8te+cAiJMNrCGn9U+6AnuzKa6ET7KsfnfQN5DZOwYl3Z9hdIYIOb/7GR/JeUs6WoOXXMXkDyJP6wY+VFM5eIT+dEqDs/b95KyzV/LKf7yAvVD2/KXnENIhsWMivG7KgrgmssL6akaUsvMhsHecJGqgE4j5e8OZiNWWLJ1/+a4QKOtQTio+z43Jp/YrBYKJRoIo+ufy9vrIE70kO0nMX6HLtcbltbBgsEW1zBrMGbFaCfpEZ4PW/xlYQcajWiC1YUdrQumFzyIFH9LRC9tOX5o/NrZCd2iFsg+NEIJ4yTa5rclvrT4XkjbMj+rKPqoxTZplrFdecn9MG3cF9QctHKyTF4p1viu7lDZc9S/QnG/3N0Smtds7CxqKx6KD19/tomJHGr6Q4eDptNLza9rWj5MSfH3sbGjcK/yQKqO9R99UffYqrfE7DWG2I6C4PjyjI1MT+D0gRuWFm3IKsJsAdl40UoldjyXxQuI4UkvFrhA1NwlkcNo9YWy8VHUyFkv152kVn4Rzde5QKbWlFFyacdPMnqGxWiyk6LnH7RIFaFGwnCqICt+cI6ShAOw3meCjrUpeXqurYnljhd3MYse6 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ewnNN2wlIpaZn0eb07+SKO0kjd9x7wOePN2m9rLCJ61swWeKXaAvkWzK9Ym6d7Ik2VBdJMkwGH7Uk+abzlme14g/Y00HzbBsqSyO6VvYxCe2RaPud2ptSlbdDawOd0Q8STOCDMiKi9rJ+Xk9ElH1ZNHkUuoHpKRtQG4noZ/MMKLctXklwcOZAfV2ke4fuZ0hqQrirmTxO/PC+3zVmPuqNdDUIaUPHDXmd12XvpdTtxBZbgwkaJGe9DLT4iizeu0pqzmWsLnK7X6QZsNqTixZZMoq1rtMdFbcc2Qx3sbDcAHpzdWix+7Pq5nWbK0QR4RNQpzwlNJEArkr1qERLiO+DcNKFTuOES5YvSGDXF3DWbRZ/fzfCkqF3AOuY3wlTOQ2dcLR9zbdD82sXqGcBnqlwbSXnuSQG9U3FPKQE+iEoGbXjxlXdQj1vkn9GWDBu6R4bA0nUJbVB4JZyh3DxfHfmDJeNHzrYMtEIUt2jZtbyk9KXSrNSc+dMAVfL62BCjUStTD4IMiRoeEaV2y+IZxq9FS3txBIE9Xhb7PTGakZRRAzMl44Bee6E20lmj3OnMhh7EokisMD2YFyfumwkITzmlHToIienkJkPXSIQfw3bPNt7K9UMAIw/QFw3hwhea8l X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 711e122f-8dd3-4f33-8fe8-08dcf29f928a X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 13:43:55.3250 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yCxjwq1L99v4Bz0qDpTDo0qQMKPTvXwtxe5NKqG/QuEfb2g0vyd4HtQFvhLNKeg0Y7IHqsqrOXjoHqPash9cLA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB9928 X-MDID: 1729604641-uYmGoQ50e_iR X-MDID-O: eu1;ams;1729604641;uYmGoQ50e_iR;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Convert seq_file-related neighbour functionality to use neighbour::hash and the related for_each macro. Signed-off-by: Gilad Naaman --- net/core/neighbour.c | 104 ++++++++++++++++++++----------------------- 1 file changed, 48 insertions(+), 56 deletions(-) diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 7df4cfc0ac9a..80bb1eef7edf 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3226,43 +3226,53 @@ EXPORT_SYMBOL(neigh_xmit); #ifdef CONFIG_PROC_FS -static struct neighbour *neigh_get_first(struct seq_file *seq) +static struct neighbour *neigh_get_valid(struct seq_file *seq, + struct neighbour *n, + loff_t *pos) { struct neigh_seq_state *state = seq->private; struct net *net = seq_file_net(seq); + + if (!net_eq(dev_net(n->dev), net)) + return NULL; + + if (state->neigh_sub_iter) { + loff_t fakep = 0; + void *v; + + v = state->neigh_sub_iter(state, n, pos ? pos : &fakep); + if (!v) + return NULL; + if (pos) + return v; + } + + if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) + return n; + + if (READ_ONCE(n->nud_state) & ~NUD_NOARP) + return n; + + return NULL; +} + +static struct neighbour *neigh_get_first(struct seq_file *seq) +{ + struct neigh_seq_state *state = seq->private; struct neigh_hash_table *nht = state->nht; - struct neighbour *n = NULL; - int bucket; + struct neighbour *n, *tmp; state->flags &= ~NEIGH_SEQ_IS_PNEIGH; - for (bucket = 0; bucket < (1 << nht->hash_shift); bucket++) { - n = rcu_dereference(nht->hash_buckets[bucket]); - - while (n) { - if (!net_eq(dev_net(n->dev), net)) - goto next; - if (state->neigh_sub_iter) { - loff_t fakep = 0; - void *v; - v = state->neigh_sub_iter(state, n, &fakep); - if (!v) - goto next; - } - if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) - break; - if (READ_ONCE(n->nud_state) & ~NUD_NOARP) - break; -next: - n = rcu_dereference(n->next); + while (++state->bucket < (1 << nht->hash_shift)) { + neigh_for_each_in_bucket(n, &nht->hash_heads[state->bucket]) { + tmp = neigh_get_valid(seq, n, NULL); + if (tmp) + return tmp; } - - if (n) - break; } - state->bucket = bucket; - return n; + return NULL; } static struct neighbour *neigh_get_next(struct seq_file *seq, @@ -3270,46 +3280,28 @@ static struct neighbour *neigh_get_next(struct seq_file *seq, loff_t *pos) { struct neigh_seq_state *state = seq->private; - struct net *net = seq_file_net(seq); - struct neigh_hash_table *nht = state->nht; + struct neighbour *tmp; if (state->neigh_sub_iter) { void *v = state->neigh_sub_iter(state, n, pos); + if (v) return n; } - n = rcu_dereference(n->next); - - while (1) { - while (n) { - if (!net_eq(dev_net(n->dev), net)) - goto next; - if (state->neigh_sub_iter) { - void *v = state->neigh_sub_iter(state, n, pos); - if (v) - return n; - goto next; - } - if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) - break; - if (READ_ONCE(n->nud_state) & ~NUD_NOARP) - break; -next: - n = rcu_dereference(n->next); + hlist_for_each_entry_continue(n, hash) { + tmp = neigh_get_valid(seq, n, pos); + if (tmp) { + n = tmp; + goto out; } - - if (n) - break; - - if (++state->bucket >= (1 << nht->hash_shift)) - break; - - n = rcu_dereference(nht->hash_buckets[state->bucket]); } + n = neigh_get_first(seq); +out: if (n && pos) --(*pos); + return n; } @@ -3412,7 +3404,7 @@ void *neigh_seq_start(struct seq_file *seq, loff_t *pos, struct neigh_table *tbl struct neigh_seq_state *state = seq->private; state->tbl = tbl; - state->bucket = 0; + state->bucket = -1; state->flags = (neigh_seq_flags & ~NEIGH_SEQ_IS_PNEIGH); rcu_read_lock(); From patchwork Tue Oct 22 13:43:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13845719 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3FFC319E97A for ; Tue, 22 Oct 2024 13:44:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604651; cv=fail; b=g26tJMEASVSHWIqWnowdnyQWVn1l72ngXP1O0Of3WO1uHW6cdFyjSBQBbpTCQwUInwfQpmyZgPvbx0k3hYDRxfiwHrF8pO9FLU0iwivigFK5nbgNsNdyx9YFZSjqH4mPhXGd+t3I7wSnl0IBff3eXtrLxu+lCH/FyTXCCsOoW04= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604651; c=relaxed/simple; bh=Zo4lWdC90BiUcyHjNrZkCXVRKTCz8HIVW2bMI+t6ZRc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=vFhhi3mCYoC2TS2wK3LQQ2RaHsQ8OI+7A0odz3tKdH5nkaI2/XW7kjo9MERDS0/0MjrLw78g/sNHISJuHhl3B2SwHdg1F4CSrKCDDaBLSzTpzTAbIpM0zephtkoJjWROJoQbMeeni4L0TfPWI2CfaWTWU9TOpJ7WXmbaCGB7uL4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=YDo+/ZlO; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="YDo+/ZlO" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 489BF480065; Tue, 22 Oct 2024 13:44:01 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=R3geVBIqvgSvXhx1p2qzmLaRmVPwmTuy01nhbXx2E7nFxFfOCW8Ge8Gg6qVHolxUJfPJJ6GsObyTypsUFoiuUxmwVjdU86Iirg8rzlmbVyeCgb2WVMk0haGtOENaFBOytxVCbGlq3qRfRKmNMottkT2NzvcL81NLSi5nr1oJ63RNJgQCkt6Il/dnHGRhtdSG3t82bOymmSsFs/NSowKaX/lSTVFFED6cQA4Vfk56IofEb4DmbD188o9cob1tozaReYKbGvsfhuZvM/F3pBGV5PeFMnVb87cb4eOnwTHHOvL3XrKNtB47ns5pUA5ktfXJzBVhP9ysdhMs3hB2bCReGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PHp/unJ4ay0NEuM5eBhoP6ZDDlzcKP5aZQ9HI9wX2Go=; b=H3FVeoXjxcb3PpaG95y7spiFoLg4LbvyB7UMTfDT5fznVAKKNGVg0DaMndda5xiK/V21Wf59+Hl/lJKlCuuWS59wOWeQLacGptniiZas62dIal8kEqcFGUw7UaHrIet0xykcz4o04VLIbcKc2hh2QVEYctNUUESvsfiGW9SBimgZ8UXIlTbf0zLZaQwga4jFNHKnAoHCOGQ5Kjb7a+hs5Nd4+euWl2NljKR2fO7DAiq4iGkEj6Y6A6GVBAG1DDN/vE1aZ+zRwTEWN/YmpdGfA50acmZgfA6s4lvlBs4EMA497jbzr6Vyz00LTI+3Ik4W16xo9m9y4SuSIxnrcG24DQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PHp/unJ4ay0NEuM5eBhoP6ZDDlzcKP5aZQ9HI9wX2Go=; b=YDo+/ZlO/CKIGOc1RbZ8FA/5ZDyBf77Q/qAycyQo81gBvgJCPb6RGe9pLL7mWAMVNTTVjtIJPFpKzLu1BSR18tLuHHMBYqqTCM2GVfYLT3h1Xt3jIxLwTHbbP8FXXCSYW62kdrm+FZxoAidnntXZG8yT1YZKlMX0vtaHFzEXQ5k= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV2PR08MB9928.eurprd08.prod.outlook.com (2603:10a6:150:be::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 13:43:56 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 13:43:56 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v7 4/6] neighbour: Convert iteration to use hlist+macro Date: Tue, 22 Oct 2024 13:43:39 +0000 Message-ID: <20241022134343.3354111-5-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241022134343.3354111-1-gnaaman@drivenets.com> References: <20241022134343.3354111-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0192.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::36) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV2PR08MB9928:EE_ X-MS-Office365-Filtering-Correlation-Id: 95ea71a3-f3fd-4640-4d2a-08dcf29f931e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: atRNM2jMYISi4o9qTHrgzs2nkb/7gcO3JORlxLgv6idut1cTxvl8uV/y6VcSEVq1hs9ycQ6GNK08Oed0cwNoaozLm00K0OdunTheP5fXtNtmBcJhuAs0gs1WGCQwVx5/KcnHamtj9Tfh5s9aFFyHfG2HzdBcISqRIcRqIpV5LTMPu+DYsP8I2LpqQyMFpCQWGcbt3OwrENowMaCxksl8X+ADpZjFSiucBwlKOcoBbECRbCAb/ZJbt19JcMZP4jdleZOMloPqYEXsxn2iw+ZjSUpcp8mSuWVBedGWhNVDy5Z4PjDhYDzGaMC5tmArTNVYSwtVHVGmq1Pr0WbBVN0xMs1vN1KZ/oUdPJd+I0j8aJMC/+gOUeG8IkHrqTZDYB7286IWWf7KcIwGgLxC2hPMJAs2Y1vn9093wJs/1VYcs6dcBvwE9aoXSb+uz7pswJFGKOIytPzcfa3omw5qLtaoK+Kt4V0dFDHttKgsQhq5LdobhkEwOZid+/aYQ6UqQOG1Vh+xJZqcdCJDU/lrSTMISzyBjuRv6c/u5gC/MoFn9mxRqj4sMSEYvrtgGJH3/1AEnS2/+mlzpySDaGD7mqwPrfIy10dpd/kNFigKsh7kU8Iv8xZnjYoXxqtEB5vTe08zpUF+TZumQ2c++Q6MWb2kAh9PttrbjUufyNtnAHP/w8bykbidtsCvs+/1Mc0c7VcLffB76pRAI4F69hWIMeTSit85ZN1A9JDgbEF/na0nzRWm0sr0ZcXcTDSR0YQuVPmT3R5rQvO1hydtTay/0GPf73GuWMbRO5kQL7AMl6UsslvTpzryc7DwNwGv4db9VECkSeLeKumJ8nztLjUib4A6kcUQyxtPrTMV23duqv5L8hzRr7xuxga4W9Qwa9vqSTCuRdWm5Ogz+ZuUSNPESXZ3yfArqyJbpy2q9eGdGRDGNC32rtp0iBZd+7/Y2huj5GQvpLpQVNylCA2osIzBxa5dyFh61OqzjfU6cK8r6dODjTdixSh16ReMuTYXgjmjdo/M5DHdGhV0MUa/88/qyb4hnIF1obaZuVEp95boJrlfpVf48XdUpps0J94cqBIBjhyBNFJ+vxmf9KFruI384UMhXgLuVFhUVQpgyW1TaXgOhDm5a6fcFJKTvzRV7om95VeOYDqyef2nMoF9n93JUXbVqQoJ51nEZKVtSYnF4ewejXH56HHVSCI4pqoVqD/8+t8BUVWyO2/8uoUBZYg2iIgTZ2ab8TzbTps1a1b28bTKNx/WhjHqmYTGp6aXBisnQ/OQoa/Jw4pJTPWmZVi4CUlPv7ARYXQvlcdcrHmFUiicYw6epL/d4QZQdSgJMYMlsJoDPQloSDOsFfvUb9yDpVkuo0T8ntZbS6thmtrbZqtX2Ps= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mT8/yp/6cG8lLb1bsXaMIb1iuE/JyO4C6/Zz8Uia1dpb6o6zF45QJpEWIsLv/nFmhtwprAhFYr5VBXxX6lnQbySepb6K04MM213Q/jNMYpUPkbGW9uezOrihYpKlDTFqbeU89qzQ5Zxq0OxFoQj0IUiZOVEEVOgNVZIESHLBxdJ4U3JluIwRb0sCUMqYs5atq2FlfGKtxzLMJXxYFaduaaZy1NYOX4e3a38MgwrTdHRmUlTz7vmaP40RJD5d3ZsZkKT8tj+PA749pGKMVD1NzwE+cgaeM/4K9lLcPGtYljA4H/n2BsG3wii46ArXUjlIfjKKgHtIG/pueWPR4J0os8Cp+zMcs4NBdwnLwItg/SBeudG9n+uvK4/CwW5nk/dkkpfVHTaveqvLN1PRh0n1KE99AVIExDgkPehxfXUEPCyMIlgcHQ6uEOujwkoEp1UXdGXP2suwvdiCqa1V+ehdU7EatgvedfReB3Vvf44rcNtbN04D2Hj30HfSrnfjaelUxqMIW1jRNuknypghE3yf9ZwJIZ8xeUJG631jwnWY1jHMxHUTdv9BI4HzW6Z7yY3WuC5VxjWCvULCopDWXJUYJO4qeJmyeCpt9Sys4uhxVT+uNb7iarVOUL1yPb6oYDNQGgZzDfZiKG7BZIKr/8Uy9xaJyFtrwE2FZzMStZbkpvVSKJtF8HqJNs8Z1vw48Zt1qdpHrH+uuUEgVi9W+q/Wut/JFFnZ3tgyPuAq1OWGdo39lZMfddx4BVWgr0ZIREFr1nQR/7PeM0Ots28nJN6GfxWf4VO081kWkPvKkiPLF3kqQM4SNCPcBx8L6mlwPhawfX3dN5WykQ44oCPVJ68F6Vtt6sUmnQg3BHnFoPCuGjv51QTdeNJvf5cIs46FrPaZcUc7Dp+0iOeDZqwc6D8ZAB44p300WXluQn1LGlxtH+wPajJu8avpybE8Uu3jske8e/O48e2+CZ2ypqbkT9AxhZ8It2UcAHJEVDDFc9MXUouYHwXC32gEUahem4x59Ke5xyOsBWd2/m9n5HtR3oiIh9TDzkhX+EsNt7nDwfEkTbQKpykjvtQ3eSoC0PDevMv2XCjAX8ytdqQ3blASHU2CoIBEKeTzj8ZHMFA6EMIcX2cP24yg1PmG1KU1dv1mnLfiT+8LOIlr+X/K4qwNJEHpSJevbr5r/VOUq1epwiUsSJWEX/fIv1audBn2yVu/5Bq75Ng3KrQTVaL556GLynqd+/JUnSa5BXfmmvqa0n6rfypB4GY0uy5YFz/F24K5+FQJvGXpttU6u6dLAaGPUzGaiy6CfzVU6G4UdP7a5CxsnZj4W3sEdi+gSKQHmz8c217f8Nn9OINcGaV98qRCqNLhJBcLGZ3EhESu+s+5PpeKpvHYdp7KEQQtF8NsxeUUGi4uhx6GpfT7KQ9eg1SaWlx7NOgE5ElJmWTFECoGqqe8Hz0orn95AEDtwrnuThD3Q3BTwWIeAMq/2/ugcW9zwM5DEo9hCccD0nLAML1LbfAGibATNuew/1vCSVRTaKKlu2aWy+jiR+6dj4URGYdEUvjSRif5DmyqjGq4b3OcBEfd4zJHFJo1xzHa7mod05RKS8vI X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: kzxC18TYhjniV/ciPhQh3PAS2YsJZJavd0OrIsdoeIjzBHnD35AE/LPiab+ZZ2xsFFebOXvGqokhVOvsBI6ka3V2KoqS2Ckd93QibhNEbEOPYDtdvbfI+k7EmJkhcWWKbf1C+kLQNbZpoadZFICpPGZT17FEvmHoHk1sLXFHBmCIbsbbcuWGucY5SxLc4YEffQuE+SXCCdfKrlQpbkb9L9Qx4PcrdGSZGK6Vn/0iLoh11rbv5jAkYBXUeeiFHKtA2bQET7dBvjmFodtTHGzqb/efb6ioi5/VuxKYvnXAUDBRJzVJ42b1gSySu2fdUHis6nzVFsPOXQ5SCeH3PWqTegtlTdAClaEOGPn1c90+TWVhQe9MG8a5RV774mmF22Xc+drXJFr4fQrFRckTGIj0Yc4jg4MNdTBoz0fRUQ3805GoXW6Q5s9cTfXBbc+t2V/gNkgJ710VNPMxdabPlUdf9cFKCqvJoCyR7/7gt37Et9lYoPwYi8QQxv+yt0Xr4u49v+qNsZy9qlCFZ2jvh8RTNBGX+nt+0tKZNr9krZGa06idpKA549tiCzUna4vULl/AQR0eV2xeVmL+8OI9uTyGCIEq9ZUfmz8MAMAo22y196HLhPfsknTk8U8wF4CrO1ky X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 95ea71a3-f3fd-4640-4d2a-08dcf29f931e X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 13:43:56.2938 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: huMRhTvtzH47gKUN0UDknPhjJvkZow0VUlmOQhnpcnAN1yTL9vawKkKeKtCltug8pKl/rqcsVtKoaOV293Ev4g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB9928 X-MDID: 1729604642-Rt5JxVeV5kXi X-MDID-O: eu1;ams;1729604642;Rt5JxVeV5kXi;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove all usage of the bare neighbour::next pointer, replacing them with neighbour::hash and its for_each macro. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 5 +---- net/core/neighbour.c | 47 ++++++++++++++++------------------------- 2 files changed, 19 insertions(+), 33 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 69aaacd1419f..68b1970d9045 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -309,12 +309,9 @@ static inline struct neighbour *___neigh_lookup_noref( u32 hash_val; hash_val = hash(pkey, dev, nht->hash_rnd) >> (32 - nht->hash_shift); - for (n = rcu_dereference(nht->hash_buckets[hash_val]); - n != NULL; - n = rcu_dereference(n->next)) { + neigh_for_each_in_bucket(n, &nht->hash_heads[hash_val]) if (n->dev == dev && key_eq(n, pkey)) return n; - } return NULL; } diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 80bb1eef7edf..e2f7699693f0 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -388,11 +388,11 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, lockdep_is_held(&tbl->lock)); for (i = 0; i < (1 << nht->hash_shift); i++) { - struct neighbour *n; struct neighbour __rcu **np = &nht->hash_buckets[i]; + struct hlist_node *tmp; + struct neighbour *n; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { if (dev && n->dev != dev) { np = &n->next; continue; @@ -620,18 +620,14 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, return old_nht; for (i = 0; i < (1 << old_nht->hash_shift); i++) { - struct neighbour *n, *next; + struct hlist_node *tmp; + struct neighbour *n; - for (n = rcu_dereference_protected(old_nht->hash_buckets[i], - lockdep_is_held(&tbl->lock)); - n != NULL; - n = next) { + neigh_for_each_in_bucket_safe(n, tmp, &old_nht->hash_heads[i]) { hash = tbl->hash(n->primary_key, n->dev, new_nht->hash_rnd); hash >>= (32 - new_nht->hash_shift); - next = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); rcu_assign_pointer(n->next, rcu_dereference_protected( @@ -726,11 +722,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, goto out_tbl_unlock; } - for (n1 = rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock)); - n1 != NULL; - n1 = rcu_dereference_protected(n1->next, - lockdep_is_held(&tbl->lock))) { + neigh_for_each_in_bucket(n1, &nht->hash_heads[hash_val]) { if (dev == n1->dev && !memcmp(n1->primary_key, n->primary_key, key_len)) { if (want_ref) neigh_hold(n1); @@ -982,10 +974,11 @@ static void neigh_connect(struct neighbour *neigh) static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); - struct neighbour *n; + struct neigh_hash_table *nht; struct neighbour __rcu **np; + struct hlist_node *tmp; + struct neighbour *n; unsigned int i; - struct neigh_hash_table *nht; NEIGH_CACHE_STAT_INC(tbl, periodic_gc_runs); @@ -1012,8 +1005,7 @@ static void neigh_periodic_work(struct work_struct *work) for (i = 0 ; i < (1 << nht->hash_shift); i++) { np = &nht->hash_buckets[i]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; write_lock(&n->lock); @@ -2763,9 +2755,8 @@ static int neigh_dump_table(struct neigh_table *tbl, struct sk_buff *skb, for (h = s_h; h < (1 << nht->hash_shift); h++) { if (h > s_h) s_idx = 0; - for (n = rcu_dereference(nht->hash_buckets[h]), idx = 0; - n != NULL; - n = rcu_dereference(n->next)) { + idx = 0; + neigh_for_each_in_bucket(n, &nht->hash_heads[h]) { if (idx < s_idx || !net_eq(dev_net(n->dev), net)) goto next; if (neigh_ifindex_filtered(n->dev, filter->dev_idx) || @@ -3132,9 +3123,7 @@ void neigh_for_each(struct neigh_table *tbl, void (*cb)(struct neighbour *, void for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; - for (n = rcu_dereference(nht->hash_buckets[chain]); - n != NULL; - n = rcu_dereference(n->next)) + neigh_for_each_in_bucket(n, &nht->hash_heads[chain]) cb(n, cookie); } read_unlock_bh(&tbl->lock); @@ -3146,18 +3135,18 @@ EXPORT_SYMBOL(neigh_for_each); void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)) { - int chain; struct neigh_hash_table *nht; + int chain; nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); for (chain = 0; chain < (1 << nht->hash_shift); chain++) { - struct neighbour *n; struct neighbour __rcu **np; + struct hlist_node *tmp; + struct neighbour *n; np = &nht->hash_buckets[chain]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); From patchwork Tue Oct 22 13:43:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13845720 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4000B19E97C for ; Tue, 22 Oct 2024 13:44:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604651; cv=fail; b=Tp/1DbfH7wq/OWFIv8+yI8o6oEhnN781LGo6WO/9gWodx4+gZER2Qy/MPQy2NP0cwZD+qqt7RJeNJu9lx9L4ebJxG5egVcfkj+p2ZhOpl1xMQbOkQJxiPgCunzqeOu5tT8hacDeb/gUaYSye0T1vv30kcOwm7c1X7Vtobr81Nio= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604651; c=relaxed/simple; bh=Vej1c6mEEm1DDopDmadYwsUKNEaKer1nKBW/GZq7igo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QPssV2LJoNJe092ujme4f4FqTADg0g2BuWw1f3R9YafLSb1y+Tg13OZ734C8KQ+zs0UjNg0airrl6bVszYMtCUKUz7zCBJ6kxpky+9HCNZWXo38LY4nYQflf/Q9Gab5tcD9W0XzMVJ2VhKfngbIrNHmGsWRcdBfyHlR0u4BYMMk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=WOcNLexU; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="WOcNLexU" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 2813C480063; Tue, 22 Oct 2024 13:44:01 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sQ8Bg2eHk/MHX27fMLr/CGX2cbqySiuV671GoPGWIbGG31DHG59Rw6xBH+5wl6v1tpKYNd8qQXdqtN0DSc1d+ddBh5tt4+agHOi8YmuU38jSY7MyNOWTXl78VPUDyfKpUpXjxzECYONb3moHD2EjpLCcI/R7w+E5jAPw+wcmu7AnbRtxhLj4jGQgJbq7HD6XhvBU7cOlnSyQ8b+flu07ciZYOBG8fBHnyEQ3qk6sFWseQ83mnFhNVavhZw0LEp8peSRhC0BkkL9GPFK/52jf7uA2pD0llIakAl92a3FpOsguH5qtU3vEo2NnJZKBKCpTHMAo08ymtiIv0GheQohfSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/EdaahXXGmmv6NwV3IYLc1ucddY5sEKmvznadCTwOXA=; b=T+tGdOiusz1X8w98oUv9chJCZq5lf6PeDSZK+q/5Fn5yqURjMSg5ASYYQ1owZZr3MkatiBTVf3wTU2Gku9DBGCkO/zRG95JaW1PzeTY9+Qz7EVCHavDvoEak7ZhsYr+7DTfW6z2xqIYqKyiatg7849qCK6eP4IZjLIDYZydt8vn01sWJFi2DT/QjDipUE7kua5Pe2tSQsoH1WR52rIs9XU2jIFqUsHBX0kKF5riJaMteE6CwaLsLXHEbO93x8ZmubrfklpLslso3wGdRVC+9J1XJOROCJ3frzleagGGupA2VFTCXRIbs7PthD9mDw4WIU3iAFlZ0/lr7fUzmiShRvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/EdaahXXGmmv6NwV3IYLc1ucddY5sEKmvznadCTwOXA=; b=WOcNLexUFY+i/BUDOe977xZqI0hjqsRdW3i/RxLkXRl5N1gDPW8icofAiG4VTsBh4XQfAtCofZr4FU7AE5AJG+8hvwA1NjhtM18g7M66e7PtbCdpeIhgkjDEVvP+wxoLxrW7j1faA6+rMKiQHknGXFrL3uvYGNKpKHP+nShCnk4= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV2PR08MB9928.eurprd08.prod.outlook.com (2603:10a6:150:be::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 13:43:57 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 13:43:57 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v7 5/6] neighbour: Remove bare neighbour::next pointer Date: Tue, 22 Oct 2024 13:43:40 +0000 Message-ID: <20241022134343.3354111-6-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241022134343.3354111-1-gnaaman@drivenets.com> References: <20241022134343.3354111-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0192.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::36) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV2PR08MB9928:EE_ X-MS-Office365-Filtering-Correlation-Id: 235873a6-204c-4f1b-8af8-08dcf29f93ad X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: gvXXySIBlGrweHcLfWnM8BngnLJVXxltL6N0Noh4OYxuYBRaJ58P6UCV1CnjLB19TWUqXcZoawsXNURJIjnd5KImc6oBiPvZv+fWpC8pj3yl6ysNChMHAjwYKzmhzzhZvtjRdW6P8yfiliYAN2QNg/wqv5F8G7or9fdImm8hMxZS1Uitoc/2EP975rMQ8htvO6KbUXWymuJV089Bg442vavAa9kB56omIliKbW/pUeksycGcwDIEqUXaNXXuLDuIiFsEvkpS8actGSodocAI0p6n+fVwDMimIrkyRT81JlvoyDVbbkyeeLUX2aFAWrCP8+TBGpv+acg4rIfIiBe/hwCOjBhhOuAVwa5ZQEgJw4pgeoQUZr28R30uhP5knRrlPPektyAOj4CuyBuSGYg2wSTdq9jaq4BtllByQjwSICNSEetS+V32JsOLix2JdnJU23eVsnELfzlo4q/yGBsOK1xYmhQTNo9APXcqW3oUEA9p3Tkt7raEodBDFclO+q4QmM0BPdf1ManfREYa7RmAmfy1ZzrwqMY2hzBEfAHepoNNADt4u6mAaPVan9F1wntxR0e8+Oa66n6bvf5km0DkRvxgheACWkQAI7+1Gs5WFl2huejjIpO7CwaEcNGC5WxuhiMQk3omxHLVIXn2N1G2svB3pmqB2WLd0tnBCmC31YzWW/9K3lMcYtnt6ouShNHfu4Rt5TCFK4EXi9uwPCbYZfBmQkpJ59qCwsN2zKlPrt0bZl3v3nN396mhBmGZlYR2GLhySzRB9bBA0PNJDOIx9aRUOVwZ5AWz1QwQ+IGBVH6rXjuJT1V9gtgjopg9tC1uWOjZAh35UmBIu72WbxJ4Jl2QLv+/Ek5MaVSC+1u1IAMMa+6H5cKVqZ3eBiaS3tQgvyUOj5nUhEfYesxY9zgkzxCl1WVHkpgPlhWtudyrMvm9WWoYvw9dQEViqHA4+59Fawz8WxVGVwSMnBZciBnTnlLEDRcIXDI8PcQC0gHQYRuMII44IdtNSOPMrcyjf99jnzPFmi3yXpGE1OiRPiCJ9IHKtIRq9cwA+dSdVoErDxb+jUy6yAJ26t+f8fwTnfDuBn16dfqO2ZkIjALOPili0y60YNlzu+YGUABRjHvmMtJXVKENZiGo9CGLv/aoRrsw2UrbRoXfYXL1am4Uxl//iyPBQx+bwUbNeIen+iCOVzFAXsVlbbcGr3zKhxAeDtkQwVoqviADPGoSSR3mbLc1dzv1KOI1306tTVmcJtucbs9LfoXgEIvTMePecrltixN8HaQl3nSoaW49+yqbB9RBQYK4vb1UXKKPTHce8QVxoopXXy2Hd+vcVATG0JQ85XvwKR7vUMM97b9lmWAjvkz5ovoBC/jVhm5poeQaVKSJtPg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qFMRxi1DVH5/S5x8QlEYtmuQJ+erHA/ny9109xrDM7C8bS5Nu3/ywwsKUQYwZZDEdMwg99eqRcy432nU5hSF6eNIlXY4fQEBGmJZo9Ei+TmqOOzJg4+Qh3xoVW6v4R96ZJw5Ij3Nkn6I3qxhSC5KQF2kn6Aire1CSPuVpuSKaXAHRwqE1aU4UzVPfPBOy+YdEKFzVNxpKFc62NLbgFXdf6Jq2rPQtLJLx8mEpu13e8gq7oj+koaIUnsGWibSHkDGKwdltQ0Krzk9zh0Sd6v2AZslf5BaO394QLB6PW7LZCzNaVBzmKl9HGJnwESRc0HjnQkQLIYJlufCR29N5TdIowE0QQ/1a9J22SFkrT/gEUKKJigW9lYBjIL6mI5yc2a8tTVl8fuT14zE22IGgcleFLeeINGt4tdaDZQb4vOym/s73ZdIfgCjXCYRzhuIc2QnAn4GvYTx6tnaUe0lJWZeps5em1Jo+aCCCXyLBXjhKy7ywBVEEoiHYcgh500lgbO+b9GH22GUOaIuU4KZFYtgRVhOzpaSKlBexZsLjTZXDsV9Lal9wqlKIl7uNatzE7C/QKnReucOaYjtwQcpZG278Kj6Sphl/pe0yOuIEQUA3LDJvnljtNmpJAECJ+aHhRVeQzALDo/BY3OXHS74GqyzgnROFXwOcDx8PvkkXa7ZG01QF+9itqrPKcIBnV5ppEmpxKNeLeLTe1XJTN9i+R2kTa7CkLJQNYLzZsmo0s4NdGPkwIX8snDzpE1YQeLy0NsrZalFb+vAaUl3yr9DmehbDpC5OZNPS87pSLYw6VjRFI986JGmtOtE8BCoEbB39DEIEIRMhTCUY8RBiB77UGaywbbbGjpAGRI6cjuEL32nSN3BGVQyKs7+LcGH2S2rGGmmfBjIMatdWx7cpGe/uHBF6os9uieoQCFAyoa5/664Diph3VOQXcbSv/b1KXx1XWk/SF6RoeRnCp+/BEftVKqTuY7Gd7EhGVvGH1A41NhYv5ADFrMBEPFszyK4x6jcqByp4JX0mnwg9n2al4C+r2N3f5zBiQ+v0MKZVNnJ9BMdLLzc3ofwdAg0rz3hvCyZHNBlah1OoWT1wC+gbiIATfx/NrY7nxhlBTlRhlEt21XFfbs1fdzG+MpRjnAsmyndM+xYdBM90s5hHyP3J+y6+aFSScFG+DkPRWduPKYyyyD+zOpMHCthDYNgDMk2/mrIuK2zd8UkCAQxvvG9ulB1rdFafwn/R2omCZ3i1fNd/OHzIDw40RZpvaWlKqBykbhagJIaM824wOBpmE7+n1hYeVNCvLZAP5FFyAhTqeep6de2nGan59wR9E601upcuyZg5D3hmJD7v5jytaEvXKsiGitD8CHNaSGM302ZfDFnJb2G+f9PJaQJZgMGqP4ovR5cY6YKXU9FT1EcMwAmISHUsPZr2z55wvQcHKcHy66WtC1GnErVgKxX1km85TwO969OuqNMuTyvzTsEHKLi/hGvxt0Q4YjKfMUW4Ytxi/AjJDh8KhgdXQHxItCc7nANXGOKPOEn9B3MeoGo60se/e/H1pat6mfaIR0MZxqaD9pUdWcUP5z0ig9QQlFpcv1cere/iqxZ X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: WBNv9PrXIrUNm0Kn0CwLLdFYC+kNSgsptdHmDFQPhayuZjHWNOnxAb9ZwTjwEIB/zm+sy4z+oqEinVSqPC01NUxdVkTI4oZQuzSGJfvDoeC6n5CeeML/RCpjh9cCmUzHPmqZBAd9teWWxVlNah8Tsmut3wcLxxs66YWuEZWGmiO9N1J7lQqKox4w6OHV2blfY6DMOIr89OvyLSCTS9wYjd1WtkzShGALSmIWuDgDFVGWxca1xo6loaEFRVYvKqc49ye81pyK6n+up44+KHAit4Cidrmbo9DpNdObTiZWgFd9nOb3Yc8stJO9/lxTmb1od58JmPU9YDM4vSYq6wCpLbEZ5l2nlDpDt3Z/LueFLegMECCI8nJhljqRgb9SteM9FYygI1xqYaqr+BHliIm/fIk2dLe9rzoFnagK+wYBl2q01oOKpktdMPBFckNiS82AZ4+/p5d3Hb0PRWjfDREM4p54CJtU2xHl4ra3aGr3RUnznFI9nrgEJaKksfvwGU0qElsFj1KR3XwODuIeG+8aE4sMVAjTV6utlQmmjhGmXGBqOvlQ/xqeCEViUNE1iUjuLGVuV4H3KjyMobxgKedekkrc/fMNM/5PufUiPiKz5RboYsW/mw7xabtWR+fg9RgO X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 235873a6-204c-4f1b-8af8-08dcf29f93ad X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 13:43:57.2352 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MljDt71aN+yfWRlUXN5UqTZUAx64N/U9ZnrnWgvO1anqY4CB61eeCpT6i9/U4lCp7qWj8XsamAUCvNsH3aYUgA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB9928 X-MDID: 1729604641-jd6bYiFu9LVS X-MDID-O: eu1;ams;1729604641;jd6bYiFu9LVS;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove the now-unused neighbour::next pointer, leaving struct neighbour solely with the hlist_node implementation. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 4 +- net/core/neighbour.c | 120 ++++++---------------------------------- net/ipv4/arp.c | 2 +- 3 files changed, 18 insertions(+), 108 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 68b1970d9045..0244fbd22a1f 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -135,7 +135,6 @@ struct neigh_statistics { #define NEIGH_CACHE_STAT_INC(tbl, field) this_cpu_inc((tbl)->stats->field) struct neighbour { - struct neighbour __rcu *next; struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; @@ -191,7 +190,6 @@ struct pneigh_entry { #define NEIGH_NUM_HASH_RND 4 struct neigh_hash_table { - struct neighbour __rcu **hash_buckets; struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; @@ -352,7 +350,7 @@ int __neigh_event_send(struct neighbour *neigh, struct sk_buff *skb, int neigh_update(struct neighbour *neigh, const u8 *lladdr, u8 new, u32 flags, u32 nlmsg_pid); void __neigh_set_probe_once(struct neighbour *neigh); -bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl); +bool neigh_remove_one(struct neighbour *ndel); void neigh_changeaddr(struct neigh_table *tbl, struct net_device *dev); int neigh_ifdown(struct neigh_table *tbl, struct net_device *dev); int neigh_carrier_down(struct neigh_table *tbl, struct net_device *dev); diff --git a/net/core/neighbour.c b/net/core/neighbour.c index e2f7699693f0..02bc1feab611 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -205,18 +205,12 @@ static void neigh_update_flags(struct neighbour *neigh, u32 flags, int *notify, } } -static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, - struct neigh_table *tbl) +bool neigh_remove_one(struct neighbour *n) { bool retval = false; write_lock(&n->lock); if (refcount_read(&n->refcnt) == 1) { - struct neighbour *neigh; - - neigh = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); - rcu_assign_pointer(*np, neigh); hlist_del_rcu(&n->hash); neigh_mark_dead(n); retval = true; @@ -227,29 +221,6 @@ static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, return retval; } -bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl) -{ - struct neigh_hash_table *nht; - void *pkey = ndel->primary_key; - u32 hash_val; - struct neighbour *n; - struct neighbour __rcu **np; - - nht = rcu_dereference_protected(tbl->nht, - lockdep_is_held(&tbl->lock)); - hash_val = tbl->hash(pkey, ndel->dev, nht->hash_rnd); - hash_val = hash_val >> (32 - nht->hash_shift); - - np = &nht->hash_buckets[hash_val]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock)))) { - if (n == ndel) - return neigh_del(n, np, tbl); - np = &n->next; - } - return false; -} - static int neigh_forced_gc(struct neigh_table *tbl) { int max_clean = atomic_read(&tbl->gc_entries) - @@ -277,7 +248,7 @@ static int neigh_forced_gc(struct neigh_table *tbl) remove = true; write_unlock(&n->lock); - if (remove && neigh_remove_one(n, tbl)) + if (remove && neigh_remove_one(n)) shrunk++; if (shrunk >= max_clean) break; @@ -388,22 +359,15 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, lockdep_is_held(&tbl->lock)); for (i = 0; i < (1 << nht->hash_shift); i++) { - struct neighbour __rcu **np = &nht->hash_buckets[i]; struct hlist_node *tmp; struct neighbour *n; neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { - if (dev && n->dev != dev) { - np = &n->next; + if (dev && n->dev != dev) continue; - } - if (skip_perm && n->nud_state & NUD_PERMANENT) { - np = &n->next; + if (skip_perm && n->nud_state & NUD_PERMANENT) continue; - } - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); @@ -532,46 +496,26 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { - size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); - size_t size = (1 << shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets; + size_t size = (1 << shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads; struct neigh_hash_table *ret; int i; - hash_heads = NULL; - ret = kmalloc(sizeof(*ret), GFP_ATOMIC); if (!ret) return NULL; if (size <= PAGE_SIZE) { - buckets = kzalloc(size, GFP_ATOMIC); - - if (buckets) { - hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); - if (!hash_heads) - kfree(buckets); - } + hash_heads = kzalloc(size, GFP_ATOMIC); } else { - buckets = (struct neighbour __rcu **) + hash_heads = (struct hlist_head *) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); - kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); - - if (buckets) { - hash_heads = (struct hlist_head *) - __get_free_pages(GFP_ATOMIC | __GFP_ZERO, - get_order(hash_heads_size)); - kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); - if (!hash_heads) - free_pages((unsigned long)buckets, get_order(size)); - } + kmemleak_alloc(hash_heads, size, 1, GFP_ATOMIC); } - if (!buckets || !hash_heads) { + if (!hash_heads) { kfree(ret); return NULL; } - ret->hash_buckets = buckets; ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) @@ -584,23 +528,14 @@ static void neigh_hash_free_rcu(struct rcu_head *head) struct neigh_hash_table *nht = container_of(head, struct neigh_hash_table, rcu); - size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); - size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets = nht->hash_buckets; + size_t size = (1 << nht->hash_shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads = nht->hash_heads; - if (size <= PAGE_SIZE) { - kfree(buckets); - } else { - kmemleak_free(buckets); - free_pages((unsigned long)buckets, get_order(size)); - } - - if (hash_heads_size < PAGE_SIZE) { + if (size < PAGE_SIZE) { kfree(hash_heads); } else { kmemleak_free(hash_heads); - free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + free_pages((unsigned long)hash_heads, get_order(size)); } kfree(nht); } @@ -629,11 +564,6 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, hash >>= (32 - new_nht->hash_shift); - rcu_assign_pointer(n->next, - rcu_dereference_protected( - new_nht->hash_buckets[hash], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(new_nht->hash_buckets[hash], n); hlist_del_rcu(&n->hash); hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } @@ -738,10 +668,6 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, list_add_tail(&n->managed_list, &n->tbl->managed_list); if (want_ref) neigh_hold(n); - rcu_assign_pointer(n->next, - rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(nht->hash_buckets[hash_val], n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); @@ -975,7 +901,6 @@ static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); struct neigh_hash_table *nht; - struct neighbour __rcu **np; struct hlist_node *tmp; struct neighbour *n; unsigned int i; @@ -1003,7 +928,6 @@ static void neigh_periodic_work(struct work_struct *work) goto out; for (i = 0 ; i < (1 << nht->hash_shift); i++) { - np = &nht->hash_buckets[i]; neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; @@ -1014,7 +938,7 @@ static void neigh_periodic_work(struct work_struct *work) if ((state & (NUD_PERMANENT | NUD_IN_TIMER)) || (n->flags & NTF_EXT_LEARNED)) { write_unlock(&n->lock); - goto next_elt; + continue; } if (time_before(n->used, n->confirmed) && @@ -1025,9 +949,6 @@ static void neigh_periodic_work(struct work_struct *work) (state == NUD_FAILED || !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); @@ -1035,9 +956,6 @@ static void neigh_periodic_work(struct work_struct *work) continue; } write_unlock(&n->lock); - -next_elt: - np = &n->next; } /* * It's fine to release lock here, even if hash table @@ -1984,7 +1902,7 @@ static int neigh_delete(struct sk_buff *skb, struct nlmsghdr *nlh, NETLINK_CB(skb).portid, extack); write_lock_bh(&tbl->lock); neigh_release(neigh); - neigh_remove_one(neigh, tbl); + neigh_remove_one(neigh); write_unlock_bh(&tbl->lock); out: @@ -3141,24 +3059,18 @@ void __neigh_for_each_release(struct neigh_table *tbl, nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); for (chain = 0; chain < (1 << nht->hash_shift); chain++) { - struct neighbour __rcu **np; struct hlist_node *tmp; struct neighbour *n; - np = &nht->hash_buckets[chain]; neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); release = cb(n); if (release) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); - } else - np = &n->next; + } write_unlock(&n->lock); if (release) neigh_cleanup_and_release(n); diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c index 11c1519b3699..cb9a7ed8abd3 100644 --- a/net/ipv4/arp.c +++ b/net/ipv4/arp.c @@ -1215,7 +1215,7 @@ int arp_invalidate(struct net_device *dev, __be32 ip, bool force) NEIGH_UPDATE_F_ADMIN, 0); write_lock_bh(&tbl->lock); neigh_release(neigh); - neigh_remove_one(neigh, tbl); + neigh_remove_one(neigh); write_unlock_bh(&tbl->lock); } From patchwork Tue Oct 22 13:43:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13845725 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7A0919F13B for ; Tue, 22 Oct 2024 13:44:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604653; cv=fail; b=E1Z3sD1KnBVaDAP7wS4vRieMfsJQNRV5ur0OUKcYFyKYSpZXNboUoJ34chJxP7xnJmkqmuHR7LZtUeNMB48t26qlUoFCc35Ou/BdZuwcm/s1PrpqPu2H8tmDWKDHALHr7woFjlbFIE3iXUxsAoVKsM53FZd79mDFeqTCb6h+iwU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729604653; c=relaxed/simple; bh=OKCiOG3C5JsHkVQR+qOb3PLVSTNYOm9e0aXZzix9zhk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=cJg/Dq9dYfu4y3omm/RE+QNmGxU6Wdm9n2dtypu925k2JJF7dxmYqJ4pL9JSIGCULwxs/Lndz4+cWDV0gMUDBiHmTsVFg8LL6pWzhvfCGXa2iBF/PBnMMEO5fmtpBpKSGJjgvc3hW3YjE3/OchYBZGetzPzTLZeNY00mjEIGRpk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=o9RaLwNZ; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="o9RaLwNZ" Received: from dispatch1-eu1.ppe-hosted.com (ip6-localhost [127.0.0.1]) by dispatch1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 75DDA340661 for ; Tue, 22 Oct 2024 13:44:10 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2105.outbound.protection.outlook.com [104.47.18.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 6AB2448005C; Tue, 22 Oct 2024 13:44:01 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LWLj+oF5ijgdPzJDcxbSfhZ+lwz7+sXiDsCEx7LS5AGpLznyJ3c7VXwmXTbjXth6Q4oaVhuuOh+cYLxxNzTvp/HjCGdtmMq4TV2GG/FuFu3BUNdDs/LYfMNt4nNrmNJuMOs1AsrgQ//ExiUuucPD0RBgdqiq1Vp0V5QlexZMWou+8P5OqIUmJC9Sg2lkz9UHn/4yuYPkJot2xtzG8w7hbZPQZ37opuMldNQ6CVG2rybcoCZjYEQ/qQn0qD+kHTCTNm/yrfk//xRcj8cmL75FDYbO0Yr6RTespQ5XSNuBDm2IQeRMYcaseF79CdS1MmPXS3GK686KOWlIqQ0niN7dAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=23ZsMgA+4a6qSMX6V/8kkIwrJxqx0JJwTX3PBPxX/YI=; b=N/G7MUC86wBARSHk4jvMHTWyXvwYUSCwEuvJRWpAebW15XgfJB4ESD2wfBLhQBaqf9jPMAqfk++WvJzY19YxqjfuJ243Ao4B3JxwWeyjMmNeUotr2xRDwijFTPnvemgxZVcJVO9JwWDiKGvg812SnC6SNqoFuwfBp+6HuMA015SqcD3Lq2BhggHQqdH/FbYi4EkUfMkbOne14CMNbbVDW/m5oRdiXbWrzf5MR2uYlgXtCQ3wYi1AMG0HqscFFsXmZCTRQoCswk9buN6XVFTpQmrEnUUWZqLQhq/fVaSE7Eptge4gpiFrQPDafRvwG0VR+eD6PXOWraRSDD9ObO7bdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=23ZsMgA+4a6qSMX6V/8kkIwrJxqx0JJwTX3PBPxX/YI=; b=o9RaLwNZ39YmApHVkk83M4N6/duBN+ArbMDvuBsab5DEZM0+hd2NS2hL+FMUJoG+GF6ThdZsdwwpftH2q4dcyMw4kw1tQ339/0pspSMZ1a+Q4KiNeaDXwAV6ly0XWm2Oetux3Y3I+G393rAZXf6QBhOM38sJmni7HjEfcS3EhgY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV2PR08MB9928.eurprd08.prod.outlook.com (2603:10a6:150:be::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 13:43:58 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 13:43:58 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v7 6/6] neighbour: Create netdev->neighbour association Date: Tue, 22 Oct 2024 13:43:41 +0000 Message-ID: <20241022134343.3354111-7-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241022134343.3354111-1-gnaaman@drivenets.com> References: <20241022134343.3354111-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0192.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a::36) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV2PR08MB9928:EE_ X-MS-Office365-Filtering-Correlation-Id: c8494e53-450b-417e-168f-08dcf29f9441 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|52116014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: sSMC3sENgO6+F7CiEt9HJ0RdSIj1NIB/lfoTtzdbKvyj5DjlFq1U/z/OOcxZP3YrXPqAq2kc5k24zJe9GlNuxKOGJtdu2I2wwLxlmo6qoQP6ZekV99+sGBk6SYHOSjl5HWNuMVp/C3+YAKsIUePikgAaO/J0JD+Wjy88flUXdJTHEMeL+kx0eZu7oP441XLUtywQZmZdNp9mTAYkjegjBA8plC06htxKl4DPK93GV4HyY/DsYKIRy04M4oSS+9BbgK/GNaQE6OsZe3VZm8jGFciD6p8pY9XlrybRPMS7PVWjKblSozTRNMXYvBwFo20ttTxGtYBG4GcXUtWkeLaDjEPp2zZgg9I1RWpMizmihI5SrhSUeqirk/qyMy92Pv1XSAXVTYM5unbJ7LyMT3PbVZJ+VjHYu3USUmOzRHTqyjJrv3JrykkTeo4+evoyEd9s6ZXuAH2K+qPQIAk6yuHbbmXfr82ylXdZVQKhrZf+77HMmHI0eFmwZWtxWlnQSYz/B0y3oBNBwkW4do3i96kDEV/dJ6owY6kyawQ+OofKmQeomUW8wXF1Ljdioq470jsJGFQVCHbhNJIEgv6ZSRCoyuklv3Y1aFBlw52Lib7ZaZ9nmJ/kr3wfDe1jjTRDuyB2+Ak+O8hzBk/F0ljG3f+GNrn3IsvFJTdkZsuk5euIxcLwEgCuO4SUUe6DwJfAbIGvJKvDIvDkG8elU9eYe016Ti8rCKirKfqoWsxLMmPOIjxogvuv/ufwpajJPvSuiMPwHKNg674MwIIuAKL4REIkkpOAgv+3PNbtefqscuDesRH2wD7NMtBf+VXNQf1L1MwdCr20nJ6PDWMGT8m86SpflTBe5UQLVEV0TAxGjGAFZitwWDyAmCfB9Yvcqx0J5Rk9h2XmPClEuLdx+7hUwW0fWQd/j/n3FyubpSPqzdid8YDob6P5taGfXmI8TYzKFne5f0+WeZ7Haj6jS0edlw9fULJglf3bOOMY3Gft8Rjng2bhKxsfbe8CzxUV7Z9l3CErIPphE6j7v2woL0Dc3ydAeBZCsuIehGBZOmv+DE8hbXJIL/5zSH7efrP1ZPW8ki3Xu9tHCQwGJacz5wdKCuPix84NVQnvlmS+SQtzLbml3VwkXo/XzbNkHoUgI2U8WPmK+DqLrEMpLs0isy/CCT+rTXIbKfaebQ1eTq8DgUDy4uO8Fv+u0J/LvspEWbd4iEmbQYRWwUHv7QLZ3mTm3UAnzfTRs44kpE3EMJwDiBA2ngborhacaq0y6/EXlwSyRyXTyRhF8OiSxpI4hB0GsUDZXzRxvsV6eJKRLiOs59VdF6Io5DhTwY6cKB4vLKZEphwwEt0kUKvQSSluQE4QekPTf2gNZQjPKZF5hYvAjBOqOhM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(52116014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: LNT7DC6aJXU0aEk5wYHo7O2ST0l87zpKxXEP89GzkwAxS+bjbQo2+VokAyDo5kN42OMJR8q93eAtaLd1cUVZM8mBNk/Tdz7/C4vsRmRUEBnDZCKt5EqycDeclXD5cM5Tmepfuer8JYedvbUIz78HYdgoIh4ReIMXaWucg9cqtMYAgCDWbIrdYrdLLj3t++DJt6FZ2Pue7h+VM753I+3z9Gdv9jKBNjCAHO6ICKmxTB9fD+cd0KoxM34MzYtPc3223LDSAFRsW79xdEgBt8VTBbwCp9CEKO/NnoouGFslI+p5Tmm+e08Mu2kUIrdjGI8sf+oFcruYDtBydensRIKaFUGxVejbLpgAkRen/divtXOFDGxEwk980WeHnQqyFGyJ5RAhCC5YmJUXZty4xuj2sM9ZDj9W7BlBlw5X06xv9fXnvBQaf+VOEhaDpp7RNUfvxWCVe6fFGbWnKHhincHV+49J2MjeS80GH2S51mZRaLMc0kvIQIfOG7wIlKSLtpfIWXbEYCpL0qYc2WcteMux4HvFIh4p9TjHHbC+68kXJaFf0I9l5TWeNz6yF+MFTtdWsftoV8kGKFc7iwqwbHy3cS8wc0HvizCXnJRnvrr8dL+OrQp2YUE4c0B9qmylw97PkIs4EiaFkQ1H8JhySoF/IXwf14OePSv3KeP+hA31OxfNjqQ6aHa8mL+ve6sJttN7JbPOChV69Dc446PM+WrzaNhiaYEwXVqSXdziNE1Q/oAg0wrwiBO4IZJxJQ91IvJ+ghMdhcNxD8FONxoTM5EGrxcEVvbLHx3KHhFJ6GPsSl2C00/IppG48/WPIOERzcvT8u87TEJUdatxu7eZjzRrg8qpm+XgyqBkhZax/H3dUy0tOwKsybWdx1r+/FGbTcI5Oiypzd8qUKvguvbf+3zlMEeQnq+WALdvKCiiQNWRsKyyiYiHqlU06nK73pF213hRrwixeaKVlYNgIQ8k5NEf2sCAs3MCIcpXhKGUaQ0/pd4QDPV5p+DxR1kcl1h2u+Mg9NRhI8whXy6I8ENYqBItpK/Iuxf1f/uWpC2lRFxVjCB2FVbNv73eV4J8tsny5lIZJI3u0rBaAYnp0Eq/XLW0IUEmsK8t8TkSkgI+AXIaDkSNTPv25HOzEpCaqKSC+/lB7GRg1J9iMT/TchqItFItdL/F3b3VIM6X8lyNW68LtDgGcBpW254JE2VqfwgyRxVuErrhBoP2YZBguBdrvMDKLjAjxGW4DboszIq62+xhELS2d4jC+fKMhBDRuDGou/171sDgFW9FFQFaBLsGmjuQylETHQj2g71B1dv9xPk1OcaySjZehhxXiXuV5wtKpkqcTaedl8Sv2TCSRnStVjcV7rDXR14u7NpmFhKRVy/Adk8IEWuWIQcQ0Upe4v0Rs5oVINm8LWU+3K1AcbZeFDoWkwpH57nvyQSPOAh/a46FM4GemJR/Hwa5YZe1bzQiP7Bk4/N/uWafjR7QwrHMj5xuliwlJvl+Ae3IDBF37L/2IidJIF2PotZXnowPflC7uonvnzNzeVRhYJzEJkBns4jbCV8SXI99rIsNC8KHAqZ0R3T2rdwdSk070nKPLJcVkk3k X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: b4fmWipyu/cIiRD57lJFtNpI6PZHfoJsxqxPboum6sFtqUcvsiZrhWlGulULNGp5MY4mVaiN+DScfrIVNb3D+DytlTeGSidJ5u/j8t9ohNzXpyo+ZbdQidmMAzrkju76kHw7lali9YA+GEoejwUMIUyWzSD1NHDvyq663B5wVVDzGBr5pQsg0Qh3uUrAtURv+1VOMg3W8TlR7X/Ml2ATCUSfsy5lPandFEwhZwh10bPis8mM1dn3Sn9lJLfZEVwHQW9m9hIJ5r8/6sIMLEHYkMTR5oqc273Fii3BueJ3zHsRtdHlYKD+LZoP2PY9/LOOa7gEO8WGUTy01oyOOR+eCuZykoNVn5q0JmeMsf5yGVv6vur+Nx+JQJ9cQM8q+dnQ5ozpnXpngzRzbmV2G7ZQ2X8ePSoRvMNZMDim8W0V2nuXtrQ2btG3stOe8iPwBy0pPWWquAcjsi0pWk675jCXqfjqKe9UDhBvKTk6olz4+A4gGCZekrDRQFji1HJQn/LtIyKPm4Awdh1MUodyNCJtazfSN/AwMUV2gFJcum8sauNiBgm112c+HmqcQYlI/jpvjalQvFhxnnA87h6DpIkFsQZdn4xsqk6Pj/Loj/Nw2pdD/8I7/qcEuVAcXERP0byN X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: c8494e53-450b-417e-168f-08dcf29f9441 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 13:43:58.2097 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wuLYuIpHzPFcej3AOPdrIwY9x+5Mwq2gKw39ia8oIur+InrdGCNPCNL1Nd0PGpv5daAZEllEU0LRATvpmCo5Xg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB9928 X-MDID: 1729604642-JEhC97yFqzIM X-MDID-O: eu1;ams;1729604642;JEhC97yFqzIM;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Create a mapping between a netdev and its neighoburs, allowing for much cheaper flushes. Signed-off-by: Gilad Naaman --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 7 ++ include/net/neighbour.h | 9 +- include/net/neighbour_tables.h | 12 +++ net/core/neighbour.c | 96 +++++++++++-------- 5 files changed, 80 insertions(+), 45 deletions(-) create mode 100644 include/net/neighbour_tables.h diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst index db6192b2bb50..2edb6ac1cab4 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -189,4 +189,5 @@ u64 max_pacing_offload_horizon struct_napi_config* napi_config unsigned_long gro_flush_timeout u32 napi_defer_hard_irqs +struct hlist_head neighbours[2] =================================== =========================== =================== =================== =================================================================================== diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 8feaca12655e..80bde95cc302 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -52,6 +52,7 @@ #include #include #include +#include struct netpoll_info; struct device; @@ -2034,6 +2035,9 @@ enum netdev_reg_state { * @napi_defer_hard_irqs: If not zero, provides a counter that would * allow to avoid NIC hard IRQ, on busy queues. * + * @neighbours: List heads pointing to this device's neighbours' + * dev_list, one per address-family. + * * FIXME: cleanup struct net_device such that network protocol info * moves out. */ @@ -2443,6 +2447,9 @@ struct net_device { */ struct net_shaper_hierarchy *net_shaper_hierarchy; #endif + + struct hlist_head neighbours[NEIGH_NR_TABLES]; + u8 priv[] ____cacheline_aligned __counted_by(priv_len); } ____cacheline_aligned; diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 0244fbd22a1f..bb345ce8bbf8 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -29,6 +29,7 @@ #include #include #include +#include /* * NUD stands for "neighbor unreachability detection" @@ -136,6 +137,7 @@ struct neigh_statistics { struct neighbour { struct hlist_node hash; + struct hlist_node dev_list; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -236,13 +238,6 @@ struct neigh_table { struct pneigh_entry **phash_buckets; }; -enum { - NEIGH_ARP_TABLE = 0, - NEIGH_ND_TABLE = 1, - NEIGH_NR_TABLES, - NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ -}; - static inline int neigh_parms_family(struct neigh_parms *p) { return p->tbl->family; diff --git a/include/net/neighbour_tables.h b/include/net/neighbour_tables.h new file mode 100644 index 000000000000..bcffbe8f7601 --- /dev/null +++ b/include/net/neighbour_tables.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _NET_NEIGHBOUR_TABLES_H +#define _NET_NEIGHBOUR_TABLES_H + +enum { + NEIGH_ARP_TABLE = 0, + NEIGH_ND_TABLE = 1, + NEIGH_NR_TABLES, + NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ +}; + +#endif diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 02bc1feab611..f4a772c71f2f 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -61,6 +61,25 @@ static int pneigh_ifdown_and_unlock(struct neigh_table *tbl, static const struct seq_operations neigh_stat_seq_ops; #endif +static struct hlist_head *neigh_get_dev_table(struct net_device *dev, int family) +{ + int i; + + switch (family) { + default: + DEBUG_NET_WARN_ON_ONCE(1); + fallthrough; /* to avoid panic by null-ptr-deref */ + case AF_INET: + i = NEIGH_ARP_TABLE; + break; + case AF_INET6: + i = NEIGH_ND_TABLE; + break; + } + + return &dev->neighbours[i]; +} + /* Neighbour hash table buckets are protected with rwlock tbl->lock. @@ -212,6 +231,7 @@ bool neigh_remove_one(struct neighbour *n) write_lock(&n->lock); if (refcount_read(&n->refcnt) == 1) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); retval = true; } @@ -352,48 +372,42 @@ static void pneigh_queue_purge(struct sk_buff_head *list, struct net *net, static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, bool skip_perm) { - int i; - struct neigh_hash_table *nht; - - nht = rcu_dereference_protected(tbl->nht, - lockdep_is_held(&tbl->lock)); + struct hlist_head *dev_head; + struct hlist_node *tmp; + struct neighbour *n; - for (i = 0; i < (1 << nht->hash_shift); i++) { - struct hlist_node *tmp; - struct neighbour *n; + dev_head = neigh_get_dev_table(dev, tbl->family); - neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { - if (dev && n->dev != dev) - continue; - if (skip_perm && n->nud_state & NUD_PERMANENT) - continue; + hlist_for_each_entry_safe(n, tmp, dev_head, dev_list) { + if (skip_perm && n->nud_state & NUD_PERMANENT) + continue; - hlist_del_rcu(&n->hash); - write_lock(&n->lock); - neigh_del_timer(n); - neigh_mark_dead(n); - if (refcount_read(&n->refcnt) != 1) { - /* The most unpleasant situation. - We must destroy neighbour entry, - but someone still uses it. - - The destroy will be delayed until - the last user releases us, but - we must kill timers etc. and move - it to safe state. - */ - __skb_queue_purge(&n->arp_queue); - n->arp_queue_len_bytes = 0; - WRITE_ONCE(n->output, neigh_blackhole); - if (n->nud_state & NUD_VALID) - n->nud_state = NUD_NOARP; - else - n->nud_state = NUD_NONE; - neigh_dbg(2, "neigh %p is stray\n", n); - } - write_unlock(&n->lock); - neigh_cleanup_and_release(n); + hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); + write_lock(&n->lock); + neigh_del_timer(n); + neigh_mark_dead(n); + if (refcount_read(&n->refcnt) != 1) { + /* The most unpleasant situation. + * We must destroy neighbour entry, + * but someone still uses it. + * + * The destroy will be delayed until + * the last user releases us, but + * we must kill timers etc. and move + * it to safe state. + */ + __skb_queue_purge(&n->arp_queue); + n->arp_queue_len_bytes = 0; + WRITE_ONCE(n->output, neigh_blackhole); + if (n->nud_state & NUD_VALID) + n->nud_state = NUD_NOARP; + else + n->nud_state = NUD_NONE; + neigh_dbg(2, "neigh %p is stray\n", n); } + write_unlock(&n->lock); + neigh_cleanup_and_release(n); } } @@ -669,6 +683,10 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, if (want_ref) neigh_hold(n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); + + hlist_add_head_rcu(&n->dev_list, + neigh_get_dev_table(dev, tbl->family)); + write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -950,6 +968,7 @@ static void neigh_periodic_work(struct work_struct *work) !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3069,6 +3088,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, release = cb(n); if (release) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); } write_unlock(&n->lock);