From patchwork Thu Oct 17 07:04:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13839526 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A91BB157E9F for ; Thu, 17 Oct 2024 07:05:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148715; cv=fail; b=sDa/iKnPDqVjjt43gRDWl9lzUq/FB7oqbReXpyQyiIABYYJLd8yuajkAAyZxi18pcwqQ3jiCzgZYFii7aFmnpXTo9C1cPf36oCRPw8GlE/ufgB4x++bM0wvO62nbO5RTkcpSkH3wcKSvi9wOvYXRnuUYoNBmDXN35IfuSMlQdGs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148715; c=relaxed/simple; bh=T+HLM7kYBiNhj32OMoMaS2Ud/Ymy2vZaxhbfZbD09JI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=tE1KU8MXHR8XSuDy4aUZinE2E3Gn612n/qsx3G7iAZD/m2N4k91Hd4mIYqjTaml5vySR5JfjSzCVtSqrNJ7yX7utQvbx5bQd+us/PM1p/VW4cDgu3ezdTbXwqAEjTVXk3Hemww1vUDd0wYgtobyNtDR1sSKCBasf2Jg/Duz3hrc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=HLgHEIFa; arc=fail smtp.client-ip=185.132.181.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="HLgHEIFa" Received: from dispatch1-eu1.ppe-hosted.com (ip6-localhost [127.0.0.1]) by dispatch1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 93314242080 for ; Thu, 17 Oct 2024 07:05:06 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2177.outbound.protection.outlook.com [104.47.17.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 2E00DB0006F; Thu, 17 Oct 2024 07:04:58 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xOrqhFNSErBRwMiFuKlVIz6dohnk4MZK7k9tSFMsnO7/cO2jjcPGVHzYHtbL3SsakjbjrTMkq7step2iQQYtACSFq5pA3jmUvqC0Doa6WGIQ7wP/2ONZIK6VJMRp3XCyLPCLoAPnRTsa3TtVrOUb0mKyPk2awAJS3GxM5SikBMcMbnT+U/mlgJfi2dGkvZpz2vZp4zKinSwaioM8p69W7V5CgLTm6Q+Qdzs1uGxR4J5ugo5yvfkd24qFz/s74o4MB+Y5IhBrSaShIpOJTC/6KvvyAL5EDuJY8TNIZj/BpcQqRLKlDzp7ciH2YgBP4Cj2jz+E0shblCoQgV/epYDh1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XIfnBpyQ3DQM/afkECK6Vaqlj2HbGYLTwLExlShUpBg=; b=RXm5CNt70mRGulFt0Abm4LU43W5Phow9XMfhTRleCNlmWbovl1lrO20x1koibHAa1A2o2hQaJg66VMXU1OXpFHIdnVw5nOHgTx+vf/3xv0PX3YpMYXAGH6Cj5G8fLfoQXkKy86V8jlibScZt6aNn/X92MmnaEBJNonxa9a8MFVyS3c+TkgK5WEsVGAqHjYcJH0mQp8ewe4F59k18pZr2K19u/W8bkaauDkk4mEFdXFStQszMq2WP8DJHxf0VLkEA5LUZbDibclOlcFcsVaqV0gZZjX1SSfm5L1R2t0110V9ldrr/nZarIRAUH/tnf2uYQH5P6+usVNqP9oC7IWTsRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XIfnBpyQ3DQM/afkECK6Vaqlj2HbGYLTwLExlShUpBg=; b=HLgHEIFanc9syI7ykKufrpvBbmAF+ySyP5uR2pwRVbudse7S4iORf+7cljBHxeBQcWhuWc2AAyM8p23n3VNUR4mO35QNQ2FJqCDNjRwkY/n/iNEFd3q8Q3fq1v0JcGP6MB3ihes4CeUVQN4LDa4pldhYa7Fo23/Jxqoyu0uA9ZA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.21; Thu, 17 Oct 2024 07:04:57 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.019; Thu, 17 Oct 2024 07:04:57 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima , Ido Schimmel , Petr Machata Subject: [PATCH net-next v5 1/6] Add hlist_node to struct neighbour Date: Thu, 17 Oct 2024 07:04:36 +0000 Message-ID: <20241017070445.4013745-2-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241017070445.4013745-1-gnaaman@drivenets.com> References: <20241017070445.4013745-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0017.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::29) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AM0PR08MB5363:EE_ X-MS-Office365-Filtering-Correlation-Id: 7b0a28b4-9292-47a1-7269-08dcee7a0219 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: LlQ0Is9vVTEmtaDvf91GGcoQL5qgFk5fa+ZC+mWWrf2slivJKZ3KeHTf5Pv+2A21mVH5tHSVRBSdlo3KWRlDLYqhwLDhVRyUmtGE3SLyZkMBR2GxFq+aHjemd1lMMvviULc9nXsxTTHbCfSyaXcYkEf9cTs7jEbibG7sk0Iy2ezGl3QHhEBOiYDiuuB2gzwpeNNtWiTxISahUgCfHN1mqKZjm+/I861cVCUB3PeaJ8yu8WQW0r+QkI4xwoS7RTMrQW0qnbL04qS+s9KXFuZZag6MXdL+j9dtKeek/dMudPLyG7tKQ09eLATmr5fk5PXU6AD4YDzDz1VBhy6evzhaJl1xMnBTHpw6PJ5iC9N2OjdImTSeBq5pf50CskBRpb0bQk6S4MyMTWHF+VbnunoycK1e5olq6nCRnK932ujcQD8rDvMeO7fseHvC5tF3EUiaz1L8emd9eE403+IronPohgKg24/nHfnclEfgiT/WofZ5A71yeKcngePcLfXnFeiWvTOsrjBhVp2YTwtXy4p+iyfodEMAEqpGzsjcIhlfiGNX4AzzBlKa3dU7vvrtWak9w3b0YIteJIbsmUNNvBFCiSTgN1GDK1OHu497C2xvQYru0/1NCYJ5gQ7HPOp6YSeIgYXZNENGmT6GySOdV0FVq+fRlpE0j8iszaCjsjlaB931n2BQe5V7xgfh8UMozh8ShmwMDKyg79v42ntPHFXsRfkZGTV4TqTvFqVvGrsMpzb0jsO3J8lp0iChK+y+xEY06NFw77NTljDu96O0oCjn6LAIRLWo2Jsc2m7o3Gerz3iSDExxYL0zGqhv9hcPG63Qpg1mpOdFRhUQUfT5PsrK4u2RSlamDA9Gu+0eW91K5Df3wP23S4wRp1+VoGIyf7W17d6x9FIxQMBdt0jiQqDakwvFL9epTjFETELDZF0ASmw8oXzIQZ27d2SM5GAYKkASHwbcDF+y80KIQfC64c2oYf7DLwG3STEIZZJoIq5W0B433GUHv7qaYOcDAGmPiU53TuICLjesckWgvGCcHEQLuYKYEJGA3YLSaTKyXBy0nLRUCY9z1ChUwMu6en5pbvJJivdY2i4b196oaMHKcEdLE4cl5j+m3JfyfIpecjp3hyjtOHJs5ThI23cHX6g1kF8mJc20qUsEOzQseMEE2JYQAjht5QM2pBkEX49LdWB/1qQ695PSZ715lw7jBHXqcBbmIw7BEHkeDau4jl51DyfhWXEtEQWIsFo23DydzYt/Q7sLdfqNOPOyYh31jxALcjPR+34zczAXEByLvO2kKGQ+iTQ6nMlWWoNZyYhN9ZnG50zB5tSj5SGFLTGVpZwL3ZHAlbNKm/PavXvPpdzwXc35v4bZiGhGXEwSy3khUnIVRY0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(52116014)(366016)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ICBSjPChfNMNqAIkFBRy3HSAfCr1Oik0te7dm77TYC9XIH+4p0avyppa9h2iW0kdRApXBPfkX0teJLQPq+Am8IxKktawlJMWaW9bnL1PCPnc7tVHk4a1k3S/h/I2aJaGfx8wP0hz8lNxKEAuRixvKKNyGgDCfNpzJF7YB/QvtJGWsQqTgxI4yPop6XNbBuUUh6X6NJ4fCX94pSEEWou+9ceXS6lZIrslMqyWp8pTAb8rx22AzuqsdV6rwcyH2VQrLccm3IJJlvWGieFhSAKViP+PE6UpnOpENFgXBLVQG5i/CgoB37RcXd7QkO8e7pNlHM5Nl4OYz+Ycwko/zZJKN22RIJ/kdz9JK4WBR73Sc0g+HhRC4IV00NNIqUDfz/HslNfstSSgOXXmSm70j7nXloGIEqyJ0s+liLG2+f2ltvMlVkw/aZxn+kFzAjGVWk0RMN6pV0j3KvGoWFXSbGXWoS2Axksc/15/Xp4om3K3zJOz2rh49Ka4ciMI2mpfDKz6CRl/d48SLl1knTV64gOhJL0WeXUmArtp1CFw6sJIRmQZkGAsCppnNsS5VL+Cz49dnzSGldr++cnx1qCFdmiZ/gulvmabCkFe0LmeDiLBnqNr9TCNJiF1PrOP9xttNDoQGmWV37UkT9FCaBCWkfZkKyOnURJKXsHy1emO+vA1yKk3jds/WtLxPDdtWzQOAwTaRBBzRO6po0ATOV9aUcbNAvrhfIp+5SgLeFDk8rghRRAFQyfez5TdjJLy0EE2lbrk7UPxCMyhz922WMTYZH+yLQyfPTp/nRORE92WRMKCi6hwjCgS7RWsts9rBeVTEvgN4Omnd2rihM8uJ3W6cCw4VyHyDYpas2E5WYL1vNrY8jwh4jgX3MrN2Jx4bA4EfVSSvHAt7nY1hd3q5a2pFXVKP40LTWC00T6XQLc46VFvL7jij8ZVPNGs9QXEJzqjjD1qaeJAK49YnRUncbnBjemA9cw8XCFqtUVQbGmRzyy/B07BhGW67EKxFKzmhdkMym7doOcApQHzCGnC3CNa9gpaX9Rm46288E55XB76QWNjrS0IBGSTSG51FQGYvtgWe/+q/uLcDgBoZ320tWPxfTJquyte643wrn1HX8QE5BhrqNJeuH4JQH8CzEkvr1rOnH3e5y0Zx2FParFwqS6GQ27+YZRjMb5ODCpZ502uQo9OI93cykTz8eOVMQAQ+/5WmBpu1+sOJTBAA6WxbDIXUz+sGdIrFy70M45SJkURuC+Tg/HDBorf0F+6+glHNN046k0RYMXqHs2xiP3kj33yMpIrs73EXKrjoblLsj9LQamO/T6CyFE/UTBI41Cr5/k3J8s3Of9EJM2OCyh0YjMkHfllVSkfoZKY5UbAMDoITnKncV535kaYuTN4E4IL37h+Wi/DaMKgUPYzErD3e0Vh/unJFBjmlOyWDSGzq7IGLC2FmptcLULEU8vAknUzZx/xGbAAsEQwK7JL1WGvUfvtZm1AoBZn+l12QBnvRP9Us3tHI6cUu8Z+/vwtXBMjdUAd+FXLtQxtUH+ojLRMJUygcyzdB9Q5j/sc6uNJncJFmzQk06iMOLE5QO80m4ZtHEaaB2ZU X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: nP8IA9iDcilEsY++31IOPpnZ8R4N7onG9gw/FEFy190/b+AlLf6rojjlTo+VYK7MXvAv61QB4ApMK4CSTZZO4oaOIFp3ywdEYMjRE/go8eFViJZjK4Zc7th4HLO7ZviUR576P+qNwvZrMpH8zwwGic8CMUL1tAH7CGlb7Ayg9nwgEfS9ZfPjJCl3DGniQniFnLhYcrjgO+reH8dAmxhqMBPUqRnqItUcL8+nALuX3mFZj1H0qf/OzROif6bdOg1J0W2StVzUyaK4kpnAhU5qxg2x+Af8e6bN+aRWvuTzt28oXNPERsvQSi0fORLkVCsc7TEHy+UnW0d1sG48cd50GP0PfnbhwAlyVLxYciagTlZjFcyZVrcGPsnBfvzOiylWUDoN6tr4HQKyPXFz7j5gKvp1qQajXizE10qQqzJ6i7k65ZKPhqQA2L7tbq7mSOPW0ItEKdbz/n3u3kCl0VMVmfbCuuwz6lvmD/BYRg+M943fm2AZZL5yMBtXFQR5okpwrsCjLgzlxkB5efvcJNw6j++2zkdZFWSbPSUNQK6L+/mVzwUNHwPq7Dp5Six+Tw0jdwJ1nBSEEAoRs4OCBDRN4MZvYAuS1GDUao+1tGuQMzFJUNtUgonPs3rQkI4fc4yc X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7b0a28b4-9292-47a1-7269-08dcee7a0219 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 07:04:56.9719 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wcPBex0AQzWMd20jXLe3/rzb5ghQNpZsgOPbzUc8MsDJRMyvLpOQ2SyKgYorEfkSrkLc2ZPCfTeG+q9Zne4aJw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363 X-MDID: 1729148698-J8U93uoXb2em X-MDID-O: eu1;fra;1729148698;J8U93uoXb2em;;3e2ef0aab6a0ad8a3f1c1b41b7049f4c X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Add a doubly-linked node to neighbours, so that they can be deleted without iterating the entire bucket they're in. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 ++ net/core/neighbour.c | 40 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 3887ed9e5026..0402447854c7 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -136,6 +136,7 @@ struct neigh_statistics { struct neighbour { struct neighbour __rcu *next; + struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -191,6 +192,7 @@ struct pneigh_entry { struct neigh_hash_table { struct neighbour __rcu **hash_buckets; + struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; struct rcu_head rcu; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 395ae1626eef..45c8df801dfb 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -217,6 +217,7 @@ static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, neigh = rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock)); rcu_assign_pointer(*np, neigh); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); retval = true; } @@ -403,6 +404,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); neigh_mark_dead(n); @@ -530,27 +532,47 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { + size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); size_t size = (1 << shift) * sizeof(struct neighbour *); - struct neigh_hash_table *ret; struct neighbour __rcu **buckets; + struct hlist_head *hash_heads; + struct neigh_hash_table *ret; int i; + hash_heads = NULL; + ret = kmalloc(sizeof(*ret), GFP_ATOMIC); if (!ret) return NULL; if (size <= PAGE_SIZE) { buckets = kzalloc(size, GFP_ATOMIC); + + if (buckets) { + hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); + if (!hash_heads) + kfree(buckets); + } } else { buckets = (struct neighbour __rcu **) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); + + if (buckets) { + hash_heads = (struct hlist_head *) + __get_free_pages(GFP_ATOMIC | __GFP_ZERO, + get_order(hash_heads_size)); + kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); + if (!hash_heads) + free_pages((unsigned long)buckets, get_order(size)); + } } - if (!buckets) { + if (!buckets || !hash_heads) { kfree(ret); return NULL; } ret->hash_buckets = buckets; + ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) neigh_get_hash_rnd(&ret->hash_rnd[i]); @@ -564,6 +586,8 @@ static void neigh_hash_free_rcu(struct rcu_head *head) rcu); size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); struct neighbour __rcu **buckets = nht->hash_buckets; + size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); + struct hlist_head *hash_heads = nht->hash_heads; if (size <= PAGE_SIZE) { kfree(buckets); @@ -571,6 +595,13 @@ static void neigh_hash_free_rcu(struct rcu_head *head) kmemleak_free(buckets); free_pages((unsigned long)buckets, get_order(size)); } + + if (hash_heads_size < PAGE_SIZE) { + kfree(hash_heads); + } else { + kmemleak_free(hash_heads); + free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + } kfree(nht); } @@ -607,6 +638,8 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, new_nht->hash_buckets[hash], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(new_nht->hash_buckets[hash], n); + hlist_del_rcu(&n->hash); + hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } } @@ -717,6 +750,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, rcu_dereference_protected(nht->hash_buckets[hash_val], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(nht->hash_buckets[hash_val], n); + hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -1002,6 +1036,7 @@ static void neigh_periodic_work(struct work_struct *work) rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3131,6 +3166,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); } else np = &n->next; From patchwork Thu Oct 17 07:04:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13839521 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E7DE16BE0D for ; Thu, 17 Oct 2024 07:05:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148705; cv=fail; b=Xv6ffHMZEubgDqa1eHN/bwBzlLELKI4KvGZMfEqTJJVSPEncpegmu97j4zMKgZyPg0mecNCF5IAgkDQb7AnckdGkphLm9KjDRTo9N9dmmYznrbCH0xlZfdqVVfehOLhIFja2vXo8B1teWh6BKzro6lyrLYlxW39NgQWXKlIZFbQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148705; c=relaxed/simple; bh=sSYg5o9VZ2/hjLM+aTRJ5u1BXsfnpO8WjD4giqqScTI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=pieNxNQkwxSAnidb5w83WLU8MR/pwhetdFsQVWnKla+UStsf5B+7Onw9lCx5qqMooMhckoN+QkgWIB2TXjL0aGJ+e52In3VCcN9y7N6YQUBsSP+klzqWwxd8wngWdu0uncuwsCcvXxchkofBvMoWs7m5pKJeV21xK2bJIoPUL1o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=BaCs97nO; arc=fail smtp.client-ip=185.132.181.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="BaCs97nO" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 63DB86C006C; Thu, 17 Oct 2024 07:04:59 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=zQKlgm9mgpC8zgW3eV/vdtigWGDoFJaTPeh9nnVG3j3Lb3enjW7F3HrJplRZYJYrzfEdgSa/kC51SsozBq8hMA3EAWsKJtJPdY7ykEQ1nYbz1qH6v4Arw3u83v/UTe5dfQal0kFnzq/kgNShUSCGms6b3LPM20emk1YtyNJaCx9HsS0wdxEsRsULpYzf3pzQq1xv77sccRAU4h3Wdk/9VArx65gpQGk1ZK6Gg0yyuzudW7OhYtYCvieuOynpCpj1Ckx/1eJ87TCE69Z5YP09qaynpxyAOEqcBFKaICEIwxQl4xs9sRKa/G/2q0W8MnmBd3i0pDY2BluYK6ZkIuQJqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IEuhQli1Jx/cU4u2rkX0ZCwGoxvZXDbLreniCkxzGXs=; b=hGJ1CY9TQAr5j4POB8P6vgXKfaKAjrAYRegBXVD2eaOhN51ht/WiqlCzLmRrF6RWfpAxhJcpnd6PiE4fQleThIBEhnWbGqmkJH+vG4YcrKWAfyexTwTg6h7q/6kWOMNO8M9YeyEoqfd6bfoNN1mUsY7UT9ABr+O6ysMN/TN36+QPR5iWLMFjkQrhaG56gsTQnqBOM4WUrmGlnq/PWk0Xl3+hhiX8ZcHpN8xXbQ/BK2SDO/YvtAu3fxzvFVyJXARC1ys72yMEhE6mTqk1XJf+ULlzeUUeMgh1ytiB9FnREqH+0qbiHreUPH6bzPE3xp86LTYFpGKPgL8Nl1jKq6ZvJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IEuhQli1Jx/cU4u2rkX0ZCwGoxvZXDbLreniCkxzGXs=; b=BaCs97nO1S4arha8856MXwgmxa4LNlUWj8JNRlRU6ZrPJrZvy2Gfq9UBSBlOqVZWPY6CaO5GaPWBQnY/yyNpAJ3M5ws3hQ/QiVP3ptf3cXiqTQHsLfK5Nmy41epsjwmPlqE8uth+Oj6O2I/KHFxrXhvlt7WyY7Czkdqpw7lwRJ8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.21; Thu, 17 Oct 2024 07:04:58 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.019; Thu, 17 Oct 2024 07:04:58 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima , Ido Schimmel , Petr Machata Subject: [PATCH net-next v5 2/6] Define neigh_for_each Date: Thu, 17 Oct 2024 07:04:37 +0000 Message-ID: <20241017070445.4013745-3-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241017070445.4013745-1-gnaaman@drivenets.com> References: <20241017070445.4013745-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0017.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::29) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AM0PR08MB5363:EE_ X-MS-Office365-Filtering-Correlation-Id: 1601adc2-3555-49bf-4187-08dcee7a02ed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: ytCHiGbonooDacC7ORKuTMxw60sUi2m/hwaVDp7BSIzClUUWUJJh95DYrwNjI04+ObpwWd/PsUFqoJVEzbPFeC+iurb3gDTqjAEyhIfAyV0rJ/7lwr/4yHehzVs/6OXd4wyHoUxGJXOTc0cASD6OwEwC60huir8K693LRq6IF2BG1f6HKWx7PYzCfwBbFPOh7Ezb7HQtIOlp1qvWBuprhPbI0Tc6L4by99H4QFW2BivBep7CqaJKYyL8Jml9huyesj2vQ8nNNFY5ahEN1qMJRq/aL2k9xxPZYBgR18FNAxFQBRlK5hRlkkRVCrTqJYUuc2B1MrCa2lal+liWekZQXWvwjn1nAbPO8vANd3mbUaQjeKSrQ1h7pSewiOu+HpDpnvtgLuxWzs75SHeCi1scdQ3gyjvZ5A3+YL5xqTReKsDRW77sj/za9QURp+H0eoC9+6gxvqXfrl5zAPlkW52EJUmg0SKSeVovHOtYAuwxT5zr26F38vMo6XGbelgQw+PwAhnlk61Zji/46qEWjsxEcBbvw9ZJwtmR3NksfXz1eR19OVwpAvytLMCiAIBmb51Hx+HxK386P/u3N2n55NBd2VkJRcgvetPDLLH8DWfcx28JuAy4m8n398JvsrS8tKR5wtzUQq0CJosL7iLhw3m0Y41bGPeur/XhP5T54egcHypeIdId3Zqsqw9dta7EqP2N8y6SPAppXLLE9tlq1xgyLLULQ0HX2TmYGc0gDSASoTheYVzzHoT/B9HsRrmwH/zEyYo9woij55blPuN8phcFhuPxXoh8iw/k8+dRLlbBY4fC4cbQL8fT9z6KPJTnaL+oVm+nDvOTS3imuae6OIx+2yJVDfCRIZY9tHjC7hmMLgof0jn09n7FgwE3Lp/Pps0wVt4edJMd73j38PBISE+SMATa+OlfC40H33QE+5yH81QoI3zeiCJrkJ3eX6pmSI5VrFjfdfDC/+7CSJDDHMrs+trX05f9MlVZHF8B/PT2o6LEQF+wWLCiUHeijDPeJg4NSX6oxtAft6uZu1l1AyEIKfksYcxi8ExbJYyPxV1unClUe2DQ9eKIGVwpBr3XCwiEleP0xJhlc12+24lDlkQnHV4pAUyYzapDxdncIMsRJR0LD4f8FJSMEXDtXd9kHYmqM81mZuCrvdJR2ttj8Bh3dWFbVDez2XcokM+cCeG6RMVR0Ph/SYEOsuR9EYgWyzrXXpOPED1cw9kG/iS/Ad9In+5crsmDP5trax49cPCCgd468ZkV/ftBaermOVih20IuBpr/6+Yi06sgBkjt+fDBkQ8ase7StrffDiEaq3Ue0Rkx7UA9qYc5W60qc9bqx325zYvJ0U6k346MGzpVQ2wQvbF1xP5NuTheavlPySgtKdg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(52116014)(366016)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hNHDpm4BcfcG6degMCvUnLJzVKc4cUEE8Pg+OBhKosh3eqbYZhRNwVbagx9ro34oyFV+q/eHnHKKKEv/Sum+kyvLLJMeSeJpjOJKuvMr1bo8kMj3ZvR1bo86eM5t1er8BwgRzYwylTftxgMaRbtLJz7ieXEZFkOhK0N38mMvpgcJM3qh+jGWwv1CSMM3VTg5cc3SdSoV+6qSe18yBSbKH5ugLE526p6h8hxH+gzd3sWjYEiC9pN7bEjK4J+ERAvhQI+qxYH68BzmGRV7Lvi9pLjpAtENc6mtqri+JXTncPcCvzAAXb+b5uNweYW7SfB3eUGexrMTFjLFts4sYNQaKHFMohYnjKrBHXxE0IZS0BuXQ6SPC7+YD3uFrOzexWNtfVH9hvdJiKLLYsIPHSMNGIMQK1ZpjmydfpUzXsFFfgVI/7kFUHRoVISZIojwyzZ12EotBMZ3odhVTuBFZ+hzsy/+84aMi2R22vQnCeJ1tPSpKDvqKFSyvW16AcxOl0unzeUtg6ZPK9XXMCoGN4kp4mDMxV0HyHbTry+W60jhNJgaaLdYlVBPnUGUXWbbQ91JZLn2PJBPGpPHRihejFAFqZa2HPDNmqyugeTxwR1Hrer8KP8UH8VnO0ux4hBoCspqhlVEpQdLvBnZQL17Seg5Ys5mOD+fRY7PbIrngJFHK4vaD71jL0ouxNcfuzS4xKKd8VgejbICgDKQZcxBvGgNKFxDjRfoRAGONJiUKbGvQEMTw43PbG0AyevFfcGzRo8fvUtB30aZ6ExTm/jSameN2O999j1oaTOYuM/JKq7aPxDjU9kax7EcP6+qviThpc/ioZSs+38cRJ7nriboGFlhh94nX2GLr6sshR5c/qe+FsthR5GxMhTj6RLV84BjwaEXGtazHPz9jneOuANTep1pIp20Ls5zQpAAgbikdohmxHR+SCT0241xyAR1OqXtJTQeTrT/jeTYo0pktXkp+u4J0BwC4FAngGcVKGUwruO7Yz38UAoqC59Cmn9OZaKJsr3MhZNPjpsIr2rEDFNUdAJtGOJPtF502XKXU8BvsVDZWCjUCQ53hGV53ILYHOJpmItf7fmPkoY07A5hmmVvBRr1lnUC2HZK1+ZWLabnaMSSS2y9Z6HRDmrXHYiMkg+A7WaGxA4Ume0RJSrqNXKkCciULRpmFG1HGoULp+Tg5MiJWlxLOw6nDNTpNQNOJu+Tib4XDt7oNBgC0HR6DpC3vxTs9SRcCx5Cg0PLCj/Q0wTcUFxtr0dw+9bvMxKfms6OHCtKfnH91XIp652faXsglUqru+7zEJbkVhUFjYyec3n8ShkIeR+CXbkLXEYO4/0LYOHWXxVF59fqUuQ5HT4A0V4ycrsUXWfbQPcBOC0Y7OXypgBLJptQnhUZLGUFWZgot7R7mpmdxS6apsrQko80T+iAEgfrFsw8mrRdaEZR65uTf79quo+ozuIpdZptmlvQx8BQKUvt+whpEJxq3JU+s1cGifeETQJaube2tFzWIAP5hr7oR9G+Pm75z78kvP5yhXyUrNaHK6jytU1XINRwtGp+khvkxtnx8gg8GUGCsk1gmal33O4esWNa+3hdNT7IgTvo X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: WrLNpI4vS31MCtH8sXhs6C8y5kQjYRs+V8SR8Umu4i9ASTe42MVZOLcWyxMTlobf1EdUtaJcx286QbDHwJkIJCpyk2ooNPr1+/7miUYamMyXRdpBVWn6K691T2ZdHyBSTH7jijEnpDd1Zbn0qOnLe3hZ2AAcKEdjceiWE2Uly2dCmtmjsx1s90bBApgVBuPmMiUaBZ4tdUzkhP5Lkp1UNXFNQYmG9gpfhJfBw7f2GPS5fWb80+zenUAq3wH7FnaxrwuQDR04k/6koMUl49N3ZC0j1E9z599chjROJxeKXcUWUkBTYZo4xJaJqqkgA2LWLHYyZ1UqIOnV4wsDq9Y21GBWmriyBeZo7axGxDW/B+tEFyX3p4ll/AGv7+QpMxbSNFAopVDNAT5RU68zyk5VzNgM8e4awex52noDLvHpUa7waDEnQpP1ff797FkkiygNWmanMz2xTbh7L3DdwVeNXqMpyDRytGfQyWVFHU26oaPQ4vhoIhISvy8BDo6JdYJLF/YNcYSf7OT1HBVsD3icyUkfzEHa5bQMfU7NByOKlnO1tN8v0onyNSK9+kp75KohAPhqxCqyz2lHNjXnwryP1zSgkqkt3mUuRKHmjBgJesqojXdoOnpnIa7P3p3uCfFv X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1601adc2-3555-49bf-4187-08dcee7a02ed X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 07:04:58.3603 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BrrMQ6AYkcpJlgVqf7lq1j2Oohdmje++RjR++/H3lFU3DY2UAP5DvwlqzBtJSVRhRK4v8VRbvSUP7iqy018GNw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363 X-MDID: 1729148700-HDMhOCGWz_Gt X-MDID-O: eu1;fra;1729148700;HDMhOCGWz_Gt;;3e2ef0aab6a0ad8a3f1c1b41b7049f4c X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Define neigh_for_each in neighbour.h and move old definition to its only point of usage within the mlxsw driver. Signed-off-by: Gilad Naaman --- .../ethernet/mellanox/mlxsw/spectrum_router.c | 24 +++++++++++++++++-- include/net/neighbour.h | 4 ++-- net/core/neighbour.c | 22 ----------------- 3 files changed, 24 insertions(+), 26 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c index 800dfb64ec83..de62587c5a63 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c @@ -3006,6 +3006,26 @@ static void mlxsw_sp_neigh_rif_made_sync_each(struct neighbour *n, void *data) rms->err = -ENOMEM; } +static void mlxsw_sp_neigh_for_each(struct neigh_table *tbl, + void *cookie) +{ + struct neigh_hash_table *nht; + int chain; + + rcu_read_lock(); + nht = rcu_dereference(tbl->nht); + + read_lock_bh(&tbl->lock); /* avoid resizes */ + for (chain = 0; chain < (1 << nht->hash_shift); chain++) { + struct neighbour *n; + + neigh_for_each(n, &nht->hash_heads[chain]) + mlxsw_sp_neigh_rif_made_sync_each(n, cookie); + } + read_unlock_bh(&tbl->lock); + rcu_read_unlock(); +} + static int mlxsw_sp_neigh_rif_made_sync(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_rif *rif) { @@ -3014,12 +3034,12 @@ static int mlxsw_sp_neigh_rif_made_sync(struct mlxsw_sp *mlxsw_sp, .rif = rif, }; - neigh_for_each(&arp_tbl, mlxsw_sp_neigh_rif_made_sync_each, &rms); + mlxsw_sp_neigh_for_each(&arp_tbl, &rms); if (rms.err) goto err_arp; #if IS_ENABLED(CONFIG_IPV6) - neigh_for_each(&nd_tbl, mlxsw_sp_neigh_rif_made_sync_each, &rms); + mlxsw_sp_neigh_for_each(&nd_tbl, &rms); #endif if (rms.err) goto err_nd; diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 0402447854c7..37303656ab65 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -277,6 +277,8 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; +#define neigh_for_each(pos, head) hlist_for_each_entry(pos, head, hash) + static inline bool neigh_key_eq32(const struct neighbour *n, const void *pkey) { return *(const u32 *)n->primary_key == *(const u32 *)pkey; @@ -390,8 +392,6 @@ static inline struct net *pneigh_net(const struct pneigh_entry *pneigh) } void neigh_app_ns(struct neighbour *n); -void neigh_for_each(struct neigh_table *tbl, - void (*cb)(struct neighbour *, void *), void *cookie); void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)); int neigh_xmit(int fam, struct net_device *, const void *, struct sk_buff *); diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 45c8df801dfb..d9c458e6f627 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3120,28 +3120,6 @@ static int neigh_get(struct sk_buff *in_skb, struct nlmsghdr *nlh, return err; } -void neigh_for_each(struct neigh_table *tbl, void (*cb)(struct neighbour *, void *), void *cookie) -{ - int chain; - struct neigh_hash_table *nht; - - rcu_read_lock(); - nht = rcu_dereference(tbl->nht); - - read_lock_bh(&tbl->lock); /* avoid resizes */ - for (chain = 0; chain < (1 << nht->hash_shift); chain++) { - struct neighbour *n; - - for (n = rcu_dereference(nht->hash_buckets[chain]); - n != NULL; - n = rcu_dereference(n->next)) - cb(n, cookie); - } - read_unlock_bh(&tbl->lock); - rcu_read_unlock(); -} -EXPORT_SYMBOL(neigh_for_each); - /* The tbl->lock must be held as a writer and BH disabled. */ void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)) From patchwork Thu Oct 17 07:04:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13839523 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AD4C155A24 for ; Thu, 17 Oct 2024 07:05:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148708; cv=fail; b=b6x8a5UphSsN7A9Vh8rJdF/k0KNI5FGGj6putJ/XCSkFaGbmCQ0ygCsD28pxk4DOH2h1OL/fZLrb+d7GV+dn7ZOIpc1wZPpTWVTOuJo0ISsPAqcga35BwS2QYtGnQb/8nbJ2A8UtfYKqw7kJ2x57phS7xB0Lg7Jnfg9KVYMDIRk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148708; c=relaxed/simple; bh=xsE1AlvaE1VQi4zzjopRp5s1KCpdLJ5WUeO/ZVXTYe0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Khlo7jB0+gZunn671cku9DljM1JJYcMEQVE7f/GqZsgmbpFweU+6C1nj35nWu3DtXN/CwBJryANYb/xN1ndHu+CD7q7nyPPOuVNzJ/Ms8fKiYjHH3S9O+iK8BTxPMtEl1c4mGfuPd7mIbrBWgMiYo4Ko2oa37QopEKVK7lndc/4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=VVcLVp02; arc=fail smtp.client-ip=185.132.181.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="VVcLVp02" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2107.outbound.protection.outlook.com [104.47.18.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 05FA3B0005D; Thu, 17 Oct 2024 07:05:01 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BZXG5affyP9SbTnK0fInyuR3Sg3l69aCDlaxVj1srxi1nxkBALreFGEM2HKaxaF6jQymFZdfjVTfAEWBr1QwPRnD+JEjTtqjpB7d4qiYlXxdrJWXmRAQWm3SM+AFjwdgPH4Uvj/TWtYuVZBQ71Rjx8vIQd4xHZ5b8bvF0qb3HmB3jpuCCwHs8Pwt1UedYsWbqGnNsNFRC/QAR7ZjWUVoU1GRbAv1PgptidYEhxH+aEzuTswgbUncylps+LCxuRfxa2yIwQ+wUNUIHJrDQu2WDMW18+4NjEtRpDCC0iQ5gQaKcwk5oG7o//XG0WAAmSh/Une0o6FguosGLmFq/aRoCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MVy1rlRe9uEwK5FAC73MM3aNvNz9X6UkYTQlSbCABXY=; b=U+4yh6QsJlM/P7RuUwLhjvaOheLLTQBljmj/DJU/xTTTZOCzT1/OZV3Wn1UBIz6JcgpFJ6HIc7/U/OlP9FSeIsIOcrj5pkTMiI1WVbIMm81XZiOCO37r0AUP8aG4Db10cbJWDKgwZo0t3+Qnq+/wolW6emkQxpB8YxIDCQSJAKBp5V+D7x988xxDgvItPdfbKtPK4uQnvfnOgqiVb2KNSmqGAw7LbGWCjerwNTw4JT8vBmq9qiQZiHboMHYWjvycScq4L6bKB1iA3V15OFMXs/BKe64C2r5SmnhVP1VQV5t7ThBQoIZd4XXdSWgCbdfwKuy1vtU7kdzYAkBjQZPNrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MVy1rlRe9uEwK5FAC73MM3aNvNz9X6UkYTQlSbCABXY=; b=VVcLVp02FUkFqMJ73xjBUXNzM9SzPfITr1HuzooztkAUHMQelocyxykP8p8GGfsCtH/CMuoLnWmcMyuSpLg5e0sDjPzZRd5G5Gj36aDxq6zlUEqAF5zH/xxSv3CrGlNizHcRiIgAdy1WsxgbPnpu+cEUBGX12xs00cEWx+KsUr8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.21; Thu, 17 Oct 2024 07:05:00 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.019; Thu, 17 Oct 2024 07:05:00 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima , Ido Schimmel , Petr Machata Subject: [PATCH net-next v5 3/6] Convert neigh_* seq_file functions to use hlist Date: Thu, 17 Oct 2024 07:04:38 +0000 Message-ID: <20241017070445.4013745-4-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241017070445.4013745-1-gnaaman@drivenets.com> References: <20241017070445.4013745-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0017.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::29) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AM0PR08MB5363:EE_ X-MS-Office365-Filtering-Correlation-Id: 59642937-7d68-4d1c-018b-08dcee7a03eb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: GCLebh25prj6IMU2Fy1cPOgEDom6aNcGFSoZ2KygTR9aqAOb2RML/ts0k60nUTwWYIg0PdQN3nE2BQpFklaW/MJ0ddx+Id5N5PsffkG2QvRCawh/de0CW4oW40a7ZZoy74qKdpBweRZPbne/+xR05TqMBxtV0Pmrip0/3OEjxEIubC1xO1Ez18C5earohXjMRYZ+lRbcmX/ikUhmqcEFf0oTJbvRctAGaBGb1P/Fvo2iwNmrHurguY5gvzR0JcqH+9SZhQBovvZLTY24fDtDnJbj8xF+mbch3QgJ+G2GeMiYY736p/qyfxYJ4Ln6zxx9xTG3D5oXx0y1u4PR0bQb5Qgfdu5IFTkmF+c6RmHhWajd5PfdmKiDXsk/vIN2CI1wSHFHJIWUeNXc81Oz5rk9D+1B0nXTOASVbYCvkdAP+8t8QV4BtOmT0kqhFpNIQQNYA9295vf7svWWUclZP3id3QDgOsSnJV7moRK9sdQtU3aMki+08nFBbLn+JF0CYGkea1sh+ABqO4bIpbYoPnMUeA3CA91Rt7fW1DNeShtALv30nzyKM4LtJuiivhU1XDa1Wu30WJ+tHOndpV3eouWFjHiSs5ZVxrN2g8sIsz58/Ys6gD8KTMhZ8tPA0rpOEtbcDuCNBmRjteX4toJHL5eHBkWFnuBwq4Oar/aDmBd2uggSwnxVzklAJGzAE0DoLVbJScDuStgRT4YoF1J9CH7LARBTMK4PqCSJJ64arncw/YIoJbJM1q1XOHcHLr4pL8wqQgayTP4lY1PZTzStOVL0Vq+10P7NBXnlSSe6thByvz/hQJKXeTsZRkEij3hneb/CG9p9XV14YLBIG+WnT9+Ux/6YuKQEqvT3drvpg8gwahbzH8KnTreO/WVOkUzRWCwVpVbgZUrS5qI7m0Vyx195yQsoFxwb+Az5oLXupezfIvcPKbvGFW9FcGfYtQ5Mtrk+a6koKdVacvjLoqvv9JBFyY8J46yGJpSYzSt6KAsG3oH4+mTu06ZBKHa+7Z54tCXOdQjm5+osHAL7lyIqv1xk/MUrn26Jy4gzPTLi1WMh3QXo07yGJZogiP2MbA/qvrVLjHYj3XaCafs8fOxZvf17N1FuG5Mzm9z5onKIrOS0YtJk+qUAk3kWbpQCnf35+G/EqW3ntXKlq3F0Wpa8eiQf1m9vwm0OB6Jv2DFKPPaDnoleoz8BFDxTm75dRy/hO5N3tYFE04j1b/KR7vuwfOR9XGG+8+aY6Wvz6mSrflkzBtm9nClxQJ45KKUXkF5wNZ7cA+ZRc+F9L9Fev3QaGGpEi4TrNWXFk9QDfWbx7g/PyCqiWZBd43GyFltoXsDK5XlBIWAZ2cC0Yc8LiUtAZ7ApiTw635KE28sieNkcRVAbspE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(52116014)(366016)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: MUhtI9Xl45I+of7fG3OfMiPVfLHIv7O9EXjVmiGJEu14kb95MpHk2hUBRhi0H1g5IfHRRzD4AUZHrxzqfLIP0563LRiJEXufFiRtTK9JdWKU4jtpKc4MmtK8K40x9ZXhaRMhYCNeieyuSTAtaOeKiy9A3ihfrDI+4D903ubqMFGrnDVGp1BSbecLsdiTrqHRCRKNFkLEBL/KS83QzN7eLxHZagQWHYUhNEjBpcIz2WyNROC5cFxy3FIOrwTS/IYpgwbjjKynIbzU9zag3KpLNGT3egjnpGm7Xq1ocY5aZ1S6utMB4EX8qHdvwFeYbSm6V/xhDBpGMiVVD4LlrLrNHhgB3QxNjbxHOeEKICWGjJi3mEYzDXrMlPeM/ayXYGTxuBA+0sXbcwfw5qK507k2SjW+2btEpC8zIT12tvY8epCbVKQVppy+2NO34hAbDZKhVkgVaUbkuDGoxmRhgvDvthOyL6rYLAETApqL78cmYlTDBTl4X4GEj9Zi750W5TblWDaYxue7O7JXF9KD61lDb7E/pdUdBc/+C9Ol+7fElwoB7mPsaBoCq35XsuT/3GXVf1goi4tOOkPR6TGYZ8z6wmJUTVkMBzSOdjrdhpUI7UQkxnrtbRcbXqbjcioQXhN8FAjOAaySJhnUyzcsZJtJ2lNK1x3uiPalFKvrMVYdq7WsWvB7tvruriMyoEjds8B95poNoqc0jZP1mQcJoB3qMbA4t1ryJ3mEjD4EaxyQbE+ERRj0ruEHc/JFR2LTofHA5boXmEJaD7hZTBkH1C7ArLJEUebQjQsv4SF5lYtiIMOL8iz+bOOj/zqpmv63cgDaKaiCoG2PlDBVYxOyeltSAFjgrHRwDATiqRsXyJcMyMN3eDcbFnXyGHFzZz2KSgjzfDApefar/v2wPP/eYHwIacEg0yWiaIXF2AJWZch6sOFX1BXs3mOzS4S+8urNR1FfgzLOP6kaL3xnfLB+oGeMbdafoJvzaa0sJDB0T36iXUe2qqj06Qr+9FGVfFJ3yrAonXM1wgoP8ZsUCLIgjBLgzDxqyCW1n9di4bxnppwucZ8S/FTmAN/crSR7vYIqOCsFnil0VL+JLrItwf28aPfx2tGBiZ+MGbxAwKN6MOfkh+Yl8ZGDmTPiem/1owUsQJwEEPel4tY/HOSJjXdb8h6+aQ2Gj0L1X7Kh/UCoPBmuNifcDyndYqOAcPMO8LJikHU3em1jv8t0GeUZb/t+MkHCr8LLhByIxPp17Ymatt1vaHnoy7XyA/6iYVfAeosEFGj+WhXXCQ4O00o3lu8QsvqY0yOeI+YmRZrk+BJ29s1A8wsS67DAEoLS4vn/ACJt9+HN3K3ZSl9jAIkQuc4x5X4ZwvcIMIpeoWJdKuIa4qx/OQCOj6XxapLnf3ajsExODEyb1P6yGJvfxZ8QC493THm2FmOoK/SeZdDU0HjniX1qLetIvBiYRIDYDeglkAbAqKbS2sS5TEEgAFnol5kUvJMNaa+xPKtiK8hAKVFd+UsvjlMhRG4acrlQMKGyeQFjTuszDt6KxqcJLlWJWItzS94ewEIDuT7IPGT8/mH+yj7QWzoXVoc1NmQvITaXCMCTYpwR X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: LSz2qLNxqQsgXQ/Dklr1ZLsAT56Ye85v4IXDxMjCk5eClnZIqucU3468WAyuWWyn+4lTDLaLroMCN/4i1oRWf7vvfhrjT2SuLYjtHUq/vMpAlUiliP9yqEy1fouZuAfhCXUQxm5/5IyGzR3IYCiWtXcHoYRXXMg4xd4SJkT2gr1HcaiIjcJHt0+tfJYuQwpBJioiLAh34emXDRs++P39EVa7aaq+2r6ShBTJcBWqOpbhTq1cl4s9DIE6MvaWDhZ2YXfZ4ZCaoR1eQ6nV2ix2iLS4Ba07cZ/a3kGr1J52eeEmgqoCgoE8/gWw4A7vxodEEci2/9jY2QUzBTLwybrholVEiR3Jtv/Dy+zCOp2onHFoTTL4IY2W/s+F0XmVcrwxY2uNxrtrX8vKsWx6ABzHkyz1WIMXQPuhWpPS3fzYWAQ/gzBj5CXisXqj0iKscWW502qGBWgDrIDHkOpNGZvKOYnzBY8mJyl9L0n1wriy6Nb8XiHTKdX3mMu0TGP/vMeKLvwA1bJRigvTdkKqmnt015U0hqzbdIE6GD8o2EqI8jHgs+tVoaPa3wmLNoYW3JtCi1yHZSqnb3hRJSOca20tMlLRdKTdA++urZwsfeDjnNpDde9l4ZovEmD3LYuXEQ2s X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 59642937-7d68-4d1c-018b-08dcee7a03eb X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 07:05:00.0268 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FQyds9mQDzirOy8xH4jQCENGVjm/pBVXPVxyVAHAeDVyIBbrm0DoJEYOHVKumbXoCWQgwH3Vjy3lFo2HA+YZDQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363 X-MDID: 1729148701-jfwD1gcRoP6T X-MDID-O: eu1;fra;1729148701;jfwD1gcRoP6T;;3e2ef0aab6a0ad8a3f1c1b41b7049f4c X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Convert seq_file-related neighbour functionality to use neighbour::hash and the related for_each macro. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 + net/core/neighbour.c | 104 +++++++++++++++++++--------------------- 2 files changed, 50 insertions(+), 56 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 37303656ab65..14f08a2e4f74 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -278,6 +278,8 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; #define neigh_for_each(pos, head) hlist_for_each_entry(pos, head, hash) +#define neigh_first_entry(bucket) \ + hlist_entry_safe((bucket)->first, struct neighbour, hash) static inline bool neigh_key_eq32(const struct neighbour *n, const void *pkey) { diff --git a/net/core/neighbour.c b/net/core/neighbour.c index d9c458e6f627..e4e31d2ca2ea 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3204,43 +3204,53 @@ EXPORT_SYMBOL(neigh_xmit); #ifdef CONFIG_PROC_FS -static struct neighbour *neigh_get_first(struct seq_file *seq) +static struct neighbour *neigh_get_valid(struct seq_file *seq, + struct neighbour *n, + loff_t *pos) { struct neigh_seq_state *state = seq->private; struct net *net = seq_file_net(seq); + + if (!net_eq(dev_net(n->dev), net)) + return NULL; + + if (state->neigh_sub_iter) { + loff_t fakep = 0; + void *v; + + v = state->neigh_sub_iter(state, n, pos ? pos : &fakep); + if (!v) + return NULL; + if (pos) + return v; + } + + if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) + return n; + + if (READ_ONCE(n->nud_state) & ~NUD_NOARP) + return n; + + return NULL; +} + +static struct neighbour *neigh_get_first(struct seq_file *seq) +{ + struct neigh_seq_state *state = seq->private; struct neigh_hash_table *nht = state->nht; - struct neighbour *n = NULL; - int bucket; + struct neighbour *n, *tmp; state->flags &= ~NEIGH_SEQ_IS_PNEIGH; - for (bucket = 0; bucket < (1 << nht->hash_shift); bucket++) { - n = rcu_dereference(nht->hash_buckets[bucket]); - - while (n) { - if (!net_eq(dev_net(n->dev), net)) - goto next; - if (state->neigh_sub_iter) { - loff_t fakep = 0; - void *v; - v = state->neigh_sub_iter(state, n, &fakep); - if (!v) - goto next; - } - if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) - break; - if (READ_ONCE(n->nud_state) & ~NUD_NOARP) - break; -next: - n = rcu_dereference(n->next); + while (++state->bucket < (1 << nht->hash_shift)) { + neigh_for_each(n, &nht->hash_heads[state->bucket]) { + tmp = neigh_get_valid(seq, n, NULL); + if (tmp) + return tmp; } - - if (n) - break; } - state->bucket = bucket; - return n; + return NULL; } static struct neighbour *neigh_get_next(struct seq_file *seq, @@ -3248,46 +3258,28 @@ static struct neighbour *neigh_get_next(struct seq_file *seq, loff_t *pos) { struct neigh_seq_state *state = seq->private; - struct net *net = seq_file_net(seq); - struct neigh_hash_table *nht = state->nht; + struct neighbour *tmp; if (state->neigh_sub_iter) { void *v = state->neigh_sub_iter(state, n, pos); + if (v) return n; } - n = rcu_dereference(n->next); - - while (1) { - while (n) { - if (!net_eq(dev_net(n->dev), net)) - goto next; - if (state->neigh_sub_iter) { - void *v = state->neigh_sub_iter(state, n, pos); - if (v) - return n; - goto next; - } - if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) - break; - if (READ_ONCE(n->nud_state) & ~NUD_NOARP) - break; -next: - n = rcu_dereference(n->next); + hlist_for_each_entry_continue(n, hash) { + tmp = neigh_get_valid(seq, n, pos); + if (tmp) { + n = tmp; + goto out; } - - if (n) - break; - - if (++state->bucket >= (1 << nht->hash_shift)) - break; - - n = rcu_dereference(nht->hash_buckets[state->bucket]); } + n = neigh_get_first(seq); +out: if (n && pos) --(*pos); + return n; } @@ -3390,7 +3382,7 @@ void *neigh_seq_start(struct seq_file *seq, loff_t *pos, struct neigh_table *tbl struct neigh_seq_state *state = seq->private; state->tbl = tbl; - state->bucket = 0; + state->bucket = -1; state->flags = (neigh_seq_flags & ~NEIGH_SEQ_IS_PNEIGH); rcu_read_lock(); From patchwork Thu Oct 17 07:04:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13839522 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 203F51C1AD6 for ; Thu, 17 Oct 2024 07:05:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148707; cv=fail; b=LzmAVZrNOi7d8hCU0nkOhQlvpttIJUysWcJyu6cvUbEj2NMGVRQSujWvgpNyVipbBG8yl8nIJxdWs7T50XtzR00RK1cY8mvYzynfAqU7WRQ7xt8dBHikCYIBdUPrQg66yPxLHivUoyUjFfuSJ1aQqJ9Kv61woBnWY2kSMRku7oI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148707; c=relaxed/simple; bh=ob9evqrrXV1K2i5/wJcxzuC0+0fmPRPwHmKT5d92GQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Z9mEMQRrh/W92E6MJNpXR2isYGIdHhhi799fgI7GxYBaR+2XoPSDGMfNIknoNGj+IqlFioAeWyHpLYou15UN55LXE2kJ30uJIkk9n8f5snXHI4P1gLodUvBb4poNOV05fDAZpchJmz8NqSvxQD/kG9wNV4/JfJ9blYP3PM8toi0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=StUBxYCQ; arc=fail smtp.client-ip=185.183.29.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="StUBxYCQ" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2111.outbound.protection.outlook.com [104.47.17.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 7366320005A; Thu, 17 Oct 2024 07:05:02 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q+Jj4A3zzfURBe4WxvvqzEGWvkcYGdQu3Xw1A9aI3VDN0y3zN/vCaiFinVFLaWImj5la+4F1VTeoVQ+IC8UYOg7ktOIpT9+ffwKpWSDSt7T11wDABvy5bH3NhntNH5qM8dQxIzisDWqAEsJsMEo7pO5+00cMTq1HaoIf57Gk54taV9x+5PZwLtgoVaJ+lE+uaxQ+bdWztiXfdT7dwWNyK8mfRy6nzzcwOpfMmVjWZP8SNli/Peqob3MlUXOP4S3N/WIAIuuc20gjA89vYE/djJfSj+mPn1Y6ff73JxMiHXK5lVS9bSQrH16EKADTF6HfZtDfyZ0VYo+yPZM0v0//8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ae0dcIy+Pn3EjEsk3zsYExCoJ4S+szA+rXitOlvVQm8=; b=AXKxlAegejLXbHeBCTflX4YQFyBNNCW0paSuUm1jMl+jP3fhGx1v2wrDY3aCjP4rhW/rFA5huxMoIC72b3UjutwJuqMJ/8UVTcjw+nSa4S211a40JLx9zhpn6g7+YFRTwbWHnFRU6bDmzXtirw4fvDB4SbX37/tXOh6rBpN58bYc8QPbo2y+uc3l0asJsEHAq61xL83ZIXtee1dX8qdWUhEK8QYoJDGT+cM3W9w1dcBJYEbK42++dPV0tWHMGjp5LYiOLHLvgQyL5zcho3/A1qW+Eu4CcnPtUH7CGmfMRskvrz8J5objkNW1vpTlDN26MsN159K1sPJTLq9T0v3zyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ae0dcIy+Pn3EjEsk3zsYExCoJ4S+szA+rXitOlvVQm8=; b=StUBxYCQLdRtyokNuaNRxaMiIve0haKXBdtsbAiQIuZ+YzLul3adnhQRib5eSpm/1hMtksfioe6sDU/r+lkzO/Fl9vpwjmsJSuI9osmQ/HAvE4KLVpMFZg9vbYr7fGl0a20pOvx6CS+owwWx3Pp7HlI5WHSXHD7p9N53szLIkRI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.21; Thu, 17 Oct 2024 07:05:01 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.019; Thu, 17 Oct 2024 07:05:01 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima , Ido Schimmel , Petr Machata Subject: [PATCH net-next v5 4/6] Convert neighbour iteration to use hlist+macro Date: Thu, 17 Oct 2024 07:04:39 +0000 Message-ID: <20241017070445.4013745-5-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241017070445.4013745-1-gnaaman@drivenets.com> References: <20241017070445.4013745-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0017.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::29) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AM0PR08MB5363:EE_ X-MS-Office365-Filtering-Correlation-Id: f527203a-6770-4384-ff6f-08dcee7a04c4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: iBxTP5jl0wYX6IiiOW1fi3jcJHmjV7hFCd2pMgiUfXGuwBrj5xkRB2t3IwF/PimKCqrkfdacLWmQ6NSvn1asiVHWRu/6XaKQGWohOtfhhIXXQwmT57anCg2BaNpMnNnvqlCW6WDVQVAuvM8TCirgF6wdVQob5Af8VEXx527WFjy2wI9LinBY0oP/ZwHH0fHUHEjw+22UxPdBgiWfNW+iJwChY7HTKoa96t+iWL+YXj3nZ6J5AvHRc/Q75mxvIT/KP/mZVcW100PmQ/gcFwOpDveGHF2fUjFIwAW7k84yb5WnJD7VLn1sW0x8QTD1fvuuagkda9N1XRgUoHvEdkJAR3zt/1tBVyxrSK4kGsPr2lvZbDV8qrgbXkjiYzvn7//jIx4iEvRhm9kwmSacPrKCu80m+a4g9HAbKG91rIV7uRzQzA6+niSbfnjFwgBSXRrncmgFyf7sMqZnXDQVzrgbD2KhcR5HDiv/ydV/aXTm31TS0uZulIhNEOgNpY2fxhCkU/9gYHwVihcyM7vdzLCnlE/vj7MYZMAKHJPwm5by7xybDqs95Wwfs7Y3UsiYI2ogmTr6bEWccisB7jslkEYMzN8/hKf1ETbKAwwkagm4dLqBLv/Pb5oGZIkGJhHwYMJFp3mIvILTxfB3E1S1jqVaquSMCe4mxjDtckzG/gtrgrGeQpPZZO+sOrgDxIWJ1IZV9BquavystY6AvZmsGHgp6uEs4fDNn4rGvXYrQ9bz6OkEg5kcfuhvKgTY6+Gmp2tYeTEbqjrmzN+SAGsWpZZ8gOBwIR+9eEG5cPQTH/64ZWqokOUmocc12iasYT+LDskmSUZs3965bmXBIEW3bQPlESIelMldHn8vyoLNOLKPpzoJauY1lorBeJbMeF3V12zgzWT3kVYLk9nFlk+MSiX8FqV1DWLzmhxqjETRSc2l09mJ/JyT29Atf9utLNft1D8sJE686xUzuGpcFFpdVNalaQTaLxnrpfi/EtiS5DgiA0sRu6OAmbO8ShEJf5UAe6wFAxmr8AtDdTBeqUZiCjs5Mic+6UQzE/MhvNcZbPepKf9RhhYaPDJkFROooshZGs1jrFQRtAQ9n/jMCAPu+Z3hHwUKdTSdD/zCeYslq3ZE+C6/zQH9lW2oBHDzHCcNufZ2vLjmmyECXv2RGHgNxIY69IStnmAAq7Kc5KJPvH+ogzeMTl90QPrvZlrVmJ1aA0Vmf+brHzr++Emrs85nRkog6VltxKHe6QQ/mZPqlRwTR9v8yXZiWKkddQFu9B0WzaBOjnoongkgyeolYLqoXed31H/8UlUs9kDimLjIxH7nrBHFdPfMdYtuJvDY+Itryu+Xu5/bIapohAbx76IcOGWv//tLytgCNZiyH6+wb1jUcVg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(52116014)(366016)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6QW+TWCj2rZEl+GAT7e5XVB3oL4HUT6XiayLkh/M7GNIMZ15c5CkkljolzcU+IfDTbRz0GAGoi/RsDSXBIQtbWtyFoNK7a986uk+pPiZm2q4UzrtTlB23cr5q3/v/8Xn4LwnF7lpb0Xey+4cO6zXyFhWdxoxSXhPUDIdrFi2SOU/DFzRGPnHt/qx+x6PApTiYpqB/6/tzeQ8N5zmoGP4msmt90ZvX5DJbLfXNxGUJzL5eVphe+45Xj84x7XmVnH9z/+1Tmu1ahMECV5bRP3R3Pg2vTsDdReO8IZa3ctU0mnuQ0xSlW1/9qoWXsTCZaPyzZGgltzrO99n1KfZEhKBT79MJKLeRIKx7opb7Cve1kjVgt/p7/ArVvhH1VjhwmqBpLcK8IevQksLwG/cqfCKpIhWUEq3bHIzf7iao3I6d4cwQucQ6T2IyCLCMn6aOaFucYbEC9bz86EeAAf2T3gjxAfVjGoZ1TaTkq8/NbBs+5c2Lpe29niTGsbErgszDIeGmCmoHSIIsVeGVLGTOeRvia26Q1Mjg+US2Nf86dg0NKq72eY27zoJ34gT0yb9t5My/eyWubFcJbDR8+IIEwTPA+d8thV4koeUl/5fj6z26qr6U+F/feqD1PQUGd8IINnNBY1epoKMvQ44lz/uDeq+T4MxIZHCnXw157eu9Ykh8APH7SvNBFn9dxfkl13fDKgg+/bt1RC2i/QVs6iFsyX31YF+ixi3/FvIEMBszu6Yp3fgdMLsa7JGrG6Jz3fnBSFKkjLzaob67L8Isa36Nlc4mNe8SlN5a+fhUXrgy4TRZLsRdUDIDwRtFCZARozcgh3Sp82o3ejk+BGe7j3WLSfBGfmiXzfzn3nCPOtnYfisJi6OfXNH8ngJPSA9Cl/aCa05/GdbYEIjC0qZ9cUssgBX8/9dQLnR9umrWJFhIQeJCstEGnEavMroK03rfUhKtFniEP1+OjU839u+4wbS3miJMOcRvBDNU4lQVtHVxHrBfO84a+l3VL366kwcWshC7kBbF7f+qi6SKrOFNvkEC2UZbHC7fWPDMSft2rOAhmkk0ZUmNUAprK/zHDHYresbNUNoqjEEBcn8Js935yBB8YCF7nwsftKxor/TM6p5gEcHDz50w7UaEt7K39aKw/qPQ7oLmJ/J5zSYKzAQJuyFCYor6AiBSNPTa5DPkBS8CN34gD0qsEVTRCjgMDRNTYqN5jS+qE0lvVhr9ZdmzlbL4sfmEJMxW7S70vJxt43zDajIPl6BScrJv5J5xwDml2MIMUoEO7kHyo04F4bTIhKPBiXXED66YjO5bzxNxad/6HlcZzYfAe15/32Z2tl45CiYT+SlIcVJcMNYo6CFzaEGY5fk8sSb6KA6BOOLt70P2n+eo2FcFYXguTTka8WH99GqkCLQ5mcGoPSSMWxdXCpVBjms1ArMLTFC5nKrZ/d0zEdbeBFtxQuwyJ1moQ7BANDslfONaKU5lwBY0KDNOYbbe6Gb0xDMWSQHXV6LMiujUh8dpAHTxl9+9EUNYOyvmzdkNmAD2axx/oNSl03cyZpOsAWzvAiiev1pCnwXUnGHCsj2ZpkHxdidit/JFdHFPBeAmxdz X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: p/WGA4V6ZriXKTLjDfb3C6dPkPtsAtTGKhta0qXU+L6x/qxZBC1VGIxI6UmKIAy4pvVVAu/T7YQXHkR/LIAGSKOvgvxCDV7es32kiuJOoAjU3tiXvX4LmEJKlD3MyzHlRwp9ouND7GO8rxv+uGwq7LaCGqn0Ar/s9kme3aG5XB/i6YxerAud63oOGeUr+9axE2R5hz2Z9FjOfqzhjRMNnEzwqy2+Ze3+J5PqAiciS5te1OpI747ZtqNhA7MgN/XktkMYfPgnRK24MXPEnBmnGIY+4AGcoijPSsl7FxBuvGCD4sfY5ln/L9HoWw3AwGmF3bsIjEwGa88AhXOpPQvJ9nWxEowq63mf0gWBLm9gul0b7hKF5hnI400wB8zPDG921ZHDBD5zp8NIqEa44Au8TP1KjHLk+pfCk52c7ZG7Dd7dQS5am6PUDVAd55fCWjS+6OefVUVqzxuMuMGX/sUjE8TCQX4ROZxxoELE5WJRsfbpSHf96NrC2dHolSQWu/VJRhnDM8IqECafZhFSpwD5mhctIW3/Ln5pgk/y8ap8S40nBzzp1Te6Z1EeVnGU7ApH3k2zU8tMyTLKHKy6+CAO6BTt4wXaIhEoCR9QKivvByLnWoTzG5pd+ZW1IRc66KUn X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: f527203a-6770-4384-ff6f-08dcee7a04c4 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 07:05:01.4496 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VgBsvnmJKHej9wBvd+9kUX8qX15T787Fcg7d+YUSBGmO1C97LW5bL2VSL+hv4y/xaRIMo25NpwUcfqytdKRNAA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363 X-MDID: 1729148703-w1N3UEng1keP X-MDID-O: eu1;ams;1729148703;w1N3UEng1keP;;3e2ef0aab6a0ad8a3f1c1b41b7049f4c X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove all usage of the bare neighbour::next pointer, replacing them with neighbour::hash and its for_each macro. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 7 +++---- net/core/neighbour.c | 34 +++++++++++++--------------------- 2 files changed, 16 insertions(+), 25 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 14f08a2e4f74..96528a6cd74b 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -278,6 +278,8 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; #define neigh_for_each(pos, head) hlist_for_each_entry(pos, head, hash) +#define neigh_for_each_safe(pos, tmp, head) \ + hlist_for_each_entry_safe(pos, tmp, head, hash) #define neigh_first_entry(bucket) \ hlist_entry_safe((bucket)->first, struct neighbour, hash) @@ -309,12 +311,9 @@ static inline struct neighbour *___neigh_lookup_noref( u32 hash_val; hash_val = hash(pkey, dev, nht->hash_rnd) >> (32 - nht->hash_shift); - for (n = rcu_dereference(nht->hash_buckets[hash_val]); - n != NULL; - n = rcu_dereference(n->next)) { + neigh_for_each(n, &nht->hash_heads[hash_val]) if (n->dev == dev && key_eq(n, pkey)) return n; - } return NULL; } diff --git a/net/core/neighbour.c b/net/core/neighbour.c index e4e31d2ca2ea..a1dd419655a1 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -391,8 +391,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, struct neighbour *n; struct neighbour __rcu **np = &nht->hash_buckets[i]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each(n, &nht->hash_heads[i]) { if (dev && n->dev != dev) { np = &n->next; continue; @@ -621,11 +620,9 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, for (i = 0; i < (1 << old_nht->hash_shift); i++) { struct neighbour *n, *next; + struct hlist_node *tmp; - for (n = rcu_dereference_protected(old_nht->hash_buckets[i], - lockdep_is_held(&tbl->lock)); - n != NULL; - n = next) { + neigh_for_each_safe(n, tmp, &old_nht->hash_heads[i]) { hash = tbl->hash(n->primary_key, n->dev, new_nht->hash_rnd); @@ -726,11 +723,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, goto out_tbl_unlock; } - for (n1 = rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock)); - n1 != NULL; - n1 = rcu_dereference_protected(n1->next, - lockdep_is_held(&tbl->lock))) { + neigh_for_each(n1, &nht->hash_heads[hash_val]) { if (dev == n1->dev && !memcmp(n1->primary_key, n->primary_key, key_len)) { if (want_ref) neigh_hold(n1); @@ -982,10 +975,11 @@ static void neigh_connect(struct neighbour *neigh) static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); - struct neighbour *n; + struct neigh_hash_table *nht; struct neighbour __rcu **np; + struct hlist_node *tmp; + struct neighbour *n; unsigned int i; - struct neigh_hash_table *nht; NEIGH_CACHE_STAT_INC(tbl, periodic_gc_runs); @@ -1012,8 +1006,7 @@ static void neigh_periodic_work(struct work_struct *work) for (i = 0 ; i < (1 << nht->hash_shift); i++) { np = &nht->hash_buckets[i]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; write_lock(&n->lock); @@ -2763,9 +2756,8 @@ static int neigh_dump_table(struct neigh_table *tbl, struct sk_buff *skb, for (h = s_h; h < (1 << nht->hash_shift); h++) { if (h > s_h) s_idx = 0; - for (n = rcu_dereference(nht->hash_buckets[h]), idx = 0; - n != NULL; - n = rcu_dereference(n->next)) { + idx = 0; + neigh_for_each(n, &nht->hash_heads[h]) { if (idx < s_idx || !net_eq(dev_net(n->dev), net)) goto next; if (neigh_ifindex_filtered(n->dev, filter->dev_idx) || @@ -3124,18 +3116,18 @@ static int neigh_get(struct sk_buff *in_skb, struct nlmsghdr *nlh, void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)) { - int chain; struct neigh_hash_table *nht; + int chain; nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; + struct hlist_node *tmp; struct neighbour __rcu **np; np = &nht->hash_buckets[chain]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); From patchwork Thu Oct 17 07:04:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13839525 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 487171C242D for ; Thu, 17 Oct 2024 07:05:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148712; cv=fail; b=JtGEwF4n53fxCMskTv0yx3eU5dhqaadj5sw7p7OLoWzOXbEb1VnFmtKVm4a1ZJqp2aTJYrnaLTJlpF2GScTQZffTxAKp7J97h7XXn3NZew63KtyxGMBelbuUxVr8MIaB9tP846Onu9jh4Z0++kvf3/WviHFC9pt1ShwZwEjg1Ao= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148712; c=relaxed/simple; bh=4YvQTxKZBaVodELexDPT9uT8Gx/1T1Cjkks7MA3KavU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=GfGUgwgVBLCZaZFbR1yQi266x4Un5MBwtovIIIWl0TNRVT0QeAYp+xg+RhfNwioj+3o0SDNNUbuMj2dmhIRFZb5mnvRrmtA+R2l8jYobkrXO9U3/BdL9R5oBEknCYjzOvTN34LA77l1ogokmNDb5iftb1PtQTlj+TOhKmqd0KNk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=at25psLP; arc=fail smtp.client-ip=185.132.181.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="at25psLP" Received: from eu1-mdac22-2.fra.proofpoint.com (unknown [10.70.45.127]) by dispatch1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id ACC58600BA; Thu, 17 Oct 2024 07:05:05 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id C159E6C006F; Thu, 17 Oct 2024 07:05:04 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KXOswnqY9E81XoZnqZxW6krwYrVlKCC7PW2rf0Muef2wnHQL74I4+okX98jKJB0z18+Z7fNLw6YdEFxn0UfG8pC3Cmx42KQohp0c9UmZHgSDgXLqOdA0XGwBflcuYdyZ0UQmieHzuWNRLemKGcP0hZvItLWg6lT3bOw+2LnhGbKE1TyCX6P7ja4aFs8B0ODG4X4u8EmnRKvWtzKnNdPjjaIcxU/D5E8TCo10c3bzdk9TnXiTkTtgZCbsNXhSfEV4W99qlz5Aj9A+CW/nu2brP8GL4jIr7IGi2tfHDuF26o5TAwrbss5nHb4F30clHh7bdp5ccHD5/t+7XdAeuko1Kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0Ct/U2sGKPu9Fp3ZIOrKbIUtB5s4HKMUL0/BjL6y6pA=; b=bcAXpJatb560MocOF5pGwAqZ0pAZDIY78+RbmxcyTs/+iqPTWnX2oOf8+3h+YriwI+/aBhURe8jxRtZi5ZjG+ZVqSmWPkfo4A2l9LFOojDygYfBVOm1s6hYCBsC/T1tGi6HWN1RMiE4bBL6he92QWyNndprA9wg2vmoOJ/XdYavxJix0/V7N/bu6ZRe8ihvE6wIXQEW7BzJBgu/Fy/lP4J695rxPVPsjZHkDUbodn1a6+20jsGNwIENDrMf4l3cCdoYmbgTkTihzT9lwTx6QLebqKSn0/JYBsPSQjoWnONQfTaJ7SxW1l8lKjpOBS1/FRgV4eEBrD1jepRrzwVZTjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0Ct/U2sGKPu9Fp3ZIOrKbIUtB5s4HKMUL0/BjL6y6pA=; b=at25psLPI0qOxlZdrbFo9eTEvDzJAW6qBn4UHZTm8YtQgR2uJ0hfY49GB9RbCe82+kVh3uVSqEFbKn8oBPXf5KCpLhRKA36djlmS17PQPlBwE/3wI9esyvpl7eR6EwYXEhScRuTlal8/nV7ZPOmUH2KM3/I7Q+H2Tx37sX658Y0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.21; Thu, 17 Oct 2024 07:05:03 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.019; Thu, 17 Oct 2024 07:05:03 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima , Ido Schimmel , Petr Machata Subject: [PATCH net-next v5 5/6] Remove bare neighbour::next pointer Date: Thu, 17 Oct 2024 07:04:40 +0000 Message-ID: <20241017070445.4013745-6-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241017070445.4013745-1-gnaaman@drivenets.com> References: <20241017070445.4013745-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0017.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::29) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AM0PR08MB5363:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d8466da-2596-460a-a60a-08dcee7a0586 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: pRK0II/iIoLQammkdrQwhvRnROIoVedb5UKvNtjy2auoHdLBCRKG1pz2up075qzb3e/Snt2Xvh4+WqR2NWDNcbt61zxGXD7WqsYprgzxbEP3z0yWzYImAqy3jhhatEX/d1VgRDT9Y+iKKJSZFEmE9zV3l1sfFJe+pZOWLGERk2Qr4pJ8OHm7ObULEgC/NxMLzE9FFVxNB+zIBo+g/fyZjyTXSu8T9kDwN8TSmt+mDwUmx2uXEViJKuZm91O11YAnX8uMqT8YKW3KTU0aOcHhuRgX1ExW088yXUHZUHFajrMoS3e91US0NTHPiahvuNVw/fyL+bKu8Tu5fA1Zohhqvx3+YabiEi+HkoQdO1lE3NebngR7l+Nolmw/0IVSBsA7jIV4h/V79cDfsGfYrvgbep3mQoDjpxB7w5bfOl6UgSrc00qOuZOsE/lVbcP9VequMYd0ywlQkclW5EVQkYxLlV7QEEKz0VDFhCqcUy81gJyRNOu01OnZOIviYxSd75WZRcmVEefaB+NROGp7ayM8mW/dobSE8hGxZCL+/fRnkt8vQwrfYPRv93zVFNKggQhvHagTY/4OkT5N8uErzgf3dnoqoWI5JlbruSVi2x9Ef5cQEd17mz5H/hEfrvKr0i6AVRB3P3VUO2a5qvj3df2cf/uMyRUxxILwkD7EphenY7xdSFtYoEPBC2Px6Gb2cEHY6Bzut4rjAjOliPU0j4vYdTl5BnaGsOg8sFE5lMFGWpJr+64YZFCO1eg1OyX8D7Xwm7oSPe0BWSlUuZCsUbQRPdy/12EQiF4/qwMoYOZCuD5jFtpJIj3GAgLRUUF7bBz/HlGhanqQYiDXwB2j2dVPQalghQC4CYUNCcVj2K1Xn2Xo7rayEs953kJK8BDis7LvmtBlY6L5hyaJVIqVbe/abnKwZ6/pPQqnPqxn/i6lQncunr/SzICT9NhvRF4I2zetTRL8MtKtjevdeznAsLcluEdWjcQVWb7F2despnSF62LRIzolXhOpO/dVY6VSa5l7tRu2q0VMW5fnu1zr0fwqB1gYiR/o0QbPmZAcSJ0wAN6jYXF4DWQ2UdGxOuNH/WzrUBln4SnOynFXFFkMf3mF14NmYDTMehpt1HYcTQrUCE4EQJUdHWImtbV+YN6Ot+rW0NLpTg9rqoYURBD/Kvpryezhrb09HRp0tRCJe84iw5h0ZJ3nHkiXIE3rUOJC+BYIhUpzPJWphCTs7kYe20ETYdSq4tSwcFlEYcs8BFr30FArJZniB+Esinga1Ghoe60kVwyWLEj6hYq3bI7wPdacOYO/vdWgY8c6Wog6qQq7zMVbYQb+j9XEIWAXNl03Z7RUT/g1DkqT4qH9PmzchJgqW7LAw2V6rFv/0z3SyiJs2Os= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(52116014)(366016)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KVkgZqUhuriX7sDWYsKv+7Aq0NMPmpYh4axC6HMNQHdl4Nl5U9WdKIAn6gItEogmoTWng8QF2L66LAf5ZDTc+JZrk3YaGF+fdjxGv7IlQ+aI76Z3ufgpOO4M9LV79TLvKM8rmfcXx2nSACfu10GUcXhiyxK0YWDzwyoudpp18pLUJuUMT8pTF/fkDDiMKYTb5fhEvGtWXOwx9CB8pG1asUDLF6KY3RQRzOJ4yUnv1Xk5JNxOUNhJSTgr1DtAvb7TtFLreMXG/PpUBGv2NEBHLhR2nHti0rK+4m6sJYCGyLC2tKqW0frxUF3eSbFGGFITqBpJc7SlgsPBdCv3msmw5bMDnF0dwB1DurwbKKuVg7VenCGH6/K4GJxaA/DaIHa3R6Rs/KO+gAvL8g4by8iivcZr96NxuJvb67d6Ahn0QnT4fdsI0+uEsS1eaCoP8noY8JpgF3yC2TI2jcC2xbpAYYPVhJP3LWvNH94yyJ8ruoh7CYSplH25LfnrMIpyetMv809bFKbqmVHYgS4Wwv3KSM2AoCyIsZA06sKx/KkTXxPvASP7ZhweFABPjEbWQvHVsUB5s5fZxpV1GCN9aQOfFJsKXKCEi+FRlxj8UgwQ0uUCHgRlQPk7x4SF4izxL2y4Ocd0q1ceaAPVq9vdyvmWmsXC43kUhlrk3pdLNz0vHDDFf6MqwGRKoZmovBedJ8g6YeApGA5GxAp5K0GEtplt9289GdLlgwLrrQpz84lb34n/FAZjlMdiysaMolFBmsblzV8YBXMfstFaUN6Va9zmIYUXly7skglQ3lByn8Nmfuq1xopYn4SYpHK86QvIQSi23vj9vJApF3D5CZwu/gFY0cAJvVoZSvwkHys5TDiFxJ2WZapGyQ1i2/RAPtp46PKMg3nENtb1XzIj/T3M0gdZGHR+hb7a9r+pWMY/Vq+Y83f8r4KpZojnzbFecwzJx1TraxQY6rcb/gE16zGFhwKbcDWeYiWQiDbI48UdLe9pON9tXT/8KUJ044nkX20u5jXr1Qnn06JPDHNNtSHqLR7Sf5TcxQ6rd/bLLGmk/0+gqoTsrR6IhmBju+F9A1RFFnF3xngyoYi4En7wdHlP+FhXPtHA2X55Uj9SbwUvLtqljuDIS7Qe6W9XS8KTGBtFiaQ8forelIv68Zv5CHb467f55B4nspAxy++LkeqRcw2ZAC5GqZywZu8ucuPhbOrpCGdQy2QeaFdPc5TC44uB+yE47Dv443P6GK5wabYgMs+Cb+eVzxsenCSIWM5iFCtKVNwKdN9abvhSZVK8hxhcx0z/I75/j+D+js2ar82ZGCrLqLOFwlJ/+h4xDH1YFlmST00xJCKDHuEIXucoSBgHpgSZAx8wgW/oSSTjg9UvJo85rojsTmC6aUIYodG807cyqG2ZdEdOtp9Chwn3HfgjlVZxWLDK/EM+yWb7Bbij08N9sFzq3RIgTOSPESUPcx16tLI1gRUViisFladgqaMlFL3AxHgyiV8UiYas4pMOIrTdljLfi7kE98xBC/1CTPcB7xZWFRaIm0Ii8ouOu6E2rN7XryA5src4STNWD0gOg2qfYLZLAmM6U12AA3davjatbGwf X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: cc5WZV/oTEIokeD162cz2ebNJnAlV24DGJJqQGfJYp2Hf07ZiuayC2XQbbC36YPpqtMzMfqCz8JWkFrIsRVymgNa6iNL9g6qzqOneNKBEUyk2yAbmKnG7Cl/BLXIelT4vAYP03sur5qfZL8DbrEK+6/Euyea8ZGCOaoDa0ml58gzhmdidRfOKM30DzkDiY1cwkAjlzrfxLf8gpUUKzemwKi7wMMmT3Lv7krRoawGpXHscF4OdKdaLK1AElLJG3aO9GoZi2f/25ZvGA0Xj2K5xEh+C3rJsXckAy4JdYDTwp4yiDNTyfIRwh2Xs0LWNuL27b3Q0rfBjmSt2D1LqpY+C23DAxQuQaWBbBGcYC/X0pmU4LKJh/0aBnPR/DAgSbrup+Dlf9LKR5ko1aARtEZmYbl4i6/yhGmXcig2vRu6DZXMC7hMlXlZfiHZSY5oIa2lsA3Kgx3IYBzQcXHEfkFWHgH96vB4XzvVUGBjCNg4A3ek1Rj5YdGreriokKHD5CmL5wM4+44qilv1EXkjKdzzkAITydqUlcra7UF7OR3s3ESalnFh4pR9yxQfKaWWDbUqEaluD985wWJ6n/QNxM89kkzyGdmQusJTNm1EFNe/x1ZrdfutJ1By3RHAGHmshOgb X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5d8466da-2596-460a-a60a-08dcee7a0586 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 07:05:03.6325 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hZjVM32lGxU9pwSoVOdB7RUAZSpcWfKB0bAlFuvV06h12elHwMdyNek8fjhOC5mdUwRNXopSHmBL7RtFOzDe8Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363 X-MDID: 1729148705-2v3JimaQR17Z X-MDID-O: eu1;fra;1729148705;2v3JimaQR17Z;;3e2ef0aab6a0ad8a3f1c1b41b7049f4c X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove the now-unused neighbour::next pointer, leaving struct neighbour solely with the hlist_node implementation. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 - net/core/neighbour.c | 132 ++++++++-------------------------------- 2 files changed, 24 insertions(+), 110 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 96528a6cd74b..61c85f5e1235 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -135,7 +135,6 @@ struct neigh_statistics { #define NEIGH_CACHE_STAT_INC(tbl, field) this_cpu_inc((tbl)->stats->field) struct neighbour { - struct neighbour __rcu *next; struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; @@ -191,7 +190,6 @@ struct pneigh_entry { #define NEIGH_NUM_HASH_RND 4 struct neigh_hash_table { - struct neighbour __rcu **hash_buckets; struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index a1dd419655a1..1c49515b850b 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -205,49 +205,24 @@ static void neigh_update_flags(struct neighbour *neigh, u32 flags, int *notify, } } -static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, - struct neigh_table *tbl) -{ - bool retval = false; - - write_lock(&n->lock); - if (refcount_read(&n->refcnt) == 1) { - struct neighbour *neigh; - - neigh = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); - rcu_assign_pointer(*np, neigh); - hlist_del_rcu(&n->hash); - neigh_mark_dead(n); - retval = true; - } - write_unlock(&n->lock); - if (retval) - neigh_cleanup_and_release(n); - return retval; -} - bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl) { struct neigh_hash_table *nht; - void *pkey = ndel->primary_key; - u32 hash_val; - struct neighbour *n; - struct neighbour __rcu **np; + bool retval = false; nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); - hash_val = tbl->hash(pkey, ndel->dev, nht->hash_rnd); - hash_val = hash_val >> (32 - nht->hash_shift); - np = &nht->hash_buckets[hash_val]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock)))) { - if (n == ndel) - return neigh_del(n, np, tbl); - np = &n->next; + write_lock(&ndel->lock); + if (refcount_read(&ndel->refcnt) == 1) { + hlist_del_rcu(&ndel->hash); + neigh_mark_dead(ndel); + retval = true; } - return false; + write_unlock(&ndel->lock); + if (retval) + neigh_cleanup_and_release(ndel); + return retval; } static int neigh_forced_gc(struct neigh_table *tbl) @@ -389,20 +364,13 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, for (i = 0; i < (1 << nht->hash_shift); i++) { struct neighbour *n; - struct neighbour __rcu **np = &nht->hash_buckets[i]; neigh_for_each(n, &nht->hash_heads[i]) { - if (dev && n->dev != dev) { - np = &n->next; + if (dev && n->dev != dev) continue; - } - if (skip_perm && n->nud_state & NUD_PERMANENT) { - np = &n->next; + if (skip_perm && n->nud_state & NUD_PERMANENT) continue; - } - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); @@ -531,9 +499,7 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { - size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); - size_t size = (1 << shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets; + size_t size = (1 << shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads; struct neigh_hash_table *ret; int i; @@ -544,33 +510,17 @@ static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) if (!ret) return NULL; if (size <= PAGE_SIZE) { - buckets = kzalloc(size, GFP_ATOMIC); - - if (buckets) { - hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); - if (!hash_heads) - kfree(buckets); - } + hash_heads = kzalloc(size, GFP_ATOMIC); } else { - buckets = (struct neighbour __rcu **) + hash_heads = (struct hlist_head *) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); - kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); - - if (buckets) { - hash_heads = (struct hlist_head *) - __get_free_pages(GFP_ATOMIC | __GFP_ZERO, - get_order(hash_heads_size)); - kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); - if (!hash_heads) - free_pages((unsigned long)buckets, get_order(size)); - } + kmemleak_alloc(hash_heads, size, 1, GFP_ATOMIC); } - if (!buckets || !hash_heads) { + if (!hash_heads) { kfree(ret); return NULL; } - ret->hash_buckets = buckets; ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) @@ -583,23 +533,14 @@ static void neigh_hash_free_rcu(struct rcu_head *head) struct neigh_hash_table *nht = container_of(head, struct neigh_hash_table, rcu); - size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets = nht->hash_buckets; - size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); + size_t size = (1 << nht->hash_shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads = nht->hash_heads; - if (size <= PAGE_SIZE) { - kfree(buckets); - } else { - kmemleak_free(buckets); - free_pages((unsigned long)buckets, get_order(size)); - } - - if (hash_heads_size < PAGE_SIZE) { + if (size < PAGE_SIZE) { kfree(hash_heads); } else { kmemleak_free(hash_heads); - free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + free_pages((unsigned long)hash_heads, get_order(size)); } kfree(nht); } @@ -619,7 +560,7 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, return old_nht; for (i = 0; i < (1 << old_nht->hash_shift); i++) { - struct neighbour *n, *next; + struct neighbour *n; struct hlist_node *tmp; neigh_for_each_safe(n, tmp, &old_nht->hash_heads[i]) { @@ -627,14 +568,7 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, new_nht->hash_rnd); hash >>= (32 - new_nht->hash_shift); - next = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); - rcu_assign_pointer(n->next, - rcu_dereference_protected( - new_nht->hash_buckets[hash], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(new_nht->hash_buckets[hash], n); hlist_del_rcu(&n->hash); hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } @@ -739,10 +673,6 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, list_add_tail(&n->managed_list, &n->tbl->managed_list); if (want_ref) neigh_hold(n); - rcu_assign_pointer(n->next, - rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(nht->hash_buckets[hash_val], n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); @@ -976,7 +906,6 @@ static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); struct neigh_hash_table *nht; - struct neighbour __rcu **np; struct hlist_node *tmp; struct neighbour *n; unsigned int i; @@ -1004,7 +933,6 @@ static void neigh_periodic_work(struct work_struct *work) goto out; for (i = 0 ; i < (1 << nht->hash_shift); i++) { - np = &nht->hash_buckets[i]; neigh_for_each_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; @@ -1015,7 +943,7 @@ static void neigh_periodic_work(struct work_struct *work) if ((state & (NUD_PERMANENT | NUD_IN_TIMER)) || (n->flags & NTF_EXT_LEARNED)) { write_unlock(&n->lock); - goto next_elt; + continue; } if (time_before(n->used, n->confirmed) && @@ -1026,9 +954,6 @@ static void neigh_periodic_work(struct work_struct *work) (state == NUD_FAILED || !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); @@ -1036,9 +961,6 @@ static void neigh_periodic_work(struct work_struct *work) continue; } write_unlock(&n->lock); - -next_elt: - np = &n->next; } /* * It's fine to release lock here, even if hash table @@ -3124,22 +3046,16 @@ void __neigh_for_each_release(struct neigh_table *tbl, for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; struct hlist_node *tmp; - struct neighbour __rcu **np; - np = &nht->hash_buckets[chain]; neigh_for_each_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); release = cb(n); if (release) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); - } else - np = &n->next; + } write_unlock(&n->lock); if (release) neigh_cleanup_and_release(n); From patchwork Thu Oct 17 07:04:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13839524 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94E011C2448 for ; Thu, 17 Oct 2024 07:05:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148711; cv=fail; b=b2GgXccHzySm6MmPvXHENxm71AqrKVybAMjXxRJv4PekKAFhW41Jb5ZyZmc5+bZ981MmUjA5m5sLwHcjrVXvRKAbbR0MdSUAnHVuWR6WoDs3S6UowIxqHlOjq6lW1DiJHyFyf+hmwhctXQ1711eAKDTqrF0etfjCPEJ5pF6WvX0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729148711; c=relaxed/simple; bh=2nvz6Gt2jetXL4OGsyajI1dWDOsqb56ZjkAz3pR2kBI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=BVQA/yEiNpccZfsm9CySc6H9qyV7UVGCaKVSODhKt6SV8vANUDE+xzcHlOxMDOy7wMTBpN9dRQC+TsxzNzWs0VSFcN811OHMbQpzNGruSN0Cdp8iLGBIa0uOgNvPPCPlod4UY1F36JyrKKtisRdf5cuiCjw7aF5X/BZ5k5YlGqU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=tDdvFHWp; arc=fail smtp.client-ip=185.132.181.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="tDdvFHWp" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2176.outbound.protection.outlook.com [104.47.17.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id DFF1E740051; Thu, 17 Oct 2024 07:05:05 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Z9x3Wc4b9AuqMC4YI/4CNaHWKusw1Ui4+iE09zz13/5lVCjE6Aa0JmWKdUOZcjHvp0cumeiSHlSO4FXRQd/+U/PMtn5sdILdp3lYkao/wM8EIzHReuGOm7XwLs/0Y0QSIrLqPq2eff+HSlA11z6Q2ZxbIPtuVQ72yGOLKyXWdao6RnnJgXeXwvMbpZH1JwAf26qbESWKC+cpd29ZUoobVaB1fZLfD/vsNeihUPdF63wORzgV+UAO87e+12EPFpxE2BqQtAbFfTRn6C/O1UgNz5xX91ilxYOKwX0lp6leFuRXMpP6fGFYpwZUSvZtDBj19PNERur4rQHSlH079fAHhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C+UV91uQGAw0ZbvSSc4XSgE9poaYb8rDLZ6NwnGx8uk=; b=RVVpyENpwKx9VUN2xFq//bsM2WrJ0C92vxcs6o02VeAwG69iSDtoMjaCLm1qEdBx0gWMNSKuCE2IcD9vr7LtuLMH57s7ZF5+gGQTyuu6FI9FNq3QJPhfRdNRxUZDlLEgoHPpgJ9VdrHia+5Ver/S9c7/RdTeDYm/vOwSdV4pNP8akitZmY3iVFfikzTeS6DnC/3oSYOQhZTQHqz5GAaLw06jLnONB45F09zvp+t6wsPqG7uc1knuRa4W7GAuM4PxRhgCmy3oBJ9mU/WeTni9vMr8Rm3VLnjULBpIFVqrhvMhgIjBBkt8kh1i92JPy1gWO3hGcDfNg5QE7gmkmZl6XQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C+UV91uQGAw0ZbvSSc4XSgE9poaYb8rDLZ6NwnGx8uk=; b=tDdvFHWp0OZl77TIcWXl7Wl1pVg5DiR6toc4mUxWd9CrZBV2U6md94WEkCfoPtgYKxJF0+mfc8WUU1LUheikwHm1wsdnC3r0hTaIAyrmB8/mCE/dPQStVR1m2i74Yn6fOSyM9OFSTUSLx+i+y0Plf7ud2NP/3ByuZPBrIfccOdY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AM0PR08MB5363.eurprd08.prod.outlook.com (2603:10a6:208:188::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.21; Thu, 17 Oct 2024 07:05:04 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.019; Thu, 17 Oct 2024 07:05:04 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima , Ido Schimmel , Petr Machata Subject: [PATCH net-next v5 6/6] Create netdev->neighbour association Date: Thu, 17 Oct 2024 07:04:41 +0000 Message-ID: <20241017070445.4013745-7-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241017070445.4013745-1-gnaaman@drivenets.com> References: <20241017070445.4013745-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0017.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::29) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AM0PR08MB5363:EE_ X-MS-Office365-Filtering-Correlation-Id: 39d292b5-168b-4566-568e-08dcee7a06c5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: pzhNJtSLBdj+QZNcHm7gsI2ZZ91pBljPqob02OMiKVKnjqPTsrherFtLyDSDmVnnpvtcuQ7WcId8helvM/2Y+zkxWflniNnY5ALULz4Xg6HQ3KfJP/3xAAQxCtPP2mMIeCgFBX+BoHUao9VgdsNZC1+UDGMEGXGHYskaxFUcBQZwsYea00/IcZKFwkP8iDtlRZd1VnXmi5fJaDbfvpdZxWxgyP+9yWAaF5LoW0IZZdabwva0dP0dEP8zNFG2V6basB0pQPtBMlree0626BdxRtJEMwdk91Z7q5aqHSb7gEBscK6W+/nbckVQrJPBhZJa81FVN5NOHBwpiP0FVbJdV1tRawiPJ0jcV4itPme2EK5mLBAEinLVuXakzukjGaOVPORwCUOQF+ydALUJuvOEQ4eRls/A3a6EA2vbnfcuBUgn6ptjbxIVJeQeWD8T7AzDkm72l58LIcv+PciXFofw0/6mgjdKpC0WhwwhHDG23dc32kLTG876B//8LHGWssbd8eab7f50bnUDaGOX+pr36XG28+LZEckr0oFm5rznN9x2X7g2gDk/83wV5FJffF1nt6lQuRvFVLQ2LHeUIiQn/1I+AXBk+qMNFYtok9MDfxxPj1obOGkzAbJG+n1QQq9GQm1vEXeCfym4jsL4fx8lkLUDLDQk8/+lc1VZc0Gurk6KmUdWy+K+WSgIYTEJy5NXl4EJ+MKpesvtdvEczv2i1TXkrsWVW6Tqu8PC7r717dHdngbPlT4ucJKDCbIgfKHrEN8/Up4yLNMPUkx1k3mrM0N3zOjf0deL6ICFzAjQZc83cjr7F0PANqHGgtGqhCELu/jkrp1W4kWnIEhreRGrhbpURbwU7P5fL1mlDvb7ssQchT90Ye/LzNT99ixhdjT/KWNwfesfs73oXdn3rCY50uc1rG3hU/TJFTqmPG0pzJl5+kP00pl7Xk2gUIx+aJDz+ysu+VRdWv3Q39h5x/KPgrcrfSoIgYTgVp5ZLrJ2SClU87Ff/UlnxyuvRhGLH3x5TCK1zfUzHmFUF9z3SeNtV6DXevKpweUzKcOITFeBwTx5wrwqlav9kuHs2cIybiGS0BEyB1BSrcl7zHEj3fsWFfot3v1iIg4YRxb+8DZT3dMcYTRI7WxBlabYG6g+OfOKA3hiO7AZkv71bFywHSTk0qZdkZqn98fHA3dhGfdlyYoKJ8e2FTD406kweSV6StULb6yZbWoPvR8WisqI5XDfWmyM2RuwuOrvEg+cpnS4AZ71r5L0W8TW7W+6UxYmdGUML4eJDIot1BuXONQ5a24yH9XNlIrbICmeM9Ln2gcNCzu7jj1PPMSZdhITcfQUADIxBlWWxzQtsIcjw/w2zBA05uf+aeWLQWRQD4gMo/ssHaw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(52116014)(366016)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: X85Rmsq6hW6z9W9f28en1okY+bOb417SjlvzzD4wAp9PPrfXP/q2dQNuaGe3O0Uf9dnonUqyOh5Rw3iorflBAG9HH80LpNz7bbOiszgicbRtTyYf4AqaxScXjWmQlJgHqBjz2iCXsOOCYajZ5hAhRU9m46BPkuSnOe6ZUNsTsaqACsu3JOkxuwGHBLruqAkf7wSPKNw2dL6aKBG2V3rbcD3du3GHPEfGRqe9a/+lhkNmkY9ViaVqplC25RB2Z35U8MVItJO91yEp5s5G+mGLV5WPCBy4aE/dN3/f0fhfNTofb+HXzUQIa+2pey+NSsMzKgsUOH8qJVsmYY++cxPykFU4H+DzTH6a4DpaF99I7CF6sSRgg63mwY3subICUShag2oCqxQXfGi9Rw3Rw2iTfifS0kAWieAtAFQMPUTFHwSILesUHBLlPdT7waucXK2UvFKCaAvDw8e4tDO3/Oi/iC4mxLtTNS0FAKVmjncbSZ5WQvZOSMT316KAXkHQQsicNP14nsUeXGZCkkUn/m30HlTlmHrTuiAjuIl2X/cMuoMRX8q2q0TQd00t2myZhjdWPmT4pQoR2H7sP26LB70X1D/Uam1v3l3JLK0MpDUvTtAEJo5L6JPVCybvs+aJ/KD1n4EACnHtkJRqumRgPoeDsctiVd8qkaoUzekMTfDnQmuDQHeezxHP2sBsZG4Jay9UocXlzKacp2vUI6F5gwm5ee3WSSzcXowlbA9ZcURK9caJlZAh/qv5ftcm+kNCmFEXs8BnrRn8ww3WFGuYJB2hwsGGbGjjYvOEPV93O4GMQ4CEjbob7z8usTHewxCTEHi5LvRx3GCtfxB1uEQ0w6MycHIgpfjExGCPtGXW7vsPgW/HYH6XMsYQ8kTnNJje7dJAUZO2dKwZpPj16MH3JwxTNuT3zkM3CLJXwRgbHs5540RFUo6u6gwZ30591hYNPcf8AXTeD6xFc7fU/f9vYQ0lbZF306DeZQIulDchCHZRLO1lyuzv0Zt1x2dWqioEkuTtiEsZ8Nhazjuu2Z04PAiUMVadtneTmxQiGUNO1FEn/vGXAtKmXPHSikRSKcRAbiMSqMJdiUfb0gvduGrzuF0FwtiyIKMjLFV/uvvGYzpATKTvoX9OqDQtFda566PzGGdgkrjqYibSTegS2L+xrCgmLkz9Z7RjBQSY1PVOMrXaOcOLxJkywXd5WSAmtoXOlZuLroRE8/9HOCVkqUuZ8Wk4qkuRAMhCR/TbzfjPTMrUqOZnSdzfTRaI670Kp5rY/GqH2hlnXHf/Yqq62ECfiH7oVrWlGnp7x2hD4POwwFoUC7y8P/tjr1iR6fqtfHF0xw6kUwkfOf1/INADKx7vzHcEMg/sEQF5UefzIIOqHu3xwojAFJ7bWSxLCnngcvabqs+F568H+kETZTAyTbbe9WgrRF1q4lavlIjyGfXyZgGo72ie1nndqPAAIhiVmGoIX9R4rLNesyxFXlA8dKie2EhIiMu6gQST+vKDHXOzhdIerPsYcNGKpuiegGa0UKwC9e4aje73r6XVHxMCW3YhWLQpoKt5Br0jhDNrLekqJnG3i2TEcRWEcnNV/t/FJjZxRLxw X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: nBJ29tcPWFECbfYykDFkNNamytZMJX0o0MzH43pcWUrctcfzmpRRkrqYYJsli9kshlMMrKHgzC+jaR8gVurjs9r6SAJOf80C2tXzKI/yTukotcidTlNIqRqPNyvHkT5JA7I+Zjaqnx2IV5OdZMw0MPJg7i+DRX7Zu6tBHXNrdoIoSMenZzCncxfx9FrVkox2YNIRt82kCxJ4dSan2vBQE0RQLCDudFU+ljCSO3rxt7dG9+0lmufy80sQgY6EVyrBVKs1n9PdoOa0IpHzx/j6O/xxVAk7zl5Av8bE7NnsAGSuPC7mK3Ym9UYpUgGOTmYwWf98ZpSSHki8o+6RgKoL7PqvFy1tT0uQ93QNkPPRS4nRu5jZiEkTMGrKqdj6bKufKAX0+yB/jB0TQpAI0eFb8v61V00hYP0pjamGQJuu8uptzEnwXjOpTaLP4e1Wv7fyTr58w+tLZUCLZ1BhFXf8FAjhWZWo9LJutQ4A5DTXZBQthEQBFeqQoU7eK4HnxjUpo3C6YFIQq1CilZKYfWxCy5bYLNIAbUNfiLEbDo2I/bmzCfzk7zZkTJn422+S7+ZZztpHQSPguSYhRK4FFsDShSAw36cjVnanafw+FtezdD0IQyHf8VD0MC+bt0KD6PTL X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 39d292b5-168b-4566-568e-08dcee7a06c5 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2024 07:05:04.8075 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8trmRHluVw9lvh2rc0Hmj9r9mGHWGsV/fQ+HuZK+z7ZFbea6VMLWKWNwmauuL1O31SIQe4tds2QOFsJDmAMF7g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5363 X-MDID: 1729148706-bUWKCgVCvL3p X-MDID-O: eu1;fra;1729148706;bUWKCgVCvL3p;;3e2ef0aab6a0ad8a3f1c1b41b7049f4c X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Create a mapping between a netdev and its neighoburs, allowing for much cheaper flushes. Signed-off-by: Gilad Naaman --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 7 ++ include/net/neighbour.h | 9 +- include/net/neighbour_tables.h | 12 +++ net/core/neighbour.c | 95 +++++++++++-------- 5 files changed, 80 insertions(+), 44 deletions(-) create mode 100644 include/net/neighbour_tables.h diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst index db6192b2bb50..2edb6ac1cab4 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -189,4 +189,5 @@ u64 max_pacing_offload_horizon struct_napi_config* napi_config unsigned_long gro_flush_timeout u32 napi_defer_hard_irqs +struct hlist_head neighbours[2] =================================== =========================== =================== =================== =================================================================================== diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 8feaca12655e..80bde95cc302 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -52,6 +52,7 @@ #include #include #include +#include struct netpoll_info; struct device; @@ -2034,6 +2035,9 @@ enum netdev_reg_state { * @napi_defer_hard_irqs: If not zero, provides a counter that would * allow to avoid NIC hard IRQ, on busy queues. * + * @neighbours: List heads pointing to this device's neighbours' + * dev_list, one per address-family. + * * FIXME: cleanup struct net_device such that network protocol info * moves out. */ @@ -2443,6 +2447,9 @@ struct net_device { */ struct net_shaper_hierarchy *net_shaper_hierarchy; #endif + + struct hlist_head neighbours[NEIGH_NR_TABLES]; + u8 priv[] ____cacheline_aligned __counted_by(priv_len); } ____cacheline_aligned; diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 61c85f5e1235..ecff9297c116 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -29,6 +29,7 @@ #include #include #include +#include /* * NUD stands for "neighbor unreachability detection" @@ -136,6 +137,7 @@ struct neigh_statistics { struct neighbour { struct hlist_node hash; + struct hlist_node dev_list; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -236,13 +238,6 @@ struct neigh_table { struct pneigh_entry **phash_buckets; }; -enum { - NEIGH_ARP_TABLE = 0, - NEIGH_ND_TABLE = 1, - NEIGH_NR_TABLES, - NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ -}; - static inline int neigh_parms_family(struct neigh_parms *p) { return p->tbl->family; diff --git a/include/net/neighbour_tables.h b/include/net/neighbour_tables.h new file mode 100644 index 000000000000..bcffbe8f7601 --- /dev/null +++ b/include/net/neighbour_tables.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _NET_NEIGHBOUR_TABLES_H +#define _NET_NEIGHBOUR_TABLES_H + +enum { + NEIGH_ARP_TABLE = 0, + NEIGH_ND_TABLE = 1, + NEIGH_NR_TABLES, + NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ +}; + +#endif diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 1c49515b850b..2683fb68f5b5 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -61,6 +61,25 @@ static int pneigh_ifdown_and_unlock(struct neigh_table *tbl, static const struct seq_operations neigh_stat_seq_ops; #endif +static struct hlist_head *neigh_get_dev_table(struct net_device *dev, int family) +{ + int i; + + switch (family) { + default: + DEBUG_NET_WARN_ON_ONCE(1); + fallthrough; /* to avoid panic by null-ptr-deref */ + case AF_INET: + i = NEIGH_ARP_TABLE; + break; + case AF_INET6: + i = NEIGH_ND_TABLE; + break; + } + + return &dev->neighbours[i]; +} + /* Neighbour hash table buckets are protected with rwlock tbl->lock. @@ -216,6 +235,7 @@ bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl) write_lock(&ndel->lock); if (refcount_read(&ndel->refcnt) == 1) { hlist_del_rcu(&ndel->hash); + hlist_del_rcu(&ndel->dev_list); neigh_mark_dead(ndel); retval = true; } @@ -356,47 +376,42 @@ static void pneigh_queue_purge(struct sk_buff_head *list, struct net *net, static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, bool skip_perm) { - int i; - struct neigh_hash_table *nht; - - nht = rcu_dereference_protected(tbl->nht, - lockdep_is_held(&tbl->lock)); + struct hlist_head *dev_head; + struct hlist_node *tmp; + struct neighbour *n; - for (i = 0; i < (1 << nht->hash_shift); i++) { - struct neighbour *n; + dev_head = neigh_get_dev_table(dev, tbl->family); - neigh_for_each(n, &nht->hash_heads[i]) { - if (dev && n->dev != dev) - continue; - if (skip_perm && n->nud_state & NUD_PERMANENT) - continue; + hlist_for_each_entry_safe(n, tmp, dev_head, dev_list) { + if (skip_perm && n->nud_state & NUD_PERMANENT) + continue; - hlist_del_rcu(&n->hash); - write_lock(&n->lock); - neigh_del_timer(n); - neigh_mark_dead(n); - if (refcount_read(&n->refcnt) != 1) { - /* The most unpleasant situation. - We must destroy neighbour entry, - but someone still uses it. - - The destroy will be delayed until - the last user releases us, but - we must kill timers etc. and move - it to safe state. - */ - __skb_queue_purge(&n->arp_queue); - n->arp_queue_len_bytes = 0; - WRITE_ONCE(n->output, neigh_blackhole); - if (n->nud_state & NUD_VALID) - n->nud_state = NUD_NOARP; - else - n->nud_state = NUD_NONE; - neigh_dbg(2, "neigh %p is stray\n", n); - } - write_unlock(&n->lock); - neigh_cleanup_and_release(n); + hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); + write_lock(&n->lock); + neigh_del_timer(n); + neigh_mark_dead(n); + if (refcount_read(&n->refcnt) != 1) { + /* The most unpleasant situation. + * We must destroy neighbour entry, + * but someone still uses it. + * + * The destroy will be delayed until + * the last user releases us, but + * we must kill timers etc. and move + * it to safe state. + */ + __skb_queue_purge(&n->arp_queue); + n->arp_queue_len_bytes = 0; + WRITE_ONCE(n->output, neigh_blackhole); + if (n->nud_state & NUD_VALID) + n->nud_state = NUD_NOARP; + else + n->nud_state = NUD_NONE; + neigh_dbg(2, "neigh %p is stray\n", n); } + write_unlock(&n->lock); + neigh_cleanup_and_release(n); } } @@ -674,6 +689,10 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, if (want_ref) neigh_hold(n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); + + hlist_add_head_rcu(&n->dev_list, + neigh_get_dev_table(dev, tbl->family)); + write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -955,6 +974,7 @@ static void neigh_periodic_work(struct work_struct *work) !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3054,6 +3074,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, release = cb(n); if (release) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); } write_unlock(&n->lock);