From patchwork Fri Oct 4 13:47:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13822393 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27BCC2101A7 for ; Fri, 4 Oct 2024 13:47:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049645; cv=none; b=iq6FxOSirLTP4hZn41pZaYAxsCpCze2WXFjmD29XpQQJd6xPinqD7KTP70AT/ADWJCeAu5S3FIwhGJZuEF4XRs6+CIBJ4bjHPaH5qiOwVr9Fjc1Jj+CdWlZQ/BnnU4NYXDQtxIME2PF4eaRBaQq9L5+cqg+lhuBHqQG31TjloxM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049645; c=relaxed/simple; bh=e1CRcsMpd5M2RcJa27XDZ1qMDqPgHbZQ7mxZ0lawOs4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Mfqun9J2hselpzgDIkDgTdxUwEjueylboo1WbimGKgzY1SE+NLr5XlwxMxQ1Qs17N33VHIltGZYrr7iuF0trf9Vk9O5vA5J/4WAYo3z1AyY+gmrhvF6gESqk1E8pcaF8NQAkMwtG49hCovDnvtjdxTxXLFnRSNAK2zw0QqvKGlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VUZNIy8c; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VUZNIy8c" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e287876794aso2985687276.0 for ; Fri, 04 Oct 2024 06:47:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728049643; x=1728654443; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AOnP393jP1F/mUC/uyT/bfXU+E03qSB5aem28E6VSU8=; b=VUZNIy8czBke6IPcDqfLop6p3PoRpDS3bPz0g5CkUL92OPrZDuyOaY/er690+M+oTd QlCmtDOG3nSGQxSi9sSsXut0/yT977JqODrjvCX1QwSnaE+kUmF7Z+A/6h7Mtxh6MAT8 yVN5jRkQLimlfwikVM1d9AX11RbUzV9QkwHLEUunlxFj8s6YVIXF4FgjanVz2M3VBzRz JbKltUqG+OfQZFOtuBRJZOwzzYQsLHs2WFzwZYIk0eOuJU51bMdiOBzPCJcM065mLN3d W8oUklSVnGQfTzKRm3yds2eUAJZmtxQk56/pb37p1ElZvzllXGlZmN8+QA7rF5FmkxVb Ew+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728049643; x=1728654443; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AOnP393jP1F/mUC/uyT/bfXU+E03qSB5aem28E6VSU8=; b=Aml6IIkYXxTyKETarHMiHxrds2vehcehd23+vnS0f/bnibUNFQ1L5wYpYRKVfZ55ec Zcgmr9uxDpGz5VOPh4XRKql9tmrU45zwt2DOYsQiGEmFEa/e2oYE7Zcti7aR7XTcK4X3 p29QYeGbbPslrLHCHd57j4W+f4NGcaOM0YwPkyV7/t08gMfY8025CIHoQWOdmKGvAcqf 1X6OCzYvJGg2VAuDE2kzo2N+CoBY79J1NWt7Y0TvnbaAqv6UXLXFWY5kZ1CKEv9SbPhU Cl6JgmCel07JKkfbJzMZfBQiFB85FwD/CzT5BjOU+djYoX9RDiObB79LN0k7Sdducw6G dPHQ== X-Forwarded-Encrypted: i=1; AJvYcCXpxlZQCg2M33L0FgCbyZz+IBmu1IQ17WfTZx/sI2vVMfcZ3uDj1dJlW+lIsett+aMeVwfbGwE=@vger.kernel.org X-Gm-Message-State: AOJu0YxRGaFgqydvMYBk4tKm1gUNenQze9wnxTQ6HZ9ibh/aTzlqw/Um e2wNAwxNTqI/FMBhwK64NuM7WqHGHIcaw9OmvD7cXAr9waR14PpDvVAKvqsSY4W5m0ULd42zL+H Jr3hrGlEfTg== X-Google-Smtp-Source: AGHT+IGT7VrT47gIJk5s+DZe0/qm2fS2qnzx2SyAUbxE+0e+EgfOvP8QC1Iw9Lmp37zfK+kJpwEnOnk2bXEHVQ== X-Received: from edumazet1.c.googlers.com ([fda3:e722:ac3:cc00:f7:ea0b:ac12:11d6]) (user=edumazet job=sendgmr) by 2002:a5b:c01:0:b0:e11:584c:76e2 with SMTP id 3f1490d57ef6-e28936be41bmr6126276.2.1728049643105; Fri, 04 Oct 2024 06:47:23 -0700 (PDT) Date: Fri, 4 Oct 2024 13:47:17 +0000 In-Reply-To: <20241004134720.579244-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241004134720.579244-1-edumazet@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog Message-ID: <20241004134720.579244-2-edumazet@google.com> Subject: [PATCH net-next 1/4] ipv4: remove fib_devindex_hashfn() From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: David Ahern , Kuniyuki Iwashima , Alexandre Ferrieux , netdev@vger.kernel.org, eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org fib_devindex_hashfn() converts a 32bit ifindex value to a 8bit hash. It makes no sense doing this from fib_info_hashfn() and fib_find_info_nh(). It is better to keep as many bits as possible to let fib_info_hashfn_result() have better spread. Only fib_info_devhash_bucket() needs to make this operation, we can 'inline' trivial fib_devindex_hashfn() in it. Signed-off-by: Eric Dumazet Reviewed-by: Kuniyuki Iwashima --- net/ipv4/fib_semantics.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index 1a847ba4045876c91948bcec45cb5eccc0ef1f31..1219d1b325910322dd978f3962a4cafa8e8db10b 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -322,17 +322,12 @@ static inline int nh_comp(struct fib_info *fi, struct fib_info *ofi) return 0; } -static inline unsigned int fib_devindex_hashfn(unsigned int val) -{ - return hash_32(val, DEVINDEX_HASHBITS); -} - static struct hlist_head * fib_info_devhash_bucket(const struct net_device *dev) { u32 val = net_hash_mix(dev_net(dev)) ^ dev->ifindex; - return &fib_info_devhash[fib_devindex_hashfn(val)]; + return &fib_info_devhash[hash_32(val, DEVINDEX_HASHBITS)]; } static unsigned int fib_info_hashfn_1(int init_val, u8 protocol, u8 scope, @@ -362,10 +357,10 @@ static inline unsigned int fib_info_hashfn(struct fib_info *fi) fi->fib_priority); if (fi->nh) { - val ^= fib_devindex_hashfn(fi->nh->id); + val ^= fi->nh->id; } else { for_nexthops(fi) { - val ^= fib_devindex_hashfn(nh->fib_nh_oif); + val ^= nh->fib_nh_oif; } endfor_nexthops(fi) } @@ -380,7 +375,7 @@ static struct fib_info *fib_find_info_nh(struct net *net, struct fib_info *fi; unsigned int hash; - hash = fib_info_hashfn_1(fib_devindex_hashfn(cfg->fc_nh_id), + hash = fib_info_hashfn_1(cfg->fc_nh_id, cfg->fc_protocol, cfg->fc_scope, (__force u32)cfg->fc_prefsrc, cfg->fc_priority); From patchwork Fri Oct 4 13:47:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13822394 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEE3D212F0B for ; Fri, 4 Oct 2024 13:47:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049647; cv=none; b=lv7EKacFp/mrAobq9yQs9CPZSK6zST8J57L8kN4wiAghlStc7xxQEboiOLnGqYL0Gjs+EQ/hmlsD9H6EVI3savrX9og3pabvyuqVr11N8uvceS6o98kwVTxl2XRQQf/nB0qJqjPpNi9eeP29E6JjnFC0yFs+rR2M5G4jPnxxbCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049647; c=relaxed/simple; bh=a72KhzjwyFh3nz7XfPh4moZi8YtJdVMW5XG/AFMUBFo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RAKb/A8lxIykxFrVJdk9IxgztHj7xYNMPuJ2CbEyOCD3mBIsmEx/CewEkbaGypNlhueFrO7FTtOn4+MpKOBAzm+OVKX9lBsu0YuFTek6smgYj2k3yHSBmuFhDvXubpAeJhDnpp25hDaiPvpX8a7sma4jZ+TLzBnqMtYTZr48gA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eT5svHqm; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eT5svHqm" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e262e764f46so3492232276.1 for ; Fri, 04 Oct 2024 06:47:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728049645; x=1728654445; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yvrrepST4GJRUm66NiZVOohqpKiv6IlBESIes/aP9+s=; b=eT5svHqmE+OZzPi23uSqn9y8G1jwfjtEv2/XPp/TMoi/EZ5yl9d6DKpquofg3xeASQ lyIwlgcwR5bkQYZ3bSdWpGL0+DWeZa0qcd6E2I6VvGm/HW/D1mk27iy57zIqkuB+6Yrf WTlcavVbpjyYj8H0lbV5cY0iaGdeq+b6sjUEDwyhfMO5Ns9EHtQxq1gua02uICy0xRfZ k+TjkGUdYDle9sLEob7g0bU9inqbGfUjgGpqbDqRn7AWWkFLwUHJJ0JimVy8DJv8ZsZI uDRUpBn8snCFGPsvkmSK6SiBVr7/d9piY65EcbHM0dCtCAO+4M0s1u+NdmvYLU5+TAMY auRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728049645; x=1728654445; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yvrrepST4GJRUm66NiZVOohqpKiv6IlBESIes/aP9+s=; b=Wu38gJOwfMNm0Af2LKpSmJUmrzdkUkEM5qXOMVTBQ/MsQd4nL7iN/snbQ+x2Ai0wEb oJZOJxSMaInsFujIVAbVyTc2P6gcknjy+pO+vVP5dMI98UcNUbVkXSwLpfoXAa8JxpAu b6Dh8n0fnIY1463g9Jk/gAti8ki/RE7up98QA1HxapWdXiqA3oYbXO2lD01WvJ9Vv7J7 vN0jdg/I764kUnIgR9D1tB+HIyZeF0q0iUQhU8hG06im3QSPZEownS6fLpYZeaSHo+qS 2jtgGEmy98T+AuZxUEmVCeCQ/uQbY9A02NXMGKK2Q+Yxec6IHYpC8YpvW7reGd5OmaoX up8w== X-Forwarded-Encrypted: i=1; AJvYcCVEMRo/C7kBkbeZwdS8Bj8yrXbjmLDcIweDM57r3wM43e9p5S94p2qTvKpMVN12k9Hr5H0gDM8=@vger.kernel.org X-Gm-Message-State: AOJu0YyNI0iM3qLpEM9nE1xNzkcXKB8qm+nEdL7VbGurnjqi5ogWw0vz MJrmEFsS/q+58P+h4p4KbFUqDxMR7WF6pFAoQyUzesuO9zKTEuF4Ms59//eCIso3LLERA1m+XTV B/5xRZAboeA== X-Google-Smtp-Source: AGHT+IGzRV898qBhhKaW9DBm2fq/XmwGCW4N8pTQ4fudT+j/im7WilGv2r4Aa1adN861v65p/QkTaHMCIxU8vw== X-Received: from edumazet1.c.googlers.com ([fda3:e722:ac3:cc00:f7:ea0b:ac12:11d6]) (user=edumazet job=sendgmr) by 2002:a25:b191:0:b0:e26:177c:86ed with SMTP id 3f1490d57ef6-e28936bb059mr1733276.1.1728049644870; Fri, 04 Oct 2024 06:47:24 -0700 (PDT) Date: Fri, 4 Oct 2024 13:47:18 +0000 In-Reply-To: <20241004134720.579244-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241004134720.579244-1-edumazet@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog Message-ID: <20241004134720.579244-3-edumazet@google.com> Subject: [PATCH net-next 2/4] ipv4: use rcu in ip_fib_check_default() From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: David Ahern , Kuniyuki Iwashima , Alexandre Ferrieux , netdev@vger.kernel.org, eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org fib_info_devhash[] is not resized in fib_info_hash_move(). fib_nh structs are already freed after an rcu grace period. This will allow to remove fib_info_lock in the following patch. Signed-off-by: Eric Dumazet Reviewed-by: Kuniyuki Iwashima --- net/ipv4/fib_semantics.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index 1219d1b325910322dd978f3962a4cafa8e8db10b..e0ffb4ffd95d0f9ebc796c3129bc2f494fb478dd 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -275,7 +275,7 @@ void fib_release_info(struct fib_info *fi) change_nexthops(fi) { if (!nexthop_nh->fib_nh_dev) continue; - hlist_del(&nexthop_nh->nh_hash); + hlist_del_rcu(&nexthop_nh->nh_hash); } endfor_nexthops(fi) } /* Paired with READ_ONCE() from fib_table_lookup() */ @@ -431,28 +431,23 @@ static struct fib_info *fib_find_info(struct fib_info *nfi) } /* Check, that the gateway is already configured. - * Used only by redirect accept routine. + * Used only by redirect accept routine, under rcu_read_lock(); */ int ip_fib_check_default(__be32 gw, struct net_device *dev) { struct hlist_head *head; struct fib_nh *nh; - spin_lock(&fib_info_lock); - head = fib_info_devhash_bucket(dev); - hlist_for_each_entry(nh, head, nh_hash) { + hlist_for_each_entry_rcu(nh, head, nh_hash) { if (nh->fib_nh_dev == dev && nh->fib_nh_gw4 == gw && !(nh->fib_nh_flags & RTNH_F_DEAD)) { - spin_unlock(&fib_info_lock); return 0; } } - spin_unlock(&fib_info_lock); - return -1; } @@ -1606,7 +1601,7 @@ struct fib_info *fib_create_info(struct fib_config *cfg, if (!nexthop_nh->fib_nh_dev) continue; head = fib_info_devhash_bucket(nexthop_nh->fib_nh_dev); - hlist_add_head(&nexthop_nh->nh_hash, head); + hlist_add_head_rcu(&nexthop_nh->nh_hash, head); } endfor_nexthops(fi) } spin_unlock_bh(&fib_info_lock); From patchwork Fri Oct 4 13:47:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13822395 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A698212EF1 for ; Fri, 4 Oct 2024 13:47:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049649; cv=none; b=LTro3DzTNBSuwSlZ5OlFQ6HdHVL4D6aoLOQ+ig54zXlFWQPwSQHUoivS5H4VcJ0fwa2cIhmK2Jc5wDK7C+GuiY5TdzvZhYCFRtOP6pZVqG6KHShgpLs9BxQXQzfgNzMdhsnuQBlhcTnNntiC9N7j6EmCryKPZyPn8B5hwpOVEQg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049649; c=relaxed/simple; bh=FdPCmnOZd/iEpeME2+r5ld8cauR1LfAxYemibWg3fok=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mTXuR1tyjcXskERE6VG5E1RwYh1zXhTTiFmM8YC/or9qqtjxceNCVmiID6hFn6schAdnbzJGTGFoeVeHBFVdV84eGEHLr+ug5C7d8Ku2/z3Nnlb6UtRzfMLxVcvS63qz6rXthZr4xnIiNffCxlp/3WJ6MOi8TPy1lprLm7CV+mo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4CY/7vRB; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4CY/7vRB" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e0082c1dd0so44807137b3.3 for ; Fri, 04 Oct 2024 06:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728049646; x=1728654446; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GkcdOzyp0Sswgvcw/W6M2O1tJ+cYqAFTbyGLaQgssj4=; b=4CY/7vRBSa1b41et7WXXAT/5ABslFDU/5DGCo4rWnU7onRVnjPqxckeDlWfhBTrmf8 IB7n4TtFUlcK2G8IqCAe4U/9m+BaTmnRbA4qu2GuxJgzR6hwpqf18igBLWX77mYA3y7H Ui17fKTX00GBVBSY+zhjGJUtysB9vRnfTD8nxoWD57HVxBXKNv5bSqFuhiu2sNMr6BIb JK9DGj6z46Ss6CiRn8bJeHeJQt5YXQml5S+o96VnX9kw6XTKtVFvnUSSHOZvUVlDrESD 4PrWUiFfPM3/xsc35DViNDEM9H/pegZLpRmmczErqMC+NQ8z1nrCcFpnAuOGI+PptqIp FyZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728049646; x=1728654446; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GkcdOzyp0Sswgvcw/W6M2O1tJ+cYqAFTbyGLaQgssj4=; b=TkQ1uWBk/Ucyf/w6vBOcCHEM0VN6lbmWzY5JZWR2Mi13bMZmh6N5PAjvoteICWsCpY f6k0Xu12buic8+Z92tYWoVN4j6iCXxgVF5jsyncvQAz4RxAbiK4OmB2/DCa/P3JySquj +wQtyUQtaOkVRniR6beeJ14zPIUPtumh+x9AAnDzvpJ+TDT+vw+OnfCp90vl5drvFpVH lM+fjx4qEIMeMTbXPkwOsddRF2m6m2/bIqEQh7qYgdoYJDkIYaetHeX6anHwIlvu5v8J gPMykZXpIbaMQkE03gu1cPF1yddPo5h+/cdPPP8yutRpcJqxEQ383Rj8xKGkD6NMTqKz QUag== X-Forwarded-Encrypted: i=1; AJvYcCWl6J9ahr+QU+nRXCB00ykucqQb6R9jO9GybJidvtoC3wRskSZYfcpF05dqYog2OBqoJOuIIlc=@vger.kernel.org X-Gm-Message-State: AOJu0Yx0GZbYkF7I0nK4FtGUHfO3kMg87yaeNB/CNwB51kJLFAFFyksH 4kuiR4dD/f5D0ErkqChubkIt4Vg9uHe4kWE5rRkLTpKys19ory4rYEZXIlXRrz3S2DerYI2w/IX MItL5z1GTKw== X-Google-Smtp-Source: AGHT+IEpr8cXEWbGKzVB+mIvO1lJMHJ3m7V1DK1EyTo1tpdhw2B2lZCrpU6V+18LK7adxFBTRJsv26mhnA3fGw== X-Received: from edumazet1.c.googlers.com ([fda3:e722:ac3:cc00:f7:ea0b:ac12:11d6]) (user=edumazet job=sendgmr) by 2002:a25:d804:0:b0:e28:8f00:896a with SMTP id 3f1490d57ef6-e289393a895mr1633276.8.1728049646391; Fri, 04 Oct 2024 06:47:26 -0700 (PDT) Date: Fri, 4 Oct 2024 13:47:19 +0000 In-Reply-To: <20241004134720.579244-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241004134720.579244-1-edumazet@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog Message-ID: <20241004134720.579244-4-edumazet@google.com> Subject: [PATCH net-next 3/4] ipv4: remove fib_info_lock From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: David Ahern , Kuniyuki Iwashima , Alexandre Ferrieux , netdev@vger.kernel.org, eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org After the prior patch, fib_info_lock became redundant because all of its users are holding RTNL. BH protection is not needed. Remove the READ_ONCE()/WRITE_ONCE() annotations around fib_info_cnt, since it is protected by RTNL. Signed-off-by: Eric Dumazet Reviewed-by: Kuniyuki Iwashima --- net/ipv4/fib_semantics.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index e0ffb4ffd95d0f9ebc796c3129bc2f494fb478dd..ece779bfb8f6bec67eb7751761df9a4f158020a8 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -50,7 +50,6 @@ #include "fib_lookup.h" -static DEFINE_SPINLOCK(fib_info_lock); static struct hlist_head *fib_info_hash; static struct hlist_head *fib_info_laddrhash; static unsigned int fib_info_hash_size; @@ -260,12 +259,11 @@ EXPORT_SYMBOL_GPL(free_fib_info); void fib_release_info(struct fib_info *fi) { - spin_lock_bh(&fib_info_lock); + ASSERT_RTNL(); if (fi && refcount_dec_and_test(&fi->fib_treeref)) { hlist_del(&fi->fib_hash); - /* Paired with READ_ONCE() in fib_create_info(). */ - WRITE_ONCE(fib_info_cnt, fib_info_cnt - 1); + fib_info_cnt--; if (fi->fib_prefsrc) hlist_del(&fi->fib_lhash); @@ -282,7 +280,6 @@ void fib_release_info(struct fib_info *fi) WRITE_ONCE(fi->fib_dead, 1); fib_info_put(fi); } - spin_unlock_bh(&fib_info_lock); } static inline int nh_comp(struct fib_info *fi, struct fib_info *ofi) @@ -1266,7 +1263,7 @@ static void fib_info_hash_move(struct hlist_head *new_info_hash, unsigned int old_size = fib_info_hash_size; unsigned int i; - spin_lock_bh(&fib_info_lock); + ASSERT_RTNL(); old_info_hash = fib_info_hash; old_laddrhash = fib_info_laddrhash; fib_info_hash_size = new_size; @@ -1303,8 +1300,6 @@ static void fib_info_hash_move(struct hlist_head *new_info_hash, } } - spin_unlock_bh(&fib_info_lock); - kvfree(old_info_hash); kvfree(old_laddrhash); } @@ -1380,6 +1375,7 @@ struct fib_info *fib_create_info(struct fib_config *cfg, int nhs = 1; struct net *net = cfg->fc_nlinfo.nl_net; + ASSERT_RTNL(); if (cfg->fc_type > RTN_MAX) goto err_inval; @@ -1422,8 +1418,7 @@ struct fib_info *fib_create_info(struct fib_config *cfg, err = -ENOBUFS; - /* Paired with WRITE_ONCE() in fib_release_info() */ - if (READ_ONCE(fib_info_cnt) >= fib_info_hash_size) { + if (fib_info_cnt >= fib_info_hash_size) { unsigned int new_size = fib_info_hash_size << 1; struct hlist_head *new_info_hash; struct hlist_head *new_laddrhash; @@ -1582,7 +1577,7 @@ struct fib_info *fib_create_info(struct fib_config *cfg, refcount_set(&fi->fib_treeref, 1); refcount_set(&fi->fib_clntref, 1); - spin_lock_bh(&fib_info_lock); + fib_info_cnt++; hlist_add_head(&fi->fib_hash, &fib_info_hash[fib_info_hashfn(fi)]); @@ -1604,7 +1599,6 @@ struct fib_info *fib_create_info(struct fib_config *cfg, hlist_add_head_rcu(&nexthop_nh->nh_hash, head); } endfor_nexthops(fi) } - spin_unlock_bh(&fib_info_lock); return fi; err_inval: From patchwork Fri Oct 4 13:47:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13822396 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D5E62139B0 for ; Fri, 4 Oct 2024 13:47:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049651; cv=none; b=cmGqGg+x5B0j0i6Yv8YrPikFjhqLduKXmZ7cqtzT9Y7sX8oXb2DySyWi5Hxev3BlRPp5TMxOna7/CuD0VwK+yJyUz4N5ad4R3qAYprDCLwFPJcLvwUnGK/D1/S7ESPJf9EPHCGUw+vimiU2m8i8zsJUlqldqxRiClDxk3+0dPzQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728049651; c=relaxed/simple; bh=FTzNJNOMI0maseYOV5BKV36sxfk5x1kC1qb63hOFfew=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=p8bEn6uQaCMzfBBJbhomj0h/NwimonIqYVd4SwEIJaQju4SiBW2ZeDVQvwAIckiMFZoiOKhc9AC9pSVIPxeL4ZLn4Lh+zs05OrF9uQueJ/W05aw444ySXUrPKRL3YziapN4B+EreNRxB6e1y+sPxN5D9gRMELogxg1mtoJXX9bk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=s+VgZ2AN; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="s+VgZ2AN" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e2605ce4276so3636890276.3 for ; Fri, 04 Oct 2024 06:47:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728049648; x=1728654448; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xA/hAHSrZwcpgvbZKlPEB+5PNe5KsZLBA1YCK5/7P1M=; b=s+VgZ2ANf2eK9aom9grXoMBeaMN0ghWzxQglpRFHqg7YgFCUSs2rSL8mShwC5RCO9W SDJNfh4/rZk7EZYypR8ONb2yOz521i6mtl/mb2gebxaKmMPbKGYxHB0fDNn77jVUilQ+ yRoSW/uiOOv+sWirhFLNl+u8w5euje5Sb1M1KmvyOcMQM98rKhs2kqYk2+lYpO/Nh567 cPpC1CPEPFS9Pe71lvZGfPbvZF/L2GBc+rUMvK+KNvfwsL6pRDIG14W38FjFxGzasLSw BF8mqGpO0p8bJzM07QNL2QhmbzHDJqzJGDXkxEAetNUvgDKN7ynjaZqkIs1ZMaLW99EE JuIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728049648; x=1728654448; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xA/hAHSrZwcpgvbZKlPEB+5PNe5KsZLBA1YCK5/7P1M=; b=m9YJzFxzzD61e5c1Sbw4wr/9mZddAnEzwy5vxLmvpdwgPzF5Gx07C6UtdHAXR1Bc/I 7tLJpbtd/5Oix/bfE/Ss8iVbXppdcwgHv8oAkJbsUyozltnUHzI9ApF0F29ieedTbW3j joJZ3tygiGiM2ut000ATJFePyTMr10lQLJ9+LREJb9YSJQz1QgUubokRN2ibg7h/rIeb MF/ibEhgax0i9/gE+/IVHUAaTAviXVQIP5AEnVkK9V+gDOfMpG8wJGbVhtn74Sm8NTdb ItAOdptpIbriyAJ4MdCwJ7Njehh/6aKVMYkx/8AzjNWmQmCXebQqiRlfPJWAzlMouq47 eBVg== X-Forwarded-Encrypted: i=1; AJvYcCVCgPvLsCninLTqSHOpG0HL46atx0Kh0IoP/Q84lv4LomfxN9PCfz3L0ZutKFp3aolQLvFPSzQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yx3OWh3r77cc+JZ2VZMIr/yYHMxK0/9MxfwBwJWhbEJzVQQRBz4 +1xIoKoFRqiQK4HJIJ96891zqaSuX1XfooQOTwtm5TviwnJFvikKNR4XbRV10G/OFWW8tJ27Jrb BIDkp9mBDmg== X-Google-Smtp-Source: AGHT+IEVnbC477oBfefyG9in4R227iIWNg6crArrYdA5kArbITMF48+AbLXrPc268uCT8ozrYM6oiBvkecnzdw== X-Received: from edumazet1.c.googlers.com ([fda3:e722:ac3:cc00:f7:ea0b:ac12:11d6]) (user=edumazet job=sendgmr) by 2002:a5b:49:0:b0:e0e:c9bc:3206 with SMTP id 3f1490d57ef6-e28936e2e94mr1949276.5.1728049648419; Fri, 04 Oct 2024 06:47:28 -0700 (PDT) Date: Fri, 4 Oct 2024 13:47:20 +0000 In-Reply-To: <20241004134720.579244-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241004134720.579244-1-edumazet@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog Message-ID: <20241004134720.579244-5-edumazet@google.com> Subject: [PATCH net-next 4/4] ipv4: remove fib_info_devhash[] From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: David Ahern , Kuniyuki Iwashima , Alexandre Ferrieux , netdev@vger.kernel.org, eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org Upcoming per-netns RTNL conversion needs to get rid of shared hash tables. fib_info_devhash[] is one of them. It is unclear why we used a hash table, because a single hlist_head per net device was cheaper and scalable. Signed-off-by: Eric Dumazet Reviewed-by: Kuniyuki Iwashima --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 2 ++ net/ipv4/fib_semantics.c | 35 ++++++++----------- 3 files changed, 18 insertions(+), 20 deletions(-) diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst index 22b07c814f4a4575d255fdf472d07c549536e543..a8e2a7ce0383343464800be8db31aeddd791f086 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -83,6 +83,7 @@ unsigned_int allmulti bool uc_promisc unsigned_char nested_level struct_in_device* ip_ptr read_mostly read_mostly __in_dev_get +struct hlist_head fib_nh_head struct_inet6_dev* ip6_ptr read_mostly read_mostly __in6_dev_get struct_vlan_info* vlan_info struct_dsa_port* dsa_ptr diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 4d20c776a4ff3d0e881b8d9b99901edb35f66da2..cda20a3fe1adf54c1e6df5b5a8882ef7830e1b46 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2209,6 +2209,8 @@ struct net_device { /* Protocol-specific pointers */ struct in_device __rcu *ip_ptr; + struct hlist_head fib_nh_head; + #if IS_ENABLED(CONFIG_VLAN_8021Q) struct vlan_info __rcu *vlan_info; #endif diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index ece779bfb8f6bec67eb7751761df9a4f158020a8..d2cee5c314f5e76530ac564f49b433822bb0a272 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -56,10 +56,6 @@ static unsigned int fib_info_hash_size; static unsigned int fib_info_hash_bits; static unsigned int fib_info_cnt; -#define DEVINDEX_HASHBITS 8 -#define DEVINDEX_HASHSIZE (1U << DEVINDEX_HASHBITS) -static struct hlist_head fib_info_devhash[DEVINDEX_HASHSIZE]; - /* for_nexthops and change_nexthops only used when nexthop object * is not set in a fib_info. The logic within can reference fib_nh. */ @@ -319,12 +315,9 @@ static inline int nh_comp(struct fib_info *fi, struct fib_info *ofi) return 0; } -static struct hlist_head * -fib_info_devhash_bucket(const struct net_device *dev) +static struct hlist_head *fib_nh_head(struct net_device *dev) { - u32 val = net_hash_mix(dev_net(dev)) ^ dev->ifindex; - - return &fib_info_devhash[hash_32(val, DEVINDEX_HASHBITS)]; + return &dev->fib_nh_head; } static unsigned int fib_info_hashfn_1(int init_val, u8 protocol, u8 scope, @@ -435,11 +428,11 @@ int ip_fib_check_default(__be32 gw, struct net_device *dev) struct hlist_head *head; struct fib_nh *nh; - head = fib_info_devhash_bucket(dev); + head = fib_nh_head(dev); hlist_for_each_entry_rcu(nh, head, nh_hash) { - if (nh->fib_nh_dev == dev && - nh->fib_nh_gw4 == gw && + DEBUG_NET_WARN_ON_ONCE(nh->fib_nh_dev != dev); + if (nh->fib_nh_gw4 == gw && !(nh->fib_nh_flags & RTNH_F_DEAD)) { return 0; } @@ -1595,7 +1588,7 @@ struct fib_info *fib_create_info(struct fib_config *cfg, if (!nexthop_nh->fib_nh_dev) continue; - head = fib_info_devhash_bucket(nexthop_nh->fib_nh_dev); + head = fib_nh_head(nexthop_nh->fib_nh_dev); hlist_add_head_rcu(&nexthop_nh->nh_hash, head); } endfor_nexthops(fi) } @@ -1948,12 +1941,12 @@ void fib_nhc_update_mtu(struct fib_nh_common *nhc, u32 new, u32 orig) void fib_sync_mtu(struct net_device *dev, u32 orig_mtu) { - struct hlist_head *head = fib_info_devhash_bucket(dev); + struct hlist_head *head = fib_nh_head(dev); struct fib_nh *nh; hlist_for_each_entry(nh, head, nh_hash) { - if (nh->fib_nh_dev == dev) - fib_nhc_update_mtu(&nh->nh_common, dev->mtu, orig_mtu); + DEBUG_NET_WARN_ON_ONCE(nh->fib_nh_dev != dev); + fib_nhc_update_mtu(&nh->nh_common, dev->mtu, orig_mtu); } } @@ -1967,7 +1960,7 @@ void fib_sync_mtu(struct net_device *dev, u32 orig_mtu) */ int fib_sync_down_dev(struct net_device *dev, unsigned long event, bool force) { - struct hlist_head *head = fib_info_devhash_bucket(dev); + struct hlist_head *head = fib_nh_head(dev); struct fib_info *prev_fi = NULL; int scope = RT_SCOPE_NOWHERE; struct fib_nh *nh; @@ -1981,7 +1974,8 @@ int fib_sync_down_dev(struct net_device *dev, unsigned long event, bool force) int dead; BUG_ON(!fi->fib_nhs); - if (nh->fib_nh_dev != dev || fi == prev_fi) + DEBUG_NET_WARN_ON_ONCE(nh->fib_nh_dev != dev); + if (fi == prev_fi) continue; prev_fi = fi; dead = 0; @@ -2131,7 +2125,7 @@ int fib_sync_up(struct net_device *dev, unsigned char nh_flags) } prev_fi = NULL; - head = fib_info_devhash_bucket(dev); + head = fib_nh_head(dev); ret = 0; hlist_for_each_entry(nh, head, nh_hash) { @@ -2139,7 +2133,8 @@ int fib_sync_up(struct net_device *dev, unsigned char nh_flags) int alive; BUG_ON(!fi->fib_nhs); - if (nh->fib_nh_dev != dev || fi == prev_fi) + DEBUG_NET_WARN_ON_ONCE(nh->fib_nh_dev != dev); + if (fi == prev_fi) continue; prev_fi = fi;