From patchwork Fri Feb 10 20:27:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13136405 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FA76C636D4 for ; Fri, 10 Feb 2023 20:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233371AbjBJUpD (ORCPT ); Fri, 10 Feb 2023 15:45:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233506AbjBJUo5 (ORCPT ); Fri, 10 Feb 2023 15:44:57 -0500 Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 996F270714 for ; Fri, 10 Feb 2023 12:44:56 -0800 (PST) Received: by mail-oi1-x234.google.com with SMTP id cz14so5458978oib.12 for ; Fri, 10 Feb 2023 12:44:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8INlq6xo7R2+lXLjUMCK9seRY08jAOSgCaggF49ap3w=; b=550GZBtXSaMVqkuV+x4ZeRfkti/ZaiQI59c5OGJDTxZ8zjOuwbF182k05ZfnE5hZUd MDRZijzTox4DbeUQGPOBv4k+KXf72pB2IkMM96QJJp0mkzoW1YYK4CrqONSgxFsUeBve PG343lZ3LQzm7T7+2pWSEZyy8FZOuqSFvWMHR7LXtLY9KNw++tW2mlAdURdc7Vg80JCF OyoMqj4GE5XipjMDTEngCTGefSLmarYYIFIvW93ueYGb2xgapehobb4uqpSJlYb0gThG uG03rCft54jB6DaCUTPJSyK5xXrq3GbmNbibGebFNSUNYiHI4136bd5Nue6Sev0408H6 Qpuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8INlq6xo7R2+lXLjUMCK9seRY08jAOSgCaggF49ap3w=; b=qzgVI65oqKG5K43A05OmwtnowXnjLQIObi3V1v2rx69lBayg+qWSLWM0TTBOvWXYs2 vFBXYbFXCKVHXStF5+5F2eqefxEYEEPp/UnP9KgGrL3KuYLmsodQ2p3TS+IZaGI27iOX ETIEDSgFMspZptQXtqoD5xi1QsYS4U2rwCnt/8uNMcVH2fElX0IWkEN7tpIRW2RWK281 uYGocbEjScIaa0TEy8m6SkLO0vR623VJxUX1PcOeBh0HPIBwMcqC/JvnyCic24xOcNsZ xLH/euDmf95boXZIPtVVbRswXX2oDkfiv/L+j4FaCnZ8yRAWZkzECvMwSy1UPDealftP 8vXw== X-Gm-Message-State: AO0yUKX4t4oWV8FuQ6p1o5u/WTdx5lB4cK9tgbDcNOGuu4EiQw7cPAgv HFASHZ9gsxBcS0V+hyJicN0KgfdbFHV8V37N X-Google-Smtp-Source: AK7set9UkyFX+JrM3JHyb8Pne0rPsT4YKru4YsnR0Ibbs9mhpJC7rivJ3nx7RH3vc+G6lWFBrMfhGg== X-Received: by 2002:a05:6808:7da:b0:375:45c3:3e8e with SMTP id f26-20020a05680807da00b0037545c33e8emr7118593oij.59.1676061895777; Fri, 10 Feb 2023 12:44:55 -0800 (PST) Received: from localhost.localdomain ([2804:14d:5c5e:4698:2ce0:9122:6880:760c]) by smtp.gmail.com with ESMTPSA id v23-20020a9d5a17000000b0068bc8968753sm2396681oth.17.2023.02.10.12.44.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 12:44:55 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Pedro Tammela Subject: [PATCH net-next 1/3] net/sched: act_nat: transition to percpu stats and rcu Date: Fri, 10 Feb 2023 17:27:24 -0300 Message-Id: <20230210202725.446422-2-pctammela@mojatatu.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210202725.446422-1-pctammela@mojatatu.com> References: <20230210202725.446422-1-pctammela@mojatatu.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The tc action act_nat was using shared stats and taking the per action lock in the datapath. Improve it by using percpu stats and rcu. perf before: - 10.48% tcf_nat_act - 81.83% _raw_spin_lock 81.08% native_queued_spin_lock_slowpath perf after: - 0.48% tcf_nat_act tdc results: 1..27 ok 1 7565 - Add nat action on ingress with default control action ok 2 fd79 - Add nat action on ingress with pipe control action ok 3 eab9 - Add nat action on ingress with continue control action ok 4 c53a - Add nat action on ingress with reclassify control action ok 5 76c9 - Add nat action on ingress with jump control action ok 6 24c6 - Add nat action on ingress with drop control action ok 7 2120 - Add nat action on ingress with maximum index value ok 8 3e9d - Add nat action on ingress with invalid index value ok 9 f6c9 - Add nat action on ingress with invalid IP address ok 10 be25 - Add nat action on ingress with invalid argument ok 11 a7bd - Add nat action on ingress with DEFAULT IP address ok 12 ee1e - Add nat action on ingress with ANY IP address ok 13 1de8 - Add nat action on ingress with ALL IP address ok 14 8dba - Add nat action on egress with default control action ok 15 19a7 - Add nat action on egress with pipe control action ok 16 f1d9 - Add nat action on egress with continue control action ok 17 6d4a - Add nat action on egress with reclassify control action ok 18 b313 - Add nat action on egress with jump control action ok 19 d9fc - Add nat action on egress with drop control action ok 20 a895 - Add nat action on egress with DEFAULT IP address ok 21 2572 - Add nat action on egress with ANY IP address ok 22 37f3 - Add nat action on egress with ALL IP address ok 23 6054 - Add nat action on egress with cookie ok 24 79d6 - Add nat action on ingress with cookie ok 25 4b12 - Replace nat action with invalid goto chain control ok 26 b811 - Delete nat action with valid index ok 27 a521 - Delete nat action with invalid index Reviewed-by: Jamal Hadi Salim Signed-off-by: Pedro Tammela --- include/net/tc_act/tc_nat.h | 10 ++++-- net/sched/act_nat.c | 68 ++++++++++++++++++++++++++----------- 2 files changed, 55 insertions(+), 23 deletions(-) diff --git a/include/net/tc_act/tc_nat.h b/include/net/tc_act/tc_nat.h index c14407160..c869274ac 100644 --- a/include/net/tc_act/tc_nat.h +++ b/include/net/tc_act/tc_nat.h @@ -5,13 +5,17 @@ #include #include -struct tcf_nat { - struct tc_action common; - +struct tcf_nat_parms { __be32 old_addr; __be32 new_addr; __be32 mask; u32 flags; + struct rcu_head rcu; +}; + +struct tcf_nat { + struct tc_action common; + struct tcf_nat_parms __rcu *parms; }; #define to_tcf_nat(a) ((struct tcf_nat *)a) diff --git a/net/sched/act_nat.c b/net/sched/act_nat.c index 74c74be33..fb986d97c 100644 --- a/net/sched/act_nat.c +++ b/net/sched/act_nat.c @@ -40,6 +40,7 @@ static int tcf_nat_init(struct net *net, struct nlattr *nla, struct nlattr *est, bool bind = flags & TCA_ACT_FLAGS_BIND; struct nlattr *tb[TCA_NAT_MAX + 1]; struct tcf_chain *goto_ch = NULL; + struct tcf_nat_parms *nparm, *oparm; struct tc_nat *parm; int ret = 0, err; struct tcf_nat *p; @@ -59,8 +60,8 @@ static int tcf_nat_init(struct net *net, struct nlattr *nla, struct nlattr *est, index = parm->index; err = tcf_idr_check_alloc(tn, &index, a, bind); if (!err) { - ret = tcf_idr_create(tn, index, est, a, - &act_nat_ops, bind, false, flags); + ret = tcf_idr_create_from_flags(tn, index, est, a, &act_nat_ops, + bind, flags); if (ret) { tcf_idr_cleanup(tn, index); return ret; @@ -79,19 +80,31 @@ static int tcf_nat_init(struct net *net, struct nlattr *nla, struct nlattr *est, err = tcf_action_check_ctrlact(parm->action, tp, &goto_ch, extack); if (err < 0) goto release_idr; + + nparm = kzalloc(sizeof(*nparm), GFP_KERNEL); + if (!nparm) { + err = -ENOMEM; + goto release_idr; + } + + nparm->old_addr = parm->old_addr; + nparm->new_addr = parm->new_addr; + nparm->mask = parm->mask; + nparm->flags = parm->flags; + p = to_tcf_nat(*a); spin_lock_bh(&p->tcf_lock); - p->old_addr = parm->old_addr; - p->new_addr = parm->new_addr; - p->mask = parm->mask; - p->flags = parm->flags; - goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch); + oparm = rcu_replace_pointer(p->parms, nparm, lockdep_is_held(&p->tcf_lock)); spin_unlock_bh(&p->tcf_lock); + if (goto_ch) tcf_chain_put_by_act(goto_ch); + if (oparm) + kfree_rcu(oparm, rcu); + return ret; release_idr: tcf_idr_release(*a, bind); @@ -103,6 +116,7 @@ TC_INDIRECT_SCOPE int tcf_nat_act(struct sk_buff *skb, struct tcf_result *res) { struct tcf_nat *p = to_tcf_nat(a); + struct tcf_nat_parms *parms; struct iphdr *iph; __be32 old_addr; __be32 new_addr; @@ -113,18 +127,16 @@ TC_INDIRECT_SCOPE int tcf_nat_act(struct sk_buff *skb, int ihl; int noff; - spin_lock(&p->tcf_lock); - tcf_lastuse_update(&p->tcf_tm); - old_addr = p->old_addr; - new_addr = p->new_addr; - mask = p->mask; - egress = p->flags & TCA_NAT_FLAG_EGRESS; - action = p->tcf_action; + tcf_action_update_bstats(&p->common, skb); - bstats_update(&p->tcf_bstats, skb); + action = READ_ONCE(p->tcf_action); - spin_unlock(&p->tcf_lock); + parms = rcu_dereference_bh(p->parms); + old_addr = parms->old_addr; + new_addr = parms->new_addr; + mask = parms->mask; + egress = parms->flags & TCA_NAT_FLAG_EGRESS; if (unlikely(action == TC_ACT_SHOT)) goto drop; @@ -264,15 +276,20 @@ static int tcf_nat_dump(struct sk_buff *skb, struct tc_action *a, .refcnt = refcount_read(&p->tcf_refcnt) - ref, .bindcnt = atomic_read(&p->tcf_bindcnt) - bind, }; + struct tcf_nat_parms *parms; struct tcf_t t; spin_lock_bh(&p->tcf_lock); - opt.old_addr = p->old_addr; - opt.new_addr = p->new_addr; - opt.mask = p->mask; - opt.flags = p->flags; + opt.action = p->tcf_action; + parms = rcu_dereference_protected(p->parms, lockdep_is_held(&p->tcf_lock)); + + opt.old_addr = parms->old_addr; + opt.new_addr = parms->new_addr; + opt.mask = parms->mask; + opt.flags = parms->flags; + if (nla_put(skb, TCA_NAT_PARMS, sizeof(opt), &opt)) goto nla_put_failure; @@ -289,6 +306,16 @@ static int tcf_nat_dump(struct sk_buff *skb, struct tc_action *a, return -1; } +static void tcf_nat_cleanup(struct tc_action *a) +{ + struct tcf_nat_parms *parms; + struct tcf_nat *p = to_tcf_nat(a); + + parms = rcu_dereference_protected(p->parms, 1); + if (parms) + kfree_rcu(parms, rcu); +} + static struct tc_action_ops act_nat_ops = { .kind = "nat", .id = TCA_ID_NAT, @@ -296,6 +323,7 @@ static struct tc_action_ops act_nat_ops = { .act = tcf_nat_act, .dump = tcf_nat_dump, .init = tcf_nat_init, + .cleanup = tcf_nat_cleanup, .size = sizeof(struct tcf_nat), }; From patchwork Fri Feb 10 20:27:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13136406 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 017FBC636D7 for ; Fri, 10 Feb 2023 20:45:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233394AbjBJUpF (ORCPT ); Fri, 10 Feb 2023 15:45:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233250AbjBJUpD (ORCPT ); Fri, 10 Feb 2023 15:45:03 -0500 Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C65E070714 for ; Fri, 10 Feb 2023 12:45:01 -0800 (PST) Received: by mail-ot1-x334.google.com with SMTP id 38-20020a9d04a9000000b0068f24f576c5so1629506otm.11 for ; Fri, 10 Feb 2023 12:45:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L2OELBHVTmIPYbUYyMRK6qAq6fpi8NqW2WegJ/4prVM=; b=i9yDukMRWXAmwBREZj7/JGtotgTXJieQjOyRdA3v1yCQ+cQONzuXNmdIBW8/JUqicP p4Svi5/Ld7Q4WfjyFG3KI97byBSDKu1KSiE9Nt7oII7n7iZ4P7wlKu6SCVHgmOQRIPJy uUdFXH1pAeRcUT25cRzwWe223QUKHcX5YgTob6QBvf2UdyPB13oBs84qRfBMMAln58RW G3ZcWpKjcMIc9yKF2QTJB7HS8apQoc2dqrR+80rPtK5jiEd3URiwZxm+447AOGfGaLbn 7dwZAKhGKbOeExmKNoPozLZ4SjUe9YsiLhjRQqJ/MUD9JNdJMLzZR5g7J0N0KSfJ5EnM 2IEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L2OELBHVTmIPYbUYyMRK6qAq6fpi8NqW2WegJ/4prVM=; b=DDTIq+QmLfMbpPTQd9h1fYFSUXryORL+R2UphPlUVRKhonaHXGhKUJb0NpywZn6pzy 1E5mzf0vhn3q6LQUIsoqh0HF6Btebh7Dh/A0nlwDwou5o2q8njKkAkf1CPfChCREKJJt jnXFG/muilzusmfacUxTw0k/rkYMVmgpzXILTCQEfAolzQtahCsvWpngAUS7S+T2HVLb Pf3sN3/Or0M3yszPbXfNLcOPzWDWXCDMkQT2/C01KkNLfAKtYsqnmcJXYTYKNjDbXqkU tHQJcCDCkjAbt2Fe+wB8CGy/hSUecwsp5MSiEwVDPs30ZVEZUXHkEUrbnxJnK3T/LogI YAPQ== X-Gm-Message-State: AO0yUKXBLm0sj/iZPKb8dQ6nNJ+lb1gHiJiiwhgzhvuPsOYBTtEUUIZy S672vMpLfC2h5ERLrveJAJYJFyJNAC6nnmvm X-Google-Smtp-Source: AK7set8NTx18TDPv2eGWkM7kDJBQIk+m4w+1Eqq/HaY8e7i6G+wSYTRol6X9WdMXayimRkZUyidJtA== X-Received: by 2002:a9d:4549:0:b0:68d:71d0:ed13 with SMTP id p9-20020a9d4549000000b0068d71d0ed13mr9444094oti.1.1676061901044; Fri, 10 Feb 2023 12:45:01 -0800 (PST) Received: from localhost.localdomain ([2804:14d:5c5e:4698:2ce0:9122:6880:760c]) by smtp.gmail.com with ESMTPSA id v23-20020a9d5a17000000b0068bc8968753sm2396681oth.17.2023.02.10.12.44.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 12:45:00 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Pedro Tammela Subject: [PATCH net-next 2/3] net/sched: act_connmark: transition to percpu stats and rcu Date: Fri, 10 Feb 2023 17:27:25 -0300 Message-Id: <20230210202725.446422-3-pctammela@mojatatu.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210202725.446422-1-pctammela@mojatatu.com> References: <20230210202725.446422-1-pctammela@mojatatu.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The tc action act_connmark was using shared stats and taking the per action lock in the datapath. Improve it by using percpu stats and rcu. perf before: - 13.55% tcf_connmark_act - 81.18% _raw_spin_lock 80.46% native_queued_spin_lock_slowpath perf after: - 3.12% tcf_connmark_act tdc results: 1..15 ok 1 2002 - Add valid connmark action with defaults ok 2 56a5 - Add valid connmark action with control pass ok 3 7c66 - Add valid connmark action with control drop ok 4 a913 - Add valid connmark action with control pipe ok 5 bdd8 - Add valid connmark action with control reclassify ok 6 b8be - Add valid connmark action with control continue ok 7 d8a6 - Add valid connmark action with control jump ok 8 aae8 - Add valid connmark action with zone argument ok 9 2f0b - Add valid connmark action with invalid zone argument ok 10 9305 - Add connmark action with unsupported argument ok 11 71ca - Add valid connmark action and replace it ok 12 5f8f - Add valid connmark action with cookie ok 13 c506 - Replace connmark with invalid goto chain control ok 14 6571 - Delete connmark action with valid index ok 15 3426 - Delete connmark action with invalid index Reviewed-by: Jamal Hadi Salim Signed-off-by: Pedro Tammela --- include/net/tc_act/tc_connmark.h | 9 ++- net/sched/act_connmark.c | 109 ++++++++++++++++++++----------- 2 files changed, 77 insertions(+), 41 deletions(-) diff --git a/include/net/tc_act/tc_connmark.h b/include/net/tc_act/tc_connmark.h index 1f4cb477b..e8dd77a96 100644 --- a/include/net/tc_act/tc_connmark.h +++ b/include/net/tc_act/tc_connmark.h @@ -4,10 +4,15 @@ #include -struct tcf_connmark_info { - struct tc_action common; +struct tcf_connmark_parms { struct net *net; u16 zone; + struct rcu_head rcu; +}; + +struct tcf_connmark_info { + struct tc_action common; + struct tcf_connmark_parms __rcu *parms; }; #define to_connmark(a) ((struct tcf_connmark_info *)a) diff --git a/net/sched/act_connmark.c b/net/sched/act_connmark.c index 7e63ff7e3..541e1c556 100644 --- a/net/sched/act_connmark.c +++ b/net/sched/act_connmark.c @@ -36,13 +36,15 @@ TC_INDIRECT_SCOPE int tcf_connmark_act(struct sk_buff *skb, struct nf_conntrack_tuple tuple; enum ip_conntrack_info ctinfo; struct tcf_connmark_info *ca = to_connmark(a); + struct tcf_connmark_parms *parms; struct nf_conntrack_zone zone; struct nf_conn *c; int proto; - spin_lock(&ca->tcf_lock); tcf_lastuse_update(&ca->tcf_tm); - bstats_update(&ca->tcf_bstats, skb); + tcf_action_update_bstats(&ca->common, skb); + + parms = rcu_dereference_bh(ca->parms); switch (skb_protocol(skb, true)) { case htons(ETH_P_IP): @@ -64,31 +66,31 @@ TC_INDIRECT_SCOPE int tcf_connmark_act(struct sk_buff *skb, c = nf_ct_get(skb, &ctinfo); if (c) { skb->mark = READ_ONCE(c->mark); - /* using overlimits stats to count how many packets marked */ - ca->tcf_qstats.overlimits++; - goto out; + goto count; } - if (!nf_ct_get_tuplepr(skb, skb_network_offset(skb), - proto, ca->net, &tuple)) + if (!nf_ct_get_tuplepr(skb, skb_network_offset(skb), proto, parms->net, + &tuple)) goto out; - zone.id = ca->zone; + zone.id = parms->zone; zone.dir = NF_CT_DEFAULT_ZONE_DIR; - thash = nf_conntrack_find_get(ca->net, &zone, &tuple); + thash = nf_conntrack_find_get(parms->net, &zone, &tuple); if (!thash) goto out; c = nf_ct_tuplehash_to_ctrack(thash); - /* using overlimits stats to count how many packets marked */ - ca->tcf_qstats.overlimits++; skb->mark = READ_ONCE(c->mark); nf_ct_put(c); -out: +count: + /* using overlimits stats to count how many packets marked */ + spin_lock(&ca->tcf_lock); + ca->tcf_qstats.overlimits++; spin_unlock(&ca->tcf_lock); - return ca->tcf_action; +out: + return READ_ONCE(ca->tcf_action); } static const struct nla_policy connmark_policy[TCA_CONNMARK_MAX + 1] = { @@ -104,6 +106,7 @@ static int tcf_connmark_init(struct net *net, struct nlattr *nla, struct nlattr *tb[TCA_CONNMARK_MAX + 1]; bool bind = flags & TCA_ACT_FLAGS_BIND; struct tcf_chain *goto_ch = NULL; + struct tcf_connmark_parms *nparms, *oparms; struct tcf_connmark_info *ci; struct tc_connmark *parm; int ret = 0, err; @@ -120,52 +123,66 @@ static int tcf_connmark_init(struct net *net, struct nlattr *nla, if (!tb[TCA_CONNMARK_PARMS]) return -EINVAL; + nparms = kzalloc(sizeof(*nparms), GFP_KERNEL); + if (!nparms) + return -ENOMEM; + parm = nla_data(tb[TCA_CONNMARK_PARMS]); index = parm->index; ret = tcf_idr_check_alloc(tn, &index, a, bind); if (!ret) { - ret = tcf_idr_create(tn, index, est, a, - &act_connmark_ops, bind, false, flags); + ret = tcf_idr_create_from_flags(tn, index, est, a, + &act_connmark_ops, bind, flags); if (ret) { tcf_idr_cleanup(tn, index); - return ret; + err = ret; + goto out_free; } ci = to_connmark(*a); - err = tcf_action_check_ctrlact(parm->action, tp, &goto_ch, - extack); - if (err < 0) - goto release_idr; - tcf_action_set_ctrlact(*a, parm->action, goto_ch); - ci->net = net; - ci->zone = parm->zone; + + nparms->net = net; + nparms->zone = parm->zone; ret = ACT_P_CREATED; } else if (ret > 0) { ci = to_connmark(*a); - if (bind) - return 0; - if (!(flags & TCA_ACT_FLAGS_REPLACE)) { - tcf_idr_release(*a, bind); - return -EEXIST; + if (bind) { + err = 0; + goto out_free; } - err = tcf_action_check_ctrlact(parm->action, tp, &goto_ch, - extack); - if (err < 0) + if (!(flags & TCA_ACT_FLAGS_REPLACE)) { + err = -EEXIST; goto release_idr; - /* replacing action and zone */ - spin_lock_bh(&ci->tcf_lock); - goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch); - ci->zone = parm->zone; - spin_unlock_bh(&ci->tcf_lock); - if (goto_ch) - tcf_chain_put_by_act(goto_ch); + } + + nparms->net = rtnl_dereference(ci->parms)->net; + nparms->zone = parm->zone; + ret = 0; } + err = tcf_action_check_ctrlact(parm->action, tp, &goto_ch, extack); + if (err < 0) + goto release_idr; + + spin_lock_bh(&ci->tcf_lock); + goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch); + oparms = rcu_replace_pointer(ci->parms, nparms, lockdep_is_held(&ci->tcf_lock)); + spin_unlock_bh(&ci->tcf_lock); + + if (goto_ch) + tcf_chain_put_by_act(goto_ch); + + if (oparms) + kfree_rcu(oparms, rcu); + return ret; + release_idr: tcf_idr_release(*a, bind); +out_free: + kfree(nparms); return err; } @@ -179,11 +196,14 @@ static inline int tcf_connmark_dump(struct sk_buff *skb, struct tc_action *a, .refcnt = refcount_read(&ci->tcf_refcnt) - ref, .bindcnt = atomic_read(&ci->tcf_bindcnt) - bind, }; + struct tcf_connmark_parms *parms; struct tcf_t t; spin_lock_bh(&ci->tcf_lock); + parms = rcu_dereference_protected(ci->parms, lockdep_is_held(&ci->tcf_lock)); + opt.action = ci->tcf_action; - opt.zone = ci->zone; + opt.zone = parms->zone; if (nla_put(skb, TCA_CONNMARK_PARMS, sizeof(opt), &opt)) goto nla_put_failure; @@ -201,6 +221,16 @@ static inline int tcf_connmark_dump(struct sk_buff *skb, struct tc_action *a, return -1; } +static void tcf_connmark_cleanup(struct tc_action *a) +{ + struct tcf_connmark_parms *parms; + struct tcf_connmark_info *ci = to_connmark(a); + + parms = rcu_dereference_protected(ci->parms, 1); + if (parms) + kfree_rcu(parms, rcu); +} + static struct tc_action_ops act_connmark_ops = { .kind = "connmark", .id = TCA_ID_CONNMARK, @@ -208,6 +238,7 @@ static struct tc_action_ops act_connmark_ops = { .act = tcf_connmark_act, .dump = tcf_connmark_dump, .init = tcf_connmark_init, + .cleanup = tcf_connmark_cleanup, .size = sizeof(struct tcf_connmark_info), }; From patchwork Fri Feb 10 20:27:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pedro Tammela X-Patchwork-Id: 13136407 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E39EC05027 for ; Fri, 10 Feb 2023 20:45:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233425AbjBJUpL (ORCPT ); Fri, 10 Feb 2023 15:45:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233455AbjBJUpG (ORCPT ); Fri, 10 Feb 2023 15:45:06 -0500 Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 753BE7BFF4 for ; Fri, 10 Feb 2023 12:45:05 -0800 (PST) Received: by mail-ot1-x32c.google.com with SMTP id j6-20020a9d7686000000b0068d4ba9d141so1913018otl.6 for ; Fri, 10 Feb 2023 12:45:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mojatatu-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=klTTMIfmdsm2ecsXNiJzppXWn6h+94D9gmx9QXWwnPw=; b=o8HCLPjj5QrMmIfV9mks9EmiaiWj4xQdqkMJLpr7ks4YRJTJ1hdIWNBp2uJqRa2l0x x0y9f8LEZGD0bRlDdln/WHtjE0SZnsXroT6as+g/2zQICh7wTboOBUeC+Thzx5UP2Y0+ BKBIgHEYo9M5cXRieXQUL1k/r5v8VUgoO2mD5u+7qDGLgA8CusahzZQAfKZs9EMqHCIM Do2rw8ALGGlnFSPLoTu9x7nHKKbsLfZ2l1O6IZA3gXZcOI7r9aKHmXbSfaUxXeirzobH 35GbfVtu8Du3Fz5VOG8xLRDDLPY5w8l24HOvw6kLp1E+RPOyUOarH05xko2ha5WatGag AaMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=klTTMIfmdsm2ecsXNiJzppXWn6h+94D9gmx9QXWwnPw=; b=aaLrMakolmCK1iHtvvHATrY/5mIW7BdBHj7UWyr62YcXMqYfNjCEoKmCM9X+5Tj77x uDZwIImWDZ0EdSlUoNV4bOumNg1rn/iw8GACCzXg/YX7pKnlTRDE6AvMRmtzNxesbtTw 8ETpXaptZ5RY1PMoKVSWrJq/yHLi1C47nUe5Z/RVPxci+Ttg3ULCU8Sqf40gyGbpNtMm l6z7aJiXB1OX4BBdxU0tz8jT0IyZRmA/MFkwAj8XpFAsWPr7FMLORwRN1Z7ezeA5nw2Y K4uEAa/JfLcnB4ii5kdDVdJv5XwNgJIlHaP0/YK0umQUZ0BAw7CrpCORTtAjnh4tvCwA RHOg== X-Gm-Message-State: AO0yUKX3uNkuUSlpx4KWqEpC6w3VVjZVISP05cecj6uSlnGzgK0KHyzU fEBLsHvQ7N2R9kUIN5nCgys4Dw5uEPXFamKT X-Google-Smtp-Source: AK7set9ue7VHnN8mn7PTYKj4E0NIaMUgKSpC9JzAHhsOsyYvgS5P4N0GsfdMQJ9ofUKzNVNZ3AyHcQ== X-Received: by 2002:a9d:6e96:0:b0:671:e5c1:219c with SMTP id a22-20020a9d6e96000000b00671e5c1219cmr9274851otr.21.1676061904702; Fri, 10 Feb 2023 12:45:04 -0800 (PST) Received: from localhost.localdomain ([2804:14d:5c5e:4698:2ce0:9122:6880:760c]) by smtp.gmail.com with ESMTPSA id v23-20020a9d5a17000000b0068bc8968753sm2396681oth.17.2023.02.10.12.45.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 12:45:04 -0800 (PST) From: Pedro Tammela To: netdev@vger.kernel.org Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, Pedro Tammela Subject: [PATCH net-next 3/3] net/sched: act_gate: use percpu stats Date: Fri, 10 Feb 2023 17:27:26 -0300 Message-Id: <20230210202725.446422-4-pctammela@mojatatu.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210202725.446422-1-pctammela@mojatatu.com> References: <20230210202725.446422-1-pctammela@mojatatu.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The tc action act_gate was using shared stats, move it to percpu stats. tdc results: 1..12 ok 1 5153 - Add gate action with priority and sched-entry ok 2 7189 - Add gate action with base-time ok 3 a721 - Add gate action with cycle-time ok 4 c029 - Add gate action with cycle-time-ext ok 5 3719 - Replace gate base-time action ok 6 d821 - Delete gate action with valid index ok 7 3128 - Delete gate action with invalid index ok 8 7837 - List gate actions ok 9 9273 - Flush gate actions ok 10 c829 - Add gate action with duplicate index ok 11 3043 - Add gate action with invalid index ok 12 2930 - Add gate action with cookie Reviewed-by: Jamal Hadi Salim Signed-off-by: Pedro Tammela --- net/sched/act_gate.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/net/sched/act_gate.c b/net/sched/act_gate.c index 9b8def0be..684b7a79f 100644 --- a/net/sched/act_gate.c +++ b/net/sched/act_gate.c @@ -120,10 +120,10 @@ TC_INDIRECT_SCOPE int tcf_gate_act(struct sk_buff *skb, { struct tcf_gate *gact = to_gate(a); - spin_lock(&gact->tcf_lock); - tcf_lastuse_update(&gact->tcf_tm); - bstats_update(&gact->tcf_bstats, skb); + tcf_action_update_bstats(&gact->common, skb); + + spin_lock(&gact->tcf_lock); if (unlikely(gact->current_gate_status & GATE_ACT_PENDING)) { spin_unlock(&gact->tcf_lock); @@ -357,8 +357,8 @@ static int tcf_gate_init(struct net *net, struct nlattr *nla, return 0; if (!err) { - ret = tcf_idr_create(tn, index, est, a, - &act_gate_ops, bind, false, flags); + ret = tcf_idr_create_from_flags(tn, index, est, a, + &act_gate_ops, bind, flags); if (ret) { tcf_idr_cleanup(tn, index); return ret;