From patchwork Mon Sep 5 03:38:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12965443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0178ECAAA1 for ; Mon, 5 Sep 2022 03:39:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234698AbiIEDjM (ORCPT ); Sun, 4 Sep 2022 23:39:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbiIEDjK (ORCPT ); Sun, 4 Sep 2022 23:39:10 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A83EF2ED7D for ; Sun, 4 Sep 2022 20:39:09 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id b144so2657264pfb.7 for ; Sun, 04 Sep 2022 20:39:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date; bh=6COL0uv9jVcnHwrCkLTkl2eKFIaAF2AYoaUOPinQbCw=; b=CIyIOtpNS+IsSU8H/N8UqA2zWuyWLg8ivovPZmtoqpWgf45hb1DprECphVjg/KLdEE 198ea74fVkKtLrKYyD/+cFDHLREQjVGND3HVrUBUpNK8uo16Sctmt1U3Ww1KP5GqtY76 cxnd7oLGKxP9CU2fEUxRvHi0Fq9kUt0ZgEclgFvSIbLcY+3heLuYf8wSnLP0rqlplmeD QE8AfSaQSC6y96OuZryl0vB788SLSVrbTJZ6ZoF423bTW+W3PYdVO9+NaS2CuzxSFAqu C+BoRwBiy/6KG06JghYi62BOEkFKs3QNoXMvMGzoQ1AyLs0yUJ2QL7zlchBhR9OXihP6 H+ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date; bh=6COL0uv9jVcnHwrCkLTkl2eKFIaAF2AYoaUOPinQbCw=; b=RaoYiC+nxMFZWcQoW+VEFHyGHiyfe9bVcwjzYtxubY7UnpDq84nC8w6NwLyYfEBISg HlwAlqXEUQM16tWEyQIyv4aGNGKeHLNIPlRzq7d0cF3ZQaO24PwflZHZ04B5KItnCX0m 6Q0Y+f9I1YhSm+g/AY9QT3/bQl8pYqmyQKVXU5Mw68N5XocfpUhiwUEA5Xh8Hx5aJoK8 io54C+A/9NVQarGFBQ0TThs++nbDBYRCRCWnb095E+X0u4LqV/GKN67eCbDKI+/I4TSg QCs5xTOwvUteHePuJrhqtmOlpPPhTs50pWuJwN4wDHrRwejwkueFoKVqSsP7s59/QqZl kBBQ== X-Gm-Message-State: ACgBeo2zGaZsuFyp/k9uEYCT5FKRxPBz0CruSJRfcrf4lJIsdvAjt0Vw 1iwhtkGUVcFOD5OjfOurR3ukXarcKqqjlkw= X-Google-Smtp-Source: AA6agR42MFhwCDd07hV2TrxxYh20m6w7qAggUA+kTH70qvW+lE39nZa/UTqgBj1JriqphwLQyc1OPA== X-Received: by 2002:a63:be0f:0:b0:434:2d63:5036 with SMTP id l15-20020a63be0f000000b004342d635036mr5921383pgf.488.1662349148205; Sun, 04 Sep 2022 20:39:08 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([43.228.180.230]) by smtp.gmail.com with ESMTPSA id o11-20020a62cd0b000000b0052cc561f320sm6403467pfg.54.2022.09.04.20.39.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Sep 2022 20:39:07 -0700 (PDT) From: Pingfan Liu To: rcu@vger.kernel.org Cc: Pingfan Liu , "Paul E. McKenney" , David Woodhouse , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , "Jason A. Donenfeld" Subject: [PATCH 1/3] rcu: remove redundant cpu affinity setting during teardown Date: Mon, 5 Sep 2022 11:38:50 +0800 Message-Id: <20220905033852.18988-1-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org At present, during the cpu teardown, rcu_boost_kthread_setaffinity() is called twice. Firstly by rcutree_offline_cpu(), then by rcutree_dead_cpu() as the CPUHP_RCUTREE_PREP cpuhp_step callback. From the scheduler's perspective, a bit in cpu_online_mask means that the cpu is visible to the scheduler. Furthermore, a bit in cpu_active_mask means that the cpu is suitable as a migration destination. Now turning back to the case in rcu offlining. sched_cpu_deactivate() has disabled the dying cpu as the migration destination before rcutree_offline_cpu(). Furthermore, if the boost kthread is on the dying cpu, it will be migrated to another suitable online cpu by the scheduler. So the affinity setting by rcutree_offline_cpu() is redundant and can be eliminated. Besides, this patch does an trival code rearrangement by unfolding rcutree_affinity_setting() into rcutree_online_cpu(), considering that the latter one is the only user of the former. Signed-off-by: Pingfan Liu Cc: "Paul E. McKenney" Cc: David Woodhouse Cc: Frederic Weisbecker Cc: Neeraj Upadhyay Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers Cc: Lai Jiangshan Cc: Joel Fernandes Cc: "Jason A. Donenfeld" To: rcu@vger.kernel.org --- kernel/rcu/tree.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 79aea7df4345..b90f6487fd45 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3978,16 +3978,6 @@ int rcutree_prepare_cpu(unsigned int cpu) return 0; } -/* - * Update RCU priority boot kthread affinity for CPU-hotplug changes. - */ -static void rcutree_affinity_setting(unsigned int cpu, int outgoing) -{ - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - - rcu_boost_kthread_setaffinity(rdp->mynode, outgoing); -} - /* * Near the end of the CPU-online process. Pretty much all services * enabled, and the CPU is now very much alive. @@ -4006,7 +3996,7 @@ int rcutree_online_cpu(unsigned int cpu) if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE) return 0; /* Too early in boot for scheduler work. */ sync_sched_exp_online_cleanup(cpu); - rcutree_affinity_setting(cpu, -1); + rcu_boost_kthread_setaffinity(rdp->mynode, -1); // Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear(TICK_DEP_BIT_RCU); @@ -4029,8 +4019,6 @@ int rcutree_offline_cpu(unsigned int cpu) rnp->ffmask &= ~rdp->grpmask; raw_spin_unlock_irqrestore_rcu_node(rnp, flags); - rcutree_affinity_setting(cpu, cpu); - // nohz_full CPUs need the tick for stop-machine to work quickly tick_dep_set(TICK_DEP_BIT_RCU); return 0; From patchwork Mon Sep 5 03:38:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12965444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60407ECAAA1 for ; Mon, 5 Sep 2022 03:39:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234831AbiIEDjP (ORCPT ); Sun, 4 Sep 2022 23:39:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbiIEDjO (ORCPT ); Sun, 4 Sep 2022 23:39:14 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1F3B2ED50 for ; Sun, 4 Sep 2022 20:39:12 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id s206so7039221pgs.3 for ; Sun, 04 Sep 2022 20:39:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=rC6QPh/wv70zl3rSlxgqvQT2056BZtJ33zin0YGx0pI=; b=J4qPGfmpIIRZcQWl3gELg0nnnQoKgBkUmTHjIhufnFeoWu2aim27oZJaa/NXKMhlbd o5uLzwp1ayt9LaqTk163d4TM3pn5+o7qiIUhmJcoVopNkHdjGFO96e3/SuuCakriQWKw ss+xnDtdBDwIbsqdegrTQFTyGvS8O03J1WQ2gcssUqk1y9cyvP7gduHYtHrbiKHWNAMA 7Obw362dPp2uJbl8HL3FgsMrwNzqtgJ7JVJhntyVt4G3+Jju/bW9/MjjBokTQnr4OE0z zIeejY51CcXIicGSX6svsZvueZ4+OmuGqrz1vLB1glAl/ehhVMoQMbkDqyRYq/zRghZ1 UnQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=rC6QPh/wv70zl3rSlxgqvQT2056BZtJ33zin0YGx0pI=; b=B81Q0XQ8XXlGt+BI/ZNWKw6+pAr/+pZjc5eva/5diJVwwnbH0hLLJulzMj6Nxzo+XM 1kprpbQMe5EpqFDRl3TYtXIJWM18iTfhIkH3lUza38Ikq3WRZnEi4SELxCiGb+41vYqd IljoXHIZZ+seQCt81T/IQ76JICeMvIJdIJDEgpbk19QrKR3FOt08cT7SMIVyCREVOp3s sce/tjlarivE5odLazntrrhFaWAjGlwj/ibyR36pMSbdFFakvdQLYDIUsWTkGFikkHmb hB+5Fdf3PAd2LnwYDKXi0LEfUlaFzJBEEUAKDktFAAW+MNkIAxdcRjVReVHuhIcQgqsf GU5g== X-Gm-Message-State: ACgBeo033GbuXEbOSGakBAfMuK9GisZCWaO9k9UbzANp9D4Bnelg5SSR 4i7GvM6T/DisFVJ5qW7/gjGDa7ul28naa4A= X-Google-Smtp-Source: AA6agR4BiG7h3iIK8n8L34lUDPp7pOZhTVuBT5J5gSfmF0Z8Gs/aeHevZkCfO6QhlBR2aheb1N4PeQ== X-Received: by 2002:a63:2048:0:b0:41c:daad:450d with SMTP id r8-20020a632048000000b0041cdaad450dmr39258797pgm.240.1662349151966; Sun, 04 Sep 2022 20:39:11 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([43.228.180.230]) by smtp.gmail.com with ESMTPSA id o11-20020a62cd0b000000b0052cc561f320sm6403467pfg.54.2022.09.04.20.39.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Sep 2022 20:39:11 -0700 (PDT) From: Pingfan Liu To: rcu@vger.kernel.org Cc: Pingfan Liu , "Paul E. McKenney" , David Woodhouse , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , "Jason A. Donenfeld" Subject: [PATCH 2/3] rcu: simplify the prototype of rcu_boost_kthread_setaffinity() Date: Mon, 5 Sep 2022 11:38:51 +0800 Message-Id: <20220905033852.18988-2-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220905033852.18988-1-kernelfans@gmail.com> References: <20220905033852.18988-1-kernelfans@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org Previously, the outgoing parameter is needed by rcutree_affinity_setting() since when rcutree_offline_cpu() is called, ->qsmaskinitnext contains the outgoing cpu. Now, it is not necessary and the prototype of rcu_boost_kthread_setaffinity() can be simplified. Signed-off-by: Pingfan Liu Cc: "Paul E. McKenney" Cc: David Woodhouse Cc: Frederic Weisbecker Cc: Neeraj Upadhyay Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers Cc: Lai Jiangshan Cc: Joel Fernandes Cc: "Jason A. Donenfeld" To: rcu@vger.kernel.org --- kernel/rcu/tree.c | 6 +++--- kernel/rcu/tree_plugin.h | 12 +++++------- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index b90f6487fd45..f0535bd210b6 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -145,7 +145,7 @@ static void rcu_report_qs_rnp(unsigned long mask, struct rcu_node *rnp, unsigned long gps, unsigned long flags); static void rcu_init_new_rnp(struct rcu_node *rnp_leaf); static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf); -static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu); +static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp); static void invoke_rcu_core(void); static void rcu_report_exp_rdp(struct rcu_data *rdp); static void sync_sched_exp_online_cleanup(int cpu); @@ -2170,7 +2170,7 @@ int rcutree_dead_cpu(unsigned int cpu) WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); /* Adjust any no-longer-needed kthreads. */ - rcu_boost_kthread_setaffinity(rnp, -1); + rcu_boost_kthread_setaffinity(rnp); // Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear(TICK_DEP_BIT_RCU); return 0; @@ -3996,7 +3996,7 @@ int rcutree_online_cpu(unsigned int cpu) if (rcu_scheduler_active == RCU_SCHEDULER_INACTIVE) return 0; /* Too early in boot for scheduler work. */ sync_sched_exp_online_cleanup(cpu); - rcu_boost_kthread_setaffinity(rdp->mynode, -1); + rcu_boost_kthread_setaffinity(rdp->mynode); // Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear(TICK_DEP_BIT_RCU); diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 438ecae6bd7e..0bf6de185af5 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1217,11 +1217,10 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp) * served by the rcu_node in question. The CPU hotplug lock is still * held, so the value of rnp->qsmaskinit will be stable. * - * We don't include outgoingcpu in the affinity set, use -1 if there is - * no outgoing CPU. If there are no CPUs left in the affinity set, - * this function allows the kthread to execute on any CPU. + * If there are no CPUs left in the affinity set, this function allows the + * kthread to execute on any CPU. */ -static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu) +static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp) { struct task_struct *t = rnp->boost_kthread_task; unsigned long mask = rcu_rnp_online_cpus(rnp); @@ -1234,8 +1233,7 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu) return; mutex_lock(&rnp->boost_kthread_mutex); for_each_leaf_node_possible_cpu(rnp, cpu) - if ((mask & leaf_node_cpu_bit(rnp, cpu)) && - cpu != outgoingcpu) + if ((mask & leaf_node_cpu_bit(rnp, cpu))) cpumask_set_cpu(cpu, cm); cpumask_and(cm, cm, housekeeping_cpumask(HK_TYPE_RCU)); if (cpumask_empty(cm)) @@ -1261,7 +1259,7 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp) { } -static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu) +static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp) { } From patchwork Mon Sep 5 03:38:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12965445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C6ECECAAA1 for ; Mon, 5 Sep 2022 03:39:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235618AbiIEDjT (ORCPT ); Sun, 4 Sep 2022 23:39:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbiIEDjS (ORCPT ); Sun, 4 Sep 2022 23:39:18 -0400 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2F272ED50 for ; Sun, 4 Sep 2022 20:39:16 -0700 (PDT) Received: by mail-pg1-x531.google.com with SMTP id b196so7024094pga.7 for ; Sun, 04 Sep 2022 20:39:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=qxkPzWWZI92LxvOm1fKr9hzMe+kooWrVanvPGMIZF1Y=; b=hRKyV0j0Vpa127A0xdJNV/mPFlFEPGv7yHytbIw8lt3bStgKQ2hH/Sp7EbgkQ+2KMY JKmqU8jJwn8Zw0XBa0c9h3PVmjev2Ztf756o7ew8dVBtX6JvJ3nxvsbM8oIHigCMF1g7 3jfJuS64h8GjrAsX+jDKly6g53O3VtfRtsjXRszX2dS5PNEDa2vznif6vod0L+ttFMYW 3/LZtaWyJC1UehsLY1C9gjuU7biiXBJ+LFhLtYiOX/81TN6nC8nQeT3xDnd+qlFZm9Vl M+1S55OgW1awoxzE9NuAQA2HQSAbG+gRllD8LmJq4larksV2NZvSgpgHhv5I8yBd23c+ 3SNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=qxkPzWWZI92LxvOm1fKr9hzMe+kooWrVanvPGMIZF1Y=; b=a6RFqGRYVphexqoJehw54e/YZYW9TkhappSkpAoGP/ACOWTobA5JDKYPVJXdyedUuF H6pclHZB2ceOolN97jMHQnLnUd9GUGtSOsi8kK0nY+rH7+3KP1Q9ky/6gcVXL6P8Jaix txTlpmhuk+ECoP2kvc8QpUExgvI4A7pvBRO5CYSTQt10Yazpf1jPnc2gTdqm3UXhRCgQ uv+7e73InznpQBRh8r99RtuZ1FO09gP6ld1C9OmGrd18wf3uNsAAdl8MzvwvoUACa78T 1vzW6Qq7dUvFPVfYxLMldp+2pFSjnKUG7h/SGCRu0q9rLYuNWr8qCETD2k5j4/h4ozOE FpKw== X-Gm-Message-State: ACgBeo1+l8U449GPqYR35MMgR2744/GIQwEg2ISknhQ75//BskbbOKL3 UhPX90a2XTm/z3aweaGgwz1L3CdlT001QUo= X-Google-Smtp-Source: AA6agR4LLcd9oIDcxCtr/9bJXEI8SFWV1McZoV8UoI6XCXklpXIkxDjyrq6vnhXNIs9kD+PISpsSjA== X-Received: by 2002:a63:5007:0:b0:42a:29:2780 with SMTP id e7-20020a635007000000b0042a00292780mr39591460pgb.45.1662349155957; Sun, 04 Sep 2022 20:39:15 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([43.228.180.230]) by smtp.gmail.com with ESMTPSA id o11-20020a62cd0b000000b0052cc561f320sm6403467pfg.54.2022.09.04.20.39.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Sep 2022 20:39:15 -0700 (PDT) From: Pingfan Liu To: rcu@vger.kernel.org Cc: Pingfan Liu , "Paul E. McKenney" , David Woodhouse , Frederic Weisbecker , Neeraj Upadhyay , Josh Triplett , Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Joel Fernandes , "Jason A. Donenfeld" Subject: [PATCH 3/3] rcu: Keep qsmaskinitnext fresh for rcu_boost_kthread_setaffinity() Date: Mon, 5 Sep 2022 11:38:52 +0800 Message-Id: <20220905033852.18988-3-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220905033852.18988-1-kernelfans@gmail.com> References: <20220905033852.18988-1-kernelfans@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org rcutree_online_cpu() is concurrent, so there is the following race scene: CPU 1 CPU2 mask_old = rcu_rnp_online_cpus(rnp); ... mask_new = rcu_rnp_online_cpus(rnp); ... set_cpus_allowed_ptr(t, cm); set_cpus_allowed_ptr(t, cm); Consequently, the old mask will overwrite the new mask in the task's cpus_ptr. Since there is a mutex ->boost_kthread_mutex, using it to build an order, then the latest ->qsmaskinitnext will be fetched for updating cpus_ptr. Notes about the cpu teardown: The cpu hot-removing initiator executes rcutree_dead_cpu() for the teardown cpu. If in future, an initiator can hot-remove several cpus at a time, it executes rcutree_dead_cpu() in series for each cpu. So it is still race-free without the mutex. But considering this is a cold path, applying that redundant mutex is harmless. Signed-off-by: Pingfan Liu Cc: "Paul E. McKenney" Cc: David Woodhouse Cc: Frederic Weisbecker Cc: Neeraj Upadhyay Cc: Josh Triplett Cc: Steven Rostedt Cc: Mathieu Desnoyers Cc: Lai Jiangshan Cc: Joel Fernandes Cc: "Jason A. Donenfeld" To: rcu@vger.kernel.org --- kernel/rcu/tree_plugin.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 0bf6de185af5..b868ac6c6ac8 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1223,7 +1223,7 @@ static void rcu_spawn_one_boost_kthread(struct rcu_node *rnp) static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp) { struct task_struct *t = rnp->boost_kthread_task; - unsigned long mask = rcu_rnp_online_cpus(rnp); + unsigned long mask; cpumask_var_t cm; int cpu; @@ -1232,6 +1232,11 @@ static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp) if (!zalloc_cpumask_var(&cm, GFP_KERNEL)) return; mutex_lock(&rnp->boost_kthread_mutex); + /* + * Relying on the lock to serialize, so the latest qsmaskinitnext is for + * cpus_ptr. + */ + mask = rcu_rnp_online_cpus(rnp); for_each_leaf_node_possible_cpu(rnp, cpu) if ((mask & leaf_node_cpu_bit(rnp, cpu))) cpumask_set_cpu(cpu, cm);