From patchwork Thu Jan 30 18:49:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954851 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9A49179BC; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262967; cv=none; b=C3CiRgfxuGmOTo5zW5h+SqmHzU5OJjTNKY6TIxuYR8xXAbRmL9TuFlqOO/oyHHPSX2M5+Mm/4pMNTkaBHud62UwC4OpbuR0Q2qBrfcWFUPFxBi173PqlCDsFgylq4lK5Ix1GIAsmc0m5bb8YpmCJOVpLJh9+Scar75DNJ5Ogfzg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262967; c=relaxed/simple; bh=AWn5SFxUzI8JsIufek7SpD9CBh9ZRnNH+qA6EOcFSqo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rPoo8wWOQvddHMpY+nG5AiPjGAn8bq79jkwh1S2NPo7MNS178FMN4Mx81hti8HUiKVCnAFxx0pDGv7NDsvFrXGXAmhtlYoF8+vg17Z/hkxHi4OxjtYNuwIwUE9hLDaYTWyaHFZVqN/azfV3oDUG3H0qGfoGPhPOahIq/tjQbbpE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UmBoRQYU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UmBoRQYU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7D75CC4CED2; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262967; bh=AWn5SFxUzI8JsIufek7SpD9CBh9ZRnNH+qA6EOcFSqo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UmBoRQYUylOkaYKgPlDbmdgRXtkkRqXsNpISCQWLiGoCZ7Yw5KJviPWEH2DrnGa4J eqGjGQ3myKMvr3X91jvmVBqQfQWNvz2VBuFFCNQJ151DW2TQyalyae9rZu2qEPXjvW z0S2amaYE1QjLUPGQtVoXp/hUfYieN/P4gl9Zxr9GK7w01IO05xoNj3vosBkE303Ib nulYKx7/TSd98b2I53jL2sgNFPGCnGL+TOmu7HAeXuN4yr39g9OYr50XdK8Y2iUTy2 bxQFoFNQdxoY7BS/yEL5QcUUTUqfIEQLY/bliyAv+CeXm7iYKJXkqKdFWaD5uZr4bW VvKzSe3BRIo4w== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 29482CE37D9; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu v2 1/7] doc: Add broken-timing possibility to stallwarn.rst Date: Thu, 30 Jan 2025 10:49:19 -0800 Message-Id: <20250130184925.1651665-1-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Currently, stallwarn.rst does not mention the fact that timer bugs can result in false-positive RCU CPU stall warnings. This commit therefore adds this to the list. Signed-off-by: Paul E. McKenney --- Documentation/RCU/stallwarn.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/RCU/stallwarn.rst b/Documentation/RCU/stallwarn.rst index 30080ff6f406..d1ccd6039a8c 100644 --- a/Documentation/RCU/stallwarn.rst +++ b/Documentation/RCU/stallwarn.rst @@ -96,6 +96,13 @@ warnings: the ``rcu_.*timer wakeup didn't happen for`` console-log message, which will include additional debugging information. +- A timer issue causes time to appear to jump forward, so that RCU + believes that the RCU CPU stall-warning timeout has been exceeded + when in fact much less time has passed. This could be due to + timer hardware bugs, timer driver bugs, or even corruption of + the "jiffies" global variable. These sorts of timer hardware + and driver bugs are not uncommon when testing new hardware. + - A low-level kernel issue that either fails to invoke one of the variants of rcu_eqs_enter(true), rcu_eqs_exit(true), ct_idle_enter(), ct_idle_exit(), ct_irq_enter(), or ct_irq_exit() on the one From patchwork Thu Jan 30 18:49:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954855 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 154F81EC00D; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; cv=none; b=g5uvij5lRzKWoWUB0ysAeCuzQ+wvOgnrEei+4kJ4AIADlCQM2OPM/jlSxgpYEH7br0l0NYB7ifS65kkyUp+gGP4edpmbS3c36wGG7+4lCw8Sb+je9rX2zl1JhJ3LnZVtCjVg4APZkZLqaNIhWSmZyTZVBfnq5v4H+Z5lo/Sk9SI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; c=relaxed/simple; bh=CQAifkc0lIqDeOEOsjSPO+PQBOplNs56QQpJryBbrLU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NZk7moKxV6QMzc+r46mSUOO3aCdGVkYDgePK5gZbdzE4WBFBiI+HMYQqIILp5t2/gVZsltyf5mYdrFXhkEsRXrukXfQJnj3jOlaTs22qBFS2zVdPwDx55ydZQPup0mkhSQHnnGXgbAfOom9c2I4ZkLMj9kXWlusqbLQ2GgrSV1s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nbMpymJh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nbMpymJh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 879D9C4CED3; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262967; bh=CQAifkc0lIqDeOEOsjSPO+PQBOplNs56QQpJryBbrLU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nbMpymJhYCgyrCRMgJvQYua/kjgi52QCjdX3cJIC4uBSSAcK06dpRwLzxGjJboibf 97Ukll5QifFjMk6oj+ssrXONqShmnpanscEyME2QjQfvujAlrpABU/CDqTkWKutdV9 5dJKFjqKZfxD2oFzsD1x5qlDyreEPiwYpu+5ozZT2OPycpvMory+KQe7USRqkFJtbA AhDWOmbBfeJ9Ml2bIwSDcnoTy75a54rAIWzIkaXICOMhGVC5UnT1U20keYIPTT2scw I6SOWIXGX+Pq1rYTvGpt69nrgbHINJNEcBkKqYVU41LKoTxmzZG0bTvwphkBvN2c3a Djyu1PiTG7q3w== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 2B129CE37DA; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Jonathan Corbet , Peter Zijlstra , Mark Rutland , Boqun Feng , linux-doc@vger.kernel.org Subject: [PATCH rcu v2 2/7] docs: Improve discussion of this_cpu_ptr(), add raw_cpu_ptr() Date: Thu, 30 Jan 2025 10:49:20 -0800 Message-Id: <20250130184925.1651665-2-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Most of the this_cpu_*() operations may be used in preemptible code, but not this_cpu_ptr(), and for good reasons. Therefore, better explain the reasons and call out raw_cpu_ptr() as an alternative in certain very special cases. Signed-off-by: Paul E. McKenney Cc: Jonathan Corbet Cc: Peter Zijlstra Cc: Mark Rutland Cc: Boqun Feng Cc: --- Documentation/core-api/this_cpu_ops.rst | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/Documentation/core-api/this_cpu_ops.rst b/Documentation/core-api/this_cpu_ops.rst index 91acbcf30e9b..533ac5dd5750 100644 --- a/Documentation/core-api/this_cpu_ops.rst +++ b/Documentation/core-api/this_cpu_ops.rst @@ -138,12 +138,22 @@ get_cpu/put_cpu sequence requires. No processor number is available. Instead, the offset of the local per cpu area is simply added to the per cpu offset. -Note that this operation is usually used in a code segment when -preemption has been disabled. The pointer is then used to -access local per cpu data in a critical section. When preemption -is re-enabled this pointer is usually no longer useful since it may -no longer point to per cpu data of the current processor. - +Note that this operation can only be used in code segments where +smp_processor_id() may be used, for example, where preemption has been +disabled. The pointer is then used to access local per cpu data in a +critical section. When preemption is re-enabled this pointer is usually +no longer useful since it may no longer point to per cpu data of the +current processor. + +The special cases where it makes sense to obtain a per-CPU pointer in +preemptible code are addressed by raw_cpu_ptr(), but such use cases need +to handle cases where two different CPUs are accessing the same per cpu +variable, which might well be that of a third CPU. These use cases are +typically performance optimizations. For example, SRCU implements a pair +of counters as a pair of per-CPU variables, and rcu_read_lock_nmisafe() +uses raw_cpu_ptr() to get a pointer to some CPU's counter, and uses +atomic_inc_long() to handle migration between the raw_cpu_ptr() and +the atomic_inc_long(). Per cpu variables and offsets ----------------------------- From patchwork Thu Jan 30 18:49:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954853 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1543A1BEF84; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; cv=none; b=spC5t8iiOXWxT0Fj36NuPpCPd3itbNF4vJGKEJ9V/idR6wU1b+5SWgj+Kdi8w9nizYCY9nzZuPRPFFotLeOKMjlDud20aMPlmMVJXR3qRwVz1Zp+kzmZYniD15c97/wEb4CehLmISXUnPN0dpmfSkfshJzY/h70bGP2PcWuhCY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; c=relaxed/simple; bh=hppjbdGwAgZh5PQvuEjTAlQFTjFVaIiFFrVhDyXxF0Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EsARO1t8nckDaRH2qtDQGzmkdbQbr2IL6AU29RHFlg0ilaPodarp4gFqJC9qI9mOxM6SSiCmwAN5XNd0qdXNwE5s77geGSanUrYeVK+EP9Eele9wCw2bYOBjlu/oygIuAxKZ+qPlKncxF9mLJyZkxrItK1X8p1VcGh/Lgz1pP3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DDg8V2FQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DDg8V2FQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9802FC4CEE1; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262967; bh=hppjbdGwAgZh5PQvuEjTAlQFTjFVaIiFFrVhDyXxF0Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DDg8V2FQLussyptYSfVR/V73YPFEE7HkQGCsNzFNPiAcW44Ze0Xo1LLTNxLdFtbhY YUkSpEMoXl/YK177X08sJG+4ZmunDooTFpYamnunaZ/5jc+MVoOusLO/mFFBuJSKfN qmJd+hKmuaiT9VJiYPR1j9Uxn+S+47/3SpdhTTqhXO3l11otSdU9/9OeB/c+Hhuxjo i0XfHFpeiv0bjPX8ZunAK6VvEXD6gqLRxOkfX4mnqzxfnm8Oq73Dy+FMA91auKY47k JZLzN78EYA9cZciKKHEBNVTIVrYq9zqFgg4U+jYWQT6twZGyMxkmq3rYRvJmcR2TAR IFfocnS1HCdaQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 2E509CE37DC; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Jens Axboe Subject: [PATCH rcu v2 3/7] rcu: Document self-propagating callbacks Date: Thu, 30 Jan 2025 10:49:21 -0800 Message-Id: <20250130184925.1651665-3-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit documents the fact that a given RCU callback function can repost itself. Reported-by: Jens Axboe Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 24f1cb292a92..befe35058c49 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3128,7 +3128,7 @@ module_param(enable_rcu_lazy, bool, 0444); * critical sections have completed. * * Use this API instead of call_rcu() if you don't want the callback to be - * invoked after very long periods of time, which can happen on systems without + * delayed for very long periods of time, which can happen on systems without * memory pressure and on systems which are lightly loaded or mostly idle. * This function will cause callbacks to be invoked sooner than later at the * expense of extra power. Other than that, this function is identical to, and @@ -3159,6 +3159,12 @@ EXPORT_SYMBOL_GPL(call_rcu_hurry); * might well execute concurrently with RCU read-side critical sections * that started after call_rcu() was invoked. * + * It is perfectly legal to repost an RCU callback, potentially with + * a different callback function, from within its callback function. + * The specified function will be invoked after another full grace period + * has elapsed. This use case is similar in form to the common practice + * of reposting a timer from within its own handler. + * * RCU read-side critical sections are delimited by rcu_read_lock() * and rcu_read_unlock(), and may be nested. In addition, but only in * v5.0 and later, regions of code across which interrupts, preemption, From patchwork Thu Jan 30 18:49:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954852 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 153CB18FDCE; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; cv=none; b=UBXLJU+g8AK/Sye5jZNnFnEYQ2GiTYSHNVHgdbm2o0rL9mcj5/XQ4TDuZv1l0W/6iERrZ6336qlm1/c7bTyTzKf8Gj0HDCFmOokQy+YF6gFt03mtayfMwSCqAhvl/oISUXeSK6dr9hTw2fADY3rCG+A0rPUX+MBDRHBsIJhf8DM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; c=relaxed/simple; bh=yoC9OP6s92AXVsVN10VonOg90I/4EWnjKGqOWBcb8Lw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KIYs7vTEdVstljQoeaYxvGbueTqCNOsRRLjrrXa9Qyez7ltvZA+7LktuZ1cjcjKUKT75IkSEw+gkeRuVOMJDcRS0UfJqHAk3j57e3fh3Y3aDvb7xScXC10ag9cbr7w7wuSR2jPM8s2NU5blZb9J01MPfKlogVJLtuWQOOqbA1DY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jvqP9Dli; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jvqP9Dli" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 98F78C4CEE3; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262967; bh=yoC9OP6s92AXVsVN10VonOg90I/4EWnjKGqOWBcb8Lw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jvqP9DliyLTTXhwpDWyw/+lsoaYgg3soWJKUazXcrVY3fQPpUcLpfMow7Z8urlBlZ qXpmAHWpsiE3VTtg57z04R6pjPCwbp05dYifmI2QOeZ6jhz8Mqtg2qhnJaHV67NQCK 3fNF4ZL1arasyixGawkdqYDH3i67hS3VALtj/9RgyojN/OcriXHBd3gtrxFlhiJ1L4 YHJljPW3CjfD7nL5Rq8++d88NbEFsWY0o8CNEHPNOBi/GVs8oOoNYdgmVEfLRKvtVk MneLr40EivztD0KgEkjDUEYsruni2DEMbUvPkpzkyy5FwX6hwwbFrsKFcX+nC+TEQC E0ONRO0UczmMg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 30D3ACE37DD; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu v2 4/7] srcu: Point call_srcu() to call_rcu() for detailed memory ordering Date: Thu, 30 Jan 2025 10:49:22 -0800 Message-Id: <20250130184925.1651665-4-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit causes the call_srcu() kernel-doc header to reference that of call_rcu() for detailed memory-ordering guarantees. Signed-off-by: Paul E. McKenney --- kernel/rcu/srcutree.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index b83c74c4dcc0..f87a9fb6d6bb 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -1398,8 +1398,12 @@ static void __call_srcu(struct srcu_struct *ssp, struct rcu_head *rhp, * read-side critical sections are delimited by srcu_read_lock() and * srcu_read_unlock(), and may be nested. * - * The callback will be invoked from process context, but must nevertheless - * be fast and must not block. + * The callback will be invoked from process context, but with bh + * disabled. The callback function must therefore be fast and must + * not block. + * + * See the description of call_rcu() for more detailed information on + * memory ordering guarantees. */ void call_srcu(struct srcu_struct *ssp, struct rcu_head *rhp, rcu_callback_t func) From patchwork Thu Jan 30 18:49:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954854 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1548F1E9B01; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; cv=none; b=RviSXaHIMAzMYkZydeAGqu/dtTlis6n0oXKvv0cn5H5cSGRy2gqEX5Uo5Z6gjgelHLeZRq0G6Ehtb1QZenUAL0/qoHqmQH0kCJRG5Oty/SM5r01cp/pjKtYHanVXfLnLIHzuYVZOQnvEFJZ6JNBr9T4OiZ8DkqRmv0zFat6ONvY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; c=relaxed/simple; bh=qCA8DvUvboeyMmYJDVVZIq4KuZNK5Bsr/7UY0MrYMtg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nMSkZ6lW03aopRUQ/ykcjXTvrt+kpNfUUAhVNqgGuJxx1VbM94zC/IQw2n382VZrdNgi/bNaGoxOKV6+vZnt2B6OH2OytRwDoOUwody0NpUAbbGb0a/csajcvjmQ+gwLIWd6FlgqDB4vNB3C2n03GybIF5Q7nKXmZVha9Y4Cb2M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Qd9FuL5y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Qd9FuL5y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9D2C5C4CEE6; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262967; bh=qCA8DvUvboeyMmYJDVVZIq4KuZNK5Bsr/7UY0MrYMtg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Qd9FuL5yWf23TkS4i/wqvRv2B3lVLhZk3fWTlCP24VnZFTQFTjXJh5RcLJC9ABvRp FYJX/334ivPtoLsuoyKQsBE+YzJyBlxDE7xB2pefV9lZczNiADONZR8nBqw5vT3Jhv 2bxIacQFW0rYFvsolbRNe0olso+wX2Jomsqk1jecCr8CDxGUJKGfohbqbS/3yVlITi QQ36doawtr+mkkfTVVZFRxkyMEy9WN4IBV1M9dIes6BKCz2ode+KdDdlHTD3vV82ZN D1+VjshvyvAvSdqQmuFJluqyOR1jIYmCEbJn/J1l4M8E60hwfdBlFFy72mmNDBDZeD DggcAOG0hZDJg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 33570CE37DF; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu v2 5/7] rcu: Add CONFIG_RCU_LAZY delays to call_rcu() kernel-doc header Date: Thu, 30 Jan 2025 10:49:23 -0800 Message-Id: <20250130184925.1651665-5-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit adds a description of the energy-efficiency delays that call_rcu() can impose, along with a pointer to call_rcu_hurry() for latency-sensitive kernel code. Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index befe35058c49..229f427b8c82 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3193,6 +3193,13 @@ EXPORT_SYMBOL_GPL(call_rcu_hurry); * * Implementation of these memory-ordering guarantees is described here: * Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst. + * + * Specific to call_rcu() (as opposed to the other call_rcu*() functions), + * in kernels built with CONFIG_RCU_LAZY=y, call_rcu() might delay for many + * seconds before starting the grace period needed by the corresponding + * callback. This delay can significantly improve energy-efficiency + * on low-utilization battery-powered devices. To avoid this delay, + * in latency-sensitive kernel code, use call_rcu_hurry(). */ void call_rcu(struct rcu_head *head, rcu_callback_t func) { From patchwork Thu Jan 30 18:49:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954856 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 158BA1F03E8; Thu, 30 Jan 2025 18:49:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; cv=none; b=NnPa4xHWsxZLsvNJG6Pz6nXs4rV1dhWfV3Gdu5w4MeUZFy2HOGH3ueUVxq5WkAF1Cj4ontmuGonCLB63EPzdC2qOjf2gExch3Shl5ViZ34bhn5qzOJoD9sfaJBUAE38SsEN8qbkN1TtRLakfSH9MkC2YXuelVpTwQmvO3J/7qio= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; c=relaxed/simple; bh=VydZE0P4L/LZcwHPe6EIAqUlXj2KVbd6m5WrTYx83ZM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AVPdS6m3Oh+geQ8SLADL4UvmSRel+ai+pAypSCyR4Q8O4dxlnA2HgJDhicr9AXutJen4yEWs+UyWz+ahnqpX6ekmTZQFvevOpnFbj40AJHfF6cRJ0tFvzy43jinXnmZW8uJjxL/HqhL0Nh22rg1V1Avc/lSakKCzLlnCoaY0dnI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=cf3dyeag; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="cf3dyeag" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D93FAC4CEE4; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262967; bh=VydZE0P4L/LZcwHPe6EIAqUlXj2KVbd6m5WrTYx83ZM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cf3dyeagGO3lHsz3fzg9A13l+2SU9nM4ekQ3sbU3Jgxh1M3694mKQ1FCOgLwvK6ye xHt5pcpwi7Yjq9t0ekONsh04sN9QcUKOi3/Ky3vpQ+g/X0hYJVIEHCNZ+aRVRRp1Rp mY3QbrNhPQ7soGaB15oPM5IewP2b3ih1MLa8T4jtX1XYIv753xrjk6WLaYznLVqheU dPoM0VN47I/nPut3L3oBKgENiCgmMtlOKP4oozcDaGl8CqbQ+WEckjL0za+rt4oUGo 56vErVxpzKo7dnfaodY7HfQjhkh/SNZjfwoRa+eg1+KKQ282LYPuugsNBQUMV8R+O8 uOp9CFDqGB9pQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 36154CE37E0; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" Subject: [PATCH rcu v2 6/7] rcu: Clarify RCU_LAZY and RCU_LAZY_DEFAULT_OFF help text Date: Thu, 30 Jan 2025 10:49:24 -0800 Message-Id: <20250130184925.1651665-6-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This commit wordsmiths the RCU_LAZY and RCU_LAZY_DEFAULT_OFF Kconfig options' help text. Signed-off-by: Paul E. McKenney --- kernel/rcu/Kconfig | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig index b9b6bc55185d..2bb22dac3b5a 100644 --- a/kernel/rcu/Kconfig +++ b/kernel/rcu/Kconfig @@ -323,21 +323,27 @@ config RCU_LAZY depends on RCU_NOCB_CPU default n help - To save power, batch RCU callbacks and flush after delay, memory - pressure, or callback list growing too big. + To save power, batch RCU callbacks and delay starting the + corresponding grace period for multiple seconds. The grace + period will be started after this delay, in case of memory + pressure, or if the corresponding CPU's callback list grows + too large. - Requires rcu_nocbs=all to be set. + These delays happen only on rcu_nocbs CPUs, that is, CPUs + whose callbacks have been offloaded. - Use rcutree.enable_rcu_lazy=0 to turn it off at boot time. + Use the rcutree.enable_rcu_lazy=0 kernel-boot parameter to + globally disable these delays. config RCU_LAZY_DEFAULT_OFF bool "Turn RCU lazy invocation off by default" depends on RCU_LAZY default n help - Allows building the kernel with CONFIG_RCU_LAZY=y yet keep it default - off. Boot time param rcutree.enable_rcu_lazy=1 can be used to switch - it back on. + Build the kernel with CONFIG_RCU_LAZY=y, but cause the kernel + to boot with these energy-efficiency delays disabled. Use the + rcutree.enable_rcu_lazy=0 kernel-boot parameter to override + the this option at boot time, thus re-enabling these delays. config RCU_DOUBLE_CHECK_CB_TIME bool "RCU callback-batch backup time check" From patchwork Thu Jan 30 18:49:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 13954857 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 857511F0E51; Thu, 30 Jan 2025 18:49:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; cv=none; b=JS0IDvgRn/AiHHMAQM0RIxTmdnMoyu8HsJQ5QzpQo+UZkg7QAeRp8KJPRPfnFu/gtZrN8chzsexUaoqtlMN9IyO4Kyu5fJfbtpilg+WX39VrFKCj+Lx9kPgA+0Ek8FLRwreKvR5GFx3MLNttpjVdzmXqer25DNLkOvHek69QUZs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738262968; c=relaxed/simple; bh=8xv5dwr+Q1Ceg+Jl4Hr+mTM9cUS7fLdtpRRTPhhkq7c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KZZ0KOcINpBmnqQu6WV0suD3yGKorN4I50Ck49Are7yOQq+MRacxf/B1JmPO7ganTeiqcnoOJ+xMg8C2mtuDwoT29wZ2BYibn5fu+3zROLXaY8+Sg5WHCBmcHZ1ESuXPC1wZ6UNAPlmzcIl8BIUrjOfPI5vV8H4B6OrafLXLwYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g0oK3vtc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g0oK3vtc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E74AFC4CEEA; Thu, 30 Jan 2025 18:49:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738262968; bh=8xv5dwr+Q1Ceg+Jl4Hr+mTM9cUS7fLdtpRRTPhhkq7c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g0oK3vtcQN6Bt1VRaYHQxgHKa8a7V1EXUkqHXMtyQ7HWQu/xQcerIiCu+avsSAsuH ATRLfPeA23ddNaFJx61ri9+qB3PxPxfxxHfLW5/QSxgZMfjoqSL7dfXIZOjErTlDU/ bbB/PDcIIbit1K/5EBZ0V00vdDzWuVAwmQqglvsG7GuytgEkfGqgT6qfQIoZvkK2Dd UWNukYbIPX+HykfeEgx71Xj/bae8DezeJmuzs+z5IBAyM+7OvBOSNtI65CLGW7ngTX NJ1CEJO3qBJQ4hMKzliNSjZVGdenUdmQZThKlCU7hVttMQ0/hrIhqaQ/bhS9z9Ci7b xbmLoW22nuVDQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 38AC3CE37E2; Thu, 30 Jan 2025 10:49:27 -0800 (PST) From: "Paul E. McKenney" To: rcu@vger.kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, rostedt@goodmis.org, "Paul E. McKenney" , Joe Perches Subject: [PATCH rcu v2 7/7] rcu: Remove references to old grace-period-wait primitives Date: Thu, 30 Jan 2025 10:49:25 -0800 Message-Id: <20250130184925.1651665-7-paulmck@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> References: <851fd11f-6397-4411-983b-96f7234326d5@paulmck-laptop> Precedence: bulk X-Mailing-List: rcu@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The rcu_barrier_sched(), synchronize_sched(), and synchronize_rcu_bh() RCU API members have been gone for many years. This commit therefore removes non-historical instances of them. Reported-by: Joe Perches Signed-off-by: Paul E. McKenney --- Documentation/RCU/rcubarrier.rst | 5 +---- include/linux/rcupdate.h | 17 +++++++---------- 2 files changed, 8 insertions(+), 14 deletions(-) diff --git a/Documentation/RCU/rcubarrier.rst b/Documentation/RCU/rcubarrier.rst index 6da7f66da2a8..12a7b059654f 100644 --- a/Documentation/RCU/rcubarrier.rst +++ b/Documentation/RCU/rcubarrier.rst @@ -329,10 +329,7 @@ Answer: was first added back in 2005. This is because on_each_cpu() disables preemption, which acted as an RCU read-side critical section, thus preventing CPU 0's grace period from completing - until on_each_cpu() had dealt with all of the CPUs. However, - with the advent of preemptible RCU, rcu_barrier() no longer - waited on nonpreemptible regions of code in preemptible kernels, - that being the job of the new rcu_barrier_sched() function. + until on_each_cpu() had dealt with all of the CPUs. However, with the RCU flavor consolidation around v4.20, this possibility was once again ruled out, because the consolidated diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 48e5c03df1dd..3bb554723074 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -806,11 +806,9 @@ do { \ * sections, invocation of the corresponding RCU callback is deferred * until after the all the other CPUs exit their critical sections. * - * In v5.0 and later kernels, synchronize_rcu() and call_rcu() also - * wait for regions of code with preemption disabled, including regions of - * code with interrupts or softirqs disabled. In pre-v5.0 kernels, which - * define synchronize_sched(), only code enclosed within rcu_read_lock() - * and rcu_read_unlock() are guaranteed to be waited for. + * Both synchronize_rcu() and call_rcu() also wait for regions of code + * with preemption disabled, including regions of code with interrupts or + * softirqs disabled. * * Note, however, that RCU callbacks are permitted to run concurrently * with new RCU read-side critical sections. One way that this can happen @@ -865,11 +863,10 @@ static __always_inline void rcu_read_lock(void) * rcu_read_unlock() - marks the end of an RCU read-side critical section. * * In almost all situations, rcu_read_unlock() is immune from deadlock. - * In recent kernels that have consolidated synchronize_sched() and - * synchronize_rcu_bh() into synchronize_rcu(), this deadlock immunity - * also extends to the scheduler's runqueue and priority-inheritance - * spinlocks, courtesy of the quiescent-state deferral that is carried - * out when rcu_read_unlock() is invoked with interrupts disabled. + * This deadlock immunity also extends to the scheduler's runqueue + * and priority-inheritance spinlocks, courtesy of the quiescent-state + * deferral that is carried out when rcu_read_unlock() is invoked with + * interrupts disabled. * * See rcu_read_lock() for more information. */