[v10,0/4] Add support for AArch64 AMUv1-based average freq

Message

Beata Michalska Jan. 31, 2025, 4:24 p.m. UTC

Hi All,

This series adds support for obtaining an average CPU frequency based on
a hardware provided feedback. The average frequency is being exposed via
dedicated yet optional cpufreq sysfs attribute - cpuinfo_avg_freq.
The architecture specific bits are being provided for AArch64, caching on
existing implementation for FIE and AMUv1 support: the frequency scale
factor, updated on each sched tick, serving as a base for retrieving
the frequency for a given CPU, representing an average frequency
reported between the ticks.

The changes have been rather lightly (due to some limitations) tested on
an FVP model.

Note that, this series depends on [6] and [7] !

Relevant discussions:
[1] https://lore.kernel.org/all/[e-mail]/
[2] https://lore.kernel.org/all/7eozim2xnepacnnkzxlbx34hib4otycnbn4dqymfziqou5lw5u@5xzpv3t7sxo3/
[3] https://lore.kernel.org/all/[e-mail]/
[4] https://lore.kernel.org/lkml/[e-mail]/T/#m4e74cb5a0aaa353c60fedc6cfb95ab7a6e381e3c
[5] https://lore.kernel.org/all/[e-mail]/

[6] https://lore.kernel.org/all/[e-mail]/
[7] https://lore.kernel.org/all/[e-mail]/

v10:
- Assume '0' is still somewhat a valid frequency value (as per discussion)
  and that gets applied for show_cpuinfo for x86
- Excluded patch:
	"arm64: amu: Delay allocating cpumask for AMU FIE support"
  which has been sent out as a separate change at [7]
- Added info on EAGAIN error within the docs
- Switched to while loop instead of goto statement for arch_freq_get_on_cpu
  implementation

v9:
- Moved changes to arch_freq_get_on_cpu to a separate patch

v8:
- Drop introducing new function and reuse arch_freq_get_on_cpu, guarding its use
  in scaling_cur_freq sysfs handler with dedicated config for x86

v7:
- Dropping 'arch_topology: init capacity_freq_ref to 0' patch from the series
  as this one has been sent separately as an independent change
  [https://lore.kernel.org/all/[e-mail]/]
- Including in the series change that introduces new sysfs entry [PATCH 1/4]
- Consequently modifying previously arch_freq_get_on_cpu to match reqs for new
  sysfs attribute
- Dropping an idea of considering a CPU that has been idle for a while as a
  valid source of information for obtaining an AMU-counter based frequency
- Some minor cosmetic changes

v6:
 - delay allocating cpumask for AMU FIE support instead of invalidating the mask
   upon failure to register cpufreq policy notifications
 - drop the change to cpufreq core (for cpuinfo_cur_freq) as this one will be
   sent as a separate change

v5:
 - Fix invalid access to cpumask
 - Reworked finding reference cpu when getting the freq

v4:
- dropping seqcount
- fixing identifying active cpu within given policy
- skipping full dynticks cpus when retrieving the freq
- bringing back plugging in arch_freq_get_on_cpu into cpuinfo_cur_freq

v3:
- dropping changes to cpufreq_verify_current_freq
- pulling in changes from Ionela initializing capacity_freq_ref to 0
  (thanks for that!)  and applying suggestions made by her during last review:
	- switching to arch_scale_freq_capacity and arch_scale_freq_ref when
	  reversing freq scale factor computation
	- swapping shift with multiplication
- adding time limit for considering last scale update as valid
- updating frequency scale factor upon entering idle

v2:
- Splitting the patches
- Adding comment for full dyntick mode
- Plugging arch_freq_get_on_cpu into cpufreq_verify_current_freq instead
  of in show_cpuinfo_cur_freq to allow the framework to stay more in sync
  with potential freq changes

CC: Jonathan Corbet <[e-mail]>
CC: Thomas Gleixner <[e-mail]>
CC: Ingo Molnar <[e-mail]>
CC: Borislav Petkov <[e-mail]>
CC: Dave Hansen <[e-mail]>
CC: H. Peter Anvin <[e-mail]>
CC: Phil Auld <[e-mail]>
CC: [e-mail]
CC: [e-mail]

Beata Michalska (4):
  cpufreq: Allow arch_freq_get_on_cpu to return an error
  cpufreq: Introduce an optional cpuinfo_avg_freq sysfs entry
  arm64: Provide an AMU-based version of arch_freq_get_on_cpu
  arm64: Update AMU-based freq scale factor on entering idle

 Documentation/admin-guide/pm/cpufreq.rst |  17 ++-
 arch/arm64/kernel/topology.c             | 125 +++++++++++++++++++++--
 arch/x86/kernel/cpu/aperfmperf.c         |   2 +-
 arch/x86/kernel/cpu/proc.c               |   7 +-
 drivers/cpufreq/Kconfig.x86              |  12 +++
 drivers/cpufreq/cpufreq.c                |  38 ++++++-
 include/linux/cpufreq.h                  |   2 +-
 7 files changed, 183 insertions(+), 20 deletions(-)

Comments

Catalin Marinas Feb. 17, 2025, 9:32 p.m. UTC | #1

On Fri, 31 Jan 2025 16:24:35 +0000, Beata Michalska wrote:
> This series adds support for obtaining an average CPU frequency based on
> a hardware provided feedback. The average frequency is being exposed via
> dedicated yet optional cpufreq sysfs attribute - cpuinfo_avg_freq.
> The architecture specific bits are being provided for AArch64, caching on
> existing implementation for FIE and AMUv1 support: the frequency scale
> factor, updated on each sched tick, serving as a base for retrieving
> the frequency for a given CPU, representing an average frequency
> reported between the ticks.
> 
> [...]

Applied to arm64 (for-next/amuv1-avg-freq), thanks!

[1/4] cpufreq: Allow arch_freq_get_on_cpu to return an error
      https://git.kernel.org/arm64/c/38e480d4fcac
[2/4] cpufreq: Introduce an optional cpuinfo_avg_freq sysfs entry
      https://git.kernel.org/arm64/c/fbb4a4759b54
[3/4] arm64: Provide an AMU-based version of arch_freq_get_on_cpu
      https://git.kernel.org/arm64/c/dd871ac1237f
[4/4] arm64: Update AMU-based freq scale factor on entering idle
      https://git.kernel.org/arm64/c/96b335620c59

Beata Michalska Feb. 18, 2025, midnight UTC | #2

On Mon, Feb 17, 2025 at 09:32:06PM +0000, Catalin Marinas wrote:
> On Fri, 31 Jan 2025 16:24:35 +0000, Beata Michalska wrote:
> > This series adds support for obtaining an average CPU frequency based on
> > a hardware provided feedback. The average frequency is being exposed via
> > dedicated yet optional cpufreq sysfs attribute - cpuinfo_avg_freq.
> > The architecture specific bits are being provided for AArch64, caching on
> > existing implementation for FIE and AMUv1 support: the frequency scale
> > factor, updated on each sched tick, serving as a base for retrieving
> > the frequency for a given CPU, representing an average frequency
> > reported between the ticks.
> > 
> > [...]
>
Thank you for that.

There is still a (not so) small issue with patch
[3/4] arm64: Provide an AMU-based version of arch_freq_get_on_cpu.
It did not come up while testing, sadly.
No idea how I could have missed that, nor why I made the mistake
in the first place.

The fix is pretty straightforward:

diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 6f0cab8e746b..4bac26d8e29c 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -268,7 +268,7 @@ int arch_freq_get_on_cpu(int cpu)
 
                        do {
                                ref_cpu = cpumask_next_wrap(ref_cpu, policy->cpus,
-                                                           start_cpu, false);
+                                                           start_cpu, true);

Please let me know if you want me to send new version with the fix applied.

Apologies for the inconvenience.

---
BR
Beata

> Applied to arm64 (for-next/amuv1-avg-freq), thanks!
> 
> [1/4] cpufreq: Allow arch_freq_get_on_cpu to return an error
>       https://git.kernel.org/arm64/c/38e480d4fcac
> [2/4] cpufreq: Introduce an optional cpuinfo_avg_freq sysfs entry
>       https://git.kernel.org/arm64/c/fbb4a4759b54
> [3/4] arm64: Provide an AMU-based version of arch_freq_get_on_cpu
>       https://git.kernel.org/arm64/c/dd871ac1237f
> [4/4] arm64: Update AMU-based freq scale factor on entering idle
>       https://git.kernel.org/arm64/c/96b335620c59
> 
> -- 
> Catalin
>

Catalin Marinas Feb. 18, 2025, 9:18 a.m. UTC | #3

On Tue, Feb 18, 2025 at 01:00:20AM +0100, Beata Michalska wrote:
> On Mon, Feb 17, 2025 at 09:32:06PM +0000, Catalin Marinas wrote:
> > On Fri, 31 Jan 2025 16:24:35 +0000, Beata Michalska wrote:
> > > This series adds support for obtaining an average CPU frequency based on
> > > a hardware provided feedback. The average frequency is being exposed via
> > > dedicated yet optional cpufreq sysfs attribute - cpuinfo_avg_freq.
> > > The architecture specific bits are being provided for AArch64, caching on
> > > existing implementation for FIE and AMUv1 support: the frequency scale
> > > factor, updated on each sched tick, serving as a base for retrieving
> > > the frequency for a given CPU, representing an average frequency
> > > reported between the ticks.
> > > 
> > > [...]
> >
> Thank you for that.
> 
> There is still a (not so) small issue with patch
> [3/4] arm64: Provide an AMU-based version of arch_freq_get_on_cpu.
> It did not come up while testing, sadly.
> No idea how I could have missed that, nor why I made the mistake
> in the first place.
> 
> The fix is pretty straightforward:
> 
> diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> index 6f0cab8e746b..4bac26d8e29c 100644
> --- a/arch/arm64/kernel/topology.c
> +++ b/arch/arm64/kernel/topology.c
> @@ -268,7 +268,7 @@ int arch_freq_get_on_cpu(int cpu)
>  
>                         do {
>                                 ref_cpu = cpumask_next_wrap(ref_cpu, policy->cpus,
> -                                                           start_cpu, false);
> +                                                           start_cpu, true);
> 
> Please let me know if you want me to send new version with the fix applied.

Usually we apply another patch on top with a Fixes tag or just fold it
in if no-one relies on this branch being stable. I'll do the latter, no
need to resend.

Thanks.