mbox series

[RFC,0/3] cpufreq: cppc: Add support for frequency invariance

Message ID cover.1594289009.git.viresh.kumar@linaro.org (mailing list archive)
Headers show
Series cpufreq: cppc: Add support for frequency invariance | expand

Message

Viresh Kumar July 9, 2020, 10:13 a.m. UTC
Hello,

CPPC cpufreq driver is used for ARM servers and this patch series tries
to provide frequency invariance support for them. The same is also
provided using a specific hardware extension, known as AMU (Activity
Monitors Unit), but that is optional for platforms and at least few of
them don't have it.

This patchset allows multiple parts of the kernel to provide the same
functionality, by registering with the topology core.

This is tested with some hacks, as I didn't have access to the right
hardware, on the ARM64 hikey board to check the overall functionality
and that works fine.

Ionela/Peter Puhov, it would be nice if you guys can give this a shot.

This is based of my cpufreq/arm/linux-next branch (should work on
linux-next too) + a cleanup patch [1] that i sent this morning.

--
viresh

[1] https://lore.kernel.org/lkml/a710fc4e4e0f1d2e561320130b99bcb5167d73b4.1594277563.git.viresh.kumar@linaro.org/

Viresh Kumar (3):
  arm64: topology: Add amu_counters_supported() helper
  topology: Provide generic implementation of
    arch_freq_counters_available()
  cpufreq: cppc: Add support for frequency invariance

 arch/arm64/include/asm/topology.h |   7 --
 arch/arm64/kernel/topology.c      | 165 +++++++++++++++---------------
 drivers/base/arch_topology.c      |  43 +++++++-
 drivers/cpufreq/cppc_cpufreq.c    | 138 ++++++++++++++++++++++++-
 include/linux/arch_topology.h     |   5 +-
 kernel/sched/core.c               |   1 +
 6 files changed, 263 insertions(+), 96 deletions(-)

Comments

Ionela Voinescu July 9, 2020, 12:43 p.m. UTC | #1
Hi Viresh,

I'll put all my comments here for now, as they refer more to the design
of the solution.

I hope it won't be too repetitive compared to what we previously discussed
offline. I understand you want to get additional points of view.

On Thursday 09 Jul 2020 at 15:43:32 (+0530), Viresh Kumar wrote:
> Hello,
> 
> CPPC cpufreq driver is used for ARM servers and this patch series tries
> to provide frequency invariance support for them. The same is also
> provided using a specific hardware extension, known as AMU (Activity
> Monitors Unit), but that is optional for platforms and at least few of
> them don't have it.
> 
> This patchset allows multiple parts of the kernel to provide the same
> functionality, by registering with the topology core.
> 
> This is tested with some hacks, as I didn't have access to the right
> hardware, on the ARM64 hikey board to check the overall functionality
> and that works fine.
> 
> Ionela/Peter Puhov, it would be nice if you guys can give this a shot.
> 

I believe the code is unnecessarily invasive for the functionality it
tries to introduce and it does break existing functionality.


 - (1) From code readability and design point of view, this switching
       between an architectural method and a driver method complicates
       an already complicated situation. We already have code that
       chooses between a cpufreq-based method and a counter based method
       for frequency invariance. This would basically introduce a choice
       between a cpufreq-based method through arch_set_freq_scale(), an
       architectural counter-based method through arch_set_freq_tick(),
       and another cpufreq-based method that piggy-backs on the
       architectural arch_set_freq_tick().

       As discussed offline, before I even try to begin accepting the
       possibility of this complicated mix, I would like to know why
       methods of obtaining the same thing by using the cpufreq
       arch_set_freq_scale() or even the more invasive wrapping of the
       counter read functions is not working. I believe those solutions
       would brings only a fraction of the complexity added through this
       set.

 - (2) For 1/3, the presence of AMU counters does not guarantee their
       usability for frequency invariance. I know you wanted to avoid
       the complications of AMUs being marked as supporting invariance
       after the cpufreq driver init function, but this breaks the
       scenario in which the maximum frequency is invalid.

 - (3) For 2/3, currently we support platforms that have partial support
       for AMUs, while this would not be supported here. The suggestions
       at (1) would give us this for free.


While I'm keen on having invariance support for CPPC when lacking part
or full support for AMUs, I think we should explore alternative designs.
I can try to come up with some code suggestions, but it will take a few
days as I have many other things in the air :).

I'm happy to test this when the design is agreed on.

Hope it helps,
Ionela.
Viresh Kumar July 10, 2020, 3 a.m. UTC | #2
Thanks for the quick reply Ionela.

On 09-07-20, 13:43, Ionela Voinescu wrote:
> I'll put all my comments here for now, as they refer more to the design
> of the solution.
> 
> I hope it won't be too repetitive compared to what we previously discussed
> offline.

> I understand you want to get additional points of view.

Not necessarily, I knew you would be one of the major reviewers here
:)

I posted so you don't need to review in private anymore and then the
code is somewhat updated since the previous time.

> On Thursday 09 Jul 2020 at 15:43:32 (+0530), Viresh Kumar wrote:
> I believe the code is unnecessarily invasive for the functionality it
> tries to introduce and it does break existing functionality.
> 
> 
>  - (1) From code readability and design point of view, this switching
>        between an architectural method and a driver method complicates
>        an already complicated situation. We already have code that
>        chooses between a cpufreq-based method and a counter based method
>        for frequency invariance. This would basically introduce a choice
>        between a cpufreq-based method through arch_set_freq_scale(), an
>        architectural counter-based method through arch_set_freq_tick(),
>        and another cpufreq-based method that piggy-backs on the
>        architectural arch_set_freq_tick().

I agree.

>        As discussed offline, before I even try to begin accepting the
>        possibility of this complicated mix, I would like to know why
>        methods of obtaining the same thing by using the cpufreq
>        arch_set_freq_scale()

The problem is same as that was in case of x86, we don't know the real
frequency the CPU may be running at and we need something that fires
up periodically in a guaranteed way to capture the freq-scale.

Though I am thinking now if we can trust the target_index() helper and
keep updating the freq-scale based on the delta between last call to
it and the latest call. I am not sure if it will be sufficient.

>        or even the more invasive wrapping of the
>        counter read functions is not working.

I am not sure I understood this one.

>  - (2) For 1/3, the presence of AMU counters does not guarantee their
>        usability for frequency invariance. I know you wanted to avoid
>        the complications of AMUs being marked as supporting invariance
>        after the cpufreq driver init function, but this breaks the
>        scenario in which the maximum frequency is invalid.

Is that really a scenario ? i.e. Invalid maximum frequency ? Why would
that ever happen ?

And I am not sure if this breaks anything which already exists,
because all we are doing in this case now is not registering cppc for
FI, which should be fine.

>  - (3) For 2/3, currently we support platforms that have partial support
>        for AMUs, while this would not be supported here. The suggestions
>        at (1) would give us this for free.

As both were counter based mechanisms, I thought it would be better
and more consistent if only one of them is picked. Though partial
support of AMUs would still work without the CPPC driver.
Vincent Guittot July 24, 2020, 9:38 a.m. UTC | #3
On Fri, 10 Jul 2020 at 05:00, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> Thanks for the quick reply Ionela.
>
> On 09-07-20, 13:43, Ionela Voinescu wrote:
> > I'll put all my comments here for now, as they refer more to the design
> > of the solution.
> >
> > I hope it won't be too repetitive compared to what we previously discussed
> > offline.
>
> > I understand you want to get additional points of view.
>
> Not necessarily, I knew you would be one of the major reviewers here
> :)
>
> I posted so you don't need to review in private anymore and then the
> code is somewhat updated since the previous time.
>
> > On Thursday 09 Jul 2020 at 15:43:32 (+0530), Viresh Kumar wrote:
> > I believe the code is unnecessarily invasive for the functionality it
> > tries to introduce and it does break existing functionality.
> >
> >
> >  - (1) From code readability and design point of view, this switching
> >        between an architectural method and a driver method complicates
> >        an already complicated situation. We already have code that
> >        chooses between a cpufreq-based method and a counter based method
> >        for frequency invariance. This would basically introduce a choice
> >        between a cpufreq-based method through arch_set_freq_scale(), an
> >        architectural counter-based method through arch_set_freq_tick(),
> >        and another cpufreq-based method that piggy-backs on the
> >        architectural arch_set_freq_tick().
>
> I agree.
>
> >        As discussed offline, before I even try to begin accepting the
> >        possibility of this complicated mix, I would like to know why
> >        methods of obtaining the same thing by using the cpufreq
> >        arch_set_freq_scale()
>
> The problem is same as that was in case of x86, we don't know the real
> frequency the CPU may be running at and we need something that fires
> up periodically in a guaranteed way to capture the freq-scale.

Yeah it's exactly the same behavior as x86 and re using the same
mechanism seems the  best solution

The main problem is that AMU currently assumes that it will be the
only to support such tick based mechanism whereas others like cppc can
provides similar information

>
> Though I am thinking now if we can trust the target_index() helper and
> keep updating the freq-scale based on the delta between last call to
> it and the latest call. I am not sure if it will be sufficient.
>
> >        or even the more invasive wrapping of the
> >        counter read functions is not working.
>
> I am not sure I understood this one.
>
> >  - (2) For 1/3, the presence of AMU counters does not guarantee their
> >        usability for frequency invariance. I know you wanted to avoid
> >        the complications of AMUs being marked as supporting invariance
> >        after the cpufreq driver init function, but this breaks the
> >        scenario in which the maximum frequency is invalid.
>
> Is that really a scenario ? i.e. Invalid maximum frequency ? Why would
> that ever happen ?
>
> And I am not sure if this breaks anything which already exists,
> because all we are doing in this case now is not registering cppc for
> FI, which should be fine.

IIUC, AMU must wait for cpufreq drivers to be registered in order to
get the maximum freq and being able to enable its FIE support.
Could we have a sequence like:
cppc register its scale_freq_tick function
AMU can then override the tick function for cpu which supports AMU

>
> >  - (3) For 2/3, currently we support platforms that have partial support
> >        for AMUs, while this would not be supported here. The suggestions
> >        at (1) would give us this for free.
>
> As both were counter based mechanisms, I thought it would be better
> and more consistent if only one of them is picked. Though partial
> support of AMUs would still work without the CPPC driver.
>
> --
> viresh
Viresh Kumar Aug. 24, 2020, 10:49 a.m. UTC | #4
On 24-07-20, 11:38, Vincent Guittot wrote:
> Yeah it's exactly the same behavior as x86 and re using the same
> mechanism seems the  best solution
> 
> The main problem is that AMU currently assumes that it will be the
> only to support such tick based mechanism whereas others like cppc can
> provides similar information

Ionela do you have some feedback to Vincent's email?

We can't handle this apart from the tick handler, i.e. capture this
data at every tick like it is done for x86 or AMU.
Ionela Voinescu Aug. 25, 2020, 9:56 a.m. UTC | #5
Hi guys,

On Friday 24 Jul 2020 at 11:38:59 (+0200), Vincent Guittot wrote:
> On Fri, 10 Jul 2020 at 05:00, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> >
> > Thanks for the quick reply Ionela.
> >
> > On 09-07-20, 13:43, Ionela Voinescu wrote:
> > > I'll put all my comments here for now, as they refer more to the design
> > > of the solution.
> > >
> > > I hope it won't be too repetitive compared to what we previously discussed
> > > offline.
> >
> > > I understand you want to get additional points of view.
> >
> > Not necessarily, I knew you would be one of the major reviewers here
> > :)
> >
> > I posted so you don't need to review in private anymore and then the
> > code is somewhat updated since the previous time.
> >
> > > On Thursday 09 Jul 2020 at 15:43:32 (+0530), Viresh Kumar wrote:
> > > I believe the code is unnecessarily invasive for the functionality it
> > > tries to introduce and it does break existing functionality.
> > >
> > >
> > >  - (1) From code readability and design point of view, this switching
> > >        between an architectural method and a driver method complicates
> > >        an already complicated situation. We already have code that
> > >        chooses between a cpufreq-based method and a counter based method
> > >        for frequency invariance. This would basically introduce a choice
> > >        between a cpufreq-based method through arch_set_freq_scale(), an
> > >        architectural counter-based method through arch_set_freq_tick(),
> > >        and another cpufreq-based method that piggy-backs on the
> > >        architectural arch_set_freq_tick().
> >
> > I agree.
> >
> > >        As discussed offline, before I even try to begin accepting the
> > >        possibility of this complicated mix, I would like to know why
> > >        methods of obtaining the same thing by using the cpufreq
> > >        arch_set_freq_scale()
> >
> > The problem is same as that was in case of x86, we don't know the real
> > frequency the CPU may be running at and we need something that fires
> > up periodically in a guaranteed way to capture the freq-scale.
> 
> Yeah it's exactly the same behavior as x86 and re using the same
> mechanism seems the  best solution
> 
> The main problem is that AMU currently assumes that it will be the
> only to support such tick based mechanism whereas others like cppc can
> provides similar information
> 

Yes, I agree that a similar method to the use of AMUs or APERF/MPERF would
result in a more accurate frequency scale factor.

> >
> > Though I am thinking now if we can trust the target_index() helper and
> > keep updating the freq-scale based on the delta between last call to
> > it and the latest call. I am not sure if it will be sufficient.
> >
> > >        or even the more invasive wrapping of the
> > >        counter read functions is not working.
> >
> > I am not sure I understood this one.
> >

I've been putting some more thought/code into this one and I believe
something as follows might look nicer as well as cover a few corner cases
(ignore implementation details for now, they can be changed):

- Have a per cpu value that marks the use of either AMUs, CPPC, or
  cpufreq for freq invariance (can be done with per-cpu variable or with
  cpumasks)

- arch_topology.c: initialization code as follows:

	for_each_present_cpu(cpu) {
		if (freq_inv_amus_valid(cpu) &&
		    !freq_inv_set_max_ratio(cpufreq_get_hw_max_freq(cpu) * 1000,
					    arch_timer_get_rate(), cpu)) {
			per_cpu(inv_source, cpu) = INV_AMU_COUNTERS;
			continue;
		}
		if (freq_inv_cppc_counters_valid(cpu) &&
		    !freq_inv_set_max_ratio(cppc_max_perf, cppc_ref_perf, cpu)) {
			per_cpu(inv_source, cpu) = INV_CPPC_COUNTERS;
			continue;
		}
		if (!cpufreq_supports_freq_invariance() ||
		    freq_inv_set_max_ratio(cpufreq_get_hw_max_freq(cpu),
					   1, cpu)) {
			pr_info("FIE disabled: no valid source for CPU%d.", cpu);
			return 0;
		}
	}
  This would live in an equivalent of the init function we have now for
  AMU counters only (init_amu_fie), made to handle more sources and moved
  to arch_topology.c.
  The freq_inv_set_max_ratio() would be a generic version of what is now
  validate_cpu_freq_invariance_counters() (only the ratio computation and
  setting).

 - Finally, 
	void freq_inv_update_counter_refs(void)
	{
		this_cpu_write(arch_core_cycles_prev, read_corecnt());
		this_cpu_write(arch_const_cycles_prev, read_constcnt());
	}
  This would be an equivalent of init_cpu_freq_invariance_counters().
  There is the option of either read_{corecnt/constcnt}() to either do AMU
  reads or CPPC counter reads depending on inv_source, or for either arm64
  or cppc code to define the entire freq_inv_update_counter_refs().

 - Given all of the above, topology_scale_freq_tick() can then be made generic

	prev_const_cnt = this_cpu_read(arch_const_cycles_prev);
	prev_core_cnt = this_cpu_read(arch_core_cycles_prev);
	freq_inv_update_counter_refs();
	const_cnt = this_cpu_read(arch_const_cycles_prev);
	core_cnt = this_cpu_read(arch_core_cycles_prev);

I have some versions of code that do this generalisation for AMUs and cpufreq
with a common topology_set_freq_scale() that is to be used for both
arch_set_freq_scale() and arch_set_freq_tick(). But it's written with this
usecase in mind so it can be extended to use CPPC counters as source as well,
as detailed above.

So, this is basically what I had in mind when recommending "wrapping of the
counter read functions" :). This would basically reuse much of what is now
the AMU invariance code while allowing for CPPC counters as a possible source.

I'll stop here for now to see what you guys think about this.

Thanks,
Ionela.

> > >  - (2) For 1/3, the presence of AMU counters does not guarantee their
> > >        usability for frequency invariance. I know you wanted to avoid
> > >        the complications of AMUs being marked as supporting invariance
> > >        after the cpufreq driver init function, but this breaks the
> > >        scenario in which the maximum frequency is invalid.
> >
> > Is that really a scenario ? i.e. Invalid maximum frequency ? Why would
> > that ever happen ?
> >
> > And I am not sure if this breaks anything which already exists,
> > because all we are doing in this case now is not registering cppc for
> > FI, which should be fine.
> 
> IIUC, AMU must wait for cpufreq drivers to be registered in order to
> get the maximum freq and being able to enable its FIE support.
> Could we have a sequence like:
> cppc register its scale_freq_tick function
> AMU can then override the tick function for cpu which supports AMU
> 
> >
> > >  - (3) For 2/3, currently we support platforms that have partial support
> > >        for AMUs, while this would not be supported here. The suggestions
> > >        at (1) would give us this for free.
> >
> > As both were counter based mechanisms, I thought it would be better
> > and more consistent if only one of them is picked. Though partial
> > support of AMUs would still work without the CPPC driver.
> >
> > --
> > viresh
Viresh Kumar Aug. 27, 2020, 7:51 a.m. UTC | #6
On 25-08-20, 10:56, Ionela Voinescu wrote:
> I've been putting some more thought/code into this one and I believe
> something as follows might look nicer as well as cover a few corner cases
> (ignore implementation details for now, they can be changed):

I saw the other patchset you sent where AMU can be used as the backend
for CPPC driver, which means that if AMU IP is present on the platform
it will be used by the CPPC to get the perf counts, right ?

> - Have a per cpu value that marks the use of either AMUs, CPPC, or
>   cpufreq for freq invariance (can be done with per-cpu variable or with
>   cpumasks)
> 
> - arch_topology.c: initialization code as follows:
> 
> 	for_each_present_cpu(cpu) {
> 		if (freq_inv_amus_valid(cpu) &&
> 		    !freq_inv_set_max_ratio(cpufreq_get_hw_max_freq(cpu) * 1000,
> 					    arch_timer_get_rate(), cpu)) {
> 			per_cpu(inv_source, cpu) = INV_AMU_COUNTERS;
> 			continue;
> 		}
> 		if (freq_inv_cppc_counters_valid(cpu) &&
> 		    !freq_inv_set_max_ratio(cppc_max_perf, cppc_ref_perf, cpu)) {
> 			per_cpu(inv_source, cpu) = INV_CPPC_COUNTERS;
> 			continue;
> 		}
> 		if (!cpufreq_supports_freq_invariance() ||
> 		    freq_inv_set_max_ratio(cpufreq_get_hw_max_freq(cpu),
> 					   1, cpu)) {
> 			pr_info("FIE disabled: no valid source for CPU%d.", cpu);
> 			return 0;
> 		}
> 	}

Based on that (your other patchset), I think this can get further
simplified to whomsoever can register first for freq invariance.

i.e. if CPPC registers for it first then there is no need to check
AMUs further (as CPPC will be using AMUs anyway), else we will
fallback to AMU, else cpufreq.

Is that understanding correct ?
Ionela Voinescu Aug. 27, 2020, 11:27 a.m. UTC | #7
Hi Viresh,

On Thursday 27 Aug 2020 at 13:21:49 (+0530), Viresh Kumar wrote:
> On 25-08-20, 10:56, Ionela Voinescu wrote:
> > I've been putting some more thought/code into this one and I believe
> > something as follows might look nicer as well as cover a few corner cases
> > (ignore implementation details for now, they can be changed):
> 
> I saw the other patchset you sent where AMU can be used as the backend
> for CPPC driver, which means that if AMU IP is present on the platform
> it will be used by the CPPC to get the perf counts, right ?
> 
> > - Have a per cpu value that marks the use of either AMUs, CPPC, or
> >   cpufreq for freq invariance (can be done with per-cpu variable or with
> >   cpumasks)
> > 
> > - arch_topology.c: initialization code as follows:
> > 
> > 	for_each_present_cpu(cpu) {
> > 		if (freq_inv_amus_valid(cpu) &&
> > 		    !freq_inv_set_max_ratio(cpufreq_get_hw_max_freq(cpu) * 1000,
> > 					    arch_timer_get_rate(), cpu)) {
> > 			per_cpu(inv_source, cpu) = INV_AMU_COUNTERS;
> > 			continue;
> > 		}
> > 		if (freq_inv_cppc_counters_valid(cpu) &&
> > 		    !freq_inv_set_max_ratio(cppc_max_perf, cppc_ref_perf, cpu)) {
> > 			per_cpu(inv_source, cpu) = INV_CPPC_COUNTERS;
> > 			continue;
> > 		}
> > 		if (!cpufreq_supports_freq_invariance() ||
> > 		    freq_inv_set_max_ratio(cpufreq_get_hw_max_freq(cpu),
> > 					   1, cpu)) {
> > 			pr_info("FIE disabled: no valid source for CPU%d.", cpu);
> > 			return 0;
> > 		}
> > 	}
> 
> Based on that (your other patchset), I think this can get further
> simplified to whomsoever can register first for freq invariance.
> 

I don't see it as anyone registering for freq invariance, rather the
freq invariance framework chooses its source of information (AMU, CPPC,
cpufreq).

> i.e. if CPPC registers for it first then there is no need to check
> AMUs further (as CPPC will be using AMUs anyway), else we will
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Not necessarily. Even if AMUs are present, they are only used for CPPC's
delivered and reference performance counters if the ACPI _CPC entry
specifies FFH as method:

  ResourceTemplate(){Register(FFixedHW, 0x40, 0, 1, 0x4)},
  ResourceTemplate(){Register(FFixedHW, 0x40, 0, 0, 0x4)},

> fallback to AMU, else cpufreq.
> 
> Is that understanding correct ?

While I understand your point (accessing AMUs through CPPC's read
functions to implement invariance) I don't think it's worth tying the
two together.

I see the two functionalities as independent:
 - frequency invariance with whichever source of information is valid
   (AMUs, cpufreq, etc) is separate from
 - CPPC's delivered and reference performance counters, which currently
   are used in cpufreq's .get() function.

Therefore, taking each of the scenarios one by one:
 - All CPUs support AMUs: the freq invariance initialisation code will
   find AMUs valid and it will use them to set the scale factor;
   completely independently, if the FFH method is specified for CPPC's
   delivered and reference performance counters, it will also use
   AMUs, even if, let's say, invariance is disabled.

 - None of the CPUs support AMUs, but the _CPC entry specifies some
   platform specific counters for delivered and reference performance.
   With the current mainline code neither cpufreq or counter based
   invariance is supported, but the CPPC counters can be used in the
   cppc_cpufreq driver for the .get() function.

   But with the above new functionality we can detect that AMUs are not
   supported and expose the CPPC counters to replace them in
   implementing invariance.

 - Mixed scenarios are also supported if we play our cards right and
   implement the above per-cpu.


I'm thinking that having some well defined invariance sources might work
well: it will simplify the init function (go through all registered
sources and choose (per-cpu) the one that's valid) and allow for
otherwise generic invariance support. Something like:

enum freq_inv_source {INV_CPUFREQ, INV_AMU_COUNTERS, INV_CPPC_COUNTERS};

struct freq_inv_source {
	enum freq_inv_source source;
	bool (*valid)(int cpu);
	u64 (*read_corecnt)(int cpu);
	u64 (*read_constcnt)(int cpu);
	u64 (*max_rate)(int cpu);
	u64 (*ref_rate)(int cpu);
}

I am in the middle of unifying AMU counter and cpufreq invariance through
something like this, so if you like the idea and you don't think I'm
stepping too much on your toes with this, I can consider the usecase in
my (what should be) generic support. So in the end this might end up
being just a matter of adding a new invariance source (CPPC counters).

My only worry is that while I know how a cpufreq source behaves and how
AMU counters behave, I'm not entirely sure what to expect from CPPC
counters: if they are always appropriate for updates on the tick (not
blocking), if they both stop during idle, if there is save/restore
functionality before/after idle, etc.

What do you think?

Regards,
Ionela.

> -- 
> viresh
Viresh Kumar Aug. 31, 2020, 11:26 a.m. UTC | #8
On 27-08-20, 12:27, Ionela Voinescu wrote:
> I don't see it as anyone registering for freq invariance, rather the
> freq invariance framework chooses its source of information (AMU, CPPC,
> cpufreq).

Yeah, either way is fine for me.

> > i.e. if CPPC registers for it first then there is no need to check
> > AMUs further (as CPPC will be using AMUs anyway), else we will
>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Not necessarily. Even if AMUs are present, they are only used for CPPC's
> delivered and reference performance counters if the ACPI _CPC entry
> specifies FFH as method:
> 
>   ResourceTemplate(){Register(FFixedHW, 0x40, 0, 1, 0x4)},
>   ResourceTemplate(){Register(FFixedHW, 0x40, 0, 0, 0x4)},

Right.

> While I understand your point (accessing AMUs through CPPC's read
> functions to implement invariance) I don't think it's worth tying the
> two together.
> 
> I see the two functionalities as independent:
>  - frequency invariance with whichever source of information is valid
>    (AMUs, cpufreq, etc) is separate from
>  - CPPC's delivered and reference performance counters, which currently
>    are used in cpufreq's .get() function.
> 
> Therefore, taking each of the scenarios one by one:
>  - All CPUs support AMUs: the freq invariance initialisation code will
>    find AMUs valid and it will use them to set the scale factor;
>    completely independently, if the FFH method is specified for CPPC's
>    delivered and reference performance counters, it will also use
>    AMUs, even if, let's say, invariance is disabled.
> 
>  - None of the CPUs support AMUs, but the _CPC entry specifies some
>    platform specific counters for delivered and reference performance.
>    With the current mainline code neither cpufreq or counter based
>    invariance is supported, but the CPPC counters can be used in the
>    cppc_cpufreq driver for the .get() function.
> 
>    But with the above new functionality we can detect that AMUs are not
>    supported and expose the CPPC counters to replace them in
>    implementing invariance.
> 
>  - Mixed scenarios are also supported if we play our cards right and
>    implement the above per-cpu.
> 
> 
> I'm thinking that having some well defined invariance sources might work
> well: it will simplify the init function (go through all registered
> sources and choose (per-cpu) the one that's valid) and allow for
> otherwise generic invariance support. Something like:
> 
> enum freq_inv_source {INV_CPUFREQ, INV_AMU_COUNTERS, INV_CPPC_COUNTERS};
> 
> struct freq_inv_source {
> 	enum freq_inv_source source;
> 	bool (*valid)(int cpu);
> 	u64 (*read_corecnt)(int cpu);
> 	u64 (*read_constcnt)(int cpu);
> 	u64 (*max_rate)(int cpu);
> 	u64 (*ref_rate)(int cpu);
> }
> 
> I am in the middle of unifying AMU counter and cpufreq invariance through
> something like this, so if you like the idea and you don't think I'm
> stepping too much on your toes with this, I can consider the usecase in
> my (what should be) generic support. So in the end this might end up
> being just a matter of adding a new invariance source (CPPC counters).

Okay, if you have already started working on that, no issues from my
side. I can just get the relevant stuff from CPPC added once you
provide that layer..

> My only worry is that while I know how a cpufreq source behaves and how
> AMU counters behave, I'm not entirely sure what to expect from CPPC

Neither do I :)

> counters: if they are always appropriate for updates on the tick (not
> blocking),

The update stuff may sleep here and so I had to do stuff in the
irq-work handler in my patch.

> if they both stop during idle, if there is save/restore
> functionality before/after idle, etc.

This I will check.
Viresh Kumar Oct. 5, 2020, 7:58 a.m. UTC | #9
On 27-08-20, 12:27, Ionela Voinescu wrote:
> I am in the middle of unifying AMU counter and cpufreq invariance through
> something like this, so if you like the idea and you don't think I'm
> stepping too much on your toes with this, I can consider the usecase in
> my (what should be) generic support. So in the end this might end up
> being just a matter of adding a new invariance source (CPPC counters).

Any update on this ?
Ionela Voinescu Oct. 5, 2020, 11:16 p.m. UTC | #10
Hi Viresh,

On Monday 05 Oct 2020 at 13:28:22 (+0530), Viresh Kumar wrote:
> On 27-08-20, 12:27, Ionela Voinescu wrote:
> > I am in the middle of unifying AMU counter and cpufreq invariance through
> > something like this, so if you like the idea and you don't think I'm
> > stepping too much on your toes with this, I can consider the usecase in
> > my (what should be) generic support. So in the end this might end up
> > being just a matter of adding a new invariance source (CPPC counters).
> 
> Any update on this ?
> 

I have some code for this, but not yet in the final state I wanted to
bring it to. The code has some dependencies/conflicts with the FFH support
and in small part with the new BL_SWITCHER patches so I wanted to get
those through first.

I'm in the middle of some distractions now, so probably it will take a
around two-three more weeks before I submit the code for review.

Sorry for the delay,
Ionela.

> -- 
> viresh