diff mbox

[v1,1/2] intel_pstate: Use the cpu load to determine the PercentPerformance

Message ID 1446542840-14982-1-git-send-email-philippe.longepe@linux.intel.com (mailing list archive)
State Superseded, archived
Headers show

Commit Message

plongepe Nov. 3, 2015, 9:27 a.m. UTC 
Aperf and Mperf counter are not enough to determine the Target P-state
because they measure performance only when the targeted processor is
in the C0 state (active state).
Because of that, we were computing the average P-state during the last
period which can be very different from the average frequency
(or percentage of performance).

As defined in the SDM (section 14.2), the PercentPerformance is defined by:

PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);

The PercentBusy (or load) can be estimated as the ratio of the mperf
counter running at a constant frequency only during active periods (C0)
and the time stamp counter running at the same frequency but also
during idle.

So, PercentBusy = 100 * (delta_mperf / delta_tsc)

and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
				(delta_aperf / delta_mperf)
That can be simplified with:
PercentPerformance = 100 * (delta_aperf / delta_tsc)

Signed-off-by: Philippe Longepe <philippe.longepe@linux.intel.com>
Signed-off-by: Stephane Gasparini <stephane.gasparini@linux.intel.com>
---
 drivers/cpufreq/intel_pstate.c | 87 +++++++++++++++---------------------------
 1 file changed, 31 insertions(+), 56 deletions(-)

Comments

Rafael J. Wysocki Nov. 7, 2015, 1:09 a.m. UTC | #1 
On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
> Aperf and Mperf counter are not enough to determine the Target P-state
> because they measure performance only when the targeted processor is
> in the C0 state (active state).
> Because of that, we were computing the average P-state during the last
> period which can be very different from the average frequency
> (or percentage of performance).
> 
> As defined in the SDM (section 14.2), the PercentPerformance is defined by:
> 
> PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);
> 
> The PercentBusy (or load) can be estimated as the ratio of the mperf
> counter running at a constant frequency only during active periods (C0)
> and the time stamp counter running at the same frequency but also
> during idle.
> 
> So, PercentBusy = 100 * (delta_mperf / delta_tsc)
> 
> and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
> 				(delta_aperf / delta_mperf)
> That can be simplified with:
> PercentPerformance = 100 * (delta_aperf / delta_tsc)
> 
> Signed-off-by: Philippe Longepe <philippe.longepe@linux.intel.com>
> Signed-off-by: Stephane Gasparini <stephane.gasparini@linux.intel.com>

Srinivas, I need your ACK here and for the [2/2].

Thanks,
Rafael

--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
srinivas pandruvada Nov. 7, 2015, 1:14 a.m. UTC | #2 
On Sat, 2015-11-07 at 02:09 +0100, Rafael J. Wysocki wrote:
> On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
> > Aperf and Mperf counter are not enough to determine the Target P
> > -state
> > because they measure performance only when the targeted processor
> > is
> > in the C0 state (active state).
> > Because of that, we were computing the average P-state during the
> > last
> > period which can be very different from the average frequency
> > (or percentage of performance).
> > 
> > As defined in the SDM (section 14.2), the PercentPerformance is
> > defined by:
> > 
> > PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);
> > 
> > The PercentBusy (or load) can be estimated as the ratio of the
> > mperf
> > counter running at a constant frequency only during active periods
> > (C0)
> > and the time stamp counter running at the same frequency but also
> > during idle.
> > 
> > So, PercentBusy = 100 * (delta_mperf / delta_tsc)
> > 
> > and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
> > 				(delta_aperf / delta_mperf)
> > That can be simplified with:
> > PercentPerformance = 100 * (delta_aperf / delta_tsc)
> > 
> > Signed-off-by: Philippe Longepe <philippe.longepe@linux.intel.com>
> > Signed-off-by: Stephane Gasparini <
> > stephane.gasparini@linux.intel.com>
> 
> Srinivas, I need your ACK here and for the [2/2].
> 
These needs further validation to merge to upstream kernel.

Thanks,
Srinivas


> Thanks,
> Rafael
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Doug Smythies Nov. 21, 2015, 4:21 p.m. UTC | #3 
Philippe, Stephane,

The resulting load / CPU frequency response curve has some interesting qualities.
The resulting step function load response time is much much faster than
the current solution, and yet there is not much tendency for the system to oscillate,
or at least the oscillation magnitude is smallish.

I agree that some sort of real load calculation needs to be brought back to the intel_pstate driver.
  
On 2015.11.06 17:14 Srinivas Pandruvada wrote:
On Sat, 2015-11-07 at 02:09 +0100, Rafael J. Wysocki wrote:
> On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
>> Aperf and Mperf counter are not enough to determine the Target P
>> -state
>> because they measure performance only when the targeted processor
>> is
>> in the C0 state (active state).
>> Because of that, we were computing the average P-state during the
>> last
>> period which can be very different from the average frequency
>> (or percentage of performance).
>> 
>> As defined in the SDM (section 14.2), the PercentPerformance is
>> defined by:
>> 
>> PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);
>> 
>> The PercentBusy (or load) can be estimated as the ratio of the
>> mperf
>> counter running at a constant frequency only during active periods
>> (C0)
>> and the time stamp counter running at the same frequency but also
>> during idle.
>> 
>> So, PercentBusy = 100 * (delta_mperf / delta_tsc)
>> 
>> and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
>> 				(delta_aperf / delta_mperf)
>> That can be simplified with:
>> PercentPerformance = 100 * (delta_aperf / delta_tsc)

Yes, but the last time I tried to bring back actual load calculations
In a similar way it was pointed out that one should not do it this way.
[1] SDM also states:

"Only the IA32_APERF/IA32_MPERF ratio is architecturally defined;
software should not attach meaning to the content of the individual of IA32_APERF or IA32_MPERF MSRs."

Note that I have been working on a "legal" way to calculate load [2] and
I have been using the method for about a month now. (note based on a
different patch set starting point).

>> -	duration_us = ktime_us_delta(cpu->sample.time,
>> -				     cpu->last_sample_time);
>> -	if (duration_us > sample_time * 3) {
>> -		sample_ratio = div_fp(int_tofp(sample_time),
>> -				      int_tofp(duration_us));
>> -		core_busy = mul_fp(core_busy, sample_ratio);

While the duration stuff has issues that can sometimes result in incorrectly driving the
target pstate downwards, rather than deleting it there might still be value in fixing
it using the actual load information.

References:
[1] http://marc.info/?l=linux-pm&m=143094005601231&w=2
[2] double u double u double u .dot smythies dot com/~doug/linux/intel_pstate/build58/0006-intel_pstate-Change-utilization-calculation-to-a-leg.txt


--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
plongepe Nov. 23, 2015, 1:28 p.m. UTC | #4 
Hi Smythies,

Yes, the cpu load gives good results for almost all use cases except IO 
workloads
(it needs an additionnal patch to boost the IOs).

That's why we will enable it only on Atoms first but not yet on 
core/servers (more
validation is needed).


On 21/11/2015 17:21, Doug Smythies wrote:
> Philippe, Stephane,
>
> The resulting load / CPU frequency response curve has some interesting qualities.
> The resulting step function load response time is much much faster than
> the current solution, and yet there is not much tendency for the system to oscillate,
> or at least the oscillation magnitude is smallish.
>
> I agree that some sort of real load calculation needs to be brought back to the intel_pstate driver.
>    
> On 2015.11.06 17:14 Srinivas Pandruvada wrote:
> On Sat, 2015-11-07 at 02:09 +0100, Rafael J. Wysocki wrote:
>> On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
>>> Aperf and Mperf counter are not enough to determine the Target P
>>> -state
>>> because they measure performance only when the targeted processor
>>> is
>>> in the C0 state (active state).
>>> Because of that, we were computing the average P-state during the
>>> last
>>> period which can be very different from the average frequency
>>> (or percentage of performance).
>>>
>>> As defined in the SDM (section 14.2), the PercentPerformance is
>>> defined by:
>>>
>>> PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);
>>>
>>> The PercentBusy (or load) can be estimated as the ratio of the
>>> mperf
>>> counter running at a constant frequency only during active periods
>>> (C0)
>>> and the time stamp counter running at the same frequency but also
>>> during idle.
>>>
>>> So, PercentBusy = 100 * (delta_mperf / delta_tsc)
>>>
>>> and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
>>> 				(delta_aperf / delta_mperf)
>>> That can be simplified with:
>>> PercentPerformance = 100 * (delta_aperf / delta_tsc)
> Yes, but the last time I tried to bring back actual load calculations
> In a similar way it was pointed out that one should not do it this way.
> [1] SDM also states:
>
> "Only the IA32_APERF/IA32_MPERF ratio is architecturally defined;
> software should not attach meaning to the content of the individual of IA32_APERF or IA32_MPERF MSRs."
Individual counters are not architecturally defined but their ratio is 
defined.
MERF and TSC counters are both counting at the same frequency (max non 
turbo freq)
but MPERF is counting only in active state (C0) and TSC is counting all 
the time.
So, delta_mperf/delta_tsc is representing the load.
>
> Note that I have been working on a "legal" way to calculate load [2] and
> I have been using the method for about a month now. (note based on a
> different patch set starting point).
>
>>> -	duration_us = ktime_us_delta(cpu->sample.time,
>>> -				     cpu->last_sample_time);
>>> -	if (duration_us > sample_time * 3) {
>>> -		sample_ratio = div_fp(int_tofp(sample_time),
>>> -				      int_tofp(duration_us));
>>> -		core_busy = mul_fp(core_busy, sample_ratio);
> While the duration stuff has issues that can sometimes result in incorrectly driving the
> target pstate downwards, rather than deleting it there might still be value in fixing
> it using the actual load information.
>
> References:
> [1] http://marc.info/?l=linux-pm&m=143094005601231&w=2
> [2] double u double u double u .dot smythies dot com/~doug/linux/intel_pstate/build58/0006-intel_pstate-Change-utilization-calculation-to-a-leg.txt
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Doug Smythies Nov. 24, 2015, 1:33 a.m. UTC | #5 
On 2015.11.23 05:29 plongepe wrote:
> On 21/11/2015 17:21, Doug Smythies wrote:
>> On 2015.11.06 17:14 Srinivas Pandruvada wrote:
>> On Sat, 2015-11-07 at 02:09 +0100, Rafael J. Wysocki wrote:
>>> On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
>>>> Aperf and Mperf counter are not enough to determine the Target P
>>>> -state
>>>> because they measure performance only when the targeted processor
>>>> is
>>>> in the C0 state (active state).
>>>> Because of that, we were computing the average P-state during the
>>>> last
>>>> period which can be very different from the average frequency
>>>> (or percentage of performance).
>>>>
>>>> As defined in the SDM (section 14.2), the PercentPerformance is
>>>> defined by:
>>>>
>>>> PercentPerformance = PercentBusy * (delta_aperf / delta_mperf);
>>>>
>>>> The PercentBusy (or load) can be estimated as the ratio of the
>>>> mperf
>>>> counter running at a constant frequency only during active periods
>>>> (C0)
>>>> and the time stamp counter running at the same frequency but also
>>>> during idle.
>>>>
>>>> So, PercentBusy = 100 * (delta_mperf / delta_tsc)
>>>>
>>>> and, PercentPerformance = 100 * (delta_mperf / delta_tsc) *
>>>> 				(delta_aperf / delta_mperf)
>>>> That can be simplified with:
>>>> PercentPerformance = 100 * (delta_aperf / delta_tsc)
>>
>> Yes, but the last time I tried to bring back actual load calculations
>> In a similar way it was pointed out that one should not do it this way.
>> [1] SDM also states:
>>
>> "Only the IA32_APERF/IA32_MPERF ratio is architecturally defined;
>> software should not attach meaning to the content of the individual of IA32_APERF or IA32_MPERF MSRs."
>
> Individual counters are not architecturally defined but their ratio is 
> defined.
> MERF and TSC counters are both counting at the same frequency (max non 
> turbo freq)
> but MPERF is counting only in active state (C0) and TSC is counting all 
> the time.
> So, delta_mperf/delta_tsc is representing the load.

I agree.
However, Intel should decide once and for all if mperf / tsc is allowed or not.
It was used one time, then it was deleted, then I tried to bring it back,
then I was told it wasn't allowed, now you are trying to bring it back.

... Doug


--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
srinivas pandruvada Nov. 24, 2015, 1:44 a.m. UTC | #6 
On Mon, 2015-11-23 at 17:33 -0800, Doug Smythies wrote:
> On 2015.11.23 05:29 plongepe wrote:
> > On 21/11/2015 17:21, Doug Smythies wrote:
> >> On 2015.11.06 17:14 Srinivas Pandruvada wrote:
> >> On Sat, 2015-11-07 at 02:09 +0100, Rafael J. Wysocki wrote:
> >>> On Tuesday, November 03, 2015 10:27:19 AM Philippe Longepe wrote:
[CUT.]
> However, Intel should decide once and for all if mperf / tsc is allowed or not.
We can't apply this uniformly across all CPUs. So this patchset will not
be applied to all CPUs, only where this is allowed and verified. 
BTW, these patches need cleanup.

Thanks,
Srinivas
> It was used one time, then it was deleted, then I tried to bring it back,
> then I was told it wasn't allowed, now you are trying to bring it back.
> 
> ... Doug
> 
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-pm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c
index 93a3c63..421903f 100644
--- a/drivers/cpufreq/intel_pstate.c
+++ b/drivers/cpufreq/intel_pstate.c
@@ -69,7 +69,7 @@  static inline int ceiling_fp(int32_t x)
 }
 
 struct sample {
-	int32_t core_pct_busy;
+	int32_t cpu_load;
 	u64 aperf;
 	u64 mperf;
 	u64 tsc;
@@ -993,21 +993,39 @@  static void intel_pstate_get_cpu_pstates(struct cpudata *cpu)
 	intel_pstate_set_pstate(cpu, cpu->pstate.min_pstate, false);
 }
 
-static inline void intel_pstate_calc_busy(struct cpudata *cpu)
+static inline int32_t intel_pstate_calc_busy(struct cpudata *cpu)
 {
 	struct sample *sample = &cpu->sample;
-	int64_t core_pct;
+	struct pstate_data *pstate = &cpu->pstate;
+	int64_t core_busy_ratio;
 
-	core_pct = int_tofp(sample->aperf) * int_tofp(100);
-	core_pct = div64_u64(core_pct, int_tofp(sample->mperf));
+	/*
+	 * The load can be estimated as the ratio of the mperf counter
+	 * running at a constant frequency only during active periods
+	 * (C0) and the time stamp counter running at the same frequency
+	 * also during C-states.
+	 */
+	sample->cpu_load = div64_u64(100 * sample->mperf, sample->tsc);
+
+	/*
+	 * The target P-state can be estimated with the following formula:
+	 * PercentPerformance = PercentBusy * (delta_aperf/delta_mperf);
+	 * (see Section 14.2 from Intel Software Developer Manual)
+	 * with PercentBusy = 100 * (delta_mperf / delta_tsc) and
+	 * PercentPerformance can be simplified with:
+	 * (delta_mperf * delta_aperf) / (delta_tsc * delta_mperf) =
+	 * delta_aperf / delta_tsc. Finally, we normalize core_busy_ratio,
+	 * which was our actual percent performance to what we requested
+	 * during the last sample period. The result will be a percentage of
+	 * busy at a specified pstate.
+	 */
+	core_busy_ratio = div64_u64(int_tofp(100) * sample->aperf *
+		pstate->max_pstate, sample->tsc * pstate->current_pstate);
 
-	sample->freq = fp_toint(
-		mul_fp(int_tofp(
-			cpu->pstate.max_pstate_physical *
-			cpu->pstate.scaling / 100),
-			core_pct));
+	sample->freq = div64_u64(sample->aperf * pstate->max_pstate *
+		pstate->scaling, sample->mperf);
 
-	sample->core_pct_busy = (int32_t)core_pct;
+	return core_busy_ratio;
 }
 
 static inline void intel_pstate_sample(struct cpudata *cpu)
@@ -1036,8 +1054,6 @@  static inline void intel_pstate_sample(struct cpudata *cpu)
 	cpu->sample.mperf -= cpu->prev_mperf;
 	cpu->sample.tsc -= cpu->prev_tsc;
 
-	intel_pstate_calc_busy(cpu);
-
 	cpu->prev_aperf = aperf;
 	cpu->prev_mperf = mperf;
 	cpu->prev_tsc = tsc;
@@ -1059,47 +1075,6 @@  static inline void intel_pstate_set_sample_time(struct cpudata *cpu)
 	mod_timer_pinned(&cpu->timer, jiffies + delay);
 }
 
-static inline int32_t intel_pstate_get_scaled_busy(struct cpudata *cpu)
-{
-	int32_t core_busy, max_pstate, current_pstate, sample_ratio;
-	s64 duration_us;
-	u32 sample_time;
-
-	/*
-	 * core_busy is the ratio of actual performance to max
-	 * max_pstate is the max non turbo pstate available
-	 * current_pstate was the pstate that was requested during
-	 * 	the last sample period.
-	 *
-	 * We normalize core_busy, which was our actual percent
-	 * performance to what we requested during the last sample
-	 * period. The result will be a percentage of busy at a
-	 * specified pstate.
-	 */
-	core_busy = cpu->sample.core_pct_busy;
-	max_pstate = int_tofp(cpu->pstate.max_pstate_physical);
-	current_pstate = int_tofp(cpu->pstate.current_pstate);
-	core_busy = mul_fp(core_busy, div_fp(max_pstate, current_pstate));
-
-	/*
-	 * Since we have a deferred timer, it will not fire unless
-	 * we are in C0.  So, determine if the actual elapsed time
-	 * is significantly greater (3x) than our sample interval.  If it
-	 * is, then we were idle for a long enough period of time
-	 * to adjust our busyness.
-	 */
-	sample_time = pid_params.sample_rate_ms  * USEC_PER_MSEC;
-	duration_us = ktime_us_delta(cpu->sample.time,
-				     cpu->last_sample_time);
-	if (duration_us > sample_time * 3) {
-		sample_ratio = div_fp(int_tofp(sample_time),
-				      int_tofp(duration_us));
-		core_busy = mul_fp(core_busy, sample_ratio);
-	}
-
-	return core_busy;
-}
-
 static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu)
 {
 	int32_t busy_scaled;
@@ -1111,7 +1086,7 @@  static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu)
 	from = cpu->pstate.current_pstate;
 
 	pid = &cpu->pid;
-	busy_scaled = intel_pstate_get_scaled_busy(cpu);
+	busy_scaled = intel_pstate_calc_busy(cpu);
 
 	ctl = pid_calc(pid, busy_scaled);
 
@@ -1119,7 +1094,7 @@  static inline void intel_pstate_adjust_busy_pstate(struct cpudata *cpu)
 	intel_pstate_set_pstate(cpu, cpu->pstate.current_pstate - ctl, true);
 
 	sample = &cpu->sample;
-	trace_pstate_sample(fp_toint(sample->core_pct_busy),
+	trace_pstate_sample(fp_toint(busy_scaled),
 		fp_toint(busy_scaled),
 		from,
 		cpu->pstate.current_pstate,