sched/schedutil: Don't set next_freq to UINT_MAX

Commit Message

Rafael J. Wysocki May 9, 2018, 8:56 a.m. UTC

On Wed, May 9, 2018 at 10:41 AM, Viresh Kumar <[e-mail]> wrote:
> On 08-05-18, 22:54, Rafael J. Wysocki wrote:
>> On Tue, May 8, 2018 at 8:42 AM, Viresh Kumar <[e-mail]> wrote:
>> > The schedutil driver sets sg_policy->next_freq to UINT_MAX on certain
>> > occasions:
>> > - In sugov_start(), when the schedutil governor is started for a group
>> >   of CPUs.
>> > - And whenever we need to force a freq update before rate-limit
>> >   duration, which happens when:
>> >   - there is an update in cpufreq policy limits.
>> >   - Or when the utilization of DL scheduling class increases.
>> >
>> > In return, get_next_freq() doesn't return a cached next_freq value but
>> > instead recalculates the next frequency. This has some side effects
>> > though and may significantly delay a required increase in frequency.
>> >
>> > In sugov_update_single() we try to avoid decreasing frequency if the CPU
>> > has not been idle recently. Consider this scenario, the available range
>> > of frequencies for a CPU are from 800 MHz to 2.5 GHz and current
>> > frequency is 800 MHz. From one of the call paths
>> > sg_policy->need_freq_update is set to true and hence
>> > sg_policy->next_freq is set to UINT_MAX. Now if the CPU had been busy,
>> > next_f will always be less than UINT_MAX, whatever the value of next_f
>> > is. And so even when we wanted to increase the frequency, we will
>> > overwrite next_f with UINT_MAX and will not change the frequency
>> > eventually. This will continue until the time CPU stays busy. This isn't
>> > cross checked with any specific test cases, but rather based on general
>> > code review.
>> >
>> > Fix that by not resetting the sg_policy->need_freq_update flag from
>> > sugov_should_update_freq() but get_next_freq() and we wouldn't need to
>> > overwrite sg_policy->next_freq anymore.
>> >
>> > Cc: 4.12+ <[e-mail]> # 4.12+
>> > Fixes: b7eaf1aab9f8 ("cpufreq: schedutil: Avoid reducing frequency of busy CPUs prematurely")
>>
>> The rest of the chantelog is totally disconnected from this commit.
>
> I added the "Fixes" tag because this is exactly the commit after which
> this problem started, isn't it?
>
>> So the problem is that sugov_update_single() doesn't check the special
>> UNIT_MAX value before assigning sg_policy->next_freq to next_f.  Fair
>> enough.
>>
>> I don't see why the patch is the right fix for that, however.
>
> I thought not overwriting next_freq makes things much simpler and easy
> to review.

I'm kind of concerned about updating the limits via sysfs in which
case the cached next frequency may be out of range, so it's better to
invalidate it right away then.

> What else do you have in mind to solve this problem ?

Something like the below?

---
 kernel/sched/cpufreq_schedutil.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Viresh Kumar May 9, 2018, 9:15 a.m. UTC | #1

On 09-05-18, 10:56, Rafael J. Wysocki wrote:
> I'm kind of concerned about updating the limits via sysfs in which
> case the cached next frequency may be out of range, so it's better to
> invalidate it right away then.

That should not be a problem as __cpufreq_driver_target() will anyway
clamp the target frequency to be within limits, whatever the cached
value of next_freq is.

And we aren't invalidating the cached next freq immediately currently
as well, as we are waiting until the next time the util update handler
is called to set sg_policy->next_freq to UINT_MAX.

> > What else do you have in mind to solve this problem ?
> 
> Something like the below?
> 
> ---
>  kernel/sched/cpufreq_schedutil.c |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> Index: linux-pm/kernel/sched/cpufreq_schedutil.c
> ===================================================================
> --- linux-pm.orig/kernel/sched/cpufreq_schedutil.c
> +++ linux-pm/kernel/sched/cpufreq_schedutil.c
> @@ -305,7 +305,8 @@ static void sugov_update_single(struct u
>       * Do not reduce the frequency if the CPU has not been idle
>       * recently, as the reduction is likely to be premature then.
>       */
> -    if (busy && next_f < sg_policy->next_freq) {
> +    if (busy && next_f < sg_policy->next_freq &&
> +        sg_policy->next_freq != UINT_MAX) {
>          next_f = sg_policy->next_freq;
> 
>          /* Reset cached freq as next_freq has changed */

This will fix the problem we have identified currently, but adding a
special meaning to next_freq == UINT_MAX invites more hidden corner
cases like the one we just found. IMHO, using next_freq only for the
*real* frequency values makes its usage more transparent and readable.
And we already have the need_freq_update flag which we can use for
this special purpose, as is done in my patch.

Rafael J. Wysocki May 9, 2018, 9:23 a.m. UTC | #2

On Wed, May 9, 2018 at 11:15 AM, Viresh Kumar <[e-mail]> wrote:
> On 09-05-18, 10:56, Rafael J. Wysocki wrote:
>> I'm kind of concerned about updating the limits via sysfs in which
>> case the cached next frequency may be out of range, so it's better to
>> invalidate it right away then.
>
> That should not be a problem as __cpufreq_driver_target() will anyway
> clamp the target frequency to be within limits, whatever the cached
> value of next_freq is.

The fast switch case doesn't use it, though.

> And we aren't invalidating the cached next freq immediately currently
> as well, as we are waiting until the next time the util update handler
> is called to set sg_policy->next_freq to UINT_MAX.
>
>> > What else do you have in mind to solve this problem ?
>>
>> Something like the below?
>>
>> ---
>>  kernel/sched/cpufreq_schedutil.c |    3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> Index: linux-pm/kernel/sched/cpufreq_schedutil.c
>> ===================================================================
>> --- linux-pm.orig/kernel/sched/cpufreq_schedutil.c
>> +++ linux-pm/kernel/sched/cpufreq_schedutil.c
>> @@ -305,7 +305,8 @@ static void sugov_update_single(struct u
>>       * Do not reduce the frequency if the CPU has not been idle
>>       * recently, as the reduction is likely to be premature then.
>>       */
>> -    if (busy && next_f < sg_policy->next_freq) {
>> +    if (busy && next_f < sg_policy->next_freq &&
>> +        sg_policy->next_freq != UINT_MAX) {
>>          next_f = sg_policy->next_freq;
>>
>>          /* Reset cached freq as next_freq has changed */
>
> This will fix the problem we have identified currently, but adding a
> special meaning to next_freq == UINT_MAX invites more hidden corner
> cases like the one we just found. IMHO, using next_freq only for the
> *real* frequency values makes its usage more transparent and readable.
> And we already have the need_freq_update flag which we can use for
> this special purpose, as is done in my patch.

So I prefer to do the above as a -stable fix and make the UNIT_MAX
change on top of that.

Viresh Kumar May 9, 2018, 9:30 a.m. UTC | #3

On 09-05-18, 11:23, Rafael J. Wysocki wrote:
> On Wed, May 9, 2018 at 11:15 AM, Viresh Kumar <[e-mail]> wrote:
> > On 09-05-18, 10:56, Rafael J. Wysocki wrote:
> >> I'm kind of concerned about updating the limits via sysfs in which
> >> case the cached next frequency may be out of range, so it's better to
> >> invalidate it right away then.
> >
> > That should not be a problem as __cpufreq_driver_target() will anyway
> > clamp the target frequency to be within limits, whatever the cached
> > value of next_freq is.
> 
> The fast switch case doesn't use it, though.

cpufreq_driver_fast_switch() does the same clamping :)

> > And we aren't invalidating the cached next freq immediately currently
> > as well, as we are waiting until the next time the util update handler
> > is called to set sg_policy->next_freq to UINT_MAX.
> >
> >> > What else do you have in mind to solve this problem ?
> >>
> >> Something like the below?
> >>
> >> ---
> >>  kernel/sched/cpufreq_schedutil.c |    3 ++-
> >>  1 file changed, 2 insertions(+), 1 deletion(-)
> >>
> >> Index: linux-pm/kernel/sched/cpufreq_schedutil.c
> >> ===================================================================
> >> --- linux-pm.orig/kernel/sched/cpufreq_schedutil.c
> >> +++ linux-pm/kernel/sched/cpufreq_schedutil.c
> >> @@ -305,7 +305,8 @@ static void sugov_update_single(struct u
> >>       * Do not reduce the frequency if the CPU has not been idle
> >>       * recently, as the reduction is likely to be premature then.
> >>       */
> >> -    if (busy && next_f < sg_policy->next_freq) {
> >> +    if (busy && next_f < sg_policy->next_freq &&
> >> +        sg_policy->next_freq != UINT_MAX) {
> >>          next_f = sg_policy->next_freq;
> >>
> >>          /* Reset cached freq as next_freq has changed */
> >
> > This will fix the problem we have identified currently, but adding a
> > special meaning to next_freq == UINT_MAX invites more hidden corner
> > cases like the one we just found. IMHO, using next_freq only for the
> > *real* frequency values makes its usage more transparent and readable.
> > And we already have the need_freq_update flag which we can use for
> > this special purpose, as is done in my patch.
> 
> So I prefer to do the above as a -stable fix and make the UNIT_MAX
> change on top of that.

Okay, that's fine with me. Will send the next version now :)

Just to make sure, you are fine with the "Fixes" tag now (since you
objected to that earlier) ?

Rafael J. Wysocki May 9, 2018, 9:32 a.m. UTC | #4

On Wed, May 9, 2018 at 11:30 AM, Viresh Kumar <[e-mail]> wrote:
> On 09-05-18, 11:23, Rafael J. Wysocki wrote:
>> On Wed, May 9, 2018 at 11:15 AM, Viresh Kumar <[e-mail]> wrote:
>> > On 09-05-18, 10:56, Rafael J. Wysocki wrote:
>> >> I'm kind of concerned about updating the limits via sysfs in which
>> >> case the cached next frequency may be out of range, so it's better to
>> >> invalidate it right away then.
>> >
>> > That should not be a problem as __cpufreq_driver_target() will anyway
>> > clamp the target frequency to be within limits, whatever the cached
>> > value of next_freq is.
>>
>> The fast switch case doesn't use it, though.
>
> cpufreq_driver_fast_switch() does the same clamping :)
>
>> > And we aren't invalidating the cached next freq immediately currently
>> > as well, as we are waiting until the next time the util update handler
>> > is called to set sg_policy->next_freq to UINT_MAX.
>> >
>> >> > What else do you have in mind to solve this problem ?
>> >>
>> >> Something like the below?
>> >>
>> >> ---
>> >>  kernel/sched/cpufreq_schedutil.c |    3 ++-
>> >>  1 file changed, 2 insertions(+), 1 deletion(-)
>> >>
>> >> Index: linux-pm/kernel/sched/cpufreq_schedutil.c
>> >> ===================================================================
>> >> --- linux-pm.orig/kernel/sched/cpufreq_schedutil.c
>> >> +++ linux-pm/kernel/sched/cpufreq_schedutil.c
>> >> @@ -305,7 +305,8 @@ static void sugov_update_single(struct u
>> >>       * Do not reduce the frequency if the CPU has not been idle
>> >>       * recently, as the reduction is likely to be premature then.
>> >>       */
>> >> -    if (busy && next_f < sg_policy->next_freq) {
>> >> +    if (busy && next_f < sg_policy->next_freq &&
>> >> +        sg_policy->next_freq != UINT_MAX) {
>> >>          next_f = sg_policy->next_freq;
>> >>
>> >>          /* Reset cached freq as next_freq has changed */
>> >
>> > This will fix the problem we have identified currently, but adding a
>> > special meaning to next_freq == UINT_MAX invites more hidden corner
>> > cases like the one we just found. IMHO, using next_freq only for the
>> > *real* frequency values makes its usage more transparent and readable.
>> > And we already have the need_freq_update flag which we can use for
>> > this special purpose, as is done in my patch.
>>
>> So I prefer to do the above as a -stable fix and make the UNIT_MAX
>> change on top of that.
>
> Okay, that's fine with me. Will send the next version now :)
>
> Just to make sure, you are fine with the "Fixes" tag now (since you
> objected to that earlier) ?

OK, so to be clear, I'm going to queue up the simple patch I posted
with a FIxes: tag.  I'll resend it with a changelog shortly.

Then please send a UINT_MAX change on top of that and it won't be an
urgent fix any more.

Patch

Index: linux-pm/kernel/sched/cpufreq_schedutil.c
===================================================================
--- linux-pm.orig/kernel/sched/cpufreq_schedutil.c
+++ linux-pm/kernel/sched/cpufreq_schedutil.c
@@ -305,7 +305,8 @@  static void sugov_update_single(struct u
      * Do not reduce the frequency if the CPU has not been idle
      * recently, as the reduction is likely to be premature then.
      */
-    if (busy && next_f < sg_policy->next_freq) {
+    if (busy && next_f < sg_policy->next_freq &&
+        sg_policy->next_freq != UINT_MAX) {
         next_f = sg_policy->next_freq;

         /* Reset cached freq as next_freq has changed */