From patchwork Wed Jul 1 09:07:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635843 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29DD76C1 for ; Wed, 1 Jul 2020 09:09:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1424D2077D for ; Wed, 1 Jul 2020 09:09:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729165AbgGAJJ1 (ORCPT ); Wed, 1 Jul 2020 05:09:27 -0400 Received: from foss.arm.com ([217.140.110.172]:47742 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726255AbgGAJJ0 (ORCPT ); Wed, 1 Jul 2020 05:09:26 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4A83E1045; Wed, 1 Jul 2020 02:09:25 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 85CE13F68F; Wed, 1 Jul 2020 02:09:23 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/8] cpufreq: allow drivers to flag custom support for freq invariance Date: Wed, 1 Jul 2020 10:07:44 +0100 Message-Id: <20200701090751.7543-2-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The scheduler's Frequency Invariance Engine (FIE) is providing a frequency scale correction factor that helps achieve more accurate load-tracking by conveying information about the currently selected frequency relative to the maximum supported frequency of a CPU. In some cases this is achieved by passing information from cpufreq drivers about the frequency selection done by cpufreq. Given that most drivers follow a similar process of selecting and setting of frequency, there is a strong case for moving the setting of the frequency scale factor from the cpufreq drivers frequency switch callbacks (target_index() and fast_switch()), to the cpufreq core functions that call them. In preparation for this, acknowledge that there are still drivers who's frequency setting process is custom and therefore these drivers will want to provide and flag custom support for the setting of the scheduler's frequency invariance (FI) scale factor as well. Prepare for this by introducing a new flag: CPUFREQ_CUSTOM_SET_FREQ_SCALE. Examples of users of this flag are: - drivers that do not implement the callbacks that lend themselves to triggering the setting of the FI scale factor, - drivers that implement the appropriate callbacks but which have an atypical implementation. Currently, given that all drivers call arch_set_freq_scale() directly, flag all users with CPUFREQ_CUSTOM_SET_FREQ_SCALE. These driver changes are also useful to maintain bisection between the FI switch from the drivers to the core. Signed-off-by: Ionela Voinescu Cc: Rafael J. Wysocki Cc: Viresh Kumar --- drivers/cpufreq/cpufreq-dt.c | 3 ++- drivers/cpufreq/qcom-cpufreq-hw.c | 3 ++- drivers/cpufreq/scmi-cpufreq.c | 3 ++- drivers/cpufreq/scpi-cpufreq.c | 3 ++- drivers/cpufreq/vexpress-spc-cpufreq.c | 3 ++- include/linux/cpufreq.h | 10 +++++++++- 6 files changed, 19 insertions(+), 6 deletions(-) diff --git a/drivers/cpufreq/cpufreq-dt.c b/drivers/cpufreq/cpufreq-dt.c index 944d7b45afe9..8e0571a49d1e 100644 --- a/drivers/cpufreq/cpufreq-dt.c +++ b/drivers/cpufreq/cpufreq-dt.c @@ -331,7 +331,8 @@ static int cpufreq_exit(struct cpufreq_policy *policy) static struct cpufreq_driver dt_cpufreq_driver = { .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | - CPUFREQ_IS_COOLING_DEV, + CPUFREQ_IS_COOLING_DEV | + CPUFREQ_CUSTOM_SET_FREQ_SCALE, .verify = cpufreq_generic_frequency_table_verify, .target_index = set_target, .get = cpufreq_generic_get, diff --git a/drivers/cpufreq/qcom-cpufreq-hw.c b/drivers/cpufreq/qcom-cpufreq-hw.c index 573630c23aca..e13780beb373 100644 --- a/drivers/cpufreq/qcom-cpufreq-hw.c +++ b/drivers/cpufreq/qcom-cpufreq-hw.c @@ -337,7 +337,8 @@ static struct freq_attr *qcom_cpufreq_hw_attr[] = { static struct cpufreq_driver cpufreq_qcom_hw_driver = { .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | - CPUFREQ_IS_COOLING_DEV, + CPUFREQ_IS_COOLING_DEV | + CPUFREQ_CUSTOM_SET_FREQ_SCALE, .verify = cpufreq_generic_frequency_table_verify, .target_index = qcom_cpufreq_hw_target_index, .get = qcom_cpufreq_hw_get, diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c index fb42e3390377..16ab4ecc75e4 100644 --- a/drivers/cpufreq/scmi-cpufreq.c +++ b/drivers/cpufreq/scmi-cpufreq.c @@ -223,7 +223,8 @@ static struct cpufreq_driver scmi_cpufreq_driver = { .name = "scmi", .flags = CPUFREQ_STICKY | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | - CPUFREQ_IS_COOLING_DEV, + CPUFREQ_IS_COOLING_DEV | + CPUFREQ_CUSTOM_SET_FREQ_SCALE, .verify = cpufreq_generic_frequency_table_verify, .attr = cpufreq_generic_attr, .target_index = scmi_cpufreq_set_target, diff --git a/drivers/cpufreq/scpi-cpufreq.c b/drivers/cpufreq/scpi-cpufreq.c index b0f5388b8854..6b5f56dc3ca3 100644 --- a/drivers/cpufreq/scpi-cpufreq.c +++ b/drivers/cpufreq/scpi-cpufreq.c @@ -197,7 +197,8 @@ static struct cpufreq_driver scpi_cpufreq_driver = { .name = "scpi-cpufreq", .flags = CPUFREQ_STICKY | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | - CPUFREQ_IS_COOLING_DEV, + CPUFREQ_IS_COOLING_DEV | + CPUFREQ_CUSTOM_SET_FREQ_SCALE, .verify = cpufreq_generic_frequency_table_verify, .attr = cpufreq_generic_attr, .get = scpi_cpufreq_get_rate, diff --git a/drivers/cpufreq/vexpress-spc-cpufreq.c b/drivers/cpufreq/vexpress-spc-cpufreq.c index 4e8b1dee7c9a..e0a1a3367ec5 100644 --- a/drivers/cpufreq/vexpress-spc-cpufreq.c +++ b/drivers/cpufreq/vexpress-spc-cpufreq.c @@ -496,7 +496,8 @@ static struct cpufreq_driver ve_spc_cpufreq_driver = { .name = "vexpress-spc", .flags = CPUFREQ_STICKY | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | - CPUFREQ_NEED_INITIAL_FREQ_CHECK, + CPUFREQ_NEED_INITIAL_FREQ_CHECK | + CPUFREQ_CUSTOM_SET_FREQ_SCALE, .verify = cpufreq_generic_frequency_table_verify, .target_index = ve_spc_cpufreq_set_target, .get = ve_spc_cpufreq_get_rate, diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 3494f6763597..42668588f9f8 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -293,7 +293,7 @@ __ATTR(_name, 0644, show_##_name, store_##_name) struct cpufreq_driver { char name[CPUFREQ_NAME_LEN]; - u8 flags; + u16 flags; void *driver_data; /* needed by all drivers */ @@ -417,6 +417,14 @@ struct cpufreq_driver { */ #define CPUFREQ_IS_COOLING_DEV BIT(7) +/* + * Set by drivers which implement the necessary calls to the scheduler's + * frequency invariance engine. The use of this flag will result in the + * default arch_set_freq_scale calls being skipped in favour of custom + * driver calls. + */ +#define CPUFREQ_CUSTOM_SET_FREQ_SCALE BIT(8) + int cpufreq_register_driver(struct cpufreq_driver *driver_data); int cpufreq_unregister_driver(struct cpufreq_driver *driver_data); From patchwork Wed Jul 1 09:07:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE716618 for ; Wed, 1 Jul 2020 09:09:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A0A032083E for ; Wed, 1 Jul 2020 09:09:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728969AbgGAJJc (ORCPT ); Wed, 1 Jul 2020 05:09:32 -0400 Received: from foss.arm.com ([217.140.110.172]:47772 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726255AbgGAJJb (ORCPT ); Wed, 1 Jul 2020 05:09:31 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EDD0B106F; Wed, 1 Jul 2020 02:09:30 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3489D3F68F; Wed, 1 Jul 2020 02:09:29 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/8] cpufreq: move invariance setter calls in cpufreq core Date: Wed, 1 Jul 2020 10:07:45 +0100 Message-Id: <20200701090751.7543-3-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Valentin Schneider To properly scale its per-entity load-tracking signals, the task scheduler needs to be given a frequency scale factor, i.e. some image of the current frequency the CPU is running at. Currently, this scale can be computed either by using counters (APERF/MPERF on x86, AMU on arm64), or by piggy-backing on the frequency selection done by cpufreq. For the latter, drivers have to explicitly set the scale factor themselves, despite it being purely boiler-plate code: the required information depends entirely on the kind of frequency switch callback implemented by the driver, i.e. either of: target_index(), target(), fast_switch() and setpolicy(). The fitness of those callbacks with regard to driving the Frequency Invariance Engine (FIE) is studied below: target_index() ============== Documentation states that the chosen frequency "must be determined by freq_table[index].frequency". It isn't clear if it *has* to be that frequency, or if it can use that frequency value to do some computation that ultimately leads to a different frequency selection. All drivers go for the former, while the vexpress-spc-cpufreq has an atypical implementation. Thefore, the hook works on the asusmption the core can use freq_table[index].frequency. target() ======= This has been flagged as deprecated since: commit 9c0ebcf78fde ("cpufreq: Implement light weight ->target_index() routine") It also doesn't have that many users: cpufreq-nforce2.c:371:2: .target = nforce2_target, cppc_cpufreq.c:416:2: .target = cppc_cpufreq_set_target, pcc-cpufreq.c:573:2: .target = pcc_cpufreq_target, Should we care about drivers using this hook, we may be able to exploit cpufreq_freq_transition_{being, end}(). Otherwise, if FIE support is desired in their current state, arch_set_freq_scale() could still be called directly by the driver, while CPUFREQ_CUSTOM_SET_FREQ_SCALE could be used to mark support for it. fast_switch() ============= This callback *has* to return the frequency that was selected. setpolicy() =========== This callback does not have any designated way of informing what was the end choice. But there are only two drivers using setpolicy(), and none of them have current FIE support: drivers/cpufreq/longrun.c:281: .setpolicy = longrun_set_policy, drivers/cpufreq/intel_pstate.c:2215: .setpolicy = intel_pstate_set_policy, The intel_pstate is known to use counter-driven frequency invariance. If FIE support is desired in their current state, arch_set_freq_scale() could still be called directly by the driver, while CPUFREQ_CUSTOM_SET_FREQ_SCALE could be used to mark support for it. Conclusion ========== Given that the significant majority of current FIE enabled drivers use callbacks that lend themselves to triggering the setting of the FIE scale factor in a generic way, move the invariance setter calls to cpufreq core, while filtering drivers that flag custom support using CPUFREQ_CUSTOM_SET_FREQ_SCALE. Signed-off-by: Valentin Schneider Signed-off-by: Ionela Voinescu Cc: Rafael J. Wysocki Cc: Viresh Kumar --- drivers/cpufreq/cpufreq.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index 0128de3603df..83b58483a39b 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -2046,9 +2046,16 @@ EXPORT_SYMBOL(cpufreq_unregister_notifier); unsigned int cpufreq_driver_fast_switch(struct cpufreq_policy *policy, unsigned int target_freq) { + unsigned int freq; + target_freq = clamp_val(target_freq, policy->min, policy->max); + freq = cpufreq_driver->fast_switch(policy, target_freq); + + if (freq && !(cpufreq_driver->flags & CPUFREQ_CUSTOM_SET_FREQ_SCALE)) + arch_set_freq_scale(policy->related_cpus, freq, + policy->cpuinfo.max_freq); - return cpufreq_driver->fast_switch(policy, target_freq); + return freq; } EXPORT_SYMBOL_GPL(cpufreq_driver_fast_switch); @@ -2140,7 +2147,7 @@ int __cpufreq_driver_target(struct cpufreq_policy *policy, unsigned int relation) { unsigned int old_target_freq = target_freq; - int index; + int index, retval; if (cpufreq_disabled()) return -ENODEV; @@ -2171,7 +2178,14 @@ int __cpufreq_driver_target(struct cpufreq_policy *policy, index = cpufreq_frequency_table_target(policy, target_freq, relation); - return __target_index(policy, index); + retval = __target_index(policy, index); + + if (!retval && !(cpufreq_driver->flags & CPUFREQ_CUSTOM_SET_FREQ_SCALE)) + arch_set_freq_scale(policy->related_cpus, + policy->freq_table[index].frequency, + policy->cpuinfo.max_freq); + + return retval; } EXPORT_SYMBOL_GPL(__cpufreq_driver_target); From patchwork Wed Jul 1 09:07:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E33CD618 for ; Wed, 1 Jul 2020 09:09:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7A4D2077D for ; Wed, 1 Jul 2020 09:09:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729469AbgGAJJn (ORCPT ); Wed, 1 Jul 2020 05:09:43 -0400 Received: from foss.arm.com ([217.140.110.172]:47798 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729413AbgGAJJf (ORCPT ); Wed, 1 Jul 2020 05:09:35 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 63272113E; Wed, 1 Jul 2020 02:09:34 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9B2323F68F; Wed, 1 Jul 2020 02:09:32 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/8] cpufreq,drivers: remove setting of frequency scale factor Date: Wed, 1 Jul 2020 10:07:46 +0100 Message-Id: <20200701090751.7543-4-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org As a result of setting the frequency scale factor in cpufreq core, after callbacks that lend themselves to trigger it, remove this functionality from the driver side. For these drivers, the CPUFREQ_CUSTOM_SET_FREQ_SCALE flag is also removed, to enable the use of the generic code on the cpufreq core side. Signed-off-by: Ionela Voinescu Cc: Rafael J. Wysocki Cc: Viresh Kumar --- drivers/cpufreq/cpufreq-dt.c | 13 ++----------- drivers/cpufreq/qcom-cpufreq-hw.c | 12 ++---------- drivers/cpufreq/scmi-cpufreq.c | 21 ++++++--------------- drivers/cpufreq/scpi-cpufreq.c | 6 +----- 4 files changed, 11 insertions(+), 41 deletions(-) diff --git a/drivers/cpufreq/cpufreq-dt.c b/drivers/cpufreq/cpufreq-dt.c index 8e0571a49d1e..9fd4ce774f12 100644 --- a/drivers/cpufreq/cpufreq-dt.c +++ b/drivers/cpufreq/cpufreq-dt.c @@ -40,16 +40,8 @@ static int set_target(struct cpufreq_policy *policy, unsigned int index) { struct private_data *priv = policy->driver_data; unsigned long freq = policy->freq_table[index].frequency; - int ret; - - ret = dev_pm_opp_set_rate(priv->cpu_dev, freq * 1000); - if (!ret) { - arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); - } - - return ret; + return dev_pm_opp_set_rate(priv->cpu_dev, freq * 1000); } /* @@ -331,8 +323,7 @@ static int cpufreq_exit(struct cpufreq_policy *policy) static struct cpufreq_driver dt_cpufreq_driver = { .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | - CPUFREQ_IS_COOLING_DEV | - CPUFREQ_CUSTOM_SET_FREQ_SCALE, + CPUFREQ_IS_COOLING_DEV, .verify = cpufreq_generic_frequency_table_verify, .target_index = set_target, .get = cpufreq_generic_get, diff --git a/drivers/cpufreq/qcom-cpufreq-hw.c b/drivers/cpufreq/qcom-cpufreq-hw.c index e13780beb373..e5d1ee7746a4 100644 --- a/drivers/cpufreq/qcom-cpufreq-hw.c +++ b/drivers/cpufreq/qcom-cpufreq-hw.c @@ -85,8 +85,6 @@ static int qcom_cpufreq_hw_target_index(struct cpufreq_policy *policy, if (icc_scaling_enabled) qcom_cpufreq_set_bw(policy, freq); - arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); return 0; } @@ -113,7 +111,6 @@ static unsigned int qcom_cpufreq_hw_fast_switch(struct cpufreq_policy *policy, { void __iomem *perf_state_reg = policy->driver_data; int index; - unsigned long freq; index = policy->cached_resolved_idx; if (index < 0) @@ -121,11 +118,7 @@ static unsigned int qcom_cpufreq_hw_fast_switch(struct cpufreq_policy *policy, writel_relaxed(index, perf_state_reg); - freq = policy->freq_table[index].frequency; - arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); - - return freq; + return policy->freq_table[index].frequency; } static int qcom_cpufreq_hw_read_lut(struct device *cpu_dev, @@ -337,8 +330,7 @@ static struct freq_attr *qcom_cpufreq_hw_attr[] = { static struct cpufreq_driver cpufreq_qcom_hw_driver = { .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | - CPUFREQ_IS_COOLING_DEV | - CPUFREQ_CUSTOM_SET_FREQ_SCALE, + CPUFREQ_IS_COOLING_DEV, .verify = cpufreq_generic_frequency_table_verify, .target_index = qcom_cpufreq_hw_target_index, .get = qcom_cpufreq_hw_get, diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c index 16ab4ecc75e4..a91a45c90274 100644 --- a/drivers/cpufreq/scmi-cpufreq.c +++ b/drivers/cpufreq/scmi-cpufreq.c @@ -48,16 +48,11 @@ static unsigned int scmi_cpufreq_get_rate(unsigned int cpu) static int scmi_cpufreq_set_target(struct cpufreq_policy *policy, unsigned int index) { - int ret; struct scmi_data *priv = policy->driver_data; struct scmi_perf_ops *perf_ops = handle->perf_ops; u64 freq = policy->freq_table[index].frequency; - ret = perf_ops->freq_set(handle, priv->domain_id, freq * 1000, false); - if (!ret) - arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); - return ret; + return perf_ops->freq_set(handle, priv->domain_id, freq * 1000, false); } static unsigned int scmi_cpufreq_fast_switch(struct cpufreq_policy *policy, @@ -66,14 +61,11 @@ static unsigned int scmi_cpufreq_fast_switch(struct cpufreq_policy *policy, struct scmi_data *priv = policy->driver_data; struct scmi_perf_ops *perf_ops = handle->perf_ops; - if (!perf_ops->freq_set(handle, priv->domain_id, - target_freq * 1000, true)) { - arch_set_freq_scale(policy->related_cpus, target_freq, - policy->cpuinfo.max_freq); - return target_freq; - } + if (perf_ops->freq_set(handle, priv->domain_id, + target_freq * 1000, true)) + return 0; - return 0; + return target_freq; } static int @@ -223,8 +215,7 @@ static struct cpufreq_driver scmi_cpufreq_driver = { .name = "scmi", .flags = CPUFREQ_STICKY | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | - CPUFREQ_IS_COOLING_DEV | - CPUFREQ_CUSTOM_SET_FREQ_SCALE, + CPUFREQ_IS_COOLING_DEV, .verify = cpufreq_generic_frequency_table_verify, .attr = cpufreq_generic_attr, .target_index = scmi_cpufreq_set_target, diff --git a/drivers/cpufreq/scpi-cpufreq.c b/drivers/cpufreq/scpi-cpufreq.c index 6b5f56dc3ca3..5a399fb847b9 100644 --- a/drivers/cpufreq/scpi-cpufreq.c +++ b/drivers/cpufreq/scpi-cpufreq.c @@ -60,9 +60,6 @@ scpi_cpufreq_set_target(struct cpufreq_policy *policy, unsigned int index) if (clk_get_rate(priv->clk) != rate) return -EIO; - arch_set_freq_scale(policy->related_cpus, freq, - policy->cpuinfo.max_freq); - return 0; } @@ -197,8 +194,7 @@ static struct cpufreq_driver scpi_cpufreq_driver = { .name = "scpi-cpufreq", .flags = CPUFREQ_STICKY | CPUFREQ_HAVE_GOVERNOR_PER_POLICY | CPUFREQ_NEED_INITIAL_FREQ_CHECK | - CPUFREQ_IS_COOLING_DEV | - CPUFREQ_CUSTOM_SET_FREQ_SCALE, + CPUFREQ_IS_COOLING_DEV, .verify = cpufreq_generic_frequency_table_verify, .attr = cpufreq_generic_attr, .get = scpi_cpufreq_get_rate, From patchwork Wed Jul 1 09:07:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6D786C1 for ; Wed, 1 Jul 2020 09:09:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7CA720722 for ; Wed, 1 Jul 2020 09:09:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729441AbgGAJJk (ORCPT ); Wed, 1 Jul 2020 05:09:40 -0400 Received: from foss.arm.com ([217.140.110.172]:47820 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726255AbgGAJJj (ORCPT ); Wed, 1 Jul 2020 05:09:39 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0E55E11FB; Wed, 1 Jul 2020 02:09:38 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2DC843F68F; Wed, 1 Jul 2020 02:09:36 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Liviu Dudau Subject: [PATCH 4/8] cpufreq,vexpress-spc: fix Frequency Invariance (FI) for bL switching Date: Wed, 1 Jul 2020 10:07:47 +0100 Message-Id: <20200701090751.7543-5-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org In the majority of cases, the index argument to cpufreq's target_index() is meant to identify the frequency that is requested from the hardware, according to the frequency table: policy->freq_table[index].frequency. After successfully requesting it from the hardware, this value, together with the maximum hardware frequency (policy->cpuinfo.max_freq) are used as arguments to arch_set_freq_scale(), in order to set the task scheduler frequency scale factor. This is a normalized indication of a CPU's current performance. But for the vexpress-spc-cpufreq driver, when big.LITTLE switching [1] is enabled, there are three issues with using the above information for setting the FI scale factor: - cur_freq: policy->freq_table[index].frequency is not the frequency requested from the hardware. ve_spc_cpufreq_set_rate() will convert from this virtual frequency to an actual frequency, which is then requested from the hardware. For the A7 cluster, the virtual frequency is half the actual frequency. The use of the virtual policy->freq_table frequency results in an incorrect FI scale factor. - max_freq: policy->cpuinfo.max_freq does not correctly identify the maximum frequency of the physical cluster. This value identifies the maximum frequency achievable by the big-LITTLE pair, that is the maximum frequency of the big CPU. But when the LITTLE CPU in the group is used, the hardware maximum frquency passed to arch_set_freq_scale() is incorrect. - missing a scale factor update: when switching clusters, the driver recalculates the frequency of the old clock domain based on the requests of the remaining CPUs in the domain and asks for a clock change. But this does not result in an update in the scale factor. Therefore, introduce a local function bLs_set_sched_freq_scale() that helps call arch_set_freq_scale() with correct information for the is_bL_switching_enabled() case, while maintaining the old, more efficient, call site of arch_set_freq_scale() for when cluster switching is disabled. Also, because of these requirements in computing the scale factor, this driver is the only one that maintains custom support for FI, which is marked by the presence of the CPUFREQ_CUSTOM_SET_FREQ_SCALE flag. [1] https://lwn.net/Articles/481055/ Signed-off-by: Ionela Voinescu Cc: Viresh Kumar Cc: Sudeep Holla Cc: Rafael J. Wysocki Cc: Liviu Dudau --- drivers/cpufreq/vexpress-spc-cpufreq.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/cpufreq/vexpress-spc-cpufreq.c b/drivers/cpufreq/vexpress-spc-cpufreq.c index e0a1a3367ec5..f2caf67d4050 100644 --- a/drivers/cpufreq/vexpress-spc-cpufreq.c +++ b/drivers/cpufreq/vexpress-spc-cpufreq.c @@ -55,6 +55,8 @@ static atomic_t cluster_usage[MAX_CLUSTERS + 1]; static unsigned int clk_big_min; /* (Big) clock frequencies */ static unsigned int clk_little_max; /* Maximum clock frequency (Little) */ +static inline u32 get_table_max(struct cpufreq_frequency_table *table); + static DEFINE_PER_CPU(unsigned int, physical_cluster); static DEFINE_PER_CPU(unsigned int, cpu_last_req_freq); @@ -87,6 +89,18 @@ static unsigned int find_cluster_maxfreq(int cluster) return max_freq; } +static void bLs_set_sched_freq_scale(int cluster, unsigned long cur_freq) +{ + unsigned long max_freq = get_table_max(freq_table[cluster]); + int j; + + for_each_online_cpu(j) { + if (cluster == per_cpu(physical_cluster, j)) + arch_set_freq_scale(get_cpu_mask(j), cur_freq, + max_freq); + } +} + static unsigned int clk_get_cpu_rate(unsigned int cpu) { u32 cur_cluster = per_cpu(physical_cluster, cpu); @@ -154,6 +168,9 @@ ve_spc_cpufreq_set_rate(u32 cpu, u32 old_cluster, u32 new_cluster, u32 rate) mutex_unlock(&cluster_lock[new_cluster]); + if (bLs) + bLs_set_sched_freq_scale(new_cluster, new_rate); + /* Recalc freq for old cluster when switching clusters */ if (old_cluster != new_cluster) { /* Switch cluster */ @@ -170,7 +187,11 @@ ve_spc_cpufreq_set_rate(u32 cpu, u32 old_cluster, u32 new_cluster, u32 rate) pr_err("%s: clk_set_rate failed: %d, old cluster: %d\n", __func__, ret, old_cluster); } + mutex_unlock(&cluster_lock[old_cluster]); + + if (new_rate) + bLs_set_sched_freq_scale(old_cluster, new_rate); } return 0; @@ -200,7 +221,7 @@ static int ve_spc_cpufreq_set_target(struct cpufreq_policy *policy, ret = ve_spc_cpufreq_set_rate(cpu, actual_cluster, new_cluster, freqs_new); - if (!ret) { + if (!is_bL_switching_enabled() && !ret) { arch_set_freq_scale(policy->related_cpus, freqs_new, policy->cpuinfo.max_freq); } From patchwork Wed Jul 1 09:07:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635857 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D625A618 for ; Wed, 1 Jul 2020 09:10:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7634207C4 for ; Wed, 1 Jul 2020 09:10:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729463AbgGAJJn (ORCPT ); Wed, 1 Jul 2020 05:09:43 -0400 Received: from foss.arm.com ([217.140.110.172]:47844 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726255AbgGAJJl (ORCPT ); Wed, 1 Jul 2020 05:09:41 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 071C61396; Wed, 1 Jul 2020 02:09:41 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 42E793F68F; Wed, 1 Jul 2020 02:09:39 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/8] cpufreq: report whether cpufreq supports Frequency Invariance (FI) Date: Wed, 1 Jul 2020 10:07:48 +0100 Message-Id: <20200701090751.7543-6-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Now that the update of the FI scale factor is done in cpufreq core for selected functions - target_index() and fast_switch(), and drivers can mark themselves as providing custom support for FI as well, by calling arch_set_freq_scale() themselves, we can provide feedback to the task scheduler (and not only) on whether cpufreq supports FI. For this purpose, provide error and debug messages, together with an external function to expose whether the cpufreq drivers support FI, by using a static key. Signed-off-by: Ionela Voinescu Cc: Rafael J. Wysocki Cc: Viresh Kumar --- drivers/cpufreq/cpufreq.c | 26 ++++++++++++++++++++++++++ include/linux/cpufreq.h | 5 +++++ 2 files changed, 31 insertions(+) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index 83b58483a39b..60b5272c5d80 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -59,6 +59,9 @@ static struct cpufreq_driver *cpufreq_driver; static DEFINE_PER_CPU(struct cpufreq_policy *, cpufreq_cpu_data); static DEFINE_RWLOCK(cpufreq_driver_lock); +/* Mark support for the scheduler's frequency invariance engine */ +static DEFINE_STATIC_KEY_FALSE(cpufreq_set_freq_scale); + /* Flag to suspend/resume CPUFreq governors */ static bool cpufreq_suspended; @@ -67,6 +70,26 @@ static inline bool has_target(void) return cpufreq_driver->target_index || cpufreq_driver->target; } +static inline +void enable_cpufreq_freq_invariance(struct cpufreq_driver *driver) +{ + if ((driver->flags & CPUFREQ_CUSTOM_SET_FREQ_SCALE) || + ((driver->target_index || driver->fast_switch) + && !(driver->target || driver->setpolicy))) { + + static_branch_enable_cpuslocked(&cpufreq_set_freq_scale); + pr_debug("%s: Driver %s can provide frequency invariance.", + __func__, driver->name); + } else + pr_err("%s: Driver %s cannot provide frequency invariance.", + __func__, driver->name); +} + +bool cpufreq_sets_freq_scale(void) +{ + return static_branch_likely(&cpufreq_set_freq_scale); +} + /* internal prototypes */ static unsigned int __cpufreq_get(struct cpufreq_policy *policy); static int cpufreq_init_governor(struct cpufreq_policy *policy); @@ -2713,6 +2736,8 @@ int cpufreq_register_driver(struct cpufreq_driver *driver_data) cpufreq_driver = driver_data; write_unlock_irqrestore(&cpufreq_driver_lock, flags); + enable_cpufreq_freq_invariance(cpufreq_driver); + if (driver_data->setpolicy) driver_data->flags |= CPUFREQ_CONST_LOOPS; @@ -2782,6 +2807,7 @@ int cpufreq_unregister_driver(struct cpufreq_driver *driver) cpus_read_lock(); subsys_interface_unregister(&cpufreq_interface); remove_boost_sysfs_file(); + static_branch_disable_cpuslocked(&cpufreq_set_freq_scale); cpuhp_remove_state_nocalls_cpuslocked(hp_online); write_lock_irqsave(&cpufreq_driver_lock, flags); diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 42668588f9f8..8b6369d657bd 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -217,6 +217,7 @@ void refresh_frequency_limits(struct cpufreq_policy *policy); void cpufreq_update_policy(unsigned int cpu); void cpufreq_update_limits(unsigned int cpu); bool have_governor_per_policy(void); +bool cpufreq_sets_freq_scale(void); struct kobject *get_governor_parent_kobj(struct cpufreq_policy *policy); void cpufreq_enable_fast_switch(struct cpufreq_policy *policy); void cpufreq_disable_fast_switch(struct cpufreq_policy *policy); @@ -237,6 +238,10 @@ static inline unsigned int cpufreq_get_hw_max_freq(unsigned int cpu) { return 0; } +static inline bool cpufreq_sets_freq_scale(void) +{ + return false; +} static inline void disable_cpufreq(void) { } #endif From patchwork Wed Jul 1 09:07:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635855 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69BCD739 for ; Wed, 1 Jul 2020 09:10:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B8672083E for ; Wed, 1 Jul 2020 09:10:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729485AbgGAJJp (ORCPT ); Wed, 1 Jul 2020 05:09:45 -0400 Received: from foss.arm.com ([217.140.110.172]:47876 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729471AbgGAJJo (ORCPT ); Wed, 1 Jul 2020 05:09:44 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C5002139F; Wed, 1 Jul 2020 02:09:43 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0CA303F68F; Wed, 1 Jul 2020 02:09:41 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] arch_topology,cpufreq,sched/core: constify arch_* cpumasks Date: Wed, 1 Jul 2020 10:07:49 +0100 Message-Id: <20200701090751.7543-7-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Valentin Schneider The passed cpumask arguments to: - arch_set_freq_scale(), - arch_set_thermal_pressure(), and - arch_freq_counters_available() are only iterated over, so reflect this in the prototype. This also allows to pass system cpumasks like cpu_online_mask without getting a warning. Signed-off-by: Valentin Schneider Signed-off-by: Ionela Voinescu Cc: Catalin Marinas Cc: Will Deacon Cc: Sudeep Holla Cc: Rafael J. Wysocki Cc: Viresh Kumar Cc: Ingo Molnar Cc: Peter Zijlstra --- arch/arm64/kernel/topology.c | 2 +- drivers/base/arch_topology.c | 4 ++-- drivers/cpufreq/cpufreq.c | 5 +++-- include/linux/arch_topology.h | 4 ++-- include/linux/cpufreq.h | 3 ++- kernel/sched/core.c | 2 +- 6 files changed, 11 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 0801a0f3c156..9a9f2b8dedf5 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -253,7 +253,7 @@ static int __init init_amu_fie(void) } late_initcall_sync(init_amu_fie); -bool arch_freq_counters_available(struct cpumask *cpus) +bool arch_freq_counters_available(const struct cpumask *cpus) { return amu_freq_invariant() && cpumask_subset(cpus, amu_fie_cpus); diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 4d0a0038b476..8447e1f30340 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -21,13 +21,13 @@ #include #include -__weak bool arch_freq_counters_available(struct cpumask *cpus) +__weak bool arch_freq_counters_available(const struct cpumask *cpus) { return false; } DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE; -void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, +void arch_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq, unsigned long max_freq) { unsigned long scale; diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index 60b5272c5d80..161b8089b0f6 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -177,8 +177,9 @@ u64 get_cpu_idle_time(unsigned int cpu, u64 *wall, int io_busy) } EXPORT_SYMBOL_GPL(get_cpu_idle_time); -__weak void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, - unsigned long max_freq) +__weak void arch_set_freq_scale(const struct cpumask *cpus, + unsigned long cur_freq, + unsigned long max_freq) { } EXPORT_SYMBOL_GPL(arch_set_freq_scale); diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h index 0566cb3314ef..4be0315700cb 100644 --- a/include/linux/arch_topology.h +++ b/include/linux/arch_topology.h @@ -30,7 +30,7 @@ static inline unsigned long topology_get_freq_scale(int cpu) return per_cpu(freq_scale, cpu); } -bool arch_freq_counters_available(struct cpumask *cpus); +bool arch_freq_counters_available(const struct cpumask *cpus); DECLARE_PER_CPU(unsigned long, thermal_pressure); @@ -39,7 +39,7 @@ static inline unsigned long topology_get_thermal_pressure(int cpu) return per_cpu(thermal_pressure, cpu); } -void arch_set_thermal_pressure(struct cpumask *cpus, +void arch_set_thermal_pressure(const struct cpumask *cpus, unsigned long th_pressure); struct cpu_topology { diff --git a/include/linux/cpufreq.h b/include/linux/cpufreq.h index 8b6369d657bd..23398133c24b 100644 --- a/include/linux/cpufreq.h +++ b/include/linux/cpufreq.h @@ -1003,7 +1003,8 @@ static inline void sched_cpufreq_governor_change(struct cpufreq_policy *policy, extern void arch_freq_prepare_all(void); extern unsigned int arch_freq_get_on_cpu(int cpu); -extern void arch_set_freq_scale(struct cpumask *cpus, unsigned long cur_freq, +extern void arch_set_freq_scale(const struct cpumask *cpus, + unsigned long cur_freq, unsigned long max_freq); /* the following are really really optional */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index f518af52d0fb..b44a42b1236c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3645,7 +3645,7 @@ unsigned long long task_sched_runtime(struct task_struct *p) DEFINE_PER_CPU(unsigned long, thermal_pressure); -void arch_set_thermal_pressure(struct cpumask *cpus, +void arch_set_thermal_pressure(const struct cpumask *cpus, unsigned long th_pressure) { int cpu; From patchwork Wed Jul 1 09:07:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635853 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8BFD618 for ; Wed, 1 Jul 2020 09:10:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C416B207C4 for ; Wed, 1 Jul 2020 09:10:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729371AbgGAJJ7 (ORCPT ); Wed, 1 Jul 2020 05:09:59 -0400 Received: from foss.arm.com ([217.140.110.172]:47902 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729494AbgGAJJr (ORCPT ); Wed, 1 Jul 2020 05:09:47 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9949213FD; Wed, 1 Jul 2020 02:09:46 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D2AF93F68F; Wed, 1 Jul 2020 02:09:44 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/8] arch_topology,arm64: define arch_scale_freq_invariant() Date: Wed, 1 Jul 2020 10:07:50 +0100 Message-Id: <20200701090751.7543-8-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Valentin Schneider arch_scale_freq_invariant() is used by schedutil to determine whether the scheduler's load-tracking signals are frequency invariant. Its definition is overridable, though by default it is hardcoded to 'true' if arch_scale_freq_capacity() is defined ('false' otherwise). This behaviour is not overridden on arm, arm64 and other users of the generic arch topology driver, which is somewhat precarious: arch_scale_freq_capacity() will always be defined, yet not all cpufreq drivers are guaranteed to drive the frequency invariance scale factor setting. In other words, the load-tracking signals may very well *not* be frequency invariant. Now that cpufreq can be queried on whether the current driver is driving the Frequency Invariance (FI) scale setting, the current situation can be improved. This combines the query of whether cpufreq supports the setting of the frequency scale factor, with whether all online CPUs are counter-based FI enabled. While cpufreq FI enablement applies at system level, for all CPUs, counter-based FI support could also be used for only a subset of CPUs to set the invariance scale factor. Therefore, if cpufreq-based FI support is present, we consider the system to be invariant. If missing, we require all online CPUs to be counter-based FI enabled in order for the full system to be considered invariant. If the system ends up not being invariant, a new condition is needed in the counter initialization code that disables all scale factor setting based on counters. Precedence of counters over cpufreq use is not important here. The invariant status is only given to the system if all CPUs have at least one method of setting the frequency scale factor. Signed-off-by: Valentin Schneider Signed-off-by: Ionela Voinescu Cc: Catalin Marinas Cc: Will Deacon Cc: Sudeep Holla --- arch/arm64/kernel/topology.c | 7 +++++++ drivers/base/arch_topology.c | 6 ++++++ include/linux/arch_topology.h | 4 ++++ 3 files changed, 17 insertions(+) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 9a9f2b8dedf5..4064d39bb66d 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -246,6 +246,13 @@ static int __init init_amu_fie(void) static_branch_enable(&amu_fie_key); } + /* + * If the system is not fully invariant after AMU init, disable + * partial use of counters for frequency invariance. + */ + if (!topology_scale_freq_invariant()) + static_branch_disable(&amu_fie_key); + free_valid_mask: free_cpumask_var(valid_cpus); diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 8447e1f30340..8686771b866d 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -21,6 +21,12 @@ #include #include +bool topology_scale_freq_invariant(void) +{ + return cpufreq_sets_freq_scale() || + arch_freq_counters_available(cpu_online_mask); +} + __weak bool arch_freq_counters_available(const struct cpumask *cpus) { return false; diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h index 4be0315700cb..6272fbc05cde 100644 --- a/include/linux/arch_topology.h +++ b/include/linux/arch_topology.h @@ -53,6 +53,10 @@ struct cpu_topology { }; #ifdef CONFIG_GENERIC_ARCH_TOPOLOGY + +extern bool topology_scale_freq_invariant(void); +#define arch_scale_freq_invariant() topology_scale_freq_invariant() + extern struct cpu_topology cpu_topology[NR_CPUS]; #define topology_physical_package_id(cpu) (cpu_topology[cpu].package_id) From patchwork Wed Jul 1 09:07:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ionela Voinescu X-Patchwork-Id: 11635851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE236618 for ; Wed, 1 Jul 2020 09:09:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DCC8C2083E for ; Wed, 1 Jul 2020 09:09:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729471AbgGAJJz (ORCPT ); Wed, 1 Jul 2020 05:09:55 -0400 Received: from foss.arm.com ([217.140.110.172]:47936 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729371AbgGAJJu (ORCPT ); Wed, 1 Jul 2020 05:09:50 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A2FE01424; Wed, 1 Jul 2020 02:09:49 -0700 (PDT) Received: from e108754-lin.cambridge.arm.com (unknown [10.1.198.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DE3533F68F; Wed, 1 Jul 2020 02:09:47 -0700 (PDT) From: Ionela Voinescu To: rjw@rjwysocki.net, viresh.kumar@linaro.org, catalin.marinas@arm.com, sudeep.holla@arm.com, will@kernel.org, linux@armlinux.org.uk, valentin.schneider@arm.com Cc: mingo@redhat.com, peterz@infradead.org, dietmar.eggemann@arm.com, ionela.voinescu@arm.com, linux-pm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 8/8] cpufreq: make schedutil the default for arm and arm64 Date: Wed, 1 Jul 2020 10:07:51 +0100 Message-Id: <20200701090751.7543-9-ionela.voinescu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200701090751.7543-1-ionela.voinescu@arm.com> References: <20200701090751.7543-1-ionela.voinescu@arm.com> Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Valentin Schneider schedutil is already a hard-requirement for EAS, which has lead to making it default on arm (when CONFIG_BIG_LITTLE), see: commit 8fdcca8e254a ("cpufreq: Select schedutil when using big.LITTLE") One thing worth pointing out is that schedutil isn't only relevant for asymmetric CPU capacity systems; for instance, schedutil is the only governor that honours util-clamp performance requests. Another good example of this is x86 switching to using it by default in: commit a00ec3874e7d ("cpufreq: intel_pstate: Select schedutil as the default governor") Arguably it should be made the default for all architectures, but it seems better to wait for them to also gain frequency invariance powers. Make it the default for arm && arm64 for now. Signed-off-by: Valentin Schneider Signed-off-by: Ionela Voinescu Cc: Catalin Marinas Cc: Will Deacon Cc: Russell King Cc: Rafael J. Wysocki Cc: Viresh Kumar --- drivers/cpufreq/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cpufreq/Kconfig b/drivers/cpufreq/Kconfig index e91750132552..2c7171e0b001 100644 --- a/drivers/cpufreq/Kconfig +++ b/drivers/cpufreq/Kconfig @@ -37,7 +37,7 @@ config CPU_FREQ_STAT choice prompt "Default CPUFreq governor" default CPU_FREQ_DEFAULT_GOV_USERSPACE if ARM_SA1100_CPUFREQ || ARM_SA1110_CPUFREQ - default CPU_FREQ_DEFAULT_GOV_SCHEDUTIL if BIG_LITTLE + default CPU_FREQ_DEFAULT_GOV_SCHEDUTIL if ARM64 || ARM default CPU_FREQ_DEFAULT_GOV_SCHEDUTIL if X86_INTEL_PSTATE && SMP default CPU_FREQ_DEFAULT_GOV_PERFORMANCE help