| Message ID | 20190914085251.18816-48-jgross@suse.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | xen: add core scheduling support | expand |
On Sat, 2019-09-14 at 10:52 +0200, Juergen Gross wrote: > Add a scheduling granularity enum ("cpu", "core", "socket") for > specification of the scheduling granularity. Initially it is set to > "cpu", this can be modified by the new boot parameter (x86 only) > "sched-gran". > > According to the selected granularity sched_granularity is set after > all cpus are online. > > A test is added for all sched resources holding the same number of > cpus. Fall back to core- or cpu-scheduling in that case. > > Signed-off-by: Juergen Gross <jgross@suse.com> > Reviewed-by: Dario Faggioli <dfaggioli@suse.com> Regards
On 26.09.19 11:46, Dario Faggioli wrote: > On Sat, 2019-09-14 at 10:52 +0200, Juergen Gross wrote: >> Add a scheduling granularity enum ("cpu", "core", "socket") for >> specification of the scheduling granularity. Initially it is set to >> "cpu", this can be modified by the new boot parameter (x86 only) >> "sched-gran". >> >> According to the selected granularity sched_granularity is set after >> all cpus are online. >> >> A test is added for all sched resources holding the same number of >> cpus. Fall back to core- or cpu-scheduling in that case. >> >> Signed-off-by: Juergen Gross <jgross@suse.com> >> > Reviewed-by: Dario Faggioli <dfaggioli@suse.com> Does this still stand with moving all code of this patch to cpupool.c? That would avoid making some variables/functions globally visible. Juergen
On Thu, 2019-09-26 at 14:37 +0200, Jürgen Groß wrote: > On 26.09.19 11:46, Dario Faggioli wrote: > > On Sat, 2019-09-14 at 10:52 +0200, Juergen Gross wrote: > > > Add a scheduling granularity enum ("cpu", "core", "socket") for > > > specification of the scheduling granularity. Initially it is set > > > to > > > "cpu", this can be modified by the new boot parameter (x86 only) > > > "sched-gran". > > > > > > According to the selected granularity sched_granularity is set > > > after > > > all cpus are online. > > > > > > A test is added for all sched resources holding the same number > > > of > > > cpus. Fall back to core- or cpu-scheduling in that case. > > > > > > Signed-off-by: Juergen Gross <jgross@suse.com> > > > > > Reviewed-by: Dario Faggioli <dfaggioli@suse.com> > > Does this still stand with moving all code of this patch to > cpupool.c? That would avoid making some variables/functions globally > visible. > Yep, I saw this being discussed during review of other patches, but forgot to mention it here, sorry. Yes, I'm fine with that, and I'm fine with the consequences of doing that on this patch (i.e., moving to cpupool.c). The Reviewed-by still holds. Regards
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 288dc6c042..3f88adae97 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -22,6 +22,7 @@ config X86 select HAS_PASSTHROUGH select HAS_PCI select HAS_PDX + select HAS_SCHED_GRANULARITY select HAS_UBSAN select HAS_VPCI if !PV_SHIM_EXCLUSIVE && HVM select NEEDS_LIBELF diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 16829f6274..e9247871a8 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -63,6 +63,9 @@ config HAS_GDBSX config HAS_IOPORTS bool +config HAS_SCHED_GRANULARITY + bool + config NEEDS_LIBELF bool diff --git a/xen/common/cpupool.c b/xen/common/cpupool.c index 7f173f5099..4226f7ded8 100644 --- a/xen/common/cpupool.c +++ b/xen/common/cpupool.c @@ -866,6 +866,8 @@ static int __init cpupool_init(void) unsigned int cpu; int err; + scheduler_gran_init(); + cpupool0 = cpupool_create(0, 0, &err); BUG_ON(cpupool0 == NULL); cpupool_put(cpupool0); diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 800d2405a7..b940083a26 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -34,6 +34,7 @@ #include <xen/cpu.h> #include <xen/preempt.h> #include <xen/event.h> +#include <xen/warning.h> #include <public/sched.h> #include <xsm/xsm.h> #include <xen/err.h> @@ -61,6 +62,23 @@ unsigned int __read_mostly sched_granularity = 1; bool __read_mostly sched_disable_smt_switching; cpumask_var_t sched_res_mask; +#ifdef CONFIG_HAS_SCHED_GRANULARITY +static int __init sched_select_granularity(const char *str) +{ + if ( strcmp("cpu", str) == 0 ) + opt_sched_granularity = SCHED_GRAN_cpu; + else if ( strcmp("core", str) == 0 ) + opt_sched_granularity = SCHED_GRAN_core; + else if ( strcmp("socket", str) == 0 ) + opt_sched_granularity = SCHED_GRAN_socket; + else + return -EINVAL; + + return 0; +} +custom_param("sched-gran", sched_select_granularity); +#endif + /* Common lock for free cpus. */ static DEFINE_SPINLOCK(sched_free_cpu_lock); @@ -2617,6 +2635,66 @@ void scheduler_enable(void) scheduler_active = true; } +static unsigned int __init sched_check_granularity(void) +{ + unsigned int cpu; + unsigned int siblings, gran = 0; + + if ( opt_sched_granularity == SCHED_GRAN_cpu ) + return 1; + + for_each_online_cpu ( cpu ) + { + siblings = cpumask_weight(sched_get_opt_cpumask(opt_sched_granularity, + cpu)); + if ( gran == 0 ) + gran = siblings; + else if ( gran != siblings ) + return 0; + } + + sched_disable_smt_switching = true; + + return gran; +} + +/* Setup data for selected scheduler granularity. */ +void __init scheduler_gran_init(void) +{ + unsigned int gran = 0; + const char *fallback = NULL; + + while ( gran == 0 ) + { + gran = sched_check_granularity(); + + if ( gran == 0 ) + { + switch ( opt_sched_granularity ) + { + case SCHED_GRAN_core: + opt_sched_granularity = SCHED_GRAN_cpu; + fallback = "Asymmetric cpu configuration.\n" + "Falling back to sched-gran=cpu.\n"; + break; + case SCHED_GRAN_socket: + opt_sched_granularity = SCHED_GRAN_core; + fallback = "Asymmetric cpu configuration.\n" + "Falling back to sched-gran=core.\n"; + break; + default: + ASSERT_UNREACHABLE(); + break; + } + } + } + + if ( fallback ) + warning_add(fallback); + + sched_granularity = gran; +} + /* Initialise the data structures. */ void __init scheduler_init(void) { diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h index f1e8f44f62..53158fe4ea 100644 --- a/xen/include/xen/sched-if.h +++ b/xen/include/xen/sched-if.h @@ -626,5 +626,6 @@ affinity_balance_cpumask(const struct sched_unit *unit, int step, void sched_rm_cpu(unsigned int cpu); const cpumask_t *sched_get_opt_cpumask(enum sched_gran opt, unsigned int cpu); +void scheduler_gran_init(void); #endif /* __XEN_SCHED_IF_H__ */
Add a scheduling granularity enum ("cpu", "core", "socket") for specification of the scheduling granularity. Initially it is set to "cpu", this can be modified by the new boot parameter (x86 only) "sched-gran". According to the selected granularity sched_granularity is set after all cpus are online. A test is added for all sched resources holding the same number of cpus. Fall back to core- or cpu-scheduling in that case. Signed-off-by: Juergen Gross <jgross@suse.com> --- RFC V2: - fixed freeing of sched_res when merging cpus - rename parameter to "sched-gran" (Jan Beulich) - rename parameter option from "thread" to "cpu" (Jan Beulich) V1: - rename scheduler_smp_init() to scheduler_gran_init(), let it be called by cpupool_init() - avoid using literal cpu number 0 in scheduler_percpu_init() (Jan Beulich) - style correction (Jan Beulich) - fallback to smaller granularity instead of panic in case of unbalanced cpu configuration V2: - style changes (Jan Beulich) - introduce CONFIG_HAS_SCHED_GRANULARITY (Jan Beulich) --- xen/arch/x86/Kconfig | 1 + xen/common/Kconfig | 3 ++ xen/common/cpupool.c | 2 ++ xen/common/schedule.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/sched-if.h | 1 + 5 files changed, 85 insertions(+)