From patchwork Sat Sep 14 08:52:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 11145565 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98F3D112B for ; Sat, 14 Sep 2019 08:54:23 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6998520717 for ; Sat, 14 Sep 2019 08:54:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6998520717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i93oF-00082I-3D; Sat, 14 Sep 2019 08:53:35 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i93oD-00080D-LL for xen-devel@lists.xenproject.org; Sat, 14 Sep 2019 08:53:33 +0000 X-Inumbo-ID: 0e37e663-d6cd-11e9-95c1-12813bfff9fa Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0e37e663-d6cd-11e9-95c1-12813bfff9fa; Sat, 14 Sep 2019 08:53:01 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 4A747B663; Sat, 14 Sep 2019 08:52:59 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Sat, 14 Sep 2019 10:52:17 +0200 Message-Id: <20190914085251.18816-14-jgross@suse.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190914085251.18816-1-jgross@suse.com> References: <20190914085251.18816-1-jgross@suse.com> Subject: [Xen-devel] [PATCH v3 13/47] xen/sched: add is_running indicator to struct sched_unit X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Tim Deegan , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Meng Xu , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Add an is_running indicator to struct sched_unit which will be set whenever the unit is being scheduled. Switch scheduler code to use unit->is_running instead of vcpu->is_running for scheduling decisions. At the same time introduce a state_entry_time field in struct sched_unit being updated whenever the is_running indicator is changed. Use that new field in the schedulers instead of the similar vcpu field. Signed-off-by: Juergen Gross Reviewed-by: Dario Faggioli --- RFC V2: - fix arm build, don't drop v->is_running V3: - avoid holes in structures (Jan Beulich) --- xen/common/sched_credit.c | 12 +++++++----- xen/common/sched_credit2.c | 18 +++++++++--------- xen/common/sched_rt.c | 2 +- xen/common/schedule.c | 13 ++++++++++--- xen/include/xen/sched.h | 6 ++++++ 5 files changed, 33 insertions(+), 18 deletions(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 2bf11b0616..c0f6ef831e 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -709,7 +709,7 @@ __csched_vcpu_is_migrateable(const struct csched_private *prv, struct vcpu *vc, * The caller is supposed to have already checked that vc is also * not running. */ - ASSERT(!vc->is_running); + ASSERT(!vc->sched_unit->is_running); return !__csched_vcpu_is_cache_hot(prv, svc) && cpumask_test_cpu(dest_cpu, mask); @@ -1033,7 +1033,8 @@ csched_unit_insert(const struct scheduler *ops, struct sched_unit *unit) lock = unit_schedule_lock_irq(unit); - if ( !__vcpu_on_runq(svc) && vcpu_runnable(vc) && !vc->is_running ) + if ( !__vcpu_on_runq(svc) && vcpu_runnable(vc) && + !vc->sched_unit->is_running ) runq_insert(svc); unit_schedule_unlock_irq(lock, unit); @@ -1646,8 +1647,9 @@ csched_runq_steal(int peer_cpu, int cpu, int pri, int balance_step) * vCPUs with useful soft affinities in some sort of bitmap * or counter. */ - if ( vc->is_running || (balance_step == BALANCE_SOFT_AFFINITY && - !has_soft_affinity(vc->sched_unit)) ) + if ( vc->sched_unit->is_running || + (balance_step == BALANCE_SOFT_AFFINITY && + !has_soft_affinity(vc->sched_unit)) ) continue; affinity_balance_cpumask(vc->sched_unit, balance_step, cpumask_scratch); @@ -1855,7 +1857,7 @@ csched_schedule( (unsigned char *)&d); } - runtime = now - current->runstate.state_entry_time; + runtime = now - current->sched_unit->state_entry_time; if ( runtime < 0 ) /* Does this ever happen? */ runtime = 0; diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 910440d5d9..a6083289f6 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1284,7 +1284,7 @@ runq_insert(const struct scheduler *ops, struct csched2_unit *svc) ASSERT(&svc->rqd->runq == runq); ASSERT(!is_idle_vcpu(svc->vcpu)); - ASSERT(!svc->vcpu->is_running); + ASSERT(!svc->vcpu->sched_unit->is_running); ASSERT(!(svc->flags & CSFLAG_scheduled)); list_for_each( iter, runq ) @@ -1341,8 +1341,8 @@ static inline bool is_preemptable(const struct csched2_unit *svc, if ( ratelimit <= CSCHED2_RATELIMIT_TICKLE_TOLERANCE ) return true; - ASSERT(svc->vcpu->is_running); - return now - svc->vcpu->runstate.state_entry_time > + ASSERT(svc->vcpu->sched_unit->is_running); + return now - svc->vcpu->sched_unit->state_entry_time > ratelimit - CSCHED2_RATELIMIT_TICKLE_TOLERANCE; } @@ -2932,7 +2932,7 @@ csched2_dom_cntl( { svc = csched2_unit(v->sched_unit); lock = unit_schedule_lock(svc->vcpu->sched_unit); - if ( v->is_running ) + if ( v->sched_unit->is_running ) { unsigned int cpu = v->processor; struct csched2_runqueue_data *rqd = c2rqd(ops, cpu); @@ -3205,8 +3205,8 @@ csched2_runtime(const struct scheduler *ops, int cpu, if ( prv->ratelimit_us ) { s_time_t ratelimit_min = MICROSECS(prv->ratelimit_us); - if ( snext->vcpu->is_running ) - ratelimit_min = snext->vcpu->runstate.state_entry_time + + if ( snext->vcpu->sched_unit->is_running ) + ratelimit_min = snext->vcpu->sched_unit->state_entry_time + MICROSECS(prv->ratelimit_us) - now; if ( ratelimit_min > min_time ) min_time = ratelimit_min; @@ -3303,7 +3303,7 @@ runq_candidate(struct csched2_runqueue_data *rqd, * no point forcing it to do so until rate limiting expires. */ if ( !yield && prv->ratelimit_us && vcpu_runnable(scurr->vcpu) && - (now - scurr->vcpu->runstate.state_entry_time) < + (now - scurr->vcpu->sched_unit->state_entry_time) < MICROSECS(prv->ratelimit_us) ) { if ( unlikely(tb_init_done) ) @@ -3314,7 +3314,7 @@ runq_candidate(struct csched2_runqueue_data *rqd, } d; d.dom = scurr->vcpu->domain->domain_id; d.vcpu = scurr->vcpu->vcpu_id; - d.runtime = now - scurr->vcpu->runstate.state_entry_time; + d.runtime = now - scurr->vcpu->sched_unit->state_entry_time; __trace_var(TRC_CSCHED2_RATELIMIT, 1, sizeof(d), (unsigned char *)&d); @@ -3562,7 +3562,7 @@ csched2_schedule( if ( snext != scurr ) { ASSERT(snext->rqd == rqd); - ASSERT(!snext->vcpu->is_running); + ASSERT(!snext->vcpu->sched_unit->is_running); runq_remove(snext); __set_bit(__CSFLAG_scheduled, &snext->flags); diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c index 8b5ae159b2..a47cc598a9 100644 --- a/xen/common/sched_rt.c +++ b/xen/common/sched_rt.c @@ -906,7 +906,7 @@ rt_unit_insert(const struct scheduler *ops, struct sched_unit *unit) { replq_insert(ops, svc); - if ( !vc->is_running ) + if ( !unit->is_running ) runq_insert(ops, svc); } unit_schedule_unlock_irq(lock, unit); diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 92a74b731b..05bc42d7e9 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -411,6 +411,8 @@ int sched_init_vcpu(struct vcpu *v, unsigned int processor) { get_sched_res(v->processor)->curr = unit; v->is_running = 1; + unit->is_running = 1; + unit->state_entry_time = NOW(); } else { @@ -731,7 +733,8 @@ static void vcpu_migrate_finish(struct vcpu *v) * context_saved(); and in any case, if the bit is cleared, then * someone else has already done the work so we don't need to. */ - if ( v->is_running || !test_bit(_VPF_migrating, &v->pause_flags) ) + if ( v->sched_unit->is_running || + !test_bit(_VPF_migrating, &v->pause_flags) ) return; old_cpu = new_cpu = v->processor; @@ -785,7 +788,7 @@ static void vcpu_migrate_finish(struct vcpu *v) * because they both happen in (different) spinlock regions, and those * regions are strictly serialised. */ - if ( v->is_running || + if ( v->sched_unit->is_running || !test_and_clear_bit(_VPF_migrating, &v->pause_flags) ) { sched_spin_unlock_double(old_lock, new_lock, flags); @@ -1673,8 +1676,10 @@ static void schedule(void) * switch, else lost_records resume will not work properly. */ - ASSERT(!next->is_running); + ASSERT(!next->sched_unit->is_running); next->is_running = 1; + next->sched_unit->is_running = 1; + next->sched_unit->state_entry_time = now; pcpu_schedule_unlock_irq(lock, cpu); @@ -1696,6 +1701,8 @@ void context_saved(struct vcpu *prev) smp_wmb(); prev->is_running = 0; + prev->sched_unit->is_running = 0; + prev->sched_unit->state_entry_time = NOW(); /* Check for migration request /after/ clearing running flag. */ smp_mb(); diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 949bb6b696..d4698b3f0d 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -270,8 +270,14 @@ struct sched_unit { struct sched_resource *res; unsigned int unit_id; + /* Currently running on a CPU? */ + bool is_running; /* Does soft affinity actually play a role (given hard affinity)? */ bool soft_aff_effective; + + /* Last time unit got (de-)scheduled. */ + uint64_t state_entry_time; + /* Bitmask of CPUs on which this VCPU may run. */ cpumask_var_t cpu_hard_affinity; /* Used to save affinity during temporary pinning. */