From patchwork Mon Nov 3 16:54:41 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 5218381 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 3632B9F349 for ; Mon, 3 Nov 2014 16:58:54 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 46FC22012F for ; Mon, 3 Nov 2014 16:58:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5406320136 for ; Mon, 3 Nov 2014 16:58:51 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XlKw4-0002a8-8l; Mon, 03 Nov 2014 16:56:56 +0000 Received: from mail-wi0-f180.google.com ([209.85.212.180]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XlKvb-0002HH-Mf for linux-arm-kernel@lists.infradead.org; Mon, 03 Nov 2014 16:56:28 +0000 Received: by mail-wi0-f180.google.com with SMTP id hi2so6972029wib.7 for ; Mon, 03 Nov 2014 08:56:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Mt30CBLF8DcC7pT1vZ2vMUcvUUNtoAvyRwKkokAPPh8=; b=KdB/KPEZRrZiSE5Lb7vooMCVdRZwu8VkVw5QSVa3W5o8kUySQEhPS8milUvdD4Bq3q CKETGASkuusOLvc9jUbyinXLTg3tXmBdM603+dpjyNoUXyIWxNYpzGzDJI3DaPGWSPf6 FNfxyb7Zj9q+qSvnCKBsHR8V2Ec4X7FjnjTPO+8F9RIvQkrO8oK9OmN2xPrP7PoFv3vj Q66PxV1o0nL2NgBkUuJEKOFk7elNmLmNVLvkwVqxErdxVgXjHcKVVs+QpJ68BJCWSwcb j2SXBWvVIimaH2uhNmTEfgtK6QlHvuKVoT/s8OGEWptfsNN4YnMw1VMrLgMyeRqbTMSl /RPg== X-Gm-Message-State: ALoCoQmp7GuYkFhPjc+PomLRH8Eu70znB4SoU1xyvH0xtA80jF5Ek1QFoNWdkM2/mAuOsLaDgC77 X-Received: by 10.194.185.167 with SMTP id fd7mr4620869wjc.108.1415033765544; Mon, 03 Nov 2014 08:56:05 -0800 (PST) Received: from lmenx30s.lme.st.com (LPuteaux-656-01-48-212.w82-127.abo.wanadoo.fr. [82.127.83.212]) by mx.google.com with ESMTPSA id s8sm22798078wjx.9.2014.11.03.08.56.03 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Nov 2014 08:56:04 -0800 (PST) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, preeti@linux.vnet.ibm.com, Morten.Rasmussen@arm.com, kamalesh@linux.vnet.ibm.com, linux-arm-kernel@lists.infradead.org Subject: [PATCH v9 04/10] sched: Make sched entity usage tracking scale-invariant Date: Mon, 3 Nov 2014 17:54:41 +0100 Message-Id: <1415033687-23294-5-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1415033687-23294-1-git-send-email-vincent.guittot@linaro.org> References: <1415033687-23294-1-git-send-email-vincent.guittot@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20141103_085627_939004_9666AFBD X-CRM114-Status: GOOD ( 14.86 ) X-Spam-Score: -0.7 (/) Cc: nicolas.pitre@linaro.org, riel@redhat.com, linaro-kernel@lists.linaro.org, Vincent Guittot , efault@gmx.de, Ben Segall , Paul Turner , Morten Rasmussen X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Morten Rasmussen Apply frequency scale-invariance correction factor to usage tracking. Each segment of the running_load_avg geometric series is now scaled by the current frequency so the utilization_avg_contrib of each entity will be invariant with frequency scaling. As a result, utilization_load_avg which is the sum of utilization_avg_contrib, becomes invariant too. So the usage level that is returned by get_cpu_usage, stays relative to the max frequency as the cpu_capacity which is is compared against. Then, we want the keep the load tracking values in a 32bits type, which implies that the max value of {runnable|running}_avg_sum must be lower than 2^32/88761=48388 (88761 is the max weigth of a task). As LOAD_AVG_MAX = 47742, arch_scale_freq_capacity must return a value less than (48388/47742) << SCHED_CAPACITY_SHIFT = 1037 (SCHED_SCALE_CAPACITY = 1024). So we define the range to [0..SCHED_SCALE_CAPACITY] in order to avoid overflow. cc: Paul Turner cc: Ben Segall Signed-off-by: Morten Rasmussen Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a96affd..a5039da 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2266,6 +2266,8 @@ static u32 __compute_runnable_contrib(u64 n) return contrib + runnable_avg_yN_sum[n]; } +unsigned long __weak arch_scale_freq_capacity(struct sched_domain *sd, int cpu); + /* * We can represent the historical contribution to runnable average as the * coefficients of a geometric series. To do this we sub-divide our runnable @@ -2294,7 +2296,7 @@ static u32 __compute_runnable_contrib(u64 n) * load_avg = u_0` + y*(u_0 + u_1*y + u_2*y^2 + ... ) * = u_0 + u_1*y + u_2*y^2 + ... [re-labeling u_i --> u_{i+1}] */ -static __always_inline int __update_entity_runnable_avg(u64 now, +static __always_inline int __update_entity_runnable_avg(u64 now, int cpu, struct sched_avg *sa, int runnable, int running) @@ -2302,6 +2304,7 @@ static __always_inline int __update_entity_runnable_avg(u64 now, u64 delta, periods; u32 runnable_contrib; int delta_w, decayed = 0; + unsigned long scale_freq = arch_scale_freq_capacity(NULL, cpu); delta = now - sa->last_runnable_update; /* @@ -2337,7 +2340,8 @@ static __always_inline int __update_entity_runnable_avg(u64 now, if (runnable) sa->runnable_avg_sum += delta_w; if (running) - sa->running_avg_sum += delta_w; + sa->running_avg_sum += delta_w * scale_freq + >> SCHED_CAPACITY_SHIFT; sa->avg_period += delta_w; delta -= delta_w; @@ -2358,7 +2362,8 @@ static __always_inline int __update_entity_runnable_avg(u64 now, if (runnable) sa->runnable_avg_sum += runnable_contrib; if (running) - sa->running_avg_sum += runnable_contrib; + sa->running_avg_sum += runnable_contrib * scale_freq + >> SCHED_CAPACITY_SHIFT; sa->avg_period += runnable_contrib; } @@ -2366,7 +2371,8 @@ static __always_inline int __update_entity_runnable_avg(u64 now, if (runnable) sa->runnable_avg_sum += delta; if (running) - sa->running_avg_sum += delta; + sa->running_avg_sum += delta * scale_freq + >> SCHED_CAPACITY_SHIFT; sa->avg_period += delta; return decayed; @@ -2474,8 +2480,8 @@ static inline void __update_group_entity_contrib(struct sched_entity *se) static inline void update_rq_runnable_avg(struct rq *rq, int runnable) { - __update_entity_runnable_avg(rq_clock_task(rq), &rq->avg, runnable, - runnable); + __update_entity_runnable_avg(rq_clock_task(rq), cpu_of(rq), &rq->avg, + runnable, runnable); __update_tg_runnable_avg(&rq->avg, &rq->cfs); } #else /* CONFIG_FAIR_GROUP_SCHED */ @@ -2553,6 +2559,7 @@ static inline void update_entity_load_avg(struct sched_entity *se, { struct cfs_rq *cfs_rq = cfs_rq_of(se); long contrib_delta, utilization_delta; + int cpu = cpu_of(rq_of(cfs_rq)); u64 now; /* @@ -2564,7 +2571,7 @@ static inline void update_entity_load_avg(struct sched_entity *se, else now = cfs_rq_clock_task(group_cfs_rq(se)); - if (!__update_entity_runnable_avg(now, &se->avg, se->on_rq, + if (!__update_entity_runnable_avg(now, cpu, &se->avg, se->on_rq, cfs_rq->curr == se)) return;