From patchwork Thu Feb 9 13:59:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9564631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 24CE9601C3 for ; Thu, 9 Feb 2017 14:01:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 108FB28499 for ; Thu, 9 Feb 2017 14:01:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 056FB284D1; Thu, 9 Feb 2017 14:01:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9CB9028499 for ; Thu, 9 Feb 2017 14:01:34 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbpFn-0004AF-Dg; Thu, 09 Feb 2017 13:59:19 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cbpFm-000468-Er for xen-devel@lists.xenproject.org; Thu, 09 Feb 2017 13:59:18 +0000 Received: from [193.109.254.147] by server-3.bemta-6.messagelabs.com id C4/46-22349-6B57C985; Thu, 09 Feb 2017 13:59:18 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrHIsWRWlGSWpSXmKPExsXiVRvkpLu1dE6 Ewa3dMhbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8aH+doFp6Ur+rofsDUwvhPtYuTiEBKYzihx 6eNWZhCHRWANq8Ss/c1sII6EwCVWiWtbljN2MXICOTESC28sZIWwqyXmHr/OBGILCahI3Ny+i gli1G9GiS0951lAEsICehJHjv5gh7CjJO7+PgtmswkYSLzZsRdskIiAksS9VZPBBjED1ZxZ3s wMYrMIqEocmbiPDcTmFfCQuHZmLlgvp4C3xPt7a9khFntJLPu7FGyXqICcxMrLLawQ9YISJ2c +AYpzAM3UlFi/Sx9ivLzE9rdzmCcwisxCUjULoWoWkqoFjMyrGDWKU4vKUot0DQ30kooy0zNK chMzc4A8M73c1OLixPTUnMSkYr3k/NxNjMDwZwCCHYz3lgUcYpTkYFIS5ZUtmBMhxJeUn1KZk VicEV9UmpNafIhRg4NDYMLZudOZpFjy8vNSlSR4P5YA1QkWpaanVqRl5gAjFKZUgoNHSYT3Kk iat7ggMbc4Mx0idYpRl2PXrssvmYTAZkiJ8zIB411IAKQoozQPbgQsWVxilJUS5mUEOlCIpyC 1KDezBFX+FaM4B6OSMO9zkFU8mXklcJteAR3BBHTE9dOzQI4oSURISTUwLviv6zozgvnfryPn bTeq1Cg+UO36HBHeGNr7ue/inpu+rDuclh8KVV0qpHnafIrO56pEV71remVFsacurmpYcsZY7 ZJ4ZqHWRYZoPaEIjfosrnmyLhGONqbN1hs/h0q4+0V8KQ2R7J9W0Gdj8fNsgY7Aqy5ZwZOFjo dncx05NCN4srjNpkwlluKMREMt5qLiRACUMHp9EQMAAA== X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-10.tower-27.messagelabs.com!1486648757!62998656!1 X-Originating-IP: [74.125.82.66] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 44906 invoked from network); 9 Feb 2017 13:59:17 -0000 Received: from mail-wm0-f66.google.com (HELO mail-wm0-f66.google.com) (74.125.82.66) by server-10.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 9 Feb 2017 13:59:17 -0000 Received: by mail-wm0-f66.google.com with SMTP id u63so3010050wmu.2 for ; Thu, 09 Feb 2017 05:59:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=/zED02rLwsAZUUSHCOiR4j/sglg/zmlIQJBut9yegYk=; b=JIuUKOcPkehMPQ38qSOjdgOcEc+zR5g9PY3VUMeeaAXj/DQ3oMFzNDSeMP7ljX4I1S OgBSHhvqZmcuMHV/0n7j3fFmSfy4gXfiY4MloVAemZC4PDEWdQ7cLa9rqWBluqKfcGkj 2aHm8FJX/eHadbXWFrTHVfDK5JYifAgV4B3AGda1Br5wEw5ZKiZcQNkG/acpsbELhMiA jpyfSJTbsT+cYQMHtP7+k8oyNQS46yJmOoP7uuZ20py63tSdIsaf9YF9f55x68jUURFV Jvsx9sNBTtu/WWk7WqWuEfDbem63jC508vKHyaVVrnc3vuTP4AzEttkdwF+ZZoS18tYs BISw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=/zED02rLwsAZUUSHCOiR4j/sglg/zmlIQJBut9yegYk=; b=Mxc4Lb864uJQ3550h6kRStk7HM+NsxUQp2HXDtTlq0OiTHfYAhlRxgbBl0o7k2ATBS LB/agueRsONSgUOy6/1ZQHDDzZItnr148DQJoEoYAkMWcG2uWfl9eedC0fvWvRMRj+Z3 2Vj9rR8F8ekT4jvRtlyue5hQTT2F9KhdIjAvuRnJHoeWTQ2/NYLZYXygIkDGRKq4KAtd 1wl2b33N9UEoJ9p5fQEIc8G+LLU0wOHVn9iDyDCFVKjmmQrs7dfe5CtkEPC5O1QhSkjd GQ5erUbCcztqEZaGESy48eq1wEvJl0u9OMWr/CwxJqsbKKyElYTh3+yVtlwVqTNFT9SJ 40TA== X-Gm-Message-State: AMke39kVmjTe38lfbkpfJagUy70nWltUDqQOgbdyrGdK8gi0Y+rUrVmJQG0byVx5+Hs1yg== X-Received: by 10.28.137.203 with SMTP id l194mr22891184wmd.63.1486648757012; Thu, 09 Feb 2017 05:59:17 -0800 (PST) Received: from Solace.fritz.box ([80.66.223.139]) by smtp.gmail.com with ESMTPSA id w17sm18692415wra.28.2017.02.09.05.59.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Feb 2017 05:59:16 -0800 (PST) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 09 Feb 2017 14:59:15 +0100 Message-ID: <148664875514.595.468730348267549995.stgit@Solace.fritz.box> In-Reply-To: <148664844741.595.10506268024432565895.stgit@Solace.fritz.box> References: <148664844741.595.10506268024432565895.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap , Anshul Makkar Subject: [Xen-devel] [PATCH v2 08/10] xen: credit2: don't miss accounting while doing a credit reset. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP A credit reset basically means going through all the vCPUs of a runqueue and altering their credits, as a consequence of a 'scheduling epoch' having come to an end. Blocked or runnable vCPUs are fine, all the credits they've spent running so far have been accounted to them when they were scheduled out. But if a vCPU is running on a pCPU, when a reset event occurs (on another pCPU), that does not get properly accounted. Let's therefore begin to do so, for better accuracy and fairness. In fact, after this patch, we see this in a trace: csched2:schedule cpu 10, rq# 1, busy, not tickled csched2:burn_credits d1v5, credit = 9998353, delta = 202996 runstate_continue d1v5 running->running ... csched2:schedule cpu 12, rq# 1, busy, not tickled csched2:burn_credits d1v6, credit = -1327, delta = 9999544 csched2:reset_credits d0v13, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v14, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v7, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:burn_credits d1v5, credit = 201805, delta = 9796548 csched2:reset_credits d1v5, credit_start = 201805, credit_end = 10201805, mult = 1 csched2:burn_credits d1v6, credit = -1327, delta = 0 csched2:reset_credits d1v6, credit_start = -1327, credit_end = 9998673, mult = 1 Which shows how d1v5 actually executed for ~9.796 ms, on pCPU 10, when reset_credit() is executed, on pCPU 12, because of d1v6's credits going below 0. Without this patch, this 9.796ms are not accounted to anyone. With this patch, d1v5 is charged for that, and its credits drop down from 9796548 to 201805. And this is important, as it means that it will begin the new epoch with 10201805 credits, instead of 10500000 (which he would have, before this patch). Basically, we were forgetting one round of accounting in epoch x, for the vCPUs that are running at the time the epoch ends. And this meant favouring a little bit these same vCPUs, in epoch x+1, providing them with the chance of execute longer than their fair share. Signed-off-by: Dario Faggioli Reviewed-by: George Dunlap --- Cc: George Dunlap Cc: Anshul Makkar --- xen/common/sched_credit2.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index bfb4891..8057abf 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1341,18 +1341,28 @@ static void reset_credit(const struct scheduler *ops, int cpu, s_time_t now, list_for_each( iter, &rqd->svc ) { + unsigned int svc_cpu; struct csched2_vcpu * svc; int start_credit; svc = list_entry(iter, struct csched2_vcpu, rqd_elem); + svc_cpu = svc->vcpu->processor; ASSERT(!is_idle_vcpu(svc->vcpu)); ASSERT(svc->rqd == rqd); + /* + * If svc is running, it is our responsibility to make sure, here, + * that the credit it has spent so far get accounted. + */ + if ( svc->vcpu == curr_on_cpu(svc_cpu) ) + burn_credits(rqd, svc, now); + start_credit = svc->credit; - /* And add INIT * m, avoiding integer multiplication in the - * common case. */ + /* + * Add INIT * m, avoiding integer multiplication in the common case. + */ if ( likely(m==1) ) svc->credit += CSCHED2_CREDIT_INIT; else