From patchwork Wed Mar 1 14:52:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9598527 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2F772604DC for ; Wed, 1 Mar 2017 14:55:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 245C728405 for ; Wed, 1 Mar 2017 14:55:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 178A828580; Wed, 1 Mar 2017 14:55:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E8F5928405 for ; Wed, 1 Mar 2017 14:55:04 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cj5ci-0005z8-3i; Wed, 01 Mar 2017 14:53:00 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cj5cg-0005yg-QL for xen-devel@lists.xenproject.org; Wed, 01 Mar 2017 14:52:58 +0000 Received: from [85.158.137.68] by server-4.bemta-3.messagelabs.com id 56/DE-03705-A40E6B85; Wed, 01 Mar 2017 14:52:58 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLIsWRWlGSWpSXmKPExsXiVRvkpOv5YFu EwZLPMhbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8anW++ZClpkKnrX32BuYJwj1sXIxSEkMJ1R 4v7m/SwgDovAGlaJnQ3HwBwJgUusEk8XrAdyOIGcGIlrv7sZIewKicUbN7KB2EICKhI3t69ig hj1k1Hi8I3VYAlhAT2JI0d/sEPYERLPZj4HG8QmYCDxZsdeVhBbREBJ4t6qyUDNHBzMAuESqz u5QcIsAqoSE19NBNvFK+Atse53A1g5p4CPxOmVi9lAyoWA4n/WuYKERQXkJFZebmGFKBeUODn zCQvERE2J9bv0QcLMAvIS29/OYZ7AKDILSdUshKpZSKoWMDKvYtQoTi0qSy3SNTLUSyrKTM8o yU3MzNE1NDDWy00tLk5MT81JTCrWS87P3cQIDP56BgbGHYw9e/0OMUpyMCmJ8u5etS1CiC8pP 6UyI7E4I76oNCe1+BCjBgeHwISzc6czSbHk5eelKknwPrsPVCdYlJqeWpGWmQOMT5hSCQ4eJR He9SBp3uKCxNzizHSI1ClGXY4P/YffMAmBzZAS530AUiQAUpRRmgc3ApYqLjHKSgnzMjIwMAj xFKQW5WaWoMq/YhTnYFQS5t0JMoUnM68EbtMroCOYgI54obIV5IiSRISUVAOjrq9i382l26Zc Ulx3r2/J6aReldaZmV+/psZcnbZTQctzU2Tl698SIgmXZaPvzeq3a19nt6jy+OU4k+2P0yWml n4XUVdefKO3gMnLuuL23KMCYVFP/nyLKQ5tvcp23WDqjlNqH2/9N+x5pns7Xk6/3m/n57517v Inrtj4rz/VoHX8mMSHeV8rlFiKMxINtZiLihMBHQMSmxADAAA= X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-8.tower-31.messagelabs.com!1488379977!88116150!1 X-Originating-IP: [74.125.82.66] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 62784 invoked from network); 1 Mar 2017 14:52:57 -0000 Received: from mail-wm0-f66.google.com (HELO mail-wm0-f66.google.com) (74.125.82.66) by server-8.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 1 Mar 2017 14:52:57 -0000 Received: by mail-wm0-f66.google.com with SMTP id u63so7787509wmu.2 for ; Wed, 01 Mar 2017 06:52:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=9WKb4RdiYW+W19hEJcTAbxvqot/OpfxfVjfV5n53k0E=; b=ch1UJlGX9jeDxeqCUDjLxIasbYxeOWe1nDgGRSs1Zo3pUU4Y7T7rwrJoq/31A+XZu0 RgGgweqTrgj036m5vX0t/+AXJHvwRqc4yvM2eM0/OsxK+OltKEds+IYhaoUOj0HfRQkZ LHmcAYSlrAUmyAvZgxmKesuy2WlLKWzI4fBSw1dT5WK7SEO0GaXb7wJ/4LUIvWdhTn1U M9HcuITvDqBh/iGhnrOCees+ssCOhhQkptjOhDqM3LevT3GQGl394tsa829N0K2S6OKT 2hItLY5R4//xH/1Qu3ueONI89tqhupilYgG7sNsiRWwUG1PA8sBTsMYxGi/E9yGdPRQ+ pEDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=9WKb4RdiYW+W19hEJcTAbxvqot/OpfxfVjfV5n53k0E=; b=f69AGTZUVyBj809mwdpHpAeOYiFUxB3m3Msolc/BKSZsQpt/OcdP8y8KXKGT0ZCdUF ofDtX6GtmGQhK3N4zW2E4vYw5rBegYGXOnQIe/hMRl0Kn5yPY49Bk7wZSCOA1o5GEMT7 e56erWtawzJ+blx56TTANLqEANz5jBOQztiQA/nFx8Qg2iVt64LEFZg7dIyEhIvb/ZMK pplAwA4U9eaAEx4oXpv0/2OBIpBNezraL/TdZUGkWoKNcnzR1dYB0qUSODcJ4icEb6VG RCrpPo9/5KwumYlvrVlFy97dHx7iWUbE4w6DpKKkecKbOu+mFSgyQ6ANSToa0ZgShyUc 0N+Q== X-Gm-Message-State: AMke39lDFJOcTy0Mo+wzKpfNhlzuEqbY67TkBNm8yYLIvnV9aHU3fqamfEtFD7wIF4T0/A== X-Received: by 10.28.221.11 with SMTP id u11mr3684963wmg.75.1488379976548; Wed, 01 Mar 2017 06:52:56 -0800 (PST) Received: from Solace.fritz.box ([80.66.223.93]) by smtp.gmail.com with ESMTPSA id d75sm7286559wmd.25.2017.03.01.06.52.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Mar 2017 06:52:55 -0800 (PST) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Wed, 01 Mar 2017 15:52:54 +0100 Message-ID: <148837997447.11900.4853141804344467051.stgit@Solace.fritz.box> In-Reply-To: <148837861276.11900.8292677471375175885.stgit@Solace.fritz.box> References: <148837861276.11900.8292677471375175885.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: Anshul Makkar , George Dunlap Subject: [Xen-devel] [PATCH v4 2/7] xen: credit2: don't miss accounting while doing a credit reset. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP A credit reset basically means going through all the vCPUs of a runqueue and altering their credits, as a consequence of a 'scheduling epoch' having come to an end. Blocked or runnable vCPUs are fine, all the credits they've spent running so far have been accounted to them when they were scheduled out. But if a vCPU is running on a pCPU, when a reset event occurs (on another pCPU), that does not get properly accounted. Let's therefore begin to do so, for better accuracy and fairness. In fact, after this patch, we see this in a trace: csched2:schedule cpu 10, rq# 1, busy, not tickled csched2:burn_credits d1v5, credit = 9998353, delta = 202996 runstate_continue d1v5 running->running ... csched2:schedule cpu 12, rq# 1, busy, not tickled csched2:burn_credits d1v6, credit = -1327, delta = 9999544 csched2:reset_credits d0v13, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v14, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v7, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:burn_credits d1v5, credit = 201805, delta = 9796548 csched2:reset_credits d1v5, credit_start = 201805, credit_end = 10201805, mult = 1 csched2:burn_credits d1v6, credit = -1327, delta = 0 csched2:reset_credits d1v6, credit_start = -1327, credit_end = 9998673, mult = 1 Which shows how d1v5 actually executed for ~9.796 ms, on pCPU 10, when reset_credit() is executed, on pCPU 12, because of d1v6's credits going below 0. Without this patch, this 9.796ms are not accounted to anyone. With this patch, d1v5 is charged for that, and its credits drop down from 9796548 to 201805. And this is important, as it means that it will begin the new epoch with 10201805 credits, instead of 10500000 (which he would have, before this patch). Basically, we were forgetting one round of accounting in epoch x, for the vCPUs that are running at the time the epoch ends. And this meant favouring a little bit these same vCPUs, in epoch x+1, providing them with the chance of execute longer than their fair share. Signed-off-by: Dario Faggioli Reviewed-by: George Dunlap --- Cc: Anshul Makkar --- Changes from v3: * moved at top of the series. --- xen/common/sched_credit2.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index cfb783d..d6b4a70 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1132,18 +1132,28 @@ static void reset_credit(const struct scheduler *ops, int cpu, s_time_t now, list_for_each( iter, &rqd->svc ) { + unsigned int svc_cpu; struct csched2_vcpu * svc; int start_credit; svc = list_entry(iter, struct csched2_vcpu, rqd_elem); + svc_cpu = svc->vcpu->processor; ASSERT(!is_idle_vcpu(svc->vcpu)); ASSERT(svc->rqd == rqd); + /* + * If svc is running, it is our responsibility to make sure, here, + * that the credit it has spent so far get accounted. + */ + if ( svc->vcpu == curr_on_cpu(svc_cpu) ) + burn_credits(rqd, svc, now); + start_credit = svc->credit; - /* And add INIT * m, avoiding integer multiplication in the - * common case. */ + /* + * Add INIT * m, avoiding integer multiplication in the common case. + */ if ( likely(m==1) ) svc->credit += CSCHED2_CREDIT_INIT; else