From patchwork Wed Mar 1 15:34:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9598637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 56FDB60453 for ; Wed, 1 Mar 2017 15:37:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4787728541 for ; Wed, 1 Mar 2017 15:37:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C12B2856B; Wed, 1 Mar 2017 15:37:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 80A0128541 for ; Wed, 1 Mar 2017 15:37:32 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cj6HY-00030n-8J; Wed, 01 Mar 2017 15:35:12 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cj6HX-00030h-9c for xen-devel@lists.xenproject.org; Wed, 01 Mar 2017 15:35:11 +0000 Received: from [85.158.143.35] by server-6.bemta-6.messagelabs.com id 2B/08-15112-E2AE6B85; Wed, 01 Mar 2017 15:35:10 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrJKsWRWlGSWpSXmKPExsXitHRDpK7Wq20 RBp/Pmlt83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBktLzayFLwOqOj4WNXAuN2vi5GTQ0IgROL3 vDnMIDavgJHE54Nf2UBsYYFgiY17nrKD2GwCBhJvduxlBbFFBJQlPj9fCVbPLBAkcfjWVrA4i 4CKxN1NG8DqOQV8JE6vXAw0h4NDSMBb4s86V5Awv4CkxK0vH5lBwswC1RJnNmtDXKAt0bP3B9 QFghInZz5hAbGFBNQkZsy9zDqBkW8WQscsJFWzwG5wkJi/7hMrhK0p0br9NzuErS2xbOFrZgj bU+LHzhtQcUWJKd0PoexAiY033jNB2B4S22Z9YcJUEyDRuvc4VNxWYt2691B7bSQ2XV3ACGHL S2x/O4d5ASPfKkaN4tSistQiXUNTvaSizPSMktzEzBxdQwMzvdzU4uLE9NScxKRiveT83E2Mw GhjAIIdjN+WBRxilORgUhLl3b1qW4QQX1J+SmVGYnFGfFFpTmrxIUYNDg6BCWfnTmeSYsnLz0 tVkuB1fAlUJ1iUmp5akZaZA0wHMKUSHDxKIrw3XwCleYsLEnOLM9MhUqcYFaXEeXVA+gRAEhm leXBtsBR0iVFWSpiXEegoIZ6C1KLczBJU+VeM4hyMSsK8MiBTeDLzSuCmvwJazAS0+IXKVpDF JYkIKakGRoUi1sUZjYlXn15oqhETiBe3nBPtPNXJqvhH0YsMgTZ9FX+ztwemrcrXfPu2U/GOX eSbHWl9O+7tfs3OISodt1O89lT/O3F7/5+OYZa7p/5OeDjridVMZxuX1lcOnpmz9374ejTk5V bBwr82xnnnE9/1M83PPCmVfoWbpZ4n+tdt1lVv50ruUWIpzkg01GIuKk4EALZSS3U8AwAA X-Env-Sender: prvs=226aaf366=dario.faggioli@citrix.com X-Msg-Ref: server-2.tower-21.messagelabs.com!1488382505!46217157!1 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 3054 invoked from network); 1 Mar 2017 15:35:06 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-2.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 1 Mar 2017 15:35:06 -0000 X-IronPort-AV: E=Sophos;i="5.35,226,1484006400"; d="asc'?scan'208";a="410608804" Message-ID: <1488382499.5548.144.camel@citrix.com> From: Dario Faggioli To: Jan Beulich Date: Wed, 1 Mar 2017 16:34:59 +0100 In-Reply-To: <148837861276.11900.8292677471375175885.stgit@Solace.fritz.box> References: <148837861276.11900.8292677471375175885.stgit@Solace.fritz.box> Organization: Citrix Inc. X-Mailer: Evolution 3.20.5 (3.20.5-1.fc24) MIME-Version: 1.0 Cc: George Dunlap , xen-devel@lists.xenproject.org Subject: Re: [Xen-devel] [PATCH v4 0/7] xen: credit2: improve style, and tracing; fix two bugs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP On Wed, 2017-03-01 at 15:52 +0100, Dario Faggioli wrote: > Dario Faggioli (7): >       xen: credit2: always mark a tickled pCPU as... tickled! >       xen: credit2: don't miss accounting while doing a credit reset. > And these two being bugfixes, I request them to be backported to Xen 4.7 (but not any further, as that would be difficult and pretty pointless). Since that isn't exactly trivial, especially for the first patch, I attach to this email what I actually propose to backport. I've successfully tested them on top of current tip of staging-4.7. Regards, Dario commit f36ce03fffab7526d9c94b46028a27a752e3f60e Author: Dario Faggioli Date: Wed Mar 1 16:22:43 2017 +0100 xen: credit2: don't miss accounting while doing a credit reset. A credit reset basically means going through all the vCPUs of a runqueue and altering their credits, as a consequence of a 'scheduling epoch' having come to an end. Blocked or runnable vCPUs are fine, all the credits they've spent running so far have been accounted to them when they were scheduled out. But if a vCPU is running on a pCPU, when a reset event occurs (on another pCPU), that does not get properly accounted. Let's therefore begin to do so, for better accuracy and fairness. In fact, after this patch, we see this in a trace: csched2:schedule cpu 10, rq# 1, busy, not tickled csched2:burn_credits d1v5, credit = 9998353, delta = 202996 runstate_continue d1v5 running->running ... csched2:schedule cpu 12, rq# 1, busy, not tickled csched2:burn_credits d1v6, credit = -1327, delta = 9999544 csched2:reset_credits d0v13, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v14, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v7, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:burn_credits d1v5, credit = 201805, delta = 9796548 csched2:reset_credits d1v5, credit_start = 201805, credit_end = 10201805, mult = 1 csched2:burn_credits d1v6, credit = -1327, delta = 0 csched2:reset_credits d1v6, credit_start = -1327, credit_end = 9998673, mult = 1 Which shows how d1v5 actually executed for ~9.796 ms, on pCPU 10, when reset_credit() is executed, on pCPU 12, because of d1v6's credits going below 0. Without this patch, this 9.796ms are not accounted to anyone. With this patch, d1v5 is charged for that, and its credits drop down from 9796548 to 201805. And this is important, as it means that it will begin the new epoch with 10201805 credits, instead of 10500000 (which he would have, before this patch). Basically, we were forgetting one round of accounting in epoch x, for the vCPUs that are running at the time the epoch ends. And this meant favouring a little bit these same vCPUs, in epoch x+1, providing them with the chance of execute longer than their fair share. Signed-off-by: Dario Faggioli diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 905fb12..c86f548 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -707,14 +707,23 @@ static void reset_credit(const struct scheduler *ops, int cpu, s_time_t now, list_for_each( iter, &rqd->svc ) { + unsigned int svc_cpu; struct csched2_vcpu * svc; int start_credit; svc = list_entry(iter, struct csched2_vcpu, rqd_elem); + svc_cpu = svc->vcpu->processor; BUG_ON( is_idle_vcpu(svc->vcpu) ); BUG_ON( svc->rqd != rqd ); + /* + * If svc is running, it is our responsibility to make sure, here, + * that the credit it has spent so far get accounted. + */ + if ( svc->vcpu == curr_on_cpu(svc_cpu) ) + burn_credits(rqd, svc, now); + start_credit = svc->credit; /* And add INIT * m, avoiding integer multiplication in the