From patchwork Tue Feb 28 11:52:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9595301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7C36D601D7 for ; Tue, 28 Feb 2017 11:54:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68E7F27A98 for ; Tue, 28 Feb 2017 11:54:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5D68E27F07; Tue, 28 Feb 2017 11:54:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DD23827816 for ; Tue, 28 Feb 2017 11:54:35 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cigKb-0001Ba-PL; Tue, 28 Feb 2017 11:52:37 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cigKa-0001Ax-7Q for xen-devel@lists.xenproject.org; Tue, 28 Feb 2017 11:52:36 +0000 Received: from [85.158.143.35] by server-3.bemta-6.messagelabs.com id 26/43-04244-38465B85; Tue, 28 Feb 2017 11:52:35 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpkleJIrShJLcpLzFFi42K5GNpwRLcpZWu EwaSZFhbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8bvWZuYCi5KV+w9voCtgfGLaBcjF4eQwAxG iWMNDYwgDovAGlaJTcs/M4M4EgKXWCUa7t5l72LkBHJiJB7Pn8EIYVdIPJ3TDRYXElCRuLl9F RPEqB+MEgtO/WUCSQgL6EkcOfqDHcKOkPh55AcriM0mYCDxZsdeMFtEQEni3qrJQPUcHMwC4R KrO7lBwiwCqhLtMyaB7eIV8JaYP2cjG0gJp4CPxPIbFiCmEFB42+NYkApRATmJlZdbWCGqBSV OznzCAjFQU2L9Ln2QMLOAvMT2t3OYJzCKzEJSNQuhahaSqgWMzKsY1YtTi8pSi3RN9JKKMtMz SnITM3N0DQ3M9HJTi4sT01NzEpOK9ZLzczcxAgOfAQh2MHZf9j/EKMnBpCTKm5W8NUKILyk/p TIjsTgjvqg0J7X4EKMGB4fAhLNzpzNJseTl56UqSfBOBqkTLEpNT61Iy8wBxiZMqQQHj5II74 8koDRvcUFibnFmOkTqFKMxx4NTu94wcXzqP/yGSQhskpQ4bxTIJAGQ0ozSPLhBsJRxiVFWSpi XEehMIZ6C1KLczBJU+VeM4hyMSsK8vSBTeDLzSuD2vQI6hQnolBcqYKeUJCKkpBoYffa+reRS to5uLdyzU7NG6Meq6P1753quVvtkLHTu5azEc8W7Nz7zTft/1cNh5sszjMF96VN6TOMqnHk04 1dMb5kSwbhuGUv2/Yfx7y9aBlQ+eTQ9df5/h7pA65xu6VXGjL6Bt97ZKX75rvolKsvao8NJ+s qa/rjuC5ujLebPYvvD5PSMZ1uYEktxRqKhFnNRcSIAHFFihRQDAAA= X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-12.tower-21.messagelabs.com!1488282754!59877518!1 X-Originating-IP: [209.85.128.196] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 45756 invoked from network); 28 Feb 2017 11:52:34 -0000 Received: from mail-wr0-f196.google.com (HELO mail-wr0-f196.google.com) (209.85.128.196) by server-12.tower-21.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 28 Feb 2017 11:52:34 -0000 Received: by mail-wr0-f196.google.com with SMTP id l37so1312731wrc.3 for ; Tue, 28 Feb 2017 03:52:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=FvIxmuy6bc/JlLv8T79XVi/fesyqYDxud2g5Sdw/+j8=; b=HAVMooV0fdpRwmxQtvr30WtZhwfcs2s3bZrfmARthke6ufh7BAYit5YX+NwveYnbKv 5pLc2c5RYGt6wXifnXObsi+zS36KSCE9A+KnscJsq3oNzJBxRxkUotkAx2DKcLZ442L0 s/R85Gfvs7lnLOwxDSjgvye5DM+X4zJs/D0eZQsRVLbAvDSU9qf/liUYFcWYlyxlGWMF QnP6swbdXMzX6ffw9orW8LU7rdG1CLqcUo9YgjIQ/GVQd9OLxrC1aWzG3rSQZ/7fb8Ha ioAT5gTh4wa22E6QzHpWFGoOXDhwDTL7xZa3jQf8VWGUkaqezIP5VmfTABKamv+QXiTE SiTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=FvIxmuy6bc/JlLv8T79XVi/fesyqYDxud2g5Sdw/+j8=; b=M8PwPf7XbilBqQ94fPim73h+6G7oUxELIPWRmsisBHG3SZ7qM5H/p6ilelqISx83pD Ue5tlDoifKcv5Aw/sTuRWXR/dHeST42p3EWdnXGL6/lkr3NECZ7Kt+Yj09oyxTreJcFH LO8PgjO61gyrfeCYyiID9/wMIFgkBr2mHvNwo2Y1legiujnhtjsGzYVPVT7FePCgNoaz EMkqFZnLH32fn/91t4Vhwq9Dhuu8SHTnBYnB4m0nYL+z/E/o+GtdLD92Fv6JboceK6d1 MywTqIqeLCY/Nx701ayC62Mkv9v7j24Z6h3ShTFo1R/6zvszjRo5yFb2uKPsDiigiJ7n MS/A== X-Gm-Message-State: AMke39nL0cW4M38CMzS+EknA8iNTiTv++8tqb7Om/geOg9wnUajCV4CIejdsuBtbwDsBrA== X-Received: by 10.223.164.9 with SMTP id d9mr2002159wra.146.1488282754268; Tue, 28 Feb 2017 03:52:34 -0800 (PST) Received: from Solace.fritz.box ([80.66.223.93]) by smtp.gmail.com with ESMTPSA id e74sm2371445wmd.2.2017.02.28.03.52.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Feb 2017 03:52:33 -0800 (PST) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Tue, 28 Feb 2017 12:52:32 +0100 Message-ID: <148828275224.26730.8510774707000628326.stgit@Solace.fritz.box> In-Reply-To: <148828109243.26730.2771577013485070217.stgit@Solace.fritz.box> References: <148828109243.26730.2771577013485070217.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: Anshul Makkar , George Dunlap Subject: [Xen-devel] [PATCH v3 5/7] xen: credit2: don't miss accounting while doing a credit reset. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP A credit reset basically means going through all the vCPUs of a runqueue and altering their credits, as a consequence of a 'scheduling epoch' having come to an end. Blocked or runnable vCPUs are fine, all the credits they've spent running so far have been accounted to them when they were scheduled out. But if a vCPU is running on a pCPU, when a reset event occurs (on another pCPU), that does not get properly accounted. Let's therefore begin to do so, for better accuracy and fairness. In fact, after this patch, we see this in a trace: csched2:schedule cpu 10, rq# 1, busy, not tickled csched2:burn_credits d1v5, credit = 9998353, delta = 202996 runstate_continue d1v5 running->running ... csched2:schedule cpu 12, rq# 1, busy, not tickled csched2:burn_credits d1v6, credit = -1327, delta = 9999544 csched2:reset_credits d0v13, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v14, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:reset_credits d0v7, credit_start = 10500000, credit_end = 10500000, mult = 1 csched2:burn_credits d1v5, credit = 201805, delta = 9796548 csched2:reset_credits d1v5, credit_start = 201805, credit_end = 10201805, mult = 1 csched2:burn_credits d1v6, credit = -1327, delta = 0 csched2:reset_credits d1v6, credit_start = -1327, credit_end = 9998673, mult = 1 Which shows how d1v5 actually executed for ~9.796 ms, on pCPU 10, when reset_credit() is executed, on pCPU 12, because of d1v6's credits going below 0. Without this patch, this 9.796ms are not accounted to anyone. With this patch, d1v5 is charged for that, and its credits drop down from 9796548 to 201805. And this is important, as it means that it will begin the new epoch with 10201805 credits, instead of 10500000 (which he would have, before this patch). Basically, we were forgetting one round of accounting in epoch x, for the vCPUs that are running at the time the epoch ends. And this meant favouring a little bit these same vCPUs, in epoch x+1, providing them with the chance of execute longer than their fair share. Signed-off-by: Dario Faggioli Reviewed-by: George Dunlap --- Cc: Anshul Makkar --- xen/common/sched_credit2.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index feb0f83..66b7f96 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1337,18 +1337,28 @@ static void reset_credit(const struct scheduler *ops, int cpu, s_time_t now, list_for_each( iter, &rqd->svc ) { + unsigned int svc_cpu; struct csched2_vcpu * svc; int start_credit; svc = list_entry(iter, struct csched2_vcpu, rqd_elem); + svc_cpu = svc->vcpu->processor; ASSERT(!is_idle_vcpu(svc->vcpu)); ASSERT(svc->rqd == rqd); + /* + * If svc is running, it is our responsibility to make sure, here, + * that the credit it has spent so far get accounted. + */ + if ( svc->vcpu == curr_on_cpu(svc_cpu) ) + burn_credits(rqd, svc, now); + start_credit = svc->credit; - /* And add INIT * m, avoiding integer multiplication in the - * common case. */ + /* + * Add INIT * m, avoiding integer multiplication in the common case. + */ if ( likely(m==1) ) svc->credit += CSCHED2_CREDIT_INIT; else