From patchwork Thu Feb 11 11:39:01 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 8277981 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 0F7249F88A for ; Thu, 11 Feb 2016 11:41:36 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 2036820373 for ; Thu, 11 Feb 2016 11:41:35 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2A4DF2035D for ; Thu, 11 Feb 2016 11:41:34 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aTpaV-0005zO-O3; Thu, 11 Feb 2016 11:39:07 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aTpaT-0005yd-Op for xen-devel@lists.xenproject.org; Thu, 11 Feb 2016 11:39:05 +0000 Received: from [85.158.137.68] by server-16.bemta-3.messagelabs.com id DF/D2-07451-9D27CB65; Thu, 11 Feb 2016 11:39:05 +0000 X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-2.tower-31.messagelabs.com!1455190744!21715690!1 X-Originating-IP: [74.125.82.67] X-SpamReason: No, hits=0.7 required=7.0 tests=BODY_RANDOM_LONG, RCVD_ILLEGAL_IP X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 2985 invoked from network); 11 Feb 2016 11:39:04 -0000 Received: from mail-wm0-f67.google.com (HELO mail-wm0-f67.google.com) (74.125.82.67) by server-2.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 11 Feb 2016 11:39:04 -0000 Received: by mail-wm0-f67.google.com with SMTP id g62so10140891wme.2 for ; Thu, 11 Feb 2016 03:39:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=P/pYk0EXoBcI01OyV+JRXvODNfyojSGt8Ejw0BJFBdQ=; b=Vmf8Y2GvLlSeVCZ0RfBErdjfaQDMJASja+coZLLitRa2ER/lQes+JsS54/QGHvvmGZ 3MimUUum8V0/gWV5Wa3zwS6sACgnSuRnXH/+pWRZ4FbTmqmUqtG3AiZj9WwdhI5gy6bB qw2K/PznZkFEJ7ZVwhxK3elqHP5F9YNsaFYr2Q8BTPlj4yEh4J550EWUYltbr5JEyPyP suuFXIXw+mIyIVnuQWnqJiMtXelVc7Zsnu5Xrix//Tu1b5giaLtmKekkaLdWCqpojR66 hjy3w9B5zaKw4xKplNIgpG/LzcLFm/Pld/YF61KCyl9MJrKnCvXY0vgpLfyui8Sde0pj GeDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version:content-type :content-transfer-encoding; bh=P/pYk0EXoBcI01OyV+JRXvODNfyojSGt8Ejw0BJFBdQ=; b=ZJdm7/GSm01S0JLYYTc4dHiGYPxhq1mLrwa7hX+4Mzq4OvBLOKQoYVT9JgJzE6bzYX HwWDhQxWd+EoMuNW4uo0Tf8oB6hFNiWPjIH1Vbk6e7UboLZWDP/HtP6jb2rHbQX6JrO7 xBXs3gPoqEO842D4AMH0qWpE3YAv+zSjPXxO5de4OckGFRPvfCy3vtclh7mgQLJ4LYtd 0OMf5A7AjGMvPoQsjtAwWYhB1aDRklvSnUdLjCd/LRbme3ENtTRq7S1yzKOcMQiGEjcz aR06CYwge2RwdcDnlirDkqQIqcmhfDKsyaAe1SZa6CTE6t/HLkXZ09YP+KKjrLJCwuyc CCMQ== X-Gm-Message-State: AG10YOTt6YNdMUScTUcRUUfeVZ1kGIkbaT8LigJ5FpeFaOdguGGUN6Ut2hTNv65Ha7lrVA== X-Received: by 10.194.21.135 with SMTP id v7mr45362505wje.131.1455190744372; Thu, 11 Feb 2016 03:39:04 -0800 (PST) Received: from Solace.station (net-2-35-170-8.cust.vodafonedsl.it. [2.35.170.8]) by smtp.gmail.com with ESMTPSA id v22sm7633264wmv.12.2016.02.11.03.39.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Feb 2016 03:39:03 -0800 (PST) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 11 Feb 2016 12:39:01 +0100 Message-ID: <20160211113901.20959.87801.stgit@Solace.station> In-Reply-To: <20160211113321.20959.53804.stgit@Solace.station> References: <20160211113321.20959.53804.stgit@Solace.station> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap Subject: [Xen-devel] [PATCH 3/3] xen: credit1: avoid boosting vCPUs being "just" migrated X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Moving a vCPU to a different pCPU means blocking it and waking it up (on the new pCPU). Credit1 grants BOOST priority to vCPUs that wakes up, with the aim of improving I/O latency. The end result is that vCPUs get boosted when migrating, which shouldn't happen. For instance, this causes scheduling anomalies and, potentially, performance problems, as reported here: http://lists.xen.org/archives/html/xen-devel/2015-10/msg02851.html This patch fixes things by introducing a new wakeup flag, and using it to tag the wakeups that happens because of migrations (and avoid boosting, in these cases). Signed-off-by: Dario Faggioli --- Cc: George Dunlap --- xen/common/sched_credit.c | 11 +++++++---- xen/common/schedule.c | 4 ++-- xen/include/xen/sched.h | 3 +++ 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index f728ddd..2a23a0b 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -1022,11 +1022,14 @@ csched_vcpu_wake(const struct scheduler *ops, struct vcpu *vc, unsigned wf) * more CPU resource intensive VCPUs without impacting overall * system fairness. * - * The one exception is for VCPUs of capped domains unpausing - * after earning credits they had overspent. We don't boost - * those. + * There are a couple of exceptions, when we don't want to boost: + * - VCPUs that are waking up after a migration, rather than + * after having block; + * - VCPUs of capped domains unpausing after earning credits + * they had overspent. */ - if ( svc->pri == CSCHED_PRI_TS_UNDER && + if ( !(wf & WF_migrated) && + svc->pri == CSCHED_PRI_TS_UNDER && !test_bit(CSCHED_FLAG_VCPU_PARKED, &svc->flags) ) { TRACE_2D(TRC_CSCHED_BOOST_START, vc->domain->domain_id, vc->vcpu_id); diff --git a/xen/common/schedule.c b/xen/common/schedule.c index ea74c96..c9a4f52 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -581,8 +581,8 @@ static void vcpu_migrate(struct vcpu *v) if ( old_cpu != new_cpu ) sched_move_irqs(v); - /* Wake on new CPU. */ - vcpu_wake(v); + /* Wake on new CPU (and let the scheduler know it's a migration). */ + _vcpu_wake(v, WF_migrated); } /* diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 9fdcfff..5f426ad 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -764,6 +764,9 @@ static inline struct domain *next_domain_in_cpupool( /* 'Default' wakeup. */ #define _WF_wakeup 0 #define WF_wakeup (1U<<_WF_wakeup) +/* Post-migration wakeup. */ +#define _WF_migrated 1 +#define WF_migrated (1U<<_WF_migrated) static inline int vcpu_runnable(struct vcpu *v) {