From patchwork Thu Mar 2 10:38:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9599909 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7C3D060414 for ; Thu, 2 Mar 2017 10:40:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A7FA284F4 for ; Thu, 2 Mar 2017 10:40:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4EF112858C; Thu, 2 Mar 2017 10:40:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E0A26284F4 for ; Thu, 2 Mar 2017 10:40:40 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjO80-0004GM-Ow; Thu, 02 Mar 2017 10:38:32 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjO7y-0004E2-K5 for xen-devel@lists.xenproject.org; Thu, 02 Mar 2017 10:38:30 +0000 Received: from [85.158.143.35] by server-10.bemta-6.messagelabs.com id D4/43-13192-526F7B85; Thu, 02 Mar 2017 10:38:29 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrFIsWRWlGSWpSXmKPExsXiVRvkrKv6bXu EwbETIhbft0xmcmD0OPzhCksAYxRrZl5SfkUCa0bvDoWC/eoVq7/eYW1g7JPtYuTiEBKYziix 5NYb5i5GTg4WgTWsEgeWeYMkJAQusUp8+dfBApKQEIiRuNs/n7WLkQPIrpRoO1QPEhYSUJG4u X0VE8SgH4wSi9ZMZAdJCAvoSRw5+gPK9pHYPukmmM0mYCDxZsdeVhBbREBJ4t6qyUwgM5kF9C VWf+WBuEFVYteBfrBVvECtD77bgYQ5BXwlOv9eYIVY6yPxY2Y/mC0qICex8nILmM0rIChxcuY TFoiJmhLrd+mDhJkF5CW2v53DPIFRZBaSqlkIVbOQVC1gZF7FqFGcWlSWWqRrZKKXVJSZnlGS m5iZo2toYKaXm1pcnJiempOYVKyXnJ+7iREY9gxAsINx38fIQ4ySHExKory8z7ZHCPEl5adUZ iQWZ8QXleakFh9i1ODgEJhwdu50JimWvPy8VCUJ3vNfgOoEi1LTUyvSMnOAkQlTKsHBoyTC2w qS5i0uSMwtzkyHSJ1i1OX40H/4DZMQ2Awpcd4GkCIBkKKM0jy4EbAkcYlRVkqYlxHoQCGegtS i3MwSVPlXjOIcjErCvH9BpvBk5pXAbXoFdAQT0BEvVLaCHFGSiJCSamBcIDj5/zqh0rxvx4Q+ FKWsWvbfwerjxTl5F3Y9T1CQcTyYu3JFRtr8BdZm6qsnrPP+3hSw6lH23ok8ha8+XG6s2P3he ti0wkBRA8mD75xExTuN/sUl7qs6/PACn3XXtksn1jvyLW7Nf30tfXb53j/TvGQiZ0tzLnyU8l /4rNvirVn/95pp9tbpK7EUZyQaajEXFScCAJYrpUsNAwAA X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-3.tower-21.messagelabs.com!1488451109!56775130!1 X-Originating-IP: [74.125.82.67] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 61078 invoked from network); 2 Mar 2017 10:38:29 -0000 Received: from mail-wm0-f67.google.com (HELO mail-wm0-f67.google.com) (74.125.82.67) by server-3.tower-21.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 2 Mar 2017 10:38:29 -0000 Received: by mail-wm0-f67.google.com with SMTP id n11so4176978wma.0 for ; Thu, 02 Mar 2017 02:38:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=QRTWjjhOfMEsMyAuJgxmkVhCcOiOF4YAdFdRw4cyVEI=; b=QDS3vR4B5d68hguvqVQiZAirMK6rnyCmCXM3LDchX+OH3+5IQfm1gzEYYHNBx/7EM+ BHFHHPfP5JHQQvO/tc/8ECi1Q/7dvHk0y+Muys+lv+jtjgsU6InO61HpThbCKEFzcbhs jTXYHiA08YbHw5ALg0AA4tNlzDkzuqI3veAXzdOveoH2CJx7JqDIUbmendUoJ2EnhlPh FBh1QjGcvOYMUraP/8mTDBSQGn8mvVqF//5It36k7JdnHdIi1FdHBFptwvGzxfUMC27X GVTCePlhmBfGIZbFnIgJeNt6t8BOX+ZndRTrUlhB0wGlZ/6EedrWas/LzDXmqPmDCvZl QAZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=QRTWjjhOfMEsMyAuJgxmkVhCcOiOF4YAdFdRw4cyVEI=; b=mhBGOOWmuqEE8PWqJUzF7zwnzmmBeGbtg+xi8jCCOEXCI6QsAM19jpryrOtbJldUzO MhW9EuuPbl0KaQiMpY8P+VLxIsYRs4CyIbN5E7LtWXqd/o3hWhPMK/HNXx0AS6QiZ+YL MDDfTIpQ/Nu8Y4JRsi19Qps60UBfB64rXr/2o4K5aDOjHR9Gw3pIiCKR5JbK7NmMm25X Qn6OuTLjRLbqjx8tUubMxGAMn7jgAG7Ebr0UzQ+VI7GvYwdCu4HSn3Miqh5x8bV9j6oO zyrZ3Xv6MCgi40pvsVBpLC7d6BGn3cFyGqU7VO8Yq8HPCvFD6/CXuJrYtCsUhAxkNXiY rOQA== X-Gm-Message-State: AMke39nrlf6chqv08ug/59PTRCiWLLBL3JYpL8WxHSeaTLeUzM893sILpGtS9sIBFfqoJQ== X-Received: by 10.28.130.207 with SMTP id e198mr7406566wmd.4.1488451108946; Thu, 02 Mar 2017 02:38:28 -0800 (PST) Received: from Solace.fritz.box ([80.66.223.93]) by smtp.gmail.com with ESMTPSA id q1sm10661266wmd.6.2017.03.02.02.38.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Mar 2017 02:38:28 -0800 (PST) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Thu, 02 Mar 2017 11:38:27 +0100 Message-ID: <148845110684.23452.11988704692319209140.stgit@Solace.fritz.box> In-Reply-To: <148844531279.23452.17528540110704914171.stgit@Solace.fritz.box> References: <148844531279.23452.17528540110704914171.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap Subject: [Xen-devel] [PATCH 4/6] xen: credit1: treat pCPUs more evenly during balancing. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Right now, we use cpumask_first() for going through the bus pCPUs in csched_load_balance(). This means not all pCPUs have equal chances of seeing their pending work stolen. It also means there is more runqueue lock pressure on lower ID pCPUs. To avoid all this, let's record and remember, for each NUMA node, from what pCPU we have stolen for last, and start from that the following time. Signed-off-by: Dario Faggioli --- Cc: George Dunlap --- xen/common/sched_credit.c | 37 +++++++++++++++++++++++++++++++++---- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 529b6c7..bae29a7 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -229,6 +229,7 @@ struct csched_private { uint32_t credit; int credit_balance; uint32_t runq_sort; + uint32_t *balance_bias; unsigned ratelimit_us; /* Period of master and tick in milliseconds */ unsigned tslice_ms, tick_period_us, ticks_per_tslice; @@ -548,6 +549,7 @@ csched_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu) { struct csched_private *prv = CSCHED_PRIV(ops); struct csched_pcpu *spc = pcpu; + unsigned int node = cpu_to_node(cpu); unsigned long flags; /* @@ -571,6 +573,12 @@ csched_deinit_pdata(const struct scheduler *ops, void *pcpu, int cpu) prv->master = cpumask_first(prv->cpus); migrate_timer(&prv->master_ticker, prv->master); } + if ( prv->balance_bias[node] == cpu ) + { + cpumask_and(cpumask_scratch, prv->cpus, &node_to_cpumask(node)); + if ( !cpumask_empty(cpumask_scratch) ) + prv->balance_bias[node] = cpumask_first(cpumask_scratch); + } kill_timer(&spc->ticker); if ( prv->ncpus == 0 ) kill_timer(&prv->master_ticker); @@ -610,6 +618,10 @@ init_pdata(struct csched_private *prv, struct csched_pcpu *spc, int cpu) NOW() + MILLISECS(prv->tslice_ms)); } + cpumask_and(cpumask_scratch, prv->cpus, &node_to_cpumask(cpu_to_node(cpu))); + if ( cpumask_weight(cpumask_scratch) == 1 ) + prv->balance_bias[cpu_to_node(cpu)] = cpu; + init_timer(&spc->ticker, csched_tick, (void *)(unsigned long)cpu, cpu); set_timer(&spc->ticker, NOW() + MICROSECS(prv->tick_period_us) ); @@ -1696,7 +1708,7 @@ csched_load_balance(struct csched_private *prv, int cpu, struct csched_vcpu *speer; cpumask_t workers; cpumask_t *online; - int peer_cpu, peer_node, bstep; + int peer_cpu, first_cpu, peer_node, bstep; int node = cpu_to_node(cpu); BUG_ON( cpu != snext->vcpu->processor ); @@ -1740,9 +1752,10 @@ csched_load_balance(struct csched_private *prv, int cpu, cpumask_and(&workers, &workers, &node_to_cpumask(peer_node)); __cpumask_clear_cpu(cpu, &workers); - peer_cpu = cpumask_first(&workers); - if ( peer_cpu >= nr_cpu_ids ) + first_cpu = cpumask_cycle(prv->balance_bias[peer_node], &workers); + if ( first_cpu >= nr_cpu_ids ) goto next_node; + peer_cpu = first_cpu; do { /* @@ -1770,12 +1783,18 @@ csched_load_balance(struct csched_private *prv, int cpu, if ( speer != NULL ) { *stolen = 1; + /* + * Next time we'll look for work to steal on this node, we + * will start from the next pCPU, with respect to this one, + * so we don't risk stealing always from the same ones. + */ + prv->balance_bias[peer_node] = peer_cpu; return speer; } peer_cpu = cpumask_cycle(peer_cpu, &workers); - } while( peer_cpu != cpumask_first(&workers) ); + } while( peer_cpu != first_cpu ); next_node: peer_node = cycle_node(peer_node, node_online_map); @@ -2126,6 +2145,14 @@ csched_init(struct scheduler *ops) prv = xzalloc(struct csched_private); if ( prv == NULL ) return -ENOMEM; + + prv->balance_bias = xzalloc_array(uint32_t, MAX_NUMNODES); + if ( prv->balance_bias == NULL ) + { + xfree(prv); + return -ENOMEM; + } + if ( !zalloc_cpumask_var(&prv->cpus) || !zalloc_cpumask_var(&prv->idlers) || !zalloc_cpumask_var(&prv->overloaded) ) @@ -2133,6 +2160,7 @@ csched_init(struct scheduler *ops) free_cpumask_var(prv->overloaded); free_cpumask_var(prv->idlers); free_cpumask_var(prv->cpus); + xfree(prv->balance_bias); xfree(prv); return -ENOMEM; } @@ -2179,6 +2207,7 @@ csched_deinit(struct scheduler *ops) free_cpumask_var(prv->cpus); free_cpumask_var(prv->idlers); free_cpumask_var(prv->overloaded); + xfree(prv->balance_bias); xfree(prv); } }