From patchwork Fri Oct 14 10:02:25 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9376353 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 915D46075E for ; Fri, 14 Oct 2016 10:05:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 818FB2A5F9 for ; Fri, 14 Oct 2016 10:05:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 753D52A5FC; Fri, 14 Oct 2016 10:05:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A27E52A5F9 for ; Fri, 14 Oct 2016 10:05:07 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1buzKL-0003nA-Lk; Fri, 14 Oct 2016 10:02:57 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1buzKJ-0003mi-Mj for xen-devel@lists.xenproject.org; Fri, 14 Oct 2016 10:02:55 +0000 Received: from [85.158.139.211] by server-12.bemta-5.messagelabs.com id C6/DA-09561-F4DA0085; Fri, 14 Oct 2016 10:02:55 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrFIsWRWlGSWpSXmKPExsVyMfSqi67fWoY Ig6YOdovvWyYzOTB6HP5whSWAMYo1My8pvyKBNePZsXOMBT/0KiasPcLWwDhFo4uRi0NIYBqj RN/UD8wgDovAGlaJw2sXs4A4EgKXWCWOTNoBlOEEcmIkvrw9yAhhV0v8W7CYCcQWElCRuLl9F RPEqPlMEs/nnmIFSQgL6EkcOfqDHcI2kXizuY0FxGYTMJB4s2MvWI2IgJLEvVWTwZqZBV4xSb xpXwLWwCKgKvH3wRMwm1fAReLJ69lgzZwCbhJ7WmcxQ2x2lTjc9ALsIlEBOYmVl1tYIeoFJU7 OfAJUzwE0VFNi/S59kDCzgLzE9rdzmCcwisxCUjULoWoWkqoFjMyrGDWKU4vKUot0DY30kooy 0zNKchMzc3QNDUz1clOLixPTU3MSk4r1kvNzNzECY4ABCHYw9s1yPsQoycGkJMpbq8cQIcSXl J9SmZFYnBFfVJqTWnyIUYaDQ0mCl3kNUE6wKDU9tSItMwcYjTBpCQ4eJRHeTauB0rzFBYm5xZ npEKlTjMYcW35fW8vEsW3qvbVMQix5+XmpUuK8R0FKBUBKM0rz4AbBksQlRlkpYV5GoNOEeAp Si3IzS1DlXzGKczAqCfPuB5nCk5lXArfvFdApTECnfGgDO6UkESEl1cDYkPytb1VQjl1O94lF 1un74yvqCt3u6Ycxav5pEP8/u+xNyNMfLn4MojfLJS4FM/8J4fEIl3j93C2yTeiSSYgfH+8hB VnpuVO8D+098qQ+cg9f5CT/87mzGlvP8ulHcc3jVqjy7NUsN3QuPCb274m+lPDWUt6oCCGPDK 9/cw6HND3+f7H4mhJLcUaioRZzUXEiAFFXiWMNAwAA X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-13.tower-206.messagelabs.com!1476439373!49265900!1 X-Originating-IP: [209.85.213.68] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.84; banners=-,-,- X-VirusChecked: Checked Received: (qmail 53205 invoked from network); 14 Oct 2016 10:02:53 -0000 Received: from mail-vk0-f68.google.com (HELO mail-vk0-f68.google.com) (209.85.213.68) by server-13.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 14 Oct 2016 10:02:53 -0000 Received: by mail-vk0-f68.google.com with SMTP id 83so3707737vkd.0 for ; Fri, 14 Oct 2016 03:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=1fRcWz8Sl9P1toqDm4qDjNN0jzG4U1ZRr0fmIERL9zw=; b=GJAksKDObAF/gI2LhO36ch7ol69R2/SteMSFL/4AS8OsUBe1Oq5gplbuPd2ZDbhn68 n/IjXSO8/tKKvmDKsueIAKKmLgSDCarV/EbgxYdG6YQ469a8m4SPVuXwrZyRGzi8yrXJ PLX3N4J2Ea4wmck565e6zEFrc81Zshsq3n/yNBGwKGfLZn7JnPlhC4CuG/oxWTW1QDlu u0phuSRc1jEmDOTevSD++YtbRfs33JwPExK+6X029fOPReLurfRuP5lWkuOB09ltrpeU X2cc4j9lqFeHmxLAebuZD8oka97F8uxbQfHCpQbiK7YH/Gti2onN3pcPpqucGHLXzaAj toFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=1fRcWz8Sl9P1toqDm4qDjNN0jzG4U1ZRr0fmIERL9zw=; b=RMKPhKEXAEjQeAU9tbxQE8bNiXoogTu8ZqFplC/weBya6VMPM6lqX4kU2B7XsjAMJy IrwZuhq8EFjwQ37YMNEJfE8F0SyWp7GrvAOSuviJ7/2x2a+9Ny4F6zu201DBKDQn/s/7 N7JjH+uGtni9xb3/0dDtlxU9jRW7UWpVus2CL6UcZbKJNYg+XB4UZPAB721Hy0076IOz tqwT5kc4bs9zqGnzY+WAEHzcDTV7Ftak/E3z5LXk52PqpFODWkch/5dDLEidjPfF7E74 hXz80DrM4KM0/wqjF4smOcjWcgMrX9DtISCdXWMLsCPKNSTjzsvDI4Gxa/om117swPu3 xnTQ== X-Gm-Message-State: AA6/9RkpYAHQtSB06DN4Xb0i2qLaIUsfkQdmnNsBkhR55qjiM7edNmO+oeLwWlYK74flPQ== X-Received: by 10.31.199.199 with SMTP id x190mr8658689vkf.117.1476439372746; Fri, 14 Oct 2016 03:02:52 -0700 (PDT) Received: from [10.80.3.155] (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id x195sm7078410vkd.5.2016.10.14.03.02.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Oct 2016 03:02:51 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 14 Oct 2016 11:02:25 +0100 Message-ID: <147643933300.13134.5578757035260154511.stgit@Palanthas> In-Reply-To: <147643879557.13134.13090577186331555974.stgit@Palanthas> References: <147643879557.13134.13090577186331555974.stgit@Palanthas> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: Lars Kurth , Stefano Stabellini , Wei Liu , George Dunlap , Andrew Cooper , Ian Jackson , Meng Xu , Jan Beulich Subject: [Xen-devel] [PATCH v2 3/3] docs: RTDS feature document. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Dario Faggioli --- Cc: Meng Xu Cc: George Dunlap Cc: Wei Liu Cc: Lars Kurth Cc: Andrew Cooper Cc: Ian Jackson Cc: Jan Beulich Cc: Konrad Rzeszutek Wilk Cc: Stefano Stabellini --- Changes from v1: * file renamed from rtds.pandoc to sched_rtds.pandoc; * feature name is now 'RTDS Scheduler'; * kill the 'e.g.'-s in the header (sorry!); * removed 'Architecture' line from header, as this is arch independent; * removed some text that was duplicated in other sched feature documents; it's redundant wrt what's in the wiki about schedulers, and the page is referenced. --- docs/features/sched_rtds.pandoc | 121 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 docs/features/sched_rtds.pandoc diff --git a/docs/features/sched_rtds.pandoc b/docs/features/sched_rtds.pandoc new file mode 100644 index 0000000..2843d97 --- /dev/null +++ b/docs/features/sched_rtds.pandoc @@ -0,0 +1,121 @@ +% RTDS Scheduler +% Revision 1 + +\clearpage + +# Basics +---------------- ---------------------------------------------------- + Status: **Experimental** + + Component: Hypervisor +---------------- ---------------------------------------------------- + +# Overview + +RTDS is one of the virtual CPU (vCPU) scheduler available in the Xen +hypervisor. + +RTDS is a real--time scheduler, so its purpose is enabling +**deterministic** scheduling of the virtual machine's vCPUs. It has +been originally developed in the context of the RT-Xen project. + +# User details + +RTDS is not in use by default. In order to use it as the Xen scheduler +the following parameter should be passed to the hypervisor at boot: + + `sched=rtds` + +Once the system is live, for creating a cpupool with RTDS as its +scheduler, either compile a cpupool configuration file, as described +in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in +`tools/examples/cpupool`), or use just `xl` directly: + + xl cpupool-create name=\"pool-rt\" sched=\"rtds\" cpus=[4,5,6,8] + +For checking or changing a VM's scheduling parameters from xl, do +as follows: + * `xl sched-rtds -d vm-rt` + * `xl sched-rtds -d vm-rt -t 10000 -b 25000` + +It is possible, for a multiple vCPUs VM, to change the parameters of +each vCPU individually: + * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -v 1 -p 45000 -b 12000` + +# Technical details + +Implementation entirely lives in the hypervisor. Xen has a pluggable, +hook based, architecture for schedulers. Thanks to this, RTDS code +is all contained in `xen/common/sched_rtds.c`. + +In libxl, the availability of the RTDS scheduler is advertised by +the presence of the LIBXL\_HAVE\_SCHED\_RTDS symbol. The ability of +specifying different scheduling parameters for each vcpu has been +introduced later, and is available if the following symbols are defined: + * `LIBXL\_HAVE\_VCPU\_SCHED\_PARAMS`, + * `LIBXL\_HAVE\_SCHED\_RTDS\_VCPU\_PARAMS`. + +# Limitations + +RTDS is a special purpose scheduling. This is by design, and not at +all a limitation, but it is certainly something to keep in mind when +thinking about using it. The purpose of the scheduler is enabling +deterministic and statically analyzable behavior (as per the +real-time academic literature), according to the scheduling parameters +assigned to each vCPU. + +Using RTDS a the Xen scheduler, and/or for general purpose workloads +is definitely possible, but the vCPU scheduling parameters (of both +Domain0 and of the various VMs) would probably require tweaking, with +respect to their default values. + +# Testing + +Any change done in RTDS must be tested by doing the following: + +* create a cpupool with RTDS as its scheduler, +* create a few virtual machines a move them in and out of the pool, +* create a few virtual machines, directly inside the pool, and verify + that they boot and can run some basic workload (e.g., login into them + and run simple commands), +* shutdown/reboot the virtual machines, + +The fact that the system boots fine when passing `sched=rtds` to Xen +should also be verified. + +Finally, to check that the scheduler is working properly (although only +at a macroscopic level), the following should be done: + +* create a VM with 1 vCPU and put it in the RTDS cpupool, +* set the scheduling parameters such as it has a 50% reservation, with + `xl sched-rtds -d vm -t 100000 -b 50000`, +* run a CPU-burning process inside the VM (e.g., `yes`), +* check with `xentop` (in Domain0) that the VM is getting no more than + 50% pCPU time. + +# Areas for improvement + +* Work-conserving mode to be added; +* performance assessment, especially focusing on what level of real-time + behavior the scheduler enables. + +# Known issues + +* OSSTest reports occasional failures on ARM. + +# References + +* "RT-Xen: Real-Time Virtualization" [XPDS14 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/2014_Xen_Developer_Summit_0.pdf) +* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf) +* [RT-Xen Project](https://sites.google.com/site/realtimexen/) +* [RTDS-Based-Scheduler](https://wiki.xenproject.org/wiki/RTDS-Based-Scheduler) +* "The RTDS Scheduler" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/RTDS-Based-Scheduler) +* "Xen Project Schedulers" [on the Xen-Project wiki](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers) + +# History + +------------------------------------------------------------------------ +Date Revision Version Notes +---------- -------- -------- ------------------------------------------- +2016-10-14 1 Xen 4.8 Document written +---------- -------- -------- -------------------------------------------