new file mode 100644
@@ -0,0 +1,123 @@
+% Credit2
+% Revision 1
+
+\clearpage
+
+# Basics
+---------------- ----------------------------------------------------
+ Status: e.g. **Supported**
+
+Architecture(s): e.g. x86, arm
+
+ Component: e.g. Hypervisor
+---------------- ----------------------------------------------------
+
+# Overview
+
+Credit2 is one of the virtual CPU (vCPU) scheduler available in the
+Xen hypervisor. The job of an hypervisor's virtual CPU scheduler is
+to decide, among all the various vCPUs of the various virtual machines,
+which ones should execute on the host's physical CPUs (pCPUs), at any
+given point in time.
+
+Credit2 was designed as a general purpose scheduler, with particular
+focus on improving handling of mixed workloads, scalability and
+support for low latency applications inside VMs, with respect to
+Credit1.
+
+# User details
+
+Credit2 is not in use by default. In order to use it as the Xen
+scheduler the following parameter should be passed to the hypervisor
+at boot:
+
+ `sched=credit2`
+
+Other parameters are available for tuning the behavior of Credit2
+(see `docs/misc/xen-command-line.markdown` for a complete list and
+for their meaning).
+
+Once the system is live, for creating a cpupool with Credit2 as
+its scheduler, either compile a cpupool configuration file, as
+described in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified
+in `tools/examples/cpupool`), or use just `xl` directly:
+
+ xl cpupool-create name=\"pool1\" sched=\"credit2\" cpus=[1,2]
+
+Two kind of interactions with the scheduler are possible:
+
+* checking or changing the global parameters, via, e.g.:
+ * `xl sched-credit2 -s`
+ * `xl sched-credit2 -s -p pool1`
+ * `xl sched-credit2 -s -r 100`
+* checking or changing a VM scheduling parameters, via, e.g.:
+ * `xl sched-credit2 -d vm1`
+ * `xl sched-credit2 -d vm1 -w 1024`
+
+# Technical details
+
+The implementation entirely lives in the hypervisor. Xen has a
+pluggable, hook based, architecture for schedulers. Credit2 code
+is all inside one file: `xen/common/sched_credit2.c`
+
+Global scheduling parameters, such as context switching rate
+limiting, is only available from Xen 4.8 onward. In libxl, the
+`LIBXL\_HAVE\_SCHED\_CREDIT2\_PARAMS` symbol is introduced to
+indicate their availability.
+
+# Limitations
+
+The Credit1 scheduler supports vCPU hard-affinity, vCPU soft-affinity
+and caps (see `docs/man/xl.pod.1.in` for more details). In Credit2,
+vCPU hard affinity is supported starting from Xen 4.8, while soft-affinity
+and caps are being implemented, but not supported yet in any released
+hypervisor.
+
+# Testing
+
+Any change done in Credit2 wants to be tested by doing at least the
+following:
+
+* boot the system with `sched=credit2`,
+* create a few virtual machine and verify that they boot and can
+ run some basic workload (e.g., login into them and run simple commands),
+* shutdown/reboot the virtual machines,
+* shutdown/reboot the system.
+
+Ideally, all the above steps should **also** be performed in a configuration
+where Credit2 is used as the scheduler of a cpupool, and by also doing the
+following:
+
+* move a virtual machine inside and outside a Credit2 cpupool.
+
+# Areas for improvement
+
+* Close the feature gap with Credit1 (i.e., finishing implementing vCPU
+ soft-affinity and caps);
+* vCPUs' reservations (similar to caps, but providing a vCPU with guarantees
+ about some pCPU time it will always be able to execute for);
+* benchmarking for assessing the best combination of values for the various
+ parameters (`sched\_credit2\_migrate\_resist`, `credit2\_balance\_over`,
+ `credit2\_balance\_under`)
+
+# Known issues
+
+* I/O oriented benchmarks (like network and disk throughput) have given
+ contradictory and non-conclusive results so far. Need to run more of
+ those.
+
+# References
+
+* "Scheduler development update", XenSummit Asia 2009 [whitepaper](http://www-archive.xenproject.org/files/xensummit_intel09/George_Dunlap.pdf)
+* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf)
+* "Scope and Performance of Credit-2 Scheduler" [XPDS16 Presentation](http://www.slideshare.net/xen_com_mgr/xpds16-scope-and-performance-of-credit2-scheduler-anshul-makkar-ctirix-systems-uk-ltd)
+* [Credit2 Scheduler Development](https://wiki.xenproject.org/wiki/Credit2_Scheduler_Development)
+* [Xen Project Scheduler](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers)
+
+# History
+
+------------------------------------------------------------------------
+Date Revision Version Notes
+---------- -------- -------- -------------------------------------------
+2016-10-10 1 Xen 4.8 Document written
+---------- -------- -------- -------------------------------------------
@@ -2954,8 +2954,6 @@ csched2_init(struct scheduler *ops)
struct csched2_private *prv;
printk("Initializing Credit2 scheduler\n");
- printk(" WARNING: This is experimental software in development.\n" \
- " Use at your own risk.\n");
printk(XENLOG_INFO " load_precision_shift: %d\n"
XENLOG_INFO " load_window_shift: %d\n"
Since we are marking the feature as 'Supported', remove the "this is experimental software" warning in the code at once. Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> --- Cc: George Dunlap <George.Dunlap@eu.citrix.com> Cc: Anshul Makkar <anshul.makkar@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Lars Kurth <lars.kurth@citrix.com> Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Ian Jackson <ian.jackson@eu.citrix.com> Cc: Jan Beulich <jbeulich@suse.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Stefano Stabellini <sstabellini@kernel.org> --- docs/features/credit2.pandoc | 123 ++++++++++++++++++++++++++++++++++++++++++ xen/common/sched_credit2.c | 2 - 2 files changed, 123 insertions(+), 2 deletions(-) create mode 100644 docs/features/credit2.pandoc