| Message ID | 147635653648.17095.14880201185403654377.stgit@Palanthas (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Oct 13, 2016 at 12:02:28PM +0100, Dario Faggioli wrote: > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> > --- > Cc: George Dunlap <George.Dunlap@eu.citrix.com> > Cc: Wei Liu <wei.liu2@citrix.com> > Cc: Lars Kurth <lars.kurth@citrix.com> > Cc: Andrew Cooper <andrew.cooper3@citrix.com> > Cc: Ian Jackson <ian.jackson@eu.citrix.com> > Cc: Jan Beulich <jbeulich@suse.com> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > Cc: Stefano Stabellini <sstabellini@kernel.org> > --- > docs/features/credit.pandoc | 99 +++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 99 insertions(+) > create mode 100644 docs/features/credit.pandoc > > diff --git a/docs/features/credit.pandoc b/docs/features/credit.pandoc > new file mode 100644 > index 0000000..fed0da2 > --- /dev/null > +++ b/docs/features/credit.pandoc > @@ -0,0 +1,99 @@ > +% Credit > +% Revision 1 > + > +\clearpage > + > +# Basics > +---------------- ---------------------------------------------------- > + Status: e.g. **Supported** > + Delete the "e.g." in all three documents, please. > +Architecture(s): e.g. x86, arm > + And this should probably be "any". Wei.
On 13/10/16 12:02, Dario Faggioli wrote: > diff --git a/docs/features/credit.pandoc b/docs/features/credit.pandoc > new file mode 100644 > index 0000000..fed0da2 > --- /dev/null > +++ b/docs/features/credit.pandoc Simply "Credit" as a top level feature isn't very descriptive. Can you see about working scheduler somewhere into the name? > @@ -0,0 +1,99 @@ > +% Credit > +% Revision 1 > + > +\clearpage > + > +# Basics > +---------------- ---------------------------------------------------- > + Status: e.g. **Supported** > + > +Architecture(s): e.g. x86, arm > + > + Component: e.g. Hypervisor > +---------------- ---------------------------------------------------- You should drop the e.g.'s. In cases like this where it really is just a software algorithm, I would suggest setting the architecture to all, or omitting the line entirely. I would expect that any new architecture is going gain all the schedulers without modification? > + > +# Overview > + > +Credit (also known as Credit1) is the default virtual CPU (vCPU) scheduler > +of the Xen hypervisor. The job of an hypervisor's scheduler is to decide, > +among all the various vCPUs of the various virtual machines, which ones > +should execute on the host's physical CPUs (pCPUs), at any given point in > +time. A lot of this is generic to all schedulers. I wonder whether it might be better to have a schedulers meta-feature doc which deals with the common scheduler parts, including interactions on the Xen command line, xl, etc. > + > +# User details > + > +Xen supports multiple schedulers. As said, Credit is the default, so it > +is used automatically, unless the `sched=$SCHED` (with `$SCHED` different > +than `credit`) parameter is passed to Xen via the bootloader. > + > +Once the system is live, for creating a cpupool with Credit as its > +scheduler, either compile a cpupool configuration file, as described > +in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in > +`tools/examples/cpupool`), or use just `xl` directly: I should see about ensuring that cross-references work with the HTML-generated versions of these docs. You might be able to get away with just putting in a plain hyperlink here. ~Andrew
On Thu, 2016-10-13 at 12:47 +0100, Andrew Cooper wrote: > On 13/10/16 12:02, Dario Faggioli wrote: > > > > diff --git a/docs/features/credit.pandoc > > b/docs/features/credit.pandoc > > new file mode 100644 > > index 0000000..fed0da2 > > --- /dev/null > > +++ b/docs/features/credit.pandoc > > Simply "Credit" as a top level feature isn't very descriptive. Can > you > see about working scheduler somewhere into the name? > Yep, I wasn't sure whether or not to do that. Re-thinking things, I agree that'd be better. I'll do. > > @@ -0,0 +1,99 @@ > > +% Credit > > +% Revision 1 > > + > > +\clearpage > > + > > +# Basics > > +---------------- ----------------------------------------------- > > ----- > > + Status: e.g. **Supported** > > + > > +Architecture(s): e.g. x86, arm > > + > > + Component: e.g. Hypervisor > > +---------------- ----------------------------------------------- > > ----- > > You should drop the e.g.'s. > Which I was sure I'd have done... sorry. > In cases like this where it really is just > a software algorithm, I would suggest setting the architecture to > all, > or omitting the line entirely. > Omitting the line is what I also was considering myself. Again, will do. > > +# Overview > > + > > +Credit (also known as Credit1) is the default virtual CPU (vCPU) > > scheduler > > +of the Xen hypervisor. The job of an hypervisor's scheduler is to > > decide, > > +among all the various vCPUs of the various virtual machines, which > > ones > > +should execute on the host's physical CPUs (pCPUs), at any given > > point in > > +time. > > A lot of this is generic to all schedulers. > Not really. Well, sure some is, but, at the end, this period is pretty much the only one that is present, identical to itself, in all the three documents (and I certainly can see about shortening or removing it, if we don't want that). And in fact, the rest... > I wonder whether it might be better to have a schedulers meta-feature > doc which deals with the common scheduler parts, including > interactions > on the Xen command line, xl, etc. > ...may look similar, but they're subtle differences spread around. And the more subtle those differences, the higher the amount of cross- referencing between different documents would be, making it more difficult to read and understand what the situation is for one specific scheduler. xl interface is a good example: sub-commands are very similar, but then the scheduling parameters are different for each scheduler. The way in which you create a cpupool is the same (modulo the name=""), but doesn't necessarily have to be, e.g., if we start allowing specifing some of the global parameters of the scheduler on the command line (e.g., "create a Credit cpupool, but with timeslice=10"). Not possible right now, but doable, and even convenient (I've already have plans for that :-P). So, FWIW, I would stick with different documents. > > +Once the system is live, for creating a cpupool with Credit as its > > +scheduler, either compile a cpupool configuration file, as > > described > > +in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in > > +`tools/examples/cpupool`), or use just `xl` directly: > > I should see about ensuring that cross-references work with the > HTML-generated versions of these docs. You might be able to get away > with just putting in a plain hyperlink here. > I thought about that, but then ended up following suit from your docs/feature/migration.pandoc. I'll turn this in links if that's what you think is best. Personally, I 's say it makes the _text_ document a bit less readable, but I guess the version we care about is the _HTML_ one? Anyway, I'm basically ok with anything. :-) Thanks and Regards, Dario
diff --git a/docs/features/credit.pandoc b/docs/features/credit.pandoc new file mode 100644 index 0000000..fed0da2 --- /dev/null +++ b/docs/features/credit.pandoc @@ -0,0 +1,99 @@ +% Credit +% Revision 1 + +\clearpage + +# Basics +---------------- ---------------------------------------------------- + Status: e.g. **Supported** + +Architecture(s): e.g. x86, arm + + Component: e.g. Hypervisor +---------------- ---------------------------------------------------- + +# Overview + +Credit (also known as Credit1) is the default virtual CPU (vCPU) scheduler +of the Xen hypervisor. The job of an hypervisor's scheduler is to decide, +among all the various vCPUs of the various virtual machines, which ones +should execute on the host's physical CPUs (pCPUs), at any given point in +time. + +# User details + +Xen supports multiple schedulers. As said, Credit is the default, so it +is used automatically, unless the `sched=$SCHED` (with `$SCHED` different +than `credit`) parameter is passed to Xen via the bootloader. + +Once the system is live, for creating a cpupool with Credit as its +scheduler, either compile a cpupool configuration file, as described +in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in +`tools/examples/cpupool`), or use just `xl` directly: + + xl cpupool-create name=\"pool1\" sched=\"credit\" cpus=[4,8] + +Two kind of interactions with the scheduler are possible: + +* checking or changing the global parameters, via, e.g.: + * `xl sched-credit -s` + * `xl sched-credit -s -p pool1` + * `xl sched-credit -s -t 20` +* checking or changing a VM's scheduling parameters, via, e.g.: + * `xl sched-credit -d vm1` + * `xl sched-credit -d vm1 -w 512` + +# Technical details + +Implementation entirely lives in the hypervisor. Xen has a pluggable, hook +based, architecture for schedulers. Actual Credit code is all inside one +file: `xen/common/sched_credit.c`. + +# Limitations + +In Credit, a vCPU has a priority, a status (i.e., active or inactive), +a weight and some credits... and all these things interact in a rather +involved way. Also, with years of use, things have gotten even more +complex (due to, e.g., the introduction of boosting, caps and vCPU +soft-affinity). + +Dealing with such complexity is starting to be an issue. Odd behavior +or subtle scheduling anomalies, that is not always possible to act upon, +have been identified already. [1][2][3] + +A certain lack of scalability and difficulties and weakness in dealing +with mixed workloads and VMs with low latency requirements are other +known problems. [4] For all these reasons, effort is ongoing to have +Credit2 become the new default scheduler. + +# Testing + +Any change to Credit code must to be tested by doing at least the following: + +* create a few virtual machine and verify that they boot and can + run some basic workload (e.g., login into them and run simple commands), +* shutdown/reboot the virtual machines, +* shutdown the system. + +Ideally, all the above steps should **also** be performed in a configuration +that includes cpupools, better if with pools using different schedulers, and +by also doing the following: + +* move the virtual machines between cpupools. + +# References + +* [potential non-ideal behavior on hyperthreaded systems](https://lists.xenproject.org/archives/html/xen-devel/2014-07/msg01848.html) [1] +* [long standing BOOST vs. migration bug](https://lists.xen.org/archives/html/xen-devel/2015-10/msg02851.html) [2] +* [priority handling issues](https://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01362.html) [3] +* "Scheduler development update", XenSummit Asia 2009 [whitepaper](http://www-archive.xenproject.org/files/xensummit_intel09/George_Dunlap.pdf) [4] +* "Scheduling in Xen" [XPDS15 Presentation](http://events.linuxfoundation.org/sites/events/files/slides/Faggioli_XenSummit.pdf) +* [Xen Project Scheduler](https://wiki.xenproject.org/wiki/Xen_Project_Schedulers) + +# History + +------------------------------------------------------------------------ +Date Revision Version Notes +---------- -------- -------- ------------------------------------------- +2016-10-10 1 Xen 4.8 Document written +---------- -------- -------- -------------------------------------------
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> --- Cc: George Dunlap <George.Dunlap@eu.citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Lars Kurth <lars.kurth@citrix.com> Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Ian Jackson <ian.jackson@eu.citrix.com> Cc: Jan Beulich <jbeulich@suse.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Stefano Stabellini <sstabellini@kernel.org> --- docs/features/credit.pandoc | 99 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/features/credit.pandoc