| Message ID | cover.1661330065.git.matias.vara@vates.fr (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add a new acquire resource to query vcpu statistics | expand |
On 24.08.2022 15:27, Matias Ezequiel Vara Larsen wrote: > The purpose of this RFC is to get feedback about a new acquire resource that > exposes vcpu statistics for a given domain. The current mechanism to get those > statistics is by querying the hypervisor. This mechanism relies on a hypercall > and holds the domctl spinlock during its execution. When a pv tool like xcp-rrdd > periodically samples these counters, it ends up affecting other paths that share > that spinlock. By using acquire resources, the pv tool only requires a few > hypercalls to set the shared memory region and samples are got without issuing > any other hypercall. The original idea has been suggested by Andrew Cooper to > which I have been discussing about how to implement the current PoC. You can > find the RFC patch series at [1]. The series is rebased on top of stable-4.15. > > I am currently a bit blocked on 1) what to expose and 2) how to expose it. For > 1), I decided to expose what xcp-rrdd is querying, e.g., XEN_DOMCTL_getvcpuinfo. > More precisely, xcp-rrd gets runstate.time[RUNSTATE_running]. This is a uint64_t > counter. However, the time spent in other states may be interesting too. > Regarding 2), I am not sure if simply using an array of uint64_t is enough or if > a different interface should be exposed. The remaining question is when to get > new values. For the moment, I am updating this counter during > vcpu_runstate_change(). > > The current series includes a simple pv tool that shows how this new interface is > used. This tool maps the counter and periodically samples it. > > Any feedback/help would be appreciated. Before looking more closely - was there perhaps kind-of-review feedback during the summit, which would make it more reasonable to look through this once a v2 has appeared? Jan
On Thu, Sep 29, 2022 at 05:55:50PM +0200, Jan Beulich wrote: > On 24.08.2022 15:27, Matias Ezequiel Vara Larsen wrote: > > The purpose of this RFC is to get feedback about a new acquire resource that > > exposes vcpu statistics for a given domain. The current mechanism to get those > > statistics is by querying the hypervisor. This mechanism relies on a hypercall > > and holds the domctl spinlock during its execution. When a pv tool like xcp-rrdd > > periodically samples these counters, it ends up affecting other paths that share > > that spinlock. By using acquire resources, the pv tool only requires a few > > hypercalls to set the shared memory region and samples are got without issuing > > any other hypercall. The original idea has been suggested by Andrew Cooper to > > which I have been discussing about how to implement the current PoC. You can > > find the RFC patch series at [1]. The series is rebased on top of stable-4.15. > > > > I am currently a bit blocked on 1) what to expose and 2) how to expose it. For > > 1), I decided to expose what xcp-rrdd is querying, e.g., XEN_DOMCTL_getvcpuinfo. > > More precisely, xcp-rrd gets runstate.time[RUNSTATE_running]. This is a uint64_t > > counter. However, the time spent in other states may be interesting too. > > Regarding 2), I am not sure if simply using an array of uint64_t is enough or if > > a different interface should be exposed. The remaining question is when to get > > new values. For the moment, I am updating this counter during > > vcpu_runstate_change(). > > > > The current series includes a simple pv tool that shows how this new interface is > > used. This tool maps the counter and periodically samples it. > > > > Any feedback/help would be appreciated. > > Before looking more closely - was there perhaps kind-of-review feedback > during the summit, which would make it more reasonable to look through > this once a v2 has appeared? > Yes, there was. I will submit v2 from feedback during summit. Thanks for point it. Matias
Hello all, The purpose of this RFC is to get feedback about a new acquire resource that exposes vcpu statistics for a given domain. The current mechanism to get those statistics is by querying the hypervisor. This mechanism relies on a hypercall and holds the domctl spinlock during its execution. When a pv tool like xcp-rrdd periodically samples these counters, it ends up affecting other paths that share that spinlock. By using acquire resources, the pv tool only requires a few hypercalls to set the shared memory region and samples are got without issuing any other hypercall. The original idea has been suggested by Andrew Cooper to which I have been discussing about how to implement the current PoC. You can find the RFC patch series at [1]. The series is rebased on top of stable-4.15. I am currently a bit blocked on 1) what to expose and 2) how to expose it. For 1), I decided to expose what xcp-rrdd is querying, e.g., XEN_DOMCTL_getvcpuinfo. More precisely, xcp-rrd gets runstate.time[RUNSTATE_running]. This is a uint64_t counter. However, the time spent in other states may be interesting too. Regarding 2), I am not sure if simply using an array of uint64_t is enough or if a different interface should be exposed. The remaining question is when to get new values. For the moment, I am updating this counter during vcpu_runstate_change(). The current series includes a simple pv tool that shows how this new interface is used. This tool maps the counter and periodically samples it. Any feedback/help would be appreciated. Thanks, Matias. [1] https://github.com/MatiasVara/xen/tree/feature_stats Changes in v1: - rework how the resource is allocated and released - rework when the resource is allocated that happens only when the resource is requested - rework the structure shared between the tool and Xen to make it extensible to new counters and declare it in a public header There are still the following questions: - consumer may fetch inconsistency data - resource shall be released when there are no more readers otherwise we keep updating it during a hot path - one frame can host up to 512 vcpus. Should I check to this limit when updating? Should it be possible to allocate more than one frame for vcpu counters? Matias Ezequiel Vara Larsen (2): xen/memory : Add a stats_table resource type tools/misc: Add xen-vcpus-stats tool tools/misc/Makefile | 6 +++ tools/misc/xen-vcpus-stats.c | 76 +++++++++++++++++++++++++++++ xen/arch/x86/hvm/hvm.c | 2 + xen/common/memory.c | 94 ++++++++++++++++++++++++++++++++++++ xen/common/sched/core.c | 7 +++ xen/include/public/memory.h | 3 ++ xen/include/public/vcpu.h | 10 ++++ xen/include/xen/mm.h | 2 + xen/include/xen/sched.h | 5 ++ 9 files changed, 205 insertions(+) create mode 100644 tools/misc/xen-vcpus-stats.c