| Message ID | 20191224151932.6304-7-andrew.cooper3@citrix.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Support CPUID/MSR data in migration streams | expand |
On 24.12.2019 16:19, Andrew Cooper wrote: > Migration data can be split into two parts - that which is invariant of > guest execution, and that which is not. Separate these two with the > STATIC_DATA_END record. > > The short term, we want to move the x86 CPU Policy data into the stream. > In the longer term, we want to provisionally send the static data only > to the destination as a more robust compatibility check. In both cases, > we will want a callback into the higher level toolstack. > > Mandate the presence of the STATIC_DATA_END record, and declare this v3, > along with instructions for how to compatibly interpret a v2 stream. What doesn't become clear (to me) from all of the above is why this record is needed (wanted), and hence why it is to be mandatory. After all ... > @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like: > HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect > the validity of architectural state in the context. > > +Compatibility with older versions > +================================= > + > +v3 compat with v2 > +----------------- > + > +A v3 stream is compatible with a v2 stream, but mandates the presense of a > +STATIC_DATA_END record ahead of any memory/register content. This is to ease > +the introduction of new static configuration records over time. > + > +A v3-compatible reciever interpreting a v2 stream should infer the position of > +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV > +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END > +had been sent. ... you already imply a v3 receiver can deal with its absence. Jan
On 03/01/2020 14:44, Jan Beulich wrote: > On 24.12.2019 16:19, Andrew Cooper wrote: >> Migration data can be split into two parts - that which is invariant of >> guest execution, and that which is not. Separate these two with the >> STATIC_DATA_END record. >> >> The short term, we want to move the x86 CPU Policy data into the stream. >> In the longer term, we want to provisionally send the static data only >> to the destination as a more robust compatibility check. In both cases, >> we will want a callback into the higher level toolstack. >> >> Mandate the presence of the STATIC_DATA_END record, and declare this v3, >> along with instructions for how to compatibly interpret a v2 stream. > What doesn't become clear (to me) from all of the above is why this > record is needed (wanted), and hence why it is to be mandatory. > After all ... > >> @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like: >> HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect >> the validity of architectural state in the context. >> >> +Compatibility with older versions >> +================================= >> + >> +v3 compat with v2 >> +----------------- >> + >> +A v3 stream is compatible with a v2 stream, but mandates the presense of a >> +STATIC_DATA_END record ahead of any memory/register content. This is to ease >> +the introduction of new static configuration records over time. >> + >> +A v3-compatible reciever interpreting a v2 stream should infer the position of >> +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV >> +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END >> +had been sent. > ... you already imply a v3 receiver can deal with its absence. It provides an easier way to reason about the stream, as new features get added, and this is helpful when debugging. Most importantly however, it offers a clean way to deprecate/remove support for obsolete senders. ~Andrew
Andrew Cooper writes ("[PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END"):
> Migration data can be split into two parts - that which is invariant of
> guest execution, and that which is not. Separate these two with the
> STATIC_DATA_END record.
>
> The short term, we want to move the x86 CPU Policy data into the stream.
> In the longer term, we want to provisionally send the static data only
> to the destination as a more robust compatibility check. In both cases,
> we will want a callback into the higher level toolstack.
>
> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
> along with instructions for how to compatibly interpret a v2 stream.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
diff --git a/docs/specs/libxc-migration-stream.pandoc b/docs/specs/libxc-migration-stream.pandoc index a7a8a08936..22ff306e0b 100644 --- a/docs/specs/libxc-migration-stream.pandoc +++ b/docs/specs/libxc-migration-stream.pandoc @@ -127,7 +127,7 @@ marker 0xFFFFFFFFFFFFFFFF. id 0x58454E46 ("XENF" in ASCII). -version 0x00000002. The version of this specification. +version 0x00000003. The version of this specification. options bit 0: Endianness. 0 = little-endian, 1 = big-endian. @@ -620,6 +620,21 @@ The count of pfns is: record->length/sizeof(uint64_t). \clearpage +STATIC_DATA_END +--------------- + +A static data end record marks the end of the static state. I.e. state which +is invariant of guest execution. + + + 0 1 2 3 4 5 6 7 octet + +-------------------------------------------------+ + +The end record contains no fields; its body_length is 0. + +\clearpage + + Layout ====== @@ -639,7 +654,9 @@ A typical save record for an x86 PV guest image would look like: * Image header * Domain header -* X86_PV_INFO record +* Static data records: + * X86_PV_INFO record + * STATIC_DATA_END * X86_PV_P2M_FRAMES record * Many PAGE_DATA records * X86_TSC_INFO @@ -667,6 +684,8 @@ A typical save record for an x86 HVM guest image would look like: * Image header * Domain header +* Static data records: + * STATIC_DATA_END * Many PAGE_DATA records * X86_TSC_INFO * HVM_PARAMS @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like: HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect the validity of architectural state in the context. +Compatibility with older versions +================================= + +v3 compat with v2 +----------------- + +A v3 stream is compatible with a v2 stream, but mandates the presense of a +STATIC_DATA_END record ahead of any memory/register content. This is to ease +the introduction of new static configuration records over time. + +A v3-compatible reciever interpreting a v2 stream should infer the position of +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END +had been sent. Legacy Images (x86 only) -======================== +------------------------ Restoring legacy images from older tools shall be handled by translating the legacy format image into this new format. diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c index dd9a11b4b5..7f22cf0365 100644 --- a/tools/libxc/xc_sr_common.c +++ b/tools/libxc/xc_sr_common.c @@ -36,6 +36,7 @@ static const char *const mandatory_rec_types[] = [REC_TYPE_VERIFY] = "Verify", [REC_TYPE_CHECKPOINT] = "Checkpoint", [REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST] = "Checkpoint dirty pfn list", + [REC_TYPE_STATIC_DATA_END] = "Static data end", }; const char *rec_type_to_str(uint32_t type) diff --git a/tools/libxc/xc_sr_stream_format.h b/tools/libxc/xc_sr_stream_format.h index ae7c0de393..81c9765b0a 100644 --- a/tools/libxc/xc_sr_stream_format.h +++ b/tools/libxc/xc_sr_stream_format.h @@ -73,6 +73,7 @@ struct xc_sr_rhdr #define REC_TYPE_VERIFY 0x0000000dU #define REC_TYPE_CHECKPOINT 0x0000000eU #define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST 0x0000000fU +#define REC_TYPE_STATIC_DATA_END 0x00000010U #define REC_TYPE_OPTIONAL 0x80000000U diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py index 63b3558029..d0c4f3527d 100644 --- a/tools/python/xen/migration/libxc.py +++ b/tools/python/xen/migration/libxc.py @@ -56,6 +56,7 @@ REC_TYPE_verify = 0x0000000d REC_TYPE_checkpoint = 0x0000000e REC_TYPE_checkpoint_dirty_pfn_list = 0x0000000f +REC_TYPE_static_data_end = 0x00000010 rec_type_to_str = { REC_TYPE_end : "End", @@ -74,6 +75,7 @@ REC_TYPE_verify : "Verify", REC_TYPE_checkpoint : "Checkpoint", REC_TYPE_checkpoint_dirty_pfn_list : "Checkpoint dirty pfn list", + REC_TYPE_static_data_end : "Static data end", } # page_data
Migration data can be split into two parts - that which is invariant of guest execution, and that which is not. Separate these two with the STATIC_DATA_END record. The short term, we want to move the x86 CPU Policy data into the stream. In the longer term, we want to provisionally send the static data only to the destination as a more robust compatibility check. In both cases, we will want a callback into the higher level toolstack. Mandate the presence of the STATIC_DATA_END record, and declare this v3, along with instructions for how to compatibly interpret a v2 stream. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> --- CC: George Dunlap <George.Dunlap@eu.citrix.com> CC: Ian Jackson <ian.jackson@citrix.com> CC: Jan Beulich <JBeulich@suse.com> CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Wei Liu <wl@xen.org> CC: Julien Grall <julien@xen.org> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> --- docs/specs/libxc-migration-stream.pandoc | 39 +++++++++++++++++++++++++++++--- tools/libxc/xc_sr_common.c | 1 + tools/libxc/xc_sr_stream_format.h | 1 + tools/python/xen/migration/libxc.py | 2 ++ 4 files changed, 40 insertions(+), 3 deletions(-)