Message ID | 1690487690-2428-1-git-send-email-nunodasneves@linux.microsoft.com (mailing list archive) |
---|---|
Headers | show |
Series | Introduce /dev/mshv drivers | expand |
Apologies, it seems patch 15 bouncing from some mailing lists. As a workaround, here is a link to the patch on github while we figure out a solution: https://github.com/NunoDasNeves/linux/commit/54329d18d68b0cbff4f1e0aeba00bc6dd9a21288 On 7/27/2023 12:54 PM, Nuno Das Neves wrote: > This series introduces support for creating and running guest machines > while running on the Microsoft Hypervisor. [0] > This is done via an IOCTL interface accessed through /dev/mshv, similar to > /dev/kvm. Another series introducing this support was previously posted. > [1] > > These interfaces support VMMs running in: > 1. The root patition - provided in the mshv_root module, and > 2. VTL 2 - provided in the mshv_vtl module [2] > > Patches breakdown > ----------------- > The first 7 patches are refactoring and adding some helper functions. > They provide some benefit on their own and could be applied independently > as cleanup patches. > > The following 5 patches just set things up for the driver code to come. > These are very small. They are separated so that the remaining patches are > more self-contained. > > The final 3 patches are the meat of the series: > - Patch 13 contains new header files used by the driver. > These are designed to mirror the ABI headers exported by Hyper-V. This is > done to avoid polluting hyperv-tlfs.h and help track changes to the ABIs > that are still unstable. (See FAQ below). > - Patch 14 conditionally includes these new header files into mshyperv.h > and linux/hyperv.h, in order to be able to use these files in the new > drivers while remaining independent from hyperv-tlfs.h. > - Patch 15 contains the new driver code located in drivers/hv. This is a > large amount of code and new files, but it is mostly self-contained and > all within drivers/hv - apart from the IOCTL interface itself in uapi. > > FAQ on include/uapi/hyperv/*.h > ------------------------------ > Q: > Why not just add these definitions to hyperv-tlfs.h? > A: > The intention of hyperv-tlfs.h is to contain stable definitions documented > in the public TLFS document. These new definitions don't fit that criteria, > so they should be separate. > > Q: > Why are these files named hvgdk.h, hvgdk_mini.h, hvhdk.h and hvhdk_mini.h? > A: > The precise meaning of the names reflects conventions used internally at > Microsoft. > Naming them this way makes it easy to find where particular Hyper-V > definitions come from, and check their correctness. > It also facilitates the future work of automatically generating these files. > > Q: > Why are they in uapi? > A: > In short, to keep things simple. There are many definitions needed in both > the kernel and the VMM in userspace. Separating them doesn't serve much > purpose, and makes it more laborious to import definitions from Hyper-V > code. > > Q: > The new headers redefine many things that are already in hyperv-tlfs.h - why? > A: > Some definitions are extended compared to what is documented in the TLFS. > In order to avoid adding undocumented or unstable definitions to hyperv-tlfs.h, > the new headers must compile independently. > Therefore, the new headers must redefine many things in hyperv-tlfs.h in order > to compile. > > -------------------------- > [0] "Hyper-V" is more well-known, but it really refers to the whole stack > including the hypervisor and other components that run in Windows > kernel and userspace. > [1] Previous /dev/mshv patch series and discussion: > https://lore.kernel.org/linux-hyperv/1632853875-20261-1-git-send-email-nunodasneves@linux.microsoft.com/ > [2] Virtual Secure Mode (VSM) and Virtual Trust Levels (VTL): > https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/vsm > > Nuno Das Neves (15): > hyperv-tlfs: Change shared HV_REGISTER_* defines to HV_MSR_* > mshyperv: Introduce hv_get_hypervisor_version > mshyperv: Introduce numa_node_to_proximity_domain_info > asm-generic/mshyperv: Introduce hv_recommend_using_aeoi() > hyperv: Move hv_connection_id to hyperv-tlfs > hyperv-tlfs: Introduce hv_status_to_string and hv_status_to_errno > Drivers: hv: Move hv_call_deposit_pages and hv_call_create_vp to > common code > Drivers: hv: Introduce per-cpu event ring tail > Drivers: hv: Introduce hv_output_arg_exists in hv_common > x86: hyperv: Add mshv_handler irq handler and setup function > Drivers: hv: export vmbus_isr, hv_context and hv_post_message > Documentation: Reserve ioctl number for mshv driver > uapi: hyperv: Add mshv driver headers hvhdk.h, hvhdk_mini.h, hvgdk.h, > hvgdk_mini.h > asm-generic: hyperv: Use mshv headers conditionally. Add > asm-generic/hyperv-defs.h > Drivers: hv: Add modules to expose /dev/mshv to VMMs running on > Hyper-V > > .../userspace-api/ioctl/ioctl-number.rst | 2 + > arch/arm64/hyperv/mshyperv.c | 23 +- > arch/arm64/include/asm/hyperv-tlfs.h | 25 + > arch/arm64/include/asm/mshyperv.h | 2 +- > arch/x86/hyperv/hv_init.c | 2 +- > arch/x86/hyperv/hv_proc.c | 166 +- > arch/x86/include/asm/hyperv-tlfs.h | 137 +- > arch/x86/include/asm/mshyperv.h | 13 +- > arch/x86/kernel/cpu/mshyperv.c | 71 +- > drivers/acpi/numa/srat.c | 1 + > drivers/clocksource/hyperv_timer.c | 24 +- > drivers/hv/Kconfig | 54 + > drivers/hv/Makefile | 21 + > drivers/hv/hv.c | 46 +- > drivers/hv/hv_call.c | 119 + > drivers/hv/hv_common.c | 225 +- > drivers/hv/hyperv_vmbus.h | 2 +- > drivers/hv/mshv.h | 156 ++ > drivers/hv/mshv_eventfd.c | 758 +++++++ > drivers/hv/mshv_eventfd.h | 80 + > drivers/hv/mshv_main.c | 208 ++ > drivers/hv/mshv_msi.c | 129 ++ > drivers/hv/mshv_portid_table.c | 84 + > drivers/hv/mshv_root.h | 194 ++ > drivers/hv/mshv_root_hv_call.c | 1064 +++++++++ > drivers/hv/mshv_root_main.c | 1964 +++++++++++++++++ > drivers/hv/mshv_synic.c | 689 ++++++ > drivers/hv/mshv_vtl.h | 52 + > drivers/hv/mshv_vtl_main.c | 1541 +++++++++++++ > drivers/hv/vmbus_drv.c | 3 +- > drivers/hv/xfer_to_guest.c | 28 + > include/asm-generic/hyperv-defs.h | 26 + > include/asm-generic/hyperv-tlfs.h | 77 +- > include/asm-generic/mshyperv.h | 76 +- > include/linux/hyperv.h | 11 +- > include/uapi/hyperv/hvgdk.h | 41 + > include/uapi/hyperv/hvgdk_mini.h | 1077 +++++++++ > include/uapi/hyperv/hvhdk.h | 1352 ++++++++++++ > include/uapi/hyperv/hvhdk_mini.h | 164 ++ > include/uapi/linux/mshv.h | 298 +++ > 40 files changed, 10653 insertions(+), 352 deletions(-) > create mode 100644 drivers/hv/hv_call.c > create mode 100644 drivers/hv/mshv.h > create mode 100644 drivers/hv/mshv_eventfd.c > create mode 100644 drivers/hv/mshv_eventfd.h > create mode 100644 drivers/hv/mshv_main.c > create mode 100644 drivers/hv/mshv_msi.c > create mode 100644 drivers/hv/mshv_portid_table.c > create mode 100644 drivers/hv/mshv_root.h > create mode 100644 drivers/hv/mshv_root_hv_call.c > create mode 100644 drivers/hv/mshv_root_main.c > create mode 100644 drivers/hv/mshv_synic.c > create mode 100644 drivers/hv/mshv_vtl.h > create mode 100644 drivers/hv/mshv_vtl_main.c > create mode 100644 drivers/hv/xfer_to_guest.c > create mode 100644 include/asm-generic/hyperv-defs.h > create mode 100644 include/uapi/hyperv/hvgdk.h > create mode 100644 include/uapi/hyperv/hvgdk_mini.h > create mode 100644 include/uapi/hyperv/hvhdk.h > create mode 100644 include/uapi/hyperv/hvhdk_mini.h > create mode 100644 include/uapi/linux/mshv.h >
On Thu, Jul 27, 2023 at 12:54:50PM -0700, Nuno Das Neves wrote: > Add mshv, mshv_root, and mshv_vtl modules. > - mshv provides /dev/mshv and common code, and is the parent module > - mshv_root provides APIs for creating and managing child partitions > - mshv_vtl provides VTL (Virtual Trust Level) support for VMMs Please provide a slightly more detailed description of what these modules do. This is huge patch after all. People doing code archaeology will appreciate a better commit message. For example (please correct if I'm wrong): Module mshv provides /dev/mshv and common code, and is the parent module to the other two modules. At its core, it implements an eventfd frame work, and defines some helper functions for the other modules. Module mshv_root provides APIs for creating and managing child partitions. It defines abstractions for vcpus, partitions and other things related to running a guest inside the kernel. It also exposes user space interfaces for the VMMs. Module mshv_vtl provides VTL (Virtual Trust Level) support for VMMs. It allows the VMM to run in a higher trust level than the guest but still within the same context as the guest. This is a useful feature for in guest emulation for better isolation and performance. > > Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com> > --- > drivers/hv/Kconfig | 54 + > drivers/hv/Makefile | 21 + > drivers/hv/hv_call.c | 119 ++ > drivers/hv/mshv.h | 156 +++ > drivers/hv/mshv_eventfd.c | 758 ++++++++++++ > drivers/hv/mshv_eventfd.h | 80 ++ > drivers/hv/mshv_main.c | 208 ++++ > drivers/hv/mshv_msi.c | 129 +++ > drivers/hv/mshv_portid_table.c | 84 ++ > drivers/hv/mshv_root.h | 194 ++++ > drivers/hv/mshv_root_hv_call.c | 1064 +++++++++++++++++ > drivers/hv/mshv_root_main.c | 1964 ++++++++++++++++++++++++++++++++ > drivers/hv/mshv_synic.c | 689 +++++++++++ > drivers/hv/mshv_vtl.h | 52 + > drivers/hv/mshv_vtl_main.c | 1541 +++++++++++++++++++++++++ > drivers/hv/xfer_to_guest.c | 28 + > include/uapi/linux/mshv.h | 298 +++++ > 17 files changed, 7439 insertions(+) > create mode 100644 drivers/hv/hv_call.c > create mode 100644 drivers/hv/mshv.h > create mode 100644 drivers/hv/mshv_eventfd.c > create mode 100644 drivers/hv/mshv_eventfd.h > create mode 100644 drivers/hv/mshv_main.c > create mode 100644 drivers/hv/mshv_msi.c > create mode 100644 drivers/hv/mshv_portid_table.c > create mode 100644 drivers/hv/mshv_root.h > create mode 100644 drivers/hv/mshv_root_hv_call.c > create mode 100644 drivers/hv/mshv_root_main.c > create mode 100644 drivers/hv/mshv_synic.c > create mode 100644 drivers/hv/mshv_vtl.h > create mode 100644 drivers/hv/mshv_vtl_main.c > create mode 100644 drivers/hv/xfer_to_guest.c > create mode 100644 include/uapi/linux/mshv.h > > diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig > index 00242107d62e..b150d686e902 100644 > --- a/drivers/hv/Kconfig > +++ b/drivers/hv/Kconfig > @@ -54,4 +54,58 @@ config HYPERV_BALLOON > help > Select this option to enable Hyper-V Balloon driver. > > +config MSHV > + tristate "Microsoft Hypervisor root partition interfaces: /dev/mshv" > + depends on X86_64 && HYPERV > + select EVENTFD > + select MSHV_VFIO This is not needed yet, right? I think this is just dead code right now. It can be introduced when we start upstreaming the VFIO bits. > + select MSHV_XFER_TO_GUEST_WORK > + help > + Select this option to enable core functionality for managing guest > + virtual machines running under the Microsoft Hypervisor. > + > + The interfaces are provided via a device named /dev/mshv. > + > + To compile this as a module, choose M here. > + > + If unsure, say N. > + > +config MSHV_ROOT > + tristate "Microsoft Hyper-V root partition APIs driver" > + depends on MSHV > + help > + Select this option to provide /dev/mshv interfaces specific to > + running as the root partition on Microsoft Hypervisor. > + > + To compile this as a module, choose M here. > + > + If unsure, say N. > + > +config MSHV_VTL > + tristate "Microsoft Hyper-V VTL driver" > + depends on MSHV > + select HYPERV_VTL_MODE > + select TRANSPARENT_HUGEPAGE > + help > + Select this option to enable Hyper-V VTL driver. > + Virtual Secure Mode (VSM) is a set of hypervisor capabilities and > + enlightenments offered to host and guest partitions which enables > + the creation and management of new security boundaries within > + operating system software. > + > + VSM achieves and maintains isolation through Virtual Trust Levels > + (VTLs). Virtual Trust Levels are hierarchical, with higher levels > + being more privileged than lower levels. VTL0 is the least privileged > + level, and currently only other level supported is VTL2. > + > + To compile this as a module, choose M here. > + > + If unsure, say N. The changes to the function which indicates if output pages are needed should be in this patch. > + > +config MSHV_VFIO > + bool > + > +config MSHV_XFER_TO_GUEST_WORK > + bool > + > endmenu > diff --git a/drivers/hv/Makefile b/drivers/hv/Makefile > index d76df5c8c2a9..113c79cfadb9 100644 > --- a/drivers/hv/Makefile > +++ b/drivers/hv/Makefile > @@ -2,10 +2,31 @@ > obj-$(CONFIG_HYPERV) += hv_vmbus.o > obj-$(CONFIG_HYPERV_UTILS) += hv_utils.o > obj-$(CONFIG_HYPERV_BALLOON) += hv_balloon.o > +obj-$(CONFIG_DXGKRNL) += dxgkrnl/ This is not yet upstreamed. It shouldn't be here. Does this not break the build for you? The rest is basically a copy of what was posted many moons before plus some VTL stuff, and new code for the root scheduler and async hypercall support. I've probably gone through some versions of this code already, so I only skim the code. Since this is a Microsoft only driver, I don't expect to get much review from the community -- the last few rounds were quiet. I will however let this patch series float for a while before taking any further actions just in case. If people are interested in specific bits of the code in the driver, please let Nuno and I know. Thanks, Wei.
On 8/2/2023 6:23 PM, Wei Liu wrote: > On Thu, Jul 27, 2023 at 12:54:50PM -0700, Nuno Das Neves wrote: >> Add mshv, mshv_root, and mshv_vtl modules. >> - mshv provides /dev/mshv and common code, and is the parent module >> - mshv_root provides APIs for creating and managing child partitions >> - mshv_vtl provides VTL (Virtual Trust Level) support for VMMs > > Please provide a slightly more detailed description of what these > modules do. This is huge patch after all. People doing code archaeology > will appreciate a better commit message. > > For example (please correct if I'm wrong): > > Module mshv provides /dev/mshv and common code, and is the parent module > to the other two modules. At its core, it implements an eventfd frame > work, and defines some helper functions for the other modules. > > Module mshv_root provides APIs for creating and managing child > partitions. It defines abstractions for vcpus, partitions and other > things related to running a guest inside the kernel. It also exposes > user space interfaces for the VMMs. > > Module mshv_vtl provides VTL (Virtual Trust Level) support for VMMs. It > allows the VMM to run in a higher trust level than the guest but still > within the same context as the guest. This is a useful feature for in > guest emulation for better isolation and performance. > Thanks - I will provide some more detail, including what you described. I will make a couple of changes - the eventfd framework is in mshv_root, not mshv. I will amend the mshv_vtl part a little for clarity. >> >> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com> >> --- >> drivers/hv/Kconfig | 54 + >> drivers/hv/Makefile | 21 + >> drivers/hv/hv_call.c | 119 ++ >> drivers/hv/mshv.h | 156 +++ >> drivers/hv/mshv_eventfd.c | 758 ++++++++++++ >> drivers/hv/mshv_eventfd.h | 80 ++ >> drivers/hv/mshv_main.c | 208 ++++ >> drivers/hv/mshv_msi.c | 129 +++ >> drivers/hv/mshv_portid_table.c | 84 ++ >> drivers/hv/mshv_root.h | 194 ++++ >> drivers/hv/mshv_root_hv_call.c | 1064 +++++++++++++++++ >> drivers/hv/mshv_root_main.c | 1964 ++++++++++++++++++++++++++++++++ >> drivers/hv/mshv_synic.c | 689 +++++++++++ >> drivers/hv/mshv_vtl.h | 52 + >> drivers/hv/mshv_vtl_main.c | 1541 +++++++++++++++++++++++++ >> drivers/hv/xfer_to_guest.c | 28 + >> include/uapi/linux/mshv.h | 298 +++++ >> 17 files changed, 7439 insertions(+) >> create mode 100644 drivers/hv/hv_call.c >> create mode 100644 drivers/hv/mshv.h >> create mode 100644 drivers/hv/mshv_eventfd.c >> create mode 100644 drivers/hv/mshv_eventfd.h >> create mode 100644 drivers/hv/mshv_main.c >> create mode 100644 drivers/hv/mshv_msi.c >> create mode 100644 drivers/hv/mshv_portid_table.c >> create mode 100644 drivers/hv/mshv_root.h >> create mode 100644 drivers/hv/mshv_root_hv_call.c >> create mode 100644 drivers/hv/mshv_root_main.c >> create mode 100644 drivers/hv/mshv_synic.c >> create mode 100644 drivers/hv/mshv_vtl.h >> create mode 100644 drivers/hv/mshv_vtl_main.c >> create mode 100644 drivers/hv/xfer_to_guest.c >> create mode 100644 include/uapi/linux/mshv.h >> >> diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig >> index 00242107d62e..b150d686e902 100644 >> --- a/drivers/hv/Kconfig >> +++ b/drivers/hv/Kconfig >> @@ -54,4 +54,58 @@ config HYPERV_BALLOON >> help >> Select this option to enable Hyper-V Balloon driver. >> >> +config MSHV >> + tristate "Microsoft Hypervisor root partition interfaces: /dev/mshv" >> + depends on X86_64 && HYPERV >> + select EVENTFD >> + select MSHV_VFIO > > This is not needed yet, right? I think this is just dead code right now. > > It can be introduced when we start upstreaming the VFIO bits. > Right. Removed, along with config MSHV_VFIO below. >> + select MSHV_XFER_TO_GUEST_WORK >> + help >> + Select this option to enable core functionality for managing guest >> + virtual machines running under the Microsoft Hypervisor. >> + >> + The interfaces are provided via a device named /dev/mshv. >> + >> + To compile this as a module, choose M here. >> + >> + If unsure, say N. >> + >> +config MSHV_ROOT >> + tristate "Microsoft Hyper-V root partition APIs driver" >> + depends on MSHV >> + help >> + Select this option to provide /dev/mshv interfaces specific to >> + running as the root partition on Microsoft Hypervisor. >> + >> + To compile this as a module, choose M here. >> + >> + If unsure, say N. >> + >> +config MSHV_VTL >> + tristate "Microsoft Hyper-V VTL driver" >> + depends on MSHV >> + select HYPERV_VTL_MODE >> + select TRANSPARENT_HUGEPAGE >> + help >> + Select this option to enable Hyper-V VTL driver. >> + Virtual Secure Mode (VSM) is a set of hypervisor capabilities and >> + enlightenments offered to host and guest partitions which enables >> + the creation and management of new security boundaries within >> + operating system software. >> + >> + VSM achieves and maintains isolation through Virtual Trust Levels >> + (VTLs). Virtual Trust Levels are hierarchical, with higher levels >> + being more privileged than lower levels. VTL0 is the least privileged >> + level, and currently only other level supported is VTL2. >> + >> + To compile this as a module, choose M here. >> + >> + If unsure, say N. > > The changes to the function which indicates if output pages are needed > should be in this patch. > Yes - I will add it in this patch. >> + >> +config MSHV_VFIO >> + bool >> + >> +config MSHV_XFER_TO_GUEST_WORK >> + bool >> + >> endmenu >> diff --git a/drivers/hv/Makefile b/drivers/hv/Makefile >> index d76df5c8c2a9..113c79cfadb9 100644 >> --- a/drivers/hv/Makefile >> +++ b/drivers/hv/Makefile >> @@ -2,10 +2,31 @@ >> obj-$(CONFIG_HYPERV) += hv_vmbus.o >> obj-$(CONFIG_HYPERV_UTILS) += hv_utils.o >> obj-$(CONFIG_HYPERV_BALLOON) += hv_balloon.o >> +obj-$(CONFIG_DXGKRNL) += dxgkrnl/ > > This is not yet upstreamed. It shouldn't be here. Does this not break > the build for you? > Oops! Nope, it doesn't seem to break the build... Anyway, removed. > The rest is basically a copy of what was posted many moons before plus > some VTL stuff, and new code for the root scheduler and async hypercall > support. I've probably gone through some versions of this code already, > so I only skim the code. > > Since this is a Microsoft only driver, I don't expect to get much review > from the community -- the last few rounds were quiet. I will however let > this patch series float for a while before taking any further actions > just in case. > > If people are interested in specific bits of the code in the driver, > please let Nuno and I know. > > Thanks, > Wei.