| Message ID | 20231102195938.1254489-5-stewart.hildebrand@amd.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Kconfig for PCI passthrough on ARM | expand |
On 02.11.2023 20:59, Stewart Hildebrand wrote: > --- a/xen/arch/x86/include/asm/domain.h > +++ b/xen/arch/x86/include/asm/domain.h > @@ -503,6 +503,15 @@ struct arch_domain > #define has_vpit(d) (!!((d)->arch.emulation_flags & X86_EMU_PIT)) > #define has_pirq(d) (!!((d)->arch.emulation_flags & X86_EMU_USE_PIRQ)) > > +#define is_pvh_domain(d) ({ \ > + unsigned int _emflags = (d)->arch.emulation_flags; \ > + IS_ENABLED(CONFIG_HVM) && \ > + ((_emflags == X86_EMU_LAPIC) || \ > + (_emflags == (X86_EMU_LAPIC | X86_EMU_IOAPIC))); }) I'm not convinced we want to re-introduce such a predicate; it'll be at risk of going stale when the boundary between HVM and PVH becomes more "fuzzy". > +/* PCI passthrough may be backed by qemu for non-PVH domains */ > +#define arch_needs_vpci(d) is_pvh_domain(d) Wouldn't we want to check for exactly what the comment alludes to then, i.e. whether the domain has any (specific?) device model attached? > --- a/xen/include/xen/domain.h > +++ b/xen/include/xen/domain.h > @@ -51,8 +51,17 @@ void arch_get_domain_info(const struct domain *d, > > #define is_domain_using_staticmem(d) ((d)->cdf & CDF_staticmem) > > -#define has_vpci(d) (((d)->options & XEN_DOMCTL_CDF_vpci) && \ > - IS_ENABLED(CONFIG_HAS_VPCI)) > +#define has_vpci(d) ({ \ > + const struct domain *_d = (d); \ > + bool _has_vpci = false; \ > + if ( (_d->options & XEN_DOMCTL_CDF_vpci) && IS_ENABLED(CONFIG_HAS_VPCI) ) \ > + { \ > + if ( is_hardware_domain(_d) ) \ > + _has_vpci = true; \ > + else if ( IS_ENABLED(CONFIG_HAS_VPCI_GUEST_SUPPORT) ) \ > + _has_vpci = true; \ > + } \ > + _has_vpci; }) This is a commonly executed check, and as such wants to remain as simple as possible. Wouldn't it be better anyway to prevent XEN_DOMCTL_CDF_vpci getting set for a domain which cannot possibly have vPCI? Jan
On 11/6/23 04:26, Jan Beulich wrote: > On 02.11.2023 20:59, Stewart Hildebrand wrote: >> --- a/xen/arch/x86/include/asm/domain.h >> +++ b/xen/arch/x86/include/asm/domain.h >> @@ -503,6 +503,15 @@ struct arch_domain >> #define has_vpit(d) (!!((d)->arch.emulation_flags & X86_EMU_PIT)) >> #define has_pirq(d) (!!((d)->arch.emulation_flags & X86_EMU_USE_PIRQ)) >> >> +#define is_pvh_domain(d) ({ \ >> + unsigned int _emflags = (d)->arch.emulation_flags; \ >> + IS_ENABLED(CONFIG_HVM) && \ >> + ((_emflags == X86_EMU_LAPIC) || \ >> + (_emflags == (X86_EMU_LAPIC | X86_EMU_IOAPIC))); }) > > I'm not convinced we want to re-introduce such a predicate; it'll be at > risk of going stale when the boundary between HVM and PVH becomes more > "fuzzy". OK, I'll drop it > >> +/* PCI passthrough may be backed by qemu for non-PVH domains */ >> +#define arch_needs_vpci(d) is_pvh_domain(d) > > Wouldn't we want to check for exactly what the comment alludes to then, > i.e. whether the domain has any (specific?) device model attached? This patch is primarily dealing with Arm, so I'm considering simply making it return false for now: #define arch_needs_vpci(d) ({ (void)(d); false; }) If it needs to be changed in the future when we enable vPCI for PVH domUs, we can deal with it at that time. > >> --- a/xen/include/xen/domain.h >> +++ b/xen/include/xen/domain.h >> @@ -51,8 +51,17 @@ void arch_get_domain_info(const struct domain *d, >> >> #define is_domain_using_staticmem(d) ((d)->cdf & CDF_staticmem) >> >> -#define has_vpci(d) (((d)->options & XEN_DOMCTL_CDF_vpci) && \ >> - IS_ENABLED(CONFIG_HAS_VPCI)) >> +#define has_vpci(d) ({ \ >> + const struct domain *_d = (d); \ >> + bool _has_vpci = false; \ >> + if ( (_d->options & XEN_DOMCTL_CDF_vpci) && IS_ENABLED(CONFIG_HAS_VPCI) ) \ >> + { \ >> + if ( is_hardware_domain(_d) ) \ >> + _has_vpci = true; \ >> + else if ( IS_ENABLED(CONFIG_HAS_VPCI_GUEST_SUPPORT) ) \ >> + _has_vpci = true; \ >> + } \ >> + _has_vpci; }) > > This is a commonly executed check, and as such wants to remain as simple as > possible. Wouldn't it be better anyway to prevent XEN_DOMCTL_CDF_vpci getting > set for a domain which cannot possibly have vPCI? Yes, agreed, I'll leave has_vpci alone > > Jan
On 13.11.2023 22:10, Stewart Hildebrand wrote: > On 11/6/23 04:26, Jan Beulich wrote: >> On 02.11.2023 20:59, Stewart Hildebrand wrote: >>> +/* PCI passthrough may be backed by qemu for non-PVH domains */ >>> +#define arch_needs_vpci(d) is_pvh_domain(d) >> >> Wouldn't we want to check for exactly what the comment alludes to then, >> i.e. whether the domain has any (specific?) device model attached? > > This patch is primarily dealing with Arm, so I'm considering simply making it return false for now: > > #define arch_needs_vpci(d) ({ (void)(d); false; }) But that's wrong for hwdom, as much as - strictly speaking - needs_vpci() returning false for hwdom is wrong in the PVH case. This would then at least need clarifying comments. Jan
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 5ff68e5d5979..3845b238a33f 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -195,6 +195,7 @@ config PCI_PASSTHROUGH depends on ARM_64 select HAS_PCI select HAS_VPCI + select HAS_VPCI_GUEST_SUPPORT default n help This option enables PCI device passthrough diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h index be9ed39c9d42..e4b4a12233f0 100644 --- a/xen/arch/arm/include/asm/domain.h +++ b/xen/arch/arm/include/asm/domain.h @@ -31,6 +31,8 @@ enum domain_type { #define is_domain_direct_mapped(d) ((d)->cdf & CDF_directmap) +#define arch_needs_vpci(d) ({ (void)(d); true; }) + /* * Is the domain using the host memory layout? * diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index 3bc4bb55082a..61e0edcedea9 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -2,6 +2,7 @@ /* * xen/arch/arm/vpci.c */ +#include <xen/lib.h> #include <xen/sched.h> #include <xen/vpci.h> @@ -90,8 +91,15 @@ int domain_vpci_init(struct domain *d) return ret; } else + { + if ( !IS_ENABLED(CONFIG_HAS_VPCI_GUEST_SUPPORT) ) + { + gdprintk(XENLOG_ERR, "vPCI requested but guest support not enabled\n"); + return -EINVAL; + } register_mmio_handler(d, &vpci_mmio_handler, GUEST_VPCI_ECAM_BASE, GUEST_VPCI_ECAM_SIZE, NULL); + } return 0; } diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h index cb02a4d1ebb2..57bbe842e18d 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -503,6 +503,15 @@ struct arch_domain #define has_vpit(d) (!!((d)->arch.emulation_flags & X86_EMU_PIT)) #define has_pirq(d) (!!((d)->arch.emulation_flags & X86_EMU_USE_PIRQ)) +#define is_pvh_domain(d) ({ \ + unsigned int _emflags = (d)->arch.emulation_flags; \ + IS_ENABLED(CONFIG_HVM) && \ + ((_emflags == X86_EMU_LAPIC) || \ + (_emflags == (X86_EMU_LAPIC | X86_EMU_IOAPIC))); }) + +/* PCI passthrough may be backed by qemu for non-PVH domains */ +#define arch_needs_vpci(d) is_pvh_domain(d) + #define gdt_ldt_pt_idx(v) \ ((v)->vcpu_id >> (PAGETABLE_ORDER - GDT_LDT_VCPU_SHIFT)) #define pv_gdt_ptes(v) \ diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 04d00c7c37df..2203725a2aa6 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1542,6 +1542,18 @@ void iommu_dev_iotlb_flush_timeout(struct domain *d, struct pci_dev *pdev) pcidevs_unlock(); } +static bool needs_vpci(const struct domain *d) +{ + if ( is_hardware_domain(d) ) + return false; + + if ( d == dom_io ) + /* xl pci-assignable-add assigns PCI devices to domIO */ + return false; + + return arch_needs_vpci(d); +} + int iommu_do_pci_domctl( struct xen_domctl *domctl, struct domain *d, XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl) @@ -1618,6 +1630,14 @@ int iommu_do_pci_domctl( bus = PCI_BUS(machine_sbdf); devfn = PCI_DEVFN(machine_sbdf); + if ( needs_vpci(d) && !has_vpci(d) ) + { + printk(XENLOG_G_WARNING "Cannot assign %pp to %pd: vPCI support not enabled\n", + &PCI_SBDF(seg, bus, devfn), d); + ret = -EPERM; + break; + } + pcidevs_lock(); ret = device_assigned(seg, bus, devfn); if ( domctl->cmd == XEN_DOMCTL_test_assign_device ) diff --git a/xen/include/xen/domain.h b/xen/include/xen/domain.h index ab8f06c5f6a2..c3364d6e2e44 100644 --- a/xen/include/xen/domain.h +++ b/xen/include/xen/domain.h @@ -51,8 +51,17 @@ void arch_get_domain_info(const struct domain *d, #define is_domain_using_staticmem(d) ((d)->cdf & CDF_staticmem) -#define has_vpci(d) (((d)->options & XEN_DOMCTL_CDF_vpci) && \ - IS_ENABLED(CONFIG_HAS_VPCI)) +#define has_vpci(d) ({ \ + const struct domain *_d = (d); \ + bool _has_vpci = false; \ + if ( (_d->options & XEN_DOMCTL_CDF_vpci) && IS_ENABLED(CONFIG_HAS_VPCI) ) \ + { \ + if ( is_hardware_domain(_d) ) \ + _has_vpci = true; \ + else if ( IS_ENABLED(CONFIG_HAS_VPCI_GUEST_SUPPORT) ) \ + _has_vpci = true; \ + } \ + _has_vpci; }) /* * Arch-specifics.
Select HAS_VPCI_GUEST_SUPPORT in Kconfig for enabling vPCI support for domUs. Add checks to fail guest creation if the configuration is invalid. Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com> --- As the tag implies, this patch is not intended to be merged (yet). Note that CONFIG_HAS_VPCI_GUEST_SUPPORT is not currently used in the upstream code base. It will be used by the vPCI series [1]. This patch is intended to be merged as part of the vPCI series. I'll coordinate with Volodymyr to include this in the vPCI series or resend afterwards. Meanwhile, I'll include it here until the Kconfig and xen_domctl_createdomain prerequisites have been committed. v4->v5: * replace is_system_domain() check with dom_io check * return an error in XEN_DOMCTL_assign_device (thanks Jan!) * remove CONFIG_ARM check * add needs_vpci() and arch_needs_vpci() * add HAS_VPCI_GUEST_SUPPORT check to has_vpci() v3->v4: * refuse to create domain if configuration is invalid * split toolstack change into separate patch v2->v3: * set pci flags in toolstack v1->v2: * new patch [1] https://lists.xenproject.org/archives/html/xen-devel/2023-10/msg00660.html --- xen/arch/arm/Kconfig | 1 + xen/arch/arm/include/asm/domain.h | 2 ++ xen/arch/arm/vpci.c | 8 ++++++++ xen/arch/x86/include/asm/domain.h | 9 +++++++++ xen/drivers/passthrough/pci.c | 20 ++++++++++++++++++++ xen/include/xen/domain.h | 13 +++++++++++-- 6 files changed, 51 insertions(+), 2 deletions(-)