| Message ID | 20231204094305.59267-3-roger.pau@citrix.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | x86/iommu: improve setup time of hwdom IOMMU | expand |
On 04.12.2023 10:43, Roger Pau Monne wrote: > --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c > +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c > @@ -359,21 +359,17 @@ int __read_mostly amd_iommu_min_paging_mode = 1; > static int cf_check amd_iommu_domain_init(struct domain *d) > { > struct domain_iommu *hd = dom_iommu(d); > + int pglvl = amd_iommu_get_paging_mode( > + PFN_DOWN(1UL << paging_max_paddr_bits(d))); This is a function in the paging subsystem, i.e. generally inapplicable to system domains (specifically DomIO). If this is to remain this way, the function would imo need to gain a warning. Yet better would imo be if the function was avoided for system domains. Jan
On Tue, Dec 05, 2023 at 03:32:20PM +0100, Jan Beulich wrote: > On 04.12.2023 10:43, Roger Pau Monne wrote: > > --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c > > +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c > > @@ -359,21 +359,17 @@ int __read_mostly amd_iommu_min_paging_mode = 1; > > static int cf_check amd_iommu_domain_init(struct domain *d) > > { > > struct domain_iommu *hd = dom_iommu(d); > > + int pglvl = amd_iommu_get_paging_mode( > > + PFN_DOWN(1UL << paging_max_paddr_bits(d))); > > This is a function in the paging subsystem, i.e. generally inapplicable > to system domains (specifically DomIO). If this is to remain this way, > the function would imo need to gain a warning. Yet better would imo be > if the function was avoided for system domains. I have to admit I'm confused, won't systems domains return paging_mode_hap(d) == false, and thus fallback to using paddr_bits (host paddr width?). I can avoid such domains calling into paging_max_paddr_bits() but it seems redundant, and would just be duplicated logic for a case that paging_max_paddr_bits() already handles correctly AFAICT. Would it be better for me to rename paging_max_paddr_bits() to domain_max_paddr_bits() and move it to asm/domain.h? Thanks, Roger.
On 05.12.2023 16:11, Roger Pau Monné wrote: > On Tue, Dec 05, 2023 at 03:32:20PM +0100, Jan Beulich wrote: >> On 04.12.2023 10:43, Roger Pau Monne wrote: >>> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c >>> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c >>> @@ -359,21 +359,17 @@ int __read_mostly amd_iommu_min_paging_mode = 1; >>> static int cf_check amd_iommu_domain_init(struct domain *d) >>> { >>> struct domain_iommu *hd = dom_iommu(d); >>> + int pglvl = amd_iommu_get_paging_mode( >>> + PFN_DOWN(1UL << paging_max_paddr_bits(d))); >> >> This is a function in the paging subsystem, i.e. generally inapplicable >> to system domains (specifically DomIO). If this is to remain this way, >> the function would imo need to gain a warning. Yet better would imo be >> if the function was avoided for system domains. > > I have to admit I'm confused, won't systems domains return > paging_mode_hap(d) == false, and thus fallback to using paddr_bits > (host paddr width?). True, but that check lives inside the function. > I can avoid such domains calling into paging_max_paddr_bits() but it > seems redundant, and would just be duplicated logic for a case that > paging_max_paddr_bits() already handles correctly AFAICT. Hence why I suggested a comment (warning) as alternative. > Would it be better for me to rename paging_max_paddr_bits() to > domain_max_paddr_bits() and move it to asm/domain.h? Maybe. I'm not sure exactly why the function was introduced where it is and under the name it has. It sole present caller is in cpu-policy.c, so either name/placement would look good to me. Jan
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c index 6bc73dc21052..00a25e649f22 100644 --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -359,21 +359,17 @@ int __read_mostly amd_iommu_min_paging_mode = 1; static int cf_check amd_iommu_domain_init(struct domain *d) { struct domain_iommu *hd = dom_iommu(d); + int pglvl = amd_iommu_get_paging_mode( + PFN_DOWN(1UL << paging_max_paddr_bits(d))); + + if ( pglvl < 0 ) + return pglvl; /* - * Choose the number of levels for the IOMMU page tables. - * - PV needs 3 or 4, depending on whether there is RAM (including hotplug - * RAM) above the 512G boundary. - * - HVM could in principle use 3 or 4 depending on how much guest - * physical address space we give it, but this isn't known yet so use 4 - * unilaterally. - * - Unity maps may require an even higher number. + * Choose the number of levels for the IOMMU page tables, taking into + * account unity maps. */ - hd->arch.amd.paging_mode = max(amd_iommu_get_paging_mode( - is_hvm_domain(d) - ? 1UL << (DEFAULT_DOMAIN_ADDRESS_WIDTH - PAGE_SHIFT) - : get_upper_mfn_bound() + 1), - amd_iommu_min_paging_mode); + hd->arch.amd.paging_mode = max(pglvl, amd_iommu_min_paging_mode); return 0; }
However take into account the minimum number of levels required by unity maps when setting the page table levels. The previous setting of the page table levels for PV guests based on the highest RAM address was bogus, as there can be other non-RAM regions past the highest RAM address that need to be mapped, for example device MMIO. For HVM we also take amd_iommu_min_paging_mode into account, however if unity maps require more than 4 levels attempting to add those will currently fail at the p2m level, as 4 levels is the maximum supported. Fixes: 0700c962ac2d ('Add AMD IOMMU support into hypervisor') Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> --- changes since v1: - Use paging_max_paddr_bits() instead of hardcoding DEFAULT_DOMAIN_ADDRESS_WIDTH. --- xen/drivers/passthrough/amd/pci_amd_iommu.c | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-)