| Message ID | 1649772991-10285-1-git-send-email-quic_jhugo@quicinc.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | PCI: hv: Fix multi-MSI to allow more than one MSI vector | expand |
> From: Jeffrey Hugo <quic_jhugo@quicinc.com> > Sent: Tuesday, April 12, 2022 7:17 AM > ... > If the allocation of multiple MSI vectors for multi-MSI fails in the core > PCI framework, the framework will retry the allocation as a single MSI > vector, assuming that meets the min_vecs specified by the requesting > driver. > > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR > domain to implement that for x86. The VECTOR domain does not support > multi-MSI, so the alloc will always fail and fallback to a single MSI > allocation. > > In short, Hyper-V advertises a capability it does not implement. > > Hyper-V can support multi-MSI because it coordinates with the hypervisor > to map the MSIs in the IOMMU's interrupt remapper, which is something the > VECTOR domain does not have. Therefore the fix is simple - copy what the > x86 IOMMU drivers (AMD/Intel-IR) do by removing > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's > pci_msi_prepare(). > > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft > Hyper-V VMs") > Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> Thanks for the fix! This looks good to me. Reviewed-by: Dexuan Cui <decui@microsoft.com>
On 4/12/2022 8:16 AM, Jeffrey Hugo wrote: > If the allocation of multiple MSI vectors for multi-MSI fails in the core > PCI framework, the framework will retry the allocation as a single MSI > vector, assuming that meets the min_vecs specified by the requesting > driver. > > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR > domain to implement that for x86. The VECTOR domain does not support > multi-MSI, so the alloc will always fail and fallback to a single MSI > allocation. > > In short, Hyper-V advertises a capability it does not implement. > > Hyper-V can support multi-MSI because it coordinates with the hypervisor > to map the MSIs in the IOMMU's interrupt remapper, which is something the > VECTOR domain does not have. Therefore the fix is simple - copy what the > x86 IOMMU drivers (AMD/Intel-IR) do by removing > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's > pci_msi_prepare(). > > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") > Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> > --- > drivers/pci/controller/pci-hyperv.c | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c > index d270a204..41be63e 100644 > --- a/drivers/pci/controller/pci-hyperv.c > +++ b/drivers/pci/controller/pci-hyperv.c > @@ -614,7 +614,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, > static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, > int nvec, msi_alloc_info_t *info) > { > - return pci_msi_prepare(domain, dev, nvec, info); > + int ret = pci_msi_prepare(domain, dev, nvec, info); > + > + /* > + * By using the interrupt remapper in the hypervisor IOMMU, contiguous > + * CPU vectors in not needed for multi-MSI I just noticed that "in" should be "is". > + */ > + if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI) > + info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS; > + > + return ret; > } > > /**
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index d270a204..41be63e 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -614,7 +614,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *info) { - return pci_msi_prepare(domain, dev, nvec, info); + int ret = pci_msi_prepare(domain, dev, nvec, info); + + /* + * By using the interrupt remapper in the hypervisor IOMMU, contiguous + * CPU vectors in not needed for multi-MSI + */ + if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI) + info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS; + + return ret; } /**
If the allocation of multiple MSI vectors for multi-MSI fails in the core PCI framework, the framework will retry the allocation as a single MSI vector, assuming that meets the min_vecs specified by the requesting driver. Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR domain to implement that for x86. The VECTOR domain does not support multi-MSI, so the alloc will always fail and fallback to a single MSI allocation. In short, Hyper-V advertises a capability it does not implement. Hyper-V can support multi-MSI because it coordinates with the hypervisor to map the MSIs in the IOMMU's interrupt remapper, which is something the VECTOR domain does not have. Therefore the fix is simple - copy what the x86 IOMMU drivers (AMD/Intel-IR) do by removing X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's pci_msi_prepare(). Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> --- drivers/pci/controller/pci-hyperv.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)