| Message ID | 1649856981-14649-1-git-send-email-quic_jhugo@quicinc.com (mailing list archive) |
|---|---|
| State | Handled Elsewhere |
| Delegated to: | Lorenzo Pieralisi |
| Headers | show |
| Series | [v2] PCI: hv: Fix multi-MSI to allow more than one MSI vector | expand |
On 4/13/2022 7:36 AM, Jeffrey Hugo wrote: > If the allocation of multiple MSI vectors for multi-MSI fails in the core > PCI framework, the framework will retry the allocation as a single MSI > vector, assuming that meets the min_vecs specified by the requesting > driver. > > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR > domain to implement that for x86. The VECTOR domain does not support > multi-MSI, so the alloc will always fail and fallback to a single MSI > allocation. > > In short, Hyper-V advertises a capability it does not implement. > > Hyper-V can support multi-MSI because it coordinates with the hypervisor > to map the MSIs in the IOMMU's interrupt remapper, which is something the > VECTOR domain does not have. Therefore the fix is simple - copy what the > x86 IOMMU drivers (AMD/Intel-IR) do by removing > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's > pci_msi_prepare(). > > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") > Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> > Reviewed-by: Dexuan Cui <decui@microsoft.com> > --- Ping? I don't see this in -next, nor have I seen any replies. It is possible I have missed some kind of update, but currently I'm wondering if this change is progressing or not. If there is some kind of process used in this area, I'm not familiar with it, so I would appreciate an introduction. Thanks -Jeff
On Wed, Apr 20, 2022 at 08:13:22AM -0600, Jeffrey Hugo wrote: > On 4/13/2022 7:36 AM, Jeffrey Hugo wrote: > > If the allocation of multiple MSI vectors for multi-MSI fails in the core > > PCI framework, the framework will retry the allocation as a single MSI > > vector, assuming that meets the min_vecs specified by the requesting > > driver. > > > > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR > > domain to implement that for x86. The VECTOR domain does not support > > multi-MSI, so the alloc will always fail and fallback to a single MSI > > allocation. > > > > In short, Hyper-V advertises a capability it does not implement. > > > > Hyper-V can support multi-MSI because it coordinates with the hypervisor > > to map the MSIs in the IOMMU's interrupt remapper, which is something the > > VECTOR domain does not have. Therefore the fix is simple - copy what the > > x86 IOMMU drivers (AMD/Intel-IR) do by removing > > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's > > pci_msi_prepare(). > > > > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") > > Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> > > Reviewed-by: Dexuan Cui <decui@microsoft.com> > > --- > > Ping? > > I don't see this in -next, nor have I seen any replies. It is possible I > have missed some kind of update, but currently I'm wondering if this change > is progressing or not. If there is some kind of process used in this area, > I'm not familiar with it, so I would appreciate an introduction. I expect the PCI maintainers to pick this up. If I don't see this picked up in this week I will apply it to hyperv-next. Thanks, Wei. > > Thanks > > -Jeff
On Mon, Apr 25, 2022 at 03:33:44PM +0000, Wei Liu wrote: > On Wed, Apr 20, 2022 at 08:13:22AM -0600, Jeffrey Hugo wrote: > > On 4/13/2022 7:36 AM, Jeffrey Hugo wrote: > > > If the allocation of multiple MSI vectors for multi-MSI fails in the core > > > PCI framework, the framework will retry the allocation as a single MSI > > > vector, assuming that meets the min_vecs specified by the requesting > > > driver. > > > > > > Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR > > > domain to implement that for x86. The VECTOR domain does not support > > > multi-MSI, so the alloc will always fail and fallback to a single MSI > > > allocation. > > > > > > In short, Hyper-V advertises a capability it does not implement. > > > > > > Hyper-V can support multi-MSI because it coordinates with the hypervisor > > > to map the MSIs in the IOMMU's interrupt remapper, which is something the > > > VECTOR domain does not have. Therefore the fix is simple - copy what the > > > x86 IOMMU drivers (AMD/Intel-IR) do by removing > > > X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's > > > pci_msi_prepare(). > > > > > > Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") > > > Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> > > > Reviewed-by: Dexuan Cui <decui@microsoft.com> > > > --- > > > > Ping? > > > > I don't see this in -next, nor have I seen any replies. It is possible I > > have missed some kind of update, but currently I'm wondering if this change > > is progressing or not. If there is some kind of process used in this area, > > I'm not familiar with it, so I would appreciate an introduction. > > I expect the PCI maintainers to pick this up. If I don't see this picked > up in this week I will apply it to hyperv-next. Actually I will pick this up via hyperv-next, because there is another series which will also touch this driver but at the some time depend on vmbus changes. I can fix up any potential conflicts easily. Thanks, Wei.
On 4/25/2022 9:49 AM, Wei Liu wrote: > On Mon, Apr 25, 2022 at 03:33:44PM +0000, Wei Liu wrote: >> On Wed, Apr 20, 2022 at 08:13:22AM -0600, Jeffrey Hugo wrote: >>> On 4/13/2022 7:36 AM, Jeffrey Hugo wrote: >>>> If the allocation of multiple MSI vectors for multi-MSI fails in the core >>>> PCI framework, the framework will retry the allocation as a single MSI >>>> vector, assuming that meets the min_vecs specified by the requesting >>>> driver. >>>> >>>> Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR >>>> domain to implement that for x86. The VECTOR domain does not support >>>> multi-MSI, so the alloc will always fail and fallback to a single MSI >>>> allocation. >>>> >>>> In short, Hyper-V advertises a capability it does not implement. >>>> >>>> Hyper-V can support multi-MSI because it coordinates with the hypervisor >>>> to map the MSIs in the IOMMU's interrupt remapper, which is something the >>>> VECTOR domain does not have. Therefore the fix is simple - copy what the >>>> x86 IOMMU drivers (AMD/Intel-IR) do by removing >>>> X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's >>>> pci_msi_prepare(). >>>> >>>> Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") >>>> Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> >>>> Reviewed-by: Dexuan Cui <decui@microsoft.com> >>>> --- >>> >>> Ping? >>> >>> I don't see this in -next, nor have I seen any replies. It is possible I >>> have missed some kind of update, but currently I'm wondering if this change >>> is progressing or not. If there is some kind of process used in this area, >>> I'm not familiar with it, so I would appreciate an introduction. >> >> I expect the PCI maintainers to pick this up. If I don't see this picked >> up in this week I will apply it to hyperv-next. > > Actually I will pick this up via hyperv-next, because there is another > series which will also touch this driver but at the some time depend on > vmbus changes. I can fix up any potential conflicts easily. Sounds good to me. Let me know if you do run into conflicts, and I can help. -Jeff
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index d270a204..1cbe24b 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -614,7 +614,16 @@ static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *info) { - return pci_msi_prepare(domain, dev, nvec, info); + int ret = pci_msi_prepare(domain, dev, nvec, info); + + /* + * By using the interrupt remapper in the hypervisor IOMMU, contiguous + * CPU vectors is not needed for multi-MSI + */ + if (info->type == X86_IRQ_ALLOC_TYPE_PCI_MSI) + info->flags &= ~X86_IRQ_ALLOC_CONTIGUOUS_VECTORS; + + return ret; } /**
If the allocation of multiple MSI vectors for multi-MSI fails in the core PCI framework, the framework will retry the allocation as a single MSI vector, assuming that meets the min_vecs specified by the requesting driver. Hyper-V advertises that multi-MSI is supported, but reuses the VECTOR domain to implement that for x86. The VECTOR domain does not support multi-MSI, so the alloc will always fail and fallback to a single MSI allocation. In short, Hyper-V advertises a capability it does not implement. Hyper-V can support multi-MSI because it coordinates with the hypervisor to map the MSIs in the IOMMU's interrupt remapper, which is something the VECTOR domain does not have. Therefore the fix is simple - copy what the x86 IOMMU drivers (AMD/Intel-IR) do by removing X86_IRQ_ALLOC_CONTIGUOUS_VECTORS after calling the VECTOR domain's pci_msi_prepare(). Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") Signed-off-by: Jeffrey Hugo <quic_jhugo@quicinc.com> Reviewed-by: Dexuan Cui <decui@microsoft.com> --- v2: -Fix grammatical mistake in added comment drivers/pci/controller/pci-hyperv.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)