| Message ID | 20191121005607.274347-1-david@gibson.dropbear.id.au (mailing list archive) |
|---|---|
| Headers | show |
| Series | vfio/spapr: Handle changes of master irq chip for VFIO devices | expand |
On Thu, 21 Nov 2019 11:56:02 +1100 David Gibson <david@gibson.dropbear.id.au> wrote: > Due to the way feature negotiation works in PAPR (which is a > paravirtualized platform), we can end up changing the global irq chip > at runtime, including it's KVM accelerate model. That causes > complications for VFIO devices with INTx, which wire themselves up > directly to the KVM irqchip for performance. > > This series introduces a new notifier to let VFIO devices (and > anything else that needs to in the future) know about changes to the > master irqchip. It modifies VFIO to respond to the notifier, > reconnecting itself to the new KVM irqchip as necessary. > > In particular this removes a misleading (though not wholly inaccurate) > warning that occurs when using VFIO devices on a pseries machine type > guest. > > Open question: should this go into qemu-4.2 or wait until 5.0? It's > has medium complexity / intrusiveness, but it *is* a bugfix that I > can't see a simpler way to fix. It's effectively a regression from > qemu-4.0 to qemu-4.1 (because that introduced XIVE support by > default), although not from 4.1 to 4.2. Looks reasonable to me for 4.2, the vfio changes are not as big as they appear. If Paolo approves this week, I can send a pull request, otherwise I can leave my ack for someone else as I'll be on PTO/holiday next week. Thanks, Alex > Changes since RFC: > * Fixed some incorrect error paths pointed by aw in 3/5 > * 5/5 had some problems previously, but they have been obsoleted by > other changes merged in the meantime > > David Gibson (5): > kvm: Introduce KVM irqchip change notifier > vfio/pci: Split vfio_intx_update() > vfio/pci: Respond to KVM irqchip change notifier > spapr: Handle irq backend changes with VFIO PCI devices > spapr: Work around spurious warnings from vfio INTx initialization > > accel/kvm/kvm-all.c | 18 ++++++++++++ > accel/stubs/kvm-stub.c | 12 ++++++++ > hw/ppc/spapr_irq.c | 17 +++++++++++- > hw/vfio/pci.c | 62 +++++++++++++++++++++++++++--------------- > hw/vfio/pci.h | 1 + > include/sysemu/kvm.h | 5 ++++ > 6 files changed, 92 insertions(+), 23 deletions(-) >
On Thu, Nov 21, 2019 at 09:57:38AM -0700, Alex Williamson wrote: > On Thu, 21 Nov 2019 11:56:02 +1100 > David Gibson <david@gibson.dropbear.id.au> wrote: > > > Due to the way feature negotiation works in PAPR (which is a > > paravirtualized platform), we can end up changing the global irq chip > > at runtime, including it's KVM accelerate model. That causes > > complications for VFIO devices with INTx, which wire themselves up > > directly to the KVM irqchip for performance. > > > > This series introduces a new notifier to let VFIO devices (and > > anything else that needs to in the future) know about changes to the > > master irqchip. It modifies VFIO to respond to the notifier, > > reconnecting itself to the new KVM irqchip as necessary. > > > > In particular this removes a misleading (though not wholly inaccurate) > > warning that occurs when using VFIO devices on a pseries machine type > > guest. > > > > Open question: should this go into qemu-4.2 or wait until 5.0? It's > > has medium complexity / intrusiveness, but it *is* a bugfix that I > > can't see a simpler way to fix. It's effectively a regression from > > qemu-4.0 to qemu-4.1 (because that introduced XIVE support by > > default), although not from 4.1 to 4.2. > > Looks reasonable to me for 4.2, the vfio changes are not as big as they > appear. If Paolo approves this week, I can send a pull request, > otherwise I can leave my ack for someone else as I'll be on PTO/holiday > next week. Thanks, I'm happy to take it through my tree, and expect to be sending a PR in that timescale, so an ack sounds good. I've pulled the series into my ppc-for-4.2 branch tentatively.
On Fri, 22 Nov 2019 12:18:24 +1100 David Gibson <david@gibson.dropbear.id.au> wrote: > On Thu, Nov 21, 2019 at 09:57:38AM -0700, Alex Williamson wrote: > > On Thu, 21 Nov 2019 11:56:02 +1100 > > David Gibson <david@gibson.dropbear.id.au> wrote: > > > > > Due to the way feature negotiation works in PAPR (which is a > > > paravirtualized platform), we can end up changing the global irq chip > > > at runtime, including it's KVM accelerate model. That causes > > > complications for VFIO devices with INTx, which wire themselves up > > > directly to the KVM irqchip for performance. > > > > > > This series introduces a new notifier to let VFIO devices (and > > > anything else that needs to in the future) know about changes to the > > > master irqchip. It modifies VFIO to respond to the notifier, > > > reconnecting itself to the new KVM irqchip as necessary. > > > > > > In particular this removes a misleading (though not wholly inaccurate) > > > warning that occurs when using VFIO devices on a pseries machine type > > > guest. > > > > > > Open question: should this go into qemu-4.2 or wait until 5.0? It's > > > has medium complexity / intrusiveness, but it *is* a bugfix that I > > > can't see a simpler way to fix. It's effectively a regression from > > > qemu-4.0 to qemu-4.1 (because that introduced XIVE support by > > > default), although not from 4.1 to 4.2. > > > > Looks reasonable to me for 4.2, the vfio changes are not as big as they > > appear. If Paolo approves this week, I can send a pull request, > > otherwise I can leave my ack for someone else as I'll be on PTO/holiday > > next week. Thanks, > > I'm happy to take it through my tree, and expect to be sending a PR in > that timescale, so an ack sounds good. > > I've pulled the series into my ppc-for-4.2 branch tentatively. > Tested-by: Alex Williamson <alex.williamson@redhat.com> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: Alex Williamson <alex.williamson@redhat.com>
On Thu, Nov 21, 2019 at 06:28:07PM -0700, Alex Williamson wrote: > On Fri, 22 Nov 2019 12:18:24 +1100 > David Gibson <david@gibson.dropbear.id.au> wrote: > > > On Thu, Nov 21, 2019 at 09:57:38AM -0700, Alex Williamson wrote: > > > On Thu, 21 Nov 2019 11:56:02 +1100 > > > David Gibson <david@gibson.dropbear.id.au> wrote: > > > > > > > Due to the way feature negotiation works in PAPR (which is a > > > > paravirtualized platform), we can end up changing the global irq chip > > > > at runtime, including it's KVM accelerate model. That causes > > > > complications for VFIO devices with INTx, which wire themselves up > > > > directly to the KVM irqchip for performance. > > > > > > > > This series introduces a new notifier to let VFIO devices (and > > > > anything else that needs to in the future) know about changes to the > > > > master irqchip. It modifies VFIO to respond to the notifier, > > > > reconnecting itself to the new KVM irqchip as necessary. > > > > > > > > In particular this removes a misleading (though not wholly inaccurate) > > > > warning that occurs when using VFIO devices on a pseries machine type > > > > guest. > > > > > > > > Open question: should this go into qemu-4.2 or wait until 5.0? It's > > > > has medium complexity / intrusiveness, but it *is* a bugfix that I > > > > can't see a simpler way to fix. It's effectively a regression from > > > > qemu-4.0 to qemu-4.1 (because that introduced XIVE support by > > > > default), although not from 4.1 to 4.2. > > > > > > Looks reasonable to me for 4.2, the vfio changes are not as big as they > > > appear. If Paolo approves this week, I can send a pull request, > > > otherwise I can leave my ack for someone else as I'll be on PTO/holiday > > > next week. Thanks, > > > > I'm happy to take it through my tree, and expect to be sending a PR in > > that timescale, so an ack sounds good. > > > > I've pulled the series into my ppc-for-4.2 branch tentatively. > > Tested-by: Alex Williamson <alex.williamson@redhat.com> > Reviewed-by: Alex Williamson <alex.williamson@redhat.com> > Acked-by: Alex Williamson <alex.williamson@redhat.com> Thanks!
On Thu, 21 Nov 2019 11:56:02 +1100 David Gibson <david@gibson.dropbear.id.au> wrote: > Due to the way feature negotiation works in PAPR (which is a > paravirtualized platform), we can end up changing the global irq chip > at runtime, including it's KVM accelerate model. That causes > complications for VFIO devices with INTx, which wire themselves up > directly to the KVM irqchip for performance. > > This series introduces a new notifier to let VFIO devices (and > anything else that needs to in the future) know about changes to the > master irqchip. It modifies VFIO to respond to the notifier, > reconnecting itself to the new KVM irqchip as necessary. > > In particular this removes a misleading (though not wholly inaccurate) > warning that occurs when using VFIO devices on a pseries machine type > guest. > > Open question: should this go into qemu-4.2 or wait until 5.0? It's > has medium complexity / intrusiveness, but it *is* a bugfix that I > can't see a simpler way to fix. It's effectively a regression from > qemu-4.0 to qemu-4.1 (because that introduced XIVE support by > default), although not from 4.1 to 4.2. > > Changes since RFC: > * Fixed some incorrect error paths pointed by aw in 3/5 > * 5/5 had some problems previously, but they have been obsoleted by > other changes merged in the meantime > > David Gibson (5): > kvm: Introduce KVM irqchip change notifier > vfio/pci: Split vfio_intx_update() > vfio/pci: Respond to KVM irqchip change notifier > spapr: Handle irq backend changes with VFIO PCI devices > spapr: Work around spurious warnings from vfio INTx initialization > > accel/kvm/kvm-all.c | 18 ++++++++++++ > accel/stubs/kvm-stub.c | 12 ++++++++ > hw/ppc/spapr_irq.c | 17 +++++++++++- > hw/vfio/pci.c | 62 +++++++++++++++++++++++++++--------------- > hw/vfio/pci.h | 1 + > include/sysemu/kvm.h | 5 ++++ > 6 files changed, 92 insertions(+), 23 deletions(-) > With the issue spotted in patch 3/5 fixed, the series looks good: Reviewed-by: Greg Kurz <groug@kaod.org> Then I've tried passthrough of a BCM5719 gigabit adapter to a guest. It works as expected with MSIs but if I force LSI, either through /sys or with the pci=nomsi kernel command line, I get no interrupts for the device in the guest. Note that the same device works ok with LSI in the host.