[V4,0/6] IRQ offloading for vDPA

Message

Zhu, Lingshan July 28, 2020, 4:23 a.m. UTC

This series intends to implement IRQ offloading for
vhost_vdpa.

By the feat of irq forwarding facilities like posted
interrupt on X86, irq bypass can  help deliver
interrupts to vCPU directly.

vDPA devices have dedicated hardware backends like VFIO
pass-throughed devices. So it would be possible to setup
irq offloading(irq bypass) for vDPA devices and gain
performance improvements.

In my testing, with this feature, we can save 0.1ms
in a ping between two VFs on average.
changes from V3:
(1)removed vDPA irq allocate/free helpers in vDPA core.
(2)add a new function get_vq_irq() in struct vdpa_config_ops,
upper layer driver can use this function to: A. query the
irq numbner of a vq. B. detect whether a vq is enabled.
(3)implement get_vq_irq() in ifcvf driver.
(4)in vhost_vdpa, set_status() will setup irq offloading when
setting DRIVER_OK, and unsetup when receive !DRIVER_OK.
(5)minor improvements.

changes from V2:
(1)rename struct vhost_call_ctx to vhost_vring_call
(2)add kvm_arch_end_assignment() in del_producer()
code path
(3)rename vDPA helpers to vdpa_devm_request_irq()
and vdpa_devm_free_irq(). Add better comments
for them.
(4)better comments for setup_vq_irq() and
unsetup_vq_irq()
(5)In vDPA VHOST_SET_VRING_CALL, will call vhost_vdpa_update_vq_irq()
without checking producer.irq, move this check into
vhost_vdpa_update_vq_irq(), so that get advantage of the spinlock.
(6)Add a function vhost_vdpa_clean_irq(), this function will unregister
the producer of vqs when vhost_vdpa_release(). This is safe
for control vq.
(7) minor improvements

changes from V1:
(1)dropped vfio changes.
(3)removed KVM_HVAE_IRQ_BYPASS checks
(4)locking fixes
(5)simplified vhost_vdpa_updat


Zhu Lingshan (6):
  vhost: introduce vhost_vring_call
  kvm: detect assigned device via irqbypass manager
  vDPA: add get_vq_irq() in vdpa_config_ops
  vhost_vdpa: implement IRQ offloading in vhost_vdpa
  ifcvf: implement vdpa_config_ops.get_vq_irq()
  irqbypass: do not start cons/prod when failed connect

 arch/x86/kvm/x86.c              | 12 ++++-
 drivers/vdpa/ifcvf/ifcvf_main.c | 18 ++++++-
 drivers/vhost/Kconfig           |  1 +
 drivers/vhost/vdpa.c            | 83 +++++++++++++++++++++++++++++++--
 drivers/vhost/vhost.c           | 22 ++++++---
 drivers/vhost/vhost.h           |  9 +++-
 include/linux/vdpa.h            |  6 +++
 virt/lib/irqbypass.c            | 16 ++++---
 8 files changed, 147 insertions(+), 20 deletions(-)

Comments

Jason Wang July 31, 2020, 3:18 a.m. UTC | #1

On 2020/7/28 下午12:23, Zhu Lingshan wrote:
> This series intends to implement IRQ offloading for
> vhost_vdpa.
>
> By the feat of irq forwarding facilities like posted
> interrupt on X86, irq bypass can  help deliver
> interrupts to vCPU directly.
>
> vDPA devices have dedicated hardware backends like VFIO
> pass-throughed devices. So it would be possible to setup
> irq offloading(irq bypass) for vDPA devices and gain
> performance improvements.
>
> In my testing, with this feature, we can save 0.1ms
> in a ping between two VFs on average.
> changes from V3:
> (1)removed vDPA irq allocate/free helpers in vDPA core.
> (2)add a new function get_vq_irq() in struct vdpa_config_ops,
> upper layer driver can use this function to: A. query the
> irq numbner of a vq. B. detect whether a vq is enabled.
> (3)implement get_vq_irq() in ifcvf driver.
> (4)in vhost_vdpa, set_status() will setup irq offloading when
> setting DRIVER_OK, and unsetup when receive !DRIVER_OK.
> (5)minor improvements.


Ok, I think you can go ahead to post a V5. It's not bad to start with 
get_vq_irq() and we can do any changes afterwards if it can work for 
some cases.

Thanks