[13/16] kvm: enable MSI-X capabilty for assigned device

Commit Message

Sheng Yang March 17, 2009, 3:50 a.m. UTC

The most important part here, is we emulate a page of MMIO region using a
page of memory. That's because MSI-X table was put in the region and we have to
intercept it.

(Thanks Ben to find one bug in assigned_dev_update_msix_mmio)

Signed-off-by: Sheng Yang <[e-mail]>
Signed-off-by: Yunbiao (Ben) Lin <[e-mail]>
---
 qemu/hw/device-assignment.c |  294 ++++++++++++++++++++++++++++++++++++++++++-
 qemu/hw/device-assignment.h |    6 +
 2 files changed, 295 insertions(+), 5 deletions(-)

Comments

Alex Williamson April 3, 2009, 9:27 p.m. UTC | #1

On Tue, 2009-03-17 at 11:50 +0800, Sheng Yang wrote:
> +    if (*ctrl_word & PCI_MSIX_ENABLE) {
> +        if (assigned_dev_update_msix_mmio(pci_dev) < 0) {
> +            perror("assigned_dev_update_msix_mmio");
> +            return;
> +        }
> +        if (kvm_assign_irq(kvm_context, &assigned_irq_data) < 0) {
> +            perror("assigned_dev_enable_msix: assign irq");
> +            return;
> +        }
> +        assigned_dev->irq_requested_type = assigned_irq_data.flags;
> +    }
> +}

Do we need some disable logic here?  If I toggle a bnx2 NIC in a guest,
I get the following when it attempts to come back up:

MSI-X entry number is zero!
assigned_dev_update_msix_mmio: No such device or address

Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sheng Yang April 7, 2009, 6:09 a.m. UTC | #2

On Saturday 04 April 2009 05:27:43 Alex Williamson wrote:
> On Tue, 2009-03-17 at 11:50 +0800, Sheng Yang wrote:
> > +    if (*ctrl_word & PCI_MSIX_ENABLE) {
> > +        if (assigned_dev_update_msix_mmio(pci_dev) < 0) {
> > +            perror("assigned_dev_update_msix_mmio");
> > +            return;
> > +        }
> > +        if (kvm_assign_irq(kvm_context, &assigned_irq_data) < 0) {
> > +            perror("assigned_dev_enable_msix: assign irq");
> > +            return;
> > +        }
> > +        assigned_dev->irq_requested_type = assigned_irq_data.flags;
> > +    }
> > +}
>
> Do we need some disable logic here?  If I toggle a bnx2 NIC in a guest,
> I get the following when it attempts to come back up:
>
> MSI-X entry number is zero!
> assigned_dev_update_msix_mmio: No such device or address

It seems that driver didn't fill the MMIO with any correct MSIX information, 
or the program fail to intercept it after driver set enable bit of MSIX. It's 
strange... (Have it got something to do with PM and some EXP feature you 
mentioned?)

Could you enable DEVICE_ASSSIGNMENT_DEBUG=1 in qemu/hw/device-assignment.c and 
post the output?

Thanks!

Alex Williamson April 7, 2009, 4:38 p.m. UTC | #3

On Tue, 2009-04-07 at 14:09 +0800, Sheng Yang wrote:
> On Saturday 04 April 2009 05:27:43 Alex Williamson wrote:
> >
> > Do we need some disable logic here?  If I toggle a bnx2 NIC in a guest,
> > I get the following when it attempts to come back up:
> >
> > MSI-X entry number is zero!
> > assigned_dev_update_msix_mmio: No such device or address
> 
> It seems that driver didn't fill the MMIO with any correct MSIX information, 
> or the program fail to intercept it after driver set enable bit of MSIX. It's 
> strange... (Have it got something to do with PM and some EXP feature you 
> mentioned?)

My guess was that it filled in the MSIX info, but then can't find a free
slot to reload the MSIX data when it tries to re-enable MSIX.  I hacked
the bnx2 driver to not rely on PM and EXP capabilities for this test, it
seems to work, but it's possible that I broke something.  My host also
locks up the second time I try to export this device to a guest, maybe a
problem with my bnx2 hacks, MSIX not getting reset, or prototype
hardware.  I'll see if I can find another MSIX capable device to export
to a guest.

> Could you enable DEVICE_ASSSIGNMENT_DEBUG=1 in qemu/hw/device-assignment.c and 
> post the output?

Yup, see below.  The error comes after I 'ifdown eth0; ifup eth0' in the
guest.  Note bnx2 appears to only turn on MSIX for SMP systems.  Thanks,

Alex

$ sudo qemu-system-x86_64 -hda kvm-vtd.img -m 4096 -net none -vnc :1 -pcidevice host=03:00.0 -boot c -smp 8
init_assigned_device: Registering real physical device 03:00.0 (bus=3 dev=0 func=0)
get_real_device: region 0 size 33554432 start 0xf4000000 type 512 resource_fd 19
assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2
assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2
assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2
assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2
assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2
assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2
assigned_dev_pci_read_config: (4.0): address=000a val=0x00000200 len=2
assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2
assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2
assigned_dev_pci_write_config: (4.0): address=0010 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0010 val=0xfe000000 len=4
assigned_dev_pci_read_config: (4.0): address=0010 val=0xfe000000 len=4
assigned_dev_pci_write_config: (4.0): address=0010 val=0xf4000000 len=4
assigned_dev_iomem_map: e_phys=f4000000 r_virt=0x7f14f4ba0000 type=0 len=02000000 region_num=0 
assigned_dev_iomem_map: munmap done, virt_base 0x0x7f14f4bac000
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_write_config: (4.0): address=0014 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0018 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=001c val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0020 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0024 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0030 val=0xffffffff len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0030 val=0xffff0001 len=4
assigned_dev_pci_read_config: (4.0): address=0030 val=0xffff0001 len=4
assigned_dev_pci_write_config: (4.0): address=0030 val=0x00010000 len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0x00010000 len=4
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_read_config: (4.0): address=003d val=0x00000001 len=1
assigned_dev_pci_write_config: (4.0): address=003c val=0x0000000b len=1
assigned_dev_pci_read_config: (4.0): address=000a val=0x00000200 len=2
assigned_dev_pci_read_config: (4.0): address=0000 val=0x000014e4 len=2
assigned_dev_pci_read_config: (4.0): address=0002 val=0x00001639 len=2
assigned_dev_pci_read_config: (4.0): address=000e val=0x00000080 len=1
assigned_dev_pci_read_config: (4.0): address=0000 val=0x163914e4 len=4
assigned_dev_pci_read_config: (4.0): address=000e val=0x00000080 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0008 val=0x02000020 len=4
assigned_dev_pci_read_config: (4.0): address=003d val=0x00000001 len=1
assigned_dev_pci_read_config: (4.0): address=003c val=0x0000000b len=1
assigned_dev_pci_read_config: (4.0): address=0010 val=0xf4000000 len=4
assigned_dev_pci_write_config: (4.0): address=0010 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0010 val=0xfe000000 len=4
assigned_dev_pci_write_config: (4.0): address=0010 val=0xf4000000 len=4
assigned_dev_iomem_map: e_phys=f4000000 r_virt=0x7f14f4ba0000 type=0 len=02000000 region_num=0 
BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
assigned_dev_iomem_map: munmap done, virt_base 0x0x7f14f4bac000
assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0014 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0018 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0018 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=001c val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=001c val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0020 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0024 val=0xffffffff len=4
assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_write_config: (4.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0030 val=0x00010000 len=4
assigned_dev_pci_write_config: (4.0): address=0030 val=0xfffffffe len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0xfffffffe len=4
assigned_dev_pci_read_config: (4.0): address=0030 val=0xffff0000 len=4
assigned_dev_pci_write_config: (4.0): address=0030 val=0x00010000 len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0030 val=0x00010000 len=4
assigned_dev_pci_read_config: (4.0): address=002c val=0x0000103c len=2
assigned_dev_pci_read_config: (4.0): address=002e val=0x00007055 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000442 len=2
assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000446 len=2
assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000446 len=2
assigned_dev_pci_read_config: (4.0): address=000d val=0x00000000 len=1
assigned_dev_pci_write_config: (4.0): address=000d val=0x00000040 len=1
assigned_dev_pci_write_config: NON BAR (4.0): address=000d val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0000 val=0x163914e4 len=4
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00100446 len=4
assigned_dev_pci_read_config: (4.0): address=0008 val=0x02000020 len=4
assigned_dev_pci_read_config: (4.0): address=000c val=0x00800010 len=4
assigned_dev_pci_read_config: (4.0): address=0010 val=0xf4000000 len=4
assigned_dev_pci_read_config: (4.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0018 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=001c val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=0028 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=002c val=0x7055103c len=4
assigned_dev_pci_read_config: (4.0): address=0030 val=0x00010000 len=4
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=4
assigned_dev_pci_read_config: (4.0): address=0038 val=0x00000000 len=4
assigned_dev_pci_read_config: (4.0): address=003c val=0x0000010b len=4
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_write_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0054 val=0x0000c000 len=4
msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val 0x4149
msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x14, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val 0x4151
msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x24, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val 0x4159
msix_mmio_writel: write to MSI-X entry table mmio offset 0x30, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x34, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x38, val 0x4161
msix_mmio_writel: write to MSI-X entry table mmio offset 0x40, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x44, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x48, val 0x4169
msix_mmio_writel: write to MSI-X entry table mmio offset 0x50, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x54, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x58, val 0x4171
msix_mmio_writel: write to MSI-X entry table mmio offset 0x60, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x64, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x68, val 0x4179
msix_mmio_writel: write to MSI-X entry table mmio offset 0x70, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x74, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x78, val 0x4181
msix_mmio_writel: write to MSI-X entry table mmio offset 0x80, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x84, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x88, val 0x4189
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000046 len=2
assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000446 len=2
assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000446 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_write_config: (4.0): address=0052 val=0x00008008 len=2
assigned_dev_update_msix_mmio: MSI-X data 0x4149, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x18, entry 0
!assigned_dev_update_msix_mmio: MSI-X data 0x4151, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x19, entry 1
!assigned_dev_update_msix_mmio: MSI-X data 0x4159, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1a, entry 2
!assigned_dev_update_msix_mmio: MSI-X data 0x4161, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1b, entry 3
!assigned_dev_update_msix_mmio: MSI-X data 0x4169, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1c, entry 4
!assigned_dev_update_msix_mmio: MSI-X data 0x4171, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1d, entry 5
!assigned_dev_update_msix_mmio: MSI-X data 0x4179, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1e, entry 6
!assigned_dev_update_msix_mmio: MSI-X data 0x4181, MSI-X addr_lo 0xfeeff00c
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1f, entry 7
!msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x0
assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4
assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4
msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00008008 len=2
assigned_dev_pci_write_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000046 len=2
msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x8c, val 0x1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_write_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_read_config: (4.0): address=0054 val=0x0000c000 len=4
msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val 0x4191
msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x14, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val 0x4199
msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x24, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val 0x41a1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x30, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x34, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x38, val 0x41a9
msix_mmio_writel: write to MSI-X entry table mmio offset 0x40, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x44, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x48, val 0x41b1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x50, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x54, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x58, val 0x41b9
msix_mmio_writel: write to MSI-X entry table mmio offset 0x60, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x64, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x68, val 0x41c1
msix_mmio_writel: write to MSI-X entry table mmio offset 0x70, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x74, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x78, val 0x41c9
msix_mmio_writel: write to MSI-X entry table mmio offset 0x80, val 0xfeeff00c
msix_mmio_writel: write to MSI-X entry table mmio offset 0x84, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x88, val 0x41d1
assigned_dev_pci_read_config: (4.0): address=0004 val=0x00000046 len=2
assigned_dev_pci_write_config: (4.0): address=0004 val=0x00000446 len=2
assigned_dev_pci_write_config: NON BAR (4.0): address=0004 val=0x00000446 len=2
assigned_dev_pci_read_config: (4.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (4.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (4.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (4.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (4.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (4.0): address=0052 val=0x00000008 len=2
assigned_dev_pci_write_config: (4.0): address=0052 val=0x00008008 len=2
MSI-X entry number is zero!
assigned_dev_update_msix_mmio: No such device or address
msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x3c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x5c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x6c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x7c, val 0x0
assigned_dev_pci_write_config: (4.0): address=0068 val=0x00000088 len=4
assigned_dev_pci_write_config: NON BAR (4.0): address=0068 val=0x00000088 len=4


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/qemu/hw/device-assignment.c b/qemu/hw/device-assignment.c
index d404286..b0b90c1 100644
--- a/qemu/hw/device-assignment.c
+++ b/qemu/hw/device-assignment.c
@@ -146,6 +146,7 @@  static void assigned_dev_iomem_map(PCIDevice *pci_dev, int region_num,
 {
     AssignedDevice *r_dev = container_of(pci_dev, AssignedDevice, dev);
     AssignedDevRegion *region = &r_dev->v_addrs[region_num];
+    PCIRegion *real_region = &r_dev->real_device.regions[region_num];
     uint32_t old_ephys = region->e_physbase;
     uint32_t old_esize = region->e_size;
     int first_map = (region->e_size == 0);
@@ -161,10 +162,27 @@  static void assigned_dev_iomem_map(PCIDevice *pci_dev, int region_num,
 	kvm_destroy_phys_mem(kvm_context, old_ephys,
                              TARGET_PAGE_ALIGN(old_esize));
 
-    if (e_size > 0)
+    if (e_size > 0) {
+        /* deal with MSI-X MMIO page */
+        if (real_region->base_addr <= r_dev->msix_table_addr &&
+                real_region->base_addr + real_region->size >=
+                r_dev->msix_table_addr) {
+            int offset = r_dev->msix_table_addr - real_region->base_addr;
+            ret = munmap(region->u.r_virtbase + offset, TARGET_PAGE_SIZE);
+            if (ret == 0)
+                DEBUG("munmap done, virt_base 0x%p\n",
+                        region->u.r_virtbase + offset);
+            else {
+                fprintf(stderr, "%s: fail munmap msix table!\n", __func__);
+                exit(1);
+            }
+            cpu_register_physical_memory(e_phys + offset,
+                    TARGET_PAGE_SIZE, r_dev->mmio_index);
+        }
 	ret = kvm_register_phys_mem(kvm_context, e_phys,
                                     region->u.r_virtbase,
                                     TARGET_PAGE_ALIGN(e_size), 0);
+    }
 
     if (ret != 0) {
 	fprintf(stderr, "%s: Error: create new mapping failed\n", __func__);
@@ -674,7 +692,9 @@  void assigned_dev_update_irqs()
     }
 }
 
-#if defined(KVM_CAP_DEVICE_MSI) && defined (KVM_CAP_IRQ_ROUTING)
+#ifdef KVM_CAP_IRQ_ROUTING
+
+#ifdef KVM_CAP_DEVICE_MSI
 static void assigned_dev_update_msi(PCIDevice *pci_dev, unsigned int ctrl_pos)
 {
     struct kvm_assigned_irq assigned_irq_data;
@@ -733,14 +753,154 @@  static void assigned_dev_update_msi(PCIDevice *pci_dev, unsigned int ctrl_pos)
 }
 #endif
 
+#ifdef KVM_CAP_DEVICE_MSIX
+static int assigned_dev_update_msix_mmio(PCIDevice *pci_dev)
+{
+    AssignedDevice *adev = container_of(pci_dev, AssignedDevice, dev);
+    u16 entries_nr = 0, entries_max_nr;
+    int pos = 0, i, r = 0;
+    u32 msg_addr, msg_upper_addr, msg_data, msg_ctrl;
+    struct kvm_assigned_msix_nr msix_nr;
+    struct kvm_assigned_msix_entry msix_entry;
+    void *va = adev->msix_table_page;
+
+    if (adev->cap.available & ASSIGNED_DEVICE_CAP_MSI)
+        pos = pci_dev->cap.start + PCI_CAPABILITY_CONFIG_MSI_LENGTH;
+    else
+        pos = pci_dev->cap.start;
+
+    entries_max_nr = pci_dev->config[pos + 2];
+    entries_max_nr &= PCI_MSIX_TABSIZE;
+
+    /* Get the usable entry number for allocating */
+    for (i = 0; i < entries_max_nr; i++) {
+        memcpy(&msg_ctrl, va + i * 16 + 12, 4);
+        /* 0x1 is mask bit for per vector */
+        if (msg_ctrl & 0x1)
+            continue;
+        memcpy(&msg_data, va + i * 16 + 8, 4);
+        /* Ignore unused entry even it's unmasked */
+        if (msg_data == 0)
+            continue;
+        entries_nr ++;
+    }
+
+    if (entries_nr == 0) {
+        fprintf(stderr, "MSI-X entry number is zero!\n");
+        return -EINVAL;
+    }
+    msix_nr.assigned_dev_id = calc_assigned_dev_id(adev->h_busnr,
+                                          (uint8_t)adev->h_devfn);
+    msix_nr.entry_nr = entries_nr;
+    r = kvm_assign_set_msix_nr(kvm_context, &msix_nr);
+    if (r != 0) {
+        fprintf(stderr, "fail to set MSI-X entry number for MSIX! %s\n",
+			strerror(-r));
+        return r;
+    }
+
+    free_dev_irq_entries(adev);
+    adev->irq_entries_nr = entries_nr;
+    adev->entry = calloc(entries_nr, sizeof(struct kvm_irq_routing_entry));
+    if (!adev->entry) {
+        perror("assigned_dev_update_msix_mmio: ");
+        return -errno;
+    }
+
+    msix_entry.assigned_dev_id = msix_nr.assigned_dev_id;
+    entries_nr = 0;
+    for (i = 0; i < entries_max_nr; i++) {
+        if (entries_nr >= msix_nr.entry_nr)
+            break;
+        memcpy(&msg_ctrl, va + i * 16 + 12, 4);
+        if (msg_ctrl & 0x1)
+            continue;
+        memcpy(&msg_data, va + i * 16 + 8, 4);
+        if (msg_data == 0)
+            continue;
+
+        memcpy(&msg_addr, va + i * 16, 4);
+        memcpy(&msg_upper_addr, va + i * 16 + 4, 4);
+
+        r = kvm_get_irq_route_gsi(kvm_context);
+        if (r < 0)
+            return r;
+
+        adev->entry[entries_nr].gsi = r;
+        adev->entry[entries_nr].type = KVM_IRQ_ROUTING_MSI;
+        adev->entry[entries_nr].flags = 0;
+        adev->entry[entries_nr].u.msi.address_lo = msg_addr;
+        adev->entry[entries_nr].u.msi.address_hi = msg_upper_addr;
+        adev->entry[entries_nr].u.msi.data = msg_data;
+        DEBUG("MSI-X data 0x%x, MSI-X addr_lo 0x%x\n!", msg_data, msg_addr);
+	kvm_add_routing_entry(kvm_context, &adev->entry[entries_nr]);
+
+        msix_entry.gsi = adev->entry[entries_nr].gsi;
+        msix_entry.entry = i;
+        r = kvm_assign_set_msix_entry(kvm_context, &msix_entry);
+        if (r) {
+            fprintf(stderr, "fail to set MSI-X entry! %s\n", strerror(-r));
+            break;
+        }
+        DEBUG("MSI-X entry gsi 0x%x, entry %d\n!",
+                msix_entry.gsi, msix_entry.entry);
+        entries_nr ++;
+    }
+
+    if (r == 0 && kvm_commit_irq_routes(kvm_context) < 0) {
+	    perror("assigned_dev_update_msix_mmio: kvm_commit_irq_routes");
+	    return -EINVAL;
+    }
+
+    return r;
+}
+
+static void assigned_dev_update_msix(PCIDevice *pci_dev, unsigned int ctrl_pos)
+{
+    struct kvm_assigned_irq assigned_irq_data;
+    AssignedDevice *assigned_dev = container_of(pci_dev, AssignedDevice, dev);
+    uint16_t *ctrl_word = (uint16_t *)(pci_dev->config + ctrl_pos);
+    int r;
+
+    memset(&assigned_irq_data, 0, sizeof assigned_irq_data);
+    assigned_irq_data.assigned_dev_id  =
+            calc_assigned_dev_id(assigned_dev->h_busnr,
+                    (uint8_t)assigned_dev->h_devfn);
+
+    if (assigned_dev->irq_requested_type) {
+        assigned_irq_data.flags = assigned_dev->irq_requested_type;
+        free_dev_irq_entries(assigned_dev);
+        r = kvm_deassign_irq(kvm_context, &assigned_irq_data);
+        /* -ENXIO means no assigned irq */
+        if (r && r != -ENXIO)
+            perror("assigned_dev_update_msix: deassign irq");
+    }
+    assigned_irq_data.flags = KVM_DEV_IRQ_HOST_MSIX | KVM_DEV_IRQ_GUEST_MSIX;
+
+    if (*ctrl_word & PCI_MSIX_ENABLE) {
+        if (assigned_dev_update_msix_mmio(pci_dev) < 0) {
+            perror("assigned_dev_update_msix_mmio");
+            return;
+        }
+        if (kvm_assign_irq(kvm_context, &assigned_irq_data) < 0) {
+            perror("assigned_dev_enable_msix: assign irq");
+            return;
+        }
+        assigned_dev->irq_requested_type = assigned_irq_data.flags;
+    }
+}
+#endif
+#endif
+
 static void assigned_device_pci_cap_write_config(PCIDevice *pci_dev, uint32_t address,
-                                          uint32_t val, int len)
+                                                 uint32_t val, int len)
 {
     AssignedDevice *assigned_dev = container_of(pci_dev, AssignedDevice, dev);
     unsigned int pos = pci_dev->cap.start, ctrl_pos;
 
     pci_default_cap_write_config(pci_dev, address, val, len);
-#if defined(KVM_CAP_DEVICE_MSI) && defined (KVM_CAP_IRQ_ROUTING)
+#ifdef KVM_CAP_IRQ_ROUTING
+#ifdef KVM_CAP_DEVICE_MSI
     if (assigned_dev->cap.available & ASSIGNED_DEVICE_CAP_MSI) {
         ctrl_pos = pos + PCI_MSI_FLAGS;
         if (address <= ctrl_pos && address + len > ctrl_pos)
@@ -748,16 +908,29 @@  static void assigned_device_pci_cap_write_config(PCIDevice *pci_dev, uint32_t ad
         pos += PCI_CAPABILITY_CONFIG_MSI_LENGTH;
     }
 #endif
+#ifdef KVM_CAP_DEVICE_MSIX
+    if (assigned_dev->cap.available & ASSIGNED_DEVICE_CAP_MSIX) {
+        ctrl_pos = pos + 3;
+        if (address <= ctrl_pos && address + len > ctrl_pos) {
+            ctrl_pos--; /* control is word long */
+            assigned_dev_update_msix(pci_dev, ctrl_pos);
+	}
+        pos += PCI_CAPABILITY_CONFIG_MSIX_LENGTH;
+    }
+#endif
+#endif
     return;
 }
 
 static int assigned_device_pci_cap_init(PCIDevice *pci_dev)
 {
     AssignedDevice *dev = container_of(pci_dev, AssignedDevice, dev);
+    PCIRegion *pci_region = dev->real_device.regions;
     int next_cap_pt = 0;
 
     pci_dev->cap.length = 0;
-#if defined(KVM_CAP_DEVICE_MSI) && defined (KVM_CAP_IRQ_ROUTING)
+#ifdef KVM_CAP_IRQ_ROUTING
+#ifdef KVM_CAP_DEVICE_MSI
     /* Expose MSI capability
      * MSI capability is the 1st capability in capability config */
     if (pci_find_cap_offset(dev->pdev, PCI_CAP_ID_MSI)) {
@@ -770,7 +943,113 @@  static int assigned_device_pci_cap_init(PCIDevice *pci_dev)
         next_cap_pt = 1;
     }
 #endif
+#ifdef KVM_CAP_DEVICE_MSIX
+    /* Expose MSI-X capability */
+    if (pci_find_cap_offset(dev->pdev, PCI_CAP_ID_MSIX)) {
+        int pos, entry_nr, bar_nr;
+        u32 msix_table_entry;
+        dev->cap.available |= ASSIGNED_DEVICE_CAP_MSIX;
+        memset(&pci_dev->config[pci_dev->cap.start + pci_dev->cap.length],
+               0, PCI_CAPABILITY_CONFIG_MSIX_LENGTH);
+        pos = pci_find_cap_offset(dev->pdev, PCI_CAP_ID_MSIX);
+        entry_nr = pci_read_word(dev->pdev, pos + 2) & PCI_MSIX_TABSIZE;
+        pci_dev->config[pci_dev->cap.start + pci_dev->cap.length] = 0x11;
+        pci_dev->config[pci_dev->cap.start +
+                        pci_dev->cap.length + 2] = entry_nr;
+        msix_table_entry = pci_read_long(dev->pdev, pos + PCI_MSIX_TABLE);
+        *(uint32_t *)(pci_dev->config + pci_dev->cap.start +
+                      pci_dev->cap.length + PCI_MSIX_TABLE) = msix_table_entry;
+        *(uint32_t *)(pci_dev->config + pci_dev->cap.start +
+                      pci_dev->cap.length + PCI_MSIX_PBA) =
+                    pci_read_long(dev->pdev, pos + PCI_MSIX_PBA);
+        bar_nr = msix_table_entry & PCI_MSIX_BIR;
+        msix_table_entry &= ~PCI_MSIX_BIR;
+        dev->msix_table_addr = pci_region[bar_nr].base_addr + msix_table_entry;
+        if (next_cap_pt != 0) {
+            pci_dev->config[pci_dev->cap.start + next_cap_pt] =
+                pci_dev->cap.start + pci_dev->cap.length;
+            next_cap_pt += PCI_CAPABILITY_CONFIG_MSI_LENGTH;
+        } else
+            next_cap_pt = 1;
+        pci_dev->cap.length += PCI_CAPABILITY_CONFIG_MSIX_LENGTH;
+    }
+#endif
+#endif
+
+    return 0;
+}
+
+static uint32_t msix_mmio_readl(void *opaque, target_phys_addr_t addr)
+{
+    AssignedDevice *adev = opaque;
+    unsigned int offset = addr & 0xfff;
+    void *page = adev->msix_table_page;
+    uint32_t val = 0;
 
+    memcpy(&val, (void *)((char *)page + offset), 4);
+
+    return val;
+}
+
+static uint32_t msix_mmio_readb(void *opaque, target_phys_addr_t addr)
+{
+    return ((msix_mmio_readl(opaque, addr & ~3)) >>
+            (8 * (addr & 3))) & 0xff;
+}
+
+static uint32_t msix_mmio_readw(void *opaque, target_phys_addr_t addr)
+{
+    return ((msix_mmio_readl(opaque, addr & ~3)) >>
+            (8 * (addr & 3))) & 0xffff;
+}
+
+static void msix_mmio_writel(void *opaque,
+                             target_phys_addr_t addr, uint32_t val)
+{
+    AssignedDevice *adev = opaque;
+    unsigned int offset = addr & 0xfff;
+    void *page = adev->msix_table_page;
+
+    DEBUG("write to MSI-X entry table mmio offset 0x%lx, val 0x%lx\n",
+		    addr, val);
+    memcpy((void *)((char *)page + offset), &val, 4);
+}
+
+static void msix_mmio_writew(void *opaque,
+                             target_phys_addr_t addr, uint32_t val)
+{
+    msix_mmio_writel(opaque, addr & ~3,
+                     (val & 0xffff) << (8*(addr & 3)));
+}
+
+static void msix_mmio_writeb(void *opaque,
+                             target_phys_addr_t addr, uint32_t val)
+{
+    msix_mmio_writel(opaque, addr & ~3,
+                     (val & 0xff) << (8*(addr & 3)));
+}
+
+static CPUWriteMemoryFunc *msix_mmio_write[] = {
+    msix_mmio_writeb,	msix_mmio_writew,	msix_mmio_writel
+};
+
+static CPUReadMemoryFunc *msix_mmio_read[] = {
+    msix_mmio_readb,	msix_mmio_readw,	msix_mmio_readl
+};
+
+static int assigned_dev_register_msix_mmio(AssignedDevice *dev)
+{
+    dev->msix_table_page = mmap(NULL, 0x1000,
+                                PROT_READ|PROT_WRITE,
+                                MAP_ANONYMOUS|MAP_PRIVATE, 0, 0);
+    memset(dev->msix_table_page, 0, 0x1000);
+    if (dev->msix_table_page == MAP_FAILED) {
+        fprintf(stderr, "fail allocate msix_table_page! %s\n",
+                strerror(errno));
+        return -EFAULT;
+    }
+    dev->mmio_index = cpu_register_io_memory(0,
+                        msix_mmio_read, msix_mmio_write, dev);
     return 0;
 }
 
@@ -838,6 +1117,11 @@  struct PCIDevice *init_assigned_device(AssignedDevInfo *adev, PCIBus *bus)
                     assigned_device_pci_cap_init) < 0)
         goto assigned_out;
 
+    /* intercept MSI-X entry page in the MMIO */
+    if (dev->cap.available & ASSIGNED_DEVICE_CAP_MSIX)
+        if (assigned_dev_register_msix_mmio(dev))
+            return NULL;
+
     return &dev->dev;
 
 assigned_out:
diff --git a/qemu/hw/device-assignment.h b/qemu/hw/device-assignment.h
index 5308073..85ed949 100644
--- a/qemu/hw/device-assignment.h
+++ b/qemu/hw/device-assignment.h
@@ -85,12 +85,18 @@  typedef struct {
     struct pci_dev *pdev;
     struct {
 #define ASSIGNED_DEVICE_CAP_MSI (1 << 0)
+#define ASSIGNED_DEVICE_CAP_MSIX (1 << 1)
         uint32_t available;
 #define ASSIGNED_DEVICE_MSI_ENABLED (1 << 0)
+#define ASSIGNED_DEVICE_MSIX_ENABLED (1 << 1)
+#define ASSIGNED_DEVICE_MSIX_MASKED (1 << 2)
         uint32_t state;
     } cap;
     int irq_entries_nr;
     struct kvm_irq_routing_entry *entry;
+    void *msix_table_page;
+    target_phys_addr_t msix_table_addr;
+    int mmio_index;
 } AssignedDevice;
 
 typedef struct AssignedDevInfo AssignedDevInfo;