| Message ID | 1443624989-24346-5-git-send-email-Bharat.Bhushan@freescale.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Hi Bharat, [auto build test results on v4.3-rc3 -- if it's inappropriate base, please ignore] config: x86_64-rhel (attached as .config) reproduce: git checkout 6fdf43e0b410216a2fe2d1d6e8541fb4f69557f9 # save the attached .config to linux build tree make ARCH=x86_64 All error/warnings (new ones prefixed by >>): >> ERROR: "vfio_device_map_msi" [drivers/vfio/pci/vfio-pci.ko] undefined! >> ERROR: "vfio_device_unmap_msi" [drivers/vfio/pci/vfio-pci.ko] undefined! --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation
> -----Original Message----- > From: kbuild test robot [mailto:lkp@intel.com] > Sent: Wednesday, September 30, 2015 4:33 PM > To: Bhushan Bharat-R65777 <Bharat.Bhushan@freescale.com> > Cc: kbuild-all@01.org; kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; > alex.williamson@redhat.com; christoffer.dall@linaro.org; > eric.auger@linaro.org; pranavkumar@linaro.org; marc.zyngier@arm.com; > will.deacon@arm.com; Bhushan Bharat-R65777 > <Bharat.Bhushan@freescale.com> > Subject: Re: [RFC PATCH 5/6] vfio-pci: Create iommu mapping for msi > interrupt > > Hi Bharat, > > [auto build test results on v4.3-rc3 -- if it's inappropriate base, please ignore] > > config: x86_64-rhel (attached as .config) > reproduce: > git checkout 6fdf43e0b410216a2fe2d1d6e8541fb4f69557f9 > # save the attached .config to linux build tree > make ARCH=x86_64 > > All error/warnings (new ones prefixed by >>): > > >> ERROR: "vfio_device_map_msi" [drivers/vfio/pci/vfio-pci.ko] undefined! > >> ERROR: "vfio_device_unmap_msi" [drivers/vfio/pci/vfio-pci.ko] > undefined! Yes, this is the problem, I will correct this in next version. Thanks -Bharat > > --- > 0-DAY kernel test infrastructure Open Source Technology Center > https://lists.01.org/pipermail/kbuild-all Intel Corporation -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Bharat, [auto build test results on v4.3-rc3 -- if it's inappropriate base, please ignore] config: i386-allmodconfig (attached as .config) reproduce: git checkout 6fdf43e0b410216a2fe2d1d6e8541fb4f69557f9 # save the attached .config to linux build tree make ARCH=i386 All error/warnings (new ones prefixed by >>): >> ERROR: "vfio_device_map_msi" undefined! >> ERROR: "vfio_device_unmap_msi" undefined! --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation
On Wed, 2015-09-30 at 20:26 +0530, Bharat Bhushan wrote: > An MSI-address is allocated and programmed in pcie device > during interrupt configuration. Now for a pass-through device, > try to create the iommu mapping for this allocted/programmed > msi-address. If the iommu mapping is created and the msi > address programmed in the pcie device is different from > msi-iova as per iommu programming then reconfigure the pci > device to use msi-iova as msi address. > > Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com> > --- > drivers/vfio/pci/vfio_pci_intrs.c | 36 ++++++++++++++++++++++++++++++++++-- > 1 file changed, 34 insertions(+), 2 deletions(-) > > diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c > index 1f577b4..c9690af 100644 > --- a/drivers/vfio/pci/vfio_pci_intrs.c > +++ b/drivers/vfio/pci/vfio_pci_intrs.c > @@ -312,13 +312,23 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, > int irq = msix ? vdev->msix[vector].vector : pdev->irq + vector; > char *name = msix ? "vfio-msix" : "vfio-msi"; > struct eventfd_ctx *trigger; > + struct msi_msg msg; > + struct vfio_device *device; > + uint64_t msi_addr, msi_iova; > int ret; > > if (vector >= vdev->num_ctx) > return -EINVAL; > > + device = vfio_device_get_from_dev(&pdev->dev); Have you looked at this function? I don't think we want to be doing that every time we want to poke the interrupt configuration. Also note that IOMMU mappings don't operate on devices, but groups, so maybe we want to pass the group. > + if (device == NULL) > + return -EINVAL; This would be a legitimate BUG_ON(!device) > + > if (vdev->ctx[vector].trigger) { > free_irq(irq, vdev->ctx[vector].trigger); > + get_cached_msi_msg(irq, &msg); > + msi_iova = ((u64)msg.address_hi << 32) | msg.address_lo; > + vfio_device_unmap_msi(device, msi_iova, PAGE_SIZE); > kfree(vdev->ctx[vector].name); > eventfd_ctx_put(vdev->ctx[vector].trigger); > vdev->ctx[vector].trigger = NULL; > @@ -346,12 +356,11 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, > * cached value of the message prior to enabling. > */ > if (msix) { > - struct msi_msg msg; > - > get_cached_msi_msg(irq, &msg); > pci_write_msi_msg(irq, &msg); > } > > + gratuitous newline > ret = request_irq(irq, vfio_msihandler, 0, > vdev->ctx[vector].name, trigger); > if (ret) { > @@ -360,6 +369,29 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, > return ret; > } > > + /* Re-program the new-iova in pci-device in case there is > + * different iommu-mapping created for programmed msi-address. > + */ > + get_cached_msi_msg(irq, &msg); > + msi_iova = 0; > + msi_addr = (u64)(msg.address_hi) << 32 | (u64)(msg.address_lo); > + ret = vfio_device_map_msi(device, msi_addr, PAGE_SIZE, &msi_iova); > + if (ret) { > + free_irq(irq, vdev->ctx[vector].trigger); > + kfree(vdev->ctx[vector].name); > + eventfd_ctx_put(trigger); > + return ret; > + } > + > + /* Reprogram only if iommu-mapped iova is different from msi-address */ > + if (msi_iova && (msi_iova != msi_addr)) { > + msg.address_hi = (u32)(msi_iova >> 32); > + /* Keep Lower bits from original msi message address */ > + msg.address_lo &= PAGE_MASK; > + msg.address_lo |= (u32)(msi_iova & 0x00000000ffffffff); Seems like you're making some assumptions here that are dependent on the architecture and maybe the platform. > + pci_write_msi_msg(irq, &msg); > + } > + > vdev->ctx[vector].trigger = trigger; > > return 0; -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> -----Original Message----- > From: Alex Williamson [mailto:alex.williamson@redhat.com] > Sent: Saturday, October 03, 2015 4:17 AM > To: Bhushan Bharat-R65777 <Bharat.Bhushan@freescale.com> > Cc: kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; > christoffer.dall@linaro.org; eric.auger@linaro.org; pranavkumar@linaro.org; > marc.zyngier@arm.com; will.deacon@arm.com > Subject: Re: [RFC PATCH 5/6] vfio-pci: Create iommu mapping for msi > interrupt > > On Wed, 2015-09-30 at 20:26 +0530, Bharat Bhushan wrote: > > An MSI-address is allocated and programmed in pcie device during > > interrupt configuration. Now for a pass-through device, try to create > > the iommu mapping for this allocted/programmed msi-address. If the > > iommu mapping is created and the msi address programmed in the pcie > > device is different from msi-iova as per iommu programming then > > reconfigure the pci device to use msi-iova as msi address. > > > > Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com> > > --- > > drivers/vfio/pci/vfio_pci_intrs.c | 36 > > ++++++++++++++++++++++++++++++++++-- > > 1 file changed, 34 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/vfio/pci/vfio_pci_intrs.c > > b/drivers/vfio/pci/vfio_pci_intrs.c > > index 1f577b4..c9690af 100644 > > --- a/drivers/vfio/pci/vfio_pci_intrs.c > > +++ b/drivers/vfio/pci/vfio_pci_intrs.c > > @@ -312,13 +312,23 @@ static int vfio_msi_set_vector_signal(struct > vfio_pci_device *vdev, > > int irq = msix ? vdev->msix[vector].vector : pdev->irq + vector; > > char *name = msix ? "vfio-msix" : "vfio-msi"; > > struct eventfd_ctx *trigger; > > + struct msi_msg msg; > > + struct vfio_device *device; > > + uint64_t msi_addr, msi_iova; > > int ret; > > > > if (vector >= vdev->num_ctx) > > return -EINVAL; > > > > + device = vfio_device_get_from_dev(&pdev->dev); > > Have you looked at this function? I don't think we want to be doing that > every time we want to poke the interrupt configuration. I am trying to describe what I understood, a device can have many interrupts and we should setup iommu only once, when called for the first time to enable/setup interrupt. Similarly when disabling the interrupt we should iommu-unmap when called for the last enabled interrupt for that device. Now with this understanding, should I move this map-unmap to separate functions and call them from vfio_msi_set_block() rather than in vfio_msi_set_vector_signal() > Also note that > IOMMU mappings don't operate on devices, but groups, so maybe we want > to pass the group. Yes, it operates on group. I hesitated to add an API to get group. Do you suggest to that it is ok to add API to get group from device. > > > + if (device == NULL) > > + return -EINVAL; > > This would be a legitimate BUG_ON(!device) > > > + > > if (vdev->ctx[vector].trigger) { > > free_irq(irq, vdev->ctx[vector].trigger); > > + get_cached_msi_msg(irq, &msg); > > + msi_iova = ((u64)msg.address_hi << 32) | msg.address_lo; > > + vfio_device_unmap_msi(device, msi_iova, PAGE_SIZE); > > kfree(vdev->ctx[vector].name); > > eventfd_ctx_put(vdev->ctx[vector].trigger); > > vdev->ctx[vector].trigger = NULL; > > @@ -346,12 +356,11 @@ static int vfio_msi_set_vector_signal(struct > vfio_pci_device *vdev, > > * cached value of the message prior to enabling. > > */ > > if (msix) { > > - struct msi_msg msg; > > - > > get_cached_msi_msg(irq, &msg); > > pci_write_msi_msg(irq, &msg); > > } > > > > + > > gratuitous newline > > > ret = request_irq(irq, vfio_msihandler, 0, > > vdev->ctx[vector].name, trigger); > > if (ret) { > > @@ -360,6 +369,29 @@ static int vfio_msi_set_vector_signal(struct > vfio_pci_device *vdev, > > return ret; > > } > > > > + /* Re-program the new-iova in pci-device in case there is > > + * different iommu-mapping created for programmed msi-address. > > + */ > > + get_cached_msi_msg(irq, &msg); > > + msi_iova = 0; > > + msi_addr = (u64)(msg.address_hi) << 32 | (u64)(msg.address_lo); > > + ret = vfio_device_map_msi(device, msi_addr, PAGE_SIZE, > &msi_iova); > > + if (ret) { > > + free_irq(irq, vdev->ctx[vector].trigger); > > + kfree(vdev->ctx[vector].name); > > + eventfd_ctx_put(trigger); > > + return ret; > > + } > > + > > + /* Reprogram only if iommu-mapped iova is different from msi- > address */ > > + if (msi_iova && (msi_iova != msi_addr)) { > > + msg.address_hi = (u32)(msi_iova >> 32); > > + /* Keep Lower bits from original msi message address */ > > + msg.address_lo &= PAGE_MASK; > > + msg.address_lo |= (u32)(msi_iova & 0x00000000ffffffff); > > Seems like you're making some assumptions here that are dependent on the > architecture and maybe the platform. What I tried is to map the msi page with different iova, which is page size aligned. But the offset within the page will remain same. For example, original msi address was 0x0603_0040 and we have a reserved region at 0xf000_0000. So iommu mapping is created for 0xf000_0000 =>0x0600_3000 of size 0x1000. So the new address to be programmed in device is 0xf000_0040, offset 0x40 added to base address in iommu mapping. Thanks -Bharat > > > + pci_write_msi_msg(irq, &msg); > > + } > > + > > vdev->ctx[vector].trigger = trigger; > > > > return 0; > >
On Mon, 2015-10-05 at 07:20 +0000, Bhushan Bharat wrote: > > > > -----Original Message----- > > From: Alex Williamson [mailto:alex.williamson@redhat.com] > > Sent: Saturday, October 03, 2015 4:17 AM > > To: Bhushan Bharat-R65777 <Bharat.Bhushan@freescale.com> > > Cc: kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; > > christoffer.dall@linaro.org; eric.auger@linaro.org; pranavkumar@linaro.org; > > marc.zyngier@arm.com; will.deacon@arm.com > > Subject: Re: [RFC PATCH 5/6] vfio-pci: Create iommu mapping for msi > > interrupt > > > > On Wed, 2015-09-30 at 20:26 +0530, Bharat Bhushan wrote: > > > An MSI-address is allocated and programmed in pcie device during > > > interrupt configuration. Now for a pass-through device, try to create > > > the iommu mapping for this allocted/programmed msi-address. If the > > > iommu mapping is created and the msi address programmed in the pcie > > > device is different from msi-iova as per iommu programming then > > > reconfigure the pci device to use msi-iova as msi address. > > > > > > Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com> > > > --- > > > drivers/vfio/pci/vfio_pci_intrs.c | 36 > > > ++++++++++++++++++++++++++++++++++-- > > > 1 file changed, 34 insertions(+), 2 deletions(-) > > > > > > diff --git a/drivers/vfio/pci/vfio_pci_intrs.c > > > b/drivers/vfio/pci/vfio_pci_intrs.c > > > index 1f577b4..c9690af 100644 > > > --- a/drivers/vfio/pci/vfio_pci_intrs.c > > > +++ b/drivers/vfio/pci/vfio_pci_intrs.c > > > @@ -312,13 +312,23 @@ static int vfio_msi_set_vector_signal(struct > > vfio_pci_device *vdev, > > > int irq = msix ? vdev->msix[vector].vector : pdev->irq + vector; > > > char *name = msix ? "vfio-msix" : "vfio-msi"; > > > struct eventfd_ctx *trigger; > > > + struct msi_msg msg; > > > + struct vfio_device *device; > > > + uint64_t msi_addr, msi_iova; > > > int ret; > > > > > > if (vector >= vdev->num_ctx) > > > return -EINVAL; > > > > > > + device = vfio_device_get_from_dev(&pdev->dev); > > > > Have you looked at this function? I don't think we want to be doing that > > every time we want to poke the interrupt configuration. > > I am trying to describe what I understood, a device can have many interrupts and we should setup iommu only once, when called for the first time to enable/setup interrupt. > Similarly when disabling the interrupt we should iommu-unmap when called for the last enabled interrupt for that device. Now with this understanding, should I move this map-unmap to separate functions and call them from vfio_msi_set_block() rather than in vfio_msi_set_vector_signal() Interrupts can be setup and torn down at any time and I don't see how one function or the other makes much difference. vfio_device_get_from_dev() is enough overhead that the data we need should be cached if we're going to call it with some regularity. Maybe vfio_iommu_driver_ops.open() should be called with a pointer to the vfio_device... or the vfio_group. > > Also note that > > IOMMU mappings don't operate on devices, but groups, so maybe we want > > to pass the group. > > Yes, it operates on group. I hesitated to add an API to get group. Do you suggest to that it is ok to add API to get group from device. No, the above suggestion is probably better. > > > > > + if (device == NULL) > > > + return -EINVAL; > > > > This would be a legitimate BUG_ON(!device) > > > > > + > > > if (vdev->ctx[vector].trigger) { > > > free_irq(irq, vdev->ctx[vector].trigger); > > > + get_cached_msi_msg(irq, &msg); > > > + msi_iova = ((u64)msg.address_hi << 32) | msg.address_lo; > > > + vfio_device_unmap_msi(device, msi_iova, PAGE_SIZE); > > > kfree(vdev->ctx[vector].name); > > > eventfd_ctx_put(vdev->ctx[vector].trigger); > > > vdev->ctx[vector].trigger = NULL; > > > @@ -346,12 +356,11 @@ static int vfio_msi_set_vector_signal(struct > > vfio_pci_device *vdev, > > > * cached value of the message prior to enabling. > > > */ > > > if (msix) { > > > - struct msi_msg msg; > > > - > > > get_cached_msi_msg(irq, &msg); > > > pci_write_msi_msg(irq, &msg); > > > } > > > > > > + > > > > gratuitous newline > > > > > ret = request_irq(irq, vfio_msihandler, 0, > > > vdev->ctx[vector].name, trigger); > > > if (ret) { > > > @@ -360,6 +369,29 @@ static int vfio_msi_set_vector_signal(struct > > vfio_pci_device *vdev, > > > return ret; > > > } > > > > > > + /* Re-program the new-iova in pci-device in case there is > > > + * different iommu-mapping created for programmed msi-address. > > > + */ > > > + get_cached_msi_msg(irq, &msg); > > > + msi_iova = 0; > > > + msi_addr = (u64)(msg.address_hi) << 32 | (u64)(msg.address_lo); > > > + ret = vfio_device_map_msi(device, msi_addr, PAGE_SIZE, > > &msi_iova); > > > + if (ret) { > > > + free_irq(irq, vdev->ctx[vector].trigger); > > > + kfree(vdev->ctx[vector].name); > > > + eventfd_ctx_put(trigger); > > > + return ret; > > > + } > > > + > > > + /* Reprogram only if iommu-mapped iova is different from msi- > > address */ > > > + if (msi_iova && (msi_iova != msi_addr)) { > > > + msg.address_hi = (u32)(msi_iova >> 32); > > > + /* Keep Lower bits from original msi message address */ > > > + msg.address_lo &= PAGE_MASK; > > > + msg.address_lo |= (u32)(msi_iova & 0x00000000ffffffff); > > > > Seems like you're making some assumptions here that are dependent on the > > architecture and maybe the platform. > > What I tried is to map the msi page with different iova, which is page size aligned. But the offset within the page will remain same. > For example, original msi address was 0x0603_0040 and we have a reserved region at 0xf000_0000. So iommu mapping is created for 0xf000_0000 =>0x0600_3000 of size 0x1000. > > So the new address to be programmed in device is 0xf000_0040, offset 0x40 added to base address in iommu mapping. Don't you need ~PAGE_MASK for it to work like that? The & with 0x00000000ffffffff shouldn't be needed either, certainly not with all the leading zeros. > > > + pci_write_msi_msg(irq, &msg); > > > + } > > > + > > > vdev->ctx[vector].trigger = trigger; > > > > > > return 0; > > > > > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogQWxleCBXaWxsaWFtc29u IFttYWlsdG86YWxleC53aWxsaWFtc29uQHJlZGhhdC5jb21dDQo+IFNlbnQ6IFR1ZXNkYXksIE9j dG9iZXIgMDYsIDIwMTUgNDoxNSBBTQ0KPiBUbzogQmh1c2hhbiBCaGFyYXQtUjY1Nzc3IDxCaGFy YXQuQmh1c2hhbkBmcmVlc2NhbGUuY29tPg0KPiBDYzoga3ZtYXJtQGxpc3RzLmNzLmNvbHVtYmlh LmVkdTsga3ZtQHZnZXIua2VybmVsLm9yZzsNCj4gY2hyaXN0b2ZmZXIuZGFsbEBsaW5hcm8ub3Jn OyBlcmljLmF1Z2VyQGxpbmFyby5vcmc7IHByYW5hdmt1bWFyQGxpbmFyby5vcmc7DQo+IG1hcmMu enluZ2llckBhcm0uY29tOyB3aWxsLmRlYWNvbkBhcm0uY29tDQo+IFN1YmplY3Q6IFJlOiBbUkZD IFBBVENIIDUvNl0gdmZpby1wY2k6IENyZWF0ZSBpb21tdSBtYXBwaW5nIGZvciBtc2kNCj4gaW50 ZXJydXB0DQo+IA0KPiBPbiBNb24sIDIwMTUtMTAtMDUgYXQgMDc6MjAgKzAwMDAsIEJodXNoYW4g QmhhcmF0IHdyb3RlOg0KPiA+DQo+ID4NCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0t DQo+ID4gPiBGcm9tOiBBbGV4IFdpbGxpYW1zb24gW21haWx0bzphbGV4LndpbGxpYW1zb25AcmVk aGF0LmNvbV0NCj4gPiA+IFNlbnQ6IFNhdHVyZGF5LCBPY3RvYmVyIDAzLCAyMDE1IDQ6MTcgQU0N Cj4gPiA+IFRvOiBCaHVzaGFuIEJoYXJhdC1SNjU3NzcgPEJoYXJhdC5CaHVzaGFuQGZyZWVzY2Fs ZS5jb20+DQo+ID4gPiBDYzoga3ZtYXJtQGxpc3RzLmNzLmNvbHVtYmlhLmVkdTsga3ZtQHZnZXIu a2VybmVsLm9yZzsNCj4gPiA+IGNocmlzdG9mZmVyLmRhbGxAbGluYXJvLm9yZzsgZXJpYy5hdWdl ckBsaW5hcm8ub3JnOw0KPiA+ID4gcHJhbmF2a3VtYXJAbGluYXJvLm9yZzsgbWFyYy56eW5naWVy QGFybS5jb207IHdpbGwuZGVhY29uQGFybS5jb20NCj4gPiA+IFN1YmplY3Q6IFJlOiBbUkZDIFBB VENIIDUvNl0gdmZpby1wY2k6IENyZWF0ZSBpb21tdSBtYXBwaW5nIGZvciBtc2kNCj4gPiA+IGlu dGVycnVwdA0KPiA+ID4NCj4gPiA+IE9uIFdlZCwgMjAxNS0wOS0zMCBhdCAyMDoyNiArMDUzMCwg QmhhcmF0IEJodXNoYW4gd3JvdGU6DQo+ID4gPiA+IEFuIE1TSS1hZGRyZXNzIGlzIGFsbG9jYXRl ZCBhbmQgcHJvZ3JhbW1lZCBpbiBwY2llIGRldmljZSBkdXJpbmcNCj4gPiA+ID4gaW50ZXJydXB0 IGNvbmZpZ3VyYXRpb24uIE5vdyBmb3IgYSBwYXNzLXRocm91Z2ggZGV2aWNlLCB0cnkgdG8NCj4g PiA+ID4gY3JlYXRlIHRoZSBpb21tdSBtYXBwaW5nIGZvciB0aGlzIGFsbG9jdGVkL3Byb2dyYW1t ZWQgbXNpLWFkZHJlc3MuDQo+ID4gPiA+IElmIHRoZSBpb21tdSBtYXBwaW5nIGlzIGNyZWF0ZWQg YW5kIHRoZSBtc2kgYWRkcmVzcyBwcm9ncmFtbWVkIGluDQo+ID4gPiA+IHRoZSBwY2llIGRldmlj ZSBpcyBkaWZmZXJlbnQgZnJvbSBtc2ktaW92YSBhcyBwZXIgaW9tbXUNCj4gPiA+ID4gcHJvZ3Jh bW1pbmcgdGhlbiByZWNvbmZpZ3VyZSB0aGUgcGNpIGRldmljZSB0byB1c2UgbXNpLWlvdmEgYXMg bXNpDQo+IGFkZHJlc3MuDQo+ID4gPiA+DQo+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IEJoYXJhdCBC aHVzaGFuIDxCaGFyYXQuQmh1c2hhbkBmcmVlc2NhbGUuY29tPg0KPiA+ID4gPiAtLS0NCj4gPiA+ ID4gIGRyaXZlcnMvdmZpby9wY2kvdmZpb19wY2lfaW50cnMuYyB8IDM2DQo+ID4gPiA+ICsrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKystLQ0KPiA+ID4gPiAgMSBmaWxlIGNoYW5nZWQs IDM0IGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pDQo+ID4gPiA+DQo+ID4gPiA+IGRpZmYg LS1naXQgYS9kcml2ZXJzL3ZmaW8vcGNpL3ZmaW9fcGNpX2ludHJzLmMNCj4gPiA+ID4gYi9kcml2 ZXJzL3ZmaW8vcGNpL3ZmaW9fcGNpX2ludHJzLmMNCj4gPiA+ID4gaW5kZXggMWY1NzdiNC4uYzk2 OTBhZiAxMDA2NDQNCj4gPiA+ID4gLS0tIGEvZHJpdmVycy92ZmlvL3BjaS92ZmlvX3BjaV9pbnRy cy5jDQo+ID4gPiA+ICsrKyBiL2RyaXZlcnMvdmZpby9wY2kvdmZpb19wY2lfaW50cnMuYw0KPiA+ ID4gPiBAQCAtMzEyLDEzICszMTIsMjMgQEAgc3RhdGljIGludCB2ZmlvX21zaV9zZXRfdmVjdG9y X3NpZ25hbChzdHJ1Y3QNCj4gPiA+IHZmaW9fcGNpX2RldmljZSAqdmRldiwNCj4gPiA+ID4gIAlp bnQgaXJxID0gbXNpeCA/IHZkZXYtPm1zaXhbdmVjdG9yXS52ZWN0b3IgOiBwZGV2LT5pcnEgKyB2 ZWN0b3I7DQo+ID4gPiA+ICAJY2hhciAqbmFtZSA9IG1zaXggPyAidmZpby1tc2l4IiA6ICJ2Zmlv LW1zaSI7DQo+ID4gPiA+ICAJc3RydWN0IGV2ZW50ZmRfY3R4ICp0cmlnZ2VyOw0KPiA+ID4gPiAr CXN0cnVjdCBtc2lfbXNnIG1zZzsNCj4gPiA+ID4gKwlzdHJ1Y3QgdmZpb19kZXZpY2UgKmRldmlj ZTsNCj4gPiA+ID4gKwl1aW50NjRfdCBtc2lfYWRkciwgbXNpX2lvdmE7DQo+ID4gPiA+ICAJaW50 IHJldDsNCj4gPiA+ID4NCj4gPiA+ID4gIAlpZiAodmVjdG9yID49IHZkZXYtPm51bV9jdHgpDQo+ ID4gPiA+ICAJCXJldHVybiAtRUlOVkFMOw0KPiA+ID4gPg0KPiA+ID4gPiArCWRldmljZSA9IHZm aW9fZGV2aWNlX2dldF9mcm9tX2RldigmcGRldi0+ZGV2KTsNCj4gPiA+DQo+ID4gPiBIYXZlIHlv dSBsb29rZWQgYXQgdGhpcyBmdW5jdGlvbj8gIEkgZG9uJ3QgdGhpbmsgd2Ugd2FudCB0byBiZSBk b2luZw0KPiA+ID4gdGhhdCBldmVyeSB0aW1lIHdlIHdhbnQgdG8gcG9rZSB0aGUgaW50ZXJydXB0 IGNvbmZpZ3VyYXRpb24uDQo+ID4NCj4gPiBJIGFtIHRyeWluZyB0byBkZXNjcmliZSB3aGF0IEkg dW5kZXJzdG9vZCwgYSBkZXZpY2UgY2FuIGhhdmUgbWFueQ0KPiBpbnRlcnJ1cHRzIGFuZCB3ZSBz aG91bGQgc2V0dXAgaW9tbXUgb25seSBvbmNlLCB3aGVuIGNhbGxlZCBmb3IgdGhlIGZpcnN0DQo+ IHRpbWUgdG8gZW5hYmxlL3NldHVwIGludGVycnVwdC4NCj4gPiBTaW1pbGFybHkgd2hlbiBkaXNh YmxpbmcgdGhlIGludGVycnVwdCB3ZSBzaG91bGQgaW9tbXUtdW5tYXAgd2hlbg0KPiA+IGNhbGxl ZCBmb3IgdGhlIGxhc3QgZW5hYmxlZCBpbnRlcnJ1cHQgZm9yIHRoYXQgZGV2aWNlLiBOb3cgd2l0 aCB0aGlzDQo+ID4gdW5kZXJzdGFuZGluZywgc2hvdWxkIEkgbW92ZSB0aGlzIG1hcC11bm1hcCB0 byBzZXBhcmF0ZSBmdW5jdGlvbnMgYW5kDQo+ID4gY2FsbCB0aGVtIGZyb20gdmZpb19tc2lfc2V0 X2Jsb2NrKCkgcmF0aGVyIHRoYW4gaW4NCj4gPiB2ZmlvX21zaV9zZXRfdmVjdG9yX3NpZ25hbCgp DQo+IA0KPiBJbnRlcnJ1cHRzIGNhbiBiZSBzZXR1cCBhbmQgdG9ybiBkb3duIGF0IGFueSB0aW1l IGFuZCBJIGRvbid0IHNlZSBob3cgb25lDQo+IGZ1bmN0aW9uIG9yIHRoZSBvdGhlciBtYWtlcyBt dWNoIGRpZmZlcmVuY2UuDQo+IHZmaW9fZGV2aWNlX2dldF9mcm9tX2RldigpIGlzIGVub3VnaCBv dmVyaGVhZCB0aGF0IHRoZSBkYXRhIHdlIG5lZWQNCj4gc2hvdWxkIGJlIGNhY2hlZCBpZiB3ZSdy ZSBnb2luZyB0byBjYWxsIGl0IHdpdGggc29tZSByZWd1bGFyaXR5LiAgTWF5YmUNCj4gdmZpb19p b21tdV9kcml2ZXJfb3BzLm9wZW4oKSBzaG91bGQgYmUgY2FsbGVkIHdpdGggYSBwb2ludGVyIHRv IHRoZQ0KPiB2ZmlvX2RldmljZS4uLiBvciB0aGUgdmZpb19ncm91cC4NCg0KdmZpb19pb21tdV9k cml2ZXJfb3BzLm9wZW4oKSA/IG9yIGRvIHlvdSBtZWFuIHZmaW9fcGNpX29wZW4oKSBzaG91bGQg YmUgY2FsbGVkIHdpdGggdmZpb19kZXZpY2Ugb3IgdmZpb19ncm91cCwgYW5kIHdlIHdpbGwgY2Fj aGUgdGhhdCBpbiB2ZmlvX3BjaV9kZXZpY2UgPw0KDQo+IA0KPiA+ID4gIEFsc28gbm90ZSB0aGF0 DQo+ID4gPiBJT01NVSBtYXBwaW5ncyBkb24ndCBvcGVyYXRlIG9uIGRldmljZXMsIGJ1dCBncm91 cHMsIHNvIG1heWJlIHdlDQo+ID4gPiB3YW50IHRvIHBhc3MgdGhlIGdyb3VwLg0KPiA+DQo+ID4g WWVzLCBpdCBvcGVyYXRlcyBvbiBncm91cC4gSSBoZXNpdGF0ZWQgdG8gYWRkIGFuIEFQSSB0byBn ZXQgZ3JvdXAuIERvIHlvdQ0KPiBzdWdnZXN0IHRvIHRoYXQgaXQgaXMgb2sgdG8gYWRkIEFQSSB0 byBnZXQgZ3JvdXAgZnJvbSBkZXZpY2UuDQo+IA0KPiBObywgdGhlIGFib3ZlIHN1Z2dlc3Rpb24g aXMgcHJvYmFibHkgYmV0dGVyLg0KPiANCj4gPiA+DQo+ID4gPiA+ICsJaWYgKGRldmljZSA9PSBO VUxMKQ0KPiA+ID4gPiArCQlyZXR1cm4gLUVJTlZBTDsNCj4gPiA+DQo+ID4gPiBUaGlzIHdvdWxk IGJlIGEgbGVnaXRpbWF0ZSBCVUdfT04oIWRldmljZSkNCj4gPiA+DQo+ID4gPiA+ICsNCj4gPiA+ ID4gIAlpZiAodmRldi0+Y3R4W3ZlY3Rvcl0udHJpZ2dlcikgew0KPiA+ID4gPiAgCQlmcmVlX2ly cShpcnEsIHZkZXYtPmN0eFt2ZWN0b3JdLnRyaWdnZXIpOw0KPiA+ID4gPiArCQlnZXRfY2FjaGVk X21zaV9tc2coaXJxLCAmbXNnKTsNCj4gPiA+ID4gKwkJbXNpX2lvdmEgPSAoKHU2NCltc2cuYWRk cmVzc19oaSA8PCAzMikgfCBtc2cuYWRkcmVzc19sbzsNCj4gPiA+ID4gKwkJdmZpb19kZXZpY2Vf dW5tYXBfbXNpKGRldmljZSwgbXNpX2lvdmEsIFBBR0VfU0laRSk7DQo+ID4gPiA+ICAJCWtmcmVl KHZkZXYtPmN0eFt2ZWN0b3JdLm5hbWUpOw0KPiA+ID4gPiAgCQlldmVudGZkX2N0eF9wdXQodmRl di0+Y3R4W3ZlY3Rvcl0udHJpZ2dlcik7DQo+ID4gPiA+ICAJCXZkZXYtPmN0eFt2ZWN0b3JdLnRy aWdnZXIgPSBOVUxMOyBAQCAtMzQ2LDEyICszNTYsMTEgQEANCj4gc3RhdGljDQo+ID4gPiA+IGlu dCB2ZmlvX21zaV9zZXRfdmVjdG9yX3NpZ25hbChzdHJ1Y3QNCj4gPiA+IHZmaW9fcGNpX2Rldmlj ZSAqdmRldiwNCj4gPiA+ID4gIAkgKiBjYWNoZWQgdmFsdWUgb2YgdGhlIG1lc3NhZ2UgcHJpb3Ig dG8gZW5hYmxpbmcuDQo+ID4gPiA+ICAJICovDQo+ID4gPiA+ICAJaWYgKG1zaXgpIHsNCj4gPiA+ ID4gLQkJc3RydWN0IG1zaV9tc2cgbXNnOw0KPiA+ID4gPiAtDQo+ID4gPiA+ICAJCWdldF9jYWNo ZWRfbXNpX21zZyhpcnEsICZtc2cpOw0KPiA+ID4gPiAgCQlwY2lfd3JpdGVfbXNpX21zZyhpcnEs ICZtc2cpOw0KPiA+ID4gPiAgCX0NCj4gPiA+ID4NCj4gPiA+ID4gKw0KPiA+ID4NCj4gPiA+IGdy YXR1aXRvdXMgbmV3bGluZQ0KPiA+ID4NCj4gPiA+ID4gIAlyZXQgPSByZXF1ZXN0X2lycShpcnEs IHZmaW9fbXNpaGFuZGxlciwgMCwNCj4gPiA+ID4gIAkJCSAgdmRldi0+Y3R4W3ZlY3Rvcl0ubmFt ZSwgdHJpZ2dlcik7DQo+ID4gPiA+ICAJaWYgKHJldCkgew0KPiA+ID4gPiBAQCAtMzYwLDYgKzM2 OSwyOSBAQCBzdGF0aWMgaW50IHZmaW9fbXNpX3NldF92ZWN0b3Jfc2lnbmFsKHN0cnVjdA0KPiA+ ID4gdmZpb19wY2lfZGV2aWNlICp2ZGV2LA0KPiA+ID4gPiAgCQlyZXR1cm4gcmV0Ow0KPiA+ID4g PiAgCX0NCj4gPiA+ID4NCj4gPiA+ID4gKwkvKiBSZS1wcm9ncmFtIHRoZSBuZXctaW92YSBpbiBw Y2ktZGV2aWNlIGluIGNhc2UgdGhlcmUgaXMNCj4gPiA+ID4gKwkgKiBkaWZmZXJlbnQgaW9tbXUt bWFwcGluZyBjcmVhdGVkIGZvciBwcm9ncmFtbWVkIG1zaS1hZGRyZXNzLg0KPiA+ID4gPiArCSAq Lw0KPiA+ID4gPiArCWdldF9jYWNoZWRfbXNpX21zZyhpcnEsICZtc2cpOw0KPiA+ID4gPiArCW1z aV9pb3ZhID0gMDsNCj4gPiA+ID4gKwltc2lfYWRkciA9ICh1NjQpKG1zZy5hZGRyZXNzX2hpKSA8 PCAzMiB8ICh1NjQpKG1zZy5hZGRyZXNzX2xvKTsNCj4gPiA+ID4gKwlyZXQgPSB2ZmlvX2Rldmlj ZV9tYXBfbXNpKGRldmljZSwgbXNpX2FkZHIsIFBBR0VfU0laRSwNCj4gPiA+ICZtc2lfaW92YSk7 DQo+ID4gPiA+ICsJaWYgKHJldCkgew0KPiA+ID4gPiArCQlmcmVlX2lycShpcnEsIHZkZXYtPmN0 eFt2ZWN0b3JdLnRyaWdnZXIpOw0KPiA+ID4gPiArCQlrZnJlZSh2ZGV2LT5jdHhbdmVjdG9yXS5u YW1lKTsNCj4gPiA+ID4gKwkJZXZlbnRmZF9jdHhfcHV0KHRyaWdnZXIpOw0KPiA+ID4gPiArCQly ZXR1cm4gcmV0Ow0KPiA+ID4gPiArCX0NCj4gPiA+ID4gKw0KPiA+ID4gPiArCS8qIFJlcHJvZ3Jh bSBvbmx5IGlmIGlvbW11LW1hcHBlZCBpb3ZhIGlzIGRpZmZlcmVudCBmcm9tIG1zaS0NCj4gPiA+ IGFkZHJlc3MgKi8NCj4gPiA+ID4gKwlpZiAobXNpX2lvdmEgJiYgKG1zaV9pb3ZhICE9IG1zaV9h ZGRyKSkgew0KPiA+ID4gPiArCQltc2cuYWRkcmVzc19oaSA9ICh1MzIpKG1zaV9pb3ZhID4+IDMy KTsNCj4gPiA+ID4gKwkJLyogS2VlcCBMb3dlciBiaXRzIGZyb20gb3JpZ2luYWwgbXNpIG1lc3Nh Z2UgYWRkcmVzcyAqLw0KPiA+ID4gPiArCQltc2cuYWRkcmVzc19sbyAmPSBQQUdFX01BU0s7DQo+ ID4gPiA+ICsJCW1zZy5hZGRyZXNzX2xvIHw9ICh1MzIpKG1zaV9pb3ZhICYgMHgwMDAwMDAwMGZm ZmZmZmZmKTsNCj4gPiA+DQo+ID4gPiBTZWVtcyBsaWtlIHlvdSdyZSBtYWtpbmcgc29tZSBhc3N1 bXB0aW9ucyBoZXJlIHRoYXQgYXJlIGRlcGVuZGVudCBvbg0KPiA+ID4gdGhlIGFyY2hpdGVjdHVy ZSBhbmQgbWF5YmUgdGhlIHBsYXRmb3JtLg0KPiA+DQo+ID4gV2hhdCBJIHRyaWVkIGlzIHRvIG1h cCB0aGUgbXNpIHBhZ2Ugd2l0aCBkaWZmZXJlbnQgaW92YSwgd2hpY2ggaXMgcGFnZSBzaXplDQo+ IGFsaWduZWQuIEJ1dCB0aGUgb2Zmc2V0IHdpdGhpbiB0aGUgcGFnZSB3aWxsIHJlbWFpbiBzYW1l Lg0KPiA+IEZvciBleGFtcGxlLCBvcmlnaW5hbCBtc2kgYWRkcmVzcyB3YXMgMHgwNjAzXzAwNDAg YW5kIHdlIGhhdmUgYSByZXNlcnZlZA0KPiByZWdpb24gYXQgMHhmMDAwXzAwMDAuIFNvIGlvbW11 IG1hcHBpbmcgaXMgY3JlYXRlZCBmb3IgMHhmMDAwXzAwMDANCj4gPT4weDA2MDBfMzAwMCBvZiBz aXplIDB4MTAwMC4NCj4gPg0KPiA+IFNvIHRoZSBuZXcgYWRkcmVzcyB0byBiZSBwcm9ncmFtbWVk IGluIGRldmljZSBpcyAweGYwMDBfMDA0MCwgb2Zmc2V0DQo+IDB4NDAgYWRkZWQgdG8gYmFzZSBh ZGRyZXNzIGluIGlvbW11IG1hcHBpbmcuDQo+IA0KPiBEb24ndCB5b3UgbmVlZCB+UEFHRV9NQVNL IGZvciBpdCB0byB3b3JrIGxpa2UgdGhhdD8gIFRoZSAmIHdpdGgNCj4gMHgwMDAwMDAwMGZmZmZm ZmZmIHNob3VsZG4ndCBiZSBuZWVkZWQgZWl0aGVyLCBjZXJ0YWlubHkgbm90IHdpdGggYWxsIHRo ZQ0KPiBsZWFkaW5nIHplcm9zLg0KDQpZZXMsIEkgdGhpbmsgflBBR0VfTVNLIGNhbiBiZSB1c2Vk Lg0KDQpUaGFua3MNCi1CaGFyYXQNCg0KPiANCj4gPiA+ID4gKwkJcGNpX3dyaXRlX21zaV9tc2co aXJxLCAmbXNnKTsNCj4gPiA+ID4gKwl9DQo+ID4gPiA+ICsNCj4gPiA+ID4gIAl2ZGV2LT5jdHhb dmVjdG9yXS50cmlnZ2VyID0gdHJpZ2dlcjsNCj4gPiA+ID4NCj4gPiA+ID4gIAlyZXR1cm4gMDsN Cj4gPiA+DQo+ID4gPg0KPiA+DQo+IA0KPiANCg0K -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 2015-10-06 at 08:32 +0000, Bhushan Bharat wrote: > > > > -----Original Message----- > > From: Alex Williamson [mailto:alex.williamson@redhat.com] > > Sent: Tuesday, October 06, 2015 4:15 AM > > To: Bhushan Bharat-R65777 <Bharat.Bhushan@freescale.com> > > Cc: kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; > > christoffer.dall@linaro.org; eric.auger@linaro.org; pranavkumar@linaro.org; > > marc.zyngier@arm.com; will.deacon@arm.com > > Subject: Re: [RFC PATCH 5/6] vfio-pci: Create iommu mapping for msi > > interrupt > > > > On Mon, 2015-10-05 at 07:20 +0000, Bhushan Bharat wrote: > > > > > > > > > > -----Original Message----- > > > > From: Alex Williamson [mailto:alex.williamson@redhat.com] > > > > Sent: Saturday, October 03, 2015 4:17 AM > > > > To: Bhushan Bharat-R65777 <Bharat.Bhushan@freescale.com> > > > > Cc: kvmarm@lists.cs.columbia.edu; kvm@vger.kernel.org; > > > > christoffer.dall@linaro.org; eric.auger@linaro.org; > > > > pranavkumar@linaro.org; marc.zyngier@arm.com; will.deacon@arm.com > > > > Subject: Re: [RFC PATCH 5/6] vfio-pci: Create iommu mapping for msi > > > > interrupt > > > > > > > > On Wed, 2015-09-30 at 20:26 +0530, Bharat Bhushan wrote: > > > > > An MSI-address is allocated and programmed in pcie device during > > > > > interrupt configuration. Now for a pass-through device, try to > > > > > create the iommu mapping for this allocted/programmed msi-address. > > > > > If the iommu mapping is created and the msi address programmed in > > > > > the pcie device is different from msi-iova as per iommu > > > > > programming then reconfigure the pci device to use msi-iova as msi > > address. > > > > > > > > > > Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com> > > > > > --- > > > > > drivers/vfio/pci/vfio_pci_intrs.c | 36 > > > > > ++++++++++++++++++++++++++++++++++-- > > > > > 1 file changed, 34 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/drivers/vfio/pci/vfio_pci_intrs.c > > > > > b/drivers/vfio/pci/vfio_pci_intrs.c > > > > > index 1f577b4..c9690af 100644 > > > > > --- a/drivers/vfio/pci/vfio_pci_intrs.c > > > > > +++ b/drivers/vfio/pci/vfio_pci_intrs.c > > > > > @@ -312,13 +312,23 @@ static int vfio_msi_set_vector_signal(struct > > > > vfio_pci_device *vdev, > > > > > int irq = msix ? vdev->msix[vector].vector : pdev->irq + vector; > > > > > char *name = msix ? "vfio-msix" : "vfio-msi"; > > > > > struct eventfd_ctx *trigger; > > > > > + struct msi_msg msg; > > > > > + struct vfio_device *device; > > > > > + uint64_t msi_addr, msi_iova; > > > > > int ret; > > > > > > > > > > if (vector >= vdev->num_ctx) > > > > > return -EINVAL; > > > > > > > > > > + device = vfio_device_get_from_dev(&pdev->dev); > > > > > > > > Have you looked at this function? I don't think we want to be doing > > > > that every time we want to poke the interrupt configuration. > > > > > > I am trying to describe what I understood, a device can have many > > interrupts and we should setup iommu only once, when called for the first > > time to enable/setup interrupt. > > > Similarly when disabling the interrupt we should iommu-unmap when > > > called for the last enabled interrupt for that device. Now with this > > > understanding, should I move this map-unmap to separate functions and > > > call them from vfio_msi_set_block() rather than in > > > vfio_msi_set_vector_signal() > > > > Interrupts can be setup and torn down at any time and I don't see how one > > function or the other makes much difference. > > vfio_device_get_from_dev() is enough overhead that the data we need > > should be cached if we're going to call it with some regularity. Maybe > > vfio_iommu_driver_ops.open() should be called with a pointer to the > > vfio_device... or the vfio_group. > > vfio_iommu_driver_ops.open() ? or do you mean vfio_pci_open() should be called with vfio_device or vfio_group, and we will cache that in vfio_pci_device ? vfio_pci_open() is an implementation of vfio_iommu_driver_ops.open(). The internal API between vfio and vfio bus drivers would need to have a parameter added. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 1f577b4..c9690af 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -312,13 +312,23 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, int irq = msix ? vdev->msix[vector].vector : pdev->irq + vector; char *name = msix ? "vfio-msix" : "vfio-msi"; struct eventfd_ctx *trigger; + struct msi_msg msg; + struct vfio_device *device; + uint64_t msi_addr, msi_iova; int ret; if (vector >= vdev->num_ctx) return -EINVAL; + device = vfio_device_get_from_dev(&pdev->dev); + if (device == NULL) + return -EINVAL; + if (vdev->ctx[vector].trigger) { free_irq(irq, vdev->ctx[vector].trigger); + get_cached_msi_msg(irq, &msg); + msi_iova = ((u64)msg.address_hi << 32) | msg.address_lo; + vfio_device_unmap_msi(device, msi_iova, PAGE_SIZE); kfree(vdev->ctx[vector].name); eventfd_ctx_put(vdev->ctx[vector].trigger); vdev->ctx[vector].trigger = NULL; @@ -346,12 +356,11 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, * cached value of the message prior to enabling. */ if (msix) { - struct msi_msg msg; - get_cached_msi_msg(irq, &msg); pci_write_msi_msg(irq, &msg); } + ret = request_irq(irq, vfio_msihandler, 0, vdev->ctx[vector].name, trigger); if (ret) { @@ -360,6 +369,29 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, return ret; } + /* Re-program the new-iova in pci-device in case there is + * different iommu-mapping created for programmed msi-address. + */ + get_cached_msi_msg(irq, &msg); + msi_iova = 0; + msi_addr = (u64)(msg.address_hi) << 32 | (u64)(msg.address_lo); + ret = vfio_device_map_msi(device, msi_addr, PAGE_SIZE, &msi_iova); + if (ret) { + free_irq(irq, vdev->ctx[vector].trigger); + kfree(vdev->ctx[vector].name); + eventfd_ctx_put(trigger); + return ret; + } + + /* Reprogram only if iommu-mapped iova is different from msi-address */ + if (msi_iova && (msi_iova != msi_addr)) { + msg.address_hi = (u32)(msi_iova >> 32); + /* Keep Lower bits from original msi message address */ + msg.address_lo &= PAGE_MASK; + msg.address_lo |= (u32)(msi_iova & 0x00000000ffffffff); + pci_write_msi_msg(irq, &msg); + } + vdev->ctx[vector].trigger = trigger; return 0;
An MSI-address is allocated and programmed in pcie device during interrupt configuration. Now for a pass-through device, try to create the iommu mapping for this allocted/programmed msi-address. If the iommu mapping is created and the msi address programmed in the pcie device is different from msi-iova as per iommu programming then reconfigure the pci device to use msi-iova as msi address. Signed-off-by: Bharat Bhushan <Bharat.Bhushan@freescale.com> --- drivers/vfio/pci/vfio_pci_intrs.c | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-)