[RFC,0/8] s390x/pci: Fixing s390 vfio-pci ISM support

Message

Matthew Rosato Dec. 9, 2020, 8:34 p.m. UTC

Today, ISM devices are completely disallowed for vfio-pci passthrough as
QEMU rejects the device due to an (inappropriate) MSI-X check.  Removing
this fence, however, reveals additional deficiencies in the s390x PCI
interception layer that prevent ISM devices from working correctly.
Namely, ISM block write operations have particular requirements in regards
to the alignment, size and order of writes performed that cannot be
guaranteed when breaking up write operations through the typical
vfio_pci_bar_rw paths. Furthermore, ISM requires that legacy/non-MIO
s390 PCI instructions are used, which is also not guaranteed when the I/O
is passed through the typical userspace channels.

This patchset provides a set of fixes related to enabling ISM device
passthrough and includes patches to enable use of a new vfio region that
will allow s390x PCI pass-through devices to perform s390 PCI instructions
in such a way that the same instruction issued on the guest is re-issued
on the host.

The region is intended for use with ISM devices specifically, as they do
not implement MSI-X and thus transferring I/O in this manner will not
interfere with vfio-pci MSI-X masking.  If someone can think of a good way
to also make this MSI-X compatible (and thus usable by other device types
besides ISM), I'm open to suggestions...

Associated kernel patchset:
https://lkml.org/lkml/2020/12/9/1059

Matthew Rosato (8):
  linux-headers: update against 5.10-rc7
  s390x/pci: MSI-X isn't strictly required for passthrough
  s390x/pci: fix pcistb length
  s390x/pci: Introduce the ZpciOps structure
  s390x/pci: Fix memory_region_access_valid call
  s390x/pci: Handle devices that support relaxed alignment
  s390x/pci: PCISTB via the vfio zPCI I/O region
  s390x/pci: PCILG via the vfio zPCI I/O region

 hw/s390x/s390-pci-bus.c                            |  32 ++-
 hw/s390x/s390-pci-inst.c                           | 289 ++++++++++++++-------
 hw/s390x/s390-pci-vfio.c                           | 164 ++++++++++++
 include/hw/s390x/s390-pci-bus.h                    |  24 ++
 include/hw/s390x/s390-pci-clp.h                    |   1 +
 include/hw/s390x/s390-pci-inst.h                   |   3 +
 include/hw/s390x/s390-pci-vfio.h                   |  23 ++
 include/standard-headers/asm-x86/kvm_para.h        |   1 +
 .../infiniband/hw/vmw_pvrdma/pvrdma_verbs.h        |   2 +-
 include/standard-headers/linux/vhost_types.h       |   9 +
 linux-headers/linux/vfio.h                         |   4 +
 linux-headers/linux/vfio_zdev.h                    |  33 +++
 linux-headers/linux/vhost.h                        |   4 +
 13 files changed, 486 insertions(+), 103 deletions(-)