| Message ID | 1542904555-1136-1-git-send-email-pmorel@linux.ibm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | s390x/vfio: VFIO-AP interrupt control interception | expand |
On Thu, 22 Nov 2018 17:35:49 +0100 Pierre Morel <pmorel@linux.ibm.com> wrote: > This series has 3 different type of patches: > > The first two: > s390x/vfio: ap: Finding the AP bridge > s390x/vfio: ap: Use the APdevice as a child of the APBus > > Are dealing with the QEMU Object Model and how we retrieve the > AP devices from instruction interception. > A lifting of the AP bridge, bus and device was necessary to > ease the process and allow future extensions. > > The third one is a place holder to ease reviewing: > s390x/vfio: ap: Linux uapi VFIO place holder > > The last three are really dealing with PQAP/AQIC instruction > interception and associate IOCTL calls to the VFIO AP device > driver. > s390x/cpumodel: Set up CPU model for AQIC interception > s390x/vfio: ap: Definition for AP Adapter type > s390x/vfio: ap: Implementing AP Queue Interrupt Control > > The S390 APQP/AQIC instruction is intercepted by the host > to configure the AP queues interruption redirection. > It retrieves the ISC used by the host and the one used > by the guest and setup the indicator address. > > This patch series > - define the AQIC feature in the cpumodel, > - extend the APDevice type for per card and queue interrupt handling, > - intercept the APQP/AQIC instruction, uses the S390 adapter interface > to setup the adapter > - and use a VFIO ioctl to let the VFIO-AP driver handle the host > instruction associated with the intercepted guest instruction. > > This patch serie can be tested with the Linux/KVM patch series > for the VFIO-AP driver: "s390: vfio: ap: Using GISA for AP Interrupt" > Sorry for raising concern this late, I hope it's better late than never. I have strong doubts that handling PQAP/AQCI via userspace is worth the effort. IMHO we could do what we have to do on AQCI in kernel iff the ap is done SIE interpreted, the appropriate feature is presented to the guest, and the queue in question belongs to the given guest. Or am I wrong? I do understand that doing it like this *may* end up being beneficial *if* we decide to do some sort of ap virtualization in QEMU. But I don't see it coming in the foreseeable future, and for ap virtualization we would need a solution for making the host do an NQAP and an DQAP on behalf of the guest/emulator, and not only to do the same for PQAP/QCI. In my understanding, with this, we would end up with an infrastructure that only makes sense in a perspective of some 'future features' which may never come to existence. What I ask for is, please, let us examine the other option. Regards, Halil > > Pierre Morel (6): > s390x/vfio: ap: Finding the AP bridge > s390x/vfio: ap: Use the APdevice as a child of the APBus > s390x/vfio: ap: Linux uapi VFIO place holder > s390x/cpumodel: Set up CPU model for AQIC interception > s390x/vfio: ap: Definition for AP Adapter type > s390x/vfio: ap: Implementing AP Queue Interrupt Control > > hw/s390x/ap-bridge.c | 12 ++++ > hw/s390x/ap-device.c | 22 +++++++ > hw/vfio/ap.c | 131 +++++++++++++++++++++++++++++++++++++--- > include/hw/s390x/ap-bridge.h | 1 + > include/hw/s390x/ap-device.h | 65 ++++++++++++++++++++ > include/hw/s390x/css.h | 1 + > linux-headers/linux/vfio.h | 25 ++++++++ > target/s390x/cpu_features.c | 1 + > target/s390x/cpu_features_def.h | 1 + > target/s390x/cpu_models.c | 1 + > target/s390x/gen-features.c | 1 + > target/s390x/kvm.c | 31 ++++++++++ > 12 files changed, 282 insertions(+), 10 deletions(-) >
On 29/11/2018 16:55, Halil Pasic wrote: > On Thu, 22 Nov 2018 17:35:49 +0100 > Pierre Morel <pmorel@linux.ibm.com> wrote: > >> This series has 3 different type of patches: >> >> The first two: >> s390x/vfio: ap: Finding the AP bridge >> s390x/vfio: ap: Use the APdevice as a child of the APBus >> >> Are dealing with the QEMU Object Model and how we retrieve the >> AP devices from instruction interception. >> A lifting of the AP bridge, bus and device was necessary to >> ease the process and allow future extensions. >> >> The third one is a place holder to ease reviewing: >> s390x/vfio: ap: Linux uapi VFIO place holder >> >> The last three are really dealing with PQAP/AQIC instruction >> interception and associate IOCTL calls to the VFIO AP device >> driver. >> s390x/cpumodel: Set up CPU model for AQIC interception >> s390x/vfio: ap: Definition for AP Adapter type >> s390x/vfio: ap: Implementing AP Queue Interrupt Control >> >> The S390 APQP/AQIC instruction is intercepted by the host >> to configure the AP queues interruption redirection. >> It retrieves the ISC used by the host and the one used >> by the guest and setup the indicator address. >> >> This patch series >> - define the AQIC feature in the cpumodel, >> - extend the APDevice type for per card and queue interrupt handling, >> - intercept the APQP/AQIC instruction, uses the S390 adapter interface >> to setup the adapter >> - and use a VFIO ioctl to let the VFIO-AP driver handle the host >> instruction associated with the intercepted guest instruction. >> >> This patch serie can be tested with the Linux/KVM patch series >> for the VFIO-AP driver: "s390: vfio: ap: Using GISA for AP Interrupt" >> > > Sorry for raising concern this late, I hope it's better late than > never. > > I have strong doubts that handling PQAP/AQCI via userspace is worth > the effort. IMHO we could do what we have to do on AQCI in kernel > iff the ap is done SIE interpreted, the appropriate feature is presented > to the guest, and the queue in question belongs to the given guest. Or > am I wrong? > > I do understand that doing it like this *may* end up being beneficial > *if* we decide to do some sort of ap virtualization in QEMU. But I don't > see it coming in the foreseeable future, and for ap virtualization we > would need a solution for making the host do an NQAP and an DQAP on > behalf of the guest/emulator, and not only to do the same for PQAP/QCI. > > In my understanding, with this, we would end up with an infrastructure > that only makes sense in a perspective of some 'future features' which > may never come to existence. > > What I ask for is, please, let us examine the other option. > > Regards, > Halil > > As we discussed offline, I already began to implement prototypes for both options. This is a clear design choice. If we examine the pro and contra, I can list: 1- Pro kernel implementation of the PQAP/AQIC -> rapidity of the reaction Question: is this important? Answer: NO, Why: The PQAP/AQIC is rarely used by the AP driver of the guest. exclusively during RESET of the AP queue. I do not think we need a rapid reaction there. 2- Pro userland implementation of PQAP/AQIC -> standard implementation, already used by PCI, CCW Question: is it important? Answer: YES Why: like following the standard It is easily extend-able to other virtualization implementation like interception based VFIO and emulation There is no implementation which would be really more complicated as the other, for both we will need to introduce new pro APQN (queue) structures to hold the interrupt information (ISC, NIB), for both we will need to ping the NIB in memory. So as long as there are no other opinion against the design I presented here I will continue this way while considering the comments I got on this series. Regards, Pierre
On Fri, 30 Nov 2018 14:01:42 +0100 Pierre Morel <pmorel@linux.ibm.com> wrote: > On 29/11/2018 16:55, Halil Pasic wrote: > > On Thu, 22 Nov 2018 17:35:49 +0100 > > Pierre Morel <pmorel@linux.ibm.com> wrote: > > > >> This series has 3 different type of patches: > >> > >> The first two: > >> s390x/vfio: ap: Finding the AP bridge > >> s390x/vfio: ap: Use the APdevice as a child of the APBus > >> > >> Are dealing with the QEMU Object Model and how we retrieve the > >> AP devices from instruction interception. > >> A lifting of the AP bridge, bus and device was necessary to > >> ease the process and allow future extensions. > >> > >> The third one is a place holder to ease reviewing: > >> s390x/vfio: ap: Linux uapi VFIO place holder > >> > >> The last three are really dealing with PQAP/AQIC instruction > >> interception and associate IOCTL calls to the VFIO AP device > >> driver. > >> s390x/cpumodel: Set up CPU model for AQIC interception > >> s390x/vfio: ap: Definition for AP Adapter type > >> s390x/vfio: ap: Implementing AP Queue Interrupt Control > >> > >> The S390 APQP/AQIC instruction is intercepted by the host > >> to configure the AP queues interruption redirection. > >> It retrieves the ISC used by the host and the one used > >> by the guest and setup the indicator address. > >> > >> This patch series > >> - define the AQIC feature in the cpumodel, > >> - extend the APDevice type for per card and queue interrupt handling, > >> - intercept the APQP/AQIC instruction, uses the S390 adapter interface > >> to setup the adapter > >> - and use a VFIO ioctl to let the VFIO-AP driver handle the host > >> instruction associated with the intercepted guest instruction. > >> > >> This patch serie can be tested with the Linux/KVM patch series > >> for the VFIO-AP driver: "s390: vfio: ap: Using GISA for AP Interrupt" > >> > > > > Sorry for raising concern this late, I hope it's better late than > > never. > > > > I have strong doubts that handling PQAP/AQCI via userspace is worth > > the effort. IMHO we could do what we have to do on AQCI in kernel > > iff the ap is done SIE interpreted, the appropriate feature is presented > > to the guest, and the queue in question belongs to the given guest. Or > > am I wrong? > > > > I do understand that doing it like this *may* end up being beneficial > > *if* we decide to do some sort of ap virtualization in QEMU. But I don't > > see it coming in the foreseeable future, and for ap virtualization we > > would need a solution for making the host do an NQAP and an DQAP on > > behalf of the guest/emulator, and not only to do the same for PQAP/QCI. > > > > In my understanding, with this, we would end up with an infrastructure > > that only makes sense in a perspective of some 'future features' which > > may never come to existence. > > > > What I ask for is, please, let us examine the other option. > > > > Regards, > > Halil > > > > > > As we discussed offline, I already began to implement prototypes for > both options. > This is a clear design choice. > > If we examine the pro and contra, I can list: > > 1- Pro kernel implementation of the PQAP/AQIC > -> rapidity of the reaction > > Question: is this important? > Answer: NO, > Why: The PQAP/AQIC is rarely used by the AP driver of the guest. > exclusively during RESET of the AP queue. > I do not think we need a rapid reaction there. > > > 2- Pro userland implementation of PQAP/AQIC > -> standard implementation, already used by PCI, CCW > > Question: is it important? > Answer: YES > Why: like following the standard > It is easily extend-able to other virtualization implementation like > interception based VFIO and emulation > > > There is no implementation which would be really more complicated as the > other, for both we will need to introduce new pro APQN (queue) > structures to hold the interrupt information (ISC, NIB), for both we > will need to ping the NIB in memory. > > So as long as there are no other opinion against the design I presented > here I will continue this way while considering the comments I got on > this series. > I'm a bit confused. Your first sentence reads like you in a process of providing a kernel-heavy version. Your last sentence however reads like, based on the discussion following the first sentence, you decided to not explore, if the kernel-heavy variant (we still need a cpu model feature in QEMU) is simpler. Frankly, my feeling was that a kernel heavy implementation can be done with less lines of code (considering QEMU and Linux) and without introducing new ioctl interfaces. I may be wrong. You certainly seem to dispute that a kernel-heavy implementation is likely to be simpler. If you don't want to explore that option, I would like to ask you for your permission to do it myself if my time allows. Regards, Halil