| Message ID | cover.1611081587.git.jag.raman@oracle.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Initial support for multi-process Qemu | expand |
On Tue, Jan 19, 2021 at 03:28:17PM -0500, Jagannathan Raman wrote: > Hi, > > This is the v20 of the patchset. This version has the following changes: > > [PATCH v18 08/20] io: add qio_channel_readv_full_all_eof & > qio_channel_readv_full_all helpers > - Considers fds as part of the "data" for both > qio_channel_readv_full_all_eof() & > qio_channel_readv_full_all(). > - Addresses some of the error scenarios > which were not addressed in the previous revisions > > To touch upon the history of this project, we posted the Proof Of Concept > patches before the BoF session in 2018. Subsequently, we have posted 19 > versions on the qemu-devel mailing list. You can find them by following > the links below ([1] - [19]). Following people contributed to the design and > implementation of this project: > Jagannathan Raman <jag.raman@oracle.com> > Elena Ufimtseva <elena.ufimtseva@oracle.com> > John G Johnson <john.g.johnson@oracle.com> > Stefan Hajnoczi <stefanha@redhat.com> > Konrad Wilk <konrad.wilk@oracle.com> > Kanth Ghatraju <kanth.ghatraju@oracle.com> > > We would like to thank you for your feedback in the > design and implementation of this project. Qemu wiki page: > https://wiki.qemu.org/Features/MultiProcessQEMU > > For the full concept writeup about QEMU multi-process, please > refer to docs/devel/qemu-multiprocess.rst. Also, see > docs/qemu-multiprocess.txt for usage information. > > Thank you for reviewing this series! > > [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html > [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html > [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html > [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html > [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html > [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html > [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html > [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/ > [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html > [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html > [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html > [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html > [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/ > [13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html > [14]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg768376.html > [15]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg769178.html > [16]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg771455.html > [17]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772512.html > [18]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772602.html > [19]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772980.html > > Elena Ufimtseva (8): > multi-process: add configure and usage information > io: add qio_channel_writev_full_all helper > io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all > helpers > multi-process: define MPQemuMsg format and transmission functions > multi-process: introduce proxy object > multi-process: add proxy communication functions > multi-process: Forward PCI config space acceses to the remote process > multi-process: perform device reset in the remote process > > Jagannathan Raman (11): > memory: alloc RAM from file at offset > multi-process: Add config option for multi-process QEMU > multi-process: setup PCI host bridge for remote device > multi-process: setup a machine object for remote device process > multi-process: Initialize message handler in remote device > multi-process: Associate fd of a PCIDevice with its object > multi-process: setup memory manager for remote device > multi-process: PCI BAR read/write handling for proxy & remote > endpoints > multi-process: Synchronize remote memory > multi-process: create IOHUB object to handle irq > multi-process: Retrieve PCI info from remote process > > John G Johnson (1): > multi-process: add the concept description to > docs/devel/qemu-multiprocess > > docs/devel/index.rst | 1 + > docs/devel/multi-process.rst | 966 ++++++++++++++++++++++++++++++ > docs/multi-process.rst | 64 ++ > configure | 10 + > meson.build | 5 +- > hw/remote/trace.h | 1 + > include/exec/memory.h | 2 + > include/exec/ram_addr.h | 2 +- > include/hw/pci-host/remote.h | 30 + > include/hw/pci/pci_ids.h | 3 + > include/hw/remote/iohub.h | 42 ++ > include/hw/remote/machine.h | 38 ++ > include/hw/remote/memory.h | 19 + > include/hw/remote/mpqemu-link.h | 99 +++ > include/hw/remote/proxy-memory-listener.h | 28 + > include/hw/remote/proxy.h | 48 ++ > include/io/channel.h | 78 +++ > include/qemu/mmap-alloc.h | 4 +- > include/sysemu/iothread.h | 6 + > backends/hostmem-memfd.c | 2 +- > hw/misc/ivshmem.c | 3 +- > hw/pci-host/remote.c | 75 +++ > hw/remote/iohub.c | 119 ++++ > hw/remote/machine.c | 80 +++ > hw/remote/memory.c | 65 ++ > hw/remote/message.c | 230 +++++++ > hw/remote/mpqemu-link.c | 267 +++++++++ > hw/remote/proxy-memory-listener.c | 227 +++++++ > hw/remote/proxy.c | 379 ++++++++++++ > hw/remote/remote-obj.c | 203 +++++++ > io/channel.c | 121 +++- > iothread.c | 6 + > softmmu/memory.c | 3 +- > softmmu/physmem.c | 11 +- > util/mmap-alloc.c | 7 +- > util/oslib-posix.c | 2 +- > Kconfig.host | 4 + > MAINTAINERS | 24 + > hw/Kconfig | 1 + > hw/meson.build | 1 + > hw/pci-host/Kconfig | 3 + > hw/pci-host/meson.build | 1 + > hw/remote/Kconfig | 4 + > hw/remote/meson.build | 13 + > hw/remote/trace-events | 4 + > 45 files changed, 3265 insertions(+), 36 deletions(-) > create mode 100644 docs/devel/multi-process.rst > create mode 100644 docs/multi-process.rst > create mode 100644 hw/remote/trace.h > create mode 100644 include/hw/pci-host/remote.h > create mode 100644 include/hw/remote/iohub.h > create mode 100644 include/hw/remote/machine.h > create mode 100644 include/hw/remote/memory.h > create mode 100644 include/hw/remote/mpqemu-link.h > create mode 100644 include/hw/remote/proxy-memory-listener.h > create mode 100644 include/hw/remote/proxy.h > create mode 100644 hw/pci-host/remote.c > create mode 100644 hw/remote/iohub.c > create mode 100644 hw/remote/machine.c > create mode 100644 hw/remote/memory.c > create mode 100644 hw/remote/message.c > create mode 100644 hw/remote/mpqemu-link.c > create mode 100644 hw/remote/proxy-memory-listener.c > create mode 100644 hw/remote/proxy.c > create mode 100644 hw/remote/remote-obj.c > create mode 100644 hw/remote/Kconfig > create mode 100644 hw/remote/meson.build > create mode 100644 hw/remote/trace-events > > -- > 1.8.3.1 > I will merge this when Daniel Berrange has reviewed it. Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> On Jan 21, 2021, at 12:07 PM, Stefan Hajnoczi <stefanha@redhat.com> wrote: > > On Tue, Jan 19, 2021 at 03:28:17PM -0500, Jagannathan Raman wrote: >> Hi, >> >> This is the v20 of the patchset. This version has the following changes: >> >> [PATCH v18 08/20] io: add qio_channel_readv_full_all_eof & >> qio_channel_readv_full_all helpers >> - Considers fds as part of the "data" for both >> qio_channel_readv_full_all_eof() & >> qio_channel_readv_full_all(). >> - Addresses some of the error scenarios >> which were not addressed in the previous revisions >> >> To touch upon the history of this project, we posted the Proof Of Concept >> patches before the BoF session in 2018. Subsequently, we have posted 19 >> versions on the qemu-devel mailing list. You can find them by following >> the links below ([1] - [19]). Following people contributed to the design and >> implementation of this project: >> Jagannathan Raman <jag.raman@oracle.com> >> Elena Ufimtseva <elena.ufimtseva@oracle.com> >> John G Johnson <john.g.johnson@oracle.com> >> Stefan Hajnoczi <stefanha@redhat.com> >> Konrad Wilk <konrad.wilk@oracle.com> >> Kanth Ghatraju <kanth.ghatraju@oracle.com> >> >> We would like to thank you for your feedback in the >> design and implementation of this project. Qemu wiki page: >> https://wiki.qemu.org/Features/MultiProcessQEMU >> >> For the full concept writeup about QEMU multi-process, please >> refer to docs/devel/qemu-multiprocess.rst. Also, see >> docs/qemu-multiprocess.txt for usage information. >> >> Thank you for reviewing this series! >> >> [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html >> [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html >> [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html >> [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html >> [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html >> [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html >> [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html >> [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/ >> [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html >> [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html >> [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html >> [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html >> [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/ >> [13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html >> [14]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg768376.html >> [15]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg769178.html >> [16]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg771455.html >> [17]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772512.html >> [18]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772602.html >> [19]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772980.html >> >> Elena Ufimtseva (8): >> multi-process: add configure and usage information >> io: add qio_channel_writev_full_all helper >> io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all >> helpers >> multi-process: define MPQemuMsg format and transmission functions >> multi-process: introduce proxy object >> multi-process: add proxy communication functions >> multi-process: Forward PCI config space acceses to the remote process >> multi-process: perform device reset in the remote process >> >> Jagannathan Raman (11): >> memory: alloc RAM from file at offset >> multi-process: Add config option for multi-process QEMU >> multi-process: setup PCI host bridge for remote device >> multi-process: setup a machine object for remote device process >> multi-process: Initialize message handler in remote device >> multi-process: Associate fd of a PCIDevice with its object >> multi-process: setup memory manager for remote device >> multi-process: PCI BAR read/write handling for proxy & remote >> endpoints >> multi-process: Synchronize remote memory >> multi-process: create IOHUB object to handle irq >> multi-process: Retrieve PCI info from remote process >> >> John G Johnson (1): >> multi-process: add the concept description to >> docs/devel/qemu-multiprocess >> >> docs/devel/index.rst | 1 + >> docs/devel/multi-process.rst | 966 ++++++++++++++++++++++++++++++ >> docs/multi-process.rst | 64 ++ >> configure | 10 + >> meson.build | 5 +- >> hw/remote/trace.h | 1 + >> include/exec/memory.h | 2 + >> include/exec/ram_addr.h | 2 +- >> include/hw/pci-host/remote.h | 30 + >> include/hw/pci/pci_ids.h | 3 + >> include/hw/remote/iohub.h | 42 ++ >> include/hw/remote/machine.h | 38 ++ >> include/hw/remote/memory.h | 19 + >> include/hw/remote/mpqemu-link.h | 99 +++ >> include/hw/remote/proxy-memory-listener.h | 28 + >> include/hw/remote/proxy.h | 48 ++ >> include/io/channel.h | 78 +++ >> include/qemu/mmap-alloc.h | 4 +- >> include/sysemu/iothread.h | 6 + >> backends/hostmem-memfd.c | 2 +- >> hw/misc/ivshmem.c | 3 +- >> hw/pci-host/remote.c | 75 +++ >> hw/remote/iohub.c | 119 ++++ >> hw/remote/machine.c | 80 +++ >> hw/remote/memory.c | 65 ++ >> hw/remote/message.c | 230 +++++++ >> hw/remote/mpqemu-link.c | 267 +++++++++ >> hw/remote/proxy-memory-listener.c | 227 +++++++ >> hw/remote/proxy.c | 379 ++++++++++++ >> hw/remote/remote-obj.c | 203 +++++++ >> io/channel.c | 121 +++- >> iothread.c | 6 + >> softmmu/memory.c | 3 +- >> softmmu/physmem.c | 11 +- >> util/mmap-alloc.c | 7 +- >> util/oslib-posix.c | 2 +- >> Kconfig.host | 4 + >> MAINTAINERS | 24 + >> hw/Kconfig | 1 + >> hw/meson.build | 1 + >> hw/pci-host/Kconfig | 3 + >> hw/pci-host/meson.build | 1 + >> hw/remote/Kconfig | 4 + >> hw/remote/meson.build | 13 + >> hw/remote/trace-events | 4 + >> 45 files changed, 3265 insertions(+), 36 deletions(-) >> create mode 100644 docs/devel/multi-process.rst >> create mode 100644 docs/multi-process.rst >> create mode 100644 hw/remote/trace.h >> create mode 100644 include/hw/pci-host/remote.h >> create mode 100644 include/hw/remote/iohub.h >> create mode 100644 include/hw/remote/machine.h >> create mode 100644 include/hw/remote/memory.h >> create mode 100644 include/hw/remote/mpqemu-link.h >> create mode 100644 include/hw/remote/proxy-memory-listener.h >> create mode 100644 include/hw/remote/proxy.h >> create mode 100644 hw/pci-host/remote.c >> create mode 100644 hw/remote/iohub.c >> create mode 100644 hw/remote/machine.c >> create mode 100644 hw/remote/memory.c >> create mode 100644 hw/remote/message.c >> create mode 100644 hw/remote/mpqemu-link.c >> create mode 100644 hw/remote/proxy-memory-listener.c >> create mode 100644 hw/remote/proxy.c >> create mode 100644 hw/remote/remote-obj.c >> create mode 100644 hw/remote/Kconfig >> create mode 100644 hw/remote/meson.build >> create mode 100644 hw/remote/trace-events >> >> -- >> 1.8.3.1 >> > > I will merge this when Daniel Berrange has reviewed it. > > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Thank you for confirming, Stefan!
Hi, This is the v20 of the patchset. This version has the following changes: [PATCH v18 08/20] io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all helpers - Considers fds as part of the "data" for both qio_channel_readv_full_all_eof() & qio_channel_readv_full_all(). - Addresses some of the error scenarios which were not addressed in the previous revisions To touch upon the history of this project, we posted the Proof Of Concept patches before the BoF session in 2018. Subsequently, we have posted 19 versions on the qemu-devel mailing list. You can find them by following the links below ([1] - [19]). Following people contributed to the design and implementation of this project: Jagannathan Raman <jag.raman@oracle.com> Elena Ufimtseva <elena.ufimtseva@oracle.com> John G Johnson <john.g.johnson@oracle.com> Stefan Hajnoczi <stefanha@redhat.com> Konrad Wilk <konrad.wilk@oracle.com> Kanth Ghatraju <kanth.ghatraju@oracle.com> We would like to thank you for your feedback in the design and implementation of this project. Qemu wiki page: https://wiki.qemu.org/Features/MultiProcessQEMU For the full concept writeup about QEMU multi-process, please refer to docs/devel/qemu-multiprocess.rst. Also, see docs/qemu-multiprocess.txt for usage information. Thank you for reviewing this series! [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/ [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/ [13]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg766825.html [14]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg768376.html [15]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg769178.html [16]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg771455.html [17]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772512.html [18]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772602.html [19]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg772980.html Elena Ufimtseva (8): multi-process: add configure and usage information io: add qio_channel_writev_full_all helper io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all helpers multi-process: define MPQemuMsg format and transmission functions multi-process: introduce proxy object multi-process: add proxy communication functions multi-process: Forward PCI config space acceses to the remote process multi-process: perform device reset in the remote process Jagannathan Raman (11): memory: alloc RAM from file at offset multi-process: Add config option for multi-process QEMU multi-process: setup PCI host bridge for remote device multi-process: setup a machine object for remote device process multi-process: Initialize message handler in remote device multi-process: Associate fd of a PCIDevice with its object multi-process: setup memory manager for remote device multi-process: PCI BAR read/write handling for proxy & remote endpoints multi-process: Synchronize remote memory multi-process: create IOHUB object to handle irq multi-process: Retrieve PCI info from remote process John G Johnson (1): multi-process: add the concept description to docs/devel/qemu-multiprocess docs/devel/index.rst | 1 + docs/devel/multi-process.rst | 966 ++++++++++++++++++++++++++++++ docs/multi-process.rst | 64 ++ configure | 10 + meson.build | 5 +- hw/remote/trace.h | 1 + include/exec/memory.h | 2 + include/exec/ram_addr.h | 2 +- include/hw/pci-host/remote.h | 30 + include/hw/pci/pci_ids.h | 3 + include/hw/remote/iohub.h | 42 ++ include/hw/remote/machine.h | 38 ++ include/hw/remote/memory.h | 19 + include/hw/remote/mpqemu-link.h | 99 +++ include/hw/remote/proxy-memory-listener.h | 28 + include/hw/remote/proxy.h | 48 ++ include/io/channel.h | 78 +++ include/qemu/mmap-alloc.h | 4 +- include/sysemu/iothread.h | 6 + backends/hostmem-memfd.c | 2 +- hw/misc/ivshmem.c | 3 +- hw/pci-host/remote.c | 75 +++ hw/remote/iohub.c | 119 ++++ hw/remote/machine.c | 80 +++ hw/remote/memory.c | 65 ++ hw/remote/message.c | 230 +++++++ hw/remote/mpqemu-link.c | 267 +++++++++ hw/remote/proxy-memory-listener.c | 227 +++++++ hw/remote/proxy.c | 379 ++++++++++++ hw/remote/remote-obj.c | 203 +++++++ io/channel.c | 121 +++- iothread.c | 6 + softmmu/memory.c | 3 +- softmmu/physmem.c | 11 +- util/mmap-alloc.c | 7 +- util/oslib-posix.c | 2 +- Kconfig.host | 4 + MAINTAINERS | 24 + hw/Kconfig | 1 + hw/meson.build | 1 + hw/pci-host/Kconfig | 3 + hw/pci-host/meson.build | 1 + hw/remote/Kconfig | 4 + hw/remote/meson.build | 13 + hw/remote/trace-events | 4 + 45 files changed, 3265 insertions(+), 36 deletions(-) create mode 100644 docs/devel/multi-process.rst create mode 100644 docs/multi-process.rst create mode 100644 hw/remote/trace.h create mode 100644 include/hw/pci-host/remote.h create mode 100644 include/hw/remote/iohub.h create mode 100644 include/hw/remote/machine.h create mode 100644 include/hw/remote/memory.h create mode 100644 include/hw/remote/mpqemu-link.h create mode 100644 include/hw/remote/proxy-memory-listener.h create mode 100644 include/hw/remote/proxy.h create mode 100644 hw/pci-host/remote.c create mode 100644 hw/remote/iohub.c create mode 100644 hw/remote/machine.c create mode 100644 hw/remote/memory.c create mode 100644 hw/remote/message.c create mode 100644 hw/remote/mpqemu-link.c create mode 100644 hw/remote/proxy-memory-listener.c create mode 100644 hw/remote/proxy.c create mode 100644 hw/remote/remote-obj.c create mode 100644 hw/remote/Kconfig create mode 100644 hw/remote/meson.build create mode 100644 hw/remote/trace-events