| Message ID | cover.1586165555.git.elena.ufimtseva@oracle.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Initial support for multi-process qemu | expand |
Patchew URL: https://patchew.org/QEMU/cover.1586165555.git.elena.ufimtseva@oracle.com/
Hi,
This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
/tmp/qemu-test/src/stubs/monitor.c:25:14: error: weak identifier 'qapi_event_emit' never declared [-Werror]
#pragma weak qapi_event_emit
^
1 error generated.
CC stubs/ram-block.o
make: *** [/tmp/qemu-test/src/rules.mak:69: stubs/monitor.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=db5ee80d8e5941478e687ebfe0d7ddb5', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-88fcgq5_/src/docker-src.2020-04-06-06.09.35.21273:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=db5ee80d8e5941478e687ebfe0d7ddb5
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-88fcgq5_/src'
make: *** [docker-run-test-debug@fedora] Error 2
real 4m23.786s
user 0m8.667s
The full log is available at
http://patchew.org/logs/cover.1586165555.git.elena.ufimtseva@oracle.com/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Patchew URL: https://patchew.org/QEMU/cover.1586165555.git.elena.ufimtseva@oracle.com/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===
LINK qemu-edid.exe
libqemuutil.a(qemu-sockets.o): In function `socket_get_fd':
/tmp/qemu-test/src/util/qemu-sockets.c:1079: undefined reference to `monitor_get_fd'
collect2: error: ld returned 1 exit status
make: *** [Makefile:689: qemu-ga.exe] Error 1
make: *** Waiting for unfinished jobs....
libqemuutil.a(qemu-sockets.o): In function `socket_get_fd':
/tmp/qemu-test/src/util/qemu-sockets.c:1079: undefined reference to `monitor_get_fd'
collect2: error: ld returned 1 exit status
make: *** [/tmp/qemu-test/src/rules.mak:124: qemu-io.exe] Error 1
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
sys.exit(main())
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=a29cf9a7b6a24130965fa70f18e8ab54', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-dnwlb_j0/src/docker-src.2020-04-06-06.31.29.9070:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=a29cf9a7b6a24130965fa70f18e8ab54
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-dnwlb_j0/src'
make: *** [docker-run-test-mingw@fedora] Error 2
real 4m46.514s
user 0m5.244s
The full log is available at
http://patchew.org/logs/cover.1586165555.git.elena.ufimtseva@oracle.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
On Mon, Apr 06, 2020 at 03:14:02AM -0700, no-reply@patchew.org wrote: > Patchew URL: https://patchew.org/QEMU/cover.1586165555.git.elena.ufimtseva@oracle.com/ > > > > Hi, > > This series failed the asan build test. Please find the testing commands and > their output below. If you have Docker installed, you can probably reproduce it > locally. > Hi, we have noticed this error and are in progress of resolving it. Elena > === TEST SCRIPT BEGIN === > #!/bin/bash > export ARCH=x86_64 > make docker-image-fedora V=1 NETWORK=1 > time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 > === TEST SCRIPT END === > > CC stubs/xen-common.o > CC stubs/xen-hvm.o > CC stubs/pci-host-piix.o > /tmp/qemu-test/src/stubs/monitor.c:25:14: error: weak identifier 'qapi_event_emit' never declared [-Werror] > #pragma weak qapi_event_emit > ^ > 1 error generated. > CC stubs/ram-block.o > make: *** [/tmp/qemu-test/src/rules.mak:69: stubs/monitor.o] Error 1 > make: *** Waiting for unfinished jobs.... > Traceback (most recent call last): > File "./tests/docker/docker.py", line 664, in <module> > --- > raise CalledProcessError(retcode, cmd) > subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=db5ee80d8e5941478e687ebfe0d7ddb5', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-88fcgq5_/src/docker-src.2020-04-06-06.09.35.21273:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2. > filter=--filter=label=com.qemu.instance.uuid=db5ee80d8e5941478e687ebfe0d7ddb5 > make[1]: *** [docker-run] Error 1 > make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-88fcgq5_/src' > make: *** [docker-run-test-debug@fedora] Error 2 > > real 4m23.786s > user 0m8.667s > > > The full log is available at > http://patchew.org/logs/cover.1586165555.git.elena.ufimtseva@oracle.com/testing.asan/?type=message. > --- > Email generated automatically by Patchew [https://patchew.org/]. > Please send your feedback to patchew-devel@redhat.com
From: Elena Ufimtseva <elena.ufimtseva@oracle.com> Hello Started with the presentation in October 2017 made by Marc-Andre (Red Hat) and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018, the multi-process project is now available and presented in this patchset. This first series enables the emulation of lsi53c895a in a separate process. We posted the Proof Of Concept patches [2] before the BoF session in 2018. Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5], RFC v4 [6] and v5 [7] of the patch series. This is v6 of the patch series and it addresses the previous feedback from the community. To make easier to review of the series, we have separated out some of the patches and will send them in the separate series. As per converstaion we had during the last community call, the live migration support is taken out from this series as well as asynchronous communication. The changes include the elimination of fork/exec of the remote process and instead using the launcher which is implemented in this series as a python script. Following people contributed to this patchset: John G Johnson <john.g.johnson@oracle.com> Jagannathan Raman <jag.raman@oracle.com> Elena Ufimtseva <elena.ufimtseva@oracle.com> Kanth Ghatraju <kanth.ghatraju@oracle.com> Konrad Wilk <konrad.wilk@oracle.com> For full concept writeup about QEMU disaggregation, refer to docs/devel/qemu-multiprocess.rst. Please refer to docs/qemu-multiprocess.txt for usage information. We will post separate patchsets for the following improvements for the experimental Qemu multi-process: - Live migration; - Asynchronous communication channel; - Libvirt support; We welcome all your ideas, concerns, and questions for this patchset. Thank you! [1]: http://events17.linuxfoundation.org/sites/events/files/slides/KVM%20FORUM%20multi-process.pdf [1]: https://www.youtube.com/watch?v=Kq1-coHh7lg [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html [7]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html Elena Ufimtseva (18): multi-process: Refactor machine_init and exit notifiers command-line: refractor parser code multi-process: Refactor chardev functions out of vl.c multi-process: Refactor monitor functions out of vl.c multi-process: add a command line option for debug file multi-process: introduce proxy object multi-process: Forward PCI config space acceses to the remote process multi-process: Introduce build flags to separate remote process code multi-process: add parse_cmdline in remote process multi-process: add support to parse device option multi-process: send heartbeat messages to remote multi-process: handle heartbeat messages in remote process multi-process: perform device reset in the remote process multi-process/mon: choose HMP commands based on target multi-process/mon: stub functions to enable QMP module for remote process multi-process/mon: enable QMP module support in the remote process multi-process/mon: Initialize QMP module for remote processes multi-process: add configure and usage information Jagannathan Raman (17): memory: alloc RAM from file at offset monitor: destaticize HMP commands multi-process: Add stub functions to facilate build of multi-process multi-process: Add config option for multi-process QEMU multi-process: build system for remote device process multi-process: define mpqemu-link object multi-process: add functions to synchronize proxy and remote endpoints multi-process: setup PCI host bridge for remote device multi-process: setup a machine object for remote device process multi-process: setup memory manager for remote device multi-process: remote process initialization multi-process: Initialize Proxy Object's communication channel multi-process: Connect Proxy Object with device in the remote process multi-process: PCI BAR read/write handling for proxy & remote endpoints multi-process: Synchronize remote memory multi-process: create IOHUB object to handle irq multi-process: Retrieve PCI info from remote process John G Johnson (1): multi-process: add the concept description to docs/devel/qemu-multiprocess MAINTAINERS | 35 + Makefile | 2 + Makefile.objs | 41 + Makefile.target | 104 ++- accel/Makefile.objs | 2 + accel/stubs/kvm-stub.c | 5 + accel/stubs/tcg-stub.c | 108 +++ backends/Makefile.objs | 2 + block/Makefile.objs | 5 + block/monitor/Makefile.objs | 2 + chardev/char.c | 14 + configure | 15 + docs/devel/index.rst | 1 + docs/devel/multi-process.rst | 1102 ++++++++++++++++++++++++++ docs/multi-process.rst | 85 ++ exec.c | 31 +- hmp-commands-info.hx | 10 + hmp-commands.hx | 25 +- hw/Makefile.objs | 7 + hw/block/Makefile.objs | 2 + hw/core/Makefile.objs | 19 + hw/nvram/Makefile.objs | 2 + hw/pci/Makefile.objs | 4 + hw/proxy/memory-sync.c | 212 +++++ hw/proxy/qemu-proxy.c | 488 ++++++++++++ hw/scsi/Makefile.objs | 2 + include/chardev/char.h | 2 + include/exec/address-spaces.h | 2 + include/exec/ram_addr.h | 4 +- include/hw/pci/pci_ids.h | 3 + include/hw/proxy/memory-sync.h | 37 + include/hw/proxy/qemu-proxy.h | 79 ++ include/io/mpqemu-link.h | 192 +++++ include/monitor/monitor.h | 3 + include/qemu-common.h | 8 + include/qemu-parse.h | 42 + include/qemu/log.h | 1 + include/qemu/mmap-alloc.h | 3 +- include/remote/iohub.h | 50 ++ include/remote/machine.h | 32 + include/remote/memory.h | 20 + include/remote/pcihost.h | 45 ++ include/sysemu/sysemu.h | 2 + io/Makefile.objs | 2 + io/mpqemu-link.c | 387 +++++++++ memory.c | 2 +- migration/Makefile.objs | 2 + monitor/Makefile.objs | 4 + monitor/misc.c | 84 +- monitor/monitor-internal.h | 38 + monitor/monitor.c | 37 + qapi/Makefile.objs | 2 + qemu-parse.c | 93 +++ qom/Makefile.objs | 4 + remote/Makefile.objs | 5 + remote/iohub.c | 148 ++++ remote/machine.c | 99 +++ remote/memory.c | 63 ++ remote/pcihost.c | 64 ++ remote/remote-main.c | 350 ++++++++ remote/remote-opts.c | 96 +++ remote/remote-opts.h | 15 + rules.mak | 2 +- scripts/hxtool | 35 +- scripts/mpqemu-launcher-perf-mode.py | 93 +++ scripts/mpqemu-launcher.py | 54 ++ softmmu/vl.c | 173 +--- stubs/audio.c | 12 + stubs/gdbstub.c | 23 + stubs/machine-init-done.c | 12 + stubs/migration.c | 162 ++++ stubs/monitor.c | 85 ++ stubs/net-stub.c | 100 +++ stubs/qapi-misc.c | 41 + stubs/qapi-target.c | 56 ++ stubs/replay.c | 18 + stubs/ui-stub.c | 130 +++ stubs/vl-stub.c | 171 ++++ stubs/vmstate.c | 19 + stubs/xen-mapcache.c | 22 + ui/Makefile.objs | 2 + util/Makefile.objs | 2 + util/log.c | 2 + util/machine-notify.c | 67 ++ util/mmap-alloc.c | 7 +- util/oslib-posix.c | 2 +- 86 files changed, 5406 insertions(+), 229 deletions(-) create mode 100644 docs/devel/multi-process.rst create mode 100644 docs/multi-process.rst create mode 100644 hw/proxy/memory-sync.c create mode 100644 hw/proxy/qemu-proxy.c create mode 100644 include/hw/proxy/memory-sync.h create mode 100644 include/hw/proxy/qemu-proxy.h create mode 100644 include/io/mpqemu-link.h create mode 100644 include/qemu-parse.h create mode 100644 include/remote/iohub.h create mode 100644 include/remote/machine.h create mode 100644 include/remote/memory.h create mode 100644 include/remote/pcihost.h create mode 100644 io/mpqemu-link.c create mode 100644 qemu-parse.c create mode 100644 remote/Makefile.objs create mode 100644 remote/iohub.c create mode 100644 remote/machine.c create mode 100644 remote/memory.c create mode 100644 remote/pcihost.c create mode 100644 remote/remote-main.c create mode 100644 remote/remote-opts.c create mode 100644 remote/remote-opts.h mode change 100644 => 100755 scripts/hxtool create mode 100755 scripts/mpqemu-launcher-perf-mode.py create mode 100755 scripts/mpqemu-launcher.py create mode 100644 stubs/audio.c create mode 100644 stubs/migration.c create mode 100644 stubs/net-stub.c create mode 100644 stubs/qapi-misc.c create mode 100644 stubs/qapi-target.c create mode 100644 stubs/ui-stub.c create mode 100644 stubs/vl-stub.c create mode 100644 stubs/xen-mapcache.c create mode 100644 util/machine-notify.c