| Message ID | cover.1582576372.git.jag.raman@oracle.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Initial support for multi-process qemu | expand |
Patchew URL: https://patchew.org/QEMU/cover.1582576372.git.jag.raman@oracle.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Subject: [PATCH v5 00/50] Initial support for multi-process qemu
Message-id: cover.1582576372.git.jag.raman@oracle.com
Type: series
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/20200224205533.23798-1-philmd@redhat.com -> patchew/20200224205533.23798-1-philmd@redhat.com
Switched to a new branch 'test'
046649c multi-process: add configure and usage information
7c79419 multi-process: add the concept description to docs/devel/qemu-multiprocess
b06e676 multi-process: Validate incoming commands from Proxy
cc1f6b4 multi-process: Enable support for multiple devices in remote
0060f81 multi-process/mig: Restore the VMSD in remote process
f163e9b multi-process/mig: Synchronize runstate of remote process
8bc3796 multi-process/mig: refactor runstate_check into common file
4ebbd74 multi-process/mig: Load VMSD in the proxy object
4b124b2 multi-process/mig: Send VMSD of remote to the Proxy object
b8dbad8 multi-process/mig: Enable VMSD save in the Proxy object
3253157 multi-process/mig: build migration module in the remote process
96557d0 multi-process: prevent duplicate memory initialization in remote
9a85af0 multi-process/mon: Initialize QMP module for remote processes
ca70d4a multi-process/mon: Refactor monitor/chardev functions out of vl.c
dbe3a63 multi-process/mon: enable QMP module support in the remote process
a12a802 multi-process/mon: stub functions to enable QMP module for remote process
1e6d3af multi-process/mon: choose HMP commands based on target
888e741 multi-process: perform device reset in the remote process
8f15bd8 multi-process: Use separate MMIO communication channel
64593f5 multi-process: handle heartbeat messages in remote process
93b0f96 multi-process: send heartbeat messages to remote
ad214e9 multi-process: add parse_cmdline in remote process
83bc352 multi-process: add remote options parser
7fcc704 multi-process: add remote option
24ca162 multi-process: refractor vl.c code
20c05d9 multi-process: Introduce build flags to separate remote process code
335f331 multi-process: add processing of remote device command line
413bf28 multi-process: remote: add create_done condition
d0e4f00 multi-process: remote: use fd for socket from parent process
045f8f8 multi-process: remote: add setup_devices msg processing
a8ea8aa multi-process: add qdev_proxy_add to create proxy devices
59d0bff multi-process: Retrieve PCI info from remote process
898af59 multi-process: configure remote side devices
b49c9f3 multi-process: create IOHUB object to handle irq
06dc766 multi-process: Synchronize remote memory
a88dcbd multi-process: PCI BAR read/write handling for proxy & remote endpoints
5a3fbbb mutli-process: build remote command line args
c319ce7 multi-process: introduce proxy object
7cb950d multi-process: remote process initialization
e91592e multi-process: setup memory manager for remote device
4f14ba4 multi-process: setup a machine object for remote device process
b1e58b1 multi-process: setup PCI host bridge for remote device
2785043 multi-process: add functions to synchronize proxy and remote endpoints
f83d3f3 multi-process: define mpqemu-link object
252d736 multi-process: build system for remote device process
1482849 multi-process: Add config option for multi-process QEMU
b78f0cc multi-process: Add stub functions to facilate build of multi-process
837f4a6 multi-process: add a command line option for debug file
e58fa6d multi-process: Refactor machine_init and exit notifiers
bf5b343 multi-process: memory: alloc RAM from file at offset
=== OUTPUT BEGIN ===
1/50 Checking commit bf5b3438f584 (multi-process: memory: alloc RAM from file at offset)
2/50 Checking commit e58fa6db276f (multi-process: Refactor machine_init and exit notifiers)
3/50 Checking commit 837f4a602862 (multi-process: add a command line option for debug file)
4/50 Checking commit b78f0cc3f072 (multi-process: Add stub functions to facilate build of multi-process)
ERROR: suspect code indent for conditional statements (4, 4)
#141: FILE: accel/stubs/tcg-stub.c:109:
+ while (1) {
+ }
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#155:
new file mode 100644
total: 1 errors, 1 warnings, 378 lines checked
Patch 4/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/50 Checking commit 1482849e9584 (multi-process: Add config option for multi-process QEMU)
6/50 Checking commit 252d7366e5c1 (multi-process: build system for remote device process)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#311:
new file mode 100644
total: 0 errors, 1 warnings, 250 lines checked
Patch 6/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/50 Checking commit f83d3f37432f (multi-process: define mpqemu-link object)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#20:
new file mode 100644
total: 0 errors, 1 warnings, 437 lines checked
Patch 7/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
8/50 Checking commit 278504388c18 (multi-process: add functions to synchronize proxy and remote endpoints)
9/50 Checking commit b1e58b14c4d2 (multi-process: setup PCI host bridge for remote device)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#31:
new file mode 100644
total: 0 errors, 1 warnings, 118 lines checked
Patch 9/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
10/50 Checking commit 4f14ba434d5b (multi-process: setup a machine object for remote device process)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#63:
new file mode 100644
total: 0 errors, 1 warnings, 169 lines checked
Patch 10/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/50 Checking commit e91592e1d3e5 (multi-process: setup memory manager for remote device)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#122:
new file mode 100644
total: 0 errors, 1 warnings, 163 lines checked
Patch 11/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/50 Checking commit 7cb950d0b414 (multi-process: remote process initialization)
13/50 Checking commit c319ce7e8e42 (multi-process: introduce proxy object)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#34:
new file mode 100644
WARNING: line over 80 characters
#78: FILE: hw/proxy/qemu-proxy.c:33:
+ error_setg(errp, "Remote processed is managed and launched by external program");
total: 0 errors, 2 warnings, 338 lines checked
Patch 13/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
14/50 Checking commit 5a3fbbb83e89 (mutli-process: build remote command line args)
15/50 Checking commit a88dcbd3b371 (multi-process: PCI BAR read/write handling for proxy & remote endpoints)
16/50 Checking commit 06dc7666cb21 (multi-process: Synchronize remote memory)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#36:
new file mode 100644
total: 0 errors, 1 warnings, 323 lines checked
Patch 16/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
17/50 Checking commit b49c9f3a3655 (multi-process: create IOHUB object to handle irq)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#43:
deleted file mode 100644
total: 0 errors, 1 warnings, 411 lines checked
Patch 17/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
18/50 Checking commit 898af59b01ef (multi-process: configure remote side devices)
19/50 Checking commit 59d0bff02614 (multi-process: Retrieve PCI info from remote process)
20/50 Checking commit a8ea8aa3417d (multi-process: add qdev_proxy_add to create proxy devices)
21/50 Checking commit 045f8f82de1c (multi-process: remote: add setup_devices msg processing)
22/50 Checking commit d0e4f00c3065 (multi-process: remote: use fd for socket from parent process)
23/50 Checking commit 413bf287c459 (multi-process: remote: add create_done condition)
24/50 Checking commit 335f331eed39 (multi-process: add processing of remote device command line)
25/50 Checking commit 20c05d9deccc (multi-process: Introduce build flags to separate remote process code)
26/50 Checking commit 24ca162587f0 (multi-process: refractor vl.c code)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#189:
new file mode 100644
total: 0 errors, 1 warnings, 326 lines checked
Patch 26/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
27/50 Checking commit 7fcc7047eaa0 (multi-process: add remote option)
28/50 Checking commit 83bc352512c7 (multi-process: add remote options parser)
29/50 Checking commit ad214e916e4d (multi-process: add parse_cmdline in remote process)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#59:
new file mode 100644
ERROR: line over 90 characters
#135: FILE: remote/remote-opts.c:72:
+ error_report("Option not supported for this target, %x arch_mask, %x arch_type",
total: 1 errors, 1 warnings, 148 lines checked
Patch 29/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
30/50 Checking commit 93b0f96ec29b (multi-process: send heartbeat messages to remote)
31/50 Checking commit 64593f51a86c (multi-process: handle heartbeat messages in remote process)
32/50 Checking commit 8f15bd8412a3 (multi-process: Use separate MMIO communication channel)
33/50 Checking commit 888e741a19af (multi-process: perform device reset in the remote process)
34/50 Checking commit 1e6d3af047e6 (multi-process/mon: choose HMP commands based on target)
35/50 Checking commit a12a80202f4e (multi-process/mon: stub functions to enable QMP module for remote process)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#110:
new file mode 100644
ERROR: line over 90 characters
#550: FILE: stubs/qapi-target.c:48:
+#if defined(TARGET_PPC) || defined(TARGET_ARM) || defined(TARGET_I386) || defined(TARGET_S390X) || defined(TARGET_MIPS)
total: 1 errors, 1 warnings, 730 lines checked
Patch 35/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
36/50 Checking commit dbe3a637fac7 (multi-process/mon: enable QMP module support in the remote process)
37/50 Checking commit ca70d4a7859d (multi-process/mon: Refactor monitor/chardev functions out of vl.c)
38/50 Checking commit 9a85af09cf01 (multi-process/mon: Initialize QMP module for remote processes)
39/50 Checking commit 96557d09787c (multi-process: prevent duplicate memory initialization in remote)
40/50 Checking commit 32531572e8c7 (multi-process/mig: build migration module in the remote process)
41/50 Checking commit b8dbad8073b6 (multi-process/mig: Enable VMSD save in the Proxy object)
ERROR: suspect code indent for conditional statements (4, 4)
#131: FILE: hw/proxy/qemu-proxy.c:557:
+ while (*((volatile uint64_t *)&pdev->migsize) < size) {
+ }
total: 1 errors, 0 warnings, 174 lines checked
Patch 41/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
42/50 Checking commit 4b124b25aded (multi-process/mig: Send VMSD of remote to the Proxy object)
43/50 Checking commit 4ebbd744f34b (multi-process/mig: Load VMSD in the proxy object)
44/50 Checking commit 8bc37967dcd8 (multi-process/mig: refactor runstate_check into common file)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#48:
new file mode 100644
total: 0 errors, 1 warnings, 89 lines checked
Patch 44/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
45/50 Checking commit f163e9b02e25 (multi-process/mig: Synchronize runstate of remote process)
46/50 Checking commit 0060f814e9c8 (multi-process/mig: Restore the VMSD in remote process)
47/50 Checking commit cc1f6b4970d4 (multi-process: Enable support for multiple devices in remote)
48/50 Checking commit b06e676c2119 (multi-process: Validate incoming commands from Proxy)
49/50 Checking commit 7c794195319d (multi-process: add the concept description to docs/devel/qemu-multiprocess)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#22:
new file mode 100644
total: 0 errors, 1 warnings, 1106 lines checked
Patch 49/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
50/50 Checking commit 046649c71c87 (multi-process: add configure and usage information)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#13:
new file mode 100644
total: 0 errors, 1 warnings, 86 lines checked
Patch 50/50 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/cover.1582576372.git.jag.raman@oracle.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Patchew URL: https://patchew.org/QEMU/cover.1582576372.git.jag.raman@oracle.com/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===
CC stubs/sysbus.o
CC stubs/tpm.o
Warning, treated as error:
/tmp/qemu-test/src/docs/devel/index.rst:13:toctree contains reference to nonexisting document 'multi-process'
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
make: *** [Makefile:1063: docs/devel/index.html] Error 2
make: *** Waiting for unfinished jobs....
In file included from /tmp/qemu-test/src/include/monitor/qdev.h:4,
from /tmp/qemu-test/src/stubs/monitor.c:14:
/tmp/qemu-test/src/include/hw/proxy/qemu-proxy.h:12:10: fatal error: linux/kvm.h: No such file or directory
#include <linux/kvm.h>
^~~~~~~~~~~~~
compilation terminated.
make: *** [/tmp/qemu-test/src/rules.mak:69: stubs/monitor.o] Error 1
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
sys.exit(main())
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=4f8cf4b3c5d9425cb7ce2e50962601a2', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-gx6fxsj2/src/docker-src.2020-02-24-17.00.12.3070:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=4f8cf4b3c5d9425cb7ce2e50962601a2
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-gx6fxsj2/src'
make: *** [docker-run-test-mingw@fedora] Error 2
real 3m34.769s
user 0m4.558s
The full log is available at
http://patchew.org/logs/cover.1582576372.git.jag.raman@oracle.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Patchew URL: https://patchew.org/QEMU/cover.1582576372.git.jag.raman@oracle.com/
Hi,
This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===
qemu-system-aarch64: falling back to tcg
TEST check-unit: tests/test-char
**
ERROR:/tmp/qemu-test/src/tests/test-char.c:230:char_mux_test: assertion failed (h1.last_event == 42): (3 == 42)
TEST iotest-qcow2: 004
qemu-system-aarch64: -accel kvm: invalid accelerator kvm
qemu-system-aarch64: falling back to tcg
ERROR - Bail out! ERROR:/tmp/qemu-test/src/tests/test-char.c:230:char_mux_test: assertion failed (h1.last_event == 42): (3 == 42)
make: *** [check-unit] Error 1
make: *** Waiting for unfinished jobs....
TEST check-qtest-x86_64: tests/qtest/fdc-test
TEST iotest-qcow2: 005
---
Not run: 259
Failures: 030 040 041 071 099 120 184 186 267
Failed 9 of 116 iotests
make: *** [check-tests/check-block.sh] Error 1
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
sys.exit(main())
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=084f448bfb414d66a22a2ccf133f7163', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-0vhz47_c/src/docker-src.2020-02-24-17.09.16.1174:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=084f448bfb414d66a22a2ccf133f7163
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-0vhz47_c/src'
make: *** [docker-run-test-quick@centos7] Error 2
real 14m39.906s
user 0m8.340s
The full log is available at
http://patchew.org/logs/cover.1582576372.git.jag.raman@oracle.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Jagannathan Raman <jag.raman@oracle.com> writes: > Hello > > Started with the presentation in October 2017 made by Marc-Andre (Red Hat) > and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018, > the multi-process project is now available and presented in this patchset. > This first series enables the emulation of lsi53c895a in a separate process. > > We posted the Proof Of Concept patches [2] before the BoF session in 2018. > Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5] and RFC v4 [6]. > > John & Elena presented the status of this project in KVM Forum 2019. We > appreciate the in-person and email feedback we received to improve this > patchset. We also received valuable feedback and direction on future > improvements from the bi-weekly KVM community conference. We have > incorporated all the feedback in the current version of the series, v5. > > Following people contributed to this patchset: > > John G Johnson <john.g.johnson@oracle.com> > Jagannathan Raman <jag.raman@oracle.com> > Elena Ufimtseva <elena.ufimtseva@oracle.com> > Kanth Ghatraju <kanth.ghatraju@oracle.com> > Konrad Wilk <konrad.wilk@oracle.com> > > For full concept writeup about QEMU disaggregation, refer to > docs/devel/qemu-multiprocess.rst. Please refer to > docs/qemu-multiprocess.txt for usage information. > > We are planning on making the following improvements in the future to the experimental > Qemu multi-process: > - Asynchronous communication channel; > - Performance improvements; > - Libvirt support; > - Enforcement of security policies and privileges control; > > We welcome all your ideas, concerns, and questions for this patchset. There seem to be quite a few CI failures with this series applied: https://travis-ci.org/stsquad/qemu/builds/656432858 https://app.shippable.com/github/stsquad/qemu/runs/1275/summary/console https://gitlab.com/stsquad/qemu/pipelines/122030403 https://cirrus-ci.com/build/4577637150490624
On 3/1/2020 6:57 AM, Alex Bennée wrote: > > Jagannathan Raman <jag.raman@oracle.com> writes: > >> Hello >> >> Started with the presentation in October 2017 made by Marc-Andre (Red Hat) >> and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018, >> the multi-process project is now available and presented in this patchset. >> This first series enables the emulation of lsi53c895a in a separate process. >> >> We posted the Proof Of Concept patches [2] before the BoF session in 2018. >> Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5] and RFC v4 [6]. >> >> John & Elena presented the status of this project in KVM Forum 2019. We >> appreciate the in-person and email feedback we received to improve this >> patchset. We also received valuable feedback and direction on future >> improvements from the bi-weekly KVM community conference. We have >> incorporated all the feedback in the current version of the series, v5. >> >> Following people contributed to this patchset: >> >> John G Johnson <john.g.johnson@oracle.com> >> Jagannathan Raman <jag.raman@oracle.com> >> Elena Ufimtseva <elena.ufimtseva@oracle.com> >> Kanth Ghatraju <kanth.ghatraju@oracle.com> >> Konrad Wilk <konrad.wilk@oracle.com> >> >> For full concept writeup about QEMU disaggregation, refer to >> docs/devel/qemu-multiprocess.rst. Please refer to >> docs/qemu-multiprocess.txt for usage information. >> >> We are planning on making the following improvements in the future to the experimental >> Qemu multi-process: >> - Asynchronous communication channel; >> - Performance improvements; >> - Libvirt support; >> - Enforcement of security policies and privileges control; >> >> We welcome all your ideas, concerns, and questions for this patchset. > > There seem to be quite a few CI failures with this series applied: > > https://travis-ci.org/stsquad/qemu/builds/656432858 > https://app.shippable.com/github/stsquad/qemu/runs/1275/summary/console > https://gitlab.com/stsquad/qemu/pipelines/122030403 > https://cirrus-ci.com/build/4577637150490624 Hi Alex, Thanks for pointing it out. "Patchew" also identified some errors which we are working on fixing for the next version. Patchew summarized the errors in the following page: https://patchew.org/QEMU/cover.1582576372.git.jag.raman@oracle.com/ To confirm we're compliant with Patchew, we are running docker tests before sending the patches for review next time around. We'll use the following wiki to trigger "travis-ci" tests before pushing the branch for review next time around: https://wiki.qemu.org/Testing/CI/Travis#Testing_Changes_to_Travis Are shippable, stsquad & cirrus-ci redundant if travis-ci & docker tests pass? Thank you very much! -- Jag >
Jag Raman <jag.raman@oracle.com> writes: > On 3/1/2020 6:57 AM, Alex Bennée wrote: >> Jagannathan Raman <jag.raman@oracle.com> writes: >> >>> Hello >>> Started with the presentation in October 2017 made by Marc-Andre >>> (Red Hat) >>> and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018, >>> the multi-process project is now available and presented in this patchset. >>> This first series enables the emulation of lsi53c895a in a separate process. >>> >>> We posted the Proof Of Concept patches [2] before the BoF session in 2018. >>> Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5] and RFC v4 [6]. >>> >>> John & Elena presented the status of this project in KVM Forum 2019. We >>> appreciate the in-person and email feedback we received to improve this >>> patchset. We also received valuable feedback and direction on future >>> improvements from the bi-weekly KVM community conference. We have >>> incorporated all the feedback in the current version of the series, v5. >>> >>> Following people contributed to this patchset: >>> >>> John G Johnson <john.g.johnson@oracle.com> >>> Jagannathan Raman <jag.raman@oracle.com> >>> Elena Ufimtseva <elena.ufimtseva@oracle.com> >>> Kanth Ghatraju <kanth.ghatraju@oracle.com> >>> Konrad Wilk <konrad.wilk@oracle.com> >>> >>> For full concept writeup about QEMU disaggregation, refer to >>> docs/devel/qemu-multiprocess.rst. Please refer to >>> docs/qemu-multiprocess.txt for usage information. >>> >>> We are planning on making the following improvements in the future to the experimental >>> Qemu multi-process: >>> - Asynchronous communication channel; >>> - Performance improvements; >>> - Libvirt support; >>> - Enforcement of security policies and privileges control; >>> >>> We welcome all your ideas, concerns, and questions for this patchset. >> There seem to be quite a few CI failures with this series applied: >> https://travis-ci.org/stsquad/qemu/builds/656432858 >> https://app.shippable.com/github/stsquad/qemu/runs/1275/summary/console >> https://gitlab.com/stsquad/qemu/pipelines/122030403 >> https://cirrus-ci.com/build/4577637150490624 > > Hi Alex, > > Thanks for pointing it out. > > "Patchew" also identified some errors which we are working on fixing for > the next version. Patchew summarized the errors in the following page: > https://patchew.org/QEMU/cover.1582576372.git.jag.raman@oracle.com/ > > To confirm we're compliant with Patchew, we are running docker tests > before sending the patches for review next time around. > > We'll use the following wiki to trigger "travis-ci" tests before pushing > the branch for review next time around: > https://wiki.qemu.org/Testing/CI/Travis#Testing_Changes_to_Travis > > Are shippable, stsquad & cirrus-ci redundant if travis-ci & docker > tests stsquad is just my user account, hopefully I'm not redundant ;-) They all test slightly different things but you should be able to replicate the tests locally. Travis basically tests a bunch of different configuration setups on mostly x86 hardware. Unless it's a weird library interaction issue this should replicate in your normal build environment. Shippable are cross compile tests. They use the existing docker infrastructure to cross compile for various target architectures. See "make docker" and the notes in docs/devel/testing.rst. CirrusCI tests MacOS and FreeBSD builds. You can build on the BSD's yourself, see "make vm-help". MacOSX is trickier unless you have a Mac yourself of course. > pass? > > Thank you very much!
On 3/2/2020 11:29 AM, Alex Bennée wrote: > > Jag Raman <jag.raman@oracle.com> writes: > >> On 3/1/2020 6:57 AM, Alex Bennée wrote: >>> Jagannathan Raman <jag.raman@oracle.com> writes: >>> >>>> Hello >>>> Started with the presentation in October 2017 made by Marc-Andre >>>> (Red Hat) >>>> and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018, >>>> the multi-process project is now available and presented in this patchset. >>>> This first series enables the emulation of lsi53c895a in a separate process. >>>> >>>> We posted the Proof Of Concept patches [2] before the BoF session in 2018. >>>> Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5] and RFC v4 [6]. >>>> >>>> John & Elena presented the status of this project in KVM Forum 2019. We >>>> appreciate the in-person and email feedback we received to improve this >>>> patchset. We also received valuable feedback and direction on future >>>> improvements from the bi-weekly KVM community conference. We have >>>> incorporated all the feedback in the current version of the series, v5. >>>> >>>> Following people contributed to this patchset: >>>> >>>> John G Johnson <john.g.johnson@oracle.com> >>>> Jagannathan Raman <jag.raman@oracle.com> >>>> Elena Ufimtseva <elena.ufimtseva@oracle.com> >>>> Kanth Ghatraju <kanth.ghatraju@oracle.com> >>>> Konrad Wilk <konrad.wilk@oracle.com> >>>> >>>> For full concept writeup about QEMU disaggregation, refer to >>>> docs/devel/qemu-multiprocess.rst. Please refer to >>>> docs/qemu-multiprocess.txt for usage information. >>>> >>>> We are planning on making the following improvements in the future to the experimental >>>> Qemu multi-process: >>>> - Asynchronous communication channel; >>>> - Performance improvements; >>>> - Libvirt support; >>>> - Enforcement of security policies and privileges control; >>>> >>>> We welcome all your ideas, concerns, and questions for this patchset. >>> There seem to be quite a few CI failures with this series applied: >>> https://travis-ci.org/stsquad/qemu/builds/656432858 >>> https://app.shippable.com/github/stsquad/qemu/runs/1275/summary/console >>> https://gitlab.com/stsquad/qemu/pipelines/122030403 >>> https://cirrus-ci.com/build/4577637150490624 >> >> Hi Alex, >> >> Thanks for pointing it out. >> >> "Patchew" also identified some errors which we are working on fixing for >> the next version. Patchew summarized the errors in the following page: >> https://patchew.org/QEMU/cover.1582576372.git.jag.raman@oracle.com/ >> >> To confirm we're compliant with Patchew, we are running docker tests >> before sending the patches for review next time around. >> >> We'll use the following wiki to trigger "travis-ci" tests before pushing >> the branch for review next time around: >> https://wiki.qemu.org/Testing/CI/Travis#Testing_Changes_to_Travis >> >> Are shippable, stsquad & cirrus-ci redundant if travis-ci & docker >> tests > > stsquad is just my user account, hopefully I'm not redundant ;-) Sorry I didn't realize it's your account. :) > > They all test slightly different things but you should be able to > replicate the tests locally. > > Travis basically tests a bunch of different configuration setups on > mostly x86 hardware. Unless it's a weird library interaction issue this > should replicate in your normal build environment. We'll run Travis CI also before submitting patches next time. The qemu wiki page for TravisCI says it tests MacOS. So this test should be sufficient to cover MacOS build tests I suppose. https://wiki.qemu.org/Testing/CI/Travis#Testing_Changes_to_Travis > > Shippable are cross compile tests. They use the existing docker > infrastructure to cross compile for various target architectures. See > "make docker" and the notes in docs/devel/testing.rst. We're running these docker tests locally. Thank you very much! -- Jag > > CirrusCI tests MacOS and FreeBSD builds. You can build on the BSD's > yourself, see "make vm-help". MacOSX is trickier unless you have a Mac > yourself of course. > >> pass? >> >> Thank you very much! > >
Hello Started with the presentation in October 2017 made by Marc-Andre (Red Hat) and Konrad Wilk (Oracle) [1], and continued by Jag's BoF at KVM Forum 2018, the multi-process project is now available and presented in this patchset. This first series enables the emulation of lsi53c895a in a separate process. We posted the Proof Of Concept patches [2] before the BoF session in 2018. Subsequently, we posted RFC v1 [3], RFC v2 [4], RFC v3 [5] and RFC v4 [6]. John & Elena presented the status of this project in KVM Forum 2019. We appreciate the in-person and email feedback we received to improve this patchset. We also received valuable feedback and direction on future improvements from the bi-weekly KVM community conference. We have incorporated all the feedback in the current version of the series, v5. Following people contributed to this patchset: John G Johnson <john.g.johnson@oracle.com> Jagannathan Raman <jag.raman@oracle.com> Elena Ufimtseva <elena.ufimtseva@oracle.com> Kanth Ghatraju <kanth.ghatraju@oracle.com> Konrad Wilk <konrad.wilk@oracle.com> For full concept writeup about QEMU disaggregation, refer to docs/devel/qemu-multiprocess.rst. Please refer to docs/qemu-multiprocess.txt for usage information. We are planning on making the following improvements in the future to the experimental Qemu multi-process: - Asynchronous communication channel; - Performance improvements; - Libvirt support; - Enforcement of security policies and privileges control; We welcome all your ideas, concerns, and questions for this patchset. Thank you! [1]: http://events17.linuxfoundation.org/sites/events/files/slides/KVM%20FORUM%20multi-process.pdf [1]: https://www.youtube.com/watch?v=Kq1-coHh7lg [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html Elena Ufimtseva (23): multi-process: add a command line option for debug file multi-process: introduce proxy object mutli-process: build remote command line args multi-process: configure remote side devices multi-process: add qdev_proxy_add to create proxy devices multi-process: remote: add setup_devices msg processing multi-process: remote: use fd for socket from parent process multi-process: remote: add create_done condition multi-process: add processing of remote device command line multi-process: refractor vl.c code multi-process: add remote option multi-process: add remote options parser multi-process: add parse_cmdline in remote process multi-process: send heartbeat messages to remote multi-process: handle heartbeat messages in remote process multi-process/mon: choose HMP commands based on target multi-process/mig: Load VMSD in the proxy object multi-process/mig: refactor runstate_check into common file multi-process/mig: Synchronize runstate of remote process multi-process/mig: Restore the VMSD in remote process multi-process: Enable support for multiple devices in remote multi-process: Validate incoming commands from Proxy multi-process: add configure and usage information Jagannathan Raman (26): multi-process: memory: alloc RAM from file at offset multi-process: Refactor machine_init and exit notifiers multi-process: Add stub functions to facilate build of multi-process multi-process: Add config option for multi-process QEMU multi-process: build system for remote device process multi-process: define mpqemu-link object multi-process: add functions to synchronize proxy and remote endpoints multi-process: setup PCI host bridge for remote device multi-process: setup a machine object for remote device process multi-process: setup memory manager for remote device multi-process: remote process initialization multi-process: PCI BAR read/write handling for proxy & remote endpoints multi-process: Synchronize remote memory multi-process: create IOHUB object to handle irq multi-process: Retrieve PCI info from remote process multi-process: Introduce build flags to separate remote process code multi-process: Use separate MMIO communication channel multi-process: perform device reset in the remote process multi-process/mon: stub functions to enable QMP module for remote process multi-process/mon: enable QMP module support in the remote process multi-process/mon: Refactor monitor/chardev functions out of vl.c multi-process/mon: Initialize QMP module for remote processes multi-process: prevent duplicate memory initialization in remote multi-process/mig: build migration module in the remote process multi-process/mig: Enable VMSD save in the Proxy object multi-process/mig: Send VMSD of remote to the Proxy object John G Johnson (1): multi-process: add the concept description to docs/devel/qemu-multiprocess Makefile | 2 + Makefile.objs | 45 ++ Makefile.target | 105 +++- accel/Makefile.objs | 2 + accel/stubs/kvm-stub.c | 5 + accel/stubs/tcg-stub.c | 107 ++++ backends/Makefile.objs | 2 + block/Makefile.objs | 2 + chardev/char.c | 14 + configure | 15 + docs/devel/index.rst | 1 + docs/devel/qemu-multiprocess.rst | 1102 ++++++++++++++++++++++++++++++++++++++ docs/qemu-multiprocess.txt | 86 +++ exec.c | 31 +- hmp-commands-info.hx | 10 + hmp-commands.hx | 25 +- hw/Makefile.objs | 7 + hw/block/Makefile.objs | 2 + hw/core/Makefile.objs | 19 + hw/nvram/Makefile.objs | 2 + hw/pci/Makefile.objs | 4 + hw/proxy/memory-sync.c | 212 ++++++++ hw/proxy/qemu-proxy.c | 906 +++++++++++++++++++++++++++++++ hw/scsi/Makefile.objs | 2 + include/chardev/char.h | 1 + include/exec/address-spaces.h | 2 + include/exec/ram_addr.h | 4 +- include/hw/pci/pci_ids.h | 3 + include/hw/proxy/memory-sync.h | 37 ++ include/hw/proxy/qemu-proxy.h | 107 ++++ include/hw/qdev-core.h | 2 + include/io/mpqemu-link.h | 218 ++++++++ include/monitor/monitor.h | 2 + include/monitor/qdev.h | 26 + include/qemu-common.h | 8 + include/qemu/log.h | 1 + include/qemu/mmap-alloc.h | 3 +- include/remote/iohub.h | 50 ++ include/remote/machine.h | 32 ++ include/remote/memory.h | 20 + include/remote/pcihost.h | 45 ++ include/sysemu/runstate.h | 3 + include/sysemu/sysemu.h | 2 + io/Makefile.objs | 2 + io/mpqemu-link.c | 410 ++++++++++++++ memory.c | 2 +- migration/Makefile.objs | 13 + migration/savevm.c | 63 +++ migration/savevm.h | 3 + monitor/Makefile.objs | 4 + monitor/misc.c | 84 +-- monitor/monitor-internal.h | 38 ++ monitor/monitor.c | 40 +- net/Makefile.objs | 2 + qapi/Makefile.objs | 2 + qdev-monitor.c | 259 ++++++++- qemu-options.hx | 21 + qom/Makefile.objs | 4 + remote/Makefile.objs | 6 + remote/iohub.c | 148 +++++ remote/machine.c | 99 ++++ remote/memory.c | 63 +++ remote/pcihost.c | 64 +++ remote/remote-main.c | 625 +++++++++++++++++++++ remote/remote-opts.c | 115 ++++ remote/remote-opts.h | 15 + rules.mak | 2 +- runstate.c | 41 ++ scripts/hxtool | 44 +- softmmu/vl.c | 299 ++++------- stubs/audio.c | 12 + stubs/gdbstub.c | 23 + stubs/machine-init-done.c | 4 + stubs/migration.c | 211 ++++++++ stubs/monitor.c | 72 +++ stubs/net-stub.c | 121 +++++ stubs/qapi-misc.c | 43 ++ stubs/qapi-target.c | 55 ++ stubs/replay.c | 26 + stubs/runstate-check.c | 3 + stubs/ui-stub.c | 130 +++++ stubs/vl-stub.c | 195 +++++++ stubs/vmstate.c | 19 + stubs/xen-mapcache.c | 22 + ui/Makefile.objs | 2 + util/Makefile.objs | 2 + util/log.c | 2 + util/mmap-alloc.c | 7 +- util/notify.c | 43 ++ util/oslib-posix.c | 2 +- vl-parse.c | 164 ++++++ vl.h | 43 ++ 92 files changed, 6698 insertions(+), 245 deletions(-) create mode 100644 docs/devel/qemu-multiprocess.rst create mode 100644 docs/qemu-multiprocess.txt create mode 100644 hw/proxy/memory-sync.c create mode 100644 hw/proxy/qemu-proxy.c create mode 100644 include/hw/proxy/memory-sync.h create mode 100644 include/hw/proxy/qemu-proxy.h create mode 100644 include/io/mpqemu-link.h create mode 100644 include/remote/iohub.h create mode 100644 include/remote/machine.h create mode 100644 include/remote/memory.h create mode 100644 include/remote/pcihost.h create mode 100644 io/mpqemu-link.c create mode 100644 remote/Makefile.objs create mode 100644 remote/iohub.c create mode 100644 remote/machine.c create mode 100644 remote/memory.c create mode 100644 remote/pcihost.c create mode 100644 remote/remote-main.c create mode 100644 remote/remote-opts.c create mode 100644 remote/remote-opts.h create mode 100644 runstate.c mode change 100644 => 100755 scripts/hxtool create mode 100644 stubs/audio.c create mode 100644 stubs/migration.c create mode 100644 stubs/net-stub.c create mode 100644 stubs/qapi-misc.c create mode 100644 stubs/qapi-target.c create mode 100644 stubs/ui-stub.c create mode 100644 stubs/vl-stub.c create mode 100644 stubs/xen-mapcache.c create mode 100644 vl-parse.c create mode 100644 vl.h