[00/10] iOS and Apple Silicon host support

Message

Joelle van Dyne Oct. 12, 2020, 11:29 p.m. UTC

These set of changes brings QEMU TCG to iOS devices and future Apple Silicon
devices. They were originally developed last year and have been working in the
UTM app. Recently, we ported the changes to master, re-wrote a lot of the build
script changes for meson, and broke up the patches into more distinct units.

A summary of the changes:

* `CONFIG_IOS` and `CONFIG_IOS_JIT` defined when building for iOS and
  iOS specific changes (as well as unsupported code) are gated behind it.
* slirp updated to support DNS resolving using libresolv; a separate patch will
  be submitted to the project. To allow for builds to succeed, the .gitmodule
  is temporarily changed. We're not entirely sure how cross-project patches
  should be handled here.
* A new dependency, libucontext is added since iOS does not have native ucontext
  and broken support for sigaltstack. libucontext is available as a new option
  for coroutine backend.
* On stock iOS devices, there is a workaround for running JIT code without
  any special entitlement. It requires the JIT region to be mirror mapped with
  one region RW and another one RX. To support this style of JIT, TCG is changed
  to support writing to a different code_ptr. These changes are gated by the
  `CONFIG_IOS_JIT`.
* For (recent) jailbroken iOS devices as well as upcoming Apple Silicon devices,
  there are new rules for applications supporting JIT (with the proper
  entitlement). These rules are implemented as well.
* An option to build QEMU as a shared library is added because iOS does not
  support fork().

These patches are also on Github:
  https://github.com/utmapp/qemu/tree/ios-support-master

-j

osy (10):
  configure: option to disable host block devices
  configure: cross-compiling without cross_prefix
  qemu: add support for iOS host
  meson: option to build as shared library
  slirp: update for iOS resolv fix
  coroutine: add libucontext as external library
  tcg: implement bulletproof JIT
  tcg: mirror mapping RWX pages for iOS optional
  tcg: support JIT on Apple Silicon
  block: check availablity for preadv/pwritev on mac

 .gitmodules                  |   5 +-
 accel/tcg/cpu-exec-common.c  |   2 +
 accel/tcg/cpu-exec.c         |   9 +-
 accel/tcg/tcg-all.c          |  27 +++++-
 accel/tcg/translate-all.c    | 169 ++++++++++++++++++++++++++++++++---
 block.c                      |   2 +-
 block/file-posix.c           |  50 ++++++++---
 bsd-user/main.c              |   2 +-
 configure                    | 118 ++++++++++++++++++++++--
 docs/devel/ios.rst           |  40 +++++++++
 include/exec/exec-all.h      |  10 +++
 include/sysemu/tcg.h         |   2 +-
 include/tcg/tcg-apple-jit.h  |  85 ++++++++++++++++++
 include/tcg/tcg.h            |  21 ++++-
 libucontext                  |   1 +
 linux-user/main.c            |   2 +-
 meson.build                  |  74 +++++++++++++--
 meson_options.txt            |   4 +
 net/slirp.c                  |  16 ++--
 qemu-options.hx              |  11 +++
 qga/commands-posix.c         |   6 ++
 slirp                        |   2 +-
 target/arm/arm-semi.c        |   2 +
 target/m68k/m68k-semi.c      |   2 +
 target/nios2/nios2-semi.c    |   2 +
 tcg/aarch64/tcg-target.c.inc |  48 ++++++----
 tcg/aarch64/tcg-target.h     |  23 ++++-
 tcg/arm/tcg-target.c.inc     |  33 ++++---
 tcg/arm/tcg-target.h         |   9 +-
 tcg/i386/tcg-target.c.inc    |  28 +++---
 tcg/i386/tcg-target.h        |  24 ++++-
 tcg/mips/tcg-target.c.inc    |  64 +++++++------
 tcg/mips/tcg-target.h        |   8 +-
 tcg/ppc/tcg-target.c.inc     |  55 +++++++-----
 tcg/ppc/tcg-target.h         |   8 +-
 tcg/riscv/tcg-target.c.inc   |  51 ++++++-----
 tcg/riscv/tcg-target.h       |   9 +-
 tcg/s390/tcg-target.c.inc    |  25 +++---
 tcg/s390/tcg-target.h        |  13 ++-
 tcg/sparc/tcg-target.c.inc   |  33 ++++---
 tcg/sparc/tcg-target.h       |   8 +-
 tcg/tcg-ldst.c.inc           |   2 +-
 tcg/tcg-pool.c.inc           |   9 +-
 tcg/tcg.c                    |  64 ++++++++-----
 tcg/tci/tcg-target.c.inc     |   8 +-
 tcg/tci/tcg-target.h         |   9 +-
 tests/qtest/meson.build      |   7 +-
 util/coroutine-ucontext.c    |   9 ++
 48 files changed, 965 insertions(+), 246 deletions(-)
 create mode 100644 docs/devel/ios.rst
 create mode 100644 include/tcg/tcg-apple-jit.h
 create mode 160000 libucontext

Comments

[e-mail] Oct. 13, 2020, 1:21 a.m. UTC | #1

Patchew URL: https://patchew.org/QEMU/[e-mail]/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: [e-mail]
Subject: [PATCH 00/10] iOS and Apple Silicon host support

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/[e-mail] -> patchew/[e-mail]
Switched to a new branch 'test'
b6c449f block: check availablity for preadv/pwritev on mac
86f3837 tcg: support JIT on Apple Silicon
6389bee tcg: mirror mapping RWX pages for iOS optional
8727db6 tcg: implement bulletproof JIT
e655897 coroutine: add libucontext as external library
e43d59a slirp: update for iOS resolv fix
715f3bd meson: option to build as shared library
7276968 qemu: add support for iOS host
9f6bafb configure: cross-compiling without cross_prefix
9187922 configure: option to disable host block devices

=== OUTPUT BEGIN ===
1/10 Checking commit 9187922c6d93 (configure: option to disable host block devices)
WARNING: architecture specific defines should be avoided
#22: FILE: block/file-posix.c:44:
+#if defined(CONFIG_HOST_BLOCK_DEVICE) && defined(__APPLE__) && (__MACH__)

total: 0 errors, 1 warnings, 61 lines checked

Patch 1/10 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/10 Checking commit 9f6bafb19c1d (configure: cross-compiling without cross_prefix)
3/10 Checking commit 72769683dd9c (qemu: add support for iOS host)
WARNING: architecture specific defines should be avoided
#27: FILE: block.c:56:
+#if !defined(__DragonFly__) && !defined(CONFIG_IOS)

ERROR: braces {} are necessary for all arms of this statement
#45: FILE: block/file-posix.c:189:
+    if (s->fd >= 0)
[...]

WARNING: architecture specific defines should be avoided
#79: FILE: block/file-posix.c:2325:
+#if !defined(CONFIG_IOS) && defined(__APPLE__) && defined(__MACH__)

WARNING: architecture specific defines should be avoided
#363: FILE: tcg/aarch64/tcg-target.h:151:
+#if defined(__APPLE__)

WARNING: architecture specific defines should be avoided
#369: FILE: tcg/aarch64/tcg-target.h:157:
+#if defined(__APPLE__)

total: 1 errors, 4 warnings, 316 lines checked

Patch 3/10 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/10 Checking commit 715f3bddd393 (meson: option to build as shared library)
5/10 Checking commit e43d59afc25c (slirp: update for iOS resolv fix)
6/10 Checking commit e655897c7ab2 (coroutine: add libucontext as external library)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#97: 
new file mode 160000

total: 0 errors, 1 warnings, 140 lines checked

Patch 6/10 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/10 Checking commit 8727db6cfd2d (tcg: implement bulletproof JIT)
ERROR: externs should be avoided in .c files
#52: FILE: accel/tcg/translate-all.c:65:
+extern kern_return_t mach_vm_remap(vm_map_t target_task,

WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#189: 
new file mode 100644

WARNING: architecture specific defines should be avoided
#501: FILE: tcg/aarch64/tcg-target.h:171:
+#if defined(__APPLE__)

WARNING: architecture specific defines should be avoided
#757: FILE: tcg/i386/tcg-target.h:209:
+#ifdef __APPLE__

WARNING: architecture specific defines should be avoided
#768: FILE: tcg/i386/tcg-target.h:220:
+#if defined(__APPLE__)

total: 1 errors, 4 warnings, 1673 lines checked

Patch 7/10 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

8/10 Checking commit 6389bee4f3d0 (tcg: mirror mapping RWX pages for iOS optional)
9/10 Checking commit 86f3837d7074 (tcg: support JIT on Apple Silicon)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#206: 
new file mode 100644

WARNING: architecture specific defines should be avoided
#238: FILE: include/tcg/tcg-apple-jit.h:28:
+#if defined(__aarch64__) && defined(CONFIG_DARWIN)

total: 0 errors, 2 warnings, 259 lines checked

Patch 9/10 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
10/10 Checking commit b6c449ff37a3 (block: check availablity for preadv/pwritev on mac)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/[e-mail]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [e-mail]

Joelle van Dyne Oct. 13, 2020, 2:12 a.m. UTC | #2

I did run the style check tool. Of the errors/warnings, a few are from
code moved from one place to another, re-formatting the moved code
seems to break another rule (don't include irrelevant changes).



On Mon, Oct 12, 2020 at 6:21 PM <[e-mail]> wrote:
>
> Patchew URL: https://patchew.org/QEMU/[e-mail]/
>
>
>
> Hi,
>
> This series seems to have some coding style problems. See output below for
> more information:
>
> Type: series
> Message-id: [e-mail]
> Subject: [PATCH 00/10] iOS and Apple Silicon host support
>
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
>
> Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
> From https://github.com/patchew-project/qemu
>  * [new tag]         patchew/[e-mail] -> patchew/[e-mail]
> Switched to a new branch 'test'
> b6c449f block: check availablity for preadv/pwritev on mac
> 86f3837 tcg: support JIT on Apple Silicon
> 6389bee tcg: mirror mapping RWX pages for iOS optional
> 8727db6 tcg: implement bulletproof JIT
> e655897 coroutine: add libucontext as external library
> e43d59a slirp: update for iOS resolv fix
> 715f3bd meson: option to build as shared library
> 7276968 qemu: add support for iOS host
> 9f6bafb configure: cross-compiling without cross_prefix
> 9187922 configure: option to disable host block devices
>
> === OUTPUT BEGIN ===
> 1/10 Checking commit 9187922c6d93 (configure: option to disable host block devices)
> WARNING: architecture specific defines should be avoided
> #22: FILE: block/file-posix.c:44:
> +#if defined(CONFIG_HOST_BLOCK_DEVICE) && defined(__APPLE__) && (__MACH__)
>
> total: 0 errors, 1 warnings, 61 lines checked
>
> Patch 1/10 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 2/10 Checking commit 9f6bafb19c1d (configure: cross-compiling without cross_prefix)
> 3/10 Checking commit 72769683dd9c (qemu: add support for iOS host)
> WARNING: architecture specific defines should be avoided
> #27: FILE: block.c:56:
> +#if !defined(__DragonFly__) && !defined(CONFIG_IOS)
>
> ERROR: braces {} are necessary for all arms of this statement
> #45: FILE: block/file-posix.c:189:
> +    if (s->fd >= 0)
> [...]
>
> WARNING: architecture specific defines should be avoided
> #79: FILE: block/file-posix.c:2325:
> +#if !defined(CONFIG_IOS) && defined(__APPLE__) && defined(__MACH__)
>
> WARNING: architecture specific defines should be avoided
> #363: FILE: tcg/aarch64/tcg-target.h:151:
> +#if defined(__APPLE__)
>
> WARNING: architecture specific defines should be avoided
> #369: FILE: tcg/aarch64/tcg-target.h:157:
> +#if defined(__APPLE__)
>
> total: 1 errors, 4 warnings, 316 lines checked
>
> Patch 3/10 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
>
> 4/10 Checking commit 715f3bddd393 (meson: option to build as shared library)
> 5/10 Checking commit e43d59afc25c (slirp: update for iOS resolv fix)
> 6/10 Checking commit e655897c7ab2 (coroutine: add libucontext as external library)
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #97:
> new file mode 160000
>
> total: 0 errors, 1 warnings, 140 lines checked
>
> Patch 6/10 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 7/10 Checking commit 8727db6cfd2d (tcg: implement bulletproof JIT)
> ERROR: externs should be avoided in .c files
> #52: FILE: accel/tcg/translate-all.c:65:
> +extern kern_return_t mach_vm_remap(vm_map_t target_task,
>
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #189:
> new file mode 100644
>
> WARNING: architecture specific defines should be avoided
> #501: FILE: tcg/aarch64/tcg-target.h:171:
> +#if defined(__APPLE__)
>
> WARNING: architecture specific defines should be avoided
> #757: FILE: tcg/i386/tcg-target.h:209:
> +#ifdef __APPLE__
>
> WARNING: architecture specific defines should be avoided
> #768: FILE: tcg/i386/tcg-target.h:220:
> +#if defined(__APPLE__)
>
> total: 1 errors, 4 warnings, 1673 lines checked
>
> Patch 7/10 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
>
> 8/10 Checking commit 6389bee4f3d0 (tcg: mirror mapping RWX pages for iOS optional)
> 9/10 Checking commit 86f3837d7074 (tcg: support JIT on Apple Silicon)
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
> #206:
> new file mode 100644
>
> WARNING: architecture specific defines should be avoided
> #238: FILE: include/tcg/tcg-apple-jit.h:28:
> +#if defined(__aarch64__) && defined(CONFIG_DARWIN)
>
> total: 0 errors, 2 warnings, 259 lines checked
>
> Patch 9/10 has style problems, please review.  If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
> 10/10 Checking commit b6c449ff37a3 (block: check availablity for preadv/pwritev on mac)
> === OUTPUT END ===
>
> Test command exited with code: 1
>
>
> The full log is available at
> http://patchew.org/logs/[e-mail]/testing.checkpatch/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to [e-mail]