[v6,0/7] cutils: Introduce bundle mechanism

Message

Akihiko Odaki June 15, 2022, 3:56 p.m. UTC

Developers often run QEMU without installing. The bundle mechanism
allows to look up files which should be present in installation even in
such a situation.

It is a general mechanism and can find any files located relative
to the installation tree. The build tree must have a new directory,
qemu-bundle, to represent what files the installation tree would
have for reference by the executables.

v6: Reuse get_relocated_path() in find_bundle() (Paolo Bonzini)

v5:
* Prefer qemu-bundle if it exists. (Daniel P. Berrangé)
* Check install_blobs option before installing BIOSes (Paolo Bonzini)
* Add common code to set up qemu-bundle to the top level meson.build
  (Paolo Bonzini)

v4:
* Add Daniel P. Berrangé to CC. Hopefully this helps merging his patch:
  https://mail.gnu.org/archive/html/qemu-devel/2022-06/msg02276.html
* Rebased to the latest QEMU.

v3:
* Note that the bundle mechanism is for any files located relative to the
  installation tree including but not limited to datadir. (Peter Maydell)
* Fix "bridge" typo (Philippe Mathieu-Daudé)

v2: Rebased to the latest QEMU.

Akihiko Odaki (7):
  datadir: Simplify firmware directory search
  qga: Relocate a path emitted in the help text
  Remove prefixes from path configuration macros
  cutils: Introduce bundle mechanism
  datadir: Use bundle mechanism
  ui/icons: Use bundle mechanism
  net: Use bundle mechanism

 .travis.yml                 |  2 +-
 include/qemu/cutils.h       | 21 ++++++++++++++-
 meson.build                 | 35 ++++++++++++++++--------
 net/tap.c                   |  2 +-
 pc-bios/keymaps/meson.build |  2 ++
 pc-bios/meson.build         | 19 +++++++------
 qemu-options.hx             | 11 ++++----
 qga/main.c                  |  2 +-
 scripts/oss-fuzz/build.sh   |  2 +-
 softmmu/datadir.c           | 43 +++++++-----------------------
 tests/qtest/fuzz/fuzz.c     | 15 -----------
 tests/vm/fedora             |  2 +-
 tests/vm/freebsd            |  2 +-
 tests/vm/netbsd             |  2 +-
 tests/vm/openbsd            |  2 +-
 ui/cocoa.m                  |  2 +-
 ui/gtk.c                    |  2 +-
 ui/icons/meson.build        | 32 ++++++++++++++++------
 ui/sdl2.c                   |  4 +--
 util/cutils.c               | 53 ++++++++++++++++++-------------------
 20 files changed, 132 insertions(+), 123 deletions(-)

Comments

Paolo Bonzini June 15, 2022, 7:27 p.m. UTC | #1

On 6/15/22 17:56, Akihiko Odaki wrote:
> Developers often run QEMU without installing. The bundle mechanism
> allows to look up files which should be present in installation even in
> such a situation.
> 
> It is a general mechanism and can find any files located relative
> to the installation tree. The build tree must have a new directory,
> qemu-bundle, to represent what files the installation tree would
> have for reference by the executables.

Thanks for prototyping this, I think this is appealing.  I'll take a 
look later at using meson introspection info to build the preinstall layout.

Paolo

Paolo Bonzini June 16, 2022, 9:18 a.m. UTC | #2

On 6/15/22 21:27, Paolo Bonzini wrote:
> 
> Thanks for prototyping this, I think this is appealing.  I'll take a 
> look later at using meson introspection info to build the preinstall 
> layout.

Something like this:

diff --git a/meson.build b/meson.build
index 0458b69cdf..17023d511a 100644
--- a/meson.build
+++ b/meson.build
@@ -3612,6 +3612,21 @@ if host_machine.system() == 'windows'
    alias_target('installer', nsis)
  endif

+#####################
+# Preinstalled data #
+#####################
+
+qemu_bundle_sources_stamp = configure_file(
+  output: 'qemu-bundle-sources.stamp',
+  command: ['touch', '@OUTPUT'])
+custom_target('tree with preinstalled data files',
+              build_by_default: true,
+              output: 'qemu-bundle.stamp',
+              input: qemu_bundle_sources_stamp,
+              env: {'MESON': config_host['MESON']},
+              command: files('symlink-install-tree.py'),
+              capture: true)
+
  #########################
  # Configuration summary #
  #########################
diff --git a/scripts/symlink-install-tree.py 
b/scripts/symlink-install-tree.py
new file mode 100644
index 0000000000..cd66dc06cd
--- /dev/null
+++ b/scripts/symlink-install-tree.py
@@ -0,0 +1,34 @@
+#! /usr/bin/env python3
+
+import json
+import os
+import subprocess
+
+def destdir_join(d1: str, d2: str) -> str:
+    if not d1:
+        return d2
+    if not os.path.isabs(d2):
+        return os.path.join(d1, d2)
+
+    # c:\destdir + c:\prefix must produce c:\destdir\prefix
+    if len(d2) > 1 and d2[1] == ':':
+        return d1 + d2[2:]
+    return d1 + d2
+
+meson = os.environ.get("MESON")
+out = subprocess.run([meson, 'introspect', '--installed'],
+                     stdout=subprocess.PIPE, check=True).stdout
+for source, dest in json.loads(out).items():
+    assert os.path.isabs(source)
+    bundle_dest = destdir_join('qemu-bundle', dest)
+    path = os.path.dirname(bundle_dest)
+    try:
+        os.makedirs(path, exist_ok=True)
+    except e:
+        print('error making directory {path}', file=sys.stderr)
+        raise e
+    try:
+        os.symlink(source, bundle_dest)
+    except e:
+        print('error making symbolic link {dest}', file=sys.stderr)
+        raise e

Akihiko Odaki June 24, 2022, 4:33 p.m. UTC | #3

On 2022/06/16 18:18, Paolo Bonzini wrote:
> +def destdir_join(d1: str, d2: str) -> str:
> +    if not d1:
> +        return d2
> +    if not os.path.isabs(d2):
> +        return os.path.join(d1, d2)
> +
> +    # c:\destdir + c:\prefix must produce c:\destdir\prefix
> +    if len(d2) > 1 and d2[1] == ':':
> +        return d1 + d2[2:]
> +    return d1 + d2

This is from Meson but buggy so I fixed it and opened a pull request for 
Meson:
https://github.com/mesonbuild/meson/pull/10531

The script included in v8 has the fixed version of destdir_join.

Regards,
Akihiko Odaki