diff mbox series

[RFC,v2,2/5] rust: add bindgen step as a meson dependency

Message ID ef980fb29deb81d574a7301365d9b9db72c015eb.1718101832.git.manos.pitsidianakis@linaro.org (mailing list archive)
State New, archived
Headers show
Series Implement ARM PL011 in Rust | expand

Commit Message

Manos Pitsidianakis June 11, 2024, 10:33 a.m. UTC
Add mechanism to generate rust hw targets that depend on a custom
bindgen target for rust bindings to C.

This way bindings will be created before the rust crate is compiled.

The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name
as a target:

ninja aarch64-softmmu-generated.rs

The way the bindings are generated is:

1. All required C headers are included in a single file, in our case
   rust/wrapper.h for convenience. Otherwise we'd have to provide a list
   of headers every time to the bindgen tool.

2. Meson creates a generated_rs target that runs bindgen making sure
   the architecture etc header dependencies are present.

3. The generated_rs target takes a list of files, type symbols,
   function symbols to block from being generated. This is not necessary
   for the bindings to work, but saves us time and space.

4. Meson creates rust hardware target dependencies from the rust_targets
   dictionary defined in rust/meson.build.

   Since we cannot declare a dependency on generated_rs before it is
   declared in meson.build, the rust crate targets must be defined after
   the generated_rs target for each target architecture is defined. This
   way meson sets up the dependency tree properly.

5. After compiling each rust crate with the cargo_wrapper.py script,
   its static library artifact is linked as a `whole-archive` with the
   final binary.

Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
---
 MAINTAINERS              |  2 +
 meson.build              | 87 ++++++++++++++++++++++++++++++++++++++
 rust/meson.build         | 91 ++++++++++++++++++++++++++++++++++++++++
 rust/wrapper.h           | 39 +++++++++++++++++
 scripts/cargo_wrapper.py | 10 +++++
 5 files changed, 229 insertions(+)
 create mode 100644 rust/meson.build
 create mode 100644 rust/wrapper.h

Comments

Paolo Bonzini June 17, 2024, 9:01 p.m. UTC | #1
Just one somewhat larger request, otherwise just a collection of ideas.

On Tue, Jun 11, 2024 at 12:34 PM Manos Pitsidianakis
<manos.pitsidianakis@linaro.org> wrote:
> diff --git a/rust/meson.build b/rust/meson.build
> new file mode 100644
> index 0000000000..e9660a3045
> --- /dev/null
> +++ b/rust/meson.build
> @@ -0,0 +1,91 @@
> +rust_targets = {}
> +
> +cargo_wrapper = [
> +  find_program(meson.global_source_root() / 'scripts/cargo_wrapper.py'),
> +  '--config-headers', meson.project_build_root() / 'config-host.h',
> +  '--meson-build-root', meson.project_build_root(),
> +  '--meson-build-dir', meson.current_build_dir(),
> +  '--meson-source-dir', meson.current_source_dir(),
> +]
>
> +
> +# TODO: verify rust_target_triple if given as an option
> +if rust_target_triple == ''
> +  if not supported_oses.contains(host_os)
> +    message()
> +    error('QEMU does not support `' + host_os +'` as a Rust platform.')
> +  elif not supported_cpus.contains(host_arch)
> +    message()
> +    error('QEMU does not support `' + host_arch +'` as a Rust architecture.')
> +  endif
> +  rust_target_triple = host_arch + rust_supported_oses[host_os]
> +  if host_os == 'windows' and host_arch == 'aarch64'
> +    rust_target_triple += 'llvm'
> +  endif
> +endif
> +
> +if get_option('optimization') in ['0', '1', 'g']
> +  rs_build_type = 'debug'
> +else
> +  rs_build_type = 'release'
> +endif
> +
> +rust_hw_target_list = {}
> +
> +foreach rust_hw_target, rust_hws: rust_hw_target_list
> +  foreach rust_hw_dev: rust_hws
> +    output = meson.current_build_dir() / rust_target_triple / rs_build_type / rust_hw_dev['output']
> +    crate_metadata = {
> +      'name': rust_hw_dev['name'],
> +      'output': [rust_hw_dev['output']],
> +      'output-path': output,
> +      'command': [cargo_wrapper,
> +        '--crate-dir',
> +        meson.current_source_dir() / rust_hw_dev['dirname'],
> +        '--profile',
> +        rs_build_type,
> +        '--target-triple',
> +        rust_target_triple,
> +        '--outdir',
> +        '@OUTDIR@',
> +        'build-lib'

Probably needs to add config-devices.h as well to --config-headers? I
think that we can have just one crate that is built per target,
instead of many small crates like your rust_pl011_cargo. And then that
crate is built many times with a rust/src/ hierarchy that resembles
the meson.build files we use for C.

rust/src/mod.rs:
pub mod hw;

rust/src/hw/mod.rs:
pub mod char;

rust/src/hw/char/mod.rs:
#[cfg(feature = "CONFIG_PL011")] pub mod pl011;

(perhaps just src/? And we could even move all sources down one level
to src/ for QEMU as well? But that can be done later, for now it can
be rust/src/). This basically gives Kconfig integration for free if it
works, so I'd ask you to try doing this for the next version?

Also, sooner or later we'll have to tackle building a set of common
dependencies (similar to common_ss), and then bringing those as a
dependency to the build of the per-target crates. Even if we don't get
to the point of building devices once for all targets, I'd rather at
least avoid having to build a dozen times the common deps of
procedural macros.

This is not trivial, but should not be hard either, by using build.rs
(cargo::rustc-link-search) to add the -Ldependency=/path/to/rlibs
option to the per-target rustc invocations. This may also require
grabbing the compilation log via "cargo build
--message-format=json-render-diagnostics", but that's not hard to do
in cargo_wrapper.py. And unlike the previous request about Kconfig, it
can be done after the first merge.

> +#include "qemu/osdep.h"
> +#include "qemu/module.h"
> +#include "qemu-io.h"
> +#include "sysemu/sysemu.h"
> +#include "hw/sysbus.h"
> +#include "exec/memory.h"
> +#include "chardev/char-fe.h"
> +#include "hw/clock.h"
> +#include "hw/qdev-clock.h"
> +#include "hw/qdev-properties.h"
> +#include "hw/qdev-properties-system.h"
> +#include "hw/irq.h"
> +#include "qapi/error.h"
> +#include "migration/vmstate.h"
> +#include "chardev/char-serial.h"

Please check for any headers included indirectly and whether it's
worth including them here (e.g. it seems that qapi/error.h,
qom/object.h, hw/qdev-core.h should be there).

> diff --git a/scripts/cargo_wrapper.py b/scripts/cargo_wrapper.py
> index d338effdaa..a36a4fc86d 100644
> --- a/scripts/cargo_wrapper.py
> +++ b/scripts/cargo_wrapper.py
> @@ -94,6 +94,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str]
>
>      env = os.environ
>      env["CARGO_ENCODED_RUSTFLAGS"] = cfg
> +    env["MESON_BUILD_DIR"] = str(target_dir)
> +    env["MESON_BUILD_ROOT"] = str(args.meson_build_root)
>
>      return (env, cargo_cmd)
>
> @@ -164,6 +166,14 @@ def main() -> None:
>          required=True,
>      )
>      parser.add_argument(
> +        "--meson-build-root",
> +        metavar="BUILD_ROOT",
> +        help="meson.project_build_root()",
> +        type=pathlib.Path,
> +        dest="meson_build_root",
> +        required=True,
> +    )
> +    parser.add_argument(
>          "--meson-source-dir",
>          metavar="SOURCE_DIR",
>          help="meson.current_source_dir()",

Probably all of cargo_wrapper.py can be moved to this patch.

Paolo
diff mbox series

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index 431010ddbf..ff6117f41d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4227,6 +4227,8 @@  Rust build system integration
 M: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
 S: Maintained
 F: scripts/cargo_wrapper.py
+F: rust/meson.build
+F: rust/wrapper.h
 
 Miscellaneous
 -------------
diff --git a/meson.build b/meson.build
index 3533889852..a08c975ef9 100644
--- a/meson.build
+++ b/meson.build
@@ -3876,6 +3876,93 @@  foreach target : target_dirs
     lib_deps += dep.partial_dependency(compile_args: true, includes: true)
   endforeach
 
+  if with_rust and target_type == 'system'
+      rust_bindgen = import('unstable-rust')
+      rust_bindgen_dep = declare_dependency(
+        dependencies: arch_deps + lib_deps,
+        include_directories: target_inc,
+        sources: arch_srcs + genh)
+
+      # We need one generated_rs target per target, so give them
+      # target-specific names
+      copy = fs.copyfile('rust/wrapper.h', target + '_wrapper.h')
+      generated_rs = rust_bindgen.bindgen(
+        input: copy,
+        dependencies: arch_deps + rust_bindgen_dep,
+        output: target + '-generated.rs',
+        include_directories: target_inc + include_directories('.') + include_directories('include'),
+        args: [
+          '--formatter',
+          'rustfmt',
+          '--no-doc-comments',
+          '--merge-extern-blocks',
+          '--generate-block',
+          '--generate-cstr',
+          '--impl-debug',
+          '--with-derive-default',
+          '--use-core',
+          '--ctypes-prefix',
+          'core::ffi',
+          '--blocklist-var',
+          'IPPORT_RESERVED',
+          '--blocklist-type',
+          '^.+_.*autoptr$',
+          '--blocklist-type',
+          'max_align_t',
+          '--blocklist-function',
+          'str*',
+          '--blocklist-function',
+          'ato*',
+          '--blocklist-function',
+          'p?select',
+          '--blocklist-file',
+          '.+inttypes.h$',
+          '--blocklist-file',
+          '.+limits.h$',
+          '--blocklist-file',
+          '.+ctype.h$',
+          '--blocklist-file',
+          '.+errno.h$',
+          '--blocklist-file',
+          '.+fcntl.h$',
+          '--blocklist-file',
+          '.+getopt.h$',
+          '--blocklist-file',
+          '.+assert.h$',
+          '--blocklist-file',
+          '.+stdbool.h$',
+          '--blocklist-file',
+          '.+stdio.h$',
+          '--blocklist-file',
+          '.+stdint.h$',
+          '--blocklist-file',
+          '.+stdalign.h$',
+        ],
+        c_args: c_args,
+      )
+
+      if target in rust_targets
+        rust_hw = ss.source_set()
+        foreach t: rust_targets[target]
+          rust_device_cargo = custom_target(t['name'],
+                                       output: t['output'],
+                                       depends: [generated_rs],
+                                       build_always_stale: true,
+                                       command: t['command'])
+          rust_dep = declare_dependency(link_args: [
+                                          '-Wl,--whole-archive',
+                                          t['output-path'],
+                                          '-Wl,--no-whole-archive'
+                                          ],
+                                          sources: [rust_device_cargo])
+          rust_hw.add(rust_dep)
+        endforeach
+        rust_hw_config = rust_hw.apply(config_target, strict: false)
+        arch_srcs += rust_hw_config.sources()
+        arch_deps += rust_hw_config.dependencies()
+      endif
+  endif
+
   lib = static_library('qemu-' + target,
                  sources: arch_srcs + genh,
                  dependencies: lib_deps,
diff --git a/rust/meson.build b/rust/meson.build
new file mode 100644
index 0000000000..e9660a3045
--- /dev/null
+++ b/rust/meson.build
@@ -0,0 +1,91 @@ 
+rust_targets = {}
+
+cargo_wrapper = [
+  find_program(meson.global_source_root() / 'scripts/cargo_wrapper.py'),
+  '--config-headers', meson.project_build_root() / 'config-host.h',
+  '--meson-build-root', meson.project_build_root(),
+  '--meson-build-dir', meson.current_build_dir(),
+  '--meson-source-dir', meson.current_source_dir(),
+]
+
+if get_option('b_colorout') != 'never'
+  cargo_wrapper += ['--color', 'always']
+endif
+
+rust_supported_oses = {'linux': '-unknown-linux-gnu', 'darwin': '-apple-darwin', 'windows': '-pc-windows-gnu'}
+rust_supported_cpus = ['x86_64', 'aarch64']
+
+# Future-proof the above definitions against any change in the root meson.build file:
+foreach rust_os: rust_supported_oses.keys()
+  if not supported_oses.contains(rust_os)
+    message()
+    warning('UNSUPPORTED OS VALUES IN ' + meson.current_source_dir() + '/meson.build')
+    message()
+    message('This meson.build file claims OS `+' + rust_os + '` is supported but')
+    message('it is not included in the global supported OSes list in')
+    message(meson.source_root() + '/meson.build.')
+  endif
+endforeach
+foreach rust_cpu: rust_supported_cpus
+  if not supported_cpus.contains(rust_cpu)
+    message()
+    warning('UNSUPPORTED CPU VALUES IN ' + meson.current_source_dir() + '/meson.build')
+    message()
+    message('This meson.build file claims CPU `+' + rust_cpu + '` is supported but')
+    message('it is not included in the global supported CPUs list in')
+    message(meson.source_root() + '/meson.build.')
+  endif
+endforeach
+
+if meson.is_cross_build()
+  message()
+  error('QEMU does not support cross compiling with Rust enabled.')
+endif
+
+rust_target_triple = get_option('with_rust_target_triple')
+
+# TODO: verify rust_target_triple if given as an option
+if rust_target_triple == ''
+  if not supported_oses.contains(host_os)
+    message()
+    error('QEMU does not support `' + host_os +'` as a Rust platform.')
+  elif not supported_cpus.contains(host_arch)
+    message()
+    error('QEMU does not support `' + host_arch +'` as a Rust architecture.')
+  endif
+  rust_target_triple = host_arch + rust_supported_oses[host_os]
+  if host_os == 'windows' and host_arch == 'aarch64'
+    rust_target_triple += 'llvm'
+  endif
+endif
+
+if get_option('optimization') in ['0', '1', 'g']
+  rs_build_type = 'debug'
+else
+  rs_build_type = 'release'
+endif
+
+rust_hw_target_list = {}
+
+foreach rust_hw_target, rust_hws: rust_hw_target_list
+  foreach rust_hw_dev: rust_hws
+    output = meson.current_build_dir() / rust_target_triple / rs_build_type / rust_hw_dev['output']
+    crate_metadata = {
+      'name': rust_hw_dev['name'],
+      'output': [rust_hw_dev['output']],
+      'output-path': output,
+      'command': [cargo_wrapper,
+        '--crate-dir',
+        meson.current_source_dir() / rust_hw_dev['dirname'],
+        '--profile',
+        rs_build_type,
+        '--target-triple',
+        rust_target_triple,
+        '--outdir',
+        '@OUTDIR@',
+        'build-lib'
+        ]
+      }
+    rust_targets += { rust_hw_target: [crate_metadata] }
+  endforeach
+endforeach
diff --git a/rust/wrapper.h b/rust/wrapper.h
new file mode 100644
index 0000000000..bcf808c8d7
--- /dev/null
+++ b/rust/wrapper.h
@@ -0,0 +1,39 @@ 
+/*
+ * QEMU System Emulator
+ *
+ * Copyright (c) 2003-2020 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/module.h"
+#include "qemu-io.h"
+#include "sysemu/sysemu.h"
+#include "hw/sysbus.h"
+#include "exec/memory.h"
+#include "chardev/char-fe.h"
+#include "hw/clock.h"
+#include "hw/qdev-clock.h"
+#include "hw/qdev-properties.h"
+#include "hw/qdev-properties-system.h"
+#include "hw/irq.h"
+#include "qapi/error.h"
+#include "migration/vmstate.h"
+#include "chardev/char-serial.h"
diff --git a/scripts/cargo_wrapper.py b/scripts/cargo_wrapper.py
index d338effdaa..a36a4fc86d 100644
--- a/scripts/cargo_wrapper.py
+++ b/scripts/cargo_wrapper.py
@@ -94,6 +94,8 @@  def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str]
 
     env = os.environ
     env["CARGO_ENCODED_RUSTFLAGS"] = cfg
+    env["MESON_BUILD_DIR"] = str(target_dir)
+    env["MESON_BUILD_ROOT"] = str(args.meson_build_root)
 
     return (env, cargo_cmd)
 
@@ -164,6 +166,14 @@  def main() -> None:
         required=True,
     )
     parser.add_argument(
+        "--meson-build-root",
+        metavar="BUILD_ROOT",
+        help="meson.project_build_root()",
+        type=pathlib.Path,
+        dest="meson_build_root",
+        required=True,
+    )
+    parser.add_argument(
         "--meson-source-dir",
         metavar="SOURCE_DIR",
         help="meson.current_source_dir()",