| Message ID | 20241025160209.194307-23-pbonzini@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | rust: fix CI + allow older versions of rustc and bindgen | expand |
Hi Paolo, On 10/25/24 09:02, Paolo Bonzini wrote: > Because Ubuntu 22.04 has a very old version of bindgen, that > does not have the important option --allowlist-file, it will > not be able to use --enable-rust out of the box. Instead, > install the latest version of bindgen-cli via "cargo install" > in the container, following QEMU's own documentation. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/docker/dockerfiles/ubuntu2204.docker | 5 +++++ > tests/lcitool/mappings.yml | 4 ++++ > tests/lcitool/refresh | 11 ++++++++++- > 3 files changed, 19 insertions(+), 1 deletion(-) > > diff --git a/tests/docker/dockerfiles/ubuntu2204.docker b/tests/docker/dockerfiles/ubuntu2204.docker > index ce3aa39d4f3..245ac879622 100644 > --- a/tests/docker/dockerfiles/ubuntu2204.docker > +++ b/tests/docker/dockerfiles/ubuntu2204.docker > @@ -149,6 +149,11 @@ ENV LANG "en_US.UTF-8" > ENV MAKE "/usr/bin/make" > ENV NINJA "/usr/bin/ninja" > ENV PYTHON "/usr/bin/python3" > +ENV CARGO_HOME=/usr/local/cargo > +ENV PATH=$CARGO_HOME/bin:$PATH > +RUN DEBIAN_FRONTEND=noninteractive eatmydata \ > + apt install -y --no-install-recommends cargo > +RUN cargo install bindgen-cli > # As a final step configure the user (if env is defined) > ARG USER > ARG UID > diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml > index 9c5ac87c1c2..c90b23a00f1 100644 > --- a/tests/lcitool/mappings.yml > +++ b/tests/lcitool/mappings.yml > @@ -1,4 +1,8 @@ > mappings: > + # Too old on Ubuntu 22.04; we install it from cargo instead > + bindgen: > + Ubuntu2204: > + > flake8: > OpenSUSELeap15: > > diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh > index 0f16f4d525c..a46cbbdca41 100755 > --- a/tests/lcitool/refresh > +++ b/tests/lcitool/refresh > @@ -137,6 +137,14 @@ fedora_rustup_nightly_extras = [ > 'RUN /usr/local/cargo/bin/rustup run nightly cargo install bindgen-cli\n', > ] > > +ubuntu2204_bindgen_extras = [ > + "ENV CARGO_HOME=/usr/local/cargo\n", > + 'ENV PATH=$CARGO_HOME/bin:$PATH\n', > + "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n", > + " apt install -y --no-install-recommends cargo\n", > + 'RUN cargo install bindgen-cli\n', > +] > + > def cross_build(prefix, targets): > conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix) > targets = "ENV DEF_TARGET_LIST %s\n" % (targets) > @@ -157,7 +165,8 @@ try: > trailer="".join(debian12_extras)) > generate_dockerfile("fedora", "fedora-40") > generate_dockerfile("opensuse-leap", "opensuse-leap-15") > - generate_dockerfile("ubuntu2204", "ubuntu-2204") > + generate_dockerfile("ubuntu2204", "ubuntu-2204", > + trailer="".join(ubuntu2204_bindgen_extras)) > > # > # Non-fatal Rust-enabled build Should we install the same version as the minimal one we expect (0.60, in debian 12)? All the rest of series is focused on having fixed minimal version, and this patch seems to escape this rule. Note: we can still install it using cargo, but just having a fixed version would be better.
Il ven 25 ott 2024, 20:51 Pierrick Bouvier <pierrick.bouvier@linaro.org> ha scritto: > Hi Paolo, > > On 10/25/24 09:02, Paolo Bonzini wrote: > > Because Ubuntu 22.04 has a very old version of bindgen, that > > does not have the important option --allowlist-file, it will > > not be able to use --enable-rust out of the box. Instead, > > install the latest version of bindgen-cli via "cargo install" > > in the container, following QEMU's own documentation. > > > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > > --- > > tests/docker/dockerfiles/ubuntu2204.docker | 5 +++++ > > tests/lcitool/mappings.yml | 4 ++++ > > tests/lcitool/refresh | 11 ++++++++++- > > 3 files changed, 19 insertions(+), 1 deletion(-) > > > > diff --git a/tests/docker/dockerfiles/ubuntu2204.docker > b/tests/docker/dockerfiles/ubuntu2204.docker > > index ce3aa39d4f3..245ac879622 100644 > > --- a/tests/docker/dockerfiles/ubuntu2204.docker > > +++ b/tests/docker/dockerfiles/ubuntu2204.docker > > @@ -149,6 +149,11 @@ ENV LANG "en_US.UTF-8" > > ENV MAKE "/usr/bin/make" > > ENV NINJA "/usr/bin/ninja" > > ENV PYTHON "/usr/bin/python3" > > +ENV CARGO_HOME=/usr/local/cargo > > +ENV PATH=$CARGO_HOME/bin:$PATH > > +RUN DEBIAN_FRONTEND=noninteractive eatmydata \ > > + apt install -y --no-install-recommends cargo > > +RUN cargo install bindgen-cli > > # As a final step configure the user (if env is defined) > > ARG USER > > ARG UID > > diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml > > index 9c5ac87c1c2..c90b23a00f1 100644 > > --- a/tests/lcitool/mappings.yml > > +++ b/tests/lcitool/mappings.yml > > @@ -1,4 +1,8 @@ > > mappings: > > + # Too old on Ubuntu 22.04; we install it from cargo instead > > + bindgen: > > + Ubuntu2204: > > + > > flake8: > > OpenSUSELeap15: > > > > diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh > > index 0f16f4d525c..a46cbbdca41 100755 > > --- a/tests/lcitool/refresh > > +++ b/tests/lcitool/refresh > > @@ -137,6 +137,14 @@ fedora_rustup_nightly_extras = [ > > 'RUN /usr/local/cargo/bin/rustup run nightly cargo install > bindgen-cli\n', > > ] > > > > +ubuntu2204_bindgen_extras = [ > > + "ENV CARGO_HOME=/usr/local/cargo\n", > > + 'ENV PATH=$CARGO_HOME/bin:$PATH\n', > > + "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n", > > + " apt install -y --no-install-recommends cargo\n", > > + 'RUN cargo install bindgen-cli\n', > > +] > > + > > def cross_build(prefix, targets): > > conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix) > > targets = "ENV DEF_TARGET_LIST %s\n" % (targets) > > @@ -157,7 +165,8 @@ try: > > trailer="".join(debian12_extras)) > > generate_dockerfile("fedora", "fedora-40") > > generate_dockerfile("opensuse-leap", "opensuse-leap-15") > > - generate_dockerfile("ubuntu2204", "ubuntu-2204") > > + generate_dockerfile("ubuntu2204", "ubuntu-2204", > > + trailer="".join(ubuntu2204_bindgen_extras)) > > > > # > > # Non-fatal Rust-enabled build > > Should we install the same version as the minimal one we expect (0.60, > in debian 12)? All the rest of series is focused on having fixed minimal > version, and this patch seems to escape this rule. > But in the end the operation of bindgen is quite deterministic, so if the coverage is improved we can indeed install 0.60.x. For example, if we think that user on Debian 12 might use distro bindgen together with a recent rustc (in their case, rustup-installed), then installing bindgen 0.60.x on Ubuntu would provide a similar effect. On the other hand I expect that users will just do "cargo install bindgen-cli", and Ubuntu is a pretty common distro, so that's what I went for here. Paolo Note: we can still install it using cargo, but just having a fixed > version would be better. > >
On 10/25/24 12:35, Paolo Bonzini wrote: > > > Il ven 25 ott 2024, 20:51 Pierrick Bouvier <pierrick.bouvier@linaro.org > <mailto:pierrick.bouvier@linaro.org>> ha scritto: > > Hi Paolo, > > On 10/25/24 09:02, Paolo Bonzini wrote: > > Because Ubuntu 22.04 has a very old version of bindgen, that > > does not have the important option --allowlist-file, it will > > not be able to use --enable-rust out of the box. Instead, > > install the latest version of bindgen-cli via "cargo install" > > in the container, following QEMU's own documentation. > > > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com > <mailto:pbonzini@redhat.com>> > > --- > > tests/docker/dockerfiles/ubuntu2204.docker | 5 +++++ > > tests/lcitool/mappings.yml | 4 ++++ > > tests/lcitool/refresh | 11 ++++++++++- > > 3 files changed, 19 insertions(+), 1 deletion(-) > > > > diff --git a/tests/docker/dockerfiles/ubuntu2204.docker > b/tests/docker/dockerfiles/ubuntu2204.docker > > index ce3aa39d4f3..245ac879622 100644 > > --- a/tests/docker/dockerfiles/ubuntu2204.docker > > +++ b/tests/docker/dockerfiles/ubuntu2204.docker > > @@ -149,6 +149,11 @@ ENV LANG "en_US.UTF-8" > > ENV MAKE "/usr/bin/make" > > ENV NINJA "/usr/bin/ninja" > > ENV PYTHON "/usr/bin/python3" > > +ENV CARGO_HOME=/usr/local/cargo > > +ENV PATH=$CARGO_HOME/bin:$PATH > > +RUN DEBIAN_FRONTEND=noninteractive eatmydata \ > > + apt install -y --no-install-recommends cargo > > +RUN cargo install bindgen-cli > > # As a final step configure the user (if env is defined) > > ARG USER > > ARG UID > > diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml > > index 9c5ac87c1c2..c90b23a00f1 100644 > > --- a/tests/lcitool/mappings.yml > > +++ b/tests/lcitool/mappings.yml > > @@ -1,4 +1,8 @@ > > mappings: > > + # Too old on Ubuntu 22.04; we install it from cargo instead > > + bindgen: > > + Ubuntu2204: > > + > > flake8: > > OpenSUSELeap15: > > > > diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh > > index 0f16f4d525c..a46cbbdca41 100755 > > --- a/tests/lcitool/refresh > > +++ b/tests/lcitool/refresh > > @@ -137,6 +137,14 @@ fedora_rustup_nightly_extras = [ > > 'RUN /usr/local/cargo/bin/rustup run nightly cargo install > bindgen-cli\n', > > ] > > > > +ubuntu2204_bindgen_extras = [ > > + "ENV CARGO_HOME=/usr/local/cargo\n", > > + 'ENV PATH=$CARGO_HOME/bin:$PATH\n', > > + "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n", > > + " apt install -y --no-install-recommends cargo\n", > > + 'RUN cargo install bindgen-cli\n', > > +] > > + > > def cross_build(prefix, targets): > > conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix) > > targets = "ENV DEF_TARGET_LIST %s\n" % (targets) > > @@ -157,7 +165,8 @@ try: > > trailer="".join(debian12_extras)) > > generate_dockerfile("fedora", "fedora-40") > > generate_dockerfile("opensuse-leap", "opensuse-leap-15") > > - generate_dockerfile("ubuntu2204", "ubuntu-2204") > > + generate_dockerfile("ubuntu2204", "ubuntu-2204", > > + trailer="".join(ubuntu2204_bindgen_extras)) > > > > # > > # Non-fatal Rust-enabled build > > Should we install the same version as the minimal one we expect (0.60, > in debian 12)? All the rest of series is focused on having fixed > minimal version, and > > this patch seems to escape this rule. > > > But in the end the operation of bindgen is quite deterministic, so if > the coverage is improved we can indeed install 0.60.x. For example, if > we think that user on Debian 12 might use distro bindgen together with a > recent rustc (in their case, rustup-installed), then installing bindgen > 0.60.x on Ubuntu would provide a similar effect. > I missed that the debian job covers this use case. So indeed, we can use a recent one on ubuntu. Where is the change for the debian container? > On the other hand I expect that users will just do "cargo install > bindgen-cli", and Ubuntu is a pretty common distro, so that's what I > went for here. > It's a reasonable expectation indeed. > Paolo > > Note: we can still install it using cargo, but just having a fixed > version would be better. >
Il ven 25 ott 2024, 21:47 Pierrick Bouvier <pierrick.bouvier@linaro.org> ha scritto: > > in the end the operation of bindgen is quite deterministic, so if > > the coverage is improved we can indeed install 0.60.x. For example, if > > we think that user on Debian 12 might use distro bindgen together with a > > recent rustc (in their case, rustup-installed), then installing bindgen > > 0.60.x on Ubuntu would provide a similar effect. > > > > I missed that the debian job covers this use case. So indeed, we can use > a recent one on ubuntu. > > Where is the change for the debian container? > Here: https://lore.kernel.org/qemu-devel/20241015133925.311587-2-berrange@redhat.com/ Without this series, that patch was already installing bindgen and rustc to test that --enable-rust was not enabled implicitly. Paolo > On the other hand I expect that users will just do "cargo install > > bindgen-cli", and Ubuntu is a pretty common distro, so that's what I > > went for here. > > It's a reasonable expectation indeed. >
On 10/25/24 09:02, Paolo Bonzini wrote: > Because Ubuntu 22.04 has a very old version of bindgen, that > does not have the important option --allowlist-file, it will > not be able to use --enable-rust out of the box. Instead, > install the latest version of bindgen-cli via "cargo install" > in the container, following QEMU's own documentation. > > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> > --- > tests/docker/dockerfiles/ubuntu2204.docker | 5 +++++ > tests/lcitool/mappings.yml | 4 ++++ > tests/lcitool/refresh | 11 ++++++++++- > 3 files changed, 19 insertions(+), 1 deletion(-) > > diff --git a/tests/docker/dockerfiles/ubuntu2204.docker b/tests/docker/dockerfiles/ubuntu2204.docker > index ce3aa39d4f3..245ac879622 100644 > --- a/tests/docker/dockerfiles/ubuntu2204.docker > +++ b/tests/docker/dockerfiles/ubuntu2204.docker > @@ -149,6 +149,11 @@ ENV LANG "en_US.UTF-8" > ENV MAKE "/usr/bin/make" > ENV NINJA "/usr/bin/ninja" > ENV PYTHON "/usr/bin/python3" > +ENV CARGO_HOME=/usr/local/cargo > +ENV PATH=$CARGO_HOME/bin:$PATH > +RUN DEBIAN_FRONTEND=noninteractive eatmydata \ > + apt install -y --no-install-recommends cargo > +RUN cargo install bindgen-cli > # As a final step configure the user (if env is defined) > ARG USER > ARG UID > diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml > index 9c5ac87c1c2..c90b23a00f1 100644 > --- a/tests/lcitool/mappings.yml > +++ b/tests/lcitool/mappings.yml > @@ -1,4 +1,8 @@ > mappings: > + # Too old on Ubuntu 22.04; we install it from cargo instead > + bindgen: > + Ubuntu2204: > + > flake8: > OpenSUSELeap15: > > diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh > index 0f16f4d525c..a46cbbdca41 100755 > --- a/tests/lcitool/refresh > +++ b/tests/lcitool/refresh > @@ -137,6 +137,14 @@ fedora_rustup_nightly_extras = [ > 'RUN /usr/local/cargo/bin/rustup run nightly cargo install bindgen-cli\n', > ] > > +ubuntu2204_bindgen_extras = [ > + "ENV CARGO_HOME=/usr/local/cargo\n", > + 'ENV PATH=$CARGO_HOME/bin:$PATH\n', > + "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n", > + " apt install -y --no-install-recommends cargo\n", > + 'RUN cargo install bindgen-cli\n', > +] > + > def cross_build(prefix, targets): > conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix) > targets = "ENV DEF_TARGET_LIST %s\n" % (targets) > @@ -157,7 +165,8 @@ try: > trailer="".join(debian12_extras)) > generate_dockerfile("fedora", "fedora-40") > generate_dockerfile("opensuse-leap", "opensuse-leap-15") > - generate_dockerfile("ubuntu2204", "ubuntu-2204") > + generate_dockerfile("ubuntu2204", "ubuntu-2204", > + trailer="".join(ubuntu2204_bindgen_extras)) > > # > # Non-fatal Rust-enabled build Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
On 10/25/24 13:08, Paolo Bonzini wrote: > > > Il ven 25 ott 2024, 21:47 Pierrick Bouvier <pierrick.bouvier@linaro.org > <mailto:pierrick.bouvier@linaro.org>> ha scritto: > > > in the end the operation of bindgen is quite deterministic, so if > > the coverage is improved we can indeed install 0.60.x. For > example, if > > we think that user on Debian 12 might use distro bindgen together > with a > > recent rustc (in their case, rustup-installed), then installing > bindgen > > 0.60.x on Ubuntu would provide a similar effect. > > > > I missed that the debian job covers this use case. So indeed, we can > use > a recent one on ubuntu. > > Where is the change for the debian container? > > > Here: > https://lore.kernel.org/qemu-devel/20241015133925.311587-2-berrange@redhat.com/ <https://lore.kernel.org/qemu-devel/20241015133925.311587-2-berrange@redhat.com/> > Good. As there is no based-on tag in the cover letter, I expected this to apply on current master. > Without this series, that patch was already installing bindgen and rustc > to test that --enable-rust was not enabled implicitly. > > Paolo > > > On the other hand I expect that users will just do "cargo install > > bindgen-cli", and Ubuntu is a pretty common distro, so that's what I > > went for here. > > It's a reasonable expectation indeed. >
Il ven 25 ott 2024, 22:14 Pierrick Bouvier <pierrick.bouvier@linaro.org> ha scritto: > > Here: > > > https://lore.kernel.org/qemu-devel/20241015133925.311587-2-berrange@redhat.com/ > > Good. As there is no based-on tag in the cover letter, I expected this > to apply on current master. > Yes, it's based on my next pull request but I didn't have time to post it yet (see cover letter). Paolo > > Without this series, that patch was already installing bindgen and rustc > > to test that --enable-rust was not enabled implicitly. > > > > Paolo > > > > > On the other hand I expect that users will just do "cargo install > > > bindgen-cli", and Ubuntu is a pretty common distro, so that's > what I > > > went for here. > > > > It's a reasonable expectation indeed. > > >
diff --git a/tests/docker/dockerfiles/ubuntu2204.docker b/tests/docker/dockerfiles/ubuntu2204.docker index ce3aa39d4f3..245ac879622 100644 --- a/tests/docker/dockerfiles/ubuntu2204.docker +++ b/tests/docker/dockerfiles/ubuntu2204.docker @@ -149,6 +149,11 @@ ENV LANG "en_US.UTF-8" ENV MAKE "/usr/bin/make" ENV NINJA "/usr/bin/ninja" ENV PYTHON "/usr/bin/python3" +ENV CARGO_HOME=/usr/local/cargo +ENV PATH=$CARGO_HOME/bin:$PATH +RUN DEBIAN_FRONTEND=noninteractive eatmydata \ + apt install -y --no-install-recommends cargo +RUN cargo install bindgen-cli # As a final step configure the user (if env is defined) ARG USER ARG UID diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml index 9c5ac87c1c2..c90b23a00f1 100644 --- a/tests/lcitool/mappings.yml +++ b/tests/lcitool/mappings.yml @@ -1,4 +1,8 @@ mappings: + # Too old on Ubuntu 22.04; we install it from cargo instead + bindgen: + Ubuntu2204: + flake8: OpenSUSELeap15: diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh index 0f16f4d525c..a46cbbdca41 100755 --- a/tests/lcitool/refresh +++ b/tests/lcitool/refresh @@ -137,6 +137,14 @@ fedora_rustup_nightly_extras = [ 'RUN /usr/local/cargo/bin/rustup run nightly cargo install bindgen-cli\n', ] +ubuntu2204_bindgen_extras = [ + "ENV CARGO_HOME=/usr/local/cargo\n", + 'ENV PATH=$CARGO_HOME/bin:$PATH\n', + "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n", + " apt install -y --no-install-recommends cargo\n", + 'RUN cargo install bindgen-cli\n', +] + def cross_build(prefix, targets): conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix) targets = "ENV DEF_TARGET_LIST %s\n" % (targets) @@ -157,7 +165,8 @@ try: trailer="".join(debian12_extras)) generate_dockerfile("fedora", "fedora-40") generate_dockerfile("opensuse-leap", "opensuse-leap-15") - generate_dockerfile("ubuntu2204", "ubuntu-2204") + generate_dockerfile("ubuntu2204", "ubuntu-2204", + trailer="".join(ubuntu2204_bindgen_extras)) # # Non-fatal Rust-enabled build
Because Ubuntu 22.04 has a very old version of bindgen, that does not have the important option --allowlist-file, it will not be able to use --enable-rust out of the box. Instead, install the latest version of bindgen-cli via "cargo install" in the container, following QEMU's own documentation. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> --- tests/docker/dockerfiles/ubuntu2204.docker | 5 +++++ tests/lcitool/mappings.yml | 4 ++++ tests/lcitool/refresh | 11 ++++++++++- 3 files changed, 19 insertions(+), 1 deletion(-)