| Message ID | 20230918234412.363087-2-mmaurer@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] rust: Respect HOSTCC when linking for host | expand |
On 9/18/23 20:43, Matthew Maurer wrote: > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, > resulting in build failures in hermetic environments where `cc` does not > exist. This includes both hostprogs and proc-macros. > > Since we are setting the linker to `HOSTCC`, we set the linker flavor to > `gcc` explicitly. > > Signed-off-by: Matthew Maurer <mmaurer@google.com> > --- > > Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS > should be respected as well. > > I did not switch it to use HOSTLD for two reasons: > * That variable is not globally defined - it is only available in two > subdirectories of tools/ > * C host scripts are linked by HOSTCC as well, even when linking a > collection of object files. It *prints* HOSTLD, but invokes HOSTCC. > See scripts/Makefile.host cmd_host-cmulti for an example. > [...] Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
On Mon, Sep 18, 2023 at 4:44 PM Matthew Maurer <mmaurer@google.com> wrote: > > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, > resulting in build failures in hermetic environments where `cc` does not > exist. This includes both hostprogs and proc-macros. > > Since we are setting the linker to `HOSTCC`, we set the linker flavor to > `gcc` explicitly. > > Signed-off-by: Matthew Maurer <mmaurer@google.com> > --- > > Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS > should be respected as well. > > I did not switch it to use HOSTLD for two reasons: > * That variable is not globally defined - it is only available in two > subdirectories of tools/ > * C host scripts are linked by HOSTCC as well, even when linking a > collection of object files. It *prints* HOSTLD, but invokes HOSTCC. > See scripts/Makefile.host cmd_host-cmulti for an example. Sure, that makes sense to me, thanks for pointing that out. > > rust/Makefile | 4 ++++ > scripts/Makefile.host | 4 ++++ > 2 files changed, 8 insertions(+) > > diff --git a/rust/Makefile b/rust/Makefile > index 87958e864be0..b60b7eb8c5a0 100644 > --- a/rust/Makefile > +++ b/rust/Makefile > @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE > $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE > $(call if_changed,exports) > > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' I don't think we need to do this kind of escaping. If a linker flag is passed to the linker directly as the driver has spaces such as `-z relro`, if it is instead passed to the compiler as the driver will not contain spaces (`-Wl,-z,relro`). As such, I don't think we need this escaping (famous last words). Mind submitting a v3 without it? > + > quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ > cmd_rustc_procmacro = \ > $(RUSTC_OR_CLIPPY) $(rust_common_flags) \ > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ > --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ > --crate-type proc-macro \ > --crate-name $(patsubst lib%.so,%,$(notdir $@)) $< > diff --git a/scripts/Makefile.host b/scripts/Makefile.host > index 8f7f842b54f9..dc0410cae5ca 100644 > --- a/scripts/Makefile.host > +++ b/scripts/Makefile.host > @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ > $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ > $(HOSTCXXFLAGS_$(target-stem).o) > > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' > + > # `--out-dir` is required to avoid temporaries being created by `rustc` in the > # current working directory, which may be not accessible in the out-of-tree > # modules case. > hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ > $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ > $(HOSTRUSTFLAGS_$(target-stem)) > > -- > 2.42.0.459.ge4e396fd5e-goog >
On Tue, Sep 19, 2023 at 12:44 PM Nick Desaulniers <ndesaulniers@google.com> wrote: > > On Mon, Sep 18, 2023 at 4:44 PM Matthew Maurer <mmaurer@google.com> wrote: > > > > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, > > resulting in build failures in hermetic environments where `cc` does not > > exist. This includes both hostprogs and proc-macros. > > > > Since we are setting the linker to `HOSTCC`, we set the linker flavor to > > `gcc` explicitly. > > > > Signed-off-by: Matthew Maurer <mmaurer@google.com> > > --- > > > > Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS > > should be respected as well. > > > > I did not switch it to use HOSTLD for two reasons: > > * That variable is not globally defined - it is only available in two > > subdirectories of tools/ > > * C host scripts are linked by HOSTCC as well, even when linking a > > collection of object files. It *prints* HOSTLD, but invokes HOSTCC. > > See scripts/Makefile.host cmd_host-cmulti for an example. > > Sure, that makes sense to me, thanks for pointing that out. > > > > > rust/Makefile | 4 ++++ > > scripts/Makefile.host | 4 ++++ > > 2 files changed, 8 insertions(+) > > > > diff --git a/rust/Makefile b/rust/Makefile > > index 87958e864be0..b60b7eb8c5a0 100644 > > --- a/rust/Makefile > > +++ b/rust/Makefile > > @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE > > $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE > > $(call if_changed,exports) > > > > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' > > I don't think we need to do this kind of escaping. If a linker flag is > passed to the linker directly as the driver has spaces such as `-z > relro`, if it is instead passed to the compiler as the driver will not > contain spaces (`-Wl,-z,relro`). As such, I don't think we need this > escaping (famous last words). Mind submitting a v3 without it? Thanks for clarifying off list that: ``` It's not escaping the spaces, it's escaping any quotes example: -Clink-args='-Wl,-z,relro -Wl,-linker-plugin-lto' Without the quote, the second one would go to rustc rather than the linker ``` Sorry for misunderstanding what exactly we were escaping here and why; thanks for implementing my suggestion of passing along KBUILD_HOSTLDFLAGS though. That will avoid issues for Android downstream. Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> > > > + > > quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ > > cmd_rustc_procmacro = \ > > $(RUSTC_OR_CLIPPY) $(rust_common_flags) \ > > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ > > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ > > --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ > > --crate-type proc-macro \ > > --crate-name $(patsubst lib%.so,%,$(notdir $@)) $< > > diff --git a/scripts/Makefile.host b/scripts/Makefile.host > > index 8f7f842b54f9..dc0410cae5ca 100644 > > --- a/scripts/Makefile.host > > +++ b/scripts/Makefile.host > > @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ > > $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ > > $(HOSTCXXFLAGS_$(target-stem).o) > > > > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' > > + > > # `--out-dir` is required to avoid temporaries being created by `rustc` in the > > # current working directory, which may be not accessible in the out-of-tree > > # modules case. > > hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ > > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ > > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ > > $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ > > $(HOSTRUSTFLAGS_$(target-stem)) > > > > -- > > 2.42.0.459.ge4e396fd5e-goog > > > > > -- > Thanks, > ~Nick Desaulniers
Matthew Maurer <mmaurer@google.com> writes: > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, > resulting in build failures in hermetic environments where `cc` does not > exist. This includes both hostprogs and proc-macros. > > Since we are setting the linker to `HOSTCC`, we set the linker flavor to > `gcc` explicitly. > > Signed-off-by: Matthew Maurer <mmaurer@google.com> Tested-by: Alice Ryhl <aliceryhl@google.com>
On Tue, Sep 19, 2023 at 8:44 AM Matthew Maurer <mmaurer@google.com> wrote: > > Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, > resulting in build failures in hermetic environments where `cc` does not > exist. This includes both hostprogs and proc-macros. > > Since we are setting the linker to `HOSTCC`, we set the linker flavor to > `gcc` explicitly. This sentence is unclear to me because it does not explain why we need both despite the spec mentions 'linker-flavor' is inferred from the value of 'linker'. The answer exists in this thread. " As mentioned earlier, we could pass $HOSTLD, but if the linker isn't named something accurate (e.g. if the linker is not named lld, but is lld), we need to know how to pass a flavor: " Maybe it is worthing menting that HOSTCC may not be properly named. > > Signed-off-by: Matthew Maurer <mmaurer@google.com> > --- > > Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS > should be respected as well. > > I did not switch it to use HOSTLD for two reasons: > * That variable is not globally defined - it is only available in two > subdirectories of tools/ > * C host scripts are linked by HOSTCC as well, even when linking a > collection of object files. It *prints* HOSTLD, but invokes HOSTCC. > See scripts/Makefile.host cmd_host-cmulti for an example. > > rust/Makefile | 4 ++++ > scripts/Makefile.host | 4 ++++ > 2 files changed, 8 insertions(+) > > diff --git a/rust/Makefile b/rust/Makefile > index 87958e864be0..b60b7eb8c5a0 100644 > --- a/rust/Makefile > +++ b/rust/Makefile > @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE > $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE > $(call if_changed,exports) > > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' This is equivalent to a macro in scripts/Kbuild.include escsq = $(subst $(squote),'\$(squote)',$1) I would write it directly in cmd_rustc_procmacro. > + > quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ > cmd_rustc_procmacro = \ > $(RUSTC_OR_CLIPPY) $(rust_common_flags) \ > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ > --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ > --crate-type proc-macro \ > --crate-name $(patsubst lib%.so,%,$(notdir $@)) $< > diff --git a/scripts/Makefile.host b/scripts/Makefile.host > index 8f7f842b54f9..dc0410cae5ca 100644 > --- a/scripts/Makefile.host > +++ b/scripts/Makefile.host > @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ > $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ > $(HOSTCXXFLAGS_$(target-stem).o) > > +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' > + > # `--out-dir` is required to avoid temporaries being created by `rustc` in the > # current working directory, which may be not accessible in the out-of-tree > # modules case. > hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ > + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ > + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ > $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ > $(HOSTRUSTFLAGS_$(target-stem)) > > -- > 2.42.0.459.ge4e396fd5e-goog >
diff --git a/rust/Makefile b/rust/Makefile index 87958e864be0..b60b7eb8c5a0 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE $(call if_changed,exports) +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' + quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ cmd_rustc_procmacro = \ $(RUSTC_OR_CLIPPY) $(rust_common_flags) \ + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ --crate-type proc-macro \ --crate-name $(patsubst lib%.so,%,$(notdir $@)) $< diff --git a/scripts/Makefile.host b/scripts/Makefile.host index 8f7f842b54f9..dc0410cae5ca 100644 --- a/scripts/Makefile.host +++ b/scripts/Makefile.host @@ -87,10 +87,14 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))' + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ + -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
Currently, rustc defaults to invoking `cc`, even if `HOSTCC` is defined, resulting in build failures in hermetic environments where `cc` does not exist. This includes both hostprogs and proc-macros. Since we are setting the linker to `HOSTCC`, we set the linker flavor to `gcc` explicitly. Signed-off-by: Matthew Maurer <mmaurer@google.com> --- Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS should be respected as well. I did not switch it to use HOSTLD for two reasons: * That variable is not globally defined - it is only available in two subdirectories of tools/ * C host scripts are linked by HOSTCC as well, even when linking a collection of object files. It *prints* HOSTLD, but invokes HOSTCC. See scripts/Makefile.host cmd_host-cmulti for an example. rust/Makefile | 4 ++++ scripts/Makefile.host | 4 ++++ 2 files changed, 8 insertions(+)