| Message ID | 20230118160220.776302-1-varmavinaym@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | scripts: `make rust-analyzer` for out-of-tree modules | expand |
On Wed, Jan 18, 2023 at 5:02 PM Vinay Varma <varmavinaym@gmail.com> wrote: > > Adds support for out-of-tree rust modules to use the `rust-analyzer` > make target to generate the rust-project.json file. Thanks for sending this! A few nits below... > When this parameter is passed, I have chosen > not to add the non-core modules (samples and drivers) into the result > since these are not expected to be used in third party modules. Sounds good to me. > changes are also made to the Makefile and rust/Makefile allowing the > `rust-analyzer` target to be used for out-of-tree modules as well. > > Signed-off-by: Vinay Varma <varmavinaym@gmail.com> A `Link: ` to the GitHub PR would be nice here: https://github.com/Rust-for-Linux/linux/pull/914. Possibly another one to the out-of-tree approach at https://github.com/Rust-for-Linux/rust-out-of-tree-module/pull/2. > import logging > import pathlib > import sys > +import os Please keep the `import`s sorted. > - for folder in ("samples", "drivers"): > + extra_src_dirs = ["samples", "drivers"] if external_src is None else [external_src] If you send a v2 for the above, this could be a tuple like in the original line, to minimize the diff. > + if os.path.exists(path.parent / "Makefile") and f"{name}.o" not in open(path.parent / "Makefile").read(): In Python one would typically go with the EAFP style instead (https://docs.python.org/3/glossary.html#term-EAFP), which would also reduce the duplication of the path computation. But it would take more lines... Not a big deal in any case. Thanks! Cheers, Miguel
------- Original Message ------- On Wednesday, January 18th, 2023 at 17:02, Vinay Varma <varmavinaym@gmail.com> wrote: > Adds support for out-of-tree rust modules to use the `rust-analyzer` > make target to generate the rust-project.json file. > > The change involves adding an optional parameter `external_src` to the > `generate_rust_analyzer.py` which expects the path to the out-of-tree > module's source directory. When this parameter is passed, I have chosen > not to add the non-core modules (samples and drivers) into the result > since these are not expected to be used in third party modules. Related > changes are also made to the Makefile and rust/Makefile allowing the > `rust-analyzer` target to be used for out-of-tree modules as well. > > Signed-off-by: Vinay Varma varmavinaym@gmail.com > > --- I tested this with https://github.com/Rust-for-Linux/rust-out-of-tree-module using "make -C ../rust-for-linux M=$(pwd) rust-analyzer". It produces a rust-project.json file, however it misses the actual rust_out_of_tree crate. This is due to the fact that generate_rust_analyzer.py only checks Makefile to find the crate roots, but rust-out-of-tree-module defines it in Kbuild instead. Apart from this issue, all paths I checked in the generated rust-project.json are correct. Cheers, Bjorn
On Fri, Jan 20, 2023 at 2:17 PM Björn Roy Baron <bjorn3_gh@protonmail.com> wrote: > > I tested this with https://github.com/Rust-for-Linux/rust-out-of-tree-module using "make -C ../rust-for-linux M=$(pwd) rust-analyzer". It produces a rust-project.json file, however it misses the actual rust_out_of_tree crate. This is due to the fact that generate_rust_analyzer.py only checks Makefile to find the crate roots, but rust-out-of-tree-module defines it in Kbuild instead. Thanks for testing it! Indeed, we should support out-of-tree modules using the `Kbuild` + `Makefile` approach. Having said that, I wonder if we should even attempt to perform the search for out-of-tree modules. The search loop is a simple hack which works fine for in-tree code because we know the structure of the code around us, but we cannot for out-of-tree modules. For instance, they may be using a different build system, or not mentioning the object file, etc. Perhaps we should simply ask them to give us the roots directly as a list of arguments to the script or similar. Cheers, Miguel
On Thu, Jan 19, 2023 at 12:02:20AM +0800, Vinay Varma wrote: > Adds support for out-of-tree rust modules to use the `rust-analyzer` > make target to generate the rust-project.json file. > > The change involves adding an optional parameter `external_src` to the > `generate_rust_analyzer.py` which expects the path to the out-of-tree > module's source directory. When this parameter is passed, I have chosen > not to add the non-core modules (samples and drivers) into the result > since these are not expected to be used in third party modules. Related > changes are also made to the Makefile and rust/Makefile allowing the > `rust-analyzer` target to be used for out-of-tree modules as well. > > Signed-off-by: Vinay Varma <varmavinaym@gmail.com> > --- > Makefile | 12 +++++++----- > rust/Makefile | 6 ++++-- > scripts/generate_rust_analyzer.py | 14 +++++++++----- > 3 files changed, 20 insertions(+), 12 deletions(-) > > diff --git a/Makefile b/Makefile > index f41ec8c8426b..a055a316d2a4 100644 > --- a/Makefile > +++ b/Makefile > @@ -1831,11 +1831,6 @@ rustfmt: > rustfmtcheck: rustfmt_flags = --check > rustfmtcheck: rustfmt > > -# IDE support targets > -PHONY += rust-analyzer > -rust-analyzer: > - $(Q)$(MAKE) $(build)=rust $@ > - > # Misc > # --------------------------------------------------------------------------- > > @@ -1888,6 +1883,7 @@ help: > @echo ' modules - default target, build the module(s)' > @echo ' modules_install - install the module' > @echo ' clean - remove generated files in module directory only' > + @echo ' rust-analyzer - generate rust-project.json rust-analyzer support file' > @echo '' > > endif # KBUILD_EXTMOD > @@ -2022,6 +2018,12 @@ quiet_cmd_tags = GEN $@ > tags TAGS cscope gtags: FORCE > $(call cmd,tags) > > +# IDE support targets > +PHONY += rust-analyzer > +rust-analyzer: > + $(Q)$(MAKE) $(build)=rust $@ > + > + > # Script to generate missing namespace dependencies > # --------------------------------------------------------------------------- > > diff --git a/rust/Makefile b/rust/Makefile > index 8f598a904f38..41c1435cd8d4 100644 > --- a/rust/Makefile > +++ b/rust/Makefile > @@ -389,8 +389,10 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L > $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@) > > rust-analyzer: > - $(Q)$(srctree)/scripts/generate_rust_analyzer.py $(srctree) $(objtree) \ > - $(RUST_LIB_SRC) > $(objtree)/rust-project.json > + $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ > + $(abs_srctree) $(abs_objtree) \ > + $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ > + $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json > > $(obj)/core.o: private skip_clippy = 1 > $(obj)/core.o: private skip_flags = -Dunreachable_pub > diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py > index ecc7ea9a4dcf..1546b80db554 100755 > --- a/scripts/generate_rust_analyzer.py > +++ b/scripts/generate_rust_analyzer.py > @@ -8,8 +8,9 @@ import json > import logging > import pathlib > import sys > +import os > > -def generate_crates(srctree, objtree, sysroot_src): > +def generate_crates(srctree, objtree, sysroot_src, external_src): > # Generate the configuration list. > cfg = [] > with open(objtree / "include" / "generated" / "rustc_cfg") as fd: > @@ -65,7 +66,7 @@ def generate_crates(srctree, objtree, sysroot_src): > [], > is_proc_macro=True, > ) > - crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so" > + crates[-1]["proc_macro_dylib_path"] = f"{objtree}/rust/libmacros.so" > > append_crate( > "build_error", > @@ -98,13 +99,15 @@ def generate_crates(srctree, objtree, sysroot_src): > # Then, the rest outside of `rust/`. > # > # We explicitly mention the top-level folders we want to cover. > - for folder in ("samples", "drivers"): > + extra_src_dirs = ["samples", "drivers"] if external_src is None else [external_src] > + > + for folder in extra_src_dirs: > for path in (srctree / folder).rglob("*.rs"): > logging.info("Checking %s", path) > name = path.name.replace(".rs", "") > > # Skip those that are not crate roots. > - if f"{name}.o" not in open(path.parent / "Makefile").read(): > + if os.path.exists(path.parent / "Makefile") and f"{name}.o" not in open(path.parent / "Makefile").read(): > continue > > logging.info("Adding %s", name) > @@ -123,6 +126,7 @@ def main(): > parser.add_argument("srctree", type=pathlib.Path) > parser.add_argument("objtree", type=pathlib.Path) > parser.add_argument("sysroot_src", type=pathlib.Path) > + parser.add_argument("exttree", type=pathlib.Path, nargs='?') I would prefer to don't mix '' and "" unless necessary > args = parser.parse_args() > > logging.basicConfig( > @@ -131,7 +135,7 @@ def main(): > ) > > rust_project = { > - "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src), > + "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.exttree), > "sysroot_src": str(args.sysroot_src), > } > > -- > 2.39.0 I could test it on my enviroment with rust-out-of-tree-module and rustc 1.66 it correctly generated a json file rust-project.json with correct path As following from github conversation: Reviewed-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com> Tested-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com> Thanks, Alicef
diff --git a/Makefile b/Makefile index f41ec8c8426b..a055a316d2a4 100644 --- a/Makefile +++ b/Makefile @@ -1831,11 +1831,6 @@ rustfmt: rustfmtcheck: rustfmt_flags = --check rustfmtcheck: rustfmt -# IDE support targets -PHONY += rust-analyzer -rust-analyzer: - $(Q)$(MAKE) $(build)=rust $@ - # Misc # --------------------------------------------------------------------------- @@ -1888,6 +1883,7 @@ help: @echo ' modules - default target, build the module(s)' @echo ' modules_install - install the module' @echo ' clean - remove generated files in module directory only' + @echo ' rust-analyzer - generate rust-project.json rust-analyzer support file' @echo '' endif # KBUILD_EXTMOD @@ -2022,6 +2018,12 @@ quiet_cmd_tags = GEN $@ tags TAGS cscope gtags: FORCE $(call cmd,tags) +# IDE support targets +PHONY += rust-analyzer +rust-analyzer: + $(Q)$(MAKE) $(build)=rust $@ + + # Script to generate missing namespace dependencies # --------------------------------------------------------------------------- diff --git a/rust/Makefile b/rust/Makefile index 8f598a904f38..41c1435cd8d4 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -389,8 +389,10 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@) rust-analyzer: - $(Q)$(srctree)/scripts/generate_rust_analyzer.py $(srctree) $(objtree) \ - $(RUST_LIB_SRC) > $(objtree)/rust-project.json + $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ + $(abs_srctree) $(abs_objtree) \ + $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ + $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json $(obj)/core.o: private skip_clippy = 1 $(obj)/core.o: private skip_flags = -Dunreachable_pub diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index ecc7ea9a4dcf..1546b80db554 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -8,8 +8,9 @@ import json import logging import pathlib import sys +import os -def generate_crates(srctree, objtree, sysroot_src): +def generate_crates(srctree, objtree, sysroot_src, external_src): # Generate the configuration list. cfg = [] with open(objtree / "include" / "generated" / "rustc_cfg") as fd: @@ -65,7 +66,7 @@ def generate_crates(srctree, objtree, sysroot_src): [], is_proc_macro=True, ) - crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so" + crates[-1]["proc_macro_dylib_path"] = f"{objtree}/rust/libmacros.so" append_crate( "build_error", @@ -98,13 +99,15 @@ def generate_crates(srctree, objtree, sysroot_src): # Then, the rest outside of `rust/`. # # We explicitly mention the top-level folders we want to cover. - for folder in ("samples", "drivers"): + extra_src_dirs = ["samples", "drivers"] if external_src is None else [external_src] + + for folder in extra_src_dirs: for path in (srctree / folder).rglob("*.rs"): logging.info("Checking %s", path) name = path.name.replace(".rs", "") # Skip those that are not crate roots. - if f"{name}.o" not in open(path.parent / "Makefile").read(): + if os.path.exists(path.parent / "Makefile") and f"{name}.o" not in open(path.parent / "Makefile").read(): continue logging.info("Adding %s", name) @@ -123,6 +126,7 @@ def main(): parser.add_argument("srctree", type=pathlib.Path) parser.add_argument("objtree", type=pathlib.Path) parser.add_argument("sysroot_src", type=pathlib.Path) + parser.add_argument("exttree", type=pathlib.Path, nargs='?') args = parser.parse_args() logging.basicConfig( @@ -131,7 +135,7 @@ def main(): ) rust_project = { - "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src), + "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.exttree), "sysroot_src": str(args.sysroot_src), }
Adds support for out-of-tree rust modules to use the `rust-analyzer` make target to generate the rust-project.json file. The change involves adding an optional parameter `external_src` to the `generate_rust_analyzer.py` which expects the path to the out-of-tree module's source directory. When this parameter is passed, I have chosen not to add the non-core modules (samples and drivers) into the result since these are not expected to be used in third party modules. Related changes are also made to the Makefile and rust/Makefile allowing the `rust-analyzer` target to be used for out-of-tree modules as well. Signed-off-by: Vinay Varma <varmavinaym@gmail.com> --- Makefile | 12 +++++++----- rust/Makefile | 6 ++++-- scripts/generate_rust_analyzer.py | 14 +++++++++----- 3 files changed, 20 insertions(+), 12 deletions(-)