| Message ID | OSZP286MB20611C35BF36F4EB69FD3DF6C0469@OSZP286MB2061.JPNP286.PROD.OUTLOOK.COM (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] gen_compile_commands: include targets not ending with .o | expand |
On Thu, May 25, 2023 at 9:21 PM Hu Weiwen <huww98@outlook.com> wrote: > > Currently, we only extract commands for targets ending with '.o'. But we > also have many standalone executables built in-tree. > > Remove this restriction. And to avoid some false matching, exclude > targets end with '.c' or '.h' when directly walking the directory. Can you give me some examples of false matching? Also, this patch would pick up *.i, *.s, *.lst, etc. Does it make sense to parse cmd files for those? I guess the answer is no. > > To really generate compile_commands.json that includes such target, call > this script directly with no arguments. `make compile_commands.json` > will not include them. > > Signed-off-by: Hu Weiwen <huww98@outlook.com> > --- > Since V1: Only commit message is updated. > > scripts/clang-tools/gen_compile_commands.py | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py > index 15ba56527acd..6e88c7e166fc 100755 > --- a/scripts/clang-tools/gen_compile_commands.py > +++ b/scripts/clang-tools/gen_compile_commands.py > @@ -18,8 +18,8 @@ import sys > _DEFAULT_OUTPUT = 'compile_commands.json' > _DEFAULT_LOG_LEVEL = 'WARNING' > > -_FILENAME_PATTERN = r'^\..*\.cmd$' > -_LINE_PATTERN = r'^savedcmd_[^ ]*\.o := (.* )([^ ]*\.c) *(;|$)' > +_FILENAME_PATTERN = r'^\..*(?<!\.(c|h))\.cmd$' > +_LINE_PATTERN = r'^savedcmd_[^ ]* := (.* )([^ ]*\.c) *(;|$)' > _VALID_LOG_LEVELS = ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'] > # The tools/ directory adopts a different build system, and produces .cmd > # files in a different format. Do not support it. > -- > 2.25.1 >
On Mon, May 29, 2023 at 07:13:46PM +0900, Masahiro Yamada wrote: > On Thu, May 25, 2023 at 9:21 PM Hu Weiwen <huww98@outlook.com> wrote: > > > > Currently, we only extract commands for targets ending with '.o'. But we > > also have many standalone executables built in-tree. > > > > Remove this restriction. And to avoid some false matching, exclude > > targets end with '.c' or '.h' when directly walking the directory. > > > Can you give me some examples of false matching? Examples: - drivers/scsi/scsi_devinfo_tbl.c: generated by sed -n 's/.*define *BLIST_\\([A-Z0-9_]*\\) *.*/BLIST_FLAG_NAME(\\1),/p' include/scsi/scsi_devinfo.h > drivers/scsi/scsi_devinfo_tbl.c - arch/x86/entry/vdso/vdso-image-32.c: generated by arch/x86/entry/vdso/vdso2c arch/x86/entry/vdso/vdso32.so.dbg arch/x86/entry/vdso/vdso32.so arch/x86/entry/vdso/vdso-image-32.c I think only .c file targets would have false matching, because the cmd must ends with '.c' to be matched. > > Also, this patch would pick up *.i, *.s, *.lst, etc. > > Does it make sense to parse cmd files for those? > I guess the answer is no. We are already parsing cmd files for those before this patch. I use more strict `_FILENAME_PATTERN'. But it may be hard to exclude all of these by filename. Should we use a more strict `_LINE_PATTERN'? e.g., require the cmd to begin with a compiler, not `sed' or something else. But we don't know the compiler name, adding a parameter for compiler name may break the existing usage. > > > > To really generate compile_commands.json that includes such target, call > > this script directly with no arguments. `make compile_commands.json` > > will not include them. > > > > Signed-off-by: Hu Weiwen <huww98@outlook.com> > > --- > > Since V1: Only commit message is updated. > > > > scripts/clang-tools/gen_compile_commands.py | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py > > index 15ba56527acd..6e88c7e166fc 100755 > > --- a/scripts/clang-tools/gen_compile_commands.py > > +++ b/scripts/clang-tools/gen_compile_commands.py > > @@ -18,8 +18,8 @@ import sys > > _DEFAULT_OUTPUT = 'compile_commands.json' > > _DEFAULT_LOG_LEVEL = 'WARNING' > > > > -_FILENAME_PATTERN = r'^\..*\.cmd$' > > -_LINE_PATTERN = r'^savedcmd_[^ ]*\.o := (.* )([^ ]*\.c) *(;|$)' > > +_FILENAME_PATTERN = r'^\..*(?<!\.(c|h))\.cmd$' > > +_LINE_PATTERN = r'^savedcmd_[^ ]* := (.* )([^ ]*\.c) *(;|$)' > > _VALID_LOG_LEVELS = ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'] > > # The tools/ directory adopts a different build system, and produces .cmd > > # files in a different format. Do not support it. > > -- > > 2.25.1 > > > > > -- > Best Regards > Masahiro Yamada
On Tue, May 30, 2023 at 6:04 PM 胡玮文 <huww98@outlook.com> wrote: > > On Mon, May 29, 2023 at 07:13:46PM +0900, Masahiro Yamada wrote: > > On Thu, May 25, 2023 at 9:21 PM Hu Weiwen <huww98@outlook.com> wrote: > > > > > > Currently, we only extract commands for targets ending with '.o'. But we > > > also have many standalone executables built in-tree. > > > > > > Remove this restriction. And to avoid some false matching, exclude > > > targets end with '.c' or '.h' when directly walking the directory. > > > > > > Can you give me some examples of false matching? > > Examples: > - drivers/scsi/scsi_devinfo_tbl.c: generated by > sed -n 's/.*define *BLIST_\\([A-Z0-9_]*\\) *.*/BLIST_FLAG_NAME(\\1),/p' include/scsi/scsi_devinfo.h > drivers/scsi/scsi_devinfo_tbl.c > > - arch/x86/entry/vdso/vdso-image-32.c: generated by > arch/x86/entry/vdso/vdso2c arch/x86/entry/vdso/vdso32.so.dbg arch/x86/entry/vdso/vdso32.so arch/x86/entry/vdso/vdso-image-32.c > > I think only .c file targets would have false matching, because the cmd > must ends with '.c' to be matched. I see. > > > > > Also, this patch would pick up *.i, *.s, *.lst, etc. > > > > Does it make sense to parse cmd files for those? > > I guess the answer is no. > > We are already parsing cmd files for those before this patch. True, but they are excluded by _LINE_PATTERN. > I use more > strict `_FILENAME_PATTERN'. But it may be hard to exclude all of these > by filename. I think so. > Should we use a more strict `_LINE_PATTERN'? e.g., require the cmd to > begin with a compiler, not `sed' or something else. But we don't know > the compiler name, adding a parameter for compiler name may break the > existing usage. Standalone executables are host programs. I do not think it is worthwhile complicating the code.
diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py index 15ba56527acd..6e88c7e166fc 100755 --- a/scripts/clang-tools/gen_compile_commands.py +++ b/scripts/clang-tools/gen_compile_commands.py @@ -18,8 +18,8 @@ import sys _DEFAULT_OUTPUT = 'compile_commands.json' _DEFAULT_LOG_LEVEL = 'WARNING' -_FILENAME_PATTERN = r'^\..*\.cmd$' -_LINE_PATTERN = r'^savedcmd_[^ ]*\.o := (.* )([^ ]*\.c) *(;|$)' +_FILENAME_PATTERN = r'^\..*(?<!\.(c|h))\.cmd$' +_LINE_PATTERN = r'^savedcmd_[^ ]* := (.* )([^ ]*\.c) *(;|$)' _VALID_LOG_LEVELS = ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'] # The tools/ directory adopts a different build system, and produces .cmd # files in a different format. Do not support it.
Currently, we only extract commands for targets ending with '.o'. But we also have many standalone executables built in-tree. Remove this restriction. And to avoid some false matching, exclude targets end with '.c' or '.h' when directly walking the directory. To really generate compile_commands.json that includes such target, call this script directly with no arguments. `make compile_commands.json` will not include them. Signed-off-by: Hu Weiwen <huww98@outlook.com> --- Since V1: Only commit message is updated. scripts/clang-tools/gen_compile_commands.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)