| Message ID | 20240612183205.3120248-1-irogers@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Refactor perf python module build | expand |
On 12/06/2024 19:31, Ian Rogers wrote: > Refactor the perf python module build to instead of building C files > it links libraries. To support this make static libraries for tests, > ui, util and pmu-events. Doing this allows fewer functions to be > stubbed out, importantly parse_events is no longer stubbed out which > will improve the ability to work with heterogeneous cores. > > Patches 1 to 5 add static libraries for existing parts of the perf > build. > > Patch 6 adds the python build using libraries rather than C source > files. > > Patch 7 cleans up the python dependencies and removes the no longer > needed python-ext-sources. > Reviewed-by: James Clark <james.clark@arm.com> It does require a clean build to avoid some -fPIC errors presumably because not everything that requires it gets rebuilt, for anyone who gets stuck on that. > Ian Rogers (7): > perf ui: Make ui its own library > perf pmu-events: Make pmu-events a library > perf test: Make tests its own library > perf bench: Make bench its own library > perf util: Make util its own library > perf python: Switch module to linking libraries from building source > perf python: Clean up build dependencies > > tools/perf/Build | 14 +- > tools/perf/Makefile.config | 5 + > tools/perf/Makefile.perf | 66 ++- > tools/perf/arch/Build | 4 +- > tools/perf/arch/arm/Build | 4 +- > tools/perf/arch/arm/tests/Build | 8 +- > tools/perf/arch/arm/util/Build | 10 +- > tools/perf/arch/arm64/Build | 4 +- > tools/perf/arch/arm64/tests/Build | 8 +- > tools/perf/arch/arm64/util/Build | 20 +- > tools/perf/arch/csky/Build | 2 +- > tools/perf/arch/csky/util/Build | 6 +- > tools/perf/arch/loongarch/Build | 2 +- > tools/perf/arch/loongarch/util/Build | 8 +- > tools/perf/arch/mips/Build | 2 +- > tools/perf/arch/mips/util/Build | 6 +- > tools/perf/arch/powerpc/Build | 4 +- > tools/perf/arch/powerpc/tests/Build | 6 +- > tools/perf/arch/powerpc/util/Build | 24 +- > tools/perf/arch/riscv/Build | 2 +- > tools/perf/arch/riscv/util/Build | 8 +- > tools/perf/arch/s390/Build | 2 +- > tools/perf/arch/s390/util/Build | 16 +- > tools/perf/arch/sh/Build | 2 +- > tools/perf/arch/sh/util/Build | 2 +- > tools/perf/arch/sparc/Build | 2 +- > tools/perf/arch/sparc/util/Build | 2 +- > tools/perf/arch/x86/Build | 6 +- > tools/perf/arch/x86/tests/Build | 20 +- > tools/perf/arch/x86/util/Build | 42 +- > tools/perf/bench/Build | 46 +- > tools/perf/scripts/Build | 4 +- > tools/perf/scripts/perl/Perf-Trace-Util/Build | 2 +- > .../perf/scripts/python/Perf-Trace-Util/Build | 2 +- > tools/perf/tests/Build | 140 +++---- > tools/perf/tests/workloads/Build | 12 +- > tools/perf/ui/Build | 18 +- > tools/perf/ui/browsers/Build | 14 +- > tools/perf/ui/tui/Build | 8 +- > tools/perf/util/Build | 394 +++++++++--------- > tools/perf/util/arm-spe-decoder/Build | 2 +- > tools/perf/util/cs-etm-decoder/Build | 2 +- > tools/perf/util/hisi-ptt-decoder/Build | 2 +- > tools/perf/util/intel-pt-decoder/Build | 2 +- > tools/perf/util/perf-regs-arch/Build | 18 +- > tools/perf/util/python-ext-sources | 53 --- > tools/perf/util/python.c | 271 +++++------- > tools/perf/util/scripting-engines/Build | 4 +- > tools/perf/util/setup.py | 33 +- > 49 files changed, 612 insertions(+), 722 deletions(-) > delete mode 100644 tools/perf/util/python-ext-sources >
On Thu, Jun 13, 2024 at 10:27:15AM +0100, James Clark wrote: > On 12/06/2024 19:31, Ian Rogers wrote: > > Refactor the perf python module build to instead of building C files > > it links libraries. To support this make static libraries for tests, > > ui, util and pmu-events. Doing this allows fewer functions to be > > stubbed out, importantly parse_events is no longer stubbed out which > > will improve the ability to work with heterogeneous cores. > > > > Patches 1 to 5 add static libraries for existing parts of the perf > > build. > > > > Patch 6 adds the python build using libraries rather than C source > > files. > > > > Patch 7 cleans up the python dependencies and removes the no longer > > needed python-ext-sources. > > > > Reviewed-by: James Clark <james.clark@arm.com> > > It does require a clean build to avoid some -fPIC errors presumably > because not everything that requires it gets rebuilt, for anyone who > gets stuck on that. We need to find a way to avoid requiring the 'make clean' :-/ - Arnaldo > > Ian Rogers (7): > > perf ui: Make ui its own library > > perf pmu-events: Make pmu-events a library > > perf test: Make tests its own library > > perf bench: Make bench its own library > > perf util: Make util its own library > > perf python: Switch module to linking libraries from building source > > perf python: Clean up build dependencies > > > > tools/perf/Build | 14 +- > > tools/perf/Makefile.config | 5 + > > tools/perf/Makefile.perf | 66 ++- > > tools/perf/arch/Build | 4 +- > > tools/perf/arch/arm/Build | 4 +- > > tools/perf/arch/arm/tests/Build | 8 +- > > tools/perf/arch/arm/util/Build | 10 +- > > tools/perf/arch/arm64/Build | 4 +- > > tools/perf/arch/arm64/tests/Build | 8 +- > > tools/perf/arch/arm64/util/Build | 20 +- > > tools/perf/arch/csky/Build | 2 +- > > tools/perf/arch/csky/util/Build | 6 +- > > tools/perf/arch/loongarch/Build | 2 +- > > tools/perf/arch/loongarch/util/Build | 8 +- > > tools/perf/arch/mips/Build | 2 +- > > tools/perf/arch/mips/util/Build | 6 +- > > tools/perf/arch/powerpc/Build | 4 +- > > tools/perf/arch/powerpc/tests/Build | 6 +- > > tools/perf/arch/powerpc/util/Build | 24 +- > > tools/perf/arch/riscv/Build | 2 +- > > tools/perf/arch/riscv/util/Build | 8 +- > > tools/perf/arch/s390/Build | 2 +- > > tools/perf/arch/s390/util/Build | 16 +- > > tools/perf/arch/sh/Build | 2 +- > > tools/perf/arch/sh/util/Build | 2 +- > > tools/perf/arch/sparc/Build | 2 +- > > tools/perf/arch/sparc/util/Build | 2 +- > > tools/perf/arch/x86/Build | 6 +- > > tools/perf/arch/x86/tests/Build | 20 +- > > tools/perf/arch/x86/util/Build | 42 +- > > tools/perf/bench/Build | 46 +- > > tools/perf/scripts/Build | 4 +- > > tools/perf/scripts/perl/Perf-Trace-Util/Build | 2 +- > > .../perf/scripts/python/Perf-Trace-Util/Build | 2 +- > > tools/perf/tests/Build | 140 +++---- > > tools/perf/tests/workloads/Build | 12 +- > > tools/perf/ui/Build | 18 +- > > tools/perf/ui/browsers/Build | 14 +- > > tools/perf/ui/tui/Build | 8 +- > > tools/perf/util/Build | 394 +++++++++--------- > > tools/perf/util/arm-spe-decoder/Build | 2 +- > > tools/perf/util/cs-etm-decoder/Build | 2 +- > > tools/perf/util/hisi-ptt-decoder/Build | 2 +- > > tools/perf/util/intel-pt-decoder/Build | 2 +- > > tools/perf/util/perf-regs-arch/Build | 18 +- > > tools/perf/util/python-ext-sources | 53 --- > > tools/perf/util/python.c | 271 +++++------- > > tools/perf/util/scripting-engines/Build | 4 +- > > tools/perf/util/setup.py | 33 +- > > 49 files changed, 612 insertions(+), 722 deletions(-) > > delete mode 100644 tools/perf/util/python-ext-sources > >
On 13/06/2024 15:15, Arnaldo Carvalho de Melo wrote: > On Thu, Jun 13, 2024 at 10:27:15AM +0100, James Clark wrote: >> On 12/06/2024 19:31, Ian Rogers wrote: >>> Refactor the perf python module build to instead of building C files >>> it links libraries. To support this make static libraries for tests, >>> ui, util and pmu-events. Doing this allows fewer functions to be >>> stubbed out, importantly parse_events is no longer stubbed out which >>> will improve the ability to work with heterogeneous cores. >>> >>> Patches 1 to 5 add static libraries for existing parts of the perf >>> build. >>> >>> Patch 6 adds the python build using libraries rather than C source >>> files. >>> >>> Patch 7 cleans up the python dependencies and removes the no longer >>> needed python-ext-sources. >>> >> >> Reviewed-by: James Clark <james.clark@arm.com> >> >> It does require a clean build to avoid some -fPIC errors presumably >> because not everything that requires it gets rebuilt, for anyone who >> gets stuck on that. > > We need to find a way to avoid requiring the 'make clean' :-/ > > - Arnaldo > Do we need to make it so that if any of the Makefiles are touched it does a clean? I'm assuming that was the cause of the issue I experienced here and that the Makefile and/or Build files aren't mentioned as dependencies of any target. >>> Ian Rogers (7): >>> perf ui: Make ui its own library >>> perf pmu-events: Make pmu-events a library >>> perf test: Make tests its own library >>> perf bench: Make bench its own library >>> perf util: Make util its own library >>> perf python: Switch module to linking libraries from building source >>> perf python: Clean up build dependencies >>> >>> tools/perf/Build | 14 +- >>> tools/perf/Makefile.config | 5 + >>> tools/perf/Makefile.perf | 66 ++- >>> tools/perf/arch/Build | 4 +- >>> tools/perf/arch/arm/Build | 4 +- >>> tools/perf/arch/arm/tests/Build | 8 +- >>> tools/perf/arch/arm/util/Build | 10 +- >>> tools/perf/arch/arm64/Build | 4 +- >>> tools/perf/arch/arm64/tests/Build | 8 +- >>> tools/perf/arch/arm64/util/Build | 20 +- >>> tools/perf/arch/csky/Build | 2 +- >>> tools/perf/arch/csky/util/Build | 6 +- >>> tools/perf/arch/loongarch/Build | 2 +- >>> tools/perf/arch/loongarch/util/Build | 8 +- >>> tools/perf/arch/mips/Build | 2 +- >>> tools/perf/arch/mips/util/Build | 6 +- >>> tools/perf/arch/powerpc/Build | 4 +- >>> tools/perf/arch/powerpc/tests/Build | 6 +- >>> tools/perf/arch/powerpc/util/Build | 24 +- >>> tools/perf/arch/riscv/Build | 2 +- >>> tools/perf/arch/riscv/util/Build | 8 +- >>> tools/perf/arch/s390/Build | 2 +- >>> tools/perf/arch/s390/util/Build | 16 +- >>> tools/perf/arch/sh/Build | 2 +- >>> tools/perf/arch/sh/util/Build | 2 +- >>> tools/perf/arch/sparc/Build | 2 +- >>> tools/perf/arch/sparc/util/Build | 2 +- >>> tools/perf/arch/x86/Build | 6 +- >>> tools/perf/arch/x86/tests/Build | 20 +- >>> tools/perf/arch/x86/util/Build | 42 +- >>> tools/perf/bench/Build | 46 +- >>> tools/perf/scripts/Build | 4 +- >>> tools/perf/scripts/perl/Perf-Trace-Util/Build | 2 +- >>> .../perf/scripts/python/Perf-Trace-Util/Build | 2 +- >>> tools/perf/tests/Build | 140 +++---- >>> tools/perf/tests/workloads/Build | 12 +- >>> tools/perf/ui/Build | 18 +- >>> tools/perf/ui/browsers/Build | 14 +- >>> tools/perf/ui/tui/Build | 8 +- >>> tools/perf/util/Build | 394 +++++++++--------- >>> tools/perf/util/arm-spe-decoder/Build | 2 +- >>> tools/perf/util/cs-etm-decoder/Build | 2 +- >>> tools/perf/util/hisi-ptt-decoder/Build | 2 +- >>> tools/perf/util/intel-pt-decoder/Build | 2 +- >>> tools/perf/util/perf-regs-arch/Build | 18 +- >>> tools/perf/util/python-ext-sources | 53 --- >>> tools/perf/util/python.c | 271 +++++------- >>> tools/perf/util/scripting-engines/Build | 4 +- >>> tools/perf/util/setup.py | 33 +- >>> 49 files changed, 612 insertions(+), 722 deletions(-) >>> delete mode 100644 tools/perf/util/python-ext-sources >>>
On Thu, Jun 13, 2024 at 8:10 AM James Clark <james.clark@arm.com> wrote: [...] > >> Reviewed-by: James Clark <james.clark@arm.com> > >> > >> It does require a clean build to avoid some -fPIC errors presumably > >> because not everything that requires it gets rebuilt, for anyone who > >> gets stuck on that. > > > > We need to find a way to avoid requiring the 'make clean' :-/ > > > > - Arnaldo > > > > Do we need to make it so that if any of the Makefiles are touched it > does a clean? I'm assuming that was the cause of the issue I experienced > here and that the Makefile and/or Build files aren't mentioned as > dependencies of any target. Perhaps we can do something with the FEATURE_DUMP. It'd be nice to detect build argument changes and rebuild when that occurs. I'm used to doing lots of `make clean` due to sanitizer builds, something we should probably capture through features and/or `perf version --build-options`. Anyway, out-of-scope for these patches. I need to cut a v2 due to failing to add the new `.a` files to the clean target. Thanks for the review, Ian