linux/tools/build, branch v5.10

tools feature: Fixup fast path feature detection

2020-11-03T12:24:20Z

22dd1ac91a776752 ("tools: Remove feature-libelf-mmap feature detection") correctly simplified the this feature detection, but forgot to remove the call to the removed function in the main() function for the test-all.c fast path feature detection, making it fail and thus do all the feature detection individually, fix it. $ cat /tmp/build/perf/feature/test-all.make.output test-all.c: In function ‘main’: test-all.c:188:2: error: implicit declaration of function ‘main_test_libelf_mmap’; did you mean ‘main_test_libelf’? [-Werror=implicit-function-declaration] 188 | main_test_libelf_mmap(); | ^~~~~~~~~~~~~~~~~~~~~ | main_test_libelf cc1: all warnings being treated as errors $ vim tools/build/feature/test-all.c $ rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf ;make V=1 -k O=/tmp/build/perf -C tools/perf install-bin ; perf test python $ cat /tmp/build/perf/feature/test-all.make.output $ Fixes: 22dd1ac91a776752 ("tools: Remove feature-libelf-mmap feature detection") Cc: Alexei Starovoitov Cc: Andrii Nakryiko Signed-off-by: Arnaldo Carvalho de Melo

Merge tag 'perf-tools-for-v5.10-2020-10-15' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

2020-10-17T18:47:46Z

Pull perf tools updates from Arnaldo Carvalho de Melo: - cgroup improvements for 'perf stat', allowing for compact specification of events and cgroups in the command line. - Support per thread topdown metrics in 'perf stat'. - Support sample-read topdown metric group in 'perf record' - Show start of latency in addition to its start in 'perf sched latency'. - Add min, max to 'perf script' futex-contention output, in addition to avg. - Allow usage of 'perf_event_attr->exclusive' attribute via the new ':e' event modifier. - Add 'snapshot' command to 'perf record --control', using it with Intel PT. - Support FIFO file names as alternative options to 'perf record --control'. - Introduce branch history "streams", to compare 'perf record' runs with 'perf diff' based on branch records and report hot streams. - Support PE executable symbol tables using libbfd, to profile, for instance, wine binaries. - Add filter support for option 'perf ftrace -F/--funcs'. - Allow configuring the 'disassembler_style' 'perf annotate' knob via 'perf config' - Update CascadelakeX and SkylakeX JSON vendor events files. - Add support for parsing perchip/percore JSON vendor events. - Add power9 hv_24x7 core level metric events. - Add L2 prefetch, ITLB instruction fetch hits JSON events for AMD zen1. - Enable Family 19h users by matching Zen2 AMD vendor events. - Use debuginfod in 'perf probe' when required debug files not found locally. - Display negative tid in non-sample events in 'perf script'. - Make GTK2 support opt-in - Add build test with GTK+ - Add missing -lzstd to the fast path feature detection - Add scripts to auto generate 'mmap', 'mremap' string<->id tables for use in 'perf trace'. - Show python test script in verbose mode. - Fix uncore metric expressions - Msan uninitialized use fixes. - Use condition variables in 'perf bench numa' - Autodetect python3 binary in systems without python2. - Support md5 build ids in addition to sha1. - Add build id 'perf test' regression test. - Fix printable strings in python3 scripts. - Fix off by ones in 'perf trace' in arches using libaudit. - Fix JSON event code for events referencing std arch events. - Introduce 'perf test' shell script for Arm CoreSight testing. - Add rdtsc() for Arm64 for used in the PERF_RECORD_TIME_CONV metadata event and in 'perf test tsc'. - 'perf c2c' improvements: Add "RMT Load Hit" metric, "Total Stores", fixes and documentation update. - Fix usage of reloc_sym in 'perf probe' when using both kallsyms and debuginfo files. - Do not print 'Metric Groups:' unnecessarily in 'perf list' - Refcounting fixes in the event parsing code. - Add expand cgroup event 'perf test' entry. - Fix out of bounds CPU map access when handling armv8_pmu events in 'perf stat'. - Add build-id injection 'perf bench' benchmark. - Enter namespace when reading build-id in 'perf inject'. - Do not load map/dso when injecting build-id speeding up the 'perf inject' process. - Add --buildid-all option to avoid processing all samples, just the mmap metadata events. - Add feature test to check if libbfd has buildid support - Add 'perf test' entry for PE binary format support. - Fix typos in power8 PMU vendor events JSON files. - Hide libtraceevent non API functions. * tag 'perf-tools-for-v5.10-2020-10-15' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux: (113 commits) perf c2c: Update documentation for metrics reorganization perf c2c: Add metrics "RMT Load Hit" perf c2c: Correct LLC load hit metrics perf c2c: Change header for LLC local hit perf c2c: Use more explicit headers for HITM perf c2c: Change header from "LLC Load Hitm" to "Load Hitm" perf c2c: Organize metrics based on memory hierarchy perf c2c: Display "Total Stores" as a standalone metrics perf c2c: Display the total numbers continuously perf bench: Use condition variables in numa. perf jevents: Fix event code for events referencing std arch events perf diff: Support hot streams comparison perf streams: Report hot streams perf streams: Calculate the sum of total streams hits perf streams: Link stream pair perf streams: Compare two streams perf streams: Get the evsel_streams by evsel_idx perf streams: Introduce branch history "streams" perf intel-pt: Improve PT documentation slightly perf tools: Add support for exclusive groups/events ...

tools feature: Add missing -lzstd to the fast path feature detection

2020-09-09T14:11:01Z

We were failing that due to GTK2+ and then for the ZSTD test, which made test-all.c, the fast path feature detection file to fail and thus trigger building all of the feature tests, slowing down the test. Eventually the ZSTD test would be built and would succeed, since it had the needed -lzstd, avoiding: $ cat /tmp/build/perf/feature/test-all.make.output /usr/bin/ld: /tmp/ccRRJQ4u.o: in function `main_test_libzstd': /home/acme/git/perf/tools/build/feature/test-libzstd.c:8: undefined reference to `ZSTD_createCStream' /usr/bin/ld: /home/acme/git/perf/tools/build/feature/test-libzstd.c:9: undefined reference to `ZSTD_freeCStream' collect2: error: ld returned 1 exit status $ Fix it by adding -lzstd to the test-all target. Now I need an entry to 'perf test' to make sure that /tmp/build/perf/feature/test-all.make.output is empty... Fixes: 3b1c5d9659718263 ("tools build: Implement libzstd feature check, LIBZSTD_DIR and NO_LIBZSTD defines") Reviewed-by: Alexei Budankov Acked-by: Namhyung Kim Cc: Adrian Hunter Cc: Ian Rogers Cc: Jiri Olsa Link: http://lore.kernel.org/lkml/20200904202611.GJ3753976@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Make GTK2 support opt-in

2020-09-04T20:11:59Z

This is bitrotting, nobody is stepping up to work on it, and since we treat warnings as errors, feature detection is failing in its main, faster test (tools/build/feature/test-all.c) because of the GTK+2 infobar check. So make this opt-in, at some point ditch this if nobody volunteers to take care of this. Cc: Adrian Hunter Cc: Ian Rogers Cc: Jiri Olsa Cc: Namhyung Kim Signed-off-by: Arnaldo Carvalho de Melo

tools features: Add feature test to check if libbfd has buildid support

2020-09-04T17:38:15Z

Which is needed by the PE executable support, for instance. Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Ian Rogers Cc: Jacek Caban Cc: Jiri Olsa Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Remi Bernon Signed-off-by: Arnaldo Carvalho de Melo

tools, bpf/build: Cleanup feature files on make clean

2020-08-28T12:04:27Z

The system for "Auto-detecting system features" located under tools/build/ are (currently) used by perf, libbpf and bpftool. It can contain stalled feature detection files, which are not cleaned up by libbpf and bpftool on make clean (side-note: perf tool is correct). Fix this by making the users invoke the make clean target. Some details about the changes. The libbpf Makefile already had a clean-config target (which seems to be copy-pasted from perf), but this target was not "connected" (a make dependency) to clean target. Choose not to rename target as someone might be using it. Did change the output from "CLEAN config" to "CLEAN feature-detect", to make it more clear what happens. This is related to the complaint and troubleshooting in the following link: https://lore.kernel.org/lkml/20200818122007.2d1cfe2d@carbon/ Signed-off-by: Jesper Dangaard Brouer Signed-off-by: Daniel Borkmann Acked-by: Jiri Olsa Link: https://lore.kernel.org/lkml/20200818122007.2d1cfe2d@carbon/ Link: https://lore.kernel.org/bpf/159851841661.1072907.13770213104521805592.stgit@firesoul

tools: Remove feature-libelf-mmap feature detection

2020-08-19T01:38:25Z

It's trivial to handle missing ELF_C_MMAP_READ support in libelf the way that objtool has solved it in ("774bec3fddcc objtool: Add fallback from ELF_C_READ_MMAP to ELF_C_READ"). So instead of having an entire feature detector for that, just do what objtool does for perf and libbpf. And keep their Makefiles a bit simpler. Signed-off-by: Andrii Nakryiko Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20200819013607.3607269-5-andriin@fb.com

perf build-ids: Fall back to debuginfod query if debuginfo not found

2020-08-14T12:51:14Z

During a perf-record, use the -ldebuginfod API to query a debuginfod server, should the debug data not be found in the usual system locations. If successful, the usual $HOME/.debug dir is populated. Tested with: $ find . . ./ctags-debuginfo-5.8-26.fc31.x86_64.rpm ./usr ./usr/lib ./usr/lib/debug ./usr/lib/debug/.build-id ./usr/lib/debug/.build-id/ca ./usr/lib/debug/.build-id/ca/46f6ae6a0cee57d85abc1d461c49074248908d ./usr/lib/debug/.build-id/ca/46f6ae6a0cee57d85abc1d461c49074248908d.debug ./usr/lib/debug/usr ./usr/lib/debug/usr/bin ./usr/lib/debug/usr/bin/ctags-5.8-26.fc31.x86_64.debug $ debuginfod -F . ... $ rm -rf ~/.debug/ ; mkdir ~/.debug $ perf record make tags BUILD: Doing 'make -j8' parallel build GEN tags [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.107 MB perf.data (1483 samples) ] $ find ~/.debug | grep ctags /home/jolsa/.debug/usr/bin/ctags /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/elf /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/probes $ rm -rf ~/.debug/ ; mkdir ~/.debug $ DEBUGINFOD_URLS=http://localhost:8002 perf record make tags BUILD: Doing 'make -j8' parallel build GEN tags [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.108 MB perf.data (1531 samples) ] $ find ~/.debug | grep ctag /home/jolsa/.debug/usr/bin/ctags /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/debug /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/elf /home/jolsa/.debug/usr/bin/ctags/ca46f6ae6a0cee57d85abc1d461c49074248908d/probes Note the 'debug' file is created in the last run. Note that currently the debuginfo data are downloaded only on record path, we still need add this support to perf build-id/report.. and test ;-) Tested-by: Jiri Olsa Signed-off-by: Jiri Olsa Signed-off-by: Frank Ch. Eigler Signed-off-by: Arnaldo Carvalho de Melo

tools build feature: Quote CC and CXX for their arguments

2020-08-13T13:00:19Z

When using a cross-compilation environment, such as OpenEmbedded, the CC an CXX variables are set to something more than just a command: there are arguments (such as --sysroot) that need to be passed on to the compiler so that the right set of headers and libraries are used. For the particular case that our systems detected, CC is set to the following: export CC="aarch64-linaro-linux-gcc --sysroot=/oe/build/tmp/work/machine/perf/1.0-r9/recipe-sysroot" Without quotes, detection is as follows: Auto-detecting system features: ... dwarf: [ OFF ] ... dwarf_getlocations: [ OFF ] ... glibc: [ OFF ] ... gtk2: [ OFF ] ... libbfd: [ OFF ] ... libcap: [ OFF ] ... libelf: [ OFF ] ... libnuma: [ OFF ] ... numa_num_possible_cpus: [ OFF ] ... libperl: [ OFF ] ... libpython: [ OFF ] ... libcrypto: [ OFF ] ... libunwind: [ OFF ] ... libdw-dwarf-unwind: [ OFF ] ... zlib: [ OFF ] ... lzma: [ OFF ] ... get_cpuid: [ OFF ] ... bpf: [ OFF ] ... libaio: [ OFF ] ... libzstd: [ OFF ] ... disassembler-four-args: [ OFF ] Makefile.config:414: *** No gnu/libc-version.h found, please install glibc-dev[el]. Stop. Makefile.perf:230: recipe for target 'sub-make' failed make[1]: *** [sub-make] Error 2 Makefile:69: recipe for target 'all' failed make: *** [all] Error 2 With CC and CXX quoted, some of those features are now detected. Fixes: e3232c2f39ac ("tools build feature: Use CC and CXX from parent") Signed-off-by: Daniel Díaz Reviewed-by: Thomas Hebb Cc: Alexei Starovoitov Cc: Andrii Nakryiko Cc: Daniel Borkmann Cc: Jiri Olsa Cc: John Fastabend Cc: KP Singh Cc: Martin KaFai Lau Cc: Namhyung Kim Cc: Song Liu Cc: Stephane Eranian Cc: Yonghong Song Link: http://lore.kernel.org/lkml/20200812221518.2869003-1-daniel.diaz@linaro.org Signed-off-by: Arnaldo Carvalho de Melo

Mgerge remote-tracking branch 'torvalds/master' into perf/core

2020-08-06T11:15:47Z

To sync headers, for instance, in this case tools/perf was ahead of upstream till Linus merged tip/perf/core to get the PERF_RECORD_TEXT_POKE changes: Warning: Kernel ABI header at 'tools/include/uapi/linux/perf_event.h' differs from latest version at 'include/uapi/linux/perf_event.h' diff -u tools/include/uapi/linux/perf_event.h include/uapi/linux/perf_event.h Signed-off-by: Arnaldo Carvalho de Melo