linux/tools/perf, branch v6.3

Merge tag 'perf-tools-fixes-for-v6.3-1-2023-03-09' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

2023-03-10T16:18:46Z

Pull perf tools fixes from Arnaldo Carvalho de Melo: - Add Adrian Hunter to MAINTAINERS as a perf tools reviewer - Sync various tools/ copies of kernel headers with the kernel sources, this time trying to avoid first merging with upstream to then update but instead copy from upstream so that a merge is avoided and the end result after merging this pull request is the one expected, tools/perf/check-headers.sh (mostly) happy, less warnings while building tools/perf/ - Fix counting when initial delay configured by setting perf_attr.enable_on_exec when starting workloads from the perf command line - Don't avoid emitting a PERF_RECORD_MMAP2 in 'perf inject --buildid-all' when that record comes with a build-id, otherwise we end up not being able to resolve symbols - Don't use comma as the CSV output separator the "stat+csv_output" test, as comma can appear on some tests as a modifier for an event, use @ instead, ditto for the JSON linter test - The offcpu test was looking for some bits being set on task_struct->prev_state without masking other bits not important for this specific 'perf test', fix it * tag 'perf-tools-fixes-for-v6.3-1-2023-03-09' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux: perf tools: Add Adrian Hunter to MAINTAINERS as a reviewer tools headers UAPI: Sync linux/perf_event.h with the kernel sources tools headers x86 cpufeatures: Sync with the kernel sources tools include UAPI: Sync linux/vhost.h with the kernel sources tools arch x86: Sync the msr-index.h copy with the kernel sources tools headers kvm: Sync uapi/{asm/linux} kvm.h headers with the kernel sources tools include UAPI: Synchronize linux/fcntl.h with the kernel sources tools headers: Synchronize {linux,vdso}/bits.h with the kernel sources tools headers UAPI: Sync linux/prctl.h with the kernel sources tools headers: Update the copy of x86's mem{cpy,set}_64.S used in 'perf bench' perf stat: Fix counting when initial delay configured tools headers svm: Sync svm headers with the kernel sources perf test: Avoid counting commas in json linter perf tests stat+csv_output: Switch CSV separator to @ perf inject: Fix --buildid-all not to eat up MMAP2 tools arch x86: Sync the msr-index.h copy with the kernel sources perf test: Fix offcpu test prev_state check

perf stat: Fix counting when initial delay configured

2023-03-02T20:39:03Z

When creating counters with initial delay configured, the enable_on_exec field is not set. So we need to enable the counters later. The problem is, when a workload is specified the target__none() is true. So we also need to check stat_config.initial_delay. In this change, we add a new field 'initial_delay' for struct target which could be shared by other subcommands. And define target__enable_on_exec() which returns whether enable_on_exec should be set on normal cases. Before this fix the event is not counted: $ ./perf stat -e instructions -D 100 sleep 2 Events disabled Events enabled Performance counter stats for 'sleep 2': instructions 1.901661124 seconds time elapsed 0.001602000 seconds user 0.000000000 seconds sys After fix it works: $ ./perf stat -e instructions -D 100 sleep 2 Events disabled Events enabled Performance counter stats for 'sleep 2': 404,214 instructions 1.901743475 seconds time elapsed 0.001617000 seconds user 0.000000000 seconds sys Fixes: c587e77e100fa40e ("perf stat: Do not delay the workload with --delay") Signed-off-by: Changbin Du Acked-by: Namhyung Kim Cc: Alexander Shishkin Cc: Hui Wang Cc: Ingo Molnar Cc: Jiri Olsa Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20230302031146.2801588-2-changbin.du@huawei.com Signed-off-by: Arnaldo Carvalho de Melo

Merge tag 'mm-stable-2023-02-20-13-37' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

2023-02-24T01:09:35Z

Pull MM updates from Andrew Morton: - Daniel Verkamp has contributed a memfd series ("mm/memfd: add F_SEAL_EXEC") which permits the setting of the memfd execute bit at memfd creation time, with the option of sealing the state of the X bit. - Peter Xu adds a patch series ("mm/hugetlb: Make huge_pte_offset() thread-safe for pmd unshare") which addresses a rare race condition related to PMD unsharing. - Several folioification patch serieses from Matthew Wilcox, Vishal Moola, Sidhartha Kumar and Lorenzo Stoakes - Johannes Weiner has a series ("mm: push down lock_page_memcg()") which does perform some memcg maintenance and cleanup work. - SeongJae Park has added DAMOS filtering to DAMON, with the series "mm/damon/core: implement damos filter". These filters provide users with finer-grained control over DAMOS's actions. SeongJae has also done some DAMON cleanup work. - Kairui Song adds a series ("Clean up and fixes for swap"). - Vernon Yang contributed the series "Clean up and refinement for maple tree". - Yu Zhao has contributed the "mm: multi-gen LRU: memcg LRU" series. It adds to MGLRU an LRU of memcgs, to improve the scalability of global reclaim. - David Hildenbrand has added some userfaultfd cleanup work in the series "mm: uffd-wp + change_protection() cleanups". - Christoph Hellwig has removed the generic_writepages() library function in the series "remove generic_writepages". - Baolin Wang has performed some maintenance on the compaction code in his series "Some small improvements for compaction". - Sidhartha Kumar is doing some maintenance work on struct page in his series "Get rid of tail page fields". - David Hildenbrand contributed some cleanup, bugfixing and generalization of pte management and of pte debugging in his series "mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on all architectures with swap PTEs". - Mel Gorman and Neil Brown have removed the __GFP_ATOMIC allocation flag in the series "Discard __GFP_ATOMIC". - Sergey Senozhatsky has improved zsmalloc's memory utilization with his series "zsmalloc: make zspage chain size configurable". - Joey Gouly has added prctl() support for prohibiting the creation of writeable+executable mappings. The previous BPF-based approach had shortcomings. See "mm: In-kernel support for memory-deny-write-execute (MDWE)". - Waiman Long did some kmemleak cleanup and bugfixing in the series "mm/kmemleak: Simplify kmemleak_cond_resched() & fix UAF". - T.J. Alumbaugh has contributed some MGLRU cleanup work in his series "mm: multi-gen LRU: improve". - Jiaqi Yan has provided some enhancements to our memory error statistics reporting, mainly by presenting the statistics on a per-node basis. See the series "Introduce per NUMA node memory error statistics". - Mel Gorman has a second and hopefully final shot at fixing a CPU-hog regression in compaction via his series "Fix excessive CPU usage during compaction". - Christoph Hellwig does some vmalloc maintenance work in the series "cleanup vfree and vunmap". - Christoph Hellwig has removed block_device_operations.rw_page() in ths series "remove ->rw_page". - We get some maple_tree improvements and cleanups in Liam Howlett's series "VMA tree type safety and remove __vma_adjust()". - Suren Baghdasaryan has done some work on the maintainability of our vm_flags handling in the series "introduce vm_flags modifier functions". - Some pagemap cleanup and generalization work in Mike Rapoport's series "mm, arch: add generic implementation of pfn_valid() for FLATMEM" and "fixups for generic implementation of pfn_valid()" - Baoquan He has done some work to make /proc/vmallocinfo and /proc/kcore better represent the real state of things in his series "mm/vmalloc.c: allow vread() to read out vm_map_ram areas". - Jason Gunthorpe rationalized the GUP system's interface to the rest of the kernel in the series "Simplify the external interface for GUP". - SeongJae Park wishes to migrate people from DAMON's debugfs interface over to its sysfs interface. To support this, we'll temporarily be printing warnings when people use the debugfs interface. See the series "mm/damon: deprecate DAMON debugfs interface". - Andrey Konovalov provided the accurately named "lib/stackdepot: fixes and clean-ups" series. - Huang Ying has provided a dramatic reduction in migration's TLB flush IPI rates with the series "migrate_pages(): batch TLB flushing". - Arnd Bergmann has some objtool fixups in "objtool warning fixes". * tag 'mm-stable-2023-02-20-13-37' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (505 commits) include/linux/migrate.h: remove unneeded externs mm/memory_hotplug: cleanup return value handing in do_migrate_range() mm/uffd: fix comment in handling pte markers mm: change to return bool for isolate_movable_page() mm: hugetlb: change to return bool for isolate_hugetlb() mm: change to return bool for isolate_lru_page() mm: change to return bool for folio_isolate_lru() objtool: add UACCESS exceptions for __tsan_volatile_read/write kmsan: disable ftrace in kmsan core code kasan: mark addr_has_metadata __always_inline mm: memcontrol: rename memcg_kmem_enabled() sh: initialize max_mapnr m68k/nommu: add missing definition of ARCH_PFN_OFFSET mm: percpu: fix incorrect size in pcpu_obj_full_size() maple_tree: reduce stack usage with gcc-9 and earlier mm: page_alloc: call panic() when memoryless node allocation fails mm: multi-gen LRU: avoid futile retries migrate_pages: move THP/hugetlb migration support check to simplify code migrate_pages: batch flushing TLB migrate_pages: share more code between _unmap and _move ...

perf test: Avoid counting commas in json linter

2023-02-23T13:31:28Z

Commas may appear in events like: cpu/INT_MISC.RECOVERY_CYCLES,cmask=1,edge/ which causes the count of commas to see more items than expected. Switch to counting the entries in the dictionary, which is 1 more than the number of commas. Signed-off-by: Ian Rogers Cc: Alexander Shishkin Cc: Athira Rajeev Cc: Claire Jensen Cc: Ingo Molnar Cc: Jiri Olsa Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Sumanth Korikkar Cc: Thomas Richter Link: https://lore.kernel.org/r/20230223071818.329671-2-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf tests stat+csv_output: Switch CSV separator to @

2023-02-23T13:31:13Z

Commas may appear in events like: cpu/INT_MISC.RECOVERY_CYCLES,cmask=1,edge/ which causes the commachecker to see more fields than expected. Use @ as the CSV separator to avoid this. Signed-off-by: Ian Rogers Cc: Alexander Shishkin Cc: Athira Rajeev Cc: Claire Jensen Cc: Ingo Molnar Cc: Jiri Olsa Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Sumanth Korikkar Cc: Thomas Richter Link: https://lore.kernel.org/r/20230223071818.329671-1-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf inject: Fix --buildid-all not to eat up MMAP2

2023-02-23T13:03:00Z

When MMAP2 has the PERF_RECORD_MISC_MMAP_BUILD_ID flag, it means the record already has the build-id info. So it marks the DSO as hit, to skip if the same DSO is not processed if it happens to miss the build-id later. But it missed to copy the MMAP2 record itself so it'd fail to symbolize samples for those regions. For example, the following generates 249 MMAP2 events. $ perf record --buildid-mmap -o- true | perf report --stat -i- | grep MMAP2 MMAP2 events: 249 (86.8%) Adding perf inject should not change the number of events like this $ perf record --buildid-mmap -o- true | perf inject -b | \ > perf report --stat -i- | grep MMAP2 MMAP2 events: 249 (86.5%) But when --buildid-all is used, it eats most of the MMAP2 events. $ perf record --buildid-mmap -o- true | perf inject -b --buildid-all | \ > perf report --stat -i- | grep MMAP2 MMAP2 events: 1 ( 2.5%) With this patch, it shows the original number now. $ perf record --buildid-mmap -o- true | perf inject -b --buildid-all | \ > perf report --stat -i- | grep MMAP2 MMAP2 events: 249 (86.5%) Committer testing: Before: $ perf record --buildid-mmap -o- perf stat --null sleep 1 2> /dev/null | perf inject -b | perf report --stat -i- | grep MMAP2 MMAP2 events: 58 (36.2%) $ perf record --buildid-mmap -o- perf stat --null sleep 1 2> /dev/null | perf report --stat -i- | grep MMAP2 MMAP2 events: 58 (36.2%) $ perf record --buildid-mmap -o- perf stat --null sleep 1 2> /dev/null | perf inject -b --buildid-all | perf report --stat -i- | grep MMAP2 MMAP2 events: 2 ( 1.9%) $ After: $ perf record --buildid-mmap -o- perf stat --null sleep 1 2> /dev/null | perf inject -b | perf report --stat -i- | grep MMAP2 MMAP2 events: 58 (29.3%) $ perf record --buildid-mmap -o- perf stat --null sleep 1 2> /dev/null | perf report --stat -i- | grep MMAP2 MMAP2 events: 58 (34.3%) $ perf record --buildid-mmap -o- perf stat --null sleep 1 2> /dev/null | perf inject -b --buildid-all | perf report --stat -i- | grep MMAP2 MMAP2 events: 58 (38.4%) $ Fixes: f7fc0d1c915a74ff ("perf inject: Do not inject BUILD_ID record if MMAP2 has it") Signed-off-by: Namhyung Kim Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Peter Zijlstra Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20230223070155.54251-1-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf test: Fix offcpu test prev_state check

2023-02-22T15:39:11Z

On Fedora 36, the 'perf record' offcpu profiling tests are failing. It was because the BPF checks the prev task's state being S or D but actually it has more bits set. Let's check the LSB 8 bits for the purpose of offcpu profiling. Reported-by: Arnaldo Carvalho de Melo Signed-off-by: Namhyung Kim Acked-by: Ian Rogers Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: Ingo Molnar Cc: Jiri Olsa Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20230218162724.1292657-1-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf tests stat_all_metrics: Change true workload to sleep workload for system wide check

2023-02-17T20:22:56Z

Testcase stat_all_metrics.sh fails in powerpc: 98: perf all metrics test : FAILED! Logs with verbose: [command]# ./perf test 98 -vv 98: perf all metrics test : --- start --- test child forked, pid 13262 Testing BRU_STALL_CPI Testing COMPLETION_STALL_CPI ---- Testing TOTAL_LOCAL_NODE_PUMPS_P23 Metric 'TOTAL_LOCAL_NODE_PUMPS_P23' not printed in: Error: Invalid event (hv_24x7/PM_PB_LNS_PUMP23,chip=3/) in per-thread mode, enable system wide with '-a'. Testing TOTAL_LOCAL_NODE_PUMPS_RETRIES_P01 Metric 'TOTAL_LOCAL_NODE_PUMPS_RETRIES_P01' not printed in: Error: Invalid event (hv_24x7/PM_PB_RTY_LNS_PUMP01,chip=3/) in per-thread mode, enable system wide with '-a'. ---- Based on above logs, we could see some of the hv-24x7 metric events fails, and logs suggest to run the metric event with -a option. This change happened after the commit a4b8cfcabb1d90ec ("perf stat: Delay metric parsing"), which delayed the metric parsing phase and now before metric parsing phase perf tool identifies, whether target is system-wide or not. With this change, perf_event_open will fails with workload monitoring for uncore events as expected. The perf all metric test case fails as some of the hv-24x7 metric events may need bigger workload with system wide monitoring to get the data. Fix this issue by changing current system wide check from true workload to sleep 0.01 workload. Result with the patch changes in powerpc: 98: perf all metrics test : Ok Fixes: a4b8cfcabb1d90ec ("perf stat: Delay metric parsing") Suggested-by: Ian Rogers Reviewed-by: Athira Rajeev Signed-off-by: Kajol Jain Tested-by: Disha Goel Tested-by: Ian Rogers Cc: Madhavan Srinivasan Cc: Nageswara R Sastry Cc: linuxppc-dev@lists.ozlabs.org Link: https://lore.kernel.org/r/20230215093827.124921-1-kjain@linux.ibm.com Signed-off-by: Arnaldo Carvalho de Melo

perf vendor events power10: Add JSON metric events to present CPI stall cycles in powerpc

2023-02-17T20:20:42Z

Power10 Performance Monitoring Unit (PMU) provides events to understand stall cycles of different pipeline stages. These events along with completed instructions provides useful metrics for application tuning. Patch implements the JSON changes to collect counter statistics to present the high level CPI stall breakdown metrics. New metric group is named as "CPI_STALL_RATIO" and this new metric group presents these stall metrics: - DISPATCHED_CPI ( Dispatch stall cycles per insn ) - ISSUE_STALL_CPI ( Issue stall cycles per insn ) - EXECUTION_STALL_CPI ( Execution stall cycles per insn ) - COMPLETION_STALL_CPI ( Completition stall cycles per insn ) To avoid multipling of events, PM_RUN_INST_CMPL event has been modified to use PMC5(performance monitoring counter5) instead of PMC4. This change is needed, since completion stall event is using PMC4. Usage example: ./perf stat --metric-no-group -M CPI_STALL_RATIO Performance counter stats for 'workload': 63,056,817,982 PM_CMPL_STALL # 0.28 COMPLETION_STALL_CPI 1,743,988,038,896 PM_ISSUE_STALL # 7.73 ISSUE_STALL_CPI 225,597,495,030 PM_RUN_INST_CMPL # 6.18 DISPATCHED_CPI # 37.48 EXECUTION_STALL_CPI 1,393,916,546,654 PM_DISP_STALL_CYC 8,455,376,836,463 PM_EXEC_STALL "--metric-no-group" is used for forcing PM_RUN_INST_CMPL to be scheduled in all group for more accuracy. Signed-off-by: Athira Rajeev Acked-by: Ian Rogers Cc: Andi Kleen Cc: Disha Goel Cc: James Clark Cc: Jiri Olsa Cc: Kajol Jain Cc: Madhavan Srinivasan Cc: Michael Ellerman Cc: Nageswara R Sastry Cc: Namhyung Kim Cc: linuxppc-dev@lists.ozlabs.org Link: https://lore.kernel.org/r/20230216061240.18067-1-atrajeev@linux.vnet.ibm.com Signed-off-by: Arnaldo Carvalho de Melo

perf intel-pt: Synthesize cycle events

2023-02-17T14:02:44Z

There is no good reason why we cannot synthesize "cycle" events from Intel PT just as we can synthesize "instruction" events, in particular when CYC packets are available. This enables using PT to getting much more accurate cycle profiles than regular sampling (record -e cycles) when the work last for very short periods (<10 ms). Thus, add support for this, based off of the existing IPC calculation framework. The new option to --itrace is "y" (for cYcles), as c was taken for calls. Cycle and instruction events can be synthesized together, and are by default. The only real caveat is that CYC packets are only emitted whenever some other packet is, which in practice is when a branch instruction is encountered (and not even all branches). Thus, even at no subsampling (e.g. --itrace=y0ns), it is impossible to get more accuracy than a single basic block, and all cycles spent executing that block will get attributed to the branch instruction that ends the packet. Thus, one cannot know whether the cycles came from e.g. a specific load, a mispredicted branch, or something else. When subsampling (which is the default), the cycle events will get smeared out even more, but will still be generally useful to attribute cycle counts to functions. Reviewed-by: Adrian Hunter Signed-off-by: Steinar H. Gunderson Cc: Alexander Shishkin Cc: Ingo Molnar Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20220322082452.1429091-1-sesse@google.com Signed-off-by: Arnaldo Carvalho de Melo