linux/tools/testing/selftests/Makefile, branch v5.4

kselftest: exclude failed TARGETS from runlist

2019-10-01T19:28:38Z

A TARGET which failed to be built/installed should not be included in the runlist generated inside the run_kselftest.sh script. Signed-off-by: Cristian Marussi Signed-off-by: Shuah Khan

kselftest: add capability to skip chosen TARGETS

2019-10-01T19:19:04Z

Let the user specify an optional TARGETS skiplist through the new optional SKIP_TARGETS Makefile variable. It is easier to skip at will using a reduced and well defined list of possibly problematic targets with SKIP_TARGETS than to provide a partially stripped down list of good targets using the usual TARGETS variable. Signed-off-by: Cristian Marussi Signed-off-by: Shuah Khan

selftests: Add kselftest-all and kselftest-install targets

2019-10-01T16:11:08Z

Add kselftest-all target to build tests from the top level Makefile. This is to simplify kselftest use-cases for CI and distributions where build and test systems are different. Current kselftest target builds and runs tests on a development system which is a developer use-case. Add kselftest-install target to install tests from the top level Makefile. This is to simplify kselftest use-cases for CI and distributions where build and test systems are different. This change addresses requests from developers and testers to add support for installing kselftest from the main Makefile. In addition, make the install directory the same when install is run using "make kselftest-install" or by running kselftest_install.sh. Also fix the INSTALL_PATH variable conflict between main Makefile and selftests Makefile. Signed-off-by: Shuah Khan Acked-by: Masahiro Yamada Signed-off-by: Shuah Khan

selftests: use "$(MAKE)" instead of "make"

2019-08-19T20:21:36Z

When doing "make kselftest TARGETS=bpf -j12", bpf progs end up being compiled sequentially and thus slowly. The reason is that parent make (tools/testing/selftests/Makefile) does not share its jobserver with child make (tools/testing/selftests/bpf/Makefile), therefore the latter runs with -j1. Change all instances of "make" to "$(MAKE)", so that the whole make hierarchy runs using a single jobserver. Signed-off-by: Ilya Leoshkevich Signed-off-by: Shuah Khan

kbuild: replace KBUILD_SRCTREE with boolean building_out_of_srctree

2019-07-10T15:05:09Z

Commit 25b146c5b8ce ("kbuild: allow Kbuild to start from any directory") deprecated KBUILD_SRCTREE. It is only used in tools/testing/selftest/ to distinguish out-of-tree build. Replace it with a new boolean flag, building_out_of_srctree. I also replaced the conditional ($(srctree),.) because the next commit will allow an absolute path to be used for $(srctree) even when building in the source tree. Signed-off-by: Masahiro Yamada

selftests: avoid KBUILD_OUTPUT dir cluttering with selftest objects

2019-05-14T23:37:41Z

Running "make kselftest" or building selftests when KBUILD_OUTPUT is set, will create selftest objects in the KBUILD_OUTPUT directory. This could be undesirable especially when user didn't intend to relocate selftest objects. Use KBUILD_OUTPUT/kselftest to create selftest objects instead of cluttering the main directory. Signed-off-by: Shuah Khan

selftests: fix install target to use default install path

2019-05-09T15:48:47Z

Install target fails when INSTALL_PATH is undefined. Fix install target to use "output_dir/install as the default install location. "output_dir" is either the root of selftests directory under kernel source tree or output directory specified by O= or KBUILD_OUTPUT. e.g: make -C tools/testing/selftests install make O=/tmp/kselftest -C tools/testing/selftests install export KBUILD_OUTPUT=/tmp/kselftest make -C tools/testing/selftests install In addition, add "all" target as dependency to "install" to build and install using a single command. Signed-off-by: Shuah Khan

selftests: Remove KSFT_TAP_LEVEL

2019-04-25T19:15:26Z

Since sub-testing can now be detected by indentation level, this removes KSFT_TAP_LEVEL so that subtests report their TAP header for later parsing. Signed-off-by: Kees Cook Signed-off-by: Shuah Khan

selftests: Move test output to diagnostic lines

2019-04-25T19:15:19Z

This changes the selftest output so that each test's output is prefixed with "# " as a TAP "diagnostic line". This creates a bit of a kernel-specific TAP dialect where the diagnostics precede the results. The TAP spec isn't entirely clear about this, though, so I think it's the correct solution so as to keep interactive runs making sense. If the output _followed_ the result line in the spec-suggested YAML form, each test would dump all of its output at once instead of as it went, making debugging harder. This does, however, solve the recursive TAP output problem, as sub-tests will simply be prefixed by "# ". Parsing sub-tests becomes a simple problem of just removing the first two characters of a given top-level test's diagnostic output, and parsing the results. Note that the shell construct needed to both get an exit code from the first command in a pipe and still filter the pipe (to add the "# " prefix) uses a POSIX solution rather than the bash "pipefail" option which is not supported by dash. Since some test environments may have a very minimal set of utilities available, the new prefixing code will fall back to doing line-at-a-time prefixing if perl and/or stdbuf are not available. Signed-off-by: Kees Cook Signed-off-by: Shuah Khan

selftests: Extract logic for multiple test runs

2019-04-25T19:14:38Z

This moves the logic for running multiple tests into a single "run_many" function of runner.sh. Both "run_tests" and "emit_tests" are modified to use it. Summary handling is now controlled by the "per_test_logging" shell flag. Signed-off-by: Kees Cook Signed-off-by: Shuah Khan