linux/tools/testing/kunit/kunit_tool_test.py, branch v5.9

Merge tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

2020-08-05T17:07:39Z

Pull kunit updates from Shuah Khan: - Add a generic kunit_resource API extending it to support resources that are passed in to kunit in addition kunit allocated resources. In addition, KUnit resources are now refcounted to avoid passed in resources being released while in use by kunit. - Add support for named resources. - Important bug fixes from Brendan Higgins and Will Chen * tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: kunit: tool: fix improper treatment of file location kunit: tool: fix broken default args in unit tests kunit: capture stderr on all make subprocess calls Documentation: kunit: Remove references to --defconfig kunit: add support for named resources kunit: generalize kunit_resource API beyond allocated resources

kunit: tool: fix improper treatment of file location

2020-07-17T20:17:49Z

Commit 01397e822af4 ("kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig") and commit 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse") introduced two closely related issues which built off of each other: they excessively created the build directory when not present and modified a constant (constants in Python only exist by convention). Together these issues broken a number of unit tests for KUnit tool, so fix them. Fixed up commit log to fic checkpatch commit description style error. Shuah Khan Fixes: 01397e822af4 ("kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig") Fixes: 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse") Signed-off-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: fix broken default args in unit tests

2020-07-17T20:17:42Z

Commit ddbd60c779b4 ("kunit: use --build_dir=.kunit as default") changed the default build directory for KUnit tests, but failed to update associated unit tests for kunit_tool, so update them. Fixes: ddbd60c779b4 ("kunit: use --build_dir=.kunit as default") Signed-off-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: show error if kunit results are not present

2020-06-26T20:29:10Z

Currently, if the kernel is configured incorrectly or if it crashes before any kunit tests are run, kunit finishes without error, reporting that 0 test cases were run. To fix this, an error is shown when the tap header is not found, which indicates that kunit was not able to run at all. Signed-off-by: Uriel Guajardo Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: kunit_tool: Separate out config/build/exec/parse

2020-05-11T22:53:40Z

Add new subcommands to kunit.py to allow stages of the existing 'run' subcommand to be run independently: - 'config': Verifies that .config is a subset of .kunitconfig - 'build': Compiles a UML kernel for KUnit - 'exec': Runs the kernel, and outputs the test results. - 'parse': Parses test results from a file or stdin The 'run' command continues to behave as before. Signed-off-by: David Gow Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: kunit_tool: Allow .kunitconfig to disable config items

2020-03-25T18:13:16Z

Rework kunit_tool in order to allow .kunitconfig files to better enforce that disabled items in .kunitconfig are disabled in the generated .config. Previously, kunit_tool simply enforced that any line present in .kunitconfig was also present in .config, but this could cause problems if a config option was disabled in .kunitconfig, but not listed in .config due to (for example) having disabled dependencies. To fix this, re-work the parser to track config names and values, and require values to match unless they are explicitly disabled with the "CONFIG_x is not set" comment (or by setting its value to 'n'). Those "disabled" values will pass validation if omitted from the .config, but not if they have a different value. Signed-off-by: David Gow Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: Run all KUnit tests through allyesconfig

2020-03-20T22:17:39Z

Implemented the functionality to run all KUnit tests through kunit_tool by specifying an --alltests flag, which builds UML with allyesconfig enabled, and consequently runs every KUnit test. A new function was added to kunit_kernel: make_allyesconfig. Firstly, if --alltests is specified, kunit.py triggers build_um_kernel which call make_allyesconfig. This function calls the make command, disables the broken configs that would otherwise prevent UML from building, then starts the kernel with all possible configurations enabled. All stdout and stderr is sent to test.log and read from there then fed through kunit_parser to parse the tests to the user. Also added a signal_handler in case kunit is interrupted while running. Tested: Run under different conditions such as testing with --raw_output, testing program interrupt then immediately running kunit again without --alltests and making sure to clean the console. Signed-off-by: Heidi Fahim Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: kunit_parser: make parser more robust

2020-03-20T22:15:57Z

Previously, kunit_parser did not properly handle kunit TAP output that - had any prefixes (generated from different configs e.g. CONFIG_PRINTK_TIME) - had unrelated kernel output mixed in the middle of it, which has shown up when testing with allyesconfig To remove prefixes, the parser looks for the first line that includes TAP output, "TAP version 14". It then determines the length of the string before this sequence, and strips that number of characters off the beginning of the following lines until the last KUnit output line is reached. These fixes have been tested with additional tests in the KUnitParseTest and their associated logs have also been added. Signed-off-by: Heidi Fahim Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit/kunit_tool_test: Test '--build_dir' option run

2019-12-23T17:52:41Z

This commit adds kunit tool test for the '--build_dir' option. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: Place 'test.log' under the 'build_dir'

2019-12-23T17:52:41Z

'kunit' writes the 'test.log' under the kernel source directory even though a 'build_dir' option is given. As users who use the option might expect the outputs to be placed under the specified directory, this commit modifies the logic to write the log file under the 'build_dir'. Signed-off-by: SeongJae Park Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan