linux/tools/testing/kunit, branch v5.11

kunit: tool: move kunitconfig parsing into init, make it optional

2021-01-16T00:52:12Z

LinuxSourceTree will unceremoniously crash if the user doesn't call read_kunitconfig() first in a number of functions. And currently every place we create an instance, the caller also calls create_kunitconfig() and read_kunitconfig(). Move these instead into __init__() so they can't be forgotten and to reduce copy-paste. The https://github.com/google/pytype type-checker complained that _config wasn't initialized. With this, kunit_tool now type checks under both pytype and mypy. Add an optional boolean that can be used to disable this for use cases in the future where we might not need/want to load the config. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: fix minor typing issue with None status

2021-01-16T00:51:47Z

The code to handle aggregating statuses didn't check that the status actually got set to some non-None value. Default the value to SUCCESS instead of adding a bunch of `is None` checks. This sorta follows the precedent in commit 3fc48259d525 ("kunit: Don't fail test suites if one of them is empty"). Also slightly simplify the code and add type annotations. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: surface and address more typing issues

2021-01-16T00:49:34Z

The authors of this tool were more familiar with a different type-checker, https://github.com/google/pytype. That's open source, but mypy seems more prevalent (and runs faster). And unlike pytype, mypy doesn't try to infer types so it doesn't check unanotated functions. So annotate ~all functions in kunit tool to increase type-checking coverage. Note: per https://www.python.org/dev/peps/pep-0484/, `__init__()` should be annotated as `-> None`. Doing so makes mypy discover a number of new violations. Exclude main() since we reuse `request` for the different types of requests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: Brendan Higgins Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: Fix spelling of "diagnostic" in kunit_parser

2021-01-16T00:46:02Z

Various helper functions were misspelling "diagnostic" in their names. It finally got annoying, so fix it. Signed-off-by: David Gow Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: Force the use of the 'tty' console for UML

2021-01-04T16:18:38Z

kunit_tool relies on the UML console outputting printk() output to the tty in order to get results. Since the default console driver could change, pass 'console=tty' to the kernel. This is triggered by a change[1] to use ttynull as a fallback console driver which -- by chance or by design -- seems to have changed the default console output on UML, breaking kunit_tool. While this may be fixed, we should be less fragile to such changes in the default. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=757055ae8dedf5333af17b3b5b4b70ba9bc9da4e Signed-off-by: David Gow Fixes: 757055ae8ded ("init/console: Use ttynull as a fallback when there is no console") Reported-by: Andy Shevchenko Tested-by: Andy Shevchenko Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

Merge tag 'kbuild-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

2020-12-22T22:02:39Z

Pull Kbuild updates from Masahiro Yamada: - Use /usr/bin/env for shebang lines in scripts - Remove useless -Wnested-externs warning flag - Update documents - Refactor log handling in modpost - Stop building modules without MODULE_LICENSE() tag - Make the insane combination of 'static' and EXPORT_SYMBOL an error - Improve genksyms to handle _Static_assert() * tag 'kbuild-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: Documentation/kbuild: Document platform dependency practises Documentation/kbuild: Document COMPILE_TEST dependencies genksyms: Ignore module scoped _Static_assert() modpost: turn static exports into error modpost: turn section mismatches to error from fatal() modpost: change license incompatibility to error() from fatal() modpost: turn missing MODULE_LICENSE() into error modpost: refactor error handling and clarify error/fatal difference modpost: rename merror() to error() kbuild: don't hardcode depmod path kbuild: doc: document subdir-y syntax kbuild: doc: clarify the difference between extra-y and always-y kbuild: doc: split if_changed explanation to a separate section kbuild: doc: merge 'Special Rules' and 'Custom kbuild commands' sections kbuild: doc: fix 'List directories to visit when descending' section kbuild: doc: replace arch/$(ARCH)/ with arch/$(SRCARCH)/ kbuild: doc: update the description about kbuild Makefiles Makefile.extrawarn: remove -Wnested-externs warning tweewide: Fix most Shebang lines

tweewide: Fix most Shebang lines

2020-12-08T14:30:04Z

Change every shebang which does not need an argument to use /usr/bin/env. This is needed as not every distro has everything under /usr/bin, sometimes not even bash. Signed-off-by: Finn Behrens Signed-off-by: Masahiro Yamada

kunit: kunit_tool: Correctly parse diagnostic messages

2020-12-01T22:31:04Z

Currently, kunit_tool expects all diagnostic lines in test results to contain ": " somewhere, as both the subtest header and the crash report do. Fix this to accept any line starting with (minus indent) "# " as being a valid diagnostic line. This matches what the TAP spec[1] and the draft KTAP spec[2] are expecting. [1]: http://testanything.org/tap-specification.html [2]: https://lore.kernel.org/linux-kselftest/CY4PR13MB1175B804E31E502221BC8163FD830@CY4PR13MB1175.namprd13.prod.outlook.com/T/ Signed-off-by: David Gow Acked-by: Marco Elver Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: Introduce get_file_path() helper

2020-11-30T20:56:30Z

Helper allows to derive file names depending on --build_dir argument. Signed-off-by: Andy Shevchenko Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: fix extra trailing \n in raw + parsed test output

2020-11-10T20:44:35Z

For simplcity, strip all trailing whitespace from parsed output. I imagine no one is printing out meaningful trailing whitespace via KUNIT_FAIL() or similar, and that if they are, they really shouldn't. `isolate_kunit_output()` yielded liens with trailing \n, which results in artifacty output like this: $ ./tools/testing/kunit/kunit.py run [16:16:46] [FAILED] example_simple_test [16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 [16:16:46] Expected 1 + 1 == 3, but [16:16:46] 1 + 1 == 2 [16:16:46] 3 == 3 [16:16:46] not ok 1 - example_simple_test [16:16:46] After this change: [16:16:46] # example_simple_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:29 [16:16:46] Expected 1 + 1 == 3, but [16:16:46] 1 + 1 == 2 [16:16:46] 3 == 3 [16:16:46] not ok 1 - example_simple_test [16:16:46] We should *not* be expecting lines to end with \n in kunit_tool_test.py for this reason. Do the same for `raw_output()` as well which suffers from the same issue. This is a followup to [1], but rebased onto kunit-fixes to pick up the other raw_output() fix and fixes for kunit_tool_test.py. [1] https://lore.kernel.org/linux-kselftest/20201020233219.4146059-1-dlatypov@google.com/ Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: David Gow Signed-off-by: Shuah Khan