linux/tools/testing/kunit, branch v5.12

kunit: tool: Disable PAGE_POISONING under --alltests

2021-03-11T21:37:37Z

kunit_tool maintains a list of config options which are broken under UML, which we exclude from an otherwise 'make ARCH=um allyesconfig' build used to run all tests with the --alltests option. Something in UML allyesconfig is causing segfaults when page poisining is enabled (and is poisoning with a non-zero value). Previously, this didn't occur, as allyesconfig enabled the CONFIG_PAGE_POISONING_ZERO option, which worked around the problem by zeroing memory. This option has since been removed, and memory is now poisoned with 0xAA, which triggers segfaults in many different codepaths, preventing UML from booting. Note that we have to disable both CONFIG_PAGE_POISONING and CONFIG_DEBUG_PAGEALLOC, as the latter will 'select' the former on architectures (such as UML) which don't implement __kernel_map_pages(). Ideally, we'd fix this properly by tracking down the real root cause, but since this is breaking KUnit's --alltests feature, it's worth disabling there in the meantime so the kernel can boot to the point where tests can actually run. Fixes: f289041ed4cf ("mm, page_poison: remove CONFIG_PAGE_POISONING_ZERO") Signed-off-by: David Gow Acked-by: Vlastimil Babka Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: Fix a python tuple typing error

2021-03-11T21:37:37Z

The first argument to namedtuple() should match the name of the type, which wasn't the case for KconfigEntryBase. Fixing this is enough to make mypy show no python typing errors again. Fixes 97752c39bd ("kunit: kunit_tool: Allow .kunitconfig to disable config items") Signed-off-by: David Gow Reviewed-by: Daniel Latypov Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: fix unintentional statefulness in run_kernel()

2021-02-08T23:10:22Z

This is a bug that has been present since the first version of this code. Using [] as a default parameter is dangerous, since it's mutable. Example using the REPL: >>> def bad(param = []): ... param.append(len(param)) ... print(param) ... >>> bad() [0] >>> bad() [0, 1] This wasn't a concern in the past since it would just keep appending the same values to it. E.g. before, `args` would just grow in size like: [mem=1G', 'console=tty'] [mem=1G', 'console=tty', mem=1G', 'console=tty'] But with now filter_glob, this is more dangerous, e.g. run_kernel(filter_glob='my-test*') # default modified here run_kernel() # filter_glob still applies here! That earlier `filter_glob` will affect all subsequent calls that don't specify `args`. Note: currently the kunit tool only calls run_kernel() at most once, so it's not possible to trigger any negative side-effects right now. Fixes: 6ebf5866f2e8 ("kunit: tool: add Python wrappers for running KUnit tests") Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: add support for filtering suites by glob

2021-02-08T23:10:00Z

This allows running different subsets of tests, e.g. $ ./tools/testing/kunit/kunit.py build $ ./tools/testing/kunit/kunit.py exec 'list*' $ ./tools/testing/kunit/kunit.py exec 'kunit*' This passes the "kunit_filter.glob" commandline option to the UML kernel, which currently only supports filtering by suite name. Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: make kunit_tool accept optional path to .kunitconfig fragment

2021-02-08T22:42:48Z

Currently running tests via KUnit tool means tweaking a .kunitconfig file, which you'd keep around locally and never commit. This changes makes it so users can pass in a path to a kunitconfig. One of the imagined use cases is having kunitconfig fragments in-tree to formalize interesting sets of tests for features/subsystems, e.g. $ ./tools/testing/kunit/kunit.py run --kunticonfig=fs/ext4/kunitconfig For now, this hypothetical fs/ext4/kunitconfig would contain CONFIG_KUNIT=y CONFIG_EXT4_FS=y CONFIG_EXT4_KUNIT_TESTS=y At the moment, it's not hard to manually whip up this file, but as more and more tests get added, this will get tedious. It also opens the door to documenting how to run all the tests relevant to a specific subsystem or feature as a simple one-liner. This can be seen as an analogue to tools/testing/selftests/*/config But in the case of KUnit, the tests live in the same directory as the code-under-test, so it feels more natural to allow the kunitconfig fragments to live anywhere. (Though, people could create a separate directory if wanted; this patch imposes no restrictions on the path). Signed-off-by: Daniel Latypov Reviewed-by: Brendan Higgins Tested-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: simplify kconfig is_subset_of() logic

2021-02-08T22:38:55Z

Don't use an O(nm) algorithm* and make it more readable by using a dict. *Most obviously, it does a nested for-loop over the entire other config. A bit more subtle, it calls .entries(), which constructs a set from the list for _every_ outer iteration. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: Brendan Higgins Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

minor: kunit: tool: fix unit test so it can run from non-root dir

2021-02-08T22:37:28Z

Also take this time to rename get_absolute_path() to test_data_path(). 1. the name is currently a lie. It gives relative paths, e.g. if I run from the same dir as the test file, it gives './test_data/' See https://docs.python.org/3/reference/import.html#__file__, which doesn't stipulate that implementations provide absolute paths. 2. it's only used for generating paths to tools/testing/kunit/test_data/ So we can tersen things by making it less general. Cache the absolute path to the test data files per suggestion from [1]. Using relative paths, the tests break because of this code in kunit.py if get_kernel_root_path(): os.chdir(get_kernel_root_path()) [1] https://lore.kernel.org/linux-kselftest/CABVgOSnH0gz7z5JhRCGyG1wg0zDDBTLoSUCoB-gWMeXLgVTo2w@mail.gmail.com/ Fixes: 5578d008d9e0 ("kunit: tool: fix running kunit_tool from outside kernel tree") Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: Brendan Higgins Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: use `with open()` in unit test

2021-02-08T22:35:57Z

The use of manual open() and .close() calls seems to be an attempt to keep the contents in scope. But Python doesn't restrict variables like that, so we can introduce new variables inside of a `with` and use them outside. Do so to make the code more Pythonic. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: Brendan Higgins Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: stop using bare asserts in unit test

2021-02-08T22:34:50Z

Use self.assertEqual/assertNotEqual() instead. Besides being more appropriate in a unit test, it'll also give a better error message by show the unexpected values. Also * Delete redundant check of exception types. self.assertRaises does this. * s/kall/call. There's no reason to name it this way. * This is probably a misunderstanding from the docs which uses it since `mock.call` is in scope as `call`. Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: Brendan Higgins Acked-by: Brendan Higgins Signed-off-by: Shuah Khan

kunit: tool: fix unit test cleanup handling

2021-02-08T22:33:32Z

* Stop leaking file objects. * Use self.addCleanup() to ensure we call cleanup functions even if setUp() fails. * use mock.patch.stopall instead of more error-prone manual approach Signed-off-by: Daniel Latypov Reviewed-by: David Gow Tested-by: Brendan Higgins Acked-by: Brendan Higgins Signed-off-by: Shuah Khan