git/t/t0000-basic.sh, branch v2.35.2

Merge branch 'ab/lib-subtest'

2021-10-13T22:15:57Z

Updates to the tests in t0000 to test the test framework. * ab/lib-subtest: test-lib tests: get rid of copy/pasted mock test code test-lib tests: assert 1 exit code, not non-zero test-lib tests: refactor common part of check_sub_test_lib_test*() test-lib tests: avoid subshell for "test_cmp" for readability test-lib tests: don't provide a description for the sub-tests test-lib tests: split up "write and run" into two functions test-lib tests: move "run_sub_test" to a new lib-subtest.sh

test-lib tests: get rid of copy/pasted mock test code

2021-09-22T23:22:41Z

Now that we've split up the write_sub_test_lib_test*() and run_sub_test_lib_test*() functions let's fix those tests in t0000-basic.sh that were verbosely copy/pasting earlier tests. That we caught all of them was asserted with a follow-up change that's not part of this series[1], we might add such a duplication check at some later time, but for now let's just one-off remove the duplicate boilerplate. 1. https://lore.kernel.org/git/patch-v3-6.9-bc79b29f3c-20210805T103237Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

test-lib tests: don't provide a description for the sub-tests

2021-09-22T23:22:41Z

Change the $test_description provided for the generated subtests to be constant, since the only purpose of having it is that test-lib.sh will barf if it isn't supplied. The other purpose of having it was to effectively split up the test description between the argument to test_expect_success and the argument to "write_and_run_sub_test_lib_test". Let's only use one of the two. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

test-lib tests: split up "write and run" into two functions

2021-09-22T23:22:41Z

Refactor the function to write and run tests of the test-lib.sh output into two functions. When this was added back in 565b6fa87bb (tests: refactor mechanics of testing in a sub test-lib, 2012-12-16) there was no reason to do this, but since we started supporting test arguments in 517cd55fd51 (test-lib: self-test that --verbose works, 2013-06-23) we've started to write out duplicate tests simply to test different arguments, now we'll be able to re-use them. This change doesn't consolidate any of those tests yet, it just makes it possible to do so. All the changes in t0000-basic.sh are a simple search-replacement. Since the _run_sub_test_lib_test_common() function doesn't handle running the test anymore we can do away with the sub-shell, which was used to scope an "unset" and "export" shell variables. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

t0000: avoid masking git exit value through pipes

2021-09-16T20:43:42Z

9af0b8dbe2 (t0000-basic: more commit-tree tests., 2006-04-26) adds tests for commit-tree that mask the return exit from git as described in a378fee5b07 (Documentation: add shell guidelines, 2018-10-05). Fix the tests, to avoid pipes by using a temporary file instead. Signed-off-by: Carlo Marcelo Arenas Belón Signed-off-by: Junio C Hamano

test-lib tests: move "run_sub_test" to a new lib-subtest.sh

2021-07-21T23:27:04Z

Move the "check_sub_test_lib_test()" and its sister functions to a new lib-subtest.sh. In the future (not in this series) I'd like to test test-lib's output in a more targeted and smaller test, and I'll need these functions to do that. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

t0000: fix test if run with TEST_OUTPUT_DIRECTORY

2021-07-20T16:19:02Z

Testcases in t0000 are quite special given that they many of them run nested testcases to verify that testing functionality itself works as expected. These nested testcases are realized by writing a new ad-hoc test script which again sources test-lib.sh, where the new script is created in a nested subdirectory located beneath the current trash directory. We then execute the new test script with the nested subdirectory as current working directory and explicitly re-export TEST_OUTPUT_DIRECTORY to point to that directory. While this works as expected in the general case, it falls apart when the developer has TEST_OUTPUT_DIRECTORY explicitly defined either via the environment or via config.mak and runs "make test". In that case, test-lib.sh will clobber the value that we've just carefully set up to instead contain what the developer has defined. As a result, the TEST_OUTPUT_DIRECTORY continues to point at the root output directory, not at the nested one. This issue causes breakage in the 'test_atexit is run' test case: the nested test case writes files into "../../", which is assumed to be the parent's trash directory. But because TEST_OUTPUT_DIRECTORY already points to to the root output directory, we instead end up writing those files outside of the output directory. The parent test case will then try to check whether those files still exist in its own trash directory, which thus must fail now. Fix the issue by adding a new TEST_OUTPUT_DIRECTORY_OVERRIDE variable. If set, then we'll always override the TEST_OUTPUT_DIRECTORY with its value after sourcing GIT-BUILD-OPTIONS. Signed-off-by: Patrick Steinhardt Acked-by: Jeff King Signed-off-by: Junio C Hamano

t0000: clear GIT_SKIP_TESTS before running sub-tests

2021-07-19T20:26:00Z

In t0000, we run several fake "sub-test" suites to verify the behavior of the test suite. But because we don't clear the parent environment completely, the sub-tests can be fooled by variables meant for the parent. For example: GIT_SKIP_TESTS=t1234 ./t0000-basic.sh fails when a sub-test expects its fake t1234 to actually run. This particular pattern is unlikely in practice; we're running a single script, and there is no t1234 in the real test suite anyway (not yet, at least). A more real-world example is: GIT_SKIP_TESTS=t[^0]* make test to run only the t0* tests. The fix is conceptually simple: we should clear the GIT_SKIP_TESTS variable when running the sub-tests, because its contents (if any) will be meant for the main test suite. This is easy to do centrally in our sub-test helper. But there's a catch: some of our tests do set GIT_SKIP_TESTS intentionally to test the feature. We need to allow them to continue to set it, but clear it for all the other tests. And the sub-test helper can't tell if the GIT_SKIP_TESTS it sees is from a test or not. We can handle this by adding a new option to the helper to let callers specify the skip list. I considered adding a more general "--eval" option to let callers set up the env for the sub-test however they like. That would cover this case and possible future ones. But the quoting gets awkward for the callers (since we're now 2 layers deep in evals!), so I went with the simpler more specific solution. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

test-lib tests: remove dead GIT_TEST_FRAMEWORK_SELFTEST variable

2021-05-11T03:45:17Z

Stop setting the GIT_TEST_FRAMEWORK_SELFTEST variable. This was originally needed back in 4231d1ba99 (t0000: do not get self-test disrupted by environment warnings, 2018-09-20). It hasn't been needed since I deleted the relevant code in test-lib.sh in c0eedbc009 (test-lib: remove check_var_migration, 2021-02-09), I just didn't notice that it was set here. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

tests: remove most uses of C_LOCALE_OUTPUT

2021-02-11T07:48:26Z

As a follow-up to d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) remove those uses of the now always true C_LOCALE_OUTPUT prerequisite from those tests which declare it as an argument to test_expect_{success,failure}. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano