git/t/t0000-basic.sh, branch v2.33.6

t0000: avoid masking git exit value through pipes

2021-09-16T20:43:42Z

9af0b8dbe2 (t0000-basic: more commit-tree tests., 2006-04-26) adds tests for commit-tree that mask the return exit from git as described in a378fee5b07 (Documentation: add shell guidelines, 2018-10-05). Fix the tests, to avoid pipes by using a temporary file instead. Signed-off-by: Carlo Marcelo Arenas Belón Signed-off-by: Junio C Hamano

t0000: fix test if run with TEST_OUTPUT_DIRECTORY

2021-07-20T16:19:02Z

Testcases in t0000 are quite special given that they many of them run nested testcases to verify that testing functionality itself works as expected. These nested testcases are realized by writing a new ad-hoc test script which again sources test-lib.sh, where the new script is created in a nested subdirectory located beneath the current trash directory. We then execute the new test script with the nested subdirectory as current working directory and explicitly re-export TEST_OUTPUT_DIRECTORY to point to that directory. While this works as expected in the general case, it falls apart when the developer has TEST_OUTPUT_DIRECTORY explicitly defined either via the environment or via config.mak and runs "make test". In that case, test-lib.sh will clobber the value that we've just carefully set up to instead contain what the developer has defined. As a result, the TEST_OUTPUT_DIRECTORY continues to point at the root output directory, not at the nested one. This issue causes breakage in the 'test_atexit is run' test case: the nested test case writes files into "../../", which is assumed to be the parent's trash directory. But because TEST_OUTPUT_DIRECTORY already points to to the root output directory, we instead end up writing those files outside of the output directory. The parent test case will then try to check whether those files still exist in its own trash directory, which thus must fail now. Fix the issue by adding a new TEST_OUTPUT_DIRECTORY_OVERRIDE variable. If set, then we'll always override the TEST_OUTPUT_DIRECTORY with its value after sourcing GIT-BUILD-OPTIONS. Signed-off-by: Patrick Steinhardt Acked-by: Jeff King Signed-off-by: Junio C Hamano

t0000: clear GIT_SKIP_TESTS before running sub-tests

2021-07-19T20:26:00Z

In t0000, we run several fake "sub-test" suites to verify the behavior of the test suite. But because we don't clear the parent environment completely, the sub-tests can be fooled by variables meant for the parent. For example: GIT_SKIP_TESTS=t1234 ./t0000-basic.sh fails when a sub-test expects its fake t1234 to actually run. This particular pattern is unlikely in practice; we're running a single script, and there is no t1234 in the real test suite anyway (not yet, at least). A more real-world example is: GIT_SKIP_TESTS=t[^0]* make test to run only the t0* tests. The fix is conceptually simple: we should clear the GIT_SKIP_TESTS variable when running the sub-tests, because its contents (if any) will be meant for the main test suite. This is easy to do centrally in our sub-test helper. But there's a catch: some of our tests do set GIT_SKIP_TESTS intentionally to test the feature. We need to allow them to continue to set it, but clear it for all the other tests. And the sub-test helper can't tell if the GIT_SKIP_TESTS it sees is from a test or not. We can handle this by adding a new option to the helper to let callers specify the skip list. I considered adding a more general "--eval" option to let callers set up the env for the sub-test however they like. That would cover this case and possible future ones. But the quoting gets awkward for the callers (since we're now 2 layers deep in evals!), so I went with the simpler more specific solution. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

test-lib tests: remove dead GIT_TEST_FRAMEWORK_SELFTEST variable

2021-05-11T03:45:17Z

Stop setting the GIT_TEST_FRAMEWORK_SELFTEST variable. This was originally needed back in 4231d1ba99 (t0000: do not get self-test disrupted by environment warnings, 2018-09-20). It hasn't been needed since I deleted the relevant code in test-lib.sh in c0eedbc009 (test-lib: remove check_var_migration, 2021-02-09), I just didn't notice that it was set here. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

tests: remove most uses of C_LOCALE_OUTPUT

2021-02-11T07:48:26Z

As a follow-up to d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) remove those uses of the now always true C_LOCALE_OUTPUT prerequisite from those tests which declare it as an argument to test_expect_{success,failure}. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

t0000: consistently use single quotes for outer tests

2021-01-28T20:06:26Z

When we use the sub-test helpers, we end up defining one shell snippet inside another shell snippet. So if we use single-quotes for the outer snippet, we have to use double-quotes within the inner snippet (it's included as here-doc within the outer snippet, but using a single quote would end the outer snippet early). Or vice versa we can use double quotes for the outer snippet, but then single quotes in the inner. We have some of each in the script, and neither is wrong. But it would be nice to be consistent unless there is a good reason not to. Using single quotes for the outer script is preferable, because it requires less metacharacter quoting overall. For example, in: test_expect_success 'outer' ' run_sub_test_lib_test ... <<-\EOF echo $foo && test_expect_success "inner" " echo \$bar " EOF ' we need only quote inside "inner", but not inside "outer" or the here-doc. Whereas if we flip them, we have to quote in both places: test_expect_success 'outer' " run_sub_test_lib_test ... <<-\EOF echo \$foo && test_expect_success 'inner' ' echo \$bar ' EOF " The exception is when we need a literal single-quote in an expected output here-doc. There we can either use outer double-quotes, or just use ${SQ} within the doc. I chose the latter for consistency (within this test, but also with other test scripts that face the same problem). There is one other interesting case, which is some tests that do: test_expect_success ... " do_something --run='"'!3'"' " This is rather confusing to read, but is correct. The outer script sees '!3' in single-quotes, as does the eval'd snippet. This is perhaps being overly cautious. In many interactive shells, an exclamation triggers history expansion even inside double quotes, but that is not generally true in non-interactive shells. There's some conflicting information here. Commit 784ce03d55 (t4216: avoid unnecessary subshell in test_bloom_filters_not_used, 2020-05-19) reports it as a problem with OpenBSD 6.7's /bin/sh. However, we have many instances in this script of prereqs like !LAZY_TRUE, which haven't been a problem. I left them un-escaped here to test out this theory. It's much nicer if we can not worry about this as a portability issue, so it's worth knowing. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

t0000: run cleaning test inside sub-test

2021-01-28T20:06:26Z

Our check of test_when_finished is done directly in the main script, and if we failed to clean, we complain and exit immediately. It's nicer to signal a test failure here, for a few reasons: - this gives better output to the user when run under a TAP harness like "prove" - constency; it's the only test left in the file that behaves this way - half of its "if" conditional is nonsense anyway; it picked up a reference to GIT_TEST_FAIL_PREREQS_INTERNAL in dfe1a17df9 (tests: add a special setup where prerequisites fail, 2019-05-13) along with its neighbors, even though it has nothing to do with that flag We could actually do this without a sub-test at all, and just put our two tests (one to do cleanup, and one to check that it happened) in the main script. But doing it in a subtest is conceptually cleaner (from the perspective of the main test script, we are checking only one thing), and it remains consistent with the "cleanup when failing" test directly after it, which has to happen in a sub-test (to avoid the main script complaining of the failed test). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

t0000: run prereq tests inside sub-test

2021-01-28T20:06:26Z

We test the behavior of prerequisites in t0000 by setting up fake ones in the main test script, trying to run some tests, and then seeing if those tests impacted the environment correctly. If they didn't, then we write a message and manually call exit. Instead, let's push these down into a sub-test, like many of the other tests covering the framework itself. This has a few advantages: - it does not pollute the test output with mention of skipped tests (that we know are uninteresting -- the point of the test was to see that these are skipped). - when running in a TAP harness, we get a useful test failure message (whereas when the script exits early, a tool like "prove" simply says "Dubious, test returned 1"). - we do not have to worry about different test environments, such as when GIT_TEST_FAIL_PREREQS_INTERNAL is set. Our sub-test helpers already give us a known environment. - the tests themselves are a bit easier to read, as we can just check the test-framework output to see what happened (and get the usual test_cmp diff if it failed) A few notes on the implementation: - we could do one sub-test per each individual test_expect_success. I broke it up here into a few logical groups, as I think this makes it more readable - the original tests modified environment variables inside the test bodies. Instead, I've used "true" as the body of a test we expect to run and "false" otherwise. Technically this does not confirm that the body of the "true" test actually ran. We are trusting the framework output to believe that it truly ran, which is sufficient for these tests. And I think the end result is much simpler to follow. - the nested_prereq test uses a few bare "test -f" calls; I converted these to our usual test_path_is_* helpers while moving the code around. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

t0000: keep clean-up tests together

2021-01-28T20:06:25Z

We check that test_when_finished cleans up after a test, and that it runs even after a failure. Those two were originally adjacent, but got split apart by the new test added in 477dcaddb6 (tests: do not let lazy prereqs inside `test_expect_*` turn off tracing, 2020-03-26), and then further by more lazy-prereq tests. Let's move them back together. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'sg/tests-prereq'

2020-11-25T23:24:54Z

A lazily defined test prerequisite can now be defined in terms of another lazily defined test prerequisite. * sg/tests-prereq: tests: fix description of 'test_set_prereq' tests: make sure nested lazy prereqs work reliably