git/t/Makefile, branch v2.45.2

t/Makefile: say the default target upfront

2024-02-03T02:41:35Z

Similar to how 2731d048 (Makefile: say the default target upfront., 2005-12-01) added the default target to the very beginning of the main Makefile to prevent a random rule that happens to be defined first in an included makefile fragments from becoming the default target, protect this Makefile the same way. This started to matter as we started to include config.mak.uname and that included makefile fragment does more than defining Make macros, unfortunately. Helped-by: Johannes Schindelin Signed-off-by: Junio C Hamano

t/Makefile: get UNIT_TESTS list from C sources

2024-01-31T22:41:12Z

We decide on the set of unit tests to run by asking make to expand the wildcard "t/unit-tests/bin/*". One unfortunate outcome of this is that we'll run anything in that directory, even if it is leftover cruft from a previous build. This isn't _quite_ as bad as it sounds, since in theory the unit tests executables are self-contained (so if they passed before, they'll pass even though they now have nothing to do with the checked out version of Git). But at the very least it's wasteful, and if they _do_ fail it can be quite confusing to understand why they are being run at all. This wildcarding presumably came from our handling of the regular shell-script tests, which use $(wildcard t[0-9][0-9][0-9][0-9]-*.sh). But the difference there is that those are actual tracked files. So if you checkout a different commit, they'll go away. Whereas the contents of unit-tests/bin are ignored (so not only do they stick around, but you are not even warned of the stale files via "git status"). This patch fixes the situation by looking for the actual unit-test source files and then massaging those names into the final executable names. This has two additional benefits: 1. It will notice if we failed to build one or more unit-tests for some reason (whereas the current code just runs whatever made it to the bin/ directory). 2. The wildcard should avoid other build cruft, like the pdb files we worked around in 0df903d402 (unit-tests: do not mistake `.pdb` files for being executable, 2023-09-25). Our new wildcard does make an assumption that unit tests are built from C sources. It would be a bit cleaner if we consulted UNIT_TEST_PROGRAMS from the top-level Makefile. But doing so is tricky unless we reorganize that Makefile to split the source file lists into include-able subfiles. That might be worth doing in general, but in the meantime, the assumptions made by the wildcard here seems reasonable. Note that we do need to include config.mak.uname either way, though, as we need the value of $(X) to compute the correct executable names (which would be true even if we had access to the top-level's UNIT_TEST_PROGRAMS variable). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'ps/chainlint-self-check-update'

2023-12-27T22:52:25Z

Test framework update. * ps/chainlint-self-check-update: tests: adjust whitespace in chainlint expectations

tests: adjust whitespace in chainlint expectations

2023-12-15T16:36:14Z

The "check-chainlint" target runs automatically when running tests and performs self-checks to verify that the chainlinter itself produces the expected output. Originally, the chainlinter was implemented via sed, but the infrastructure has been rewritten in fb41727b7e (t: retire unused chainlint.sed, 2022-09-01) to use a Perl script instead. The rewrite caused some slight whitespace changes in the output that are ultimately not of much importance. In order to be able to assert that the actual chainlinter errors match our expectations we thus have to ignore whitespace characters when diffing them. As the `-w` flag is not in POSIX we try to use `git diff -w --no-index` before we fall back to `diff -w -u`. To accomodate for cases where the host system has no Git installation we use the locally-compiled version of Git. This can result in problems though when the Git project's repository is using extensions that the locally-compiled version of Git doesn't understand. It will refuse to run and thus cause the checks to fail. Instead of improving the detection logic, fix our ".expect" files so that we do not need any post-processing at all anymore. This allows us to drop the `-w` flag when diffing so that we can always use diff(1) now. Note that we keep some of the post-processing of `chainlint.pl` output intact to strip leading line numbers generated by the script. Having these would cause a rippling effect whenever we add a new test that sorts into the middle of existing tests and would require us to renumerate all subsequent lines, which seems rather pointless. Signed-off-by: Patrick Steinhardt Reviewed-by: Eric Sunshine Signed-off-by: Junio C Hamano

unit-tests: do not mistake `.pdb` files for being executable

2023-11-09T23:16:27Z

When building the unit tests via CMake, the `.pdb` files are built. Those are, essentially, files containing the debug information separately from the executables. Let's not confuse them with the executables we actually want to run. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

unit tests: add TAP unit test framework

2023-11-09T23:15:32Z

This patch contains an implementation for writing unit tests with TAP output. Each test is a function that contains one or more checks. The test is run with the TEST() macro and if any of the checks fail then the test will fail. A complete program that tests STRBUF_INIT would look like #include "test-lib.h" #include "strbuf.h" static void t_static_init(void) { struct strbuf buf = STRBUF_INIT; check_uint(buf.len, ==, 0); check_uint(buf.alloc, ==, 0); check_char(buf.buf[0], ==, '\0'); } int main(void) { TEST(t_static_init(), "static initialization works); return test_done(); } The output of this program would be ok 1 - static initialization works 1..1 If any of the checks in a test fail then they print a diagnostic message to aid debugging and the test will be reported as failing. For example a failing integer check would look like # check "x >= 3" failed at my-test.c:102 # left: 2 # right: 3 not ok 1 - x is greater than or equal to three There are a number of check functions implemented so far. check() checks a boolean condition, check_int(), check_uint() and check_char() take two values to compare and a comparison operator. check_str() will check if two strings are equal. Custom checks are simple to implement as shown in the comments above test_assert() in test-lib.h. Tests can be skipped with test_skip() which can be supplied with a reason for skipping which it will print. Tests can print diagnostic messages with test_msg(). Checks that are known to fail can be wrapped in TEST_TODO(). There are a couple of example test programs included in this patch. t-basic.c implements some self-tests and demonstrates the diagnostic output for failing test. The output of this program is checked by t0080-unit-test-output.sh. t-strbuf.c shows some example unit tests for strbuf.c The unit tests will be built as part of the default "make all" target, to avoid bitrot. If you wish to build just the unit tests, you can run "make build-unit-tests". To run the tests, you can use "make unit-tests" or run the test binaries directly, as in "./t/unit-tests/bin/t-strbuf". Signed-off-by: Phillip Wood Signed-off-by: Josh Steadmon Signed-off-by: Junio C Hamano

Merge branch 'jk/chainlint-fixes'

2023-04-06T20:38:31Z

Test framework fix. * jk/chainlint-fixes: tests: skip test_eval_ in internal chain-lint tests: drop here-doc check from internal chain-linter tests: diagnose unclosed here-doc in chainlint.pl tests: replace chainlint subshell with a function tests: run internal chain-linter under "make test"

tests: run internal chain-linter under "make test"

2023-03-30T20:07:29Z

Since 69b9924b875 (t/Makefile: teach `make test` and `make prove` to run chainlint.pl, 2022-09-01), we run a single chainlint.pl process for all scripts, and then instruct each individual script to run with the equivalent of --no-chain-lint, which tells them not to redundantly run the chainlint script themselves. However, this also disables the internal linter run within the shell by eval-ing "(exit 117) && $1" and confirming we get code 117. In theory the external linter produces a superset of complaints, and we don't need the internal one anymore. However, we know there is at least one case where they differ. A test like: test_expect_success 'should fail linter' ' false && sleep 2 & pid=$! && kill $pid ' is buggy (it ignores the failure from "false", because it is backgrounded along with the sleep). The internal linter catches this, but the external one doesn't (and teaching it to do so is complicated[1]). So not only does "make test" miss this problem, but it's doubly confusing because running the script standalone does complain. Let's teach the suppression in the Makefile to only turn off the external linter (which we know is redundant, as it was already run) and leave the internal one intact. I've used a new environment variable to do this here, and intentionally did not add a "--no-ext-chain-lint" option. This is an internal optimization used by the Makefile, and not something that ordinary users would need to tweak. [1] For discussion of chainlint.pl and this case, see: https://lore.kernel.org/git/CAPig+cQtLFX4PgXyyK_AAkCvg4Aw2RAC5MmLbib-aHHgTBcDuw@mail.gmail.com/ Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

test: don't print aggregate-results command

2023-03-09T22:57:57Z

There's no value in it. Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano

test: simplify counts aggregation

2023-03-09T22:57:55Z

When the list of files as input was implemented in 6508eedf67 (t/aggregate-results: accomodate systems with small max argument list length, 2010-06-01), a much simpler solution wasn't considered. Let's just pass the directory as an argument. Signed-off-by: Felipe Contreras Signed-off-by: Junio C Hamano