git/ci, branch v2.34.2

Merge branch 'ab/sanitize-leak-ci'

2021-10-11T17:21:47Z

CI learns to run the leak sanitizer builds. * ab/sanitize-leak-ci: tests: add a test mode for SANITIZE=leak, run it in CI Makefile: add SANITIZE=leak flag to GIT-BUILD-OPTIONS

tests: add a test mode for SANITIZE=leak, run it in CI

2021-09-23T18:29:45Z

While git can be compiled with SANITIZE=leak, we have not run regression tests under that mode. Memory leaks have only been fixed as one-offs without structured regression testing. This change adds CI testing for it. We'll now build and small set of whitelisted t00*.sh tests under Linux with a new job called "linux-leaks". The CI target uses a new GIT_TEST_PASSING_SANITIZE_LEAK=true test mode. When running in that mode, we'll assert that we were compiled with SANITIZE=leak. We'll then skip all tests, except those that we've opted-in by setting "TEST_PASSES_SANITIZE_LEAK=true". A test setting "TEST_PASSES_SANITIZE_LEAK=true" setting can in turn make use of the "SANITIZE_LEAK" prerequisite, should they wish to selectively skip tests even under "GIT_TEST_PASSING_SANITIZE_LEAK=true". In the preceding commit we started doing this in "t0004-unwritable.sh" under SANITIZE=leak, now it'll combine nicely with "GIT_TEST_PASSING_SANITIZE_LEAK=true". This is how tests that don't set "TEST_PASSES_SANITIZE_LEAK=true" will be skipped under GIT_TEST_PASSING_SANITIZE_LEAK=true: $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0001-init.sh 1..0 # SKIP skip all tests in t0001 under SANITIZE=leak, TEST_PASSES_SANITIZE_LEAK not set The intent is to add more TEST_PASSES_SANITIZE_LEAK=true annotations as follow-up change, but let's start small to begin with. In ci/run-build-and-tests.sh we make use of the default "*" case to run "make test" without any GIT_TEST_* modes. SANITIZE=leak is known to fail in combination with GIT_TEST_SPLIT_INDEX=true in t0016-oidmap.sh, and we're likely to have other such failures in various GIT_TEST_* modes. Let's focus on getting the base tests passing, we can expand coverage to GIT_TEST_* modes later. It would also be possible to implement a more lightweight version of this by only relying on setting "LSAN_OPTIONS". See [1] and [2] for a discussion of that. I've opted for this approach of adding a GIT_TEST_* mode instead because it's consistent with how we handle other special test modes. Being able to add a "!SANITIZE_LEAK" prerequisite and calling "test_done" early if it isn't satisfied also means that we can more incrementally add regression tests without being forced to fix widespread and hard-to-fix leaks at the same time. We have tests that do simple checking of some tool we're interested in, but later on in the script might be stressing trace2, or common sources of leaks like "git log" in combination with the tool (e.g. the commit-graph tests). To be clear having a prerequisite could also be accomplished by using "LSAN_OPTIONS" directly. On the topic of "LSAN_OPTIONS": It would be nice to have a mode to aggregate all failures in our various scripts, see [2] for a start at doing that which sets "log_path" in "LSAN_OPTIONS". I've punted on that for now, it can be added later. As of writing this we've got major regressions between master..seen, i.e. the t000*.sh tests and more fixed since 31f9acf9ce2 (Merge branch 'ah/plugleaks', 2021-08-04) have regressed recently. See the discussion at <87czsv2idy.fsf@evledraar.gmail.com>[3] about the lack of this sort of test mode, and 0e5bba53af (add UNLEAK annotation for reducing leak false positives, 2017-09-08) for the initial addition of SANITIZE=leak. See also 09595ab381 (Merge branch 'jk/leak-checkers', 2017-09-19), 7782066f67 (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent 936e58851a (Merge branch 'ah/plugleaks', 2021-05-07) for some of the past history of "one-off" SANITIZE=leak (and more) fixes. As noted in [5] we can't support this on OSX yet until Clang 14 is released, at that point we'll probably want to resurrect that "osx-leaks" job. 1. https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer 2. https://lore.kernel.org/git/YS9OT%2Fpn5rRK9cGB@coredump.intra.peff.net/ 3. https://lore.kernel.org/git/87czsv2idy.fsf@evledraar.gmail.com/ 4. https://lore.kernel.org/git/YS9ZIDpANfsh7N+S@coredump.intra.peff.net/ 5. https://lore.kernel.org/git/20210916035603.76369-1-carenas@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Carlo Marcelo Arenas Belón Signed-off-by: Junio C Hamano

Merge branch 'tb/multi-pack-bitmaps'

2021-09-20T22:20:39Z

The reachability bitmap file used to be generated only for a single pack, but now we've learned to generate bitmaps for history that span across multiple packfiles. * tb/multi-pack-bitmaps: (29 commits) pack-bitmap: drop bitmap_index argument from try_partial_reuse() pack-bitmap: drop repository argument from prepare_midx_bitmap_git() p5326: perf tests for MIDX bitmaps p5310: extract full and partial bitmap tests midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' t7700: update to work with MIDX bitmap test knob t5319: don't write MIDX bitmaps in t5319 t5310: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t0410: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t5326: test multi-pack bitmap behavior t/helper/test-read-midx.c: add --checksum mode t5310: move some tests to lib-bitmap.sh pack-bitmap: write multi-pack bitmaps pack-bitmap: read multi-pack bitmaps pack-bitmap.c: avoid redundant calls to try_partial_reuse pack-bitmap.c: introduce 'bitmap_is_preferred_refname()' pack-bitmap.c: introduce 'nth_bitmap_object_oid()' pack-bitmap.c: introduce 'bitmap_num_objects()' midx: avoid opening multiple MIDXs when writing midx: close linked MIDXs, avoid leaking memory ...

Merge branch 'cb/ci-build-pedantic'

2021-09-10T18:46:32Z

CI update. * cb/ci-build-pedantic: ci: run a pedantic build as part of the GitHub workflow

midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP'

2021-09-01T20:56:43Z

Introduce a new 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' environment variable to also write a multi-pack bitmap when 'GIT_TEST_MULTI_PACK_INDEX' is set. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

ci: run a pedantic build as part of the GitHub workflow

2021-08-11T18:25:06Z

similar to the recently added sparse task, it is nice to know as early as possible. add a dockerized build using fedora (that usually has the latest gcc) to be ahead of the curve and avoid older ISO C issues at the same time. Signed-off-by: Carlo Marcelo Arenas Belón Signed-off-by: Junio C Hamano

ci/install-dependencies: handle "sparse" job package installs

2021-07-26T22:20:51Z

This just matches the style/location of the package installation for other jobs. There should be no functional change. I did flip the order of the options and command-name ("-y update" instead of "update -y") for consistency with other lines in the same file. Note also that we have to reorder the dependency install with the "checkout" action, so that we actually have the "ci" scripts available. Signed-off-by: Jeff King Acked-by: Johannes Schindelin Signed-off-by: Junio C Hamano

Merge branch 'dd/svn-test-wo-locale-a'

2021-07-08T20:14:58Z

"git-svn" tests assumed that "locale -a", which is used to pick an available UTF-8 locale, is available everywhere. A knob has been introduced to allow testers to specify a suitable locale to use. * dd/svn-test-wo-locale-a: t: use user-specified utf-8 locale for testing svn

t: use user-specified utf-8 locale for testing svn

2021-06-08T07:07:37Z

In some test-cases, UTF-8 locale is required. To find such locale, we're using the first available UTF-8 locale that returned by "locale -a". However, the locale(1) utility is unavailable on some systems, e.g. Linux with musl libc. However, without "locale -a", we can't guess provided UTF-8 locale. Add a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for linux-musl in our CI system. Rename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale, since we no longer prepare the variable named "a_utf8_locale", but set up a fallback value for GIT_TEST_UTF8_LOCALE instead. The fallback will be LC_ALL, LANG environment variable, or the first UTF-8 locale from output of "locale -a", in that order. Signed-off-by: Đoàn Trần Công Danh Signed-off-by: Junio C Hamano

Merge branch 'mt/parallel-checkout-part-3'

2021-05-16T12:05:23Z

The final part of "parallel checkout". * mt/parallel-checkout-part-3: ci: run test round with parallel-checkout enabled parallel-checkout: add tests related to .gitattributes t0028: extract encoding helpers to lib-encoding.sh parallel-checkout: add tests related to path collisions parallel-checkout: add tests for basic operations checkout-index: add parallel checkout support builtin/checkout.c: complete parallel checkout support make_transient_cache_entry(): optionally alloc from mem_pool