git/ci, branch v2.48.2

Merge branch 'jk/lsan-race-ignore-false-positive'

2025-01-02T21:37:08Z

CI jobs that run threaded programs under LSan has been giving false positives from time to time, which has been worked around. This is an alternative to the jk/lsan-race-with-barrier topic with much smaller change to the production code. * jk/lsan-race-ignore-false-positive: test-lib: ignore leaks in the sanitizer's thread code test-lib: check leak logs for presence of DEDUP_TOKEN test-lib: simplify leak-log checking test-lib: rely on logs to detect leaks Revert barrier-based LSan threading race workaround

Revert barrier-based LSan threading race workaround

2025-01-01T22:13:01Z

The extra "barrier" approach was too much code whose sole purpose was to work around a race that is not even ours (i.e. in LSan's teardown code). In preparation for queuing a solution taking a much-less-invasive approach, let's revert them.

Merge branch 'jk/lsan-race-with-barrier'

2025-01-01T17:21:15Z

CI jobs that run threaded programs under LSan has been giving false positives from time to time, which has been worked around. * jk/lsan-race-with-barrier: grep: work around LSan threading race with barrier index-pack: work around LSan threading race with barrier thread-utils: introduce optional barrier type Revert "index-pack: spawn threads atomically" test-lib: use individual lsan dir for --stress runs

Merge branch 'ps/weak-sha1-for-tail-sum-fix'

2025-01-01T17:21:14Z

An earlier "csum-file checksum does not have to be computed with sha1dc" topic had a few code paths that had initialized an implementation of a hash function to be used by an unmatching hash by mistake, which have been corrected. * ps/weak-sha1-for-tail-sum-fix: ci: exercise unsafe OpenSSL backend builtin/fast-import: fix segfault with unsafe SHA1 backend bulk-checkin: fix segfault with unsafe SHA1 backend

ci: exercise unsafe OpenSSL backend

2024-12-30T14:46:30Z

In the preceding commit we have fixed a segfault when using an unsafe SHA1 backend that is different from the safe one. This segfault only went by unnoticed because we never set up an unsafe backend in our CI systems. Fix this ommission by setting `OPENSSL_SHA1_UNSAFE` in our TEST-vars job. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

thread-utils: introduce optional barrier type

2024-12-30T14:18:57Z

One thread primitive we don't yet support is a barrier: it waits for all threads to reach a synchronization point before letting any of them continue. This would be useful for avoiding the LSan race we see in index-pack (and other places) by having all threads complete their initialization before any of them start to do real work. POSIX introduced a pthread_barrier_t in 2004, which does what we want. But if we want to rely on it: 1. Our Windows pthread emulation would need a new set of wrapper functions. There's a Synchronization Barrier primitive there, which was introduced in Windows 8 (which is old enough for us to depend on). 2. macOS (and possibly other systems) has pthreads but not pthread_barrier_t. So there we'd have to implement our own barrier based on the mutex and cond primitives. Those are do-able, but since we only care about avoiding races in our LSan builds, there's an easier way: make it a noop on systems without a native pthread barrier. This patch introduces a "maybe_thread_barrier" API. The clunky name (rather than just using pthread_barrier directly) should hopefully clue people in that on some systems it will do nothing. It's wired to a Makefile knob which has to be triggered manually, and we enable it for the linux-leaks CI jobs (since we know we'll have it there). There are some other possible options: - we could turn it on all the time for Linux systems based on uname. But we really only care about it for LSan builds, and there is no need to add extra code to regular builds. - we could turn it on only for LSan builds. But that would break builds on non-Linux platforms (like macOS) that otherwise should support sanitizers. - we could trigger only on the combination of Linux and LSan together. This isn't too hard to do, but the uname check isn't completely accurate. It is really about what your libc supports, and non-glibc systems might not have it (though at least musl seems to). So we'd risk breaking builds on those systems, which would need to add a new knob. Though the upside would be that running local "make SANITIZE=leak test" would be protected automatically. And of course none of this protects LSan runs from races on systems without pthread barriers. It's probably OK in practice to protect only our CI jobs, though. The race is rare-ish and most leak-checking happens through CI. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'ps/ci-meson'

2024-12-23T17:32:25Z

The meson-build procedure is integrated into CI to catch and prevent bitrotting. * ps/ci-meson: ci: wire up Meson builds t: introduce compatibility options to clar-based tests t: fix out-of-tree tests for some git-p4 tests Makefile: detect missing Meson tests meson: detect missing tests at configure time t/unit-tests: rename clar-based unit tests to have a common prefix Makefile: drop -DSUPPRESS_ANNOTATED_LEAKS ci/lib: support custom output directories when creating test artifacts

Merge branch 'ps/ci-gitlab-update'

2024-12-23T17:32:07Z

GitLab CI updates. * ps/ci-gitlab-update: ci/lib: fix "CI setup" sections with GitLab CI ci/lib: do not interpret escape sequences in `group ()` arguments ci/lib: remove duplicate trap to end "CI setup" group gitlab-ci: update macOS images to Sonoma

Merge branch 'ps/build'

2024-12-16T01:54:33Z

Build procedure update plus introduction of Meson based builds. * ps/build: (24 commits) Introduce support for the Meson build system Documentation: add comparison of build systems t: allow overriding build dir t: better support for out-of-tree builds Documentation: extract script to generate a list of mergetools Documentation: teach "cmd-list.perl" about out-of-tree builds Documentation: allow sourcing generated includes from separate dir Makefile: simplify building of templates Makefile: write absolute program path into bin-wrappers Makefile: allow "bin-wrappers/" directory to exist Makefile: refactor generators to be PWD-independent Makefile: extract script to generate gitweb.js Makefile: extract script to generate gitweb.cgi Makefile: extract script to massage Python scripts Makefile: extract script to massage Shell scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Perl scripts Makefile: consistently use PERL_PATH Makefile: generate doc versions via GIT-VERSION-GEN Makefile: generate "git.rc" via GIT-VERSION-GEN ...

Merge branch 'es/oss-fuzz'

2024-12-13T15:33:42Z

Backport oss-fuzz tests for us to our codebase. * es/oss-fuzz: fuzz: port fuzz-url-decode-mem from OSS-Fuzz fuzz: port fuzz-parse-attr-line from OSS-Fuzz fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz