git/Makefile, branch v2.48.2

Revert barrier-based LSan threading race workaround

2025-01-01T22:13:01Z

The extra "barrier" approach was too much code whose sole purpose was to work around a race that is not even ours (i.e. in LSan's teardown code). In preparation for queuing a solution taking a much-less-invasive approach, let's revert them.

thread-utils: introduce optional barrier type

2024-12-30T14:18:57Z

One thread primitive we don't yet support is a barrier: it waits for all threads to reach a synchronization point before letting any of them continue. This would be useful for avoiding the LSan race we see in index-pack (and other places) by having all threads complete their initialization before any of them start to do real work. POSIX introduced a pthread_barrier_t in 2004, which does what we want. But if we want to rely on it: 1. Our Windows pthread emulation would need a new set of wrapper functions. There's a Synchronization Barrier primitive there, which was introduced in Windows 8 (which is old enough for us to depend on). 2. macOS (and possibly other systems) has pthreads but not pthread_barrier_t. So there we'd have to implement our own barrier based on the mutex and cond primitives. Those are do-able, but since we only care about avoiding races in our LSan builds, there's an easier way: make it a noop on systems without a native pthread barrier. This patch introduces a "maybe_thread_barrier" API. The clunky name (rather than just using pthread_barrier directly) should hopefully clue people in that on some systems it will do nothing. It's wired to a Makefile knob which has to be triggered manually, and we enable it for the linux-leaks CI jobs (since we know we'll have it there). There are some other possible options: - we could turn it on all the time for Linux systems based on uname. But we really only care about it for LSan builds, and there is no need to add extra code to regular builds. - we could turn it on only for LSan builds. But that would break builds on non-Linux platforms (like macOS) that otherwise should support sanitizers. - we could trigger only on the combination of Linux and LSan together. This isn't too hard to do, but the uname check isn't completely accurate. It is really about what your libc supports, and non-glibc systems might not have it (though at least musl seems to). So we'd risk breaking builds on those systems, which would need to add a new knob. Though the upside would be that running local "make SANITIZE=leak test" would be protected automatically. And of course none of this protects LSan runs from races on systems without pthread barriers. It's probably OK in practice to protect only our CI jobs, though. The race is rare-ish and most leak-checking happens through CI. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

GIT-BUILD-OPTIONS: wire up NO_GITWEB option

2024-12-27T16:17:19Z

Building our "gitweb" interface is optional in our Makefile and in Meson and not wired up at all with CMake, but disabling it causes a couple of tests in the t950* range that pull in "t/lib-gitweb.sh". This is because the test library knows to execute gitweb-tests based on whether or not Perl is available, but we may have Perl available and still end up not building gitweb e.g. with `make test NO_GITWEB=YesPlease`. Fix this issue by wiring up a new "NO_GITWEB" build option so that we can skip these tests in case gitweb is not built. Note that this new build option requires us to move the configuration of GIT-BUILD-OPTIONS to a later point in our Meson build instructions. But as that file is only consumed by our tests at runtime this change does not cause any issues. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

GIT-BUILD-OPTIONS: sort variables alphabetically

2024-12-27T16:17:19Z

The variables declared and substituted in GIT-BUILD-OPTIONS are not ordered in any obvious way. Sort them alphabetically so that it becomes obvious where new variables should go. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'ps/build-hotfix'

2024-12-23T17:32:26Z

A topic to optionally build with meson, which has graduated to 'master' recently, has regressed the normal Makefile build, which is being corrected. * ps/build-hotfix: meson: add options to override build information GIT-VERSION-GEN: fix overriding GIT_BUILT_FROM_COMMIT and GIT_DATE GIT-VERSION-GEN: fix overriding GIT_VERSION Makefile: introduce template for GIT-VERSION-GEN Makefile: drop unneeded indirection for GIT-VERSION-GEN outputs Makefile: stop including "GIT-VERSION-FILE" in docs

Merge branch 'ps/ci-meson'

2024-12-23T17:32:25Z

The meson-build procedure is integrated into CI to catch and prevent bitrotting. * ps/ci-meson: ci: wire up Meson builds t: introduce compatibility options to clar-based tests t: fix out-of-tree tests for some git-p4 tests Makefile: detect missing Meson tests meson: detect missing tests at configure time t/unit-tests: rename clar-based unit tests to have a common prefix Makefile: drop -DSUPPRESS_ANNOTATED_LEAKS ci/lib: support custom output directories when creating test artifacts

GIT-VERSION-GEN: fix overriding GIT_VERSION

2024-12-20T20:36:45Z

GIT-VERSION-GEN tries to derive the version that Git is being built from via multiple different sources in the following order: 1. A file called "version" in the source tree's root directory, if it exists. 2. The current commit in case Git is built from a Git repository. 3. Otherwise, we use a fallback version stored in a variable which is bumped whenever a new Git version is getting tagged. It used to be possible to override the version by overriding the `GIT_VERSION` Makefile variable (e.g. `make GIT_VERSION=foo`). This worked somewhat by chance, only: `GIT-VERSION-GEN` would write the actual Git version into `GIT-VERSION-FILE`, not the overridden value, but when including the file into our Makefile we would not override the `GIT_VERSION` variable because it has already been set by the user. And because our Makefile used the variable to propagate the version to our build tools instead of using `GIT-VERSION-FILE` the resulting build artifacts used the overridden version. But that subtle mechanism broke with 4838deab65 (Makefile: refactor GIT-VERSION-GEN to be reusable, 2024-12-06) and subsequent commits because the version information is not propagated via the Makefile variable anymore, but instead via the files that `GIT-VERSION-GEN` started to write. And as the script never knew about the `GIT_VERSION` environment variable in the first place it uses one of the values listed above instead of the overridden value. Fix this issue by making `GIT-VERSION-GEN` handle the case where `GIT_VERSION` has been set via the environment. Note that this requires us to introduce a new GIT_VERSION_OVERRIDE variable that stores a potential user-provided value, either via the environment or via "config.mak". Ideally we wouldn't need it and could just continue to use GIT_VERSION for this. But unfortunately, Makefiles will first include all sub-Makefiles before figuring out whether it needs to re-make any of them [1]. Consequently, if there already is a GIT-VERSION-FILE, we would have slurped in its value of GIT_VERSION before we call GIT-VERSION-GEN, and because GIT-VERSION-GEN now uses that value as an override it would mean that the first generated value for GIT_VERSION will remain unchanged. Furthermore we have to move the include for "GIT-VERSION-FILE" after the includes for "config.mak" and related so that GIT_VERSION_OVERRIDE can be set to the value provided by "config.mak". [1]: https://www.gnu.org/software/make/manual/html_node/Remaking-Makefiles.html Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Makefile: introduce template for GIT-VERSION-GEN

2024-12-20T20:36:45Z

Introduce a new template to call GIT-VERSION-GEN. This will allow us to iterate on how exactly the script is called in subsequent commits without having to adapt all call sites every time. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Makefile: drop unneeded indirection for GIT-VERSION-GEN outputs

2024-12-20T20:36:44Z

Some of the callsites of GIT-VERSION-GEN generate the target file with a "+" suffix first and then move the file into place when the new contents are different compared to the old contents. This allows us to avoid a needless rebuild by not updating timestamps of the target file when its contents will remain unchanged anyway. In fact though, this exact logic is already handled in GIT-VERSION-GEN, so doing this manually is pointless. This is a leftover from an earlier version of 4838deab65 (Makefile: refactor GIT-VERSION-GEN to be reusable, 2024-12-06), where the script didn't handle that logic for us. Drop the needless indirection. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'ps/build'

2024-12-16T01:54:33Z

Build procedure update plus introduction of Meson based builds. * ps/build: (24 commits) Introduce support for the Meson build system Documentation: add comparison of build systems t: allow overriding build dir t: better support for out-of-tree builds Documentation: extract script to generate a list of mergetools Documentation: teach "cmd-list.perl" about out-of-tree builds Documentation: allow sourcing generated includes from separate dir Makefile: simplify building of templates Makefile: write absolute program path into bin-wrappers Makefile: allow "bin-wrappers/" directory to exist Makefile: refactor generators to be PWD-independent Makefile: extract script to generate gitweb.js Makefile: extract script to generate gitweb.cgi Makefile: extract script to massage Python scripts Makefile: extract script to massage Shell scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Perl scripts Makefile: consistently use PERL_PATH Makefile: generate doc versions via GIT-VERSION-GEN Makefile: generate "git.rc" via GIT-VERSION-GEN ...