git/ci, branch v2.42.2

Sync with 2.41.1

2024-04-19T10:38:46Z

* maint-2.41: (38 commits) Git 2.41.1 Git 2.40.2 Git 2.39.4 fsck: warn about symlink pointing inside a gitdir core.hooksPath: add some protection while cloning init.templateDir: consider this config setting protected clone: prevent hooks from running during a clone Add a helper function to compare file contents init: refactor the template directory discovery into its own function find_hook(): refactor the `STRIP_EXTENSION` logic clone: when symbolic links collide with directories, keep the latter entry: report more colliding paths t5510: verify that D/F confusion cannot lead to an RCE submodule: require the submodule path to contain directories only clone_submodule: avoid using `access()` on directories submodules: submodule paths must not contain symlinks clone: prevent clashing git dirs when cloning submodule in parallel t7423: add tests for symlinked submodule directories has_dir_name(): do not get confused by characters < '/' docs: document security issues around untrusted .git dirs ...

Sync with 2.40.2

2024-04-19T10:38:42Z

* maint-2.40: (39 commits) Git 2.40.2 Git 2.39.4 fsck: warn about symlink pointing inside a gitdir core.hooksPath: add some protection while cloning init.templateDir: consider this config setting protected clone: prevent hooks from running during a clone Add a helper function to compare file contents init: refactor the template directory discovery into its own function find_hook(): refactor the `STRIP_EXTENSION` logic clone: when symbolic links collide with directories, keep the latter entry: report more colliding paths t5510: verify that D/F confusion cannot lead to an RCE submodule: require the submodule path to contain directories only clone_submodule: avoid using `access()` on directories submodules: submodule paths must not contain symlinks clone: prevent clashing git dirs when cloning submodule in parallel t7423: add tests for symlinked submodule directories has_dir_name(): do not get confused by characters < '/' docs: document security issues around untrusted .git dirs upload-pack: disable lazy-fetching by default ...

Sync with 2.39.4

2024-04-19T10:38:37Z

* maint-2.39: (38 commits) Git 2.39.4 fsck: warn about symlink pointing inside a gitdir core.hooksPath: add some protection while cloning init.templateDir: consider this config setting protected clone: prevent hooks from running during a clone Add a helper function to compare file contents init: refactor the template directory discovery into its own function find_hook(): refactor the `STRIP_EXTENSION` logic clone: when symbolic links collide with directories, keep the latter entry: report more colliding paths t5510: verify that D/F confusion cannot lead to an RCE submodule: require the submodule path to contain directories only clone_submodule: avoid using `access()` on directories submodules: submodule paths must not contain symlinks clone: prevent clashing git dirs when cloning submodule in parallel t7423: add tests for symlinked submodule directories has_dir_name(): do not get confused by characters < '/' docs: document security issues around untrusted .git dirs upload-pack: disable lazy-fetching by default fetch/clone: detect dubious ownership of local repositories ...

Merge branch 'quicker-asan-lsan'

2024-04-16T22:00:56Z

This patch speeds up the `asan`/`lsan` jobs that are really slow enough already. Signed-off-by: Johannes Schindelin

ci(linux-asan/linux-ubsan): let's save some time

2024-04-16T21:58:04Z

Every once in a while, the `git-p4` tests flake for reasons outside of our control. It typically fails with "Connection refused" e.g. here: https://github.com/git/git/actions/runs/5969707156/job/16196057724 [...] + git p4 clone --dest=/home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git //depot Initialized empty Git repository in /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git/.git/ Perforce client error: Connect to server failed; check $P4PORT. TCP connect to localhost:9807 failed. connect: 127.0.0.1:9807: Connection refused failure accessing depot: could not run p4 Importing from //depot into /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git [...] This happens in other jobs, too, but in the `linux-asan`/`linux-ubsan` jobs it hurts the most because those jobs often take an _awfully_ long time to run, therefore re-running a failed `linux-asan`/`linux-ubsan` jobs is _very_ costly. The purpose of the `linux-asan`/`linux-ubsan` jobs is to exercise the C code of Git, anyway, and any part of Git's source code that the `git-p4` tests run and that would benefit from the attention of ASAN/UBSAN are run better in other tests anyway, as debugging C code run via Python scripts can get a bit hairy. In fact, it is not even just `git-p4` that is the problem (even if it flakes often enough to be problematic in the CI builds), but really the part about Python scripts. So let's just skip any Python parts of the tests from being run in that job. For good measure, also skip the Subversion tests because debugging C code run via Perl scripts is as much fun as debugging C code run via Python scripts. And it will reduce the time this very expensive job takes, which is a big benefit. Backported to `maint-2.39` as another step to get that branch's CI builds back to a healthy state. Backported-from: 6ba913629f (ci(linux-asan-ubsan): let's save some time, 2023-08-29) Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano Signed-off-by: Johannes Schindelin

ci: upgrade to using macos-13

2024-04-16T21:58:03Z

In April, GitHub announced that the `macos-13` pool is available: https://github.blog/changelog/2023-04-24-github-actions-macos-13-is-now-available/. It is only a matter of time until the `macos-12` pool is going away, therefore we should switch now, without pressure of a looming deadline. Since the `macos-13` runners no longer include Python2, we also drop specifically testing with Python2 and switch uniformly to Python3, see https://github.com/actions/runner-images/blob/HEAD/images/macos/macos-13-Readme.md for details about the software available on the `macos-13` pool's runners. Also, on macOS 13, Homebrew seems to install a `gcc@9` package that no longer comes with a regular `unistd.h` (there seems only to be a `ssp/unistd.h`), and hence builds would fail with: In file included from base85.c:1: git-compat-util.h:223:10: fatal error: unistd.h: No such file or directory 223 | #include | ^~~~~~~~~~ compilation terminated. The reason why we install GCC v9.x explicitly is historical, and back in the days it was because it was the _newest_ version available via Homebrew: 176441bfb58 (ci: build Git with GCC 9 in the 'osx-gcc' build job, 2019-11-27). To reinstate the spirit of that commit _and_ to fix that build failure, let's switch to the now-newest GCC version: v13.x. Backported-from: 682a868f67 (ci: upgrade to using macos-13, 2023-11-03) Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano Signed-off-by: Johannes Schindelin

Merge branch 'jk/ci-retire-allow-ref' into maint-2.42

2023-11-02T07:53:23Z

CI update. * jk/ci-retire-allow-ref: ci: deprecate ci/config/allow-ref script ci: allow branch selection through "vars"

ci: deprecate ci/config/allow-ref script

2023-08-30T22:56:11Z

Now that we have the CI_BRANCHES mechanism, there is no need for anybody to use the ci/config/allow-ref mechanism. In the long run, we can hopefully remove it and the whole "config" job, as it consumes CPU and adds to the end-to-end latency of the whole workflow. But we don't want to do that immediately, as people need time to migrate until the CI_BRANCHES change has made it into the workflow file of every branch. So let's issue a warning, which will appear in the "annotations" section below the workflow result in GitHub's web interface. And let's remove the sample allow-refs script, as we don't want to encourage anybody to use it. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

ci: allow branch selection through "vars"

2023-08-30T22:56:09Z

When we added config to skip CI for certain branches in e76eec3554 (ci: allow per-branch config for GitHub Actions, 2020-05-07), there wasn't any way to avoid spinning up a VM just to check the config. From the developer's perspective this isn't too bad, as the "skipped" branches complete successfully after running the config job (the workflow result is "success" instead of "skipped", but that is a minor lie). But we are still wasting time and GitHub's CPU to spin up a VM just to check the result of a short shell script. At the time there wasn't any way to avoid this. But they've since introduced repo-level variables that should let us do the same thing: https://github.blog/2023-01-10-introducing-required-workflows-and-configuration-variables-to-github-actions/#configuration-variables This is more efficient, and as a bonus is probably less confusing to configure (the existing system requires sticking your config on a magic ref). See the included docs for how to configure it. The code itself is pretty simple: it checks the variable and skips the config job if appropriate (and everything else depends on the config job already). There are two slight inaccuracies here: - we don't insist on branches, so this likewise applies to tag names or other refs. I think in practice this is OK, and keeping the code (and docs) short is more important than trying to be more exact. We are targeting developers of git.git and their limited workflows. - the match is done as a substring (so if you want to run CI for "foobar", then branch "foo" will accidentally match). Again, this should be OK in practice, as anybody who uses this is likely to only specify a handful of well-known names. If we want to be more exact, we can have the code check for adjoining spaces. Or even move to a more general CI_CONFIG variable formatted as JSON. I went with this scheme for the sake of simplicity. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

ci(linux-asan-ubsan): let's save some time

2023-08-29T20:54:55Z

Every once in a while, the `git-p4` tests flake for reasons outside of our control. It typically fails with "Connection refused" e.g. here: https://github.com/git/git/actions/runs/5969707156/job/16196057724 [...] + git p4 clone --dest=/home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git //depot Initialized empty Git repository in /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git/.git/ Perforce client error: Connect to server failed; check $P4PORT. TCP connect to localhost:9807 failed. connect: 127.0.0.1:9807: Connection refused failure accessing depot: could not run p4 Importing from //depot into /home/runner/work/git/git/t/trash directory.t9807-git-p4-submit/git [...] This happens in other jobs, too, but in the `linux-asan-ubsan` job it hurts the most because that job often takes over a full hour to run, therefore re-running a failed `linux-asan-ubsan` job is _very_ costly. The purpose of the `linux-asan-ubsan` job is to exercise the C code of Git, anyway, and any part of Git's source code that the `git-p4` tests run and that would benefit from the attention of ASAN/UBSAN are run better in other tests anyway, as debugging C code run via Python scripts can get a bit hairy. In fact, it is not even just `git-p4` that is the problem (even if it flakes often enough to be problematic in the CI builds), but really the part about Python scripts. So let's just skip any Python parts of the tests from being run in that job. For good measure, also skip the Subversion tests because debugging C code run via Perl scripts is as much fun as debugging C code run via Python scripts. And it will reduce the time this very expensive job takes, which is a big benefit. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano