git/t/lib-httpd, branch v2.53.0

t/lib-httpd: refactor "one-time-perl" CGI script to not depend on Perl

2025-04-07T21:47:40Z

Our Apache HTTPD setup exposes an "one_time_perl" endpoint to access repositories. If used, we execute the "apply-one-time-perl.sh" CGI script that checks whether we have a "one-time-perl" script. If so, that script gets executed so that it can munge what would be served. Once done, the script gets removed so that it doesn't execute a second time. As the name says, this functionality expects the user to pass a Perl script. This isn't really necessary though: we can just as easily implement the same thing with arbitrary scripts. Refactor the code so that we instead expect an arbitrary script to exist and rename the functionality to "one-time-script". Adapt callers to use shell utilities instead of Perl so that we can drop the PERL_TEST_HELPERS prerequisite. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

t5563: refactor for multi-stage authentication

2024-04-17T05:39:08Z

Some HTTP authentication schemes, such as NTLM- and Kerberos-based options, require more than one round trip to authenticate. Currently, these can only be supported in libcurl, since Git does not have support for this in the credential helper protocol. However, in a future commit, we'll add support for this functionality into the credential helper protocol and Git itself. Because we don't really want to implement either NTLM or Kerberos, both of which are complex protocols, we'll want to test this using a fake credential authentication scheme. In order to do so, update t5563 and its backend to allow us to accept multiple sets of credentials and respond with different behavior in each case. Since we can now provide any number of possible status codes, provide a non-specific reason phrase so we don't have to generate a more specific one based on the response. The reason phrase is mandatory according to the status-line production in RFC 7230, but clients SHOULD ignore it, and curl does (except to print it). Each entry in the authorization and challenge fields contains an ID, which indicates a corresponding credential and response. If the response is a 200 status, then we continue to execute git-http-backend. Otherwise, we print the corresponding status and response. If no ID is matched, we use the default response with a status of 401. Note that there is an implicit order to the parameters. The ID is always first and the creds or response value is always last, and therefore may contain spaces, equals signs, or other arbitrary data. Signed-off-by: brian m. carlson Signed-off-by: Junio C Hamano

t/lib-httpd: stop using legacy crypt(3) for authentication

2023-11-11T00:00:42Z

When setting up httpd for our tests, we also install a passwd and proxy-passwd file that contain the test user's credentials. These credentials currently use crypt(3) as the password encryption schema. This schema can be considered deprecated nowadays as it is not safe anymore. Quoting Apache httpd's documentation [1]: > Unix only. Uses the traditional Unix crypt(3) function with a > randomly-generated 32-bit salt (only 12 bits used) and the first 8 > characters of the password. Insecure. This is starting to cause issues in modern Linux distributions. glibc has deprecated its libcrypt library that used to provide crypt(3) in favor of the libxcrypt library. This newer replacement provides a compile time switch to disable insecure password encryption schemata, which causes crypt(3) to always return `EINVAL`. The end result is that httpd tests that exercise authentication will fail on distros that use libxcrypt without these insecure encryption schematas. Regenerate the passwd files to instead use the default password encryption schema, which is md5. While it feels kind of funny that an MD5-based encryption schema should be more secure than anything else, it is the current default and supported by all platforms. Furthermore, it really doesn't matter all that much given that these files are only used for testing purposes anyway. [1]: https://httpd.apache.org/docs/2.4/misc/password_encryptions.html Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

test-lib: set UBSAN_OPTIONS to match ASan

2023-09-21T21:10:36Z

For a long time we have used ASAN_OPTIONS to set abort_on_error. This is important because we want to notice detected problems even in programs which are expected to fail. But we never did the same for UBSAN_OPTIONS. This means that our UBSan test suite runs might silently miss some cases. It also causes a more visible effect, which is that t4058 complains about unexpected "fixes" (and this is how I noticed the issue): $ make SANITIZE=undefined CC=gcc && (cd t && ./t4058-*) ... ok 8 - git read-tree does not segfault # TODO known breakage vanished ok 9 - reset --hard does not segfault # TODO known breakage vanished ok 10 - git diff HEAD does not segfault # TODO known breakage vanished The tests themselves aren't that interesting. We have a known bug where these programs segfault, and they do when compiled without sanitizers. With UBSan, when the test runs: test_might_fail git read-tree --reset base it gets: cache-tree.c:935:9: runtime error: member access within misaligned address 0x5a5a5a5a5a5a5a5a for type 'struct cache_entry', which requires 8 byte alignment So that's garbage memory which would _usually_ cause us to segfault, but UBSan catches it and complains first about the alignment. That makes sense, but the weird thing is that UBSan then exits instead of aborting, so our test_might_fail call considers that an acceptable outcome and the test "passes". Curiously, this historically seems to have aborted, because I've run "make test" with UBSan many times (and so did our CI) and we never saw the problem. Even more curiously, I see an abort if I use clang with ASan and UBSan together, like: # this aborts! make SANITIZE=undefined,address CC=clang But not with just UBSan, and not with both when used with gcc: # none of these do make SANITIZE=undefined CC=gcc make SANITIZE=undefined CC=clang make SANITIZE=undefined,address CC=gcc Likewise moving to older versions of gcc (I tried gcc-11 and gcc-12 on my Debian system) doesn't abort. Nor does moving around in Git's history. Neither this test nor the relevant code have been touched in a while, and going back to v2.41.0 produces the same outcome (even though many UBSan CI runs have passed in the meantime). So _something_ changed on my system (and likely will soon on other people's, since this is stock Debian unstable), but I didn't track it further. I don't know why it ever aborted in the past, but we definitely should be explicit here and tell UBSan what we want to happen. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'jk/http-test-cgipassauth-unavailable-in-older-apache'

2023-05-19T16:27:07Z

We started unconditionally testing with CGIPassAuth directive but it is unavailable in older Apache that ships with CentOS 7 that has about a year of shelf-life still left. The test has conditionally been disabled when running with an ancient Apache. This was a fix for a recent regression caught before the release, so no need to mention it in the release notes. * jk/http-test-cgipassauth-unavailable-in-older-apache: t/lib-httpd: make CGIPassAuth support conditional

t/lib-httpd: make CGIPassAuth support conditional

2023-05-18T21:29:32Z

Commit 988aad99b4 (t5563: add tests for basic and anoymous HTTP access, 2023-02-27) added tests that require Apache to support the CGIPassAuth directive, which was added in Apache 2.4.13. This is fairly old (~8 years), but recent enough that we still encounter it in the wild (e.g., RHEL/CentOS 7, which is not EOL until June 2024). We can live with skipping the new tests on such a platform. But unfortunately, since the directive is used unconditionally in our apache.conf, it means the web server fails to start entirely, and we cannot run other HTTP tests at all (e.g., the basic ones in t5551). We can fix that by making the config conditional, and only triggering it for t5563. That solves the problem for t5551 (which then ignores the directive entirely). For t5563, we'd see apache complain in start_httpd; with the default setting of GIT_TEST_HTTPD, we'd then skip the whole script. But that leaves one small problem: people may set GIT_TEST_HTTPD=1 explicitly, which instructs the tests to fail (rather than skip) when we can't start the webserver (to avoid accidentally missing some tests). This could be worked around by having the user manually set GIT_SKIP_TESTS on a platform with an older Apache. But we can be a bit friendlier by doing the version check ourselves and setting an appropriate prereq. We'll use the (lack of) prereq to then skip the rest of t5563. In theory we could use the prereq to skip individual tests, but in practice this whole script depends on it. Reported-by: Todd Zullinger Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'jk/use-perl-path-consistently'

2023-04-11T20:49:13Z

Tests had a few places where we ignored PERL_PATH and blindly used /usr/bin/perl, which have been corrected. * jk/use-perl-path-consistently: t/lib-httpd: pass PERL_PATH to CGI scripts

t/lib-httpd: pass PERL_PATH to CGI scripts

2023-04-06T16:29:43Z

As discussed in t/README, tests should aim to use PERL_PATH rather than straight "perl". We usually do this automatically with a "perl" function in test-lib.sh, but a few cases need to be handled specially. One such case is the apply-one-time-perl.sh CGI, which invokes plain "perl". It should be using $PERL_PATH, but to make that work, we must also instruct Apache to pass through the variable. Prior to this patch, doing: mv /usr/bin/perl /usr/bin/my-perl make PERL_PATH=/usr/bin/my-perl test would fail t5702, t5703, and t5616. After this it passes. This is a pretty extreme case, as even if you install perl elsewhere, you'd likely still have it in your $PATH. A more realistic case is that you don't want to use the perl in your $PATH (because it's older, broken, etc) and expect PERL_PATH to consistently override that (since that's what it's documented to do). Removing it completely is just a convenient way of completely breaking it for testing purposes. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'mc/credential-helper-www-authenticate'

2023-03-17T21:03:10Z

Allow information carried on the WWW-AUthenticate header to be passed to the credential helpers. * mc/credential-helper-www-authenticate: credential: add WWW-Authenticate header to cred requests http: read HTTP WWW-Authenticate response headers t5563: add tests for basic and anoymous HTTP access

Merge branch 'jk/http-test-fixes'

2023-03-01T00:38:47Z

Various fix-ups on HTTP tests. * jk/http-test-fixes: t5559: make SSL/TLS the default t5559: fix test failures with LIB_HTTPD_SSL t/lib-httpd: enable HTTP/2 "h2" protocol, not just h2c t/lib-httpd: respect $HTTPD_PROTO in expect_askpass() t5551: drop curl trace lines without headers t5551: handle v2 protocol in cookie test t5551: simplify expected cookie file t5551: handle v2 protocol in upload-pack service test t5551: handle v2 protocol when checking curl trace t5551: stop forcing clone to run with v0 protocol t5551: handle HTTP/2 when checking curl trace t5551: lower-case headers in expected curl trace t5551: drop redundant grep for Accept-Language t5541: simplify and move "no empty path components" test t5541: stop marking "used receive-pack service" test as v0 only t5541: run "used receive-pack service" test earlier