git/t/lib-httpd.sh, branch v2.37.2

tests: add LIBCURL prerequisite to tests needing libcurl

2022-06-15T18:49:52Z

Add and use a LIBCURL prerequisite for tests added in 6dcbdc0d661 (remote: create fetch.credentialsInUrl config, 2022-06-06). These tests would get as far as emitting a couple of the warnings we were testing for, but would then die as we had no "git-remote-https" program compiled. It would be more consistent with other prerequisites (e.g. PERL for NO_PERL) to name this "CURL", but since e9184b0789a (t5561: skip tests if curl is not available, 2018-04-03) we've had that prerequisite defined for checking of we have the curl(1) program. The existing "CURL" prerequisite is only used in one place, and we should probably name it "CURL_PROGRAM", then rename "LIBCURL" to "CURL" as a follow-up, but for now (pre-v2.37.0) let's aim for the most minimal fix possible. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

send-pack: complain about "expecting report" with --helper-status

2021-10-18T20:26:52Z

When pushing to a server which erroneously omits the final ref-status report, the client side should complain about the refs for which we didn't receive the status (because we can't just assume they were updated). This works over most transports like ssh, but for http we'll print a very misleading "Everything up-to-date". It works for ssh because send-pack internally sets the status of each ref to REF_STATUS_EXPECTING_REPORT, and then if the server doesn't tell us about a particular ref, it will stay at that value. When we print the final status table, we'll see that we're still on EXPECTING_REPORT and complain then. But for http, we go through remote-curl, which invokes send-pack with "--stateless-rpc --helper-status". The latter option causes send-pack to return a machine-readable list of ref statuses to the remote helper. But ever since its inception in de1a2fdd38 (Smart push over HTTP: client side, 2009-10-30), the send-pack code has simply omitted mention of any ref which ended up in EXPECTING_REPORT. In the remote helper, we then take the absence of any status report from send-pack to mean that the ref was not even something we tried to send, and thus it prints "Everything up-to-date". Fortunately it does detect the eventual non-zero exit from send-pack, and propagates that in its own non-zero exit code. So at least a careful script invoking "git push" would notice the failure. But sending the misleading message on stderr is certainly confusing for humans (not to mention the machine-readable "push --porcelain" output, though again, any careful script should be checking the exit code from push, too). Nobody seems to have noticed because the server in this instance has to be misbehaving: it has promised to support the ref-status capability (otherwise the client will not set EXPECTING_REPORT at all), but didn't send us any. If the connection were simply cut, then send-pack would complain about getting EOF while trying to read the status. But if the server actually sends a flush packet (i.e., saying "now you have all of the ref statuses" without actually sending any), then the client ends up in this confused situation. The fix is simple: we should return an error message from "send-pack --helper-status", just like we would for any other error per-ref error condition (in the test I included, the server simply omits all ref status responses, but a more insidious version of this would skip only some of them). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

remote-curl: error on incomplete packet

2020-05-24T23:26:00Z

Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated with a partially written packet, remote-curl will blindly send the partially written packet before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the partial packet and continue reading, expecting more input. This results in a deadlock between the two processes. For a stateless connection, inspect packets before sending them and error out if a packet line packet is incomplete. Helped-by: Jeff King Signed-off-by: Denton Liu Signed-off-by: Junio C Hamano

t/lib-httpd: avoid using macOS' sed

2020-02-27T17:58:41Z

Among other differences relative to GNU sed, macOS' sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: macOS' sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that macOS' sed does not even interpret those `\x` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs macOS semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values

2019-11-23T02:16:08Z

Since 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21) we get the normalized bool values of various GIT_TEST_* environment variables via 'git env--helper'. Now, while the 'git env--helper' command itself does catch invalid values in the environment variable or in the given --default and exits with error (exit code 128 or 129, respectively), it's invoked in conditions like 'if ! git env--helper ...', which means that all invalid bool values are interpreted the same as the ordinary 'false' (exit code 1). This has led to inadvertently skipped httpd tests in our CI builds for a couple of weeks, see 3960290675 (ci: restore running httpd tests, 2019-09-06). Let's be more careful about what the test suite accepts as bool values in GIT_TEST_* environment variables, and error out loud and clear on invalid values instead of simply skipping tests. Add the 'test_bool_env' helper function to encapsulate the invocation of 'git env--helper' and the verification of its exit code, and replace all invocations of that command in our test framework and test suite with a call to this new helper (except in 't0017-env-helper.sh', of course). $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

tests: replace test_tristate with "git env--helper"

2019-06-21T16:42:49Z

The test_tristate helper introduced in 83d842dc8c ("tests: turn on network daemon tests by default", 2014-02-10) can now be better implemented with "git env--helper" to give the variables in question the standard boolean behavior. The reason for the "tristate" was to have all of false/true/auto, where "auto" meant either "false" or "true" depending on what the fallback was. With the --default option to "git env--helper" we can simply have e.g. GIT_TEST_HTTPD where we know if it's true because the user asked explicitly ("true"), or true implicitly ("auto"). This breaks backwards compatibility for explicitly setting "auto" for these variables, but I don't think anyone cares. That was always intended to be internal. This means the test_normalize_bool() code in test-lib-functions.sh goes away in addition to test_tristate(). We still need the test_skip_or_die() helper, but now it takes the variable name instead of the value, and uses "git env--bool" to distinguish a default "true" from an explicit "true" (in those "explicit true" cases we want to fail the test in question). Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

tests: use 'test_atexit' to stop httpd

2019-03-14T03:34:39Z

Use 'test_atexit' to run cleanup commands to stop httpd at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Merge branch 'jt/fetch-v2-sideband'

2019-02-14T22:28:21Z

"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache

t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache

2019-02-14T21:18:20Z

07c3c2aa16 ("tests: define GIT_TEST_SIDEBAND_ALL", 2019-01-16) added GIT_TEST_SIDEBAND_ALL to the apache.conf PassEnv list. Avoid warnings from Apache when the variable is unset, as we do for GIT_VALGRIND* and GIT_TRACE, from f628825481 ("t/lib-httpd: handle running under --valgrind", 2012-07-24) and 89c57ab3f0 ("t: pass GIT_TRACE through Apache", 2015-03-13), respectively. Signed-off-by: Todd Zullinger Signed-off-by: Junio C Hamano

t5551: test server-side ERR packet

2019-02-06T20:20:23Z

When a smart HTTP server sends an error message via pkt-line, we detect the error due to using PACKET_READ_DIE_ON_ERR_PACKET. This case was added by 2d103c31c2 (pack-protocol.txt: accept error packets in any context, 2018-12-29), but not covered by tests. Signed-off-by: Josh Steadmon Signed-off-by: Jeff King Signed-off-by: Junio C Hamano