git/t/lib-httpd.sh, branch v2.30.2

remote-curl: error on incomplete packet

2020-05-24T23:26:00Z

Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated with a partially written packet, remote-curl will blindly send the partially written packet before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the partial packet and continue reading, expecting more input. This results in a deadlock between the two processes. For a stateless connection, inspect packets before sending them and error out if a packet line packet is incomplete. Helped-by: Jeff King Signed-off-by: Denton Liu Signed-off-by: Junio C Hamano

t/lib-httpd: avoid using macOS' sed

2020-02-27T17:58:41Z

Among other differences relative to GNU sed, macOS' sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: macOS' sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that macOS' sed does not even interpret those `\x` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs macOS semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values

2019-11-23T02:16:08Z

Since 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21) we get the normalized bool values of various GIT_TEST_* environment variables via 'git env--helper'. Now, while the 'git env--helper' command itself does catch invalid values in the environment variable or in the given --default and exits with error (exit code 128 or 129, respectively), it's invoked in conditions like 'if ! git env--helper ...', which means that all invalid bool values are interpreted the same as the ordinary 'false' (exit code 1). This has led to inadvertently skipped httpd tests in our CI builds for a couple of weeks, see 3960290675 (ci: restore running httpd tests, 2019-09-06). Let's be more careful about what the test suite accepts as bool values in GIT_TEST_* environment variables, and error out loud and clear on invalid values instead of simply skipping tests. Add the 'test_bool_env' helper function to encapsulate the invocation of 'git env--helper' and the verification of its exit code, and replace all invocations of that command in our test framework and test suite with a call to this new helper (except in 't0017-env-helper.sh', of course). $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

tests: replace test_tristate with "git env--helper"

2019-06-21T16:42:49Z

The test_tristate helper introduced in 83d842dc8c ("tests: turn on network daemon tests by default", 2014-02-10) can now be better implemented with "git env--helper" to give the variables in question the standard boolean behavior. The reason for the "tristate" was to have all of false/true/auto, where "auto" meant either "false" or "true" depending on what the fallback was. With the --default option to "git env--helper" we can simply have e.g. GIT_TEST_HTTPD where we know if it's true because the user asked explicitly ("true"), or true implicitly ("auto"). This breaks backwards compatibility for explicitly setting "auto" for these variables, but I don't think anyone cares. That was always intended to be internal. This means the test_normalize_bool() code in test-lib-functions.sh goes away in addition to test_tristate(). We still need the test_skip_or_die() helper, but now it takes the variable name instead of the value, and uses "git env--bool" to distinguish a default "true" from an explicit "true" (in those "explicit true" cases we want to fail the test in question). Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

tests: use 'test_atexit' to stop httpd

2019-03-14T03:34:39Z

Use 'test_atexit' to run cleanup commands to stop httpd at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Merge branch 'jt/fetch-v2-sideband'

2019-02-14T22:28:21Z

"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache

t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache

2019-02-14T21:18:20Z

07c3c2aa16 ("tests: define GIT_TEST_SIDEBAND_ALL", 2019-01-16) added GIT_TEST_SIDEBAND_ALL to the apache.conf PassEnv list. Avoid warnings from Apache when the variable is unset, as we do for GIT_VALGRIND* and GIT_TRACE, from f628825481 ("t/lib-httpd: handle running under --valgrind", 2012-07-24) and 89c57ab3f0 ("t: pass GIT_TRACE through Apache", 2015-03-13), respectively. Signed-off-by: Todd Zullinger Signed-off-by: Junio C Hamano

t5551: test server-side ERR packet

2019-02-06T20:20:23Z

When a smart HTTP server sends an error message via pkt-line, we detect the error due to using PACKET_READ_DIE_ON_ERR_PACKET. This case was added by 2d103c31c2 (pack-protocol.txt: accept error packets in any context, 2018-12-29), but not covered by tests. Signed-off-by: Josh Steadmon Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

test-lib-functions: introduce the 'test_set_port' helper function

2019-01-07T17:24:06Z

Several test scripts run daemons like 'git-daemon' or Apache, and communicate with them through TCP sockets. To have unique ports where these daemons are accessible, the ports are usually the number of the corresponding test scripts, unless the user overrides them via environment variables, and thus all those tests and test libs contain more or less the same bit of one-liner boilerplate code to find out the port. The last patch in this series will make this a bit more complicated. Factor out finding the port for a daemon into the common helper function 'test_set_port' to avoid repeating ourselves. Take special care of test scripts with "low" numbers: - Test numbers below 1024 would result in a port that's only usable as root, so set their port to '10000 + test-nr' to make sure it doesn't interfere with other tests in the test suite. This makes the hardcoded port number in 't0410-partial-clone.sh' unnecessary, remove it. - The shell's arithmetic evaluation interprets numbers with leading zeros as octal values, which means that test number below 1000 and containing the digits 8 or 9 will trigger an error. Remove all leading zeros from the test numbers to prevent this. Note that the 'git p4' tests are unlike the other tests involving daemons in that: - 'lib-git-p4.sh' doesn't use the test's number for unique port as is, but does a bit of additional arithmetic on top [1]. - The port is not overridable via an environment variable. With this patch even 'git p4' tests will use the test's number as default port, and it will be overridable via the P4DPORT environment variable. [1] Commit fc00233071 (git-p4 tests: refactor and cleanup, 2011-08-22) introduced that "unusual" unique port computation without explaining why it was necessary (as opposed to simply using the test number as is). It seems to be just unnecessary complication, and in any case that commit came way before the "test nr as unique port" got "standardized" for other daemons in commits c44132fcf3 (tests: auto-set git-daemon port, 2014-02-10), 3bb486e439 (tests: auto-set LIB_HTTPD_PORT from test name, 2014-02-10), and bf9d7df950 (t/lib-git-svn.sh: improve svnserve tests with parallel make test, 2017-12-01). Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Merge branch 'sg/httpd-test-unflake'

2018-08-02T22:30:39Z

httpd tests saw occasional breakage due to the way its access log gets inspected by the tests, which has been updated to make them less flaky. * sg/httpd-test-unflake: t/lib-httpd: avoid occasional failures when checking access.log t/lib-httpd: add the strip_access_log() helper function t5541: clean up truncating access log