git/http-backend.c, branch v2.20.2

Merge branch 'ds/multi-pack-index'

2018-09-17T20:53:50Z

When there are too many packfiles in a repository (which is not recommended), looking up an object in these would require consulting many pack .idx files; a new mechanism to have a single file that consolidates all of these .idx files is introduced. * ds/multi-pack-index: (32 commits) pack-objects: consider packs in multi-pack-index midx: test a few commands that use get_all_packs treewide: use get_all_packs packfile: add all_packs list midx: fix bug that skips midx with alternates midx: stop reporting garbage midx: mark bad packed objects multi-pack-index: store local property multi-pack-index: provide more helpful usage info midx: clear midx on repack packfile: skip loading index if in multi-pack-index midx: prevent duplicate packfile loads midx: use midx in approximate_object_count midx: use existing midx when writing new one midx: use midx in abbreviation calculations midx: read objects from multi-pack-index config: create core.multiPackIndex setting midx: write object offsets midx: write object id fanout chunk midx: write object ids in a chunk ...

Merge branch 'mk/http-backend-content-length'

2018-09-10T17:35:42Z

The earlier attempt barfed when given a CONTENT_LENGTH that is set to an empty string. RFC 3875 is fairly clear that in this case we should not read any message body, but we've been reading through to the EOF in previous versions (which did not even pay attention to the environment variable), so keep that behaviour for now in this late update. * mk/http-backend-content-length: http-backend: allow empty CONTENT_LENGTH

http-backend: allow empty CONTENT_LENGTH

2018-09-07T19:35:51Z

According to RFC3875, empty environment variable is equivalent to unset, and for CONTENT_LENGTH it should mean zero body to read. However, unset CONTENT_LENGTH is also used for chunked encoding to indicate reading until EOF. At least, the test "large fetch-pack requests can be split across POSTs" from t5551 starts faliing, if unset or empty CONTENT_LENGTH is treated as zero length body. So keep the existing behavior as much as possible. Add a test for the case. Reported-By: Jelmer Vernooĳ Signed-off-by: Max Kirillov Signed-off-by: Junio C Hamano

treewide: use get_all_packs

2018-08-20T22:31:40Z

There are many places in the codebase that want to iterate over all packfiles known to Git. The purposes are wide-ranging, and those that can take advantage of the multi-pack-index already do. So, use get_all_packs() instead of get_packed_git() to be sure we are iterating over all packfiles. Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

Merge branch 'mk/http-backend-content-length'

2018-08-17T20:09:57Z

The http-backend (used for smart-http transport) used to slurp the whole input until EOF, without paying attention to CONTENT_LENGTH that is supplied in the environment and instead expecting the Web server to close the input stream. This has been fixed. * mk/http-backend-content-length: t5562: avoid non-portable "export FOO=bar" construct http-backend: respect CONTENT_LENGTH for receive-pack http-backend: respect CONTENT_LENGTH as specified by rfc3875 http-backend: cleanup writing to child process

http-backend: respect CONTENT_LENGTH for receive-pack

2018-07-27T17:47:52Z

Push passes to another commands, as described in https://public-inbox.org/git/20171129032214.GB32345@sigill.intra.peff.net/ As it gets complicated to correctly track the data length, instead transfer the data through parent process and cut the pipe as the specified length is reached. Do it only when CONTENT_LENGTH is set, otherwise pass the input directly to the forked commands. Add tests for cases: * CONTENT_LENGTH is set, script's stdin has more data, with all combinations of variations: fetch or push, plain or compressed body, correct or truncated input. * CONTENT_LENGTH is specified to a value which does not fit into ssize_t. Helped-by: Junio C Hamano Signed-off-by: Max Kirillov Signed-off-by: Junio C Hamano

tag: add repository argument to deref_tag

2018-06-29T17:43:39Z

Add a repository argument to allow the callers of deref_tag to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time. Signed-off-by: Jonathan Nieder Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano

object: add repository argument to parse_object

2018-06-29T17:43:38Z

Add a repository argument to allow the callers of parse_object to be more specific about which repository to act on. This is a small mechanical change; it doesn't change the implementation to handle repositories other than the_repository yet. As with the previous commits, use a macro to catch callers passing a repository other than the_repository at compile time. Signed-off-by: Jonathan Nieder Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano

http-backend: respect CONTENT_LENGTH as specified by rfc3875

2018-06-11T20:21:38Z

http-backend reads whole input until EOF. However, the RFC 3875 specifies that a script must read only as many bytes as specified by CONTENT_LENGTH environment variable. Web server may exercise the specification by not closing the script's standard input after writing content. In that case http-backend would hang waiting for the input. The issue is known to happen with IIS/Windows, for example. Make http-backend read only CONTENT_LENGTH bytes, if it's defined, rather than the whole input until EOF. If the variable is not defined, keep older behavior of reading until EOF because it is used to support chunked transfer-encoding. This commit only fixes buffered input, whcih reads whole body before processign it. Non-buffered input is going to be fixed in subsequent commit. Signed-off-by: Florian Manschwetus [mk: fixed trivial build failures and polished style issues] Helped-by: Junio C Hamano Signed-off-by: Max Kirillov Signed-off-by: Junio C Hamano

http-backend: cleanup writing to child process

2018-06-11T20:21:36Z

As explained in [1], we should not assume the reason why the writing has failed, and even if the reason is that child has existed not the reason why it have done so. So instead just say that writing has failed. [1] https://public-inbox.org/git/20180604044408.GD14451@sigill.intra.peff.net/ Signed-off-by: Max Kirillov Signed-off-by: Junio C Hamano