git/fetch-pack.c, branch v2.37.2

Merge branch 'ds/bundle-uri'

2022-06-03T21:30:34Z

Preliminary code refactoring around transport and bundle code. * ds/bundle-uri: bundle.h: make "fd" version of read_bundle_header() public remote: allow relative_url() to return an absolute url remote: move relative_url() http: make http_get_file() external fetch-pack: move --keep=* option filling to a function fetch-pack: add a deref_without_lazy_fetch_extended() dir API: add a generalized path_match_flags() function connect.c: refactor sending of agent & object-format

Merge branch 'jt/fetch-peek-optional-section'

2022-05-25T23:42:48Z

"git fetch" unnecessarily failed when an unexpected optional section appeared in the output, which has been corrected. * jt/fetch-peek-optional-section: fetch-pack: make unexpected peek result non-fatal

fetch-pack: move --keep=* option filling to a function

2022-05-16T22:02:09Z

Move the populating of the --keep=* option argument to "index-pack" to a static function, a subsequent commit will make use of it in another function. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

fetch-pack: add a deref_without_lazy_fetch_extended()

2022-05-16T22:02:09Z

Add a version of the deref_without_lazy_fetch function which can be called with custom oi_flags and to grab information about the "object_type". This will be used for the bundle-uri client in a subsequent commit. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

fetch-pack: make unexpected peek result non-fatal

2022-05-16T16:11:12Z

When a Git server responds to a fetch request, it may send optional sections before the packfile section. To handle this, the Git client calls packet_reader_peek() (see process_section_header()) in order to see what's next without consuming the line. However, as implemented, Git errors out whenever what's peeked is not an ordinary line. This is not only unexpected (here, we only need to know whether the upcoming line is the section header we want) but causes errors to include the name of a section header that is irrelevant to the cause of the error. For example, at $DAYJOB, we have seen "fatal: error reading section header 'shallow-info'" error messages when none of the repositories involved are shallow. Therefore, fix this so that the peek returns 1 if the upcoming line is the wanted section header and nothing else. Because of this change, reader->line may now be NULL later in the function, so update the error message printing code accordingly. Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano

fetch-pack: add refetch

2022-03-28T17:25:52Z

Allow a "refetch" where the contents of the local object store are ignored and a full fetch is performed, not attempting to find or negotiate common commits with the remote. A key use case is to apply a new partial clone blob/tree filter and refetch all the associated matching content, which would otherwise not be transferred when the commit objects are already present locally. Signed-off-by: Robert Coup Signed-off-by: Junio C Hamano

Merge branch 'ps/fetch-optim-with-commit-graph'

2022-02-24T00:58:03Z

A couple of optimization to "git fetch". * ps/fetch-optim-with-commit-graph: fetch: skip computing output width when not printing anything fetch-pack: use commit-graph when computing cutoff

Merge branch 'bs/forbid-i18n-of-protocol-token-in-fetch-pack'

2022-02-24T00:58:03Z

L10n support for a few error messages. * bs/forbid-i18n-of-protocol-token-in-fetch-pack: fetch-pack: parameterize message containing 'ready' keyword

fetch-pack: parameterize message containing 'ready' keyword

2022-02-11T22:37:09Z

The protocol keyword 'ready' isn't meant for translation. Pass it as parameter instead of spell it in die() message (and potentially confuse translators). Signed-off-by: Bagas Sanjaya Signed-off-by: Junio C Hamano

fetch-pack: use commit-graph when computing cutoff

2022-02-10T17:59:38Z

During packfile negotiation we iterate over all refs announced by the remote side to check whether their IDs refer to commits already known to us. If a commit is known to us already, then its date is a potential cutoff point for commits we have in common with the remote side. There is potentially a lot of commits announced by the remote depending on how many refs there are in the remote repository, and for every one of them we need to search for it in our object database and, if found, parse the corresponding object to find out whether it is a candidate for the cutoff date. This can be sped up by trying to look up commits via the commit-graph first, which is a lot more efficient. Benchmarks in a repository with about 2,1 million refs and an up-to-date commit-graph show an almost 20% speedup when mirror-fetching: Benchmark 1: git fetch +refs/*:refs/* (v2.35.0) Time (mean ± σ): 115.587 s ± 2.009 s [User: 109.874 s, System: 11.305 s] Range (min … max): 113.584 s … 118.820 s 5 runs Benchmark 2: git fetch +refs/*:refs/* (HEAD) Time (mean ± σ): 96.859 s ± 0.624 s [User: 91.948 s, System: 10.980 s] Range (min … max): 96.180 s … 97.875 s 5 runs Summary 'git fetch +refs/*:refs/* (HEAD)' ran 1.19 ± 0.02 times faster than 'git fetch +refs/*:refs/* (v2.35.0)' Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano