git/builtin/fetch.c, branch v2.37.2

Merge branch 'jc/avoid-redundant-submodule-fetch'

2022-05-25T23:42:49Z

"git fetch --recurse-submodules" from multiple remotes (either from a remote group, or "--all") used to make one extra "git fetch" in the submodules, which has been corrected. * jc/avoid-redundant-submodule-fetch: fetch: do not run a redundant fetch from submodule

fetch: do not run a redundant fetch from submodule

2022-05-18T16:08:57Z

When 7dce19d3 (fetch/pull: Add the --recurse-submodules option, 2010-11-12) introduced the "--recurse-submodule" option, the approach taken was to perform fetches in submodules only once, after all the main fetching (it may usually be a fetch from a single remote, but it could be fetching from a group of remotes using fetch_multiple()) succeeded. Later we added "--all" to fetch from all defined remotes, which complicated things even more. If your project has a submodule, and you try to run "git fetch --recurse-submodule --all", you'd see a fetch for the top-level, which invokes another fetch for the submodule, followed by another fetch for the same submodule. All but the last fetch for the submodule come from a "git fetch --recurse-submodules" subprocess that is spawned via the fetch_multiple() interface for the remotes, and the last fetch comes from the code at the end. Because recursive fetching from submodules is done in each fetch for the top-level in fetch_multiple(), the last fetch in the submodule is redundant. It only matters when fetch_one() interacts with a single remote at the top-level. While we are at it, there is one optimization that exists in dealing with a group of remote, but is missing when "--all" is used. In the former, when the group turns out to be a group of one, instead of spawning "git fetch" as a subprocess via the fetch_multiple() interface, we use the normal fetch_one() code path. Do the same when handing "--all", if it turns out that we have only one remote defined. Reviewed-by: Glen Choo Signed-off-by: Junio C Hamano

fetch: limit shared symref check only for local branches

2022-05-16T17:58:01Z

This check was introduced in 8ee5d73137f (Fix fetch/pull when run without --update-head-ok, 2008-10-13) in order to protect against replacing the ref of the active branch by mistake, for example by running git fetch origin master:master. It was later extended in 8bc1f39f411 (fetch: protect branches checked out in all worktrees, 2021-12-01) to scan all worktrees. This operation is very expensive (takes about 30s in my repository) when there are many tags or branches, and it is executed on every fetch, even if no local heads are updated at all. Limit it to protect only refs/heads/* to improve fetch performance. Signed-off-by: Orgad Shaneh Signed-off-by: Junio C Hamano

Merge branch 'rc/fetch-refetch'

2022-04-04T17:56:23Z

"git fetch --refetch" learned to fetch everything without telling the other side what we already have, which is useful when you cannot trust what you have in the local object store. * rc/fetch-refetch: docs: mention --refetch fetch option fetch: after refetch, encourage auto gc repacking t5615-partial-clone: add test for fetch --refetch fetch: add --refetch option builtin/fetch-pack: add --refetch option fetch-pack: add refetch fetch-negotiator: add specific noop initializer

fetch: after refetch, encourage auto gc repacking

2022-03-28T17:25:53Z

After invoking `fetch --refetch`, the object db will likely contain many duplicate objects. If auto-maintenance is enabled, invoke it with appropriate settings to encourage repacking/consolidation. * gc.autoPackLimit: unless this is set to 0 (disabled), override the value to 1 to force pack consolidation. * maintenance.incremental-repack.auto: unless this is set to 0, override the value to -1 to force incremental repacking. Signed-off-by: Robert Coup Signed-off-by: Junio C Hamano

fetch: add --refetch option

2022-03-28T17:25:52Z

Teach fetch and transports the --refetch option to force a full fetch without negotiating common commits with the remote. Use when applying a new partial clone filter to refetch all matching objects. Signed-off-by: Robert Coup Signed-off-by: Junio C Hamano

Merge branch 'gc/recursive-fetch-with-unused-submodules'

2022-03-25T23:38:25Z

When "git fetch --recurse-submodules" grabbed submodule commits that would be needed to recursively check out newly fetched commits in the superproject, it only paid attention to submodules that are in the current checkout of the superproject. We now do so for all submodules that have been run "git submodule init" on. * gc/recursive-fetch-with-unused-submodules: submodule: fix latent check_has_commit() bug fetch: fetch unpopulated, changed submodules submodule: move logic into fetch_task_create() submodule: extract get_fetch_task() submodule: store new submodule commits oid_array in a struct submodule: inline submodule_commits() into caller submodule: make static functions read submodules from commits t5526: create superproject commits with test helper t5526: stop asserting on stderr literally t5526: introduce test helper to assert on fetches

Merge branch 'ps/fetch-mirror-optim'

2022-03-17T00:53:07Z

Various optimization for "git fetch". * ps/fetch-mirror-optim: refs/files-backend: optimize reading of symbolic refs remote: read symbolic refs via `refs_read_symbolic_ref()` refs: add ability for backends to special-case reading of symbolic refs fetch: avoid lookup of commits when not appending to FETCH_HEAD upload-pack: look up "want" lines via commit-graph

fetch: fetch unpopulated, changed submodules

2022-03-16T23:08:59Z

"git fetch --recurse-submodules" only considers populated submodules (i.e. submodules that can be found by iterating the index), which makes "git fetch" behave differently based on which commit is checked out. As a result, even if the user has initialized all submodules correctly, they may not fetch the necessary submodule commits, and commands like "git checkout --recurse-submodules" might fail. Teach "git fetch" to fetch cloned, changed submodules regardless of whether they are populated. This is in addition to the current behavior of fetching populated submodules (which is always attempted regardless of what was fetched in the superproject, or even if nothing was fetched in the superproject). A submodule may be encountered multiple times (via the list of populated submodules or via the list of changed submodules). When this happens, "git fetch" only reads the 'populated copy' and ignores the 'changed copy'. Amend the verify_fetch_result() test helper so that we can assert on which 'copy' is being read. Signed-off-by: Glen Choo Signed-off-by: Junio C Hamano

Merge branch 'ps/fetch-atomic'

2022-03-13T22:56:16Z

"git fetch" can make two separate fetches, but ref updates coming from them were in two separate ref transactions under "--atomic", which has been corrected. * ps/fetch-atomic: fetch: make `--atomic` flag cover pruning of refs fetch: make `--atomic` flag cover backfilling of tags refs: add interface to iterate over queued transactional updates fetch: report errors when backfilling tags fails fetch: control lifecycle of FETCH_HEAD in a single place fetch: backfill tags before setting upstream fetch: increase test coverage of fetches