git/builtin/clone.c, branch v2.33.6

Sync with 2.32.4

2022-10-06T21:42:02Z

Signed-off-by: Taylor Blau

Sync with 2.31.5

2022-10-06T21:40:44Z

Signed-off-by: Taylor Blau

Sync with 2.30.6

2022-10-06T21:39:15Z

Signed-off-by: Taylor Blau

builtin/clone.c: disallow `--local` clones with symlinks

2022-10-01T04:23:38Z

When cloning a repository with `--local`, Git relies on either making a hardlink or copy to every file in the "objects" directory of the source repository. This is done through the callpath `cmd_clone()` -> `clone_local()` -> `copy_or_link_directory()`. The way this optimization works is by enumerating every file and directory recursively in the source repository's `$GIT_DIR/objects` directory, and then either making a copy or hardlink of each file. The only exception to this rule is when copying the "alternates" file, in which case paths are rewritten to be absolute before writing a new "alternates" file in the destination repo. One quirk of this implementation is that it dereferences symlinks when cloning. This behavior was most recently modified in 36596fd2df (clone: better handle symlinked files at .git/objects/, 2019-07-10), which attempted to support `--local` clones of repositories with symlinks in their objects directory in a platform-independent way. Unfortunately, this behavior of dereferencing symlinks (that is, creating a hardlink or copy of the source's link target in the destination repository) can be used as a component in attacking a victim by inadvertently exposing the contents of file stored outside of the repository. Take, for example, a repository that stores a Dockerfile and is used to build Docker images. When building an image, Docker copies the directory contents into the VM, and then instructs the VM to execute the Dockerfile at the root of the copied directory. This protects against directory traversal attacks by copying symbolic links as-is without dereferencing them. That is, if a user has a symlink pointing at their private key material (where the symlink is present in the same directory as the Dockerfile, but the key itself is present outside of that directory), the key is unreadable to a Docker image, since the link will appear broken from the container's point of view. This behavior enables an attack whereby a victim is convinced to clone a repository containing an embedded submodule (with a URL like "file:///proc/self/cwd/path/to/submodule") which has a symlink pointing at a path containing sensitive information on the victim's machine. If a user is tricked into doing this, the contents at the destination of those symbolic links are exposed to the Docker image at runtime. One approach to preventing this behavior is to recreate symlinks in the destination repository. But this is problematic, since symlinking the objects directory are not well-supported. (One potential problem is that when sharing, e.g. a "pack" directory via symlinks, different writers performing garbage collection may consider different sets of objects to be reachable, enabling a situation whereby garbage collecting one repository may remove reachable objects in another repository). Instead, prohibit the local clone optimization when any symlinks are present in the `$GIT_DIR/objects` directory of the source repository. Users may clone the repository again by prepending the "file://" scheme to their clone URL, or by adding the `--no-local` option to their `git clone` invocation. The directory iterator used by `copy_or_link_directory()` must no longer dereference symlinks (i.e., it *must* call `lstat()` instead of `stat()` in order to discover whether or not there are symlinks present). This has no bearing on the overall behavior, since we will immediately `die()` on encounter a symlink. Note that t5604.33 suggests that we do support local clones with symbolic links in the source repository's objects directory, but this was likely unintentional, or at least did not take into consideration the problem with sharing parts of the objects directory with symbolic links at the time. Update this test to reflect which options are and aren't supported. Helped-by: Johannes Schindelin Signed-off-by: Taylor Blau

clone: handle unborn branch in bare repos

2021-09-20T21:05:36Z

When cloning a repository with an unborn HEAD, we'll set the local HEAD to match it only if the local repository is non-bare. This is inconsistent with all other combinations: remote HEAD | local repo | local HEAD ----------------------------------------------- points to commit | non-bare | same as remote points to commit | bare | same as remote unborn | non-bare | same as remote unborn | bare | local default So I don't think this is some clever or subtle behavior, but just a bug in 4f37d45706 (clone: respect remote unborn HEAD, 2021-02-05). And it's easy to see how we ended up there. Before that commit, the code to set up the HEAD for an empty repo was guarded by "if (!option_bare)". That's because the only thing it did was call install_branch_config(), and we don't want to do so for a bare repository (unborn HEAD or not). That commit put the handling of unborn HEADs into the same block, since those also need to call install_branch_config(). But the unborn case has an additional side effect of calling create_symref(), and we want that to happen whether we are bare or not. This patch just pulls all of the "figure out the default branch" code out of the "!option_bare" block. Only the actual config installation is kept there. Note that this does mean we might allocate "ref" and not use it (if the remote is empty but did not advertise an unborn HEAD). But that's not really a big deal since this isn't a hot code path, and it keeps the code simple. The alternative would be handling unborn_head_target separately, but that gets confusing since its memory ownership is tangled up with the "ref" variable. There's just one new test, for the case we're fixing. The other ones in the table are handled elsewhere (the unborn non-bare case just above, and the actually-born cases in t5601, t5606, and t5609, as they do not require v2's "unborn" protocol extension). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'jk/clone-clean-upon-transport-error'

2021-06-14T04:33:26Z

Recent "git clone" left a temporary directory behind when the transport layer returned an failure. * jk/clone-clean-upon-transport-error: clone: clean up directory after transport_fetch_refs() failure

clone: clean up directory after transport_fetch_refs() failure

2021-05-19T12:14:59Z

git-clone started respecting errors from the transport subsystem in aab179d937 (builtin/clone.c: don't ignore transport_fetch_refs() errors, 2020-12-03). However, that commit didn't handle the cleanup of the filesystem quite right. The cleanup of the directory that cmd_clone() creates is done by an atexit() handler, which we control with a flag. It starts as JUNK_LEAVE_NONE ("clean up everything"), then progresses to JUNK_LEAVE_REPO when we know we have a valid repo but not working tree, and then finally JUNK_LEAVE_ALL when we have a successful checkout. Most errors cause us to die(), which then triggers the handler to do the right thing based on how far into cmd_clone() we got. But the checks added by aab179d937 instead set the "err" variable and then jump to a new "cleanup" label, which then returns our non-zero status. However, the code after the cleanup label includes setting the flag to JUNK_LEAVE_ALL, and so we accidentally leave the repository and working tree in place. One obvious option to fix this is to reorder the end of the function to set the flag first, before cleanup code, and put the label between them. But we can observe another small bug: the error return from transport_fetch_refs() is generally "-1", and we propagate that to the return value of cmd_clone(), which ultimately becomes the exit code of the process. And we try to avoid transmitting negative values via exit codes (only the low 8 bits are passed along as an unsigned value, though in practice for "-1" this at least retains the property that it's non-zero). Instead, let's just die(). That makes us consistent with rest of the code in the function. It does add a new "fatal:" line to the output, but I'd argue that's a good thing: - in the rare case that the transport code didn't say anything, now the user gets _some_ error message - even if the transport code said something like "error: ssh died of signal 9", it's nice to also say "fatal" to indicate that we considered that to be a show-stopper. Triggering this in the test suite turns out to be surprisingly difficult. Almost every error we'd encounter, including ones deep inside the transport code, cause us to just die() right there! However, one way is to put a fake wrapper around git-upload-pack that sends the complete packfile but exits with a failure code. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

hash: provide per-algorithm null OIDs

2021-04-27T07:31:39Z

Up until recently, object IDs did not have an algorithm member, only a hash. Consequently, it was possible to share one null (all-zeros) object ID among all hash algorithms. Now that we're going to be handling objects from multiple hash algorithms, it's important to make sure that all object IDs have a correct algorithm field. Introduce a per-algorithm null OID, and add it to struct hash_algo. Introduce a wrapper function as well, and use it everywhere we used to use the null_oid constant. Signed-off-by: brian m. carlson Signed-off-by: Junio C Hamano

Merge branch 'll/clone-reject-shallow'

2021-04-08T20:23:25Z

"git clone --reject-shallow" option fails the clone as soon as we notice that we are cloning from a shallow repository. * ll/clone-reject-shallow: builtin/clone.c: add --reject-shallow option

builtin/clone.c: add --reject-shallow option

2021-04-01T19:58:58Z

In some scenarios, users may want more history than the repository offered for cloning, which happens to be a shallow repository, can give them. But because users don't know it is a shallow repository until they download it to local, we may want to refuse to clone this kind of repository, without creating any unnecessary files. The '--depth=x' option cannot be used as a solution; the source may be deep enough to give us 'x' commits when cloned, but the user may later need to deepen the history to arbitrary depth. Teach '--reject-shallow' option to "git clone" to abort as soon as we find out that we are cloning from a shallow repository. Signed-off-by: Li Linchao Signed-off-by: Junio C Hamano