git/merge-recursive.c, branch v2.35.3

Merge branch 'jt/no-abuse-alternate-odb-for-submodules'

2021-10-25T23:06:56Z

Follow through the work to use the repo interface to access submodule objects in-process, instead of abusing the alternate object database interface. * jt/no-abuse-alternate-odb-for-submodules: submodule: trace adding submodule ODB as alternate submodule: pass repo to check_has_commit() object-file: only register submodule ODB if needed merge-{ort,recursive}: remove add_submodule_odb() refs: peeling non-the_repository iterators is BUG refs: teach arbitrary repo support to iterators refs: plumb repo into ref stores

Merge branch 'en/removing-untracked-fixes'

2021-10-13T22:15:57Z

Various fixes in code paths that move untracked files away to make room. * en/removing-untracked-fixes: Documentation: call out commands that nuke untracked files/directories Comment important codepaths regarding nuking untracked files/dirs unpack-trees: avoid nuking untracked dir in way of locally deleted file unpack-trees: avoid nuking untracked dir in way of unmerged file Change unpack_trees' 'reset' flag into an enum Remove ignored files by default when they are in the way unpack-trees: make dir an internal-only struct unpack-trees: introduce preserve_ignored to unpack_trees_options read-tree, merge-recursive: overwrite ignored files by default checkout, read-tree: fix leak of unpack_trees_options.dir t2500: add various tests for nuking untracked files

merge-{ort,recursive}: remove add_submodule_odb()

2021-10-08T22:06:06Z

After the parent commit and some of its ancestors, the only place commits are being accessed through alternates is in the user-facing message formatting code. Fix those, and remove the add_submodule_odb() calls. Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano

Merge branch 'jt/add-submodule-odb-clean-up'

2021-10-06T20:40:11Z

More code paths that use the hack to add submodule's object database to the set of alternate object store have been cleaned up. * jt/add-submodule-odb-clean-up: revision: remove "submodule" from opt struct repository: support unabsorbed in repo_submodule_init submodule: remove unnecessary unabsorbed fallback

unpack-trees: introduce preserve_ignored to unpack_trees_options

2021-09-27T20:38:37Z

Currently, every caller of unpack_trees() that wants to ensure ignored files are overwritten by default needs to: * allocate unpack_trees_options.dir * flip the DIR_SHOW_IGNORED flag in unpack_trees_options.dir->flags * call setup_standard_excludes AND then after the call to unpack_trees() needs to * call dir_clear() * deallocate unpack_trees_options.dir That's a fair amount of boilerplate, and every caller uses identical code. Make this easier by instead introducing a new boolean value where the default value (0) does what we want so that new callers of unpack_trees() automatically get the appropriate behavior. And move all the handling of unpack_trees_options.dir into unpack_trees() itself. While preserve_ignored = 0 is the behavior we feel is the appropriate default, we defer fixing commands to use the appropriate default until a later commit. So, this commit introduces several locations where we manually set preserve_ignored=1. This makes it clear where code paths were previously preserving ignored files when they should not have been; a future commit will flip these to instead use a value of 0 to get the behavior we want. Signed-off-by: Elijah Newren Signed-off-by: Junio C Hamano

read-tree, merge-recursive: overwrite ignored files by default

2021-09-27T20:38:37Z

This fixes a long-standing patchwork of ignored files handling in read-tree and merge-recursive, called out and suggested by Junio long ago. Quoting from commit dcf0c16ef1 ("core.excludesfile clean-up" 2007-11-16): git-read-tree takes --exclude-per-directory=, not because the flexibility was needed. Again, this was because the option predates the standardization of the ignore files. ... On the other hand, I think it makes perfect sense to fix git-read-tree, git-merge-recursive and git-clean to follow the same rule as other commands. I do not think of a valid use case to give an exclude-per-directory that is nonstandard to read-tree command, outside a "negative" test in the t1004 test script. This patch is the first step to untangle this mess. The next step would be to teach read-tree, merge-recursive and clean (in C) to use setup_standard_excludes(). History shows each of these were partially or fully fixed: * clean was taught the new trick in 1617adc7a0 ("Teach git clean to use setup_standard_excludes()", 2007-11-14). * read-tree was primarily used by checkout & merge scripts. checkout and merge later became builtins and were both fixed to use the new setup_standard_excludes() handling in fc001b526c ("checkout,merge: loosen overwriting untracked file check based on info/exclude", 2011-11-27). So the primary users were fixed, though read-tree itself was not. * merge-recursive has now been replaced as the default merge backend by merge-ort. merge-ort fixed this by using setup_standard_excludes() starting early in its implementation; see commit 6681ce5cf6 ("merge-ort: add implementation of checkout()", 2020-12-13), largely due to its design depending on checkout() and thus being influenced by the checkout code. However, merge-recursive itself was not fixed here, in part because its design meant it had difficulty differentiating between untracked files, ignored files, leftover tracked files that haven't been removed yet due to order of processing files, and files written by itself due to collisions). Make the conversion more complete by now handling read-tree and handling at least the unpack_trees() portion of merge-recursive. While merge-recursive is on its way out, fixing the unpack_trees() portion is easy and facilitates some of the later changes in this series. Note that fixing read-tree makes the --exclude-per-directory option to read-tree useless, so we remove it from the documentation (though we continue to accept it if passed). The read-tree changes happen to fix a bug in t1013. Signed-off-by: Elijah Newren Signed-off-by: Junio C Hamano

Merge branch 'jt/add-submodule-odb-clean-up' into jt/no-abuse-alternate-odb-for-submodules

2021-09-23T00:11:09Z

* jt/add-submodule-odb-clean-up: revision: remove "submodule" from opt struct repository: support unabsorbed in repo_submodule_init submodule: remove unnecessary unabsorbed fallback

Merge branch 'ds/mergies-with-sparse-index'

2021-09-20T22:20:45Z

Various mergy operations have been prepared to work efficiently with the sparse index. * ds/mergies-with-sparse-index: sparse-index: integrate with cherry-pick and rebase sequencer: ensure full index if not ORT strategy t1092: add cherry-pick, rebase tests merge-ort: expand only for out-of-cone conflicts merge: make sparse-aware with ORT diff: ignore sparse paths in diffstat

merge: make sparse-aware with ORT

2021-09-09T22:49:04Z

Allow 'git merge' to operate without expanding a sparse index, at least not immediately. The index still will be expanded in a few cases: 1. If the merge strategy is 'recursive', then we enable command_requires_full_index at the start of the merge_recursive() method. We expect sparse-index users to also have the 'ort' strategy enabled. 2. With the 'ort' strategy, if the merge results in a conflicted file, then we expand the index before updating the working tree. The loop that iterates over the worktree replaces index entries and tracks 'origintal_cache_nr' which can become completely wrong if the index expands in the middle of the operation. This safety valve is important before that loop starts. A later change will focus this to only expand if we indeed have a conflict outside of the sparse-checkout cone. 3. Other merge strategies are executed as a 'git merge-X' subcommand, and those strategies are currently protected with the 'command_requires_full_index' guard. Some test updates are required, including a mistaken 'git checkout -b' that did not specify the base branch, causing merges to be fast-forward merges. Signed-off-by: Derrick Stolee Reviewed-by: Elijah Newren Signed-off-by: Junio C Hamano

revision: remove "submodule" from opt struct

2021-09-09T21:09:30Z

Clean up a TODO in revision.h by removing the "submodule" field from struct setup_revision_opt. This field is only used to specify the ref store to use, so use rev_info->repo to determine the ref store instead. The only users of this field are merge-ort.c and merge-recursive.c. However, both these files specify the superproject as rev_info->repo and the submodule as setup_revision_opt->submodule. In order to be able to pass the submodule as rev_info->repo, all commits must be parsed with the submodule explicitly specified; this patch does that as well. (An incremental solution in which only some commits are parsed with explicit submodule will not work, because if the same commit is parsed twice in different repositories, there will be 2 heap-allocated object structs corresponding to that commit, and any flag set by the revision walking mechanism on one of them will not be reflected onto the other.) Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano