git/dir.c, branch v2.17.2

Sync with Git 2.16.4

2018-05-22T05:25:26Z

* maint-2.16: Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths

Merge branch 'bp/untracked-cache-noflush'

2018-03-08T20:36:30Z

Writing out the index file when the only thing that changed in it is the untracked cache information is often wasteful, and this has been optimized out. * bp/untracked-cache-noflush: untracked cache: use git_env_bool() not getenv() for customization dir.c: don't flag the index as dirty for changes to the untracked cache

untracked cache: use git_env_bool() not getenv() for customization

2018-02-28T21:27:10Z

GIT_DISABLE_UNTRACKED_CACHE and GIT_TEST_UNTRACKED_CACHE are only sensed for their presense by using getenv(); use git_env_bool() instead so that GIT_DISABLE_UNTRACKED_CACHE=false would work as naïvely expected. Also rename GIT_TEST_UNTRACKED_CACHE to GIT_FORCE_UNTRACKED_CACHE to express what it does more honestly. Forcing its use may be one useful thing to do while testing the feature, but testing does not have to be the only use of the knob. While at it, avoid repeated calls to git_env_bool() by capturing the return value from the first call in a static variable. Signed-off-by: Junio C Hamano

Merge branch 'nd/fix-untracked-cache-invalidation'

2018-02-27T18:33:50Z

Some bugs around "untracked cache" feature have been fixed. * nd/fix-untracked-cache-invalidation: dir.c: ignore paths containing .git when invalidating untracked cache dir.c: stop ignoring opendir() error in open_cached_dir() dir.c: fix missing dir invalidation in untracked code dir.c: avoid stat() in valid_cached_dir() status: add a failing test showing a core.untrackedCache bug

Merge branch 'nd/trace-index-ops'

2018-02-15T22:55:44Z

* nd/trace-index-ops: trace: measure where the time is spent in the index-heavy operations

Merge branch 'po/object-id'

2018-02-15T22:55:43Z

Conversion from uchar[20] to struct object_id continues. * po/object-id: sha1_file: rename hash_sha1_file_literally sha1_file: convert write_loose_object to object_id sha1_file: convert force_object_loose to object_id sha1_file: convert write_sha1_file to object_id notes: convert write_notes_tree to object_id notes: convert combine_notes_* to object_id commit: convert commit_tree* to object_id match-trees: convert splice_tree to object_id cache: clear whole hash buffer with oidclr sha1_file: convert hash_sha1_file to object_id dir: convert struct sha1_stat to use object_id sha1_file: convert pretend_sha1_file to object_id

dir.c: ignore paths containing .git when invalidating untracked cache

2018-02-07T20:27:02Z

read_directory() code ignores all paths named ".git" even if it's not a valid git repository. See treat_path() for details. Since ".git" is basically invisible to read_directory(), when we are asked to invalidate a path that contains ".git", we can safely ignore it because the slow path would not consider it anyway. This helps when fsmonitor is used and we have a real ".git" repo at worktree top. Occasionally .git/index will be updated and if the fsmonitor hook does not filter it, untracked cache is asked to invalidate the path ".git/index". Without this patch, we invalidate the root directory unncessarily, which: - makes read_directory() fall back to slow path for root directory (slower) - makes the index dirty (because UNTR extension is updated). Depending on the index size, writing it down could also be slow. A note about the new "safe_path" knob. Since this new check could be relatively expensive, avoid it when we know it's not needed. If the path comes from the index, it can't contain ".git". If it does contain, we may be screwed up at many more levels, not just this one. Noticed-by: Ævar Arnfjörð Bjarmason Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano

dir.c: don't flag the index as dirty for changes to the untracked cache

2018-02-05T20:55:49Z

The untracked cache saves its current state in the UNTR index extension. Currently, _any_ change to that state causes the index to be flagged as dirty and written out to disk. Unfortunately, the cost to write out the index can exceed the savings gained by using the untracked cache. Since it is a cache that can be updated from the current state of the working directory, there is no functional requirement that the index be written out for every change to the untracked cache. Update the untracked cache logic so that it no longer forces the index to be written to disk except in the case where the extension is being turned on or off. When some other git command requires the index to be written to disk, the untracked cache will take advantage of that to save it's updated state as well. This results in a performance win when looked at over common sequences of git commands (ie such as a status followed by add, commit, etc). After this patch, all the logic to track statistics for the untracked cache could be removed as it is only used by debug tracing used to debug the untracked cache. Signed-off-by: Ben Peart Signed-off-by: Junio C Hamano

trace: measure where the time is spent in the index-heavy operations

2018-02-02T19:20:16Z

All the known heavy code blocks are measured (except object database access). This should help identify if an optimization is effective or not. An unoptimized git-status would give something like below: 0.001791141 s: read cache ... 0.004011363 s: preload index 0.000516161 s: refresh index 0.003139257 s: git command: ... 'status' '--porcelain=2' 0.006788129 s: diff-files 0.002090267 s: diff-index 0.001885735 s: initialize name hash 0.032013138 s: read directory 0.051781209 s: git command: './git' 'status' Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano

dir.c: stop ignoring opendir() error in open_cached_dir()

2018-02-02T18:16:23Z

A follow-up to the recently fixed bugs in the untracked invalidation. If opendir() fails it should show a warning, perhaps this should die, but if this ever happens the error is probably recoverable for the user, and dying would just make things worse. Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano