git/dir.c, branch v2.33.6

use fspathhash() everywhere

2021-07-30T19:14:27Z

cf2dc1c238 (speed up alt_odb_usable() with many alternates, 2021-07-07) introduced the function fspathhash() for calculating path hashes while respecting the configuration option core.ignorecase. Call it instead of open-coding it; the resulting code is shorter and less repetitive. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

Merge branch 'ds/status-with-sparse-index'

2021-07-28T20:18:02Z

"git status" codepath learned to work with sparsely populated index without hydrating it fully. * ds/status-with-sparse-index: t1092: document bad sparse-checkout behavior fsmonitor: integrate with sparse index wt-status: expand added sparse directory entries status: use sparse-index throughout status: skip sparse-checkout percentage with sparse-index diff-lib: handle index diffs with sparse dirs dir.c: accept a directory as part of cone-mode patterns unpack-trees: unpack sparse directory entries unpack-trees: rename unpack_nondirectories() unpack-trees: compare sparse directories correctly unpack-trees: preserve cache_bottom t1092: add tests for status/add and sparse files t1092: expand repository data shape t1092: replace incorrect 'echo' with 'cat' sparse-index: include EXTENDED flag when expanding sparse-index: skip indexes with unmerged entries

Merge branch 'ew/many-alternate-optim'

2021-07-28T20:17:57Z

Optimization for repositories with many alternate object store. * ew/many-alternate-optim: oidtree: a crit-bit tree for odb_loose_cache oidcpy_with_padding: constify `src' arg make object_directory.loose_objects_subdir_seen a bitmap avoid strlen via strbuf_addstr in link_alt_odb_entry speed up alt_odb_usable() with many alternates

dir.c: accept a directory as part of cone-mode patterns

2021-07-14T20:42:49Z

When we have sparse directory entries in the index, we want to compare that directory against sparse-checkout patterns. Those pattern matching algorithms are built expecting a file path, not a directory path. This is especially important in the "cone mode" patterns which will match files that exist within the "parent directories" as well as the recursive directory matches. If path_matches_pattern_list() is given a directory, we can add a fake filename ("-") to the directory and get the same results as before, assuming we are in cone mode. Since sparse index requires cone mode patterns, this is an acceptable assumption. Reviewed-by: Elijah Newren Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

speed up alt_odb_usable() with many alternates

2021-07-08T00:21:12Z

With many alternates, the duplicate check in alt_odb_usable() wastes many cycles doing repeated fspathcmp() on every existing alternate. Use a khash to speed up lookups by odb->path. Since the kh_put_* API uses the supplied key without duplicating it, we also take advantage of it to replace both xstrdup() and strbuf_release() in link_alt_odb_entry() with strbuf_detach() to avoid the allocation and copy. In a test repository with 50K alternates and each of those 50K alternates having one alternate each (for a total of 100K total alternates); this speeds up lookup of a non-existent blob from over 16 minutes to roughly 2.7 seconds on my busy workstation. Note: all underlying git object directories were small and unpacked with only loose objects and no packs. Having to load packs increases times significantly. Signed-off-by: Eric Wong Signed-off-by: Junio C Hamano

dir.[ch]: replace dir_init() with DIR_INIT

2021-07-01T19:32:22Z

Remove the dir_init() function and replace it with a DIR_INIT macro. In many cases in the codebase we need to initialize things with a function for good reasons, e.g. needing to call another function on initialization. The "dir_init()" function was not one such case, and could trivially be replaced with a more idiomatic macro initialization pattern. The only place where we made use of its use of memset() was in dir_clear() itself, which resets the contents of an an existing struct pointer. Let's use the new "memcpy() a 'blank' struct on the stack" idiom to do that reset. Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

Merge branch 'en/dir-traversal'

2021-05-28T04:03:00Z

Fix-up to a topic that is already in 'master'. * en/dir-traversal: dir: introduce readdir_skip_dot_and_dotdot() helper dir: update stale description of treat_directory() Revert "dir: update stale description of treat_directory()" Revert "dir: introduce readdir_skip_dot_and_dotdot() helper"

dir: introduce readdir_skip_dot_and_dotdot() helper

2021-05-27T05:02:37Z

Many places in the code were doing while ((d = readdir(dir)) != NULL) { if (is_dot_or_dotdot(d->d_name)) continue; ...process d... } Introduce a readdir_skip_dot_and_dotdot() helper to make that a one-liner: while ((d = readdir_skip_dot_and_dotdot(dir)) != NULL) { ...process d... } This helper particularly simplifies checks for empty directories. Also use this helper in read_cached_dir() so that our statistics are consistent across platforms. (In other words, read_cached_dir() should have been using is_dot_or_dotdot() and skipping such entries, but did not and left it to treat_path() to detect and mark such entries as path_none.) Signed-off-by: Elijah Newren Signed-off-by: Junio C Hamano

dir: update stale description of treat_directory()

2021-05-27T05:02:37Z

The documentation comment for treat_directory() was originally written in 095952 (Teach directory traversal about subprojects, 2007-04-11) which was before the 'struct dir_struct' split its bitfield of named options into a 'flags' enum in 7c4c97c0 (Turn the flags in struct dir_struct into a single variable, 2009-02-16). When those flags changed, the comment became stale, since members like 'show_other_directories' transitioned into flags like DIR_SHOW_OTHER_DIRECTORIES. Update the comments for treat_directory() to use these flag names rather than the old member names. Signed-off-by: Derrick Stolee Signed-off-by: Elijah Newren Signed-off-by: Junio C Hamano

Revert "dir: update stale description of treat_directory()"

2021-05-27T05:02:37Z

This reverts commit 4e689d81718eb6e939cace317ea3e33cb994dcbb, to be replaced with a reworked version.