git/read-cache.c, branch v2.34.2

Merge branch 'rs/add-dry-run-without-objects'

2021-10-25T23:06:57Z

Stop "git add --dry-run" from creating new blob and tree objects. * rs/add-dry-run-without-objects: add: don't write objects with --dry-run

Merge branch 'rs/make-verify-path-really-verify-again'

2021-10-18T22:47:58Z

Recent sparse-index work broke safety against attempts to add paths with trailing slashes to the index, which has been corrected. * rs/make-verify-path-really-verify-again: read-cache: let verify_path() reject trailing dir separators again read-cache: add verify_path_internal() t3905: show failure to ignore sub-repo

add: don't write objects with --dry-run

2021-10-12T20:15:49Z

When the option --dry-run/-n is given, "git add" doesn't change the index, but still writes out new object files. Only hash the latter without writing instead to make the run as dry as possible. Use this opportunity to also make the hash_flags variable unsigned, to match the index_path() parameter it is used as. Reported-by: git.mexon@spamgourmet.com Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

Merge branch 'sg/test-split-index-fix'

2021-10-11T17:21:47Z

Test updates. * sg/test-split-index-fix: read-cache: fix GIT_TEST_SPLIT_INDEX tests: disable GIT_TEST_SPLIT_INDEX for sparse index tests read-cache: look for shared index files next to the index, too t1600-index: disable GIT_TEST_SPLIT_INDEX t1600-index: don't run git commands upstream of a pipe t1600-index: remove unnecessary redirection

read-cache: let verify_path() reject trailing dir separators again

2021-10-08T00:52:26Z

6e773527b6 (sparse-index: convert from full to sparse, 2021-03-30) made verify_path() accept trailing directory separators for directories, which is necessary for sparse directory entries. This clemency causes "git stash" to stumble over sub-repositories, though, and there may be more unintended side-effects. Avoid them by restoring the old verify_path() behavior and accepting trailing directory separators only in places that are supposed to handle sparse directory entries. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

read-cache: add verify_path_internal()

2021-10-08T00:49:39Z

Turn verify_path() into an internal function that distinguishes between valid paths and those with trailing directory separators and rename it to verify_path_internal(). Provide a wrapper with the old behavior under the old name. No functional change intended. The new function will be used in the next patch. Suggested-by: Junio C Hamano Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

Merge branch 'ab/repo-settings-cleanup'

2021-10-06T20:40:11Z

Code cleanup. * ab/repo-settings-cleanup: repository.h: don't use a mix of int and bitfields repo-settings.c: simplify the setup read-cache & fetch-negotiator: check "enum" values in switch() environment.c: remove test-specific "ignore_untracked..." variable wrapper.c: add x{un,}setenv(), and use xsetenv() in environment.c

repo-settings.c: simplify the setup

2021-09-22T20:15:00Z

Simplify the setup code in repo-settings.c in various ways, making the code shorter, easier to read, and requiring fewer hacks to do the same thing as it did before: Since 7211b9e7534 (repo-settings: consolidate some config settings, 2019-08-13) we have memset() the whole "settings" structure to -1 in prepare_repo_settings(), and subsequently relied on the -1 value. Most of the fields did not need to be initialized to -1, and because we were doing that we had the enum labels "UNTRACKED_CACHE_UNSET" and "FETCH_NEGOTIATION_UNSET" purely to reflect the resulting state created this memset() in prepare_repo_settings(). No other code used or relied on them, more on that below. For the rest most of the subsequent "are we -1, then read xyz" can simply be removed by re-arranging what we read first. E.g. when setting the "index.version" setting we should have first read "feature.experimental", so that it (and "feature.manyfiles") can provide a default for our "index.version". Instead the code setting it, added when "feature.manyFiles"[1] was created, was using the UPDATE_DEFAULT_BOOL() macro added in an earlier commit[2]. That macro is now gone, since it was only needed for this pattern of reading things in the wrong order. This also fixes an (admittedly obscure) logic error where we'd conflate an explicit "-1" value in the config with our own earlier memset() -1. We can also remove the UPDATE_DEFAULT_BOOL() wrapper added in [3]. Using it is redundant to simply using the return value from repo_config_get_bool(), which is non-zero if the provided key exists in the config. Details on edge cases relating to the memset() to -1, continued from "more on that below" above: * UNTRACKED_CACHE_KEEP: In [4] the "unset" and "keep" handling for core.untrackedCache was consolidated. But it while we understand the "keep" value, we don't handle it differently than the case of any other unknown value. So let's retain UNTRACKED_CACHE_KEEP and remove the UNTRACKED_CACHE_UNSET setting (which was always implicitly UNTRACKED_CACHE_KEEP before). We don't need to inform any code after prepare_repo_settings() that the setting was "unset", as far as anyone else is concerned it's core.untrackedCache=keep. if "core.untrackedcache" isn't present in the config. * FETCH_NEGOTIATION_UNSET & FETCH_NEGOTIATION_NONE: Since these two two enum fields added in [5] don't rely on the memzero() setting them to "-1" anymore we don't have to provide them with explicit values. 1. c6cc4c5afd2 (repo-settings: create feature.manyFiles setting, 2019-08-13) 2. 31b1de6a09b (commit-graph: turn on commit-graph by default, 2019-08-13) 3. 31b1de6a09b (commit-graph: turn on commit-graph by default, 2019-08-13) 4. ad0fb659993 (repo-settings: parse core.untrackedCache, 2019-08-13) 5. aaf633c2ad1 (repo-settings: create feature.experimental setting, 2019-08-13) Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

read-cache & fetch-negotiator: check "enum" values in switch()

2021-09-22T20:15:00Z

Change tweak_untracked_cache() in "read-cache.c" to use a switch() to have the compiler assert that we checked all possible values in the "enum untracked_cache_setting" type, and likewise remove the "default" case in fetch_negotiator_init() in favor of checking for "FETCH_NEGOTIATION_UNSET" and "FETCH_NEGOTIATION_NONE". As will be discussed in a subsequent we'll only ever have either of these set to FETCH_NEGOTIATION_NONE, FETCH_NEGOTIATION_UNSET and UNTRACKED_CACHE_UNSET within the prepare_repo_settings() function itself. In preparation for fixing that code let's add a BUG() here to mark this as unreachable code. See ad0fb659993 (repo-settings: parse core.untrackedCache, 2019-08-13) for when the "unset" and "keep" handling for core.untrackedCache was consolidated, and aaf633c2ad1 (repo-settings: create feature.experimental setting, 2019-08-13) for the addition of the "default" pattern in "fetch-negotiator.c". Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

read-cache: fix GIT_TEST_SPLIT_INDEX

2021-09-08T06:28:04Z

Running tests with GIT_TEST_SPLIT_INDEX=1 is supposed to turn on the split index feature and trigger index splitting (mostly) randomly. Alas, this has been broken since 6e37c8ed3c (read-cache.c: fix writing "link" index ext with null base oid, 2019-02-13), and GIT_TEST_SPLIT_INDEX=1 hasn't triggered any index splitting since then. This patch makes GIT_TEST_SPLIT_INDEX work again, though it doesn't restore the pre-6e37c8ed3c behavior. To understand the bug, the fix, and the behavior change we first have to look at how GIT_TEST_SPLIT_INDEX used to work before 6e37c8ed3c: There are two places where we check the value of GIT_TEST_SPLIT_INDEX, and before 6e37c8ed3c they worked like this: 1) In the lower-level do_write_index(), where, if GIT_TEST_SPLIT_INDEX is enabled, we call init_split_index(). This call merely allocates and zero-initializes 'istate->split_index', but does nothing else (i.e. doesn't fill the base/shared index with cache entries, doesn't actually write a shared index file, etc.). Pertinent to this issue, the hash of the base index remains all zeroed out. 2) In the higher-level write_locked_index(), but only when 'istate->split_index' has already been initialized. Then, if GIT_TEST_SPLIT_INDEX is enabled, it randomly sets the flag that triggers index splitting later in this function. This randomness comes from the first byte of the hash of the base index via an 'if ((first_byte & 15) < 6)' condition. However, if 'istate->split_index' hasn't been initialized (i.e. it is still NULL), then write_locked_index() just calls do_write_locked_index(), which internally calls the above mentioned do_write_index(). This means that while GIT_TEST_SPLIT_INDEX=1 usually triggered index splitting randomly, the first two index writes were always deterministic (though I suspect this was unintentional): - The initial index write never splits the index. During the first index write write_locked_index() is called with 'istate->split_index' still uninitialized, so the check in 2) is not executed. It still calls do_write_index(), though, which then executes the check in 1). The resulting all zero base index hash then leads to the 'link' extension being written to '.git/index', though a shared index file is not written: $ rm .git/index $ GIT_TEST_SPLIT_INDEX=1 git update-index --add file $ test-tool dump-split-index .git/index own c6ef71168597caec8553c83d9d0048f1ef416170 base 0000000000000000000000000000000000000000 100644 d00491fd7e5bb6fa28c517a0bb32b8b506539d4d 0 file replacements: deletions: $ ls -l .git/sharedindex.* ls: cannot access '.git/sharedindex.*': No such file or directory - The second index write always splits the index. When the index written in the previous point is read, 'istate->split_index' is initialized because of the presence of the 'link' extension. So during the second write write_locked_index() does run the check in 2), and the first byte of the all zero base index hash always fulfills the randomness condition, which in turn always triggers the index splitting. - Subsequent index writes will find the 'link' extension with a real non-zero base index hash, so from then on the check in 2) is executed and the first byte of the base index hash is as random as it gets (coming from the SHA-1 of index data including timestamps and inodes...). All this worked until 6e37c8ed3c came along, and stopped writing the 'link' extension if the hash of the base index was all zero: $ rm .git/index $ GIT_TEST_SPLIT_INDEX=1 git update-index --add file $ test-tool dump-split-index .git/index own abbd6f6458d5dee73ae8e210ca15a68a390c6fd7 not a split index $ ls -l .git/sharedindex.* ls: cannot access '.git/sharedindex.*': No such file or directory So, since the first index write with GIT_TEST_SPLIT_INDEX=1 doesn't write a 'link' extension, in the second index write 'istate->split_index' remains uninitialized, and the check in 2) is not executed, and ultimately the index is never split. Fix this by modifying write_locked_index() to make sure to check GIT_TEST_SPLIT_INDEX even if 'istate->split_index' is still uninitialized, and initialize it if necessary. The check for GIT_TEST_SPLIT_INDEX and separate init_split_index() call in do_write_index() thus becomes unnecessary, so remove it. Furthermore, add a test to 't1700-split-index.sh' to make sure that GIT_TEST_SPLIT_INDEX=1 will keep working (though only check the index splitting on the first index write, because after that it will be random). Note that this change does not restore the pre-6e37c8ed3c behaviour, as it will deterministically split the index already on the first index write. Since GIT_TEST_SPLIT_INDEX is purely a developer aid, there is no backwards compatibility issue here. The new behaviour did trigger test failures in 't0003-attributes.sh' and 't1600-index.sh', though, which have been fixed in preparatory patches in this series. Signed-off-by: SZEDER Gábor Acked-by: Derrick Stolee Signed-off-by: Junio C Hamano