aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2022-11-23Merge branch 'ab/coccicheck-incremental'Junio C Hamano11-33/+515
"make coccicheck" is time consuming. It has been made to run more incrementally. * ab/coccicheck-incremental: Makefile: don't create a ".build/.build/" for cocci, fix output spatchcache: add a ccache-alike for "spatch" cocci: run against a generated ALL.cocci cocci rules: remove <id>'s from rules that don't need them Makefile: copy contrib/coccinelle/*.cocci to build/ cocci: optimistically use COMPUTE_HEADER_DEPENDENCIES cocci: make "coccicheck" rule incremental cocci: split off "--all-includes" from SPATCH_FLAGS cocci: split off include-less "tests" from SPATCH_FLAGS Makefile: split off SPATCH_BATCH_SIZE comment from "cocci" heading Makefile: have "coccicheck" re-run if flags change Makefile: add ability to TAB-complete cocci *.patch rules cocci rules: remove unused "F" metavariable from pending rule Makefile + shared.mak: rename and indent $(QUIET_SPATCH_T)
2022-11-23Merge branch 'es/chainlint-output'Junio C Hamano22-66/+206
Teach chainlint.pl to annotate the original test definition instead of the token stream. * es/chainlint-output: chainlint: annotate original test definition rather than token stream chainlint: latch start/end position of each token chainlint: tighten accuracy when consuming input stream chainlint: add explanatory comments
2022-11-23Merge branch 'js/remove-stale-scalar-repos'Junio C Hamano2-20/+48
'scalar reconfigure -a' is taught to automatically remove scalar.repo entires which no longer exist. * js/remove-stale-scalar-repos: tests(scalar): tighten the stale `scalar.repo` test some scalar reconfigure -a: remove stale `scalar.repo` entries
2022-11-23Merge branch 'dd/bisect-helper-subcommand'Junio C Hamano3-120/+142
Fix a regression in the bisect-helper which mistakenly treats arguments to the command given to 'git bisect run' as arguments to the helper. * dd/bisect-helper-subcommand: bisect--helper: parse subcommand with OPT_SUBCOMMAND bisect--helper: move all subcommands into their own functions bisect--helper: remove unused options
2022-11-23Merge branch 'ab/submodule-helper-prep-only'Junio C Hamano12-150/+358
Preparation to remove git-submodule.sh and replace it with a builtin. * ab/submodule-helper-prep-only: submodule--helper: use OPT_SUBCOMMAND() API submodule--helper: drop "update --prefix <pfx>" for "-C <pfx> update" submodule--helper: remove --prefix from "absorbgitdirs" submodule API & "absorbgitdirs": remove "----recursive" option submodule.c: refactor recursive block out of absorb function submodule tests: test for a "foreach" blind-spot submodule--helper: fix a memory leak in "status" submodule tests: add tests for top-level flag output submodule--helper: move "config" to a test-tool
2022-11-23chainlint.pl: fix /proc/cpuinfo regexpAndreas Hasenack1-1/+1
29fb2ec3 (chainlint.pl: validate test scripts in parallel, 2022-09-01) introduced a function that gets the number of cores from /proc/cpuinfo on some systems, notably linux. The regexp it uses (^processor\s*:) fails to match the desired lines in the s390x architecture, where they look like this: processor 0: version = FF, identification = 148F67, machine = 2964 As a result, on s390x that function returns 0 as the number of cores, and the chainlint.pl script exits without doing anything. Signed-off-by: Andreas Hasenack <andreas.hasenack@canonical.com> Acked-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-22parse_object(): simplify blob conditionalÆvar Arnfjörð Bjarmason1-1/+1
Commit 8db2dad7a0 (parse_object(): check on-disk type of suspected blob, 2022-11-17) simplified the conditional for checking if we might have a blob. But we can simplify it further. In: !obj || (obj && obj->type == OBJ_BLOB) the short-circuit "OR" means "obj" will always be true on the right-hand side. The compiler almost certainly optimized that out anyway, but dropping it makes the conditional easier to understand for humans. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-22lib-httpd: extend module location auto-detectionEric Sunshine1-1/+2
Although it is possible to manually set LIB_HTTPD_PATH and LIB_HTTPD_MODULE_PATH to point at the location of `httpd` and its modules, doing so is cumbersome and easily forgotten. To address this, 0d344738dc (t/lib-http.sh: Restructure finding of default httpd location, 2010-01-02) enhanced lib-httpd.sh to automatically detect the location of `httpd` and its modules in order to facilitate out-of-the- box testing on a wider range of platforms. Follow that lead by further enhancing it to automatically detect the `httpd` modules on Void Linux, as well. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-22t5516: fail to run in verbose modeJiang Xin1-3/+3
The test case "push with config push.useBitmap" of t5516 was introduced in commit 82f67ee13f (send-pack.c: add config push.useBitmaps, 2022-06-17). It won't work in verbose mode, e.g.: $ sh t5516-fetch-push.sh --run='1,115' -v This is because "git-push" will run in a tty in this case, and the subcommand "git pack-objects" will contain an argument "--progress" instead of "-q". Adding a specific option "--quiet" to "git push" will get a stable result for t5516. Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21list-objects-filter: plug combine_filter_data leakRené Scharfe1-0/+1
filter_combine__init() allocates a struct combine_filter_data object and assigns it to the filter_data member of struct filter_options. Release it in the complementing filter_combine__free(). Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21prune: quiet ENOENT on missing directoriesEric Wong2-1/+11
$GIT_DIR/objects/pack may be removed to save inodes in shared repositories. Quiet down prune in cases where either $GIT_DIR/objects or $GIT_DIR/objects/pack is non-existent, but emit the system error in other cases to help users diagnose permissions problems or resource constraints. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21built-ins: use free() not UNLEAK() if trivial, rm dead codeÆvar Arnfjörð Bjarmason6-33/+35
For a lot of uses of UNLEAK() it would be quite tricky to release the memory involved, or we're missing the relevant *_(release|clear)() functions. But in these cases we have them already, and can just invoke them on the variable(s) involved, instead of UNLEAK(). For "builtin/worktree.c" the UNLEAK() was also added in [1], but the struct member it's unleaking was removed in [2]. The only non-"int" member of that structure is "const char *keep_locked", which comes to us via "argv" or a string literal[3]. We have good visibility via the compiler and tooling (e.g. SANITIZE=address) on bad free()-ing, but none on UNLEAK() we don't need anymore. So let's prefer releasing the memory when it's easy. For "bugreport", "worktree" and "config" we need to start using a "ret = ..." return pattern. For "builtin/bugreport.c" these UNLEAK() were added in [4], and for "builtin/config.c" in [1]. For "config" the code seen here was the only user of the "value" variable. For "ACTION_{RENAME,REMOVE}_SECTION" we need to be sure to return the right exit code in the cases where we were relying on falling through to the top-level. I think there's still a use-case for UNLEAK(), but hat it's changed since then. Using it so that "we can see the real leaks" is counter-productive in these cases. It's more useful to have UNLEAK() be a marker of the remaining odd cases where it's hard to free() the memory for whatever reason. With this change less than 20 of them remain in-tree. 1. 0e5bba53af7 (add UNLEAK annotation for reducing leak false positives, 2017-09-08) 2. d861d34a6ed (worktree: remove extra members from struct add_opts, 2018-04-24) 3. 0db4961c49b (worktree: teach `add` to accept --reason <string> with --lock, 2021-07-15) 4. 0e5bba53af7 and 00d8c311050 (commit: fix "author_ident" leak, 2022-05-12). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21revert: fix parse_options_concat() leakÆvar Arnfjörð Bjarmason2-0/+2
Free memory from parse_options_concat(), which comes from code originally added (then extended) in [1]. At this point we could get several more tests leak-free by free()-ing the xstrdup() just above the line being changed, but that one's trickier than it seems. The sequencer_remove_state() function supposedly owns it, but sometimes we don't call it. I have a fix for it, but it's non-trivial, so let's fix the easy one first. 1. c62f6ec341b (revert: add --ff option to allow fast forward when cherry-picking, 2010-03-06) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21cherry-pick: free "struct replay_opts" membersÆvar Arnfjörð Bjarmason1-0/+3
Call the release_revisions() function added in 1878b5edc03 (revision.[ch]: provide and start using a release_revisions(), 2022-04-13) in cmd_cherry_pick(), as well as freeing the xmalloc()'d "revs" member itself. This is the same change as the one made for cmd_revert() a few lines above it in fd74ac95ac3 (revert: free "struct replay_opts" members, 2022-07-01). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21rebase: don't leak on "--abort"Ævar Arnfjörð Bjarmason2-0/+2
Fix a leak in the recent 6159e7add49 (rebase --abort: improve reflog message, 2022-10-12). Before that commit we'd strbuf_release() the reflog message we were formatting, but when that code was refactored to use "ropts.head_msg" the strbuf_release() was omitted. Ideally the three users of "ropts" in cmd_rebase() should use different "ropts" variables, in practice they're completely separate, as this and the other user in the "switch" statement will "goto cleanup", which won't touch "ropts". The third caller after the "switch" is then unreachable if we take these two branches, so all of them are getting a "{ 0 }" init'd "ropts". So it's OK that we're leaving a stale pointer in "ropts.head_msg", cleaning it up was our responsibility, and it won't be used again. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21connected.c: free the "struct packed_git"Ævar Arnfjörð Bjarmason8-1/+13
The "new_pack" we allocate in check_connected() wasn't being free'd. Let's do that before we return from the function. This has leaked ever since "new_pack" was added to this function in c6807a40dcd (clone: open a shortcut for connectivity check, 2013-05-26). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21sequencer.c: fix "opts->strategy" leak in read_strategy_opts()Ævar Arnfjörð Bjarmason1-0/+1
When "read_strategy_opts()" is called we may have populated the "opts->strategy" before, so we'll need to free() it to avoid leaking memory. We populate it before because we cal get_replay_opts() from within "rebase.c" with an already populated "opts", which we then copy. Then if we're doing a "rebase -i" the sequencer API itself will promptly clobber our alloc'd version of it with its own. If this code is changed to do, instead of the added free() here a: if (opts->strategy) opts->strategy = xstrdup("another leak"); We get a couple of stacktraces from -fsanitize=leak showing how we ended up clobbering the already allocated value, i.e.: Direct leak of 6 byte(s) in 1 object(s) allocated from: #0 0x7f2e8cd45545 in __interceptor_malloc ../../../../src/libsanitizer/lsan/lsan_interceptors.cpp:75 #1 0x7f2e8cb0fcaa in __GI___strdup string/strdup.c:42 #2 0x6c4778 in xstrdup wrapper.c:39 #3 0x66bcb8 in read_strategy_opts sequencer.c:2902 #4 0x66bf7b in read_populate_opts sequencer.c:2969 #5 0x6723f9 in sequencer_continue sequencer.c:5063 #6 0x4a4f74 in run_sequencer_rebase builtin/rebase.c:348 #7 0x4a64c8 in run_specific_rebase builtin/rebase.c:753 #8 0x4a9b8b in cmd_rebase builtin/rebase.c:1824 #9 0x407a32 in run_builtin git.c:466 #10 0x407e0a in handle_builtin git.c:721 #11 0x40803d in run_argv git.c:788 #12 0x40850f in cmd_main git.c:923 #13 0x4eee79 in main common-main.c:57 #14 0x7f2e8ca9f209 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #15 0x7f2e8ca9f2bb in __libc_start_main_impl ../csu/libc-start.c:389 #16 0x405fd0 in _start (git+0x405fd0) Direct leak of 4 byte(s) in 1 object(s) allocated from: #0 0x7f2e8cd45545 in __interceptor_malloc ../../../../src/libsanitizer/lsan/lsan_interceptors.cpp:75 #1 0x7f2e8cb0fcaa in __GI___strdup string/strdup.c:42 #2 0x6c4778 in xstrdup wrapper.c:39 #3 0x4a3c31 in xstrdup_or_null git-compat-util.h:1169 #4 0x4a447a in get_replay_opts builtin/rebase.c:163 #5 0x4a4f5b in run_sequencer_rebase builtin/rebase.c:346 #6 0x4a64c8 in run_specific_rebase builtin/rebase.c:753 #7 0x4a9b8b in cmd_rebase builtin/rebase.c:1824 #8 0x407a32 in run_builtin git.c:466 #9 0x407e0a in handle_builtin git.c:721 #10 0x40803d in run_argv git.c:788 #11 0x40850f in cmd_main git.c:923 #12 0x4eee79 in main common-main.c:57 #13 0x7f2e8ca9f209 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #14 0x7f2e8ca9f2bb in __libc_start_main_impl ../csu/libc-start.c:389 #15 0x405fd0 in _start (git+0x405fd0) This can be seen in e.g. the 4th test of "t3404-rebase-interactive.sh". In the larger picture the ownership of the "struct replay_opts" is quite a mess, e.g. in this case rebase.c's static "get_replay_opts()" function partially creates it, but nothing in rebase.c will free() it. The structure is "mostly owned" by the sequencer API, but it also expects to get these partially populated versions of it. It would be better to have rebase keep track of what it allocated, and free() that, and to pass that as a "const" to the sequencer API, which would copy what it needs to its own version, and to free() that. But doing so is a much larger change, and however messy the ownership boundary is here is consistent with what we're doing already, so let's just free() this to fix the leak. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21ls-files: fix a --with-tree memory leakÆvar Arnfjörð Bjarmason3-1/+3
Fix a memory leak in overlay_tree_on_index(), we need to clear_pathspec() at some point, which might as well be after the last time we use it in the function. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21revision API: call graph_clear() in release_revisions()Ævar Arnfjörð Bjarmason3-0/+3
Call graph_clear() in release_revisions(), this will free memory allocated by e.g. this command, which will now run without memory leaks: git -P log -1 --graph --no-graph --graph Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21unpack-file: fix ancient leak in create_temp_file()Ævar Arnfjörð Bjarmason6-0/+8
Fix a leak that's been with us since 3407bb4940c (Add "unpack-file" helper that unpacks a sha1 blob into a tmpfile., 2005-04-18). See 00c8fd493af (cat-file: use streaming API to print blobs, 2012-03-07) for prior art which shows the same API pattern, i.e. free()-ing the result of read_object_file() after it's used. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21built-ins & libs & helpers: add/move destructors, fix leaksÆvar Arnfjörð Bjarmason10-1/+14
Fix various leaks in built-ins, libraries and a test helper here we were missing a call to strbuf_release(), string_list_clear() etc, or were calling them after a potential "return". Comments on individual changes: - builtin/checkout.c: Fix a memory leak that was introduced in [1]. A sibling leak introduced in [2] was recently fixed in [3]. As with [3] we should be using the wt_status_state_free_buffers() API introduced in [4]. - builtin/repack.c: Fix a leak that's been here since this use of "strbuf_release()" was added in a1bbc6c0176 (repack: rewrite the shell script in C, 2013-09-15). We don't use the variable for anything except this loop, so we can instead free it right afterwards. - builtin/rev-parse: Fix a leak that's been here since this code was added in 21d47835386 (Add a parseopt mode to git-rev-parse to bring parse-options to shell scripts., 2007-11-04). - builtin/stash.c: Fix a couple of leaks that have been here since this code was added in d4788af875c (stash: convert create to builtin, 2019-02-25), we strbuf_release()'d only some of the "struct strbuf" we allocated earlier in the function, let's release all of them. - ref-filter.c: Fix a leak in 482c1191869 (gpg-interface: improve interface for parsing tags, 2021-02-11), we don't use the "payload" variable that we ask parse_signature() to populate for us, so let's free it. - t/helper/test-fake-ssh.c: Fix a leak that's been here since this code was added in 3064d5a38c7 (mingw: fix t5601-clone.sh, 2016-01-27). Let's free the "struct strbuf" as soon as we don't need it anymore. 1. c45f0f525de (switch: reject if some operation is in progress, 2019-03-29) 2. 2708ce62d21 (branch: sort detached HEAD based on a flag, 2021-01-07) 3. abcac2e19fa (ref-filter.c: fix a leak in get_head_description, 2022-09-25) 4. 962dd7ebc3e (wt-status: introduce wt_status_state_free_buffers(), 2020-09-27). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21dir.c: free "ident" and "exclude_per_dir" in "struct untracked_cache"Ævar Arnfjörð Bjarmason2-3/+8
When the "ident" member of the structure was added in 1e8fef609e7 (untracked cache: guard and disable on system changes, 2015-03-08) this function wasn't updated to free it. Let's do so. Let's also free the "exclude_per_dir" memory we've been leaking since[1], while making sure not to free() the constant ".gitignore" string we add by default[2]. As we now have three struct members we're freeing let's change free_untracked_cache() to return early if "uc" isn't defined. We won't hand it to free() now, but that was just for convenience, once we're dealing with >=2 struct members this pattern is more convenient. 1. f9e6c649589 (untracked cache: load from UNTR index extension, 2015-03-08) 2. 039bc64e886 (core.excludesfile clean-up, 2007-11-14) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21read-cache.c: clear and free "sparse_checkout_patterns"Ævar Arnfjörð Bjarmason4-0/+8
The "sparse_checkout_patterns" member was added to the "struct index_state" in 836e25c51b2 (sparse-checkout: hold pattern list in index, 2021-03-30), but wasn't added to discard_index(). Let's do that. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21commit: discard partial cache before (re-)reading itÆvar Arnfjörð Bjarmason19-2/+29
The read_cache() in prepare_to_commit() would end up clobbering the pointer we had for a previously populated "the_index.cache_tree" in the very common case of "git commit" stressed by e.g. the tests being changed here. We'd populate "the_index.cache_tree" by calling "update_main_cache_tree" in prepare_index(), but would not end up with a "fully prepared" index. What constitutes an existing index is clearly overly fuzzy, here we'll check "active_nr" (aka "the_index.cache_nr"), but our "the_index.cache_tree" might have been malloc()'d already. Thus the code added in 11c8a74a64a (commit: write cache-tree data when writing index anyway, 2011-12-06) would end up allocating the "cache_tree", and would interact here with code added in 7168624c353 (Do not generate full commit log message if it is not going to be used, 2007-11-28). The result was a very common memory leak. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21{reset,merge}: call discard_index() before returningÆvar Arnfjörð Bjarmason2-0/+3
These two built-ins both deal with the index, but weren't discarding it. In subsequent commits we'll add more free()-ing to discard_index() that we've missed, but let's first call the existing function. We can doubtless add discard_index() (or its alias discard_cache()) to a lot more places, but let's just add it here for now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21tests: mark tests as passing with SANITIZE=leakÆvar Arnfjörð Bjarmason11-7/+10
This marks tests that have been leak-free since various recent commits, but which were not marked us such when the memory leak was fixed. These were mostly discovered with the "check" mode added in faececa53f9 (test-lib: have the "check" mode for SANITIZE=leak consider leak logs, 2022-07-28). Commits that fixed the last memory leak in these tests. Per narrowing down when they started to pass under SANITIZE=leak with "bisect": - t1022-read-tree-partial-clone.sh: 7e2619d8ff0 (list_objects_filter_options: plug leak of filter_spec strings, 2022-09-08) - t4053-diff-no-index.sh: 07a6f94a6d0 (diff-no-index: release prefixed filenames, 2022-09-07) - t6415-merge-dir-to-symlink.sh: bac92b1f39f (Merge branch 'js/ort-clean-up-after-failed-merge', 2022-08-08). - t5554-noop-fetch-negotiator.sh: 66eede4a37c (prepare_repo_settings(): plug leak of config values, 2022-09-08) - t2012-checkout-last.sh, t7504-commit-msg-hook.sh, t91{15,46,60}-git-svn-*.sh: The in-flight "pw/rebase-no-reflog-action" series, upon which this is based: https://lore.kernel.org/git/pull.1405.git.1667575142.gitgitgadget@gmail.com/ Let's mark all of these as passing with "TEST_PASSES_SANITIZE_LEAK=true", to have it regression tested, including as part of the "linux-leaks" CI job. Additionally, let's remove the "!SANITIZE_LEAK" prerequisite from tests that now pass, these were marked as failing in: - 77e56d55ba6 (diff.c: fix a double-free regression in a18d66cefb, 2022-03-17) - c4d1d526312 (tests: change some 'test $(git) = "x"' to test_cmp, 2022-03-07) These were not spotted with the new "check" mode, but manually, it doesn't cover these sort of prerequisites. There's few enough that we shouldn't bother to automate it. They'll be going away sooner than later. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-21Merge branch 'pw/rebase-no-reflog-action' into ab/various-leak-fixesJunio C Hamano3-32/+46
* pw/rebase-no-reflog-action: rebase: stop exporting GIT_REFLOG_ACTION sequencer: stop exporting GIT_REFLOG_ACTION
2022-11-21cocci: apply "pending" index-compatibility to some "builtin/*.c"Ævar Arnfjörð Bjarmason30-202/+213
Apply "index-compatibility.pending.cocci" rule to "builtin/*", but exclude those where we conflict with in-flight changes. As a result some of them end up using only "the_index", so let's have them use the more narrow "USE_THE_INDEX_VARIABLE" rather than "USE_THE_INDEX_COMPATIBILITY_MACROS". Manual changes not made by coccinelle, that were squashed in: * Whitespace-wrap argument lists for repo_hold_locked_index(), repo_read_index_preload() and repo_refresh_and_write_index(), in cases where the line became too long after the transformation. * Change "refresh_cache()" to "refresh_index()" in a comment in "builtin/update-index.c". * For those whose call was followed by perror("<macro-name>"), change it to perror("<function-name>"), referring to the new function. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cache.h & test-tool.h: add & use "USE_THE_INDEX_VARIABLE"Ævar Arnfjörð Bjarmason10-3/+9
In a preceding commit we fully applied the "index-compatibility.pending.cocci" rule to "t/helper/*". Let's now stop defining "USE_THE_INDEX_COMPATIBILITY_MACROS" in test-tool.h itself, and instead instead define "USE_THE_INDEX_VARIABLE" in the individual test helpers that need it. This mirrors how we do the same thing in the "builtin/" directory. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21{builtin/*,repository}.c: add & use "USE_THE_INDEX_VARIABLE"Ævar Arnfjörð Bjarmason6-6/+10
Split up the "USE_THE_INDEX_COMPATIBILITY_MACROS" into that setting and a more narrow "USE_THE_INDEX_VARIABLE". In the case of these built-ins we only need "the_index" variable, but not the compatibility wrapper for functions we're not using. Let's then have some users of "USE_THE_INDEX_COMPATIBILITY_MACROS" use this more narrow and descriptive define. For context: The USE_THE_INDEX_COMPATIBILITY_MACROS macro was added to test-tool.h in f8adbec9fea (cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch, 2019-01-24). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci: apply "pending" index-compatibility to "t/helper/*.c"Ævar Arnfjörð Bjarmason8-20/+21
Apply the "index-compatibility.pending.cocci" rule to the "t/helper/*" directory, a subsequent commit will extend cache.h to further narrow down the use of "USE_THE_INDEX_COMPATIBILITY_MACROS" in this area. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci & cache.h: apply variable section of "pending" index-compatibilityÆvar Arnfjörð Bjarmason19-129/+132
Mostly apply the part of "index-compatibility.pending.cocci" that renames the global variables like "active_nr", which are a shorthand to referencing (in that case) a struct member as "the_index.cache_nr". In doing so move more of "index-compatibility.pending.cocci" to "index-compatibility.cocci". In the case of "active_nr" we'd have a textual conflict with "ab/various-leak-fixes" in "next"[1]. Let's exclude that specific case while moving the rule over from "pending". 1. 407b94280f8 (commit: discard partial cache before (re-)reading it, 2022-11-08) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci & cache.h: apply a selection of "pending" index-compatibilityÆvar Arnfjörð Bjarmason11-41/+40
Apply a selection of rules in "index-compatibility.pending.cocci" tree-wide, and in doing so migrate them to "index-compatibility.cocci". As in preceding commits the only manual changes here are the macro removals in "cache.h", and the update to the '*.cocci" rules. The rest of the C code changes are the result of applying those updated rules. Move rules for some rarely used cache compatibility macros from "index-compatibility.pending.cocci" to "index-compatibility.cocci" and apply them. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci: add a index-compatibility.pending.cocciÆvar Arnfjörð Bjarmason1-0/+110
Add a coccinelle rule which covers the rest of the macros guarded by "USE_THE_INDEX_COMPATIBILITY_MACROS" cache.h. If the result of this were applied it can be reduced down to just: #ifdef USE_THE_INDEX_COMPATIBILITY_MACROS extern struct index_state the_index; #endif But that patch is just under 2000 lines, so let's first add this as a "pending", and then incrementally pick changes from it in subsequent commits. In doing that we'll migrate rules from this "index-compatibility.pending.cocci" to the "index-compatibility.cocci" created in a preceding commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21read-cache API & users: make discard_index() return voidÆvar Arnfjörð Bjarmason6-16/+19
The discard_index() function has not returned non-zero since 7a51ed66f65 (Make on-disk index representation separate from in-core one, 2008-01-14), but we've had various code in-tree still acting as though that might be the case. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cocci & cache.h: remove rarely used "the_index" compat macrosÆvar Arnfjörð Bjarmason13-29/+65
Since 4aab5b46f44 (Make read-cache.c "the_index" free., 2007-04-01) we've been undergoing a slow migration away from these macros, but haven't made much progress since f8adbec9fea (cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch, 2019-01-24). Let's move forward a bit by changing the users of those macros that are rare enough that we can convert them in one go, and then remove the compatibility shim. The only manual change to the C code here is to "cache.h", the rest is all the result of applying the new "index-compatibility.cocci". Even though it's a one-off, let's keep the coccinelle rules for now. We'll extend them in subsequent commits, and this will help anything that's in-flight or out-of-tree to migrate. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21builtin/{grep,log}.: don't define "USE_THE_INDEX_COMPATIBILITY_MACROS"Ævar Arnfjörð Bjarmason2-2/+0
Adding "USE_THE_INDEX_COMPATIBILITY_MACROS" to these two appears to have been unnecessary from the start, as going back and compiling f8adbec9fea (cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch, 2019-01-24) without that addition works. Let's not have these ask for the compatibility macros from cache.h that they don't need. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-21cache.h: remove unused "the_index" compat macrosÆvar Arnfjörð Bjarmason1-5/+0
The "active_alloc" macro added in 228e94f9357 (Move index-related variables into a structure., 2007-04-01) has not been used since 4aab5b46f44 (Make read-cache.c "the_index" free., 2007-04-01). Let's remove it. The rest of these are likewise unused, so let's not keep them around. E.g. 12cd0bf9b02 (dir: stop using the index compatibility macros, 2017-05-05) is the last use of "cache_dir_exists". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-18The thirteenth batchTaylor Blau1-0/+21
Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-18Merge branch 'en/sparse-checkout-design'Taylor Blau1-0/+1103
Design doc. * en/sparse-checkout-design: sparse-checkout.txt: new document with sparse-checkout directions
2022-11-18Merge branch 'jk/branch-delete-detached'Taylor Blau2-6/+39
Fix a bug where `git branch -d` did not work on an orphaned HEAD. * jk/branch-delete-detached: branch: gracefully handle '-d' on orphan HEAD
2022-11-18Merge branch 'mh/credential-unrecognized-attrs'Taylor Blau2-0/+3
Docfix. * mh/credential-unrecognized-attrs: docs: clarify that credential discards unrecognised attributes
2022-11-18Merge branch 'vd/skip-cache-tree-update'Taylor Blau14-3/+145
Avoid calling 'cache_tree_update()' when doing so would be redundant. * vd/skip-cache-tree-update: rebase: use 'skip_cache_tree_update' option read-tree: use 'skip_cache_tree_update' option reset: use 'skip_cache_tree_update' option unpack-trees: add 'skip_cache_tree_update' option cache-tree: add perf test comparing update and prime
2022-11-18Merge branch 'mh/increase-credential-cache-timeout'Taylor Blau1-2/+2
Update the credential-cache documentation to provide a more realistic example. * mh/increase-credential-cache-timeout: Documentation: increase example cache timeout to 1 hour
2022-11-18Merge branch 'vd/update-refs-delete'Taylor Blau2-3/+113
`git rebase --update-refs` would delete references when all `update-ref` commands in the sequencer were removed, which has been corrected. * vd/update-refs-delete: rebase --update-refs: avoid unintended ref deletion
2022-11-18Merge branch 'tb/repack-expire-to'Taylor Blau3-10/+188
"git repack" learns to send cruft objects out of the way into packfiles outside the repository. * tb/repack-expire-to: builtin/repack.c: implement `--expire-to` for storing pruned objects builtin/repack.c: write cruft packs to arbitrary locations builtin/repack.c: pass "cruft_expiration" to `write_cruft_pack` builtin/repack.c: pass "out" to `prepare_pack_objects`
2022-11-18Merge branch 'ab/sha-makefile-doc'Taylor Blau9-104/+173
Makefile comments updates and reordering to clarify knobs used to choose SHA implementations. * ab/sha-makefile-doc: Makefile: discuss SHAttered in *_SHA{1,256} discussion Makefile: document default SHA-1 backend on OSX Makefile & test-tool: replace "DC_SHA1" variable with a "define" Makefile: document SHA-1 and SHA-256 default and selection order Makefile: document default SHA-256 backend Makefile: rephrase the discussion of *_SHA1 knobs Makefile: create and use sections for "define" flag listing Makefile: correct DC_SHA1 documentation INSTALL: remove discussion of SHA-1 backends Makefile: always (re)set DC_SHA1 on fallback
2022-11-18Merge branch 'ab/misc-hook-submodule-run-command'Taylor Blau3-6/+19
Various test updates. * ab/misc-hook-submodule-run-command: run-command tests: test stdout of run_command_parallel() submodule tests: reset "trace.out" between "grep" invocations hook tests: fix redirection logic error in 96e7225b310
2022-11-18delta-islands: free island-related data after useEric Wong1-20/+51
On my use case involving 771 islands of Linux on kernel.org, this reduces memory usage by around 25MB. The bulk of that comes from free_remote_islands, since free_config_regexes only saves around 40k. This memory is saved early in the memory-intensive pack process, making it available for the remainder of the long process. Signed-off-by: Eric Wong <e@80x24.org> Co-authored-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-18parse_object(): check on-disk type of suspected blobJeff King2-4/+4
In parse_object(), we try to handle blobs by streaming rather than loading them entirely into memory. The most common case here will be that we haven't seen the object yet and check oid_object_info(), which tells us we have a blob. But we trigger this code on one other case: when we have an in-memory object struct with type OBJ_BLOB (and without its "parsed" flag set, since otherwise we'd return early from the function). This indicates that some other part of the code suspected we have a blob (e.g., it was mentioned by a tree or tag) but we haven't yet looked at the on-disk copy. In this case before hitting the streaming path, we check if we have the object on-disk at all. This is mostly pointless extra work, as the streaming path would complain if it couldn't open the object (albeit with the message "hash mismatch", which is a little misleading). But it's also insufficient to catch all problems. The streaming code will only tell us "yes, the on-disk object matches the oid". But it doesn't actually confirm that what we found was indeed a blob, and neither does repo_has_object_file(). One way to improve this would be to teach stream_object_signature() to check the type (either by returning it to us to check, or taking an "expected" type). But there's an even simpler fix here: if we suspect the object is a blob, just call oid_object_info() to confirm that we have it on-disk, and that it really is a blob. This is slightly less efficient than teaching stream_object_signature() to do it (since it has to open the object already). But this case very rarely comes up. In practice, we usually don't have any clue what the type is, in which case we already call oid_object_info(). This "suspected" case happens only when some other code created an object struct but didn't actually parse the blob, which is actually tricky to trigger at all (see the discussion of the test below). I reworked the conditional a bit so that instead of: if ((suspected_blob && oid_object_info() == OBJ_BLOB) (no_clue && oid_object_info() == OBJ_BLOB) we have the simpler: if ((suspected_blob || no_clue) && oid_object_info() == OBJ_BLOB) This is shorter, but also reflects what we really want say, which is "have we ruled out this being a blob; if not, check it on-disk". In either case, if oid_object_info() fails to tell us it's a blob, we'll skip the streaming code path and call repo_read_object_file(), just as before. And if we really do have a mismatch with the existing object struct, we'll eventually call lookup_commit(), etc, via parse_object_buffer(), which will complain that it doesn't match our existing obj->type. So this fixes one of the lingering expect_failure cases from 0616617c7e (t: introduce tests for unexpected object types, 2019-04-09). That test works by peeling a tag that claims to point to a blob (triggering us to create the struct), but really points to something else, which we later discover when we call parse_object() as part of the actual traversal). Prior to this commit, we'd quietly check the sha1 and mark the blob as "parsed". Now we correctly complain about the mismatch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com>