git/t/perf, branch v2.45.2

t/perf: avoid redundant use of cat

2024-03-16T18:08:56Z

Take care to redirect stdin, otherwise the output of wc would also contain the file name. Signed-off-by: Beat Bolli Acked-by: Taylor Blau Signed-off-by: Junio C Hamano

Merge branch 'tb/multi-pack-verbatim-reuse'

2024-01-13T00:09:56Z

Streaming spans of packfile data used to be done only from a single, primary, pack in a repository with multiple packfiles. It has been extended to allow reuse from other packfiles, too. * tb/multi-pack-verbatim-reuse: (26 commits) t/perf: add performance tests for multi-pack reuse pack-bitmap: enable reuse from all bitmapped packs pack-objects: allow setting `pack.allowPackReuse` to "single" t/test-lib-functions.sh: implement `test_trace2_data` helper pack-objects: add tracing for various packfile metrics pack-bitmap: prepare to mark objects from multiple packs for reuse pack-revindex: implement `midx_pair_to_pack_pos()` pack-revindex: factor out `midx_key_to_pack_pos()` helper midx: implement `midx_preferred_pack()` git-compat-util.h: implement checked size_t to uint32_t conversion pack-objects: include number of packs reused in output pack-objects: prepare `write_reused_pack_verbatim()` for multi-pack reuse pack-objects: prepare `write_reused_pack()` for multi-pack reuse pack-objects: pass `bitmapped_pack`'s to pack-reuse functions pack-objects: keep track of `pack_start` for each reuse pack pack-objects: parameterize pack-reuse routines over a single pack pack-bitmap: return multiple packs via `reuse_partial_packfile_from_bitmap()` pack-bitmap: simplify `reuse_partial_packfile_from_bitmap()` signature ewah: implement `bitmap_is_empty()` pack-bitmap: pass `bitmapped_pack` struct to pack-reuse functions ...

Merge branch 'js/update-urls-in-doc-and-comment'

2023-12-18T22:10:12Z

Stale URLs have been updated to their current counterparts (or archive.org) and HTTP links are replaced with working HTTPS links. * js/update-urls-in-doc-and-comment: doc: refer to internet archive doc: update links for andre-simon.de doc: switch links to https doc: update links to current pages

t/perf: add performance tests for multi-pack reuse

2023-12-14T22:38:09Z

To ensure that we don't regress either the size or runtime performance of multi-pack reuse, add a performance test to measure both of these. The test partitions the objects in GIT_TEST_PERF_LARGE_REPO into 1, 10, and 100 packs, and then tries to perform a "clone" at each stage with both single- and multi-pack reuse enabled. Note that the `repack_into_n_chunks()` function in this new test script differs from the existing `repack_into_n()`. The former partitions the repository into N equal-sized chunks, while the latter produces N packs of five commits each (plus their objects), and then another pack with the remainder. On git.git, I can produce the following results on my machine: Test this tree -------------------------------------------------------------------------------- 5332.3: clone for 1-pack scenario (single-pack reuse) 1.57(2.99+0.15) 5332.4: clone size for 1-pack scenario (single-pack reuse) 231.8M 5332.5: clone for 1-pack scenario (multi-pack reuse) 1.79(2.96+0.21) 5332.6: clone size for 1-pack scenario (multi-pack reuse) 231.7M 5332.9: clone for 10-pack scenario (single-pack reuse) 3.89(16.75+0.35) 5332.10: clone size for 10-pack scenario (single-pack reuse) 209.9M 5332.11: clone for 10-pack scenario (multi-pack reuse) 1.56(2.99+0.17) 5332.12: clone size for 10-pack scenario (multi-pack reuse) 224.4M 5332.15: clone for 100-pack scenario (single-pack reuse) 8.24(54.31+0.59) 5332.16: clone size for 100-pack scenario (single-pack reuse) 278.3M 5332.17: clone for 100-pack scenario (multi-pack reuse) 2.13(2.44+0.33) 5332.18: clone size for 100-pack scenario (multi-pack reuse) 357.9M Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

Merge branch 'vd/for-each-ref-unsorted-optimization'

2023-12-10T00:37:50Z

"git for-each-ref --no-sort" still sorted the refs alphabetically which paid non-trivial cost. It has been redefined to show output in an unspecified order, to allow certain optimizations to take advantage of. * vd/for-each-ref-unsorted-optimization: t/perf: add perf tests for for-each-ref ref-filter.c: use peeled tag for '*' format fields for-each-ref: clean up documentation of --format ref-filter.c: filter & format refs in the same callback ref-filter.c: refactor to create common helper functions ref-filter.c: rename 'ref_filter_handler()' to 'filter_one()' ref-filter.h: add functions for filter/format & format-only ref-filter.h: move contains caches into filter ref-filter.h: add max_count and omit_empty to ref_format ref-filter.c: really don't sort when using --no-sort

doc: switch links to https

2023-11-26T01:07:05Z

These sites offer https versions of their content. Using the https versions provides some protection for users. Signed-off-by: Josh Soref Signed-off-by: Junio C Hamano

t/perf: add perf tests for for-each-ref

2023-11-16T05:03:01Z

Add performance tests for 'for-each-ref'. The tests exercise different combinations of filters/formats/options, as well as the overall performance of 'git for-each-ref | git cat-file --batch-check' to demonstrate the performance difference vs. 'git for-each-ref' with "%(*fieldname)" format specifiers. All tests are run against a repository with 40k loose refs - 10k commits, each having a unique: - branch - custom ref (refs/custom/special_*) - annotated tag pointing at the commit - annotated tag pointing at the other annotated tag (i.e., a nested tag) After those tests are finished, the refs are packed with 'pack-refs --all' and the same tests are rerun. Signed-off-by: Victoria Dye Signed-off-by: Junio C Hamano

global: convert trivial usages of `test -a/-o `

2023-11-11T00:21:00Z

Our coding guidelines say to not use `test` with `-a` and `-o` because it can easily lead to bugs. Convert trivial cases where we still use these to instead instead concatenate multiple invocations of `test` via `&&` and `||`, respectively. While not all of the converted instances can cause ambiguity, it is worth getting rid of all of them regardless: - It becomes easier to reason about the code as we do not have to argue why one use of `-a`/`-o` is okay while another one isn't. - We don't encourage people to use these expressions. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

check-attr: integrate with sparse-index

2023-08-11T16:44:52Z

Set the requires-full-index to false for "check-attr". Add a test to ensure that the index is not expanded whether the files are outside or inside the sparse-checkout cone when the sparse index is enabled. The `p2000` tests demonstrate a ~63% execution time reduction for 'git check-attr' using a sparse index. Test before after ----------------------------------------------------------------------- 2000.106: git check-attr -a f2/f4/a (full-v3) 0.05 0.05 +0.0% 2000.107: git check-attr -a f2/f4/a (full-v4) 0.05 0.05 +0.0% 2000.108: git check-attr -a f2/f4/a (sparse-v3) 0.04 0.02 -50.0% 2000.109: git check-attr -a f2/f4/a (sparse-v4) 0.04 0.01 -75.0% Helped-by: Victoria Dye Signed-off-by: Shuqi Liang Signed-off-by: Junio C Hamano

Merge branch 'sl/worktree-sparse'

2023-06-23T18:21:16Z

"git worktree" learned to work better with sparse index feature. * sl/worktree-sparse: worktree: integrate with sparse-index