git/builtin/pack-objects.c, branch v2.22.2

Merge branch 'jk/delta-islands-progress-fix' into maint

2019-07-29T19:38:19Z

The codepath to compute delta islands used to spew progress output without giving the callers any way to squelch it, which has been fixed. * jk/delta-islands-progress-fix: delta-islands: respect progress flag

delta-islands: respect progress flag

2019-06-20T20:29:49Z

The delta island code always prints "Marked %d islands", even if progress has been suppressed with --no-progress or by sending stderr to a non-tty. Let's pass a progress boolean to load_delta_islands(). We already do the same thing for the progress meter in resolve_tree_islands(). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'ds/midx-too-many-packs'

2019-05-19T07:45:30Z

The code to generate the multi-pack idx file was not prepared to see too many packfiles and ran out of open file descriptor, which has been corrected. * ds/midx-too-many-packs: midx: add packs to packed_git linked list midx: pass a repository pointer

Merge branch 'ew/repack-with-bitmaps-by-default'

2019-05-13T14:50:32Z

The connectivity bitmaps are created by default in bare repositories now; also the pathname hash-cache is created by default to avoid making crappy deltas when repacking. * ew/repack-with-bitmaps-by-default: pack-objects: default to writing bitmap hash-cache t5310: correctly remove bitmaps for jgit test repack: enable bitmaps by default on bare repos

Merge branch 'jk/pack-objects-reports-num-objects-to-trace2'

2019-05-08T15:37:23Z

The "git pack-objects" command learned to report the number of objects it packed via the trace2 mechanism. * jk/pack-objects-reports-num-objects-to-trace2: pack-objects: write objects packed to trace2

midx: pass a repository pointer

2019-05-07T04:48:41Z

Much of the multi-pack-index code focuses on the multi_pack_index struct, and so we only pass a pointer to the current one. However, we will insert a dependency on the packed_git linked list in a future change, so we will need a repository reference. Inserting these parameters is a significant enough change to split out. Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

pack-objects: write objects packed to trace2

2019-04-12T06:19:46Z

This is useful when investigating performance of pushes, and other times when no progress information is written (because the pack is written to stdout). Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano

pack-bitmap: switch hash tables to use struct object_id

2019-04-01T02:57:37Z

Instead of storing unsigned char pointers in the hash tables, switch to storing instances of struct object_id. Update several internal functions and one external function to take pointers to struct object_id. Signed-off-by: brian m. carlson Signed-off-by: Junio C Hamano

pack-objects: default to writing bitmap hash-cache

2019-03-18T05:11:15Z

Enabling pack.writebitmaphashcache should always be a performance win. It costs only 4 bytes per object on disk, and the timings in ae4f07fbcc (pack-bitmap: implement optional name_hash cache, 2013-12-21) show it improving fetch and partial-bitmap clone times by 40-50%. The only reason we didn't enable it by default at the time is that early versions of JGit's bitmap reader complained about the presence of optional header bits it didn't understand. But that was changed in JGit's d2fa3987a (Use bitcheck to check for presence of OPT_FULL option, 2013-10-30), which made it into JGit v3.5.0 in late 2014. So let's turn this option on by default. It's backwards-compatible with all versions of Git, and if you are also using JGit on the same repository, you'd only run into problems using a version that's almost 5 years old. We'll drop the manual setting from all of our test scripts, including perf tests. This isn't strictly necessary, but it has two advantages: 1. If the hash-cache ever stops being enabled by default, our perf regression tests will notice. 2. We can use the modified perf tests to show off the behavior of an otherwise unconfigured repo, as shown below. These are the results of a few of a perf tests against linux.git that showed interesting results. You can see the expected speedup in 5310.4, which was noted in ae4f07fbcc. Curiously, 5310.8 did not improve (and actually got slower), despite seeing the opposite in ae4f07fbcc. I don't have an explanation for that. The tests from p5311 did not exist back then, but do show improvements (a smaller pack due to better deltas, which we found in less time). Test HEAD^ HEAD ------------------------------------------------------------------------------------- 5310.4: simulated fetch 7.39(22.70+0.25) 5.64(11.43+0.22) -23.7% 5310.8: clone (partial bitmap) 18.45(24.83+1.19) 19.94(28.40+1.36) +8.1% 5311.31: server (128 days) 0.41(1.13+0.05) 0.34(0.72+0.02) -17.1% 5311.32: size (128 days) 7.4M 7.0M -4.8% 5311.33: client (128 days) 1.33(1.49+0.06) 1.29(1.37+0.12) -3.0% Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

trace2:data: pack-objects: add trace2 regions

2019-02-22T23:28:21Z

When studying the performance of 'git push' we would like to know how much time is spent at various parts of the command. One area that could cause performance trouble is 'git pack-objects'. Add trace2 regions around the three main actions taken in this command: 1. Enumerate objects. 2. Prepare pack. 3. Write pack-file. Signed-off-by: Derrick Stolee Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano