git/packfile.c, branch v2.34.2

Merge branch 'rs/close-pack-leakfix'

2021-10-04T04:49:20Z

Leakfix. * rs/close-pack-leakfix: packfile: release bad_objects in close_pack()

packfile: release bad_objects in close_pack()

2021-09-24T16:22:46Z

Unusable entries of a damaged pack file are recorded in the oidset bad_objects. Release it when we're done with the pack. This doesn't affect intact packs because an empty oidset requires no allocation. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

Merge branch 'rs/packfile-bad-object-list-in-oidset'

2021-09-23T20:44:46Z

Replace a handcrafted data structure used to keep track of bad objects in the packfile API by an oidset. * rs/packfile-bad-object-list-in-oidset: packfile: use oidset for bad objects packfile: convert has_packed_and_bad() to object_id packfile: convert mark_bad_packed_object() to object_id midx: inline nth_midxed_pack_entry() oidset: make oidset_size() an inline function

Merge branch 'tb/multi-pack-bitmaps'

2021-09-20T22:20:39Z

The reachability bitmap file used to be generated only for a single pack, but now we've learned to generate bitmaps for history that span across multiple packfiles. * tb/multi-pack-bitmaps: (29 commits) pack-bitmap: drop bitmap_index argument from try_partial_reuse() pack-bitmap: drop repository argument from prepare_midx_bitmap_git() p5326: perf tests for MIDX bitmaps p5310: extract full and partial bitmap tests midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' t7700: update to work with MIDX bitmap test knob t5319: don't write MIDX bitmaps in t5319 t5310: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t0410: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t5326: test multi-pack bitmap behavior t/helper/test-read-midx.c: add --checksum mode t5310: move some tests to lib-bitmap.sh pack-bitmap: write multi-pack bitmaps pack-bitmap: read multi-pack bitmaps pack-bitmap.c: avoid redundant calls to try_partial_reuse pack-bitmap.c: introduce 'bitmap_is_preferred_refname()' pack-bitmap.c: introduce 'nth_bitmap_object_oid()' pack-bitmap.c: introduce 'bitmap_num_objects()' midx: avoid opening multiple MIDXs when writing midx: close linked MIDXs, avoid leaking memory ...

packfile: use oidset for bad objects

2021-09-12T23:14:32Z

Store the object ID of broken pack entries in an oidset instead of keeping only their hashes in an unsorted array. The resulting code is shorter and easier to read. It also handles the (hopefully) very rare case of having a high number of bad objects better. Helped-by: Jeff King Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

packfile: convert has_packed_and_bad() to object_id

2021-09-12T23:14:32Z

The single caller has a full object ID, so pass it on instead of just its hash member. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

packfile: convert mark_bad_packed_object() to object_id

2021-09-12T23:14:32Z

All callers have full object IDs, so pass them on instead of just their hash member. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

pack-bitmap: read multi-pack bitmaps

2021-09-01T20:56:43Z

This prepares the code in pack-bitmap to interpret the new multi-pack bitmaps described in Documentation/technical/bitmap-format.txt, which mostly involves converting bit positions to accommodate looking them up in a MIDX. Note that there are currently no writers who write multi-pack bitmaps, and that this will be implemented in the subsequent commit. Note also that get_midx_checksum() and get_midx_filename() are made non-static so they can be called from pack-bitmap.c. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

object-store.h: teach for_each_packed_object to ignore kept packs

2021-08-30T06:23:39Z

The next patch will reimplement a function that wants to iterate over packed objects while ignoring packs which are marked as kept (either in-core or on-disk). Teach for_each_packed_object() to ignore all objects from those packs by adding a new flag for each of the "kept" states that a pack can be in. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

xmmap: inform Linux users of tuning knobs on ENOMEM

2021-06-30T06:14:25Z

Linux users may benefit from additional information on how to avoid ENOMEM from mmap despite the system having enough RAM to accomodate them. We can't reliably unmap pack windows to work around the issue since malloc and other library routines may mmap without our knowledge. Signed-off-by: Eric Wong Signed-off-by: Junio C Hamano