git/reftable, branch v2.50.0

reftable: make REFTABLE_UNUSED C99 compatible

2025-05-29T16:18:53Z

Since f93b2a0424 (reftable/basics: introduce `REFTABLE_UNUSED` annotation, 2025-02-18), the reftable library was migrated to use an internal version of `UNUSED`, which unconditionally sets a GNU __attribute__ to avoid warnings function parameters that are not being used. Make the definition conditional to prevent breaking the build with non GNU compilers. Reported-by: "Randall S. Becker" Signed-off-by: Carlo Marcelo Arenas Belón Signed-off-by: Junio C Hamano

Merge branch 'ps/reftable-read-block-perffix'

2025-05-19T23:02:48Z

Performance regression in not-yet-released code has been corrected. * ps/reftable-read-block-perffix: reftable: fix perf regression when reading blocks of unwanted type

Merge branch 'ly/reftable-writer-leakfix'

2025-05-19T23:02:47Z

Leakfix. * ly/reftable-writer-leakfix: reftable/writer: fix memory leak when `writer_index_hash()` fails reftable/writer: fix memory leak when `padded_write()` fails

reftable: fix perf regression when reading blocks of unwanted type

2025-05-12T17:55:24Z

In fd888311fbc (reftable/table: move reading block into block reader, 2025-04-07), we have refactored how reftable blocks are read so that most of the logic is contained in the "block.c" subsystem itself. Most importantly, the whole logic to read the data itself is now contained in that subsystem. This change caused a significant performance regression though when reading blocks that aren't of the specific type one is searching for: Benchmark 1: update-ref: create 100k refs (revision = fd888311fbc~) Time (mean ± σ): 2.171 s ± 0.028 s [User: 1.189 s, System: 0.977 s] Range (min … max): 2.117 s … 2.206 s 10 runs Benchmark 2: update-ref: create 100k refs (revision = fd888311fbc) Time (mean ± σ): 3.418 s ± 0.030 s [User: 2.371 s, System: 1.037 s] Range (min … max): 3.377 s … 3.473 s 10 runs Summary update-ref: create 100k refs (revision = fd888311fbc~) ran 1.57 ± 0.02 times faster than update-ref: create 100k refs (revision = fd888311fbc) The root caute of the performance regression is that we changed when exactly blocks of an uninteresting type are being discarded. Previous to the refactoring in the mentioned commit we'd load the block data, read its type, notice that it's not the wanted type and discard the block. After the commit though we don't discard the block immediately, but we fully decode it only to realize that it's not the desired type. We then discard the block again, but have already performed a bunch of pointless work. Fix the regression by making `reftable_block_init()` return early in case the block is not of the desired type. This fixes the performance hit: Benchmark 1: update-ref: create 100k refs (revision = HEAD~) Time (mean ± σ): 2.712 s ± 0.018 s [User: 1.990 s, System: 0.716 s] Range (min … max): 2.682 s … 2.741 s 10 runs Benchmark 2: update-ref: create 100k refs (revision = HEAD) Time (mean ± σ): 1.670 s ± 0.012 s [User: 0.991 s, System: 0.676 s] Range (min … max): 1.652 s … 1.693 s 10 runs Summary update-ref: create 100k refs (revision = HEAD) ran 1.62 ± 0.02 times faster than update-ref: create 100k refs (revision = HEAD~) Note that the baseline performance is lower than in the original due to a couple of unrelated performance improvements that have landed since the original commit. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/writer: fix memory leak when `writer_index_hash()` fails

2025-05-12T16:19:50Z

In reftable/writer.c:writer_index_hash(), if `reftable_buf_add` failed, key allocated by `reftable_malloc` will not be insert into `obj_index_tree` thus leaks. Simple add reftable_free(key) will solve this problem. Signed-off-by: Lidong Yan <502024330056@smail.nju.edu.cn> Signed-off-by: Junio C Hamano

reftable/writer: fix memory leak when `padded_write()` fails

2025-05-12T16:19:49Z

In reftable/writer.c:padded_write(), if w->writer failed, zeroed allocated in `reftable_calloc` will leak. w->writer could be `reftable_write_data` in reftable/stack.c, and could fail due to some write error. Simply add reftable_free(zeroed) will solve this problem. Signed-off-by: Lidong Yan <502024330056@smail.nju.edu.cn> Signed-off-by: Junio C Hamano

Merge branch 'ps/reftable-api-revamp'

2025-04-29T21:21:30Z

Overhaul of the reftable API. * ps/reftable-api-revamp: reftable/table: move printing logic into test helper reftable/constants: make block types part of the public interface reftable/table: introduce iterator for table blocks reftable/table: add `reftable_table` to the public interface reftable/block: expose a generic iterator over reftable records reftable/block: make block iterators reseekable reftable/block: store block pointer in the block iterator reftable/block: create public interface for reading blocks git-zlib: use `struct z_stream_s` instead of typedef reftable/block: rename `block_reader` to `reftable_block` reftable/block: rename `block` to `block_data` reftable/table: move reading block into block reader reftable/block: simplify how we track restart points reftable/blocksource: consolidate code into a single file reftable/reader: rename data structure to "table" reftable: fix formatting of the license header

Merge branch 'ps/reftable-windows-unlink-fix'

2025-04-15T20:50:13Z

Portability fix. * ps/reftable-windows-unlink-fix: reftable: ignore file-in-use errors when unlink(3p) fails on Windows

Merge branch 'ps/reftable-sans-compat-util'

2025-04-08T18:43:14Z

Make the code in reftable library less reliant on the service routines it used to borrow from Git proper, to make it easier to use by external users of the library. * ps/reftable-sans-compat-util: Makefile: skip reftable library for Coccinelle reftable: decouple from Git codebase by pulling in "compat/posix.h" git-compat-util.h: split out POSIX-emulating bits compat/mingw: split out POSIX-related bits reftable/basics: introduce `REFTABLE_UNUSED` annotation reftable/basics: stop using `SWAP()` macro reftable/stack: stop using `sleep_millisec()` reftable/system: introduce `reftable_rand()` reftable/reader: stop using `ARRAY_SIZE()` macro reftable/basics: provide wrappers for big endian conversion reftable/basics: stop using `st_mult()` in array allocators reftable: stop using `BUG()` in trivial cases reftable/record: don't `BUG()` in `reftable_record_cmp()` reftable/record: stop using `BUG()` in `reftable_record_init()` reftable/record: stop using `COPY_ARRAY()` reftable/blocksource: stop using `xmmap()` reftable/stack: stop using `write_in_full()` reftable/stack: stop using `read_in_full()`

reftable/table: move printing logic into test helper

2025-04-07T21:53:13Z

The logic to print individual blocks in a table is hosted in the reftable library. This is only the case due to historical reasons though because users of the library had no interfaces to read blocks one by one. Otherwise, printing individual blocks has no place in the reftable library given that the format will not be generic in the first place. We have now grown a public interface to iterate through blocks contained in a table, and thus we can finally move the logic to print them into the test helper. Move over the logic and refactor it accordingly. Note that the iterator also trivially allows us to access index sections, which we previously didn't print at all. This omission wasn't intentional though, so start dumping those sections as well so that we can assert that indices are written as expected. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano