git/reftable/record.c, branch v2.46.2

global: improve const correctness when assigning string constants

2024-06-07T17:30:48Z

We're about to enable `-Wwrite-strings`, which changes the type of string constants to `const char[]`. Fix various sites where we assign such constants to non-const variables. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/record: extract function to decode key lengths

2024-04-03T16:16:50Z

We're about to refactor the binary search over restart points so that it does not need to fully decode the record keys anymore. To do so we will need to decode the record key lengths, which is non-trivial logic. Extract the logic to decode these lengths from `refatble_decode_key()` so that we can reuse it. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'ps/reftable-reflog-iteration-perf'

2024-03-21T21:55:13Z

The code to iterate over reflogs in the reftable has been optimized to reduce memory allocation and deallocation. Reviewed-by: Josh Steadmon cf. * ps/reftable-reflog-iteration-perf: refs/reftable: track last log record name via strbuf reftable/record: use scratch buffer when decoding records reftable/record: reuse message when decoding log records reftable/record: reuse refnames when decoding log records reftable/record: avoid copying author info reftable/record: convert old and new object IDs to arrays refs/reftable: reload correct stack when creating reflog iter

Merge branch 'ps/reftable-block-search-fix'

2024-03-21T21:55:12Z

The reftable code has its own custom binary search function whose comparison callback has an unusual interface, which caused the binary search to degenerate into a linear search, which has been corrected. * ps/reftable-block-search-fix: reftable/block: fix binary search over restart counter reftable/record: fix memory leak when decoding object records

reftable/record: fix memory leak when decoding object records

2024-03-07T21:59:19Z

When decoding records it is customary to reuse a `struct reftable_ref_record` across calls. Thus, it may happen that the record already holds some allocated memory. When decoding ref and log records we handle this by releasing or reallocating held memory. But we fail to do this for object records, which causes us to leak memory. Fix this memory leak by releasing object records before we decode into them. We may eventually want to reuse memory instead to avoid needless reallocations. But for now, let's just plug the leak and be done. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/record: use scratch buffer when decoding records

2024-03-05T17:10:06Z

When decoding log records we need a temporary buffer to decode the reflog entry's name, mail address and message. As this buffer is local to the function we thus have to reallocate it for every single log record which we're about to decode, which is inefficient. Refactor the code such that callers need to pass in a scratch buffer, which allows us to reuse it for multiple decodes. This reduces the number of allocations when iterating through reflogs. Before: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 2,068,487 allocs, 2,068,365 frees, 305,122,946 bytes allocated After: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 1,068,485 allocs, 1,068,363 frees, 281,122,886 bytes allocated Note that this commit also drop some redundant calls to `strbuf_reset()` right before calling `decode_string()`. The latter already knows to reset the buffer, so there is no need for these. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/record: reuse message when decoding log records

2024-03-05T17:10:06Z

Same as the preceding commit we can allocate log messages as needed when decoding log records, thus further reducing the number of allocations. Before: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 3,068,488 allocs, 3,068,366 frees, 307,122,961 bytes allocated After: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 2,068,487 allocs, 2,068,365 frees, 305,122,946 bytes allocated Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/record: reuse refnames when decoding log records

2024-03-05T17:10:06Z

When decoding a log record we always reallocate their refname arrays. This results in quite a lot of needless allocation churn. Refactor the code to grow the array as required only. Like this, we should usually only end up reallocating the array a small handful of times when iterating over many refs. Before: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 4,068,487 allocs, 4,068,365 frees, 332,011,793 bytes allocated After: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 3,068,488 allocs, 3,068,366 frees, 307,122,961 bytes allocated Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/record: avoid copying author info

2024-03-05T17:10:06Z

Each reflog entry contains information regarding the authorship of who has made the change. This authorship information is not the same as that of any of the commits that the reflog entry references, but instead corresponds to the local user that has executed the command. Thus, it is almost always the case that all reflog entries have the same author. We can make use of this fact when decoding reftable records: instead of freeing and then reallocating the authorship information of log records, we can special-case when the next record during an iteration has the exact same authorship as the preceding record. If so, then there is no need to reallocate the respective fields. This change results in two allocations less per log record that we're iterating over in the most common case. Before: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 6,068,489 allocs, 6,068,367 frees, 361,011,822 bytes allocated After: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 4,068,487 allocs, 4,068,365 frees, 332,011,793 bytes allocated An alternative would be to store the capacity of both name and email and then use `REFTABLE_ALLOC_GROW()` to conditionally reallocate the array. But reftable records are copied around quite a lot, and thus we need to be a bit mindful of the overall record size. Furthermore, a memory comparison should also be more efficient than having to copy over memory even if we wouldn't have to allocate a new array every time. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/record: convert old and new object IDs to arrays

2024-03-05T17:10:06Z

In 7af607c58d (reftable/record: store "val1" hashes as static arrays, 2024-01-03) and b31e3cc620 (reftable/record: store "val2" hashes as static arrays, 2024-01-03) we have converted ref records to store their object IDs in a static array. Convert log records to do the same so that their old and new object IDs are arrays, too. This change results in two allocations less per log record that we're iterating over. Before: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 8,068,495 allocs, 8,068,373 frees, 401,011,862 bytes allocated After: HEAP SUMMARY: in use at exit: 13,473 bytes in 122 blocks total heap usage: 6,068,489 allocs, 6,068,367 frees, 361,011,822 bytes allocated Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano