git/reftable, branch jch

reftable/system: add abstraction to mmap files

2026-04-02T17:45:44Z

In our codebase we have a couple of wrappers around mmap(3p) that allow us to reimplement the syscall on platforms that don't have it natively, like for example Windows. Other projects that embed the reftable library may have a different infra though to hook up mmap wrappers, but these are currently hard to integrate. Provide the infrastructure to let projects easily define the mmap interface with a custom struct and custom functions. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/system: add abstraction to retrieve time in milliseconds

2026-04-02T17:45:43Z

We directly call gettimeofday(3p), which may not be available on some platforms. Provide the infrastructure to let projects easily use their own implementations of this function. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/fsck: use REFTABLE_UNUSED instead of UNUSED

2026-04-02T17:45:43Z

While we have the reftable-specific `REFTABLE_UNUSED` header, we accidentally introduced a new usage of the Git-specific `UNUSED` header into the reftable library in 9051638519 (reftable: add code to facilitate consistency checks, 2025-10-07). Convert the site to use `REFTABLE_UNUSED`. Ideally, we'd move the definition of `UNUSED` into "git-compat-util.h" so that it becomes in accessible to the reftable library. But this is unfortunately not easily possible as "compat/mingw-posix.h" requires this macro, and this header is included by "compat/posix.h". Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/stack: provide fsync(3p) via system header

2026-04-02T17:45:43Z

Users of the reftable library are expected to provide their own function callback in cases they want to sync(3p) data to disk via the reftable write options. But if no such function was provided we end up calling fsync(3p) directly, which may not even be available on some systems. While dropping the explicit call to fsync(3p) would work, it would lead to an unsafe default behaviour where a project may have forgotten to set up the callback function, and that could lead to potential data loss. So this is not a great solution. Instead, drop the callback function and make it mandatory for the project to define fsync(3p). In the case of Git, we can then easily inject our custom implementation via the "reftable-system.h" header so that we continue to use `fsync_component()`. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable: introduce "reftable-system.h" header

2026-04-02T17:45:43Z

We're including a couple of standard headers like in a bunch of locations, which makes it hard for a project to plug in their own logic for making required functionality available. For us this is for example via "compat/posix.h", which already includes all of the system headers relevant to us. Introduce a new "reftable-system.h" header that allows projects to provide their own headers. This new header is supposed to contain all the project-specific bits to provide the POSIX-like environment, and some additional supporting code. With this change, we thus have the following split in our system-specific code: - "reftable/reftable-system.h" is the project-specific header that provides a POSIX-like environment. Every project is expected to provide their own implementation. - "reftable/system.h" contains the project-independent definition of the interfaces that a project needs to implement. This file should not be touched by a project. - "reftable/system.c" contains the project-specific implementation of the interfaces defined in "system.h". Again, every project is expected to provide their own implementation. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/stack: add function to check if optimization is required

2025-11-10T17:28:47Z

The reftable backend performs auto-compaction as part of its regular flow, which is required to keep the number of tables part of a stack at bay. This allows it to stay optimized. Compaction can also be triggered voluntarily by the user via the 'git pack-refs' or the 'git refs optimize' command. However, currently there is no way for the user to check if optimization is required without actually performing it. Extract out the heuristics logic from 'reftable_stack_auto_compact()' into an internal function 'update_segment_if_compaction_required()'. Then use this to add and expose `reftable_stack_compaction_required()` which will allow users to check if the reftable backend can be optimized. Signed-off-by: Karthik Nayak Acked-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/stack: return stack segments directly

2025-11-10T17:28:47Z

The `stack_table_sizes_for_compaction()` function returns individual sizes of each reftable table. This function is only called by `reftable_stack_auto_compact()` to decide which tables need to be compacted, if any. Modify the function to directly return the segments, which avoids the extra step of receiving the sizes only to pass it to `suggest_compaction_segment()`. A future commit will also add functionality for checking whether auto-compaction is necessary without performing it. This change allows code re-usability in that context. Signed-off-by: Karthik Nayak Acked-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'kn/reftable-consistency-checks'

2025-10-14T05:00:35Z

The reftable backend learned to sanity check its on-disk data more carefully. * kn/reftable-consistency-checks: refs/reftable: add fsck check for checking the table name reftable: add code to facilitate consistency checks fsck: order 'fsck_msg_type' alphabetically Documentation/fsck-msgids: remove duplicate msg id reftable: check for trailing newline in 'tables.list' refs: move consistency check msg to generic layer refs: remove unused headers

reftable: add code to facilitate consistency checks

2025-10-07T16:22:58Z

The `git refs verify` command is used to run consistency checks on the reference backends. This command is also invoked when users run 'git fsck'. While the files-backend has some fsck checks added, the reftable backend lacks such checks. Let's add the required infrastructure and a check to test for the files present in the reftable directory. Since the reftable library is treated as an independent library we should ensure that the library code works independently without knowledge about Git's internals. To do this, add both 'reftable/fsck.c' and 'reftable/reftable-fsck.h'. Which provide an entry point 'reftable_fsck_check' for running fsck checks over a provided reftable stack. The callee provides the function with callbacks to handle issue and information reporting. The added check, goes over all tables in the reftable stack validates that they have a valid name. It not, it raises an error. While here, move 'reftable/error.o' in the Makefile to retain lexicographic ordering. Signed-off-by: Karthik Nayak Signed-off-by: Junio C Hamano

reftable: check for trailing newline in 'tables.list'

2025-10-07T16:22:57Z

In the reftable format, the 'tables.list' file contains a newline separated list of tables. While we parse this file, we do not check or care about the last newline. Tighten the parser in `parse_names()` to return an appropriate error if the last newline is missing. This requires modification to `parse_names()` to now return the error while accepting the output as a third argument. Signed-off-by: Karthik Nayak Signed-off-by: Junio C Hamano