git/reftable/system.h, branch jch

reftable/system: add abstraction to mmap files

2026-04-02T17:45:44Z

In our codebase we have a couple of wrappers around mmap(3p) that allow us to reimplement the syscall on platforms that don't have it natively, like for example Windows. Other projects that embed the reftable library may have a different infra though to hook up mmap wrappers, but these are currently hard to integrate. Provide the infrastructure to let projects easily define the mmap interface with a custom struct and custom functions. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/system: add abstraction to retrieve time in milliseconds

2026-04-02T17:45:43Z

We directly call gettimeofday(3p), which may not be available on some platforms. Provide the infrastructure to let projects easily use their own implementations of this function. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable: introduce "reftable-system.h" header

2026-04-02T17:45:43Z

We're including a couple of standard headers like in a bunch of locations, which makes it hard for a project to plug in their own logic for making required functionality available. For us this is for example via "compat/posix.h", which already includes all of the system headers relevant to us. Introduce a new "reftable-system.h" header that allows projects to provide their own headers. This new header is supposed to contain all the project-specific bits to provide the POSIX-like environment, and some additional supporting code. With this change, we thus have the following split in our system-specific code: - "reftable/reftable-system.h" is the project-specific header that provides a POSIX-like environment. Every project is expected to provide their own implementation. - "reftable/system.h" contains the project-independent definition of the interfaces that a project needs to implement. This file should not be touched by a project. - "reftable/system.c" contains the project-specific implementation of the interfaces defined in "system.h". Again, every project is expected to provide their own implementation. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable: don't second-guess errors from flock interface

2025-08-12T14:41:00Z

The `flock` interface is implemented as part of "reftable/system.c" and thus needs to be implemented by the integrator between the reftable library and its parent code base. As such, we cannot rely on any specific implementation thereof. Regardless of that, users of the `flock` subsystem rely on `errno` being set to specific values. This is fragile and not documented anywhere and doesn't really make for a good interface. Refactor the code so that the implementations themselves are expected to return reftable-specific error codes. Our implementation of the `flock` subsystem already knows to do this for all error paths except one. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'ps/reftable-api-revamp'

2025-04-29T21:21:30Z

Overhaul of the reftable API. * ps/reftable-api-revamp: reftable/table: move printing logic into test helper reftable/constants: make block types part of the public interface reftable/table: introduce iterator for table blocks reftable/table: add `reftable_table` to the public interface reftable/block: expose a generic iterator over reftable records reftable/block: make block iterators reseekable reftable/block: store block pointer in the block iterator reftable/block: create public interface for reading blocks git-zlib: use `struct z_stream_s` instead of typedef reftable/block: rename `block_reader` to `reftable_block` reftable/block: rename `block` to `block_data` reftable/table: move reading block into block reader reftable/block: simplify how we track restart points reftable/blocksource: consolidate code into a single file reftable/reader: rename data structure to "table" reftable: fix formatting of the license header

reftable: fix formatting of the license header

2025-04-07T21:53:09Z

The license headers used across the reftable library doesn't follow our typical coding style for multi-line comments. Fix it. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable: ignore file-in-use errors when unlink(3p) fails on Windows

2025-02-18T22:29:18Z

Unlinking a file may fail on Windows systems when the file is still held open by another process. This is incompatible with POSIX semantics and by extension with Git's assumed semantics when unlinking files, which is that files can be unlinked regardless of whether they are still open or not. To counteract this incompatibility, we have some custom error handling in the `mingw_unlink()` wrapper that first retries the deletion with some delay, and then asks the user whether we should continue to retry. While this logic might be sensible in many callsites throughout Git, it is less when used in the reftable library. We only use unlink(3) there to delete tables which aren't referenced anymore, and the code is very aware of the limitations on Windows. As such, all calls to unlink(3p) don't perform any error checking at all and are fine with the call failing. Instead, the library provides the `reftable_stack_clean()` function, which Git knows to execute in git-pack-refs(1) after compacting a stack. The effect of this function is that all stale tables will eventually get deleted once they aren't kept open anymore. So while we're fine with unlink(3p) failing, the Windows-emulation of that function will still perform several sleeps and ultimately end up asking the user: $ git pack-refs Unlink of file 'C:/temp/jgittest/jgit/.git/reftable/0x000000000002-0x000000000004-50486d0e.ref' failed. Should I try again? (y/n) n Unlink of file 'C:/temp/jgittest/jgit/.git/reftable/0x000000000002-0x000000000004-50486d0e.ref' failed. Should I try again? (y/n) n Unlink of file 'C:/temp/jgittest/jgit/.git/reftable/0x000000000002-0x000000000004-50486d0e.ref' failed. Should I try again? (y/n) n It even asks multiple times, which is doubly annoying and puzzling to the user: 1. It asks when trying to delete the old file after having written the compacted stack. 2. It asks when reloading the stack, where it will try to unlink now-unreferenced tables. 3. It asks when calling `reftable_stack_clean()`, where it will try to unlink now-stale tables. Fix the issue by making it possible to disable this behaviour with a preprocessor define. As "git-compat-util.h" is only included from "system.h", and given that "system.h" is only ever included by headers and code that are internal to the reftable library, we can set that macro in this header without impacting anything else but the reftable library. Reported-by: Christian Reich Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable: decouple from Git codebase by pulling in "compat/posix.h"

2025-02-18T18:55:41Z

The reftable library includes "git-compat-util.h" in order to get a POSIX-like programming environment that papers over various differences between platforms. The header also brings with it a couple of helpers specific to the Git codebase though, and over time we have started to use these helpers in the reftable library, as well. This makes it very hard to use the reftable library as a standalone library without the rest of the Git codebase, so other libraries like e.g. libgit2 cannot easily use it. But now that we have removed all calls to Git-specific functionality and have split out "compat/posix.h" as a separate header we can address this. Stop including "git-compat-util.h" and instead include "compat/posix.h" to finalize the decoupling of the reftable library from the rest of the Git codebase. The only bits which remain specific to Git are "system.h" and "system.c", which projects will have to provide. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/system: introduce `reftable_rand()`

2025-02-18T18:55:38Z

Introduce a new system-level `reftable_rand()` function that generates a single unsigned integer for us. The implementation of this function is to be provided by the calling codebase, which allows us to more easily hook into pre-seeded random number generators. Adapt the two callsites where we generated random data. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'ps/zlib-ng'

2025-02-06T22:56:45Z

The code paths to interact with zlib has been cleaned up in preparation for building with zlib-ng. * ps/zlib-ng: ci: make "linux-musl" job use zlib-ng ci: switch linux-musl to use Meson compat/zlib: allow use of zlib-ng as backend git-zlib: cast away potential constness of `next_in` pointer compat/zlib: provide stubs for `deflateSetHeader()` compat/zlib: provide `deflateBound()` shim centrally git-compat-util: move include of "compat/zlib.h" into "git-zlib.h" compat: introduce new "zlib.h" header git-compat-util: drop `z_const` define compat: drop `uncompress2()` compatibility shim