git/reftable, branch v2.47.2

Merge branch 'ak/typofix-2.46-maint'

2024-10-04T21:21:40Z

Typofixes. * ak/typofix-2.46-maint: perl: fix a typo mergetool: fix a typo reftable: fix a typo trace2: fix typos

reftable: fix a typo

2024-10-03T19:06:51Z

Fix a typo in comments. Signed-off-by: Andrew Kreimer Signed-off-by: Junio C Hamano

Merge branch 'ps/reftable-concurrent-writes'

2024-09-30T23:16:14Z

Give timeout to the locking code to write to reftable. * ps/reftable-concurrent-writes: refs/reftable: reload locked stack when preparing transaction reftable/stack: allow locking of outdated stacks refs/reftable: introduce "reftable.lockTimeout"

reftable/stack: allow locking of outdated stacks

2024-09-24T16:45:25Z

In `reftable_stack_new_addition()` we first lock the stack and then check whether it is still up-to-date. If it is not we return an error to the caller indicating that the stack is outdated. This is overly restrictive in our ref transaction interface though: we lock the stack right before we start to verify the transaction, so we do not really care whether it is outdated or not. What we really want is that the stack is up-to-date after it has been locked so that we can verify queued updates against its current state while we know that it is locked for concurrent modification. Introduce a new flag `REFTABLE_STACK_NEW_ADDITION_RELOAD` that alters the behaviour of `reftable_stack_init_addition()` in this case: when we notice that it is out-of-date we reload it instead of returning an error to the caller. This logic will be wired up in the reftable backend in the next commit. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

refs/reftable: introduce "reftable.lockTimeout"

2024-09-24T16:45:25Z

When multiple concurrent processes try to update references in a repository they may try to lock the same lockfiles. This can happen even when the updates are non-conflicting and can both be applied, so it doesn't always make sense to abort the transaction immediately. Both the "loose" and "packed" backends thus have a grace period that they wait for the lock to be released that can be controlled via the config values "core.filesRefLockTimeout" and "core.packedRefsTimeout", respectively. The reftable backend doesn't have such a setting yet and instead fails immediately when it sees such a lock. But the exact same concepts apply here as they do apply to the other backends. Introduce a new "reftable.lockTimeout" config that controls how long we may wait for a "tables.list" lock to be released. The default value of this config is 100ms, which is the same default as we have it for the "loose" backend. Note that even though we also lock individual tables, this config really only applies to the "tables.list" file. This is because individual tables are only ever locked when we already hold the "tables.list" lock during compaction. When we observe such a lock we in fact do not want to compact the table at all because it is already in the process of being compacted by a concurrent process. So applying the same timeout here would not make any sense and only delay progress. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

reftable/reader: make table iterator reseekable

2024-09-16T20:57:19Z

In 67ce50ba26 (Merge branch 'ps/reftable-reusable-iterator', 2024-05-30) we have refactored the interface of reftable iterators such that they can be reused in theory. This patch series only landed the required changes on the interface level, but didn't yet implement the actual logic to make iterators reusable. As it turns out almost all of the infrastructure already does support re-seeking. The only exception is the table iterator, which does not reset its `is_finished` bit. Do so and add a couple of tests that verify that we can re-seek iterators. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

t: move reftable/stack_test.c to the unit testing framework

2024-09-08T20:24:03Z

reftable/stack_test.c exercises the functions defined in reftable/stack.{c, h}. Migrate reftable/stack_test.c to the unit testing framework. Migration involves refactoring the tests to use the unit testing framework instead of reftable's test framework and renaming the tests to be in-line with unit-tests' standards. Since some of the tests use set_test_hash() defined by reftable/test_framework.{c, h} but these files are not '#included' in the test file, copy this function in the ported test file. With the migration of stack test to the unit-tests framework, "test-tool reftable" becomes a no-op. Hence, get rid of everything that uses "test-tool reftable" alongside everything that is used to implement it. While at it, alphabetically sort the cmds[] list in helper/test-tool.c by moving the entry for "dump-reftable". Mentored-by: Patrick Steinhardt Mentored-by: Christian Couder Signed-off-by: Chandra Pratap Signed-off-by: Junio C Hamano

Merge branch 'ps/reftable-concurrent-compaction'

2024-09-03T16:15:03Z

The code path for compacting reftable files saw some bugfixes against concurrent operation. * ps/reftable-concurrent-compaction: reftable/stack: fix segfault when reload with reused readers fails reftable/stack: reorder swapping in the reloaded stack contents reftable/reader: keep readers alive during iteration reftable/reader: introduce refcounting reftable/stack: fix broken refnames in `write_n_ref_tables()` reftable/reader: inline `reader_close()` reftable/reader: inline `init_reader()` reftable/reader: rename `reftable_new_reader()` reftable/stack: inline `stack_compact_range_stats()` reftable/blocksource: drop malloc block source

Merge branch 'cp/unit-test-reftable-block'

2024-08-29T18:08:16Z

Another test for reftable library ported to the unit test framework. * cp/unit-test-reftable-block: t-reftable-block: mark unused argv/argc t-reftable-block: add tests for index blocks t-reftable-block: add tests for obj blocks t-reftable-block: add tests for log blocks t-reftable-block: remove unnecessary variable 'j' t-reftable-block: use xstrfmt() instead of xstrdup() t-reftable-block: use block_iter_reset() instead of block_iter_close() t-reftable-block: use reftable_record_key() instead of strbuf_addstr() t-reftable-block: use reftable_record_equal() instead of check_str() t-reftable-block: release used block reader t: harmonize t-reftable-block.c with coding guidelines t: move reftable/block_test.c to the unit testing framework

Merge branch 'ps/reftable-drop-generic'

2024-08-29T18:08:16Z

The code in the reftable library has been cleaned up by discarding unused "generic" interface. * ps/reftable-drop-generic: reftable: mark unused parameters in empty iterator functions reftable/generic: drop interface t/helper: refactor to not use `struct reftable_table` t/helper: use `hash_to_hex_algop()` to print hashes t/helper: inline printing of reftable records t/helper: inline `reftable_table_print()` t/helper: inline `reftable_stack_print_directory()` t/helper: inline `reftable_reader_print_file()` t/helper: inline `reftable_dump_main()` reftable/dump: drop unused `compact_stack()` reftable/generic: move generic iterator code into iterator interface reftable/iter: drop double-checking logic reftable/stack: open-code reading refs reftable/merged: stop using generic tables in the merged table reftable/merged: rename `reftable_new_merged_table()` reftable/merged: expose functions to initialize iterators