git/Documentation/technical, branch v2.47.2

docs: fix the `maintain-git` links in `technical/platform-support`

2024-10-07T22:34:16Z

These links should point to `.html` files, not to `.txt` ones. Compare also to 4945f046c7f (api docs: link to html version of api-trace2, 2022-09-16). Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

Documentation/technical: fix a typo

2024-09-23T19:40:52Z

Fix a typo in documentation. Signed-off-by: Andrew Kreimer Signed-off-by: Junio C Hamano

Merge branch 'ps/clar-unit-test'

2024-09-19T01:02:05Z

Import clar unit tests framework libgit2 folks invented for our use. * ps/clar-unit-test: Makefile: rename clar-related variables to avoid confusion clar: add CMake support t/unit-tests: convert ctype tests to use clar t/unit-tests: convert strvec tests to use clar t/unit-tests: implement test driver Makefile: wire up the clar unit testing framework Makefile: do not use sparse on third-party sources Makefile: make hdr-check depend on generated headers Makefile: fix sparse dependency on GENERATED_H clar: stop including `shellapi.h` unnecessarily clar(win32): avoid compile error due to unused `fs_copy()` clar: avoid compile error with mingw-w64 t/clar: fix compatibility with NonStop t: import the clar unit testing framework t: do not pass GIT_TEST_OPTS to unit tests with prove

t: import the clar unit testing framework

2024-09-04T15:41:36Z

Our unit testing framework is a homegrown solution. While it supports most of our needs, it is likely that the volume of unit tests will grow quite a bit in the future such that we can exercise low-level subsystems directly. This surfaces several shortcomings that the current solution has: - There is no way to run only one specific tests. While some of our unit tests wire this up manually, others don't. In general, it requires quite a bit of boilerplate to get this set up correctly. - Failures do not cause a test to stop execution directly. Instead, the test author needs to return manually whenever an assertion fails. This is rather verbose and is not done correctly in most of our unit tests. - Wiring up a new testcase requires both implementing the test function and calling it in the respective test suite's main function, which is creating code duplication. We can of course fix all of these issues ourselves, but that feels rather pointless when there are already so many unit testing frameworks out there that have those features. We line out some requirements for any unit testing framework in "Documentation/technical/unit-tests.txt". The "clar" unit testing framework, which isn't listed in that table yet, ticks many of the boxes: - It is licensed under ISC, which is compatible. - It is easily vendorable because it is rather tiny at around 1200 lines of code. - It is easily hackable due to the same reason. - It has TAP support. - It has skippable tests. - It preprocesses test files in order to extract test functions, which then get wired up automatically. While it's not perfect, the fact that clar originates from the libgit2 project means that it should be rather easy for us to collaborate with upstream to plug any gaps. Import the clar unit testing framework at commit 1516124 (Merge pull request #97 from pks-t/pks-whitespace-fixes, 2024-08-15). The framework will be wired up in subsequent commits. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

trace2: implement trace2_printf() for event target

2024-08-22T22:02:31Z

The trace2 event target does not have an implementation for trace2_printf(). While the event target is for structured events, and trace2_printf() is for unstructured, human-readable messages, it may still be useful to wrap these unstructured messages in a structured JSON object. Among other things, it may reduce confusion when manually debugging using event trace data. Add a simple implementation for the event target that wraps trace2_printf() messages in a minimal JSON object. Document this in Documentation/technical/api-trace2.txt, and bump the event format version since we're adding a new event type. Signed-off-by: Josh Steadmon Signed-off-by: Junio C Hamano

Merge branch 'tb/incremental-midx-part-1'

2024-08-19T18:07:37Z

Incremental updates of multi-pack index files. * tb/incremental-midx-part-1: midx: implement support for writing incremental MIDX chains t/t5313-pack-bounds-checks.sh: prepare for sub-directories t: retire 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' midx: implement verification support for incremental MIDXs midx: support reading incremental MIDX chains midx: teach `midx_fanout_add_midx_fanout()` about incremental MIDXs midx: teach `midx_preferred_pack()` about incremental MIDXs midx: teach `midx_contains_pack()` about incremental MIDXs midx: remove unused `midx_locate_pack()` midx: teach `fill_midx_entry()` about incremental MIDXs midx: teach `nth_midxed_offset()` about incremental MIDXs midx: teach `bsearch_midx()` about incremental MIDXs midx: introduce `bsearch_one_midx()` midx: teach `nth_bitmapped_pack()` about incremental MIDXs midx: teach `nth_midxed_object_oid()` about incremental MIDXs midx: teach `prepare_midx_pack()` about incremental MIDXs midx: teach `nth_midxed_pack_int_id()` about incremental MIDXs midx: add new fields for incremental MIDX chains Documentation: describe incremental MIDX format

Documentation: describe incremental MIDX format

2024-08-06T19:01:35Z

Prepare to implement incremental multi-pack indexes (MIDXs) over the next several commits by first describing the relevant prerequisites (like a new chunk in the MIDX format, the directory structure for incremental MIDXs, etc.) The format is described in detail in the patch contents below, but the high-level description is as follows. Incremental MIDXs live in $GIT_DIR/objects/pack/multi-pack-index.d, and each `*.midx` within that directory has a single "parent" MIDX, which is the MIDX layer immediately before it in the MIDX chain. The chain order resides in a file 'multi-pack-index-chain' in the same directory. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

Documentation: add platform support policy

2024-08-02T23:27:15Z

Supporting many platforms is only possible when we have the right tools to ensure that support. Teach platform maintainers how they can help us to help them, by explaining what kind of tooling support we would like to have, and what level of support becomes available as a result. Provide examples so that platform maintainers can see what we're asking for in practice. With this policy in place, we can make changes with stronger assurance that we are not breaking anybody we promised not to. Instead, we can feel confident that our existing testing and integration practices protect those who care from breakage. Signed-off-by: Emily Shaffer Signed-off-by: Junio C Hamano

Documentation/technical/bitmap-format.txt: add missing position table

2024-06-14T21:19:26Z

While investigating a benign Coverity warning on the new pseudo-merge implementation, I was struggling to understand the (paraphrased) below: ofs = index_end - 24 - (index->pseudo_merges.nr * sizeof(uint64_t)); for (i = 0; i < index->pseudo_merges.nr; i++) { index->pseudo_merges.v[i].at = get_be64(ofs); ofs += sizeof(uint64_t); } , in pack-bitmap.c::load_bitmap_header(). Looking at the documentation, the diagram describing the on-disk format (prior to this patch) suggested that the optional extended lookup table immediately preceded the trailing metadata portion. If that were the case, that would make the above code from load_bitmap_header() incorrect, as we'd be blindly reading into the extended offset table. But later on in the documentation there is a description of the pseudo-merge position table as immediately preceding the trailing metadata portion of the extension. And indeed, we do write the position table in pack-bitmap-write.c: /* write positions for all pseudo merges */ for (i = 0; i < writer->pseudo_merges_nr; i++) hashwrite_be64(f, pseudo_merge_ofs[i]); hashwrite_be32(f, writer->pseudo_merges_nr); hashwrite_be32(f, kh_size(writer->pseudo_merge_commits)); hashwrite_be64(f, table_start - start); hashwrite_be64(f, hashfile_total(f) - start + sizeof(uint64_t)); So this is purely a case of the diagram being out of sync with the textual description and actual implementation of the format specification. Add the missing component back to the format diagram to avoid further confusion in this area. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

Documentation/technical: describe pseudo-merge bitmaps format

2024-05-24T18:40:41Z

Prepare to implement pseudo-merge bitmaps over the next several commits by first describing the serialization format which will store the new pseudo-merge bitmaps themselves. This format is implemented as an optional extension within the bitmap v1 format, making it compatible with previous versions of Git, as well as the original .bitmap implementation within JGit. The format is described in detail in the patch contents below, but the high-level description is as follows: - An array of pseudo-merge bitmaps, each containing a pair of EWAH bitmaps: one describing the set of pseudo-merge "parents", and another describing the set of object(s) reachable from those parents. - A lookup table to determine which pseudo-merge(s) a given commit appears in. An optional extended lookup table follows when there is at least one commit which appears in multiple pseudo-merge groups. - Trailing metadata, including the number of pseudo-merge(s), number of unique parents, the offset within the .bitmap file for the pseudo-merge commit lookup table, and the size of the optional extension itself. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano