git/list-objects-filter-options.c, branch v2.36.2

pack-objects: lazily set up "struct rev_info", don't leak

2022-03-28T16:57:21Z

In the preceding [1] (pack-objects: move revs out of get_object_list(), 2022-03-22) the "repo_init_revisions()" was moved to cmd_pack_objects() so that it unconditionally took place for all invocations of "git pack-objects". We'd thus start leaking memory, which is easily reproduced in e.g. git.git by feeding e83c5163316 (Initial revision of "git", the information manager from hell, 2005-04-07) to "git pack-objects"; $ echo e83c5163316f89bfbde7d9ab23ca2e25604af290 | ./git pack-objects initial [...] ==19130==ERROR: LeakSanitizer: detected memory leaks Direct leak of 7120 byte(s) in 1 object(s) allocated from: #0 0x455308 in __interceptor_malloc (/home/avar/g/git/git+0x455308) #1 0x75b399 in do_xmalloc /home/avar/g/git/wrapper.c:41:8 #2 0x75b356 in xmalloc /home/avar/g/git/wrapper.c:62:9 #3 0x5d7609 in prep_parse_options /home/avar/g/git/diff.c:5647:2 #4 0x5d415a in repo_diff_setup /home/avar/g/git/diff.c:4621:2 #5 0x6dffbb in repo_init_revisions /home/avar/g/git/revision.c:1853:2 #6 0x4f599d in cmd_pack_objects /home/avar/g/git/builtin/pack-objects.c:3980:2 #7 0x4592ca in run_builtin /home/avar/g/git/git.c:465:11 #8 0x457d81 in handle_builtin /home/avar/g/git/git.c:718:3 #9 0x458ca5 in run_argv /home/avar/g/git/git.c:785:4 #10 0x457b40 in cmd_main /home/avar/g/git/git.c:916:19 #11 0x562259 in main /home/avar/g/git/common-main.c:56:11 #12 0x7fce792ac7ec in __libc_start_main csu/../csu/libc-start.c:332:16 #13 0x4300f9 in _start (/home/avar/g/git/git+0x4300f9) SUMMARY: LeakSanitizer: 7120 byte(s) leaked in 1 allocation(s). Aborted Narrowly fixing that commit would have been easy, just add call repo_init_revisions() right before get_object_list(), which is effectively what was done before that commit. But an unstated constraint when setting it up early is that it was needed for the subsequent [2] (pack-objects: parse --filter directly into revs.filter, 2022-03-22), i.e. we might have a --filter command-line option, and need to either have the "struct rev_info" setup when we encounter that option, or later. Let's just change the control flow so that we'll instead set up the "struct rev_info" only when we need it. Doing so leads to a bit more verbosity, but it's a lot clearer what we're doing and why. An earlier version of this commit[3] went behind opt_parse_list_objects_filter()'s back by faking up a "struct option" before calling it. Let's avoid that and instead create a blessed API for this pattern. We could furthermore combine the two get_object_list() invocations here by having repo_init_revisions() invoked on &pfd.revs, but I think clearly separating the two makes the flow clearer. Likewise redundantly but explicitly (i.e. redundant v.s. a "{ 0 }") "0" to "have_revs" early in cmd_pack_objects(). While we're at it add parentheses around the arguments to the OPT_* macros in in list-objects-filter-options.h, as we need to change those lines anyway. It doesn't matter in this case, but is good general practice. 1. https://lore.kernel.org/git/619b757d98465dbc4995bdc11a5282fbfcbd3daa.1647970119.git.gitgitgadget@gmail.com 2. https://lore.kernel.org/git/97de926904988b89b5663bd4c59c011a1723a8f5.1647970119.git.gitgitgadget@gmail.com 3. https://lore.kernel.org/git/patch-1.1-193534b0f07-20220325T121715Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Junio C Hamano

bundle: parse filter capability

2022-03-09T18:25:27Z

The v3 bundle format has capabilities, allowing newer versions of Git to create bundles with newer features. Older versions that do not understand these new capabilities will fail with a helpful warning. Create a new capability allowing Git to understand that the contained pack-file is filtered according to some object filter. Typically, this filter will be "blob:none" for a blobless partial clone. This change teaches Git to parse this capability, place its value in the bundle header, and demonstrate this understanding by adding a message to 'git bundle verify'. Since we will use gently_parse_list_objects_filter() outside of list-objects-filter-options.c, make it an external method and move its API documentation to before its declaration. Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

list-objects-filter-options: create copy helper

2022-03-09T18:25:26Z

As we add more embedded members with type 'struct list_objects_filter_options', it will be important to easily perform a deep copy across multiple such structs. Create list_objects_filter_copy() to satisfy this need. This method is recursive to match the recursive nature of the struct. Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

help: fix small typo in error message

2021-05-20T22:52:10Z

Classic string concatenation while forgetting a space character. Signed-off-by: Jean-Noël Avila Signed-off-by: Junio C Hamano

list-objects: implement object type filter

2021-04-19T21:09:11Z

While it already is possible to filter objects by some criteria in git-rev-list(1), it is not yet possible to filter out only a specific type of objects. This makes some filters less useful. The `blob:limit` filter for example filters blobs such that only those which are smaller than the given limit are returned. But it is unfit to ask only for these smallish blobs, given that git-rev-list(1) will continue to print tags, commits and trees. Now that we have the infrastructure in place to also filter tags and commits, we can improve this situation by implementing a new filter which selects objects based on their type. Above query can thus trivially be implemented with the following command: $ git rev-list --objects --filter=object:type=blob \ --filter=blob:limit=200 Furthermore, this filter allows to optimize for certain other cases: if for example only tags or commits have been selected, there is no need to walk down trees. The new filter is not yet supported in bitmaps. This is going to be implemented in a subsequent commit. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

list-objects-filter-options: fix function name in BUG

2020-11-16T22:28:25Z

Fix the function name we give in the BUG message. It's "config", not "choice". Signed-off-by: Martin Ågren Signed-off-by: Junio C Hamano

fetch: do not override partial clone filter

2020-09-28T23:11:59Z

When a fetch with the --filter argument is made, the configured default filter is set even if one already exists. This change was made in 5e46139376 ("builtin/fetch: remove unique promisor remote limitation", 2019-06-25) - in particular, changing from: * If this is the FIRST partial-fetch request, we enable partial * on this repo and remember the given filter-spec as the default * for subsequent fetches to this remote. to: * If this is a partial-fetch request, we enable partial on * this repo if not already enabled and remember the given * filter-spec as the default for subsequent fetches to this * remote. (The given filter-spec is "remembered" even if there is already an existing one.) This is problematic whenever a lazy fetch is made, because lazy fetches are made using "git fetch --filter=blob:none", but this will also happen if the user invokes "git fetch --filter=" manually. Therefore, restore the behavior prior to 5e46139376, which writes a filter-spec only if the current fetch request is the first partial-fetch one (for that remote). Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano

Merge branch 'tb/upload-pack-filters'

2020-08-12T01:04:13Z

The component to respond to "git fetch" request is made more configurable to selectively allow or reject object filtering specification used for partial cloning. * tb/upload-pack-filters: t5616: use test_i18ngrep for upload-pack errors upload-pack.c: introduce 'uploadpackfilter.tree.maxDepth' upload-pack.c: allow banning certain object filter(s) list_objects_filter_options: introduce 'list_object_filter_config_name'

list_objects_filter_options: introduce 'list_object_filter_config_name'

2020-08-04T01:03:24Z

In a subsequent commit, we will add configuration options that are specific to each kind of object filter, in which case it is handy to have a function that translates between 'enum list_objects_filter_choice' and an appropriate configuration-friendly string. Signed-off-by: Taylor Blau Acked-by: Jeff King Signed-off-by: Junio C Hamano

strvec: rename files from argv-array to strvec

2020-07-28T22:02:17Z

This requires updating #include lines across the code-base, but that's all fairly mechanical, and was done with: git ls-files '*.c' '*.h' | xargs perl -i -pe 's/argv-array.h/strvec.h/' Signed-off-by: Jeff King Signed-off-by: Junio C Hamano