git/Documentation/technical, branch v2.12.2

Merge branch 'sb/in-core-index-doc'

2017-01-31T21:14:59Z

Documentation and in-code comments updates. * sb/in-core-index-doc: documentation: retire unfinished documentation cache.h: document add_[file_]to_index cache.h: document remove_index_entry_at cache.h: document index_name_pos

Merge branch 'jk/clear-delta-base-cache-fix'

2017-01-31T21:14:59Z

A crashing bug introduced in v2.11 timeframe has been found (it is triggerable only in fast-import) and fixed. * jk/clear-delta-base-cache-fix: clear_delta_base_cache(): don't modify hashmap while iterating

documentation: retire unfinished documentation

2017-01-19T20:18:43Z

When looking for documentation for a specific function, you may be tempted to run git -C Documentation grep index_name_pos only to find the file technical/api-in-core-index.txt, which doesn't help for understanding the given function. It would be better to not find these functions in the documentation, such that people directly dive into the code instead. In the previous patches we have documented * index_name_pos() * remove_index_entry_at() * add_[file_]to_index() in cache.h We already have documentation for: * add_index_entry() * read_index() Which leaves us with a TODO for: * cache -> the_index macros * refresh_index() * discard_index() * ie_match_stat() and ie_modified(); how they are different and when to use which. * write_index() that was renamed to write_locked_index * cache_tree_invalidate_path() * cache_tree_update() Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano

clear_delta_base_cache(): don't modify hashmap while iterating

2017-01-19T19:17:20Z

On Thu, Jan 19, 2017 at 03:03:46PM +0100, Ulrich Spörlein wrote: > > I suspect the patch below may fix things for you. It works around it by > > walking over the lru list (either is fine, as they both contain all > > entries, and since we're clearing everything, we don't care about the > > order). > > Confirmed. With the patch applied, I can import the whole 55G in one go > without any crashes or aborts. Thanks much! Thanks. Here it is rolled up with a commit message. -- >8 -- Subject: clear_delta_base_cache(): don't modify hashmap while iterating Removing entries while iterating causes fast-import to access an already-freed `struct packed_git`, leading to various confusing errors. What happens is that clear_delta_base_cache() drops the whole contents of the cache by iterating over the hashmap, calling release_delta_base_cache() on each entry. That function removes the item from the hashmap. The hashmap code may then shrink the table, but the hashmap_iter struct retains an offset from the old table. As a result, the next call to hashmap_iter_next() may claim that the iteration is done, even though some items haven't been visited. The only caller of clear_delta_base_cache() is fast-import, which wants to clear the cache because it is discarding the packed_git struct for its temporary pack. So by failing to remove all of the entries, we still have references to the freed packed_git. To make things even more confusing, this doesn't seem to trigger with the test suite, because it depends on complexities like the size of the hash table, which entries got cleared, whether we try to access them before they're evicted from the cache, etc. So I've been able to identify the problem with large imports like freebsd's svn import, or a fast-export of linux.git. But nothing that would be reasonable to run as part of the normal test suite. We can fix this easily by iterating over the lru linked list instead of the hashmap. They both contain the same entries, and we can use the "safe" variant of the list iterator, which exists for exactly this case. Let's also add a warning to the hashmap API documentation to reduce the chances of getting bit by this again. Reported-by: Ulrich Spörlein Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'bw/pathspec-cleanup'

2017-01-18T23:12:15Z

Code clean-up in the pathspec API. * bw/pathspec-cleanup: pathspec: rename prefix_pathspec to init_pathspec_item pathspec: small readability changes pathspec: create strip submodule slash helpers pathspec: create parse_element_magic helper pathspec: create parse_long_magic function pathspec: create parse_short_magic function pathspec: factor global magic into its own function pathspec: simpler logic to prefix original pathspec elements pathspec: always show mnemonic and name in unsupported_magic pathspec: remove unused variable from unsupported_magic pathspec: copy and free owned memory pathspec: remove the deprecated get_pathspec function ls-tree: convert show_recursive to use the pathspec struct interface dir: convert fill_directory to use the pathspec struct interface dir: remove struct path_simplify mv: remove use of deprecated 'get_pathspec()'

pathspec: remove the deprecated get_pathspec function

2017-01-09T02:04:17Z

Now that all callers of the old 'get_pathspec' interface have been migrated to use the new pathspec struct interface it can be removed from the codebase. Since there are no more users of the '_raw' field in the pathspec struct it can also be removed. This patch also removes the old functionality of modifying the const char **argv array that was passed into parse_pathspec. Instead the constructed 'match' string (which is a pathspec element with the prefix prepended) is only stored in its corresponding pathspec_item entry. Signed-off-by: Brandon Williams Reviewed-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano

submodule-config: clarify parsing of null_sha1 element

2016-11-22T22:43:04Z

Signed-off-by: Stefan Beller Signed-off-by: Junio C Hamano Reviewed-by: Brandon Williams Signed-off-by: Junio C Hamano

submodule-config: rename commit_sha1 to treeish_name

2016-11-22T22:43:03Z

It is also possible to pass in any treeish name to lookup a submodule config. Make it clear by naming the variables accordingly. Looking up a submodule config by tree hash will come in handy in a later patch. Signed-off-by: Stefan Beller Reviewed-by: Brandon Williams Signed-off-by: Junio C Hamano

Merge branch 'nd/shallow-deepen'

2016-10-10T21:03:50Z

The existing "git fetch --depth=" option was hard to use correctly when making the history of an existing shallow clone deeper. A new option, "--deepen=", has been added to make this easier to use. "git clone" also learned "--shallow-since=" and "--shallow-exclude=" options to make it easier to specify "I am interested only in the recent N months worth of history" and "Give me only the history since that version". * nd/shallow-deepen: (27 commits) fetch, upload-pack: --deepen=N extends shallow boundary by N commits upload-pack: add get_reachable_list() upload-pack: split check_unreachable() in two, prep for get_reachable_list() t5500, t5539: tests for shallow depth excluding a ref clone: define shallow clone boundary with --shallow-exclude fetch: define shallow boundary with --shallow-exclude upload-pack: support define shallow boundary by excluding revisions refs: add expand_ref() t5500, t5539: tests for shallow depth since a specific date clone: define shallow clone boundary based on time with --shallow-since fetch: define shallow boundary with --shallow-since upload-pack: add deepen-since to cut shallow repos based on time shallow.c: implement a generic shallow boundary finder based on rev-list fetch-pack: use a separate flag for fetch in deepening mode fetch-pack.c: mark strings for translating fetch-pack: use a common function for verbose printing fetch-pack: use skip_prefix() instead of starts_with() upload-pack: move rev-list code out of check_non_tip() upload-pack: make check_non_tip() clean things up on error upload-pack: tighten number parsing at "deepen" lines ...

sha1_array: let callbacks interrupt iteration

2016-09-26T18:46:41Z

The callbacks for iterating a sha1_array must have a void return. This is unlike our usual for_each semantics, where a callback may interrupt iteration and have its value propagated. Let's switch it to the usual form, which will enable its use in more places (e.g., where we are replacing an existing iteration with a different data structure). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano