git/convert.c, branch v2.4.4

Merge branch 'sp/stream-clean-filter'

2014-10-08T20:05:32Z

When running a required clean filter, we do not have to mmap the original before feeding the filter. Instead, stream the file contents directly to the filter and process its output. * sp/stream-clean-filter: sha1_file: don't convert off_t to size_t too early to avoid potential die() convert: stream from fd to required clean filter to reduce used address space copy_fd(): do not close the input file descriptor mmap_limit: introduce GIT_MMAP_LIMIT to allow testing expected mmap size memory_limit: use git_env_ulong() to parse GIT_ALLOC_LIMIT config.c: add git_env_ulong() to parse environment variable convert: drop arguments other than 'path' from would_convert_to_git()

convert: stream from fd to required clean filter to reduce used address space

2014-08-28T17:25:15Z

The data is streamed to the filter process anyway. Better avoid mapping the file if possible. This is especially useful if a clean filter reduces the size, for example if it computes a sha1 for binary data, like git media. The file size that the previous implementation could handle was limited by the available address space; large files for example could not be handled with (32-bit) msysgit. The new implementation can filter files of any size as long as the filter output is small enough. The new code path is only taken if the filter is required. The filter consumes data directly from the fd. If it fails, the original data is not immediately available. The condition can easily be handled as a fatal error, which is expected for a required filter anyway. If the filter was not required, the condition would need to be handled in a different way, like seeking to 0 and reading the data. But this would require more restructuring of the code and is probably not worth it. The obvious approach of falling back to reading all data would not help achieving the main purpose of this patch, which is to handle large files with limited address space. If reading all data is an option, we can simply take the old code path right away and mmap the entire file. The environment variable GIT_MMAP_LIMIT, which has been introduced in a previous commit is used to test that the expected code path is taken. A related test that exercises required filters is modified to verify that the data actually has been modified on its way from the file system to the object store. Signed-off-by: Steffen Prohaska Signed-off-by: Junio C Hamano

run-command: introduce CHILD_PROCESS_INIT

2014-08-20T16:53:37Z

Most struct child_process variables are cleared using memset first after declaration. Provide a macro, CHILD_PROCESS_INIT, that can be used to initialize them statically instead. That's shorter, doesn't require a function call and is slightly more readable (especially given that we already have STRBUF_INIT, ARGV_ARRAY_INIT etc.). Helped-by: Johannes Sixt Signed-off-by: Rene Scharfe Signed-off-by: Junio C Hamano

use skip_prefix to avoid magic numbers

2014-06-20T17:44:45Z

It's a common idiom to match a prefix and then skip past it with a magic number, like: if (starts_with(foo, "bar")) foo += 3; This is easy to get wrong, since you have to count the prefix string yourself, and there's no compiler check if the string changes. We can use skip_prefix to avoid the magic numbers here. Note that some of these conversions could be much shorter. For example: if (starts_with(arg, "--foo=")) { bar = arg + 6; continue; } could become: if (skip_prefix(arg, "--foo=", &bar)) continue; However, I have left it as: if (skip_prefix(arg, "--foo=", &v)) { bar = v; continue; } to visually match nearby cases which need to actually process the string. Like: if (skip_prefix(arg, "--foo=", &v)) { bar = atoi(v); continue; } Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

replace {pre,suf}fixcmp() with {starts,ends}_with()

2013-12-05T22:13:21Z

typofix: in-code comments

2013-07-22T23:06:49Z

Signed-off-by: Ondřej Bílka Signed-off-by: Junio C Hamano

Merge branch 'lf/read-blob-data-from-index'

2013-04-22T01:39:45Z

Reduce duplicated code between convert.c and attr.c. * lf/read-blob-data-from-index: convert.c: remove duplicate code read_blob_data_from_index(): optionally return the size of blob data attr.c: extract read_index_data() as read_blob_data_from_index()

convert.c: remove duplicate code

2013-04-17T16:52:33Z

The has_cr_in_index() function is an almost 1:1 copy of read_blob_data_from_index() with some additions. Use the latter instead of using copy-pasted code. Signed-off-by: Lukas Fleischer Signed-off-by: Junio C Hamano

convert some config callbacks to parse_config_key

2013-01-23T16:41:50Z

These callers can drop some inline pointer arithmetic and magic offset constants, making them more readable and less error-prone (those constants had to match the lengths of strings, but there is no automatic verification of that fact). The "ep" pointer (presumably for "end pointer"), which points to the final key segment of the config variable, is given the more standard name "key" to describe its function rather than its derivation. Signed-off-by: Jeff King Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano

Merge branch 'jb/required-filter'

2012-02-28T21:25:57Z

* jb/required-filter: Add a setting to require a filter to be successful Conflicts: convert.c