git/git.c, branch v2.32.2

Merge branch 'ps/config-env-option-with-separate-value'

2021-05-07T03:47:37Z

"git --config-env var=val cmd" weren't accepted (only --config-env=var=val was). * ps/config-env-option-with-separate-value: git: support separate arg for `--config-env`'s value git.txt: fix synopsis of `--config-env` missing the equals sign

Merge branch 'mt/parallel-checkout-part-2'

2021-04-30T04:50:26Z

The checkout machinery has been taught to perform the actual write-out of the files in parallel when able. * mt/parallel-checkout-part-2: parallel-checkout: add design documentation parallel-checkout: support progress displaying parallel-checkout: add configuration options parallel-checkout: make it truly parallel unpack-trees: add basic support for parallel checkout

git: support separate arg for `--config-env`'s value

2021-04-30T00:46:53Z

While not documented as such, many of the top-level options like `--git-dir` and `--work-tree` support two syntaxes: they accept both an equals sign between option and its value, and they do support option and value as two separate arguments. The recently added `--config-env` option only supports the syntax with an equals sign. Mitigate this inconsistency by accepting both syntaxes and add tests to verify both work. Signed-off-by: Patrick Steinhardt Reviewed-by: Jeff King Signed-off-by: Junio C Hamano

parallel-checkout: make it truly parallel

2021-04-19T18:57:05Z

Use multiple worker processes to distribute the queued entries and call write_pc_item() in parallel for them. The items are distributed uniformly in contiguous chunks. This minimizes the chances of two workers writing to the same directory simultaneously, which could affect performance due to lock contention in the kernel. Work stealing (or any other format of re-distribution) is not implemented yet. The protocol between the main process and the workers is quite simple. They exchange binary messages packed in pkt-line format, and use PKT-FLUSH to mark the end of input (from both sides). The main process starts the communication by sending N pkt-lines, each corresponding to an item that needs to be written. These packets contain all the necessary information to load, smudge, and write the blob associated with each item. Then it waits for the worker to send back N pkt-lines containing the results for each item. The resulting packet must contain: the identification number of the item that it refers to, the status of the operation, and the lstat() data gathered after writing the file (iff the operation was successful). For now, checkout always uses a hardcoded value of 2 workers, only to demonstrate that the parallel checkout framework correctly divides and writes the queued entries. The next patch will add user configurations and define a more reasonable default, based on tests with the said settings. Co-authored-by: Nguyễn Thái Ngọc Duy Co-authored-by: Jeff Hostetler Signed-off-by: Matheus Tavares Signed-off-by: Junio C Hamano

Merge branch 'tb/precompose-prefix-simplify'

2021-04-13T22:28:51Z

Streamline the codepath to fix the UTF-8 encoding issues in the argv[] and the prefix on macOS. * tb/precompose-prefix-simplify: macOS: precompose startup_info->prefix precompose_utf8: make precompose_string_if_needed() public

macOS: precompose startup_info->prefix

2021-04-06T00:30:36Z

The "prefix" was precomposed for macOS in commit 5c327502 (MacOS: precompose_argv_prefix(), 2021-02-03). However, this commit forgot to update "startup_info->prefix" after precomposing. Move the (possible) precomposition towards the end of setup_git_directory_gently(), so that precompose_string_if_needed() can use git_config_get_bool("core.precomposeunicode") correctly. Keep prefix, startup_info->prefix and GIT_PREFIX_ENVIRONMENT all in sync. And as a result, the prefix no longer needs to be precomposed in git.c Reported-by: Dmitry Torilov Helped-by: Junio C Hamano Signed-off-by: Torsten Bögershausen Signed-off-by: Junio C Hamano

Merge branch 'tb/precompose-prefix-too'

2021-02-12T22:21:04Z

When commands are started from a subdirectory, they may have to compare the path to the subdirectory (called prefix and found out from $(pwd)) with the tracked paths. On macOS, $(pwd) and readdir() yield decomposed path, while the tracked paths are usually normalized to the precomposed form, causing mismatch. This has been fixed by taking the same approach used to normalize the command line arguments. * tb/precompose-prefix-too: MacOS: precompose_argv_prefix()

MacOS: precompose_argv_prefix()

2021-02-03T22:09:37Z

The following sequence leads to a "BUG" assertion running under MacOS: DIR=git-test-restore-p Adiarnfd=$(printf 'A\314\210') DIRNAME=xx${Adiarnfd}yy mkdir $DIR && cd $DIR && git init && mkdir $DIRNAME && cd $DIRNAME && echo "Initial" >file && git add file && echo "One more line" >>file && echo y | git restore -p . Initialized empty Git repository in /tmp/git-test-restore-p/.git/ BUG: pathspec.c:495: error initializing pathspec_item Cannot close git diff-index --cached --numstat [snip] The command `git restore` is run from a directory inside a Git repo. Git needs to split the $CWD into 2 parts: The path to the repo and "the rest", if any. "The rest" becomes a "prefix" later used inside the pathspec code. As an example, "/path/to/repo/dir-inside-repå" would determine "/path/to/repo" as the root of the repo, the place where the configuration file .git/config is found. The rest becomes the prefix ("dir-inside-repå"), from where the pathspec machinery expands the ".", more about this later. If there is a decomposed form, (making the decomposing visible like this), "dir-inside-rep°a" doesn't match "dir-inside-repå". Git commands need to: (a) read the configuration variable "core.precomposeunicode" (b) precocompose argv[] (c) precompose the prefix, if there was any The first commit, 76759c7dff53 "git on Mac OS and precomposed unicode" addressed (a) and (b). The call to precompose_argv() was added into parse-options.c, because that seemed to be a good place when the patch was written. Commands that don't use parse-options need to do (a) and (b) themselfs. The commands `diff-files`, `diff-index`, `diff-tree` and `diff` learned (a) and (b) in commit 90a78b83e0b8 "diff: run arguments through precompose_argv" Branch names (or refs in general) using decomposed code points resulting in decomposed file names had been fixed in commit 8e712ef6fc97 "Honor core.precomposeUnicode in more places" The bug report from above shows 2 things: - more commands need to handle precomposed unicode - (c) should be implemented for all commands using pathspecs Solution: precompose_argv() now handles the prefix (if needed), and is renamed into precompose_argv_prefix(). Inside this function the config variable core.precomposeunicode is read into the global variable precomposed_unicode, as before. This reading is skipped if precomposed_unicode had been read before. The original patch for preocomposed unicode, 76759c7dff53, placed precompose_argv() into parse-options.c Now add it into git.c::run_builtin() as well. Existing precompose calls in diff-files.c and others may become redundant, and if we audit the callflows that reach these places to make sure that they can never be reached without going through the new call added to run_builtin(), we might be able to remove these existing ones. But in this commit, we do not bother to do so and leave these precompose callsites as they are. Because precompose() is idempotent and can be called on an already precomposed string safely, this is safer than removing existing calls without fully vetting the callflows. There is certainly room for cleanups - this change intends to be a bug fix. Cleanups needs more tests in e.g. t/t3910-mac-os-precompose.sh, and should be done in future commits. [1] git-bugreport-2021-01-06-1209.txt (git can't deal with special characters) [2] https://lore.kernel.org/git/A102844A-9501-4A86-854D-E3B387D378AA@icloud.com/ Reported-by: Daniel Troger Helped-By: Philippe Blain Signed-off-by: Torsten Bögershausen Signed-off-by: Junio C Hamano

Merge branch 'ps/config-env-pairs'

2021-01-25T22:19:19Z

Introduce two new ways to feed configuration variable-value pairs via environment variables, and tweak the way GIT_CONFIG_PARAMETERS encodes variable/value pairs to make it more robust. * ps/config-env-pairs: config: allow specifying config entries via envvar pairs environment: make `getenv_safe()` a public function config: store "git -c" variables using more robust format config: parse more robust format in GIT_CONFIG_PARAMETERS config: extract function to parse config pairs quote: make sq_dequote_step() a public function config: add new way to pass config via `--config-env` git: add `--super-prefix` to usage string

config: add new way to pass config via `--config-env`

2021-01-12T20:03:18Z

While it's already possible to pass runtime configuration via `git -c =`, it may be undesirable to use when the value contains sensitive information. E.g. if one wants to set `http.extraHeader` to contain an authentication token, doing so via `-c` would trivially leak those credentials via e.g. ps(1), which typically also shows command arguments. To enable this usecase without leaking credentials, this commit introduces a new switch `--config-env==`. Instead of directly passing a value for the given key, it instead allows the user to specify the name of an environment variable. The value of that variable will then be used as value of the key. Co-authored-by: Jeff King Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano