git/remote-curl.c, branch v2.46.2

Merge branch 'jk/remote-wo-url'

2024-07-02T16:59:01Z

Memory ownership rules for the in-core representation of remote.*.url configuration values have been straightened out, which resulted in a few leak fixes and code clarification. * jk/remote-wo-url: remote: drop checks for zero-url case remote: always require at least one url in a remote t5801: test remote.*.vcs config t5801: make remote-testgit GIT_DIR setup more robust remote: allow resetting url list config: document remote.*.url/pushurl interaction remote: simplify url/pushurl selection remote: use strvecs to store remote url/pushurl remote: transfer ownership of memory in add_url(), etc remote: refactor alias_url() memory ownership archive: fix check for missing url

Merge branch 'ps/use-the-repository'

2024-07-02T16:59:00Z

A CPP macro USE_THE_REPOSITORY_VARIABLE is introduced to help transition the codebase to rely less on the availability of the singleton the_repository instance. * ps/use-the-repository: hex: guard declarations with `USE_THE_REPOSITORY_VARIABLE` t/helper: remove dependency on `the_repository` in "proc-receive" t/helper: fix segfault in "oid-array" command without repository t/helper: use correct object hash in partial-clone helper compat/fsmonitor: fix socket path in networked SHA256 repos replace-object: use hash algorithm from passed-in repository protocol-caps: use hash algorithm from passed-in repository oidset: pass hash algorithm when parsing file http-fetch: don't crash when parsing packfile without a repo hash-ll: merge with "hash.h" refs: avoid include cycle with "repository.h" global: introduce `USE_THE_REPOSITORY_VARIABLE` macro hash: require hash algorithm in `empty_tree_oid_hex()` hash: require hash algorithm in `is_empty_{blob,tree}_oid()` hash: make `is_null_oid()` independent of `the_repository` hash: convert `oidcmp()` and `oideq()` to compare whole hash global: ensure that object IDs are always padded hash: require hash algorithm in `oidread()` and `oidclr()` hash: require hash algorithm in `hasheq()`, `hashcmp()` and `hashclr()` hash: drop (mostly) unused `is_empty_{blob,tree}_sha1()` functions

Merge branch 'ps/no-writable-strings'

2024-06-17T22:55:58Z

Building with "-Werror -Wwrite-strings" is now supported. * ps/no-writable-strings: (27 commits) config.mak.dev: enable `-Wwrite-strings` warning builtin/merge: always store allocated strings in `pull_twohead` builtin/rebase: always store allocated string in `options.strategy` builtin/rebase: do not assign default backend to non-constant field imap-send: fix leaking memory in `imap_server_conf` imap-send: drop global `imap_server_conf` variable mailmap: always store allocated strings in mailmap blob revision: always store allocated strings in output encoding remote-curl: avoid assigning string constant to non-const variable send-pack: always allocate receive status parse-options: cast long name for OPTION_ALIAS http: do not assign string constant to non-const field compat/win32: fix const-correctness with string constants pretty: add casts for decoration option pointers object-file: make `buf` parameter of `index_mem()` a constant object-file: mark cached object buffers as const ident: add casts for fallback name and GECOS entry: refactor how we remove items for delayed checkouts line-log: always allocate the output prefix line-log: stop assigning string constant to file parent buffer ...

global: introduce `USE_THE_REPOSITORY_VARIABLE` macro

2024-06-14T17:26:33Z

Use of the `the_repository` variable is deprecated nowadays, and we slowly but steadily convert the codebase to not use it anymore. Instead, callers should be passing down the repository to work on via parameters. It is hard though to prove that a given code unit does not use this variable anymore. The most trivial case, merely demonstrating that there is no direct use of `the_repository`, is already a bit of a pain during code reviews as the reviewer needs to manually verify claims made by the patch author. The bigger problem though is that we have many interfaces that implicitly rely on `the_repository`. Introduce a new `USE_THE_REPOSITORY_VARIABLE` macro that allows code units to opt into usage of `the_repository`. The intent of this macro is to demonstrate that a certain code unit does not use this variable anymore, and to keep it from new dependencies on it in future changes, be it explicit or implicit For now, the macro only guards `the_repository` itself as well as `the_hash_algo`. There are many more known interfaces where we have an implicit dependency on `the_repository`, but those are not guarded at the current point in time. Over time though, we should start to add guards as required (or even better, just remove them). Define the macro as required in our code units. As expected, most of our code still relies on the global variable. Nearly all of our builtins rely on the variable as there is no way yet to pass `the_repository` to their entry point. For now, declare the macro in "biultin.h" to keep the required changes at least a little bit more contained. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

remote: use strvecs to store remote url/pushurl

2024-06-14T16:34:38Z

Now that the url/pushurl fields of "struct remote" own their strings, we can switch from bare arrays to strvecs. This has a few advantages: - push/clear are now one-liners - likewise the free+assigns in alias_all_urls() can use strvec_replace() - we now use size_t for storage, avoiding possible overflow - this will enable some further cleanups in future patches There's quite a bit of fallout in the code that reads these fields, as it tends to access these arrays directly. But it's mostly a mechanical replacement of "url_nr" with "url.nr", and "url[i]" with "url.v[i]", with a few variations (e.g. "*url" could become "*url.v", but I used "url.v[0]" for consistency). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

remote-curl: avoid assigning string constant to non-const variable

2024-06-07T17:30:53Z

When processing remote options, we split the option line into two by searching for a space. If there is one, we replace the space with '\0', otherwise we implicitly assume that the value is "true" and thus assign a string constant. As the return value of strchr(3P) weirdly enough is a `char *` even though it gets a `const char *` as input, the assigned-to variable also is a non-constant. This is fine though because the argument is in fact an allocated string, and thus we are allowed to modify it. But this will break once we enable `-Wwrite-strings`. Refactor the code stop splitting the fields with '\0' altogether. Instead, we can pass the length of the option name to `set_option()` and then use strncmp(3P) instead of strcmp(3P). Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

Merge branch 'ps/undecided-is-not-necessarily-sha1'

2024-05-30T21:15:11Z

Before discovering the repository details, We used to assume SHA-1 as the "default" hash function, which has been corrected. Hopefully this will smoke out codepaths that rely on such an unwarranted assumptions. * ps/undecided-is-not-necessarily-sha1: repository: stop setting SHA1 as the default object hash oss-fuzz/commit-graph: set up hash algorithm builtin/shortlog: don't set up revisions without repo builtin/diff: explicitly set hash algo when there is no repo builtin/bundle: abort "verify" early when there is no repository builtin/blame: don't access potentially unitialized `the_hash_algo` builtin/rev-parse: allow shortening to more than 40 hex characters remote-curl: fix parsing of detached SHA256 heads attr: fix BUG() when parsing attrs outside of repo attr: don't recompute default attribute source parse-options-cb: only abbreviate hashes when hash algo is known path: move `validate_headref()` to its only user path: harden validation of HEAD with non-standard hashes

Merge branch 'bc/credential-scheme-enhancement'

2024-05-08T17:18:44Z

The credential helper protocol, together with the HTTP layer, have been enhanced to support authentication schemes different from username & password pair, like Bearer and NTLM. * bc/credential-scheme-enhancement: credential: add method for querying capabilities credential-cache: implement authtype capability t: add credential tests for authtype credential: add support for multistage credential rounds t5563: refactor for multi-stage authentication docs: set a limit on credential line length credential: enable state capability credential: add an argument to keep state http: add support for authtype and credential docs: indicate new credential protocol fields credential: add a field called "ephemeral" credential: gate new fields on capability credential: add a field for pre-encoded credentials http: use new headers for each object request remote-curl: reset headers on new request credential: add an authtype field

remote-curl: fix parsing of detached SHA256 heads

2024-05-07T05:50:49Z

The dumb HTTP transport tries to read the remote HEAD reference by downloading the "HEAD" file and then parsing it via `http_fetch_ref()`. This function will either parse the file as an object ID in case it is exactly `the_hash_algo->hexsz` long, or otherwise it will check whether the reference starts with "ref :" and parse it as a symbolic ref. This is broken when parsing detached HEADs of a remote SHA256 repository because we never update `the_hash_algo` to the discovered remote object hash. Consequently, `the_hash_algo` will always be the fallback SHA1 hash algorithm, which will cause us to fail parsing HEAD altogteher when it contains a SHA256 object ID. Fix this issue by setting up `the_hash_algo` via `repo_set_hash_algo()`. While at it, let's make the expected SHA1 fallback explicit in our code, which also addresses an upcoming issue where we are going to remove the SHA1 fallback for `the_hash_algo`. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

http: add support for authtype and credential

2024-04-17T05:39:07Z

Now that we have the credential helper code set up to handle arbitrary authentications schemes, let's add support for this in the HTTP code, where we really want to use it. If we're using this new functionality, don't set a username and password, and instead set a header wherever we'd normally do so, including for proxy authentication. Since we can now handle this case, ask the credential helper to enable the appropriate capabilities. Finally, if we're using the authtype value, set "Expect: 100-continue". Any type of authentication that requires multiple rounds (such as NTLM or Kerberos) requires a 100 Continue (if we're larger than http.postBuffer) because otherwise we send the pack data before we're authenticated, the push gets a 401 response, and we can't rewind the stream. We don't know for certain what other custom schemes might require this, the HTTP/1.1 standard has required handling this since 1999, the broken HTTP server for which we disabled this (Google's) is now fixed and has been for some time, and libcurl has a 1-second fallback in case the HTTP server is still broken. In addition, it is not unreasonable to require compliance with a 25-year old standard to use new Git features. For all of these reasons, do so here. Signed-off-by: brian m. carlson Signed-off-by: Junio C Hamano