git/http.c, branch v2.40.2

Merge branch 'backport/jk/libcurl-8.7-regression-workaround' into maint-2.39

2024-04-16T21:58:53Z

Fix was added to work around a regression in libcURL 8.7.0 (which has already been fixed in their tip of the tree). * jk/libcurl-8.7-regression-workaround: remote-curl: add Transfer-Encoding header only for older curl INSTALL: bump libcurl version to 7.21.3 http: reset POSTFIELDSIZE when clearing curl handle Signed-off-by: Johannes Schindelin

Merge branch 'jk/redact-h2h3-headers-fix' into maint-2.42

2024-04-16T21:58:48Z

HTTP Header redaction code has been adjusted for a newer version of cURL library that shows its traces differently from earlier versions. * jk/redact-h2h3-headers-fix: http: update curl http/2 info matching for curl 8.3.0 http: factor out matching of curl http/2 trace lines This backport to `maint-2.39` is needed to bring the following test cases back to a working state in conjunction with recent libcurl versions: - t5559.17 GIT_TRACE_CURL redacts auth details - t5559.18 GIT_CURL_VERBOSE redacts auth details - t5559.38 cookies are redacted by default Signed-off-by: Johannes Schindelin

http: reset POSTFIELDSIZE when clearing curl handle

2024-04-10T17:24:48Z

In get_active_slot(), we return a CURL handle that may have been used before (reusing them is good because it lets curl reuse the same connection across many requests). We set a few curl options back to defaults that may have been modified by previous requests. We reset POSTFIELDS to NULL, but do not reset POSTFIELDSIZE (which defaults to "-1"). This usually doesn't matter because most POSTs will set both fields together anyway. But there is one exception: when handling a large request in remote-curl's post_rpc(), we don't set _either_, and instead set a READFUNCTION to stream data into libcurl. This can interact weirdly with a stale POSTFIELDSIZE setting, because curl will assume it should read only some set number of bytes from our READFUNCTION. However, it has worked in practice because we also manually set a "Transfer-Encoding: chunked" header, which libcurl uses as a clue to set the POSTFIELDSIZE to -1 itself. So everything works, but we're better off resetting the size manually for a few reasons: - there was a regression in curl 8.7.0 where the chunked header detection didn't kick in, causing any large HTTP requests made by Git to fail. This has since been fixed (but not yet released). In the issue, curl folks recommended setting it explicitly to -1: https://github.com/curl/curl/issues/13229#issuecomment-2029826058 and it indeed works around the regression. So even though it won't be strictly necessary after the fix there, this will help folks who end up using the affected libcurl versions. - it's consistent with what a new curl handle would look like. Since get_active_slot() may or may not return a used handle, this reduces the possibility of heisenbugs that only appear with certain request patterns. Note that the recommendation in the curl issue is to actually drop the manual Transfer-Encoding header. Modern libcurl will add the header itself when streaming from a READFUNCTION. However, that code wasn't added until 802aa5ae2 (HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown, 2019-07-22), which is in curl 7.66.0. We claim to support back to 7.19.5, so those older versions still need the manual header. This is a backport of 3242311742 (http: reset POSTFIELDSIZE when clearing curl handle, 2024-04-02) into the `maint-2.39` branch. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano Signed-off-by: Johannes Schindelin

http: update curl http/2 info matching for curl 8.3.0

2023-09-15T17:54:11Z

To redact header lines in http/2 curl traces, we have to parse past some prefix bytes that curl sticks in the info lines it passes to us. That changed once already, and we adapted in db30130165 (http: handle both "h2" and "h2h3" in curl info lines, 2023-06-17). Now it has changed again, in curl's fbacb14c4 (http2: cleanup trace messages, 2023-08-04), which was released in curl 8.3.0. Running a build of git linked against that version will fail to redact the trace (and as before, t5559 notices and complains). The format here is a little more complicated than the other ones, as it now includes a "stream id". This is not constant but is always numeric, so we can easily parse past it. We'll continue to match the old versions, of course, since we want to work with many different versions of curl. We can't even select one format at compile time, because the behavior depends on the runtime version of curl we use, not the version we build against. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

http: factor out matching of curl http/2 trace lines

2023-09-15T17:54:08Z

We have to parse out curl's http/2 trace lines so we can redact their headers. We already match two different types of lines from various vintages of curl. In preparation for adding another (which will be slightly more complex), let's pull the matching into its own function, rather than doing it in the middle of a conditional. While we're doing so, let's expand the comment a bit to describe the two matches. That probably should have been part of db30130165 (http: handle both "h2" and "h2h3" in curl info lines, 2023-06-17), but will become even more important as we add new types. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

http: handle both "h2" and "h2h3" in curl info lines

2023-06-17T16:08:31Z

When redacting auth tokens in trace output from curl, we look for http/2 headers of the form "h2h3 [header: value]". This comes from b637a41ebe (http: redact curl h2h3 headers in info, 2022-11-11). But the "h2h3" prefix changed to just "h2" in curl's fc2f1e547 (http2: support HTTP/2 to forward proxies, non-tunneling, 2023-04-14). That's in released version curl 8.1.0; linking against that version means we'll fail to correctly redact the trace. Our t5559.17 notices and fails. We can fix this by matching either prefix, which should handle both old and new versions. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Sync with 2.38.4

2023-02-06T08:43:39Z

* maint-2.38: Git 2.38.4 Git 2.37.6 Git 2.36.5 Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport

Sync with 2.37.6

2023-02-06T08:43:28Z

* maint-2.37: Git 2.37.6 Git 2.36.5 Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport

Sync with 2.36.5

2023-02-06T08:38:31Z

* maint-2.36: Git 2.36.5 Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport

Sync with 2.35.7

2023-02-06T08:37:52Z

* maint-2.35: Git 2.35.7 Git 2.34.7 http: support CURLOPT_PROTOCOLS_STR http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT Git 2.33.7 Git 2.32.6 Git 2.31.7 Git 2.30.8 apply: fix writing behind newly created symbolic links dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS clone: delay picking a transport until after get_repo_path() t5619: demonstrate clone_local() with ambiguous transport