git/Documentation/git-http-backend.txt, branch v2.8.2

Merge branch 'jk/http-backend-deadlock'

2015-06-01T19:45:09Z

Communication between the HTTP server and http_backend process can lead to a dead-lock when relaying a large ref negotiation request. Diagnose the situation better, and mitigate it by reading such a request first into core (to a reasonable limit). * jk/http-backend-deadlock: http-backend: spool ref negotiation requests to buffer t5551: factor out tag creation http-backend: fix die recursion with custom handler

http-backend: spool ref negotiation requests to buffer

2015-05-26T03:43:18Z

When http-backend spawns "upload-pack" to do ref negotiation, it streams the http request body to upload-pack, who then streams the http response back to the client as it reads. In theory, git can go full-duplex; the client can consume our response while it is still sending the request. In practice, however, HTTP is a half-duplex protocol. Even if our client is ready to read and write simultaneously, we may have other HTTP infrastructure in the way, including the webserver that spawns our CGI, or any intermediate proxies. In at least one documented case[1], this leads to deadlock when trying a fetch over http. What happens is basically: 1. Apache proxies the request to the CGI, http-backend. 2. http-backend gzip-inflates the data and sends the result to upload-pack. 3. upload-pack acts on the data and generates output over the pipe back to Apache. Apache isn't reading because it's busy writing (step 1). This works fine most of the time, because the upload-pack output ends up in a system pipe buffer, and Apache reads it as soon as it finishes writing. But if both the request and the response exceed the system pipe buffer size, then we deadlock (Apache blocks writing to http-backend, http-backend blocks writing to upload-pack, and upload-pack blocks writing to Apache). We need to break the deadlock by spooling either the input or the output. In this case, it's ideal to spool the input, because Apache does not start reading either stdout _or_ stderr until we have consumed all of the input. So until we do so, we cannot even get an error message out to the client. The solution is fairly straight-forward: we read the request body into an in-memory buffer in http-backend, freeing up Apache, and then feed the data ourselves to upload-pack. But there are a few important things to note: 1. We limit the in-memory buffer to prevent an obvious denial-of-service attack. This is a new hard limit on requests, but it's unlikely to come into play. The default value is 10MB, which covers even the ridiculous 100,000-ref negotation in the included test (that actually caps out just over 5MB). But it's configurable on the off chance that you don't mind spending some extra memory to make even ridiculous requests work. 2. We must take care only to buffer when we have to. For pushes, the incoming packfile may be of arbitrary size, and we should connect the input directly to receive-pack. There's no deadlock problem here, though, because we do not produce any output until the whole packfile has been read. For upload-pack's initial ref advertisement, we similarly do not need to buffer. Even though we may generate a lot of output, there is no request body at all (i.e., it is a GET, not a POST). [1] http://article.gmane.org/gmane.comp.version-control.git/269020 Test-adapted-from: Dennis Kaarsemaker Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

doc: put example URLs and emails inside literal backticks

2015-05-13T05:14:46Z

This makes sure that AsciiDoc does not turn them into links. Regular AsciiDoc does not catch these cases, but AsciiDoctor does treat them as links. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge commit 'doc/http-backend: missing accent grave in literal mark-up'

2014-04-09T18:45:04Z

* commit '5df05146d5cb94628a3dfc53063c802ee1152cec': doc/http-backend: missing accent grave in literal mark-up

doc/http-backend: missing accent grave in literal mark-up

2014-04-09T18:43:56Z

Signed-off-by: Thomas Ackermann Signed-off-by: Junio C Hamano

doc: remote author/documentation sections from more pages

2014-01-27T16:34:34Z

We decided at 48bb914e (doc: drop author/documentation sections from most pages, 2011-03-11) to remove "author" and "documentation" sections from our documentation. Remove a few stragglers. Signed-off-by: Michael Haggerty Reviewed-by: Jonathan Nieder Signed-off-by: Junio C Hamano

doc/http-backend: match query-string in apache half-auth example

2013-04-14T05:27:06Z

When setting up a "half-auth" repository in which reads can be done anonymously but writes require authentication, it is best if the server can require authentication for both the ref advertisement and the actual receive-pack POSTs. This alleviates the need for the admin to set http.receivepack in the repositories, and means that the client is challenged for credentials immediately, instead of partway through the push process (and git clients older than v1.7.11.7 had trouble handling these challenges). Since detecting a push during the ref advertisement requires matching the query string, and this is non-trivial to do in Apache, we have traditionally punted and instructed users to just protect "/git-receive-pack$". This patch provides the mod_rewrite recipe to actually match the ref advertisement, which is preferred. While we're at it, let's add the recipe to our test scripts so that we can be sure that it works, and doesn't get broken (either by our changes or by changes in Apache). Signed-off-by: Jeff King Acked-by: Jakub Narębski Signed-off-by: Junio C Hamano

doc/http-backend: give some lighttpd config examples

2013-04-11T14:33:21Z

The examples in the documentation are all for Apache. Let's at least cover the basics: an anonymous server, an authenticated server, and a "half auth" server with anonymous read and authenticated write. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

doc/http-backend: clarify "half-auth" repo configuration

2013-04-11T14:33:07Z

When the http-backend is set up to allow anonymous read but authenticated write, the http-backend manual suggests catching only the "/git-receive-pack" POST of the packfile, not the initial "info/refs?service=git-receive-pack" GET in which we advertise refs. This does work and is secure, as we do not allow any write during the info/refs request, and the information in the ref advertisement is the same that you would get from a fetch. However, the configuration required by the server is slightly more complex. The default `http.receivepack` setting is to allow pushes if the webserver tells us that the user authenticated, and otherwise to return a 403 ("Forbidden"). That works fine if authentication is turned on completely; the initial request requires authentication, and http-backend realizes it is OK to do a push. But for this "half-auth" state, no authentication has occurred during the initial ref advertisement. The http-backend CGI therefore does not think that pushing should be enabled, and responds with a 403. The client cannot continue, even though the server would have allowed it to run if it had provided credentials. It would be much better if the server responded with a 401, asking for credentials during the initial contact. But git-http-backend does not know about the server's auth configuration (so a 401 would be confusing in the case of a true anonymous server). Unfortunately, configuring Apache to recognize the query string and apply the auth appropriately to receive-pack (but not upload-pack) initial requests is non-trivial. The site admin can work around this by just turning on http.receivepack explicitly in its repositories. Let's document this workaround. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Documentation: the name of the system is 'Git', not 'git'

2013-02-01T21:53:33Z

Signed-off-by: Thomas Ackermann Signed-off-by: Junio C Hamano