git/serve.c, branch v2.40.3

protocol v2: add server-side "bundle-uri" skeleton

2022-12-25T07:24:23Z

Add a skeleton server-side implementation of a new "bundle-uri" command to protocol v2. This will allow conforming clients to optionally seed their initial clones or incremental fetches from URLs containing "*.bundle" files created with "git bundle create". This change only performs the basic boilerplate of advertising a new protocol v2 capability. The new 'bundle-uri' capability allows a client to request a list of bundles. Right now, the server only returns a flush packet, which corresponds to an empty advertisement. The bundle.* config namespace describes which key-value pairs will be communicated across this interface in future updates. The critical bit right now is that the new boolean uploadPack.adverstiseBundleURIs config value signals whether or not this capability should be advertised at all. An earlier version of this patch [1] used a different transfer format than the "key=value" pairs in the current implementation. The change was made to unify the protocol v2 command with the bundle lists provided by independent bundle servers. Further, the standard allows for the server to advertise a URI that contains a bundle list. This allows users automatically discovering bundle providers that are loosely associated with the origin server, but without the origin server knowing exactly which bundles are currently available. [1] https://lore.kernel.org/git/RFC-patch-v2-01.13-2fc87ce092b-20220311T155841Z-avarab@gmail.com/ The very-deep headings needed to be modified to stop at level 4 due to documentation build issues. These were not recognized in earlier builds since the file was previously in the Documentation/technical/ directory and was built in a different way. With its current location, the heavily-nested details were causing build issues and they are now replaced with a bulletted list of details. Co-authored-by: Derrick Stolee Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

serve.c: remove unnecessary include

2022-04-06T16:42:12Z

Remove include "strvec.h" from serve.c, which is orphaned since f0a35c9ce52 (serve: drop "keys" strvec, 2021-09-15) Signed-off-by: Garrit Franke Signed-off-by: Junio C Hamano

serve: reject commands used as capabilities

2021-09-15T19:25:19Z

Our table of v2 "capabilities" contains everything we might tell the client we support. But there are differences in how we expect the client to respond. Some of the entries are true capabilities (i.e., we expect the client to say "yes, I support this"), and some are ones we expect them to send as commands (with "command=ls-refs" or similar). When we receive a capability used as a command, we complain about that. But when we receive a command used as a capability (e.g., just "ls-refs" in a pkt-line by itself), we silently ignore it. This isn't really hurting anything (clients shouldn't send it, and we'll ignore it), but we can tighten up the protocol to match what we expect to happen. There are two new tests here. The first one checks a capability used as a command, which already passes. The second tests a command as a capability, which this patch fixes. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: reject bogus v2 "command=ls-refs=foo"

2021-09-15T19:25:19Z

When we see a line from the client like "command=ls-refs", we parse everything after the equals sign as a capability, which we check against our capabilities table. If we don't recognize the command (e.g., "command=foo"), we'll reject it. But in parse_command(), we use the same get_capability() parser for parsing non-command lines. So if we see "command=ls-refs=foo", we will feed "ls-refs=foo" to get_capability(), which will say "OK, that's ls-refs, with value 'foo'". But then we simply ignore the value entirely. The client is violating the spec here, which says: command = PKT-LINE("command=" key LF) key = 1*(ALPHA | DIGIT | "-_") I.e., the key is not even allowed to have an equals sign in it. Whereas a real non-command capability does allow a value: capability = PKT-LINE(key[=value] LF) So by reusing the same get_capability() parser, we are mixing up the "key" and "capability" tokens. However, since that parser tells us whether it saw an "=", we can still use it; we just need to reject any input that produces a non-NULL value field. The current behavior isn't really hurting anything (the client should never send such a request, and if it does, we just ignore the "value" part). But since it does violate the spec, let's tighten it up to prevent any surprising behavior. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: drop "keys" strvec

2021-09-15T19:25:19Z

We collect the set of capabilities the client sends us in a strvec. While this is usually small, there's no limit to the number of capabilities the client can send us (e.g., they could just send us "agent" pkt-lines over and over, and we'd keep adding them to the list). Since all code has been converted away from using this list, let's get rid of it. This avoids a potential attack where clients waste our memory. Note that we do have to replace it with a flag, because some of the flush-packet logic checks whether we've seen any valid commands or keys. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: provide "receive" function for session-id capability

2021-09-15T00:12:05Z

Rather than pulling the session-id string from the list of collected capabilities, we can handle it as soon as we receive it. This gets us closer to dropping the collected list entirely. The behavior should be the same, with one exception. Previously if the client sent us multiple session-id lines, we'd report only the first. Now we'll pass each one along to trace2. This shouldn't matter in practice, since clients shouldn't do that (and if they do, it's probably sensible to log them all). As this removes the last caller of the static has_capability(), we can remove it, as well (and in fact we must to avoid -Wunused-function complaining). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: provide "receive" function for object-format capability

2021-09-15T00:12:05Z

We get any "object-format" specified by the client by searching for it in the collected list of capabilities the client sent. We can instead just handle it as soon as they send it. This is slightly more efficient, and gets us one step closer to dropping that collected list. Note that we do still have to do our final hash check after receiving all capabilities (because they might not have sent an object-format line at all, and we still have to check that the default matches our repository algorithm). Since the check_algorithm() function would now be down to a single if() statement, I've just inlined it in its only caller. There should be no change of behavior here, except for two broken-protocol cases: - if the client sends multiple conflicting object-format capabilities (which they should not), we'll now choose the last one rather than the first. We could also detect and complain about the duplicates quite easily now, which we could not before, but I didn't do so here. - if the client sends a bogus "object-format" with no equals sign, we'll now say so, rather than "unknown object format: ''" Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: add "receive" method for v2 capabilities table

2021-09-14T17:56:19Z

We have a capabilities table that tells us what we should tell the client we are capable of, and what to do when a client gives us a particular command (e.g., "command=ls-refs"). But it doesn't tell us what to do when the client sends us back a capability (e.g., "object-format=sha256"). We just collect them all in a strvec and hope somebody can use them later. Instead, let's provide a function pointer in the table to act on these. This will eventually help us avoid collecting the strings, which will be more efficient and less prone to mischief. Using the new method is optional, which helps in two ways: - we can move existing capabilities over to this new system gradually in individual commits - some capabilities we don't actually do anything with anyway. For example, the client is free to say "agent=git/1.2.3" to us, but we do not act on the information in any way. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: return capability "value" from get_capability()

2021-09-14T17:56:19Z

When the client sends v2 capabilities, they may be simple, like: foo or have a value like: foo=bar (all of the current capabilities actually expect a value, but the protocol allows for boolean ones). We use get_capability() to make sure the client's pktline matches a capability. In doing so, we parse enough to see the "=" and the value (if any), but we immediately forget it. Nobody cares for now, because they end up parsing the values out later using has_capability(). But in preparation for changing that, let's pass back a pointer so the callers know what we found. Note that unlike has_capability(), we'll return NULL for a "simple" capability. Distinguishing these will be useful for some future patches. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

serve: rename is_command() to parse_command()

2021-09-14T17:56:19Z

The is_command() function not only tells us whether the pktline is a valid command string, but it also parses out the command (and complains if we see a duplicate). Let's rename it to make those extra functions a bit more obvious. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano