git/upload-pack.c, branch v1.8.2.2

Merge branch 'jk/peel-ref' into maint

2013-04-04T19:59:55Z

* jk/peel-ref: upload-pack: load non-tip "want" objects from disk upload-pack: make sure "want" objects are parsed upload-pack: drop lookup-before-parse optimization

upload-pack: load non-tip "want" objects from disk

2013-03-17T05:19:29Z

It is a long-time security feature that upload-pack will not serve any "want" lines that do not correspond to the tip of one of our refs. Traditionally, this was enforced by checking the objects in the in-memory hash; they should have been loaded and received the OUR_REF flag during the advertisement. The stateless-rpc mode, however, has a race condition here: one process advertises, and another receives the want lines, so the refs may have changed in the interim. To address this, commit 051e400 added a new verification mode; if the object is not OUR_REF, we set a "has_non_tip" flag, and then later verify that the requested objects are reachable from our current tips. However, we still die immediately when the object is not in our in-memory hash, and at this point we should only have loaded our tip objects. So the check_non_tip code path does not ever actually trigger, as any non-tip objects would have already caused us to die. We can fix that by using parse_object instead of lookup_object, which will load the object from disk if it has not already been loaded. We still need to check that parse_object does not return NULL, though, as it is possible we do not have the object at all. A more appropriate error message would be "no such object" rather than "not our ref"; however, we do not want to leak information about what objects are or are not in the object database, so we continue to use the same "not our ref" message that would be produced by an unreachable object. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

upload-pack: make sure "want" objects are parsed

2013-03-17T05:16:56Z

When upload-pack receives a "want" line from the client, it adds it to an object array. We call lookup_object to find the actual object, which will only check for objects already in memory. This works because we are expecting to find objects that we already loaded during the ref advertisement. We use the resulting object structs for a variety of purposes. Some of them care only about the object flags, but others care about the type of the object (e.g., ok_to_give_up), or even feed them to the revision parser (when --depth is used), which assumes that objects it receives are fully parsed. Once upon a time, this was OK; any object we loaded into memory would also have been parsed. But since 435c833 (upload-pack: use peel_ref for ref advertisements, 2012-10-04), we try to avoid parsing objects during the ref advertisement. This means that lookup_object may return an object with a type of OBJ_NONE. The resulting mess depends on the exact set of objects, but can include the revision parser barfing, or the shallow code sending the wrong set of objects. This patch teaches upload-pack to parse each "want" object as we receive it. We do not replace the lookup_object call with parse_object, as the current code is careful not to let just any object appear on a "want" line, but rather only one we have previously advertised (whereas parse_object would actually load any arbitrary object from disk). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

upload-pack: drop lookup-before-parse optimization

2013-03-17T05:16:45Z

When we receive a "have" line from the client, we want to load the object pointed to by the sha1. However, we are careful to do: o = lookup_object(sha1); if (!o || !o->parsed) o = parse_object(sha1); to avoid loading the object from disk if we have already seen it. However, since ccdc603 (parse_object: try internal cache before reading object db), parse_object already does this optimization internally. We can just call parse_object directly. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'jc/hidden-refs'

2013-02-17T23:25:57Z

Allow the server side to redact the refs/ namespace it shows to the client. Will merge to 'master'. * jc/hidden-refs: upload/receive-pack: allow hiding ref hierarchies upload-pack: simplify request validation upload-pack: share more code

upload/receive-pack: allow hiding ref hierarchies

2013-02-07T21:48:47Z

A repository may have refs that are only used for its internal bookkeeping purposes that should not be exposed to the others that come over the network. Teach upload-pack to omit some refs from its initial advertisement by paying attention to the uploadpack.hiderefs multi-valued configuration variable. Do the same to receive-pack via the receive.hiderefs variable. As a convenient short-hand, allow using transfer.hiderefs to set the value to both of these variables. Any ref that is under the hierarchies listed on the value of these variable is excluded from responses to requests made by "ls-remote", "fetch", etc. (for upload-pack) and "push" (for receive-pack). Because these hidden refs do not count as OUR_REF, an attempt to fetch objects at the tip of them will be rejected, and because these refs do not get advertised, "git push :" will not see local branches that have the same name as them as "matching" ones to be sent. An attempt to update/delete these hidden refs with an explicit refspec, e.g. "git push origin :refs/hidden/22", is rejected. This is not a new restriction. To the pusher, it would appear that there is no such ref, so its push request will conclude with "Now that I sent you all the data, it is time for you to update the refs. I saw that the ref did not exist when I started pushing, and I want the result to point at this commit". The receiving end will apply the compare-and-swap rule to this request and rejects the push with "Well, your update request conflicts with somebody else; I see there is such a ref.", which is the right thing to do. Otherwise a push to a hidden ref will always be "the last one wins", which is not a good default. Signed-off-by: Junio C Hamano

Merge branch 'nd/fetch-depth-is-broken'

2013-02-01T20:39:24Z

"git fetch --depth" was broken in at least three ways. The resulting history was deeper than specified by one commit, it was unclear how to wipe the shallowness of the repository with the command, and documentation was misleading. * nd/fetch-depth-is-broken: fetch: elaborate --depth action upload-pack: fix off-by-one depth calculation in shallow clone fetch: add --unshallow for turning shallow repo into complete one

upload-pack: simplify request validation

2013-01-29T05:05:51Z

Long time ago, we used to punt on a large (read: asking for more than 256 refs) fetch request and instead sent a full pack, because we couldn't fit many refs on the command line of rev-list we run internally to enumerate the objects to be sent. To fix this, 565ebbf (upload-pack: tighten request validation., 2005-10-24), added a check to count the number of refs in the request and matched with the number of refs we advertised, and changed the invocation of rev-list to pass "--all" to it, still keeping us under the command line argument limit. However, these days we feed the list of objects requested and the list of objects the other end is known to have via standard input, so there is no longer a valid reason to special case a full clone request. Remove the code associated with "create_full_pack" to simplify the logic. Signed-off-by: Junio C Hamano

upload-pack: share more code

2013-01-18T23:48:49Z

We mark the objects pointed at our refs with "OUR_REF" flag in two functions (mark_our_ref() and send_ref()), but we can just use the former as a helper for the latter. Update the way mark_our_ref() prepares in-core object to use lookup_unknown_object() to delay reading the actual object data, just like we did in 435c833 (upload-pack: use peel_ref for ref advertisements, 2012-10-04). Signed-off-by: Junio C Hamano

Merge branch 'nd/upload-pack-shallow-must-be-commit'

2013-01-14T16:15:44Z

A minor consistency check patch that does not have much relevance to the real world. * nd/upload-pack-shallow-must-be-commit: upload-pack: only accept commits from "shallow" line