git/remote.h, branch v2.30.2

Merge branch 'sk/force-if-includes'

2020-10-27T22:09:49Z

"git push --force-with-lease[=]" can easily be misused to lose commits unless the user takes good care of their own "git fetch". A new option "--force-if-includes" attempts to ensure that what is being force-pushed was created after examining the commit at the tip of the remote ref that is about to be force-replaced. * sk/force-if-includes: t, doc: update tests, reference for "--force-if-includes" push: parse and set flag for "--force-if-includes" push: add reflog check for "--force-if-includes"

Merge branch 'jk/refspecs-negative'

2020-10-05T21:01:54Z

"git fetch" and "git push" support negative refspecs. * jk/refspecs-negative: refspec: add support for negative refspecs

push: add reflog check for "--force-if-includes"

2020-10-03T16:59:18Z

Add a check to verify if the remote-tracking ref of the local branch is reachable from one of its "reflog" entries. The check iterates through the local ref's reflog to see if there is an entry for the remote-tracking ref and collecting any commits that are seen, into a list; the iteration stops if an entry in the reflog matches the remote ref or if the entry timestamp is older the latest entry of the remote ref's "reflog". If there wasn't an entry found for the remote ref, "in_merge_bases_many()" is called to check if it is reachable from the list of collected commits. When a local branch that is based on a remote ref, has been rewound and is to be force pushed on the remote, "--force-if-includes" runs a check that ensures any updates to the remote-tracking ref that may have happened (by push from another repository) in-between the time of the last update to the local branch (via "git-pull", for instance) and right before the time of push, have been integrated locally before allowing a forced update. If the new option is passed without specifying "--force-with-lease", or specified along with "--force-with-lease=:" it is a "no-op". Signed-off-by: Srinidhi Kaushik Signed-off-by: Junio C Hamano

refspec: add support for negative refspecs

2020-09-30T21:52:00Z

Both fetch and push support pattern refspecs which allow fetching or pushing references that match a specific pattern. Because these patterns are globs, they have somewhat limited ability to express more complex situations. For example, suppose you wish to fetch all branches from a remote except for a specific one. To allow this, you must setup a set of refspecs which match only the branches you want. Because refspecs are either explicit name matches, or simple globs, many patterns cannot be expressed. Add support for a new type of refspec, referred to as "negative" refspecs. These are prefixed with a '^' and mean "exclude any ref matching this refspec". They can only have one "side" which always refers to the source. During a fetch, this refers to the name of the ref on the remote. During a push, this refers to the name of the ref on the local side. With negative refspecs, users can express more complex patterns. For example: git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/dontwant will fetch all branches on origin into remotes/origin, but will exclude fetching the branch named dontwant. Refspecs today are commutative, meaning that order doesn't expressly matter. Rather than forcing an implied order, negative refspecs will always be applied last. That is, in order to match, a ref must match at least one positive refspec, and match none of the negative refspecs. This is similar to how negative pathspecs work. Signed-off-by: Jacob Keller Signed-off-by: Junio C Hamano

Merge branch 'jx/proc-receive-hook'

2020-09-25T22:25:39Z

"git receive-pack" that accepts requests by "git push" learned to outsource most of the ref updates to the new "proc-receive" hook. * jx/proc-receive-hook: doc: add documentation for the proc-receive hook transport: parse report options for tracking refs t5411: test updates of remote-tracking branches receive-pack: new config receive.procReceiveRefs doc: add document for capability report-status-v2 New capability "report-status-v2" for git-push receive-pack: feed report options to post-receive receive-pack: add new proc-receive hook t5411: add basic test cases for proc-receive hook transport: not report a non-head push as a branch

New capability "report-status-v2" for git-push

2020-08-27T19:47:47Z

The new introduced "proc-receive" hook may handle a command for a pseudo-reference with a zero-old as its old-oid, while the hook may create or update a reference with different name, different new-oid, and different old-oid (the reference may exist already with a non-zero old-oid). Current "report-status" protocol cannot report the status for such reference rewrite. Add new capability "report-status-v2" and new report protocol which is not backward compatible for report of git-push. If a user pushes to a pseudo-reference "refs/for/master/topic", and "receive-pack" creates two new references "refs/changes/23/123/1" and "refs/changes/24/124/1", for client without the knowledge of "report-status-v2", "receive-pack" will only send "ok/ng" directives in the report, such as: ok ref/for/master/topic But for client which has the knowledge of "report-status-v2", "receive-pack" will use "option" directives to report more attributes for the reference given by the above "ok/ng" directive. ok refs/for/master/topic option refname refs/changes/23/123/1 option new-oid ok refs/for/master/topic option refname refs/changes/24/124/1 option new-oid The client will report two new created references to the end user. Suggested-by: Junio C Hamano Suggested-by: Jeff King Signed-off-by: Jiang Xin Signed-off-by: Junio C Hamano

receive-pack: add new proc-receive hook

2020-08-27T19:47:47Z

Git calls an internal `execute_commands` function to handle commands sent from client to `git-receive-pack`. Regardless of what references the user pushes, git creates or updates the corresponding references if the user has write-permission. A contributor who has no write-permission, cannot push to the repository directly. So, the contributor has to write commits to an alternate location, and sends pull request by emails or by other ways. We call this workflow as a distributed workflow. It would be more convenient to work in a centralized workflow like what Gerrit provided for some cases. For example, a read-only user who cannot push to a branch directly can run the following `git push` command to push commits to a pseudo reference (has a prefix "refs/for/", not "refs/heads/") to create a code review. git push origin \ HEAD:refs/for// The `` in the above example can be as simple as "master", or a more complicated branch name like "foo/bar". The `` in the above example command can be the local branch name of the client side, such as "my/topic". We cannot implement a centralized workflow elegantly by using "pre-receive" + "post-receive", because Git will call the internal function "execute_commands" to create references (even the special pseudo reference) between these two hooks. Even though we can delete the temporarily created pseudo reference via the "post-receive" hook, having a temporary reference is not safe for concurrent pushes. So, add a filter and a new handler to support this kind of workflow. The filter will check the prefix of the reference name, and if the command has a special reference name, the filter will turn a specific field (`run_proc_receive`) on for the command. Commands with this filed turned on will be executed by a new handler (a hook named "proc-receive") instead of the internal `execute_commands` function. We can use this "proc-receive" command to create pull requests or send emails for code review. Suggested by Junio, this "proc-receive" hook reads the commands, push-options (optional), and send result using a protocol in pkt-line format. In the following example, the letter "S" stands for "receive-pack" and letter "H" stands for the hook. # Version and features negotiation. S: PKT-LINE(version=1\0push-options atomic...) S: flush-pkt H: PKT-LINE(version=1\0push-options...) H: flush-pkt # Send commands from server to the hook. S: PKT-LINE(

) S: ... ... S: flush-pkt # Send push-options only if the 'push-options' feature is enabled. S: PKT-LINE(push-option) S: ... ... S: flush-pkt # Receive result from the hook. # OK, run this command successfully. H: PKT-LINE(ok ) # NO, I reject it. H: PKT-LINE(ng ) # Fall through, let 'receive-pack' to execute it. H: PKT-LINE(ok ) H: PKT-LINE(option fall-through) # OK, but has an alternate reference. The alternate reference name # and other status can be given in options H: PKT-LINE(ok ) H: PKT-LINE(option refname ) H: PKT-LINE(option old-oid ) H: PKT-LINE(option new-oid ) H: PKT-LINE(option forced-update) H: ... ... H: flush-pkt After receiving a command, the hook will execute the command, and may create/update different reference. For example, a command for a pseudo reference "refs/for/master/topic" may create/update different reference such as "refs/pull/123/head". The alternate reference name and other status are given in option lines. The list of commands returned from "proc-receive" will replace the relevant commands that are sent from user to "receive-pack", and "receive-pack" will continue to run the "execute_commands" function and other routines. Finally, the result of the execution of these commands will be reported to end user. The reporting function from "receive-pack" to "send-pack" will be extended in latter commit just like what the "proc-receive" hook reports to "receive-pack". Signed-off-by: Jiang Xin Signed-off-by: Junio C Hamano

argv-array: rename to strvec

2020-07-28T22:02:17Z

The name "argv-array" isn't very good, because it describes what the data type can be used for (program argument arrays), not what it actually is (a dynamically-growing string array that maintains a NULL-terminator invariant). This leads to people being hesitant to use it for other cases where it would actually be a good fit. The existing name is also clunky to use. It's overly long, and the name often leads to saying things like "argv.argv" (i.e., the field names overlap with variable names, since they're describing the use, not the type). Let's give it a more neutral name. I settled on "strvec" because "vector" is the name for a dynamic array type in many programming languages. "strarray" would work, too, but it's longer and a bit more awkward to say (and don't we all say these things in our mind as we type them?). A more extreme direction would be a generic data structure which stores a NULL-terminated of _any_ type. That would be easy to do with void pointers, but we'd lose some type safety for the existing cases. Plus it raises questions about memory allocation and ownership. So I limited myself here to changing names only, and not semantics. If we do find a use for that more generic data type, we could perhaps implement it at a lower level and then provide type-safe wrappers around it for strings. But that can come later. This patch does the minimum to convert the struct and function names in the header and implementation, leaving a few things for follow-on patches: - files retain their original names for now - struct field names are retained for now - there's a preprocessor compat layer that lets most users remain the same for now. The exception is headers which made a manual forward declaration of the struct. I've converted them (and their dependent function declarations) here. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

stateless-connect: send response end packet

2020-05-24T23:26:00Z

Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated before the transaction is complete, remote-curl will blindly forward the packets before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the transaction and continue reading, expecting more input to continue the transaction. This results in a deadlock between the two processes. This can be seen in the following command which does not terminate: $ git -c protocol.version=2 clone https://github.com/git/git.git --shallow-since=20151012 Cloning into 'git'... whereas the v1 version does terminate as expected: $ git -c protocol.version=1 clone https://github.com/git/git.git --shallow-since=20151012 Cloning into 'git'... fatal: the remote end hung up unexpectedly Instead of blindly forwarding packets, make remote-curl insert a response end packet after proxying the responses from the remote server when using stateless_connect(). On the RPC client side, ensure that each response ends as described. A separate control packet is chosen because we need to be able to differentiate between what the remote server sends and remote-curl's control packets. By ensuring in the remote-curl code that a server cannot send response end packets, we prevent a malicious server from being able to perform a denial of service attack in which they spoof a response end packet and cause the described deadlock to happen. Reported-by: Force Charlie Helped-by: Jeff King Signed-off-by: Denton Liu Signed-off-by: Junio C Hamano

remote: drop "explicit" parameter from remote_ref_for_branch()

2020-03-03T22:56:05Z

Commit 9700fae5ee (for-each-ref: let upstream/push report the remote ref name, 2017-11-07) added a remote_ref_for_branch() helper, which is modeled after remote_for_branch(). This includes providing an "explicit" out-parameter that tells the caller whether the remote was configured by the user, or whether we picked a default name like "origin". But unlike remote names, there is no default name when the user didn't configure one. The only way the "explicit" parameter is used by the caller is to use the value returned from the helper when it is set, and use an empty string otherwise, ignoring the returned value from the helper. Let's drop the "explicit" out-parameter, and return NULL when the returned value from the helper should be ignored, to simplify the function interface. Signed-off-by: Jeff King Signed-off-by: Damien Robert Signed-off-by: Junio C Hamano