Merge branch 'main' of github.com:git/git

* 'main' of github.com:git/git: (45 commits)
  A bit more of remaining topics before -rc1
  t1800: correct test to handle Cygwin
  chainlint: colorize problem annotations and test delimiters
  ls-files: fix black space in error message
  list-objects-filter: convert filter_spec to a strbuf
  list-objects-filter: add and use initializers
  list-objects-filter: handle null default filter spec
  list-objects-filter: don't memset after releasing filter struct
  builtin/mv.c: fix possible segfault in add_slash()
  Documentation/technical: include Scalar technical doc
  t/perf: add 'GIT_PERF_USE_SCALAR' run option
  t/perf: add Scalar performance tests
  scalar-clone: add test coverage
  scalar: add to 'git help -a' command list
  scalar: implement the `help` subcommand
  git help: special-case `scalar`
  scalar: include in standard Git build & installation
  scalar: fix command documentation section header
  t: retire unused chainlint.sed
  t/Makefile: teach `make test` and `make prove` to run chainlint.pl
  ...

5 files changed, 183 insertions, 73 deletions

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 849af6da30..12c533c1a3 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -21,6 +21,7 @@ MAN1_TXT += $(filter-out \
 MAN1_TXT += git.txt
 MAN1_TXT += gitk.txt
 MAN1_TXT += gitweb.txt
+MAN1_TXT += scalar.txt
 
 # man5 / man7 guides (note: new guides should also be added to command-list.txt)
 MAN5_TXT += gitattributes.txt
@@ -116,6 +117,7 @@ TECH_DOCS += technical/parallel-checkout
 TECH_DOCS += technical/partial-clone
 TECH_DOCS += technical/racy-git
 TECH_DOCS += technical/reftable
+TECH_DOCS += technical/scalar
 TECH_DOCS += technical/send-pack-pipeline
 TECH_DOCS += technical/shallow
 TECH_DOCS += technical/trivial-merge
diff --git a/Documentation/RelNotes/2.38.0.txt b/Documentation/RelNotes/2.38.0.txt
index 01617baa98..5d9bd8c295 100644
--- a/Documentation/RelNotes/2.38.0.txt
+++ b/Documentation/RelNotes/2.38.0.txt
@@ -6,7 +6,7 @@ UI, Workflows & Features
  * "git remote show [-n] frotz" now pays attention to negative
    pathspec.
 
- * "git push" sometimes perform poorly when reachability bitmaps are
+ * "git push" sometimes performs poorly when reachability bitmaps are
    used, even in a repository where other operations are helped by
    bitmaps.  The push.useBitmaps configuration variable is introduced
    to allow disabling use of reachability bitmaps only for "git push".
@@ -27,7 +27,7 @@ UI, Workflows & Features
    what locale they are in by sending Accept-Language HTTP header, but
    this was done only for some requests but not others.
 
- * Introduce a discovery.barerepository configuration variable that
+ * Introduce a safe.barerepository configuration variable that
    allows users to forbid discovery of bare repositories.
 
  * Various messages that come from the pack-bitmap codepaths have been
@@ -79,12 +79,15 @@ UI, Workflows & Features
 
  * "git format-patch --from=<ident>" can be told to add an in-body
    "From:" line even for commits that are authored by the given
-   <ident> with "--force-in-body-from"option.
+   <ident> with "--force-in-body-from" option.
 
  * The built-in fsmonitor refuses to work on a network mounted
    repositories; a configuration knob for users to override this has
    been introduced.
 
+ * The "scalar" addition from Microsoft is now part of the core Git
+   installation.
+
 
 Performance, Internal Implementation, Development Support etc.
 
@@ -127,7 +130,7 @@ Performance, Internal Implementation, Development Support etc.
 
  * The way "git multi-pack" uses parse-options API has been improved.
 
- * A coccinelle rule (in contrib/) to encourage use of COPY_ARRAY
+ * A Coccinelle rule (in contrib/) to encourage use of COPY_ARRAY
    macro has been improved.
 
  * API tweak to make it easier to run fuzz testing on commit-graph parser.
@@ -172,6 +175,12 @@ Performance, Internal Implementation, Development Support etc.
  * Share the text used to explain configuration variables used by "git
    <subcmd>" in "git help <subcmd>" with the text from "git help config".
 
+ * "git mv A B" in a sparsely populated working tree can be asked to
+   move a path from a directory that is "in cone" to another directory
+   that is "out of cone".  Handling of such a case has been improved.
+
+ * The chainlint script for our tests has been revamped.
+
 
 Fixes since v2.37
 -----------------
@@ -297,7 +306,7 @@ Fixes since v2.37
  * "git fsck" reads mode from tree objects but canonicalizes the mode
    before passing it to the logic to check object sanity, which has
    hid broken tree objects from the checking logic.  This has been
-   corrected, but to help exiting projects with broken tree objects
+   corrected, but to help existing projects with broken tree objects
    that they cannot fix retroactively, the severity of anomalies this
    code detects has been demoted to "info" for now.
 
@@ -306,12 +315,10 @@ Fixes since v2.37
 
  * An earlier optimization discarded a tree-object buffer that is
    still in use, which has been corrected.
-   (merge 1490d7d82d jk/is-promisor-object-keep-tree-in-use later to maint).
 
  * Fix deadlocks between main Git process and subprocess spawned via
    the pipe_command() API, that can kill "git add -p" that was
    reimplemented in C recently.
-   (merge 716c1f649e jk/pipe-command-nonblock later to maint).
 
  * The sequencer machinery translated messages left in the reflog by
    mistake, which has been corrected.
@@ -319,20 +326,16 @@ Fixes since v2.37
  * xcalloc(), imitating calloc(), takes "number of elements of the
    array", and "size of a single element", in this order.  A call that
    does not follow this ordering has been corrected.
-   (merge c4bbd9bb8f sg/xcalloc-cocci-fix later to maint).
 
  * The preload-index codepath made copies of pathspec to give to
    multiple threads, which were left leaked.
-   (merge 23578904da ad/preload-plug-memleak later to maint).
 
  * Update the version of Ubuntu used for GitHub Actions CI from 18.04
    to 22.04.
-   (merge ef46584831 ds/github-actions-use-newer-ubuntu later to maint).
 
  * The auto-stashed local changes created by "git merge --autostash"
    was mixed into a conflicted state left in the working tree, which
    has been corrected.
-   (merge d3a9295ada en/merge-unstash-only-on-clean-merge later to maint).
 
  * Multi-pack index got corrupted when preferred pack changed from one
    pack to another in a certain way, which has been corrected.
diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
index af5da45d28..9515a499a3 100755
--- a/Documentation/cmd-list.perl
+++ b/Documentation/cmd-list.perl
@@ -10,7 +10,7 @@ sub format_one {
 	$state = 0;
 	open I, '<', "$name.txt" or die "No such file $name.txt";
 	while (<I>) {
-		if (/^git[a-z0-9-]*\(([0-9])\)$/) {
+		if (/^(git|scalar)[a-z0-9-]*\(([0-9])\)$/) {
 			$mansection = $1;
 			next;
 		}
diff --git a/Documentation/scalar.txt b/Documentation/scalar.txt
new file mode 100644
index 0000000000..f33436c7f6
--- /dev/null
+++ b/Documentation/scalar.txt
@@ -0,0 +1,166 @@
+scalar(1)
+=========
+
+NAME
+----
+scalar - A tool for managing large Git repositories
+
+SYNOPSIS
+--------
+[verse]
+scalar clone [--single-branch] [--branch <main-branch>] [--full-clone] <url> [<enlistment>]
+scalar list
+scalar register [<enlistment>]
+scalar unregister [<enlistment>]
+scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]
+scalar reconfigure [ --all | <enlistment> ]
+scalar diagnose [<enlistment>]
+scalar delete <enlistment>
+
+DESCRIPTION
+-----------
+
+Scalar is a repository management tool that optimizes Git for use in large
+repositories. Scalar improves performance by configuring advanced Git settings,
+maintaining repositories in the background, and helping to reduce data sent
+across the network.
+
+An important Scalar concept is the enlistment: this is the top-level directory
+of the project. It usually contains the subdirectory `src/` which is a Git
+worktree. This encourages the separation between tracked files (inside `src/`)
+and untracked files, such as build artifacts (outside `src/`). When registering
+an existing Git worktree with Scalar whose name is not `src`, the enlistment
+will be identical to the worktree.
+
+The `scalar` command implements various subcommands, and different options
+depending on the subcommand. With the exception of `clone`, `list` and
+`reconfigure --all`, all subcommands expect to be run in an enlistment.
+
+The following options can be specified _before_ the subcommand:
+
+-C <directory>::
+	Before running the subcommand, change the working directory. This
+	option imitates the same option of linkgit:git[1].
+
+-c <key>=<value>::
+	For the duration of running the specified subcommand, configure this
+	setting. This option imitates the same option of linkgit:git[1].
+
+COMMANDS
+--------
+
+Clone
+~~~~~
+
+clone [<options>] <url> [<enlistment>]::
+	Clones the specified repository, similar to linkgit:git-clone[1]. By
+	default, only commit and tree objects are cloned. Once finished, the
+	worktree is located at `<enlistment>/src`.
++
+The sparse-checkout feature is enabled (except when run with `--full-clone`)
+and the only files present are those in the top-level directory. Use
+`git sparse-checkout set` to expand the set of directories you want to see,
+or `git sparse-checkout disable` to expand to all files (see
+linkgit:git-sparse-checkout[1] for more details). You can explore the
+subdirectories outside your sparse-checkout by using `git ls-tree
+HEAD[:<directory>]`.
+
+-b <name>::
+--branch <name>::
+	Instead of checking out the branch pointed to by the cloned
+	repository's HEAD, check out the `<name>` branch instead.
+
+--[no-]single-branch::
+	Clone only the history leading to the tip of a single branch, either
+	specified by the `--branch` option or the primary branch remote's
+	`HEAD` points at.
++
+Further fetches into the resulting repository will only update the
+remote-tracking branch for the branch this option was used for the initial
+cloning. If the HEAD at the remote did not point at any branch when
+`--single-branch` clone was made, no remote-tracking branch is created.
+
+--[no-]full-clone::
+	A sparse-checkout is initialized by default. This behavior can be
+	turned off via `--full-clone`.
+
+List
+~~~~
+
+list::
+	List enlistments that are currently registered by Scalar. This
+	subcommand does not need to be run inside an enlistment.
+
+Register
+~~~~~~~~
+
+register [<enlistment>]::
+	Adds the enlistment's repository to the list of registered repositories
+	and starts background maintenance. If `<enlistment>` is not provided,
+	then the enlistment associated with the current working directory is
+	registered.
++
+Note: when this subcommand is called in a worktree that is called `src/`, its
+parent directory is considered to be the Scalar enlistment. If the worktree is
+_not_ called `src/`, it itself will be considered to be the Scalar enlistment.
+
+Unregister
+~~~~~~~~~~
+
+unregister [<enlistment>]::
+	Remove the specified repository from the list of repositories
+	registered with Scalar and stop the scheduled background maintenance.
+
+Run
+~~~
+
+scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]::
+	Run the given maintenance task (or all tasks, if `all` was specified).
+	Except for `all` and `config`, this subcommand simply hands off to
+	linkgit:git-maintenance[1] (mapping `fetch` to `prefetch` and
+	`pack-files` to `incremental-repack`).
++
+These tasks are run automatically as part of the scheduled maintenance,
+as soon as the repository is registered with Scalar. It should therefore
+not be necessary to run this subcommand manually.
++
+The `config` task is specific to Scalar and configures all those
+opinionated default settings that make Git work more efficiently with
+large repositories. As this task is run as part of `scalar clone`
+automatically, explicit invocations of this task are rarely needed.
+
+Reconfigure
+~~~~~~~~~~~
+
+After a Scalar upgrade, or when the configuration of a Scalar enlistment
+was somehow corrupted or changed by mistake, this subcommand allows to
+reconfigure the enlistment.
+
+With the `--all` option, all enlistments currently registered with Scalar
+will be reconfigured. Use this option after each Scalar upgrade.
+
+Diagnose
+~~~~~~~~
+
+diagnose [<enlistment>]::
+    When reporting issues with Scalar, it is often helpful to provide the
+    information gathered by this command, including logs and certain
+    statistics describing the data shape of the current enlistment.
++
+The output of this command is a `.zip` file that is written into
+a directory adjacent to the worktree in the `src` directory.
+
+Delete
+~~~~~~
+
+delete <enlistment>::
+	This subcommand lets you delete an existing Scalar enlistment from your
+	local file system, unregistering the repository.
+
+SEE ALSO
+--------
+linkgit:git-clone[1], linkgit:git-maintenance[1].
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/technical/scalar.txt b/Documentation/technical/scalar.txt
index 0600150b3a..921cb104c3 100644
--- a/Documentation/technical/scalar.txt
+++ b/Documentation/technical/scalar.txt
@@ -64,64 +64,3 @@ some "global" `git` options (e.g., `-c` and `-C`).
 Because `scalar` is not invoked as a Git subcommand (like `git scalar`), it is
 built and installed as its own executable in the `bin/` directory, alongside
 `git`, `git-gui`, etc.
-
-Roadmap
--------
-
-NOTE: this section will be removed once the remaining tasks outlined in this
-roadmap are complete.
-
-Scalar is a large enough project that it is being upstreamed incrementally,
-living in `contrib/` until it is feature-complete. So far, the following patch
-series have been accepted:
-
-- `scalar-the-beginning`: The initial patch series which sets up
-  `contrib/scalar/` and populates it with a minimal `scalar` command that
-  demonstrates the fundamental ideas.
-
-- `scalar-c-and-C`: The `scalar` command learns about two options that can be
-  specified before the command, `-c <key>=<value>` and `-C <directory>`.
-
-- `scalar-diagnose`: The `scalar` command is taught the `diagnose` subcommand.
-
-- `scalar-generalize-diagnose`: Move the functionality of `scalar diagnose`
-  into `git diagnose` and `git bugreport --diagnose`.
-
-- 'scalar-add-fsmonitor: Enable the built-in FSMonitor in Scalar
-  enlistments. At the end of this series, Scalar should be feature-complete
-  from the perspective of a user.
-
-Roughly speaking (and subject to change), the following series are needed to
-"finish" this initial version of Scalar:
-
-- Move Scalar to toplevel: Move Scalar out of `contrib/` and into the root of
-  `git`. This includes a variety of related updates, including:
-    - building & installing Scalar in the Git root-level 'make [install]'.
-    - builing & testing Scalar as part of CI.
-    - moving and expanding test coverage of Scalar (including perf tests).
-    - implementing 'scalar help'/'git help scalar' to display scalar
-      documentation.
-
-Finally, there are two additional patch series that exist in Microsoft's fork of
-Git, but there is no current plan to upstream them. There are some interesting
-ideas there, but the implementation is too specific to Azure Repos and/or VFS
-for Git to be of much help in general.
-
-These still exist mainly because the GVFS protocol is what Azure Repos has
-instead of partial clone, while Git is focused on improving partial clone:
-
-- `scalar-with-gvfs`: The primary purpose of this patch series is to support
-  existing Scalar users whose repositories are hosted in Azure Repos (which does
-  not support Git's partial clones, but supports its predecessor, the GVFS
-  protocol, which is used by Scalar to emulate the partial clone).
-
-  Since the GVFS protocol will never be supported by core Git, this patch series
-  will remain in Microsoft's fork of Git.
-
-- `run-scalar-functional-tests`: The Scalar project developed a quite
-  comprehensive set of integration tests (or, "Functional Tests"). They are the
-  sole remaining part of the original C#-based Scalar project, and this patch
-  adds a GitHub workflow that runs them all.
-
-  Since the tests partially depend on features that are only provided in the
-  `scalar-with-gvfs` patch series, this patch cannot be upstreamed.