git/t/perf/run, branch jch

t/perf/run: preserve GIT_PERF_* from environment

2026-01-06T23:56:36Z

If you run: GIT_PERF_LARGE_REPO=/some/path ./p1006-cat-file.sh it will use the repo in /some/path. But if you use the "run" helper script to aggregate and compare results, like this: GIT_PERF_LARGE_REPO=/some/path ./run HEAD^ HEAD p1006-cat-file.sh it will ignore that variable. This is because the presence of the LARGE_REPO variable in GIT-BUILD-OPTIONS overrides what's in the environment. This started with 4638e8806e (Makefile: use common template for GIT-BUILD-OPTIONS, 2024-12-06), which now writes even empty variables (though arguably it was wrong even before with a non-empty value, as we generally prefer the environment to take precedence over on-disk config). We had the same problem in perf-lib.sh itself, and we hacked around it with 32b74b9809 (perf: do allow `GIT_PERF_*` to be overridden again, 2025-04-04). That's what lets the direct invocation of "./p1006" work above. And in fact that was sufficient for "./run", too, until it started loading GIT-BUILD-OPTIONS itself in 5756ccd181 (t/perf: fix benchmarks with out-of-tree builds, 2025-04-28). Now it has the same problem: it clobbers any incoming GIT_PERF options from the environment. We can use the same hack here in the "run" script. It's quite ugly, but it's just short enough that I don't think it's worth trying to factor it out into a common shell library. In the long run, we might consider teaching GIT-BUILD-OPTIONS to be more gentle in overwriting existing entries. There are probably other GIT_TEST_* variables which would need the same treatment. And if and when we come up with a more complete solution, we can use it in both spots. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

t/perf: use configured PERL_PATH

2025-04-28T20:13:51Z

Our benchmarks use a couple of Perl scripts to compute results. These Perl scripts get executed directly, and as the shebang is hardcoded to "/usr/bin/perl" this will fail on any system where the Perl interpreter is located in a different path. Our build infrastructure already lets users configure the location of Perl, which ultimately gets written into the GIT-BUILD-OPTIONS file. This file is being sourced by "test-lib.sh", and consequently we already have the "PERL_PATH" variable available that contains its configured location. Use "PERL_PATH" to execute Perl scripts, which makes them work on more esoteric systems like NixOS. Furthermore, adapt the shebang to use env(1) to execute Perl so that users who have Perl in PATH, but in a non-standard location can execute the script directly. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

global: convert trivial usages of `test -a/-o `

2023-11-11T00:21:00Z

Our coding guidelines say to not use `test` with `-a` and `-o` because it can easily lead to bugs. Convert trivial cases where we still use these to instead instead concatenate multiple invocations of `test` via `&&` and `||`, respectively. While not all of the converted instances can cause ambiguity, it is worth getting rid of all of them regardless: - It becomes easier to reason about the code as we do not have to argue why one use of `-a`/`-o` is okay while another one isn't. - We don't encourage people to use these expressions. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

t: convert egrep usage to "grep -E"

2022-09-21T18:00:18Z

Despite POSIX states that: > The old egrep and fgrep commands are likely to be supported for many > years to come as implementation extensions, allowing historical > applications to operate unmodified. GNU grep 3.8 started to warn[1]: > The egrep and fgrep commands, which have been deprecated since > release 2.5.3 (2007), now warn that they are obsolescent and should > be replaced by grep -E and grep -F. Prepare for their removal in the future. [1]: https://lists.gnu.org/archive/html/info-gnu/2022-09/msg00001.html Signed-off-by: Đoàn Trần Công Danh Signed-off-by: Junio C Hamano

t/perf: add 'GIT_PERF_USE_SCALAR' run option

2022-09-02T17:02:56Z

Add a 'GIT_PERF_USE_SCALAR' environment variable (and corresponding perf config 'useScalar') to register a repository created with any of: * test_perf_fresh_repo * test_perf_default_repo * test_perf_large_repo as a Scalar enlistment. This is intended to allow a developer to test the impact of Scalar on already-defined performance scenarios. Suggested-by: Derrick Stolee Signed-off-by: Victoria Dye Signed-off-by: Junio C Hamano

t/perf/run: fix bin-wrappers computation

2021-09-22T20:26:11Z

The GIT_TEST_INSTALLED was moved from perf-lib.sh to run in df0f5021 (perf-lib.sh: remove GIT_TEST_INSTALLED from perf-lib.sh, 2019-05-07) and that included a change to how it inspected the existence of a bin-wrappers directory. However, that included a typo that made the match of bin-wrappers never work. Specifically, the assignment was mydir_abs_wrappers="$mydir_abs_wrappers/bin-wrappers" which uses the same variable before it is initialized. By changing it to mydir_abs_wrappers="$mydir_abs/bin-wrappers" We can correctly use the bin-wrappers directory. This is critical to successfully computing performance of commands that execute subcommands. The bin-wrappers ensure that the --exec-path is set correctly. Reported-by: Victoria Dye Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

perf: fix when running with TEST_OUTPUT_DIRECTORY

2021-07-02T22:47:30Z

When the TEST_OUTPUT_DIRECTORY is defined, then all test data will be written in that directory instead of the default directory located in "t/". While this works as expected for our normal tests, performance tests fail to locate and aggregate performance data because they don't know to handle TEST_OUTPUT_DIRECTORY correctly and always look at the default location. Fix the issue by adding a `--results-dir` parameter to "aggregate.perl" which identifies the directory where results are and by making the "run" script awake of the TEST_OUTPUT_DIRECTORY variable. Signed-off-by: Patrick Steinhardt Signed-off-by: Junio C Hamano

perf-lib.sh: forbid the use of GIT_TEST_INSTALLED

2019-05-08T02:00:28Z

As noted in preceding commits setting GIT_TEST_INSTALLED has never been supported or documented, and as noted in an earlier t/perf/README change to the extent that it's been documented nobody's notices that the example hasn't worked since 3c8f12c96c ("test-lib: reorder and include GIT-BUILD-OPTIONS a lot earlier", 2012-06-24). We could directly support GIT_TEST_INSTALLED for invocations without the "run" script, such as: GIT_TEST_INSTALLED=../../ ./p0000-perf-lib-sanity.sh GIT_TEST_INSTALLED=/home/avar/g/git ./p0000-perf-lib-sanity.sh But while not having this "error" will "work", it won't write the the resulting "test-results/*" files to the right place, and thus a subsequent call to aggregate.perl won't work as expected. Let's just tell the user that they need to use the "run" script, which'll correctly deal with this and set the right PERF_RESULTS_PREFIX. If someone's in desperate need of bypassing "run" for whatever reason they can trivially do so by setting "PERF_SET_GIT_TEST_INSTALLED", but not we won't have people who expect GIT_TEST_INSTALLED to just work wondering why their aggregation doesn't work, even though they're running the right "git". Signed-off-by: Ævar Arnfjörð Bjarmason

perf tests: add "bindir" prefix to git tree test results

2019-05-08T02:00:28Z

Change the output file names in test-results/ to be "test-results/bindir_" rather than just "test-results/". This is for consistency with the "build_" directories we have for built revisions, i.e. "test-results/build_". There's no user-visible functional changes here, it just makes it easier to see at a glance what "test-results" files are of what "type" as they're all explicitly grouped together now, and to grep this code to find both the run_dirs_helper() implementation and its corresponding aggregate.perl code. Note that we already guarantee that the rest of the PERF_RESULTS_PREFIX is an absolute path, and since it'll start with e.g. "/" which we munge to "_" we'll up with a readable string like "bindir_home_avar[...]". Signed-off-by: Ævar Arnfjörð Bjarmason

perf-lib.sh: remove GIT_TEST_INSTALLED from perf-lib.sh

2019-05-08T02:00:28Z

Follow-up my preceding change which fixed the immediate "./run " regression in 0baf78e7bc ("perf-lib.sh: rely on test-lib.sh for --tee handling", 2019-03-15) and entirely get rid of GIT_TEST_INSTALLED from perf-lib.sh (and aggregate.perl). As noted in that change the dance we're doing with GIT_TEST_INSTALLED perf-lib.sh isn't necessary, but there I was doing the most minimal set of changes to quickly fix a regression. But it's much simpler to never deal with the "GIT_TEST_INSTALLED" we were setting in perf-lib.sh at all. Instead the run_dirs_helper() sets the previously inferred $PERF_RESULTS_PREFIX directly. Setting this at the callsite that's already best positioned to exhaustively know about all the different cases we need to handle where PERF_RESULTS_PREFIX isn't what we want already (the empty string) makes the most sense. In one-off cases like: ./run ./p0000-perf-lib-sanity.sh ./p0000-perf-lib-sanity.sh We'll just do the right thing because PERF_RESULTS_PREFIX will be empty, and test-lib.sh takes care of finding where our git is. Any refactoring of this code needs to change both the shell code and the Perl code in aggregate.perl, because when running e.g.: ./run ../../ -- The "../../" path to a relative bindir needs to be munged to a filename containing the results, and critically aggregate.perl does not get passed the path to those aggregations, just "../..". Let's fix cases where aggregate.perl would print e.g. ".." in its report output for this, and "git" for "/home/avar/g/git", i.e. it would always pick the last element. Now'll always print the full path instead. This also makes the code sturdier, e.g. you can feed "../.." to "./run" and then an absolute path to the aggregate.perl script, as long as the absolute path and "../.." resolved to the same directory printing the aggregation will work. Also simplify the "[_*]" on the RHS of "tr -c", we're trimming everything to "_", so we don't need that. Signed-off-by: Ævar Arnfjörð Bjarmason