| Age | Commit message (Collapse) | Author | Lines |
|
A good reference for the concepts used here is:
https://mazzo.li/posts/fast-pipes.html
We don't consider huge pages or busy loops here,
but use vmsplice(), and splice() to get significant speedups:
i7-5600U-laptop $ taskset 1 yes | taskset 2 pv > /dev/null
... [4.98GiB/s]
i7-5600U-laptop $ taskset 1 src/yes | taskset 2 pv > /dev/null
... [34.1GiB/s]
IBM,9043-MRX $ taskset 1 yes | taskset 2 pv > /dev/null
... [11.6GiB/s]
IBM,9043-MRX $ taskset 1 src/yes | taskset 2 pv > /dev/null
... [175GiB/s]
Also throughput to file (on BTRFS) was seen to increase significantly.
With a Fedora 43 laptop improving from 690MiB/s to 1.1GiB/s.
* bootstrap.conf: Ensure sys/uio.h is present.
This was an existing transitive dependency.
* m4/jm-macros.m4: Define HAVE_SPLICE appropriately.
We assume vmsplice() is available if splice() is as they
were introduced at the same time to Linux and glibc.
* src/yes.c (repeat_pattern): A new function to efficiently
duplicate a pattern in a buffer with memcpy calls that double in size.
This also makes the setup for the existing write() path more efficient.
(pipe_splice_size): A new function to increase the kernel pipe buffer
if possible, and use an appropriately sized buffer based on that (25%).
(splice_write): A new function to call vmplice() when outputting
to a pipe, and also splice() if outputting to a non-pipe.
* tests/misc/yes.sh: Verify the non-pipe output case,
(main): Adjust to always calling write on the minimal buffer first,
then trying vmsplice(), then falling back to write from bigger buffer.
and the vmsplice() fallback to write() case.
* NEWS: Mention the improvement.
|
|
* tests/misc/warning-errors.sh: Simply check there is output
to stderr before checking that output induces an error.
|
|
* tests/misc/warning-errors.sh: Skip commands which have been built with
sanitizers, since standard error will not be closed and checked for
errors.
Reported by Bruno Haible.
|
|
* tests/misc/warning-errors.sh: Expect 'wc' and 'cksum' to exit
successfully if there is not an optimized 'wc -l' implementation or
CRC32 implementation.
Reported by Bruno Haible.
|
|
* NEWS: Mention the improvement.
* src/id.c (print_stuff): Call fflush for each listed user to check for
write errors.
* tests/misc/io-errors.sh: Add an invocation of 'id'.
|
|
* NEWS: Mention the improvement.
* src/groups.c (main): Call fflush for each listed user to check for
write errors.
* tests/misc/io-errors.sh: Add an invocation of 'groups'.
|
|
* tests/misc/warning-errors.sh: Add a new test to ensure
failure to write warnings is diagnosed in the exit status.
* tests/local.mk: Reference the new test.
|
|
* tests/misc/io-errors.sh: Promptness is checked in
write-errors.sh, not this test.
|
|
* tests/df/no-mtab-status-masked-proc.sh: Tweak unshare check.
* tests/local.mk: Reference new test.
* tests/misc/user.sh: Add new test using unshare -U, to verify
that whoami and logname diagnose failure correctly.
https://github.com/coreutils/coreutils/pull/195
|
|
* tests/misc/write-errors.sh: Add a shuf invocation that would run
forever if write errors weren't diagnosed.
|
|
* tests/csplit/csplit-io-err.sh: Adjust determination of ENOSPC.
* tests/csplit/split-io-err.sh: Likewise.
* tests/fold/fold-zero-width.sh: Verify full ENOSPC message.
* tests/tac/tac-continue.sh: Likewise.
* tests/misc/io-errors.sh: Likewise.
* tests/misc/write-errors.sh: Likewise.
|
|
* tests/head/head-write-error.sh: Verify ENOSPC is output.
* tests/misc/yes.sh: Likewise.
|
|
* tests/chgrp/no-x.sh: Don't harcode "Permission denied".
* tests/chmod/no-x.sh: Likewise.
* tests/du/inacc-dest.sh: Likewise.
* tests/du/no-x.sh: Likewise.
* tests/misc/nohup.sh: Likewise.
* tests/mv/part-fail.sh: Likewise.
* tests/mv/perm-1.sh: Likewise.
* tests/rm/fail-eacces.sh: Likewise.
* tests/rm/rm1.sh: Likewise.
* tests/rm/rm2.sh: Likewise.
* tests/rm/unread2.sh: Likewise.
* tests/rm/unreadable.pl: Likewise.
|
|
* tests/rm/inaccessible.sh: Remove AIX specific handling.
* tests/cp/fail-perm.sh: Likeise. Also remove confusing
EPERM/HPUX mentions.
* tests/misc/sync.sh: Don't hardcode EISDIR and EACCES.
|
|
* tests/misc/coreutils.sh: Remove the check for non-empty coreutils.h,
as the existing `print_ver_ coreutils` is sufficient to determine
support for single binary mode.
https://github.com/coreutils/coreutils/pull/189
|
|
* tests/misc/arch.sh: Ensure -- to indicate end of options is accepted.
https://github.com/coreutils/coreutils/pull/191
|
|
* tests/misc/read-errors.sh: Use getlimits_ determined EIO error string,
rather than inferring the string from bash's output.
See https://bugs.gnu.org/80353
|
|
* tests/misc/usage_vs_refs.sh: Remove copy and paste slop.
* tests/local.mk: Fix indentation.
|
|
* NEWS: Mention the improvement.
* src/nl.c: Include mcel.h.
(DEFAULT_SECTION_DELIMITERS): Resize to fit 2 multi-byte characters.
(section_del_len): New variable.
(check_section): Compare against section_del_len instead of 2.
(main): Support multi-byte characters for the -d option.
* tests/nl/multibyte.sh: New file.
* tests/nl/nl.sh: New file, moved from tests/misc/nl.sh.
* tests/local.mk (all_tests): Add the new test. Adjust the existing
tests file name.
* cfg.mk (exclude_file_name_regexp--sc_space_tab): Adjust Adjust the
existing tests file name.
|
|
* tests/misc/read-errors.sh: Avoid dash or ksh as they
do _not_ print an error message upon EIO from read.
Fixes https://bugs.gnu.org/80353
|
|
* tests/misc/nl.sh: Add a test case.
|
|
* tests/misc/sync.sh: Increase the timeout to 60s,
and restrict the case to when RUN_VERY_EXPENSIVE_TESTS=yes.
|
|
* tests/misc/usage_vs_refs.sh: Fix missing escape on literal {,
required by busybox grep.
Reported by Bruno Haible.
|
|
* tests/misc/usage_vs_getopt.sh: Fix false failure induced
by commit v9.9-239-ga06f0cf67
|
|
This is a more standard mechanism to disable markup.
* src/system.h (oputs_): Logic change to honor TERM=dumb,
rather than HELP_NO_MARKUP=something.
* doc/coreutils.texi: Adjust the description for --help.
* man/local.mk: Ensure TERM is set to something,
so that man pages have links included.
* man/viewman: Just honor users $TERM.
* tests/misc/getopt_vs_usage.sh: Remove env var complication,
as TERM is unset automatically.
* tests/misc/usage_vs_refs.sh: Likewise.
* NEWS: Adjust the change in behavior note.
|
|
* tests/misc/usage_vs_refs.sh: Properly quote arguments
to avoid false failure with paths containing spaces etc.
|
|
* tests/misc/mknod.sh: Skip a portion of the test
in the edge case of setgid directories.
|
|
* tests/misc/usage_vs_refs.sh: A new test to correlation options
listed in --help, and anchors referenced in the texinfo (html) manual.
* tests/local.mk: Add the new test.
|
|
* src/ls.c (oputs): A new function that wraps puts(),
but also highlights the --option-text portion, and
adds links to the appropriate part of the online manual.
(usage): Call oputs() rather than puts().
* doc/coreutils.texi (--help): Document new HELP_NO_MARKUP env var,
which can be used in the edge case one wants to suppress ansi escapes.
* tests/misc/getopt_vs_usage.sh: Use HELP_NO_MARKUP to ensure the
test continues to pass.
|
|
* tests/misc/sync.sh: Add the test case.
|
|
* tests/misc/sync.sh: Check 'sync fifo' with no arguments, -d, and -f
separately. Only perform the check for -f if RUN_EXPENSIVE_TESTS=yes
since it may be expensive.
|
|
* tests/cksum/cksum-c.sh: Add 'splice' to the injection set.
* tests/misc/read-errors.sh: Likewise.
From https://github.com/coreutils/coreutils/pull/167
|
|
* tests/misc/coreutils.sh: Be more flexibile by using `command -v`
to identify the binary to be executed, rather than linking
directly to the built binary.
|
|
* init.cfg (uses_strace_): New function.
(require_strace_): Use it.
tests/cksum/cksum-c.sh: Likewise.
tests/misc/read-errors.sh: Likewise.
|
|
|
|
Setup
$ ln -nsf src/coreutils foo
Before
$ ./foo; echo $?
foo: unknown program ‘foo’
Try './foo --help' for more information.
1
./foo --version; echo $?
coreutils (GNU coreutils) 9.9.172-01993
0
After
$ ./foo; echo $?
coreutils: unknown program 'foo'
1
$ ./foo --version; echo $?
coreutils: unknown program 'foo'
1
* src/coreutils.c (main): Don't process options if
we don't know they're intended for the multi-call binary.
Otherwise `foo --version` would return true, even though
foo was symlinked to the multi-call binary, but not supported.
* tests/misc/coreutils.sh: Add test cases.
* NEWS: Mention the change in behavior.
|
|
* tests/misc/read-errors.sh: Ensure each utility calls a
read system call as requested. Not doing so (say by avoiding
reading from a directory) is a layering violation.
Also ensure we diagnose the particular error encountered.
|
|
* src/paste.c (collapse_escapes): This is the central --delimiters
parsing function, so adjust to handle multi-byte chars with
mcel_scanz(). Populate a delimiters length array to support
characters of differing lengths.
(paste_serial): Use the delimiters length array to output
the appropriate delimiter.
(paste_parallel): Likewise.
* tests/paste/multi-byte.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
|
|
* tests/misc/paste.pl: Test all supported POSIX and GNU
backslash escape sequences.
|
|
* tests/misc/read-errors.sh: Add a test case.
Discussed in https://github.com/coreutils/coreutils/pull/167
|
|
* src/cksum.c (output_file, digest_check): Check if standard output has
it's error flag set after printing.
* tests/misc/write-errors.sh: Add a test case that would previously run
forever.
* NEWS: Mention the improvement. Reorder alphabetically.
|
|
* tests/misc/write-errors.sh: This was a no-op anyway
due to inadequate escaping. Also document the escaping requirement.
|
|
* src/du.c (print_size): Call write_error() if can't flush.
* tests/misc/write-errors.sh: Add a test case.
* NEWS: Mention the improvement.
|
|
* src/wc.c (write_counts): Call write_error() if any pending errors.
* tests/misc/write-errors.sh: Add a test case.
* NEWS: Mention the improvement.
|
|
* tests/misc/mktemp.pl: Moved to tests/mktemp/mktemp.pl.
* tests/local.mk (all_tests): Adjust the file name.
|
|
|
|
* tests/misc/kill.sh: Call getlimits_. Check for RTMIN and RTMAX in the
output of 'kill -l' if the system defines SIGRTMIN and SIGRTMAX.
|
|
* tests/misc/io-errors.sh: Verify that all commands diagnose write
errors, and handle a closed pipe appropriately.
* tests/local.mk: Reference the new test.
|
|
* tests/misc/tee.sh: Moved to tests/tee/tee.sh.
* tests/local.mk (all_tests): Adjust the file name.
|
|
* tests/misc/expand.pl: Move to tests/expand/expand.pl.
* tests/misc/unexpand.pl: Move to tests/unexpand/unexpand.pl.
* tests/local.mk (all_tests): Adjust file names.
|