coreutils/doc, branch v9.2

split: support split -n on larger pipe input

2023-03-07T21:41:46Z

* bootstrap.conf (gnulib_modules): Add free-posix, tmpfile. * src/split.c (copy_to_tmpfile): New function. (input_file_size): Use it to split larger files when sizes cannot easily be determined via fstat or lseek. See Bug#61386#235. * tests/split/l-chunk.sh: Mark tests of /dev/zero as very expensive since they exhaust /tmp.

du: --apparent counts only symlinks and regular

2023-03-05T00:57:16Z

Problem reported by Christoph Anton Mitterer (Bug#61884). * src/du.c (process_file): When counting apparent sizes, count only usable st_size members. * tests/du/apparent.sh: New file. * tests/local.mk (all_root_tests): Add it.

split: split more evenly with -n

2023-03-04T22:49:45Z

* src/split.c (bytes_split): New arg REM_BYTES. Use this to split more evenly. All callers changed. (lines_chunk_split, bytes_chunk_extract): Be consistent with new byte_split. * tests/split/b-chunk.sh, tests/split/l-chunk.sh: Test new behavior.

doc: tee -p: clarify operation

2023-02-28T14:02:42Z

* src/tee.c (usage): Change from describing one (non pipe) aspect to the more general point of being the option to use if working with pipes, and referencing the more detailed info below. * doc/coreutils.texi (tee invocation): s/standard/appropriate/ since the standard operation with pipes is to exit immediately upon write error. s/early/immediately/ as it's ambiguous as to what "early" is in relation to.

tee: enhance -p mode using iopoll() to detect broken pipe outputs

2023-02-28T14:02:42Z

If input is intermittent (a tty, pipe, or socket), and all remaining outputs are pipes (eg, >(cmd) process substitutions), exit early when they have all become broken pipes (and thus future writes will fail), without waiting for more input to become available, as future write attempts to these outputs will fail (SIGPIPE/EPIPE). Only provide this enhancement when pipe errors are ignored (-p mode). Note that only one output needs to be monitored at a time with iopoll(), as we only want to exit early if _all_ outputs have been removed. * src/tee.c (pipe_check): New global for iopoll mode. (main): enable pipe_check for -p, as long as output_error ignores EPIPE, and input is suitable for iopoll(). (get_next_out): Helper function for finding next valid output. (fail_output, tee_files): Break out write failure/output removal logic to helper function. (tee_files): Add out_pollable array to track which outputs are suitable for iopoll() (ie, that are pipes); track first output index that is still valid; add iopoll() broken pipe detection before calling read(), removing an output that becomes a broken pipe. * src/local.mk (src_tee_SOURCES): include src/iopoll.c. * NEWS: Mention tee -p enhancement in Improvements. * doc/coreutils.texi: Mention the new early exit behavior in the nopipe modes for the tee -p option. Suggested-by: Arsen Arsenović

cp,install,mv: add --debug to explain how a file is copied

2023-02-24T00:35:18Z

How a file is copied is dependent on the sparseness of the file, what file system it is on, what file system the destination is on, the attributes of the file, and whether they're being copied or not. Also the --reflink and --sparse options directly impact the operation. Given it's hard to reason about the combination of all of the above, the --debug option is useful for users to directly identify if copy offloading, reflinking, or sparse detection are being used. It will also be useful for tests to directly query if these operations are supported. The new output looks as follows: $ src/cp --debug src/cp file.sparse 'src/cp' -> 'file.sparse' copy offload: yes, reflink: unsupported, sparse detection: no $ truncate -s+1M file.sparse $ src/cp --debug file.sparse file.sparse.cp 'file.sparse' -> 'file.sparse.cp' copy offload: yes, reflink: unsupported, sparse detection: SEEK_HOLE $ src/cp --reflink=never --debug file.sparse file.sparse.cp 'file.sparse' -> 'file.sparse.cp' copy offload: avoided, reflink: no, sparse detection: SEEK_HOLE * doc/coreutils.texi (cp invocation): Describe the --debug option. (mv invocation): Likewise. (install invocation): Likewise. * src/copy.h: Add a new DEBUG member to cp_options, to control whether to output debug info or not. * src/copy.c (copy_debug): A new global structure to unconditionally store debug into from the last copy_reg operations. (copy_debug_string, emit_debug): New functions to print debug info. * src/cp.c: if ("--debug") x->debug=true; * src/install.c: Likewise. * src/mv.c: Likewise. * tests/cp/debug.sh: Add a new test. * tests/local.mk: Reference the new test. * NEWS: Mention the new feature.

doc: fix some spelling mistakes

2023-02-21T16:22:20Z

* doc/coreutils.texi: s/functionalty/functionality/, s/sychronize/synchronize/, s/millsecond/millisecond/ s/paramter/parameters/ * init.cfg: s/parmeters/parameters/ * scripts/build-older-versions/README.older-versions: s/vesion/version/ * tests/misc/env-S-script.sh: s/paramaters/parameters/ Fixes https://bugs.gnu.org/61681

cksum: add --raw option to output a binary digest

2023-02-06T13:09:40Z

--raw output is the most composable format, and also is a robust way to discard the file name without parsing (escaped) output. Examples: $ cksum --raw -a crc "$afile" | basenc --base16 4ACFC4F0 $ cksum --raw -a crc "$afile" | basenc --base2msbf 01001010110011111100010011110000 $ cksum --raw -a sha256 "$bfile" | basenc --base32 AAAAAAAADHLGRHAILLQWLAY6SNH7OY5OI2RKNQLSWPY3MCUM4JXQ==== * doc/coreutils.texi (cksum invocation): Describe the new feature. * src/digest.c (output_file): Inspect the new RAW_DIGEST global, and output the bytes directly if set. * src/cksum.c (output_crc): Likewise. * src/sum.c (output_bsd, output_sysv): Likewise. * tests/misc/cksum-raw.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the new feature.

cksum: accept new option: --base64 (-b)

2023-02-01T02:24:28Z

* src/digest.c [HASH_ALGO_CKSUM]: Include "base64.h" [HASH_ALGO_CKSUM] (base64_digest): New global. [HASH_ALGO_CKSUM] (enum BASE64_DIGEST_OPTION): New enum. [HASH_ALGO_CKSUM] (long_options): Add "base64". (valid_digits): Rename from hex_digits, now taking an input length argument. Adjust callers. (bsd_split_3): Rename arg from hex_digits to digest. Add new *d_len parameter for length of extracted digest. Move "i" declaration down to first use. (split_3): Rename arg from hex_digits to digest. Add new *d_len parameter for length of extracted digest. Instead of relying on "known" length of digest to find the following must-be-whitespace byte, search for the first whitespace byte. [HASH_ALGO_CKSUM] (output_file): Handle base64_digest. [HASH_ALGO_CKSUM] (main): Set base64_digest. [HASH_ALGO_CKSUM] (b64_equal): New function. (hex_equal): New function, factored out of digest_check. (digest_check) Factored part into b64_equal and hex_equal. Rename local hex_digest to digest. * tests/misc/cksum-base64.pl: Add tests. * tests/local.mk (all_tests): Add to the list. * cfg.mk (_cksum): Define. (exclude_file_name_regexp--sc_prohibit_test_backticks): Exempt new test. (exclude_file_name_regexp--sc_long_lines): Likewise. * doc/coreutils.texi (cksum invocation): Document it. (md5sum invocation) [--check]: Mention digest encoding auto-detect. * NEWS (New Features): Mention this.

doc: document --preserve=mode better

2023-01-31T21:53:58Z

* doc/coreutils.texi: Spruce up cp --preserve=mode doc.