summaryrefslogtreecommitdiffstats
path: root/src
AgeCommit message (Collapse)AuthorLines
2021-09-16cksum: fix off-by-1 bug with \r strippingPaul Eggert-4/+3
Problem reported by Jim Meyering (Bug#50611). * src/digest.c (digest_check): When stripping trailing \r, avoid subscript error before start of line.
2021-09-15maint: prefer rawmemchr to memchr when easyPaul Eggert-24/+26
* bootstrap.conf (gnulib_modules): Add rawmemchr. * src/csplit.c: Include idx.h. * src/csplit.c (record_line_starts): * src/head.c (elide_tail_lines_pipe): * src/shuf.c (next_line): * src/split.c (lines_split): * src/tail.c (pipe_lines): * src/wc.c (wc_lines): Prefer rawmemchr to memchr when rawmemchr is easy. * src/csplit.c (load_buffer): * src/head.c (struct linebuffer): Make room for a 1-byte sentinel.
2021-09-15split: avoid NULL + 1Paul Eggert-2/+5
* src/split.c (lines_chunk_split): Don’t add to a null pointer. It’s undefined behavior, and it’s unnecessarily confusing regardless.
2021-09-15digest: support windows format checksum filesPádraig Brady-20/+25
Support checksum files with CRLF line endings, which is a common gotcha for using --check on windows, or with checksum files generated on windows. Note we escape \r here to support the original coreutils format (with file name at EOL), and file names with literal \r characters as the last character of their name. * src/digest.c (filename_unescape): Convert \\r -> \r. (print_filename): Escape \r -> \\r. (output_file): Detect \r chars in file names. (digest_check): Ignore literal \r char at EOL. * tests/misc/md5sum.pl: Add a test case. * tests/misc/sha1sum.pl: Likewise. * NEWS: Mention the improvement.
2021-09-15doc: improve --help indenting in checksum utilsPádraig Brady-2/+2
* src/digest.c (usage): Indent multi-line descriptions for clarity.
2021-09-15cksum: operate in binary mode onlyPádraig Brady-14/+28
This only practically matters on windows. But given there are separate text handling options in cygwin, keep the interface simple, and avoid exposing the confusing binary/text difference here. * doc/coreutils.texi (md5sum invocation): Mention that --binary and --text are not supported by the cksum command. * src/digest.c: Set flag to use binary mode by default. (output_file): Don't distinguish text and binary modes with ' ' and '*', and just use ' ' always.
2021-09-15cksum: use --tag format by defaultPádraig Brady-4/+30
This format is a better default, since it results in simpler usage, as you don't need to specify --tag on generation or -a on checking invocations. Also it's a more general format supporting mixed and length adjusted digests. * doc/coreutils.texi (cksum invocation): Document a new --untagged option, to use the older coreutils format. (md5sum invocation): Mention that cksum doesn't support --tag. * src/digest.c: Adjust cksum(1) to default to --tag, and accept the new --untagged option. * tests/misc/b2sum.sh: Adjust accordingly. * tests/misc/cksum-a.sh: Likewise. * tests/misc/cksum-c.sh: Likewise.
2021-09-15cksum: support --zero in default modePádraig Brady-19/+24
* src/cksum.h: Thread DELIM through the output functions. * src/digest.c: Likewise. * src/sum.c: Likewise. * src/sum.h: Likewise. * src/cksum.c: Likewise. Also adjust check to allow -z with traditional output modes. Also ajust the global variable name to avoid shadowing warnings. * tests/misc/cksum-a.sh: Adjust accordingly.
2021-09-15digest: support -length specifiers on all digest tagsPádraig Brady-41/+31
This will be generally useful going forward, for sha3-256 etc. * src/digest.c: Rename b2_length to digest_length, and adjust/simplify the code to operate on this for both b2sum and cksum -a blake2b.
2021-09-15cksum: support digest detection for tagged formatPádraig Brady-3/+66
Support `cksum --check FILE` without having to specify a digest algorithm, allowing for more generic file check instructions. This also supports mixed digest checksum files, supporting more robust multi digest checks. * src/digest.c (algorithm_from_tag): A new function to identify the digest algorithm from a tagged format line. (split3): Set the algorithm depending on tag, and update the expected digest length accordingly. * tests/misc/cksum-c.sh: Add a new test. * tests/local.mk: Reference the new test. * tests/misc/md5sum.pl: Adjust to more generic error. * tests/misc/sha1sum.pl: Likewise. * doc/coreutils.texi (md5sum invocation): Mention the new -c feature. * NEWS: Mention the new feature.
2021-09-15maint: simplify b2sum to only handle BLAKE2bPádraig Brady-41/+14
Any further variants will use the cksum -a table driven mechanism. * src/digest.c: Remove BLAKE2 specific table driven code.
2021-09-15digest: add support for sm3Pádraig Brady-4/+16
Add message digest sm3, which uses the OSCCA SM3 secure hash (OSCCA GM/T 0004-2012 SM3) generic hash transformation. * bootstrap.conf: Add the sm3 module. * doc/coreutils.texi: Mention the cksum -a option. * src/digest.c: Provide support for --algorithm='sm3'. * tests/misc/sm3sum.pl: Add a new test (from Tianjia Zhang) * tests/local.mk: Reference the new test. * NEWS: Mention the new feature. Tested-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
2021-09-15cksum: add --algorithm option to select digest modePádraig Brady-60/+269
* src/digest.c: Organize HASH_ALGO_CKSUM to be table driven, and amalgamate all digest algorithms. (main): Parse all options if HASH_ALGO_CKSUM, and disallow --tag, --zero, and --check with the traditional bsd, sysv, and crc checksums for now. * src/local.mk: Reorganize to include all digest modules in cksum. * tests/misc/cksum-a.sh: Add a new test. * tests/misc/b2sum.sh: Update to default to checking with cksum, as b2sum's implementation diverges a bit from the others. * tests/local.mk: Reference the new test. * doc/coreutils.texi (cksum invocation): Adjust the summary to identify the new mode, and document the new --algorithm option. * man/cksum.x: Adjust description to be more general. * man/*sum.x: Add [See Also] section referencing cksum(1). * NEWS: Mention the new feature.
2021-09-15digest: refactor cksum(1) into digest.cPádraig Brady-627/+556
* cfg.mk: Adjust cksum.c to not require config.h and support a main (for crctab) without calling bindtextdomain(). * po/POTFILES.in: Remove cksum_pclmul.c since it no longer concerns itself with diagnostics. * src/cksum.c: Refactor to just providing stream digest, and digest printing functionality. * src/cksum.h: Adjust to the new interface. * src/cksum_pclmul.c: Remove diagnostics, and determine errors internally. * src/crctab.c: Separate from cksum.h since that's now included multiple times. * src/digest.c: Provide cksum(1) functionality if -DHASH_ALGO_CKSUM * src/local.mk: Adjust to new crctab.c and HASH_ALGO_CKSUM define.
2021-09-15cksum: document the --debug optionPádraig Brady-0/+6
This should have been part of commit v8.32-113-gb73b9fcb1 * doc/coreutils.texi (cksum invocation): Add the --debug description. * src/cksum.c (usage): Likewise. (main): Also give explicit indication when using generic hardware.
2021-09-15sum: handle EOVERFLOW for too large inputsPádraig Brady-0/+20
* src/sum.c (bsd_sum_stream): Detect overflow when updating length. (sysv_sum_stream): Likewise.
2021-09-15digest: refactor sum(1) into digest.cPádraig Brady-253/+272
Since digest will be providing all digest functionality, refactor sum.c into it. * po/POTFILES.in: sum.c no longer has translatable strings so remove. * src/digest.c: Call out to new stream interfaces in sum.c * src/local.mk: Adjust sources for the sum binary. * src/sum.c: Provide a stream interface for BSD and SYSV digests. * src/sum.h: A new file to declare the exported functions in sum.c
2021-09-15digest: add LENGTH parameter to digest to support cksumPádraig Brady-3/+3
* src/digest.c (digest_file): Add a LENGTH param, to support cksum(1), and sum(1) which output the length as part of their output.
2021-09-15maint: rename md5sum.c to more general digest.cPádraig Brady-6/+7
md5sum.c will be the base for all digest functions, so rename accordingly. * src/md5sum.c: Rename to ... * src/digest.c: ... renamed from md5sum.c * scripts/git-hooks/commit-msg: Allow digest: commit prefix. * po.POTFILES.in: Adjust to new name. * src/local.mk: Likewise.
2021-09-15doc: fix ambiguities in logname(1) and whoami(1)Pádraig Brady-1/+1
* doc/coreutils.texi (whoami invocation): Clarify it prints names, not numeric IDs. * man/whoami.x: Likewise. * man/logname.x: Reference getlogin(3). * src/logname.c: Clarify that it prints the login name, rather than the name of the effective user ID. Fixes https://bugs.gnu.org/48894
2021-09-15dircolors: add *direct* to TERM matchingnl6720-0/+1
Search for "direct color" at: https://invisible-island.net/xterm/terminfo.html * src/dircolors.hin: Add *direct* to match terminals that support direct colors (24-bit color / TrueColor). The trailing * will match entries like xterm-direct2. Addresses https://bugs.gnu.org/39827
2021-08-31doc: indicate the default algorithm in the sum(1) man pagePádraig Brady-1/+1
* src/sum.c (usage): Indicate that -r (BSD algorithm) is the default.
2021-08-31sum: always output a file name if one passedPádraig Brady-2/+2
Adjust to output the file name if any name parameter is passed. This is consistent with sum -s, cksum, and sum implementations on other platforms. This should not cause significant compat issues, as multiple fields are already output, and so already need to be parsed. * src/sum.c (bsd_sum_file): Output the file name if any name parameter is passed. * tests/misc/sum.pl: Adjust accordingly. * doc/coreutils.texi (sum invocation): Likewise. * NEWS: Mention the change in behavior.
2021-08-29basenc: fix bug49741: using wrong decoding buffer lengthAssaf Gordon-1/+3
Emil Lundberg <lundberg.emil@gmail.com> reports in https://bugs.gnu.org/49741 about a 'basenc --base64 -d' decoding bug. The input buffer length was not divisible by 3, resulting in decoding errors. * NEWS: Mention fix. * src/basenc.c (DEC_BLOCKSIZE): Change from 1024*5 to 4200 (35*3*5*8) which is divisible by 3,4,5,8 - satisfying both base32 and base64; Use compile-time verify() macro to enforce the above. * tests/misc/basenc.pl: Add test.
2021-08-27basenc: prefer signed to unsigned integersPaul Eggert-101/+87
This patch modifies basenc to prefer signed integers to unsigned, as signed are less error-prone. This patch also updates Gnulib to to latest, which updates Gnulib’s base32 and base64 modules to prefer signed to unsigned integers. * src/basenc.c: Include idx.h. (struct base2_decode_context): Use unsigned char, not unsigned for an octet that must fit in an unsigned char. (base_encode, struct base_decode_context) (base64_decode_ctx_wrapper, prepare_inbuf, base64url_encode) (base64url_decode_ctx_wrapper, base32_decode_ctx_wrapper) (base32hex_encode, base32hex_decode_ctx_wrapper, base16_encode) (base16_decode_ctx, z85_encode, Z85_HI_CTX_TO_32BIT_VAL) (z85_decoding, z85_decode_ctx, base2msbf_encode) (base2lsbf_encode, base2lsbf_decode_ctx, base2msbf_decode_ctx) (wrap_write, do_encode, do_decode, main): Prefer signed integers to unsigned. (main): Treat extremely large wrap columns as if they were infinite; that’s good enough. Since we’re now using xstrtoimax, this allows ‘-w -0’ (same as ‘-w 0’). * tests/misc/base64.pl (gen_tests): -w-0 is no longer an error.
2021-08-22df: pacify -Wsuggest-attribute=mallocPaul Eggert-1/+4
Problem found with latest Gnulib and GCC 11.2.1. * src/find-mount-point.h (find_mount_point): Add _GL_ATTRIBUTE_MALLOC and _GL_ATTRIBUTE_DEALLOC_FREE.
2021-08-22maint: use clearerr on stdin when appropriatePaul Eggert-107/+129
This is so that commands like ‘fmt - -’ read from stdin both times, even when it is a tty. Fix some other minor issues that are related. * src/blake2/b2sum.c (main): * src/cksum.c (cksum): * src/cut.c (cut_file): * src/expand-common.c (next_file): * src/fmt.c (fmt): * src/fold.c (fold_file): * src/md5sum.c (digest_file, digest_check): * src/nl.c (nl_file): * src/od.c (check_and_close): * src/paste.c (paste_parallel, paste_serial): * src/pr.c (close_file): * src/sum.c (bsd_sum_file): Use clearerr on stdin so that stdin can be read multiple times even if it is a tty. Do not assume that ferror preserves errno as POSIX does not guarantee this. Coalesce duplicate diagnostic calls. * src/blake2/b2sum.c (main): * src/fmt.c (main, fmt): Report read error, even if it's merely fclose failure. * src/fmt.c: Include die.h. (fmt): New arg FILE. Close input (reporting error) if not stdin. All callers changed. * src/ptx.c (swallow_file_in_memory): Clear stdin's EOF flag. * src/sort.c (xfclose): Remove unnecessary feof call.
2021-08-15chmod: fix use of uninitialized var if -vPaul Eggert-63/+64
Problem reported by Michael Debertol (Bug#50070). * NEWS: Mention the fix. * src/chmod.c (struct change_status): New struct, replacing the old enum Change_status. All uses changed. (describe_change): Distinguish between cases depending on whether 'stat' or its equivalent succeeded. Report a line of output even if 'stat' failed, as that matches the documentation. Rework to avoid casts. (process_file): Do not output nonsense modes computed from uninitialized storage, removing a couple of IF_LINTs. Simplify by defaulting to CH_NO_STAT.
2021-08-11df: fix bug with automountedPaul Eggert-9/+25
If the command-line argument is automounted, df would use stat info that became wrong after the following open. * NEWS: Mention the fix (bug#50012). * src/df.c (automount_stat_err): New function. This fixes the hang on fifos in a better way, by using O_NONBLOCK. (main): Use it.
2021-08-08cat: with -E fix handling of \r\n spanning buffersPádraig Brady-6/+33
We must delay handling when \r is the last character of the buffer being processed, as the next character may or may not be \n. * src/cat.c (pending_cr): A new global to record whether the last character processed (in -E mode) is '\r'. (cat): Honor pending_cr when processing the start of the buffer. (main): Honor pending_cr if no more files to process. * tests/misc/cat-E.sh: Add test cases. Fixes https://bugs.gnu.org/49925
2021-07-31uniq: pacify GCC -fanalyzerPaul Eggert-2/+2
Pacify GCC 11.1 -fanalyzer. * src/uniq.c (check_file): Use simpler test to check whether this is the first time through the loop. Although the old test was correct, the new one is easier to understand and perhaps a tiny bit more efficient.
2021-07-31numfmt: omit unnecessary pointer testPaul Eggert-1/+1
Caught by GCC 11.1 -fanalyzer. * src/numfmt.c (simple_strtod_int): Remove unnecessary test of *endptr vs NULL. Presumably this was a typo and **endptr was intended instead of *endptr, but an **endptr test is also unnecessary since c_isdigit (0) returns false.
2021-07-28maint: avoid syntax-check failures in recent ls changesPádraig Brady-8/+9
* src/ls.c: Fix ifdef indenting and long line.
2021-07-28doc: modernize usage of “disk” and “core”Paul Eggert-54/+55
In documentation and comments, don’t assume that secondary storage devices are disk devices. Similarly, don’t assume that main memory uses magnetic cores, which became obsolete in the 1970s. * src/du.c (usage): * src/ls.c (usage): * src/shred.c (usage): Reword to avoid “disk” in usage messages.
2021-07-28doc: improve ls documentationPaul Eggert-8/+7
* doc/coreutils.texi (ls invocation): Document implementation more closely. Be more consistent about style. Omit some needless words. * src/ls.c (usage): Don’t overdocument -f, as the details were wrong. Omit -1 advice as it’s a bit obsolete now that we have --zero and is a bit much for --usage output anyway.
2021-07-27ls: rename --null to --zero (Bug#49716)Paul Eggert-28/+29
* NEWS, doc/coreutils.texi (General output formatting): * src/ls.c (usage): Document this. * src/ls.c (ZERO_OPTION): Rename from NULL_OPTION. All uses changed. (long_options): Rename --null to --zero. (dired_dump_obstack, main, print_dir): Use '\n' instead of eolbyte where eolbyte must equal '\n'. (decode_switches): Decode --zero instead of --null. --zero also implies -1, -N, --color=none, --show-control-chars. Use easier-to-decipher code to set ‘format’ and ‘dired’. Reject attempts to combine --dired and --zero. * tests/local.mk: Adjust to test script renaming. * tests/ls/zero-option.sh: Rename from tests/ls/null-option.sh, and test --zero instead of --null.
2021-07-27ls: compute defaults more lazilyPaul Eggert-195/+200
* src/ls.c (enum time_type, enum sort_type, enum indicator_style) (enum Dereference_symlink, ignore_mode): Put ‘= 0’ after default values, since the code relies on static storage defaulting to zero. (enum sort_type): Reorder so that -1 can be used to represent unset. (main): Test print_with_color after parse_ls_color may have reset it. (decode_line_length): Return the line length instead of setting static storage. All uses changed. Treat line lengths exceeding PTRDIFF_MAX as infinite, to avoid pointer-subtraction glitches. (stdout_isatty): New function, to avoid calling isatty twice. (decode_switches): Calculate defaults more lazily, to avoid using syscalls or getenv during startup unless the results are more likely to be needed. Use -1 to indicate options that haven’t been set on the command line yet. Move print_with_color test from here to ‘main’. Suppress bogus GCC warning. (getenv_quoting_style): Return the quoting style instead of setting static storage. (init_column_info): New arg MAX_COLS, to avoid recalculating it. Caller changed.
2021-07-26maint: avoid recent syntax-check issuesPádraig Brady-1/+1
* .gitignore: Cater for recently added poll module. * src/stdbuf.c: Avoid false positive from sc_prohibit_readlink.
2021-07-26ls: add --null option (Bug#49716)Paul Eggert-14/+25
* NEWS, doc/coreutils.texi (General output formatting): * src/ls.c (usage): Document this. * src/ls.c (NULL_OPTION): New constant. (long_options): Add --null. (eolbyte): New static var. (dired_dump_obstack, main, print_dir, print_current_files) (print_many_per_line, print_horizontal, print_with_separator): Output eolbyte instead of '\n'. (decode_switches): Decode --null. * tests/ls/null-option.sh: New file. * tests/local.mk (all_tests): Add it.
2021-07-26ls: port to wider off_t, uid_t, gid_tPaul Eggert-10/+10
* src/ls.c (dired_pos): Now off_t, not size_t, since it counts output file offsets. (dired_dump_obstack): This obstack's file offsets are now off_t, not size_t. (format_user_or_group, format_user_or_group_width): ID arg is now uintmax_t, not unsigned long, since uid_t and gid_t values might exceed ULONG_MAX. (format_user_or_group_width): Use snprintf with NULL instead of sprintf with a discarded buffer. This avoids a stack buffer, and so should be safer.
2021-07-26ls: demacroizePaul Eggert-135/+140
Prefer functions or constants to macros where either will do. That’s cleaner, and nowadays there’s no performance reason to prefer macros. All uses changed. * src/ls.c (INITIAL_TABLE_SIZE, MIN_COLUMN_WIDTH): Now constants instead of macros. (file_or_link_mode): New function, replacing the old macro FILE_OR_LINK_MODE. (dired_outbyte): New function, replacing the old macro DIRED_PUTCHAR. (dired_outbuf): New function, replacing the old macro DIRED_FPUTS. (dired_outstring): New function, replacing the old macro DIRED_FPUTS_LITERAL. (dired_indent): New function, replacing the old macro DIRED_INDENT. (push_current_dired_pos): New function, replacing the old macro PUSH_CURRENT_DIRED_POS. (assert_matching_dev_ino): New function, replacing the old macro ASSERT_MATCHING_DEV_INO. (do_stat, do_lstat, stat_for_mode, stat_for_ino, fstat_for_ino) (signal_init, signal_restore, cmp_ctime, cmp_mtime, cmp_atime) (cmp_btime, cmp_size, cmp_name, cmp_extension) (fileinfo_name_width, cmp_width, cmp_version): No longer inline; compilers can deduce this well enough nowadays. (main): Protect unused assert with ‘if (false)’ rather than commenting it out, so that the compiler checks the code. (print_dir): Output the space and newline in the same buffer as the human-readable number they surround. (dirfirst_check): New function, replacing the old macro DIRFIRST_CHECK. Simplify by using subtraction. (off_cmp): New function, replacing the old macro longdiff. (print_long_format): No need to null-terminate the string now. (format_user_or_group): Let printf count the bytes.
2021-07-26ls: simplify sprintf usagePaul Eggert-24/+14
* src/ls.c (format_user_or_group_width, print_long_format): Use return value from sprintf instead of calling strlen on the resulting buffer, or inferring the length some other way.
2021-07-26maint: fix white spacePaul Eggert-86/+89
2021-07-26env: fix usage typoPaul Eggert-1/+1
* src/env.c (usage): Fix pluralization typo.
2021-07-02df: fix duplicated remote entries due to bind mountsKamil Dudka-2/+13
As originally reported in <https://bugzilla.redhat.com/1962515>, df invoked without -a printed duplicated entries for NFS mounts of bind mounts. This is a regression from commit v8.25-54-g1c17f61ef99, which introduced the use of a hash table. The proposed patch makes sure that the devlist entry seen the last time is used for comparison when eliminating duplicated mount entries. This way it worked before introducing the hash table. Patch co-authored by Roberto Bergantinos. * src/ls.c (struct devlist): Introduce the seen_last pointer. (devlist_for_dev): Return the devlist entry seen the last time if found. (filter_mount_list): Remember the devlist entry seen the last time for each hashed item. * NEWS: Mention the bug fix. Fixes https://bugs.gnu.org/49298
2021-06-26tail: use poll, not selectPaul Eggert-61/+39
This fixes an unlikely stack out-of-bounds write reported by Stepan Broz via Kamil Dudka (Bug#49209). * bootstrap.conf (gnulib_modules): Replace select with poll. * src/tail.c: Do not include <sys/select.h>. [!_AIX]: Include poll.h. (check_output_alive) [!_AIX]: Use poll instead of select. (tail_forever_inotify): Likewise. Simplify logic, as there is no need for a ‘while (len <= evbuf_off)’ loop.
2021-06-26maint: while (1) → while (true)Paul Eggert-47/+47
2021-06-21maint: remove redundant checks on buffer sizes in tailNikolay Nechaev-4/+3
* src/tail.c: remove redundant size checks before calls to `xwrite_stdout`
2021-06-21stat: use decomposed decimal device numbers by defaultPádraig Brady-2/+2
* src/stat.c (default_format): Use decomposed decimal representation (major,minor) in the default format. This is least ambiguous for human interpretation, and more consistent with ls for example. Fixes https://bugs.gnu.org/48960
2021-06-21stat: support more device number representationsPádraig Brady-11/+49
In preparation for changing the default device number representation (to decomposed decimal), provide more formatting options for device numbers. These new (FreeBSD compat) formatting options are added: %Hd major device number in decimal (st_dev) %Ld minor device number in decimal (st_dev) %Hr major device type in decimal (st_rdev) %Lr minor device type in decimal (st_rdev) %r (composed) device type in decimal (st_rdev) %R (composed) device type in hex (st_rdev) * doc/coreutils.texi (stat invocation): Document new formats. * src/stat.c (print_it): Handle the new %H and %L modifiers. (print_statfs): Adjust to passing the format as two chars rather than an int. Using an int was introduced in commit db42ae78, but using separate chars is cleaner and more extensible. (print_stat): Likewise. Handle any modifiers and the new 'r' format. (usage): Document the new formats. * tests/misc/stat-fmt.sh: Add a test case for new modifiers. Addresses https://bugs.gnu.org/48960