coreutils/tests/Makefile.am, branch v8.7

csplit: fix a memory leak per input buffer

2010-11-11T01:40:12Z

* src/csplit.c (free_buffer): Also free the line offsets buffers (remove_line): Also free the containing structure * tests/misc/csplit-heap: A new test to trigger with leaks of this magnitude. * tests/Makefile.am: Reference the new test * NEWS: Mention the fix Reported by David Hofstee

csplit: avoid buffer overrun when writing more than 999 files

2010-11-10T13:28:03Z

Without this fix, seq 1000 | csplit - /./ '{*}' would write the NUL-terminated file name, xx1000, into a buffer of size 6. * src/csplit.c (main): Use properly sized file name buffer. * NEWS (Bug fixes): Mention it. * tests/misc/csplit-1000: New test to trigger the bug. * tests/Makefile.am (TESTS): Add misc/csplit-1000.

stat: revert %X-%Y-%Z change; use e.g., %:X to print fractional seconds

2010-11-03T12:10:50Z

This reverts part of the recent commit 9069af45, "stat: print timestamps to full resolution", which made %X, %Y, %Z print floating point numbers. We prefer to retain portability of %X, %Y and %Z uses, while still providing access to full-resolution time stamps via modified format strings. Also make the new %W consistent. * src/stat.c: Include "xstrtol.h". (print_it): Accept a new %...:[XYZ] format directive, e.g., %:X, to print the nanoseconds portion of the corresponding time. For example, %3.3:Y prints the zero-padded, truncated, milliseconds part of the time of last modification. (print_it): Update print_func signature to match. (neg_to_zero): New helper function. (epoch_time): Remove function; replace with... (epoch_sec): New function; use timetostr. (out_ns): New function. Use "09" only when no other modifier is specified. (print_statfs): Change type of "m" to unsigned int, now that it must accommodate values larger than 255. (print_stat): Likewise. Map :X to a code of 'X' + 256. Likewise for Y, Z and W. (usage): Update. * tests/touch/60-seconds: Use %Y.%:Y in place of %Y. * tests/misc/stat-nanoseconds: New file. * tests/Makefile.am (TESTS): Add it. * NEWS (Changes in behavior): Mention this. With improvements by Pádraig Brady. Thanks to Andreas Schwab for raising the issue.

du: don't print junk when diagnosing out-of-range time stamps

2010-10-24T00:24:38Z

* src/du.c (show_date): Fix call to fputs with a buffer that contains some uninitialized data. * tests/Makefile.am (TESTS): Add du/big-timestamp. * tests/du/bigtime: New file, which checks for the bug.

tail: fix checking of currently unavailable directories

2010-10-12T10:41:17Z

* src/tail.c (tail_forever_inotify): Handle the case where tail --follow=name with inotify, is not able to add a watch on a specified directory. This may happen due to inotify resource limits or if the directory is currently missing or inaccessible. In all these cases, revert to polling which will try to reopen the file later. Note inotify returns ENOSPC when it runs out of resources, and instead we report a particular error message, lest users think one of their file systems is full. (main): Document another caveat with using inotify, where we currently don't recheck directories recreated after the initial watch is setup. * tests/tail-2/F-vs-rename: Fix the endless loop triggered by the above issue. * tests/tail-2/inotify-hash-abuse: Likewise. * tests/tail-2/wait: Don't fail in the resource exhaustion case. * tests/tail-2/F-vs-missing: A new test for this failure mode which was until now just triggered on older buggy linux kernels which returned ENOSPC constantly from inotify_add_watch(). * NEWS: Mention the fix.

stat: support printing birthtime

2010-10-01T16:43:25Z

* src/stat.c (print_stat): New %w and %W formats. (do_stat): Include %w in default format. (usage): Document new specifiers. * doc/coreutils.texi (stat invocation): Likewise. * NEWS: Likewise.

tr: fix various issues with case conversion

2010-09-29T10:05:12Z

This valid translation spec aborted: LC_ALL=en_US.iso-8859-1 tr '[:upper:]- ' '[:lower:]_' This invalid translation spec aborted: LC_ALL=en_US.iso-8859-1 tr '[:upper:] ' '[:lower:]' This was caused by commit 6efd1046, 05-01-2008, "Avoid tr case-conversion failure in some locales" This misaligned conversion spec was allowed: LC_ALL=C tr 'A-Y[:lower:]' 'a-z[:upper:]' This was caused by commit af5d0c36, 21-10-2007, "tr: do not reject an unmatched [:lower:] or [:upper:] in SET1" This misaligned spec was allowed by extending the class: LC_ALL=C tr '[:upper:] ' '[:lower:]' * src/tr.c (validate_case_classes): A new function to check alignment of case conversion classes. Also it adjusts the length of the sets so that locales with different numbers of upper and lower case characters, don't cause issues. (string2_extend): Disallow extending the case conversion class as in the above example. That is locale dependent and most likely not what the user wants. (validate): Do the simple test for "restricted" char classes earlier, so we don't redundantly do more expensive validation. (main): Remove the case class validation, and simplify. * tests/misc/tr-case-class: A new test to test the various alignment and locale issues, associated with case conversion. * tests/misc/tr: Move case conversion tests to new tr-case-class. * tests/Makefile.am: Reference the new test. * NEWS: Mention the fixes.

stat: add %m to output the mount point for a file

2010-08-27T21:16:10Z

* src/find-mount-point.c: A new file refactoring find_mount_point() out from df.c * src/find-mount-point.h: Likewise. * src/df.c: Use the new find-mount-point module. * src/stat.c (print_stat): Handle the new %m format. (find_bind_mount): A new function to return the bind mount for a file if any. (out_mount_mount): Print the bind mount for a file, or else the standard mount point given by the find-mount-point module. (usage): Document the %m format directive. * src/Makefile.am: Reference the refactored find-mount-point.c * po/POTFILES.in: Add find_mount_point.c to the translation list * doc/coreutils.texi (stat invocation): Document %m, and how it may differ from the mount point that df outputs. * test/misc/stat-mount: A new test to correlate mount points output from df and stat. * tests/Makefile.am: Reference the new test. * NEWS: Mention the new feature * THANKS: Add the author Signed-off-by: Pádraig Brady

sort: add a test case for the sort -u bug

2010-07-16T01:13:55Z

* tests/Makefile.am (TESTS): Add misc/sort-unique. * tests/misc/sort-unique: New file.

sort: parallelize internal sort

2010-07-13T00:44:46Z

This patch is by Gene Auyeung, Chris Dickens, Chen Guo, and Mike Nichols, based off of a patch by Paul Eggert, Glen Lenker, et. al., with a basic heap implementation based off of the GDSL heap, originally by Nicolas Darnis. The number of sorts done in parallel is limited to the number of available processors by default, or can be further restricted with the --parallel option. On a dual-die, 8 core Intel Xeon, results show sorting with 8 threads is almost 4 times faster than using a single thread. Timings when sorting a 96MB file: THREADS TIME (s) 1 5.10 2 2.87 4 1.75 8 1.31 Single threaded sorting has also been improved, especially for cheaper comparison operations: COMMAND BEFORE (s) AFTER (s) sort 8.822 8.716 sort -g 10.336 10.222 sort -n 3.077 2.961 LANG=C sort 2.169 2.066 * bootstrap.conf: Add heap, pthread. * coreutils.texi (sort): Describe the new --parallel option. * gl/lib/heap.c: New file. Very basic heap implementation. * gl/lib/heap.h: New file. * gl/modules/heap: New file. * src/Makefile.am: Add LIB_PTHREAD. * src/sort.c: Include heap.h, nproc.h, pthread.h. (MAX_MERGE): New macro. (SUBTHREAD_LINES_HEURISTIC, PARALLEL_OPTION): New constants. (MERGE_END, MERGE_ROOT): New constants. (struct merge_node): New struct. (struct merge_node_queue): New struct. (sortlines temp): Remove declaration. (usage, long_options, main): New option, --parallel. (specify_nthreads): New function. (mergelines): New signature, to emphasize the fact that the HI area must be part of the destination. All callers changed. (sequential_sort): New function, renamed from sortlines. Merge in the functionality of sortlines_temp. (compare_nodes): New function. (lock_node, unlock_node): New functions. (queue_destroy): New function. (queue_init): New function. (queue_insert): New function. (queue_pop): New function. (write_unique): New function. (mergelines_node): New function. (check_insert): New function. (update_parent): New function. (merge_loop): New function. (sortlines): Rewrite to support and use parallelism, with a new signature. All callers changed. (struct thread_args): New struct. (sortlines_thread): New function. (sortlines_temp): Remove. (sort): New argument NTHREADS. All uses changed. Output moved to mergelines_node. (main): disable threading if we are sorting at random. * tests/Makefile.am (TESTS): Add misc/sort-benchmark-random. * tests/misc/sort-benchmark-random: New file. Signed-off-by: Pádraig Brady