Mirror of https://https.git.savannah.gnu.org/git/coreutils.git/ https://git.shady.money/coreutils/atom?h=v8.8 2010-12-19T00:33:45Z 2010-12-19T00:33:45Z Pádraig Brady P@draigBrady.com 2010-12-18T05:27:46Z urn:sha1:0e181024c00b746a930aab6a0cfd9162d7b67ae4 * src/sort.c (main): If --parallel isn't specified, restrict the number of threads to 8 by default. If the --parallel option is specified, then allow any number of threads to be set, independent of the number of processors on the system. * doc/coreutils.texi (sort invocation): Document the changes to determining the number of threads to use. Mention the memory overhead when using multiple threads. * tests/misc/sort-spinlock-abuse: Allow single core systems that support pthreads. * tests/misc/sort-stale-thread-mem: Likewise. * tests/misc/sort-unique-segv: Likewise. * NEWS: Mention the change in behaviour. 2010-12-16T21:55:31Z Paul Eggert eggert@cs.ucla.edu 2010-12-16T21:55:13Z urn:sha1:1b31ce6982a9151d9dfe2ea3595ad7595cb9ca86 * NEWS: Document this. * src/sort.c (UNCOMPRESSED, UNREAPED, REAPED): New constants. (struct tempnode): New member 'state', to hold these constants. The pid member is now undefined if state == UNCOMPRESSED. (struct sortfile): Replace member 'pid' with member 'temp'. (uintptr): Remove. (proctab_hasher, proctab_comparator, register_proc, delete_proc): Proctab entries are now struct tempnode *, not pid_t, to handle the case where multiple tempnode objects correspond to the same pid. This avoids a race condition that can cause a hang. (register_proc): Arg is now struct tempnode *, not pid_t. All callers changed. (delete_proc): Set tempnode state to REAPED. (create_temp_file): No need to set pid member here; it's now done when the pid is known. (maybe_create_temp, create_temp): Remove PPID arg. Return struct tempnode *, not char *. All callers changed. (maybe_create_temp): Set node state to UNCOMPRESSED or UNREAPED. No need to set node->pid to 0. (open_temp): Replace NAME and PID args with a single TEMP arg. All callers changed. Wait only for unreaped children. (zaptemp): Wait for decompressor to finish before removing its temporary-file input. This avoids .nfsXXXX hassles with NFS and fixes a race (leading to a hang) regardless of NFS. (open_input_files): Adjust to new way of dealing with temp files and their subprocesses. * tests/Makefile.am (TESTS): Add misc/sort-compress-hang. * tests/misc/sort-compress-hang: New file. 2010-12-16T08:04:32Z Paul Eggert eggert@cs.ucla.edu 2010-12-16T08:03:29Z urn:sha1:f3c584d1e08646704cb46f4e5b6afc4afef3f70a * NEWS: Document this. * src/sort.c (avoid_trashing_input): The previous fix to this function didn't fix all the problems with this code. Replace it with something simpler: just copy the input file. This doesn't change the number of files, so return void instead of the updated file count. Caller changed. * tests/misc/sort-merge-fdlimit: Test for the bug. 2010-12-14T18:08:04Z Paul Eggert eggert@cs.ucla.edu 2010-12-14T18:07:36Z urn:sha1:dcb3f60d0af77ec06d58e4fa18c8ee2b80d10cc6 * tests/check.mk (TESTS_ENVIRONMENT): Default TMPDIR to /tmp, rather than to the working directory; this is more common in practice, which makes the tests more real-worldish; and it is often faster. Also, it avoids some problems with NFS cleanups. * tests/misc/sort-compress: Remove unnecessary code setting TMPDIR. * tests/misc/sort-compress-proc: Likewise. Do the final sleep only if TMPDIR is relative, which should be rarely given the change to TESTS_ENVIRONMENT. 2010-12-14T07:23:47Z Paul Eggert eggert@cs.ucla.edu 2010-12-14T07:23:17Z urn:sha1:0da4d843003e9d38624e19c24c7fb670f1bb4749 * src/sort.c (uintptr): New type. (enum procstate, struct procnode, update_proc): Remove. (proctab_hasher, proctab_comparator, register_proc, wait_proc): (reap_some): The proctab is now simply a hash of process-IDs rather than of pointers to objects with reference counts and states; this is smaller and faster and easier to understand. (nprocs): Now pid_t, not size_t, since one cannot have more than PID_MAX children. (reap): If the argument is -1, wait; if 0 (a new value), do not. Delete pid from proctab as needed. Ignore children that are not in proctab, as they are from the program that exec'ed us and are irrelevant to our success or failure. (delete_proc, reap_all): New functions. (open_temp): Register the child. (sort): Clean up all children afterwards; without this patch, 'sort' sometimes missed failures in children due to race conditions. * tests/Makefile.am (TESTS): Add misc/sort-compress-proc. * tests/misc/sort-compress-proc: New file, to test for the bugs fixed above. 2010-12-13T18:02:43Z Paul Eggert eggert@cs.ucla.edu 2010-12-13T18:02:06Z urn:sha1:6d36bd4c6418083d543c3757905c3202f2d0ee43 * tests/misc/sort-stale-thread-mem: Fix typo in comment. 2010-12-13T07:28:19Z Jim Meyering meyering@redhat.com 2010-12-13T07:07:25Z urn:sha1:8351407f874ab3d6fc0830e78a6c234bf1583e3f * tests/misc/sort-stale-thread-mem: Don't initialize fail=0 here; that is done in init.sh. This avoids a syntax-check failure. Invoke "Exit $fail" at end, too. Mark as a very expensive test. 2010-12-12T21:38:49Z Paul Eggert eggert@cs.ucla.edu 2010-12-12T21:38:19Z urn:sha1:b212633c41870cd4b68b8522041b51429b1ba1bd * tests/misc/sort-stale-thread-mem: New tests. * tests/Makefile.am (TESTS): Add it. 2010-12-11T17:44:57Z Jim Meyering meyering@redhat.com 2010-12-11T17:44:57Z urn:sha1:45ff91af2eeee4f84eff331cbbc5c1a3f4a84248 * tests/misc/printenv: Filter out LD_PRELOAD, as the comment said, not LD_LIBRARY. 2010-12-08T23:36:17Z Pádraig Brady P@draigBrady.com 2010-12-08T08:33:15Z urn:sha1:195c455d0ae3884ab2d9680ac3043aa36e9c8c3d When -n l/N is used and long lines are present that both span partitions and multiple buffers, one would get inconsistent chunk sizes. * src/split.c (main): Add a new undocumented ---io-blksize option to support full testing with varied buffer sizes. (cwrite): Refactor most handling of --elide-empty to here. (bytes_split): Remove handling of --elide-empty. (lines_chunk_split): Likewise. The specific issue here was the first handling of elide_empty_files interfered with the replenishing of the input buffer. * test/misc/split-lchunk: Add -e and the new ---io-blksize combinations to the test.