Mirror of https://https.git.savannah.gnu.org/git/coreutils.git/ https://git.shady.money/coreutils/atom?h=v8.22 2013-05-08T10:51:37Z 2013-05-08T10:51:37Z Cojocaru Alexandru xojoc@gmx.com 2013-05-07T12:47:15Z urn:sha1:465f9512b710ee2fe03c3caf65bfdccdce3544ae Use a sentinel value that's checked implicitly, rather than a bit array, to determine if an item should be output. Benchmark results for this change are: $ yes abcdfeg | head -n1MB > big-file $ for c in orig sentinel; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == orig == real 0m0.049s user 0m0.044s sys 0m0.005s == sentinel == real 0m0.035s user 0m0.032s sys 0m0.002s ## Again with --output-delimiter ## $ for c in orig sentinel; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 --output-delimiter=: big-file > /dev/null done == orig == real 0m0.106s user 0m0.103s sys 0m0.002s == sentinel == real 0m0.055s user 0m0.052s sys 0m0.003s eol_range_start: Removed. 'n-' is no longer treated specially, and instead SIZE_MAX is set for the 'hi' limit, and tested implicitly. complement_rp: Used to complement 'rp' when '--complement' is specified. ADD_RANGE_PAIR: Macro renamed to 'add_range_pair' function. * tests/misc/cut-huge-range.sh: Adjust to the SENTINEL value. Also remove the overlapping range test as this is no longer dependent on large ranges and also is already handled with the EOL-subsumed-3 test in cut.pl. 2013-05-07T12:07:02Z Cojocaru Alexandru xojoc@gmx.com 2013-05-07T12:01:46Z urn:sha1:b54b47f954c9b97bdb2dbbf51ead908ccb3a4f13 This issue was introduced in commit v8.21-43-g3e466ad * src/cut.c (set_fields): Process all range pairs when merging. * tests/misc/cut-huge-range.sh: Add a test for this edge case. Also fix an issue where we could miss reported errors due to truncation of the 'err' file. 2013-04-29T16:54:39Z Pádraig Brady P@draigBrady.com 2013-04-28T22:27:12Z urn:sha1:791919f6d9a873ae7452a7e1d71e2fe9e0fd4104 Ensure appropriate functions are inlined. This was seen to be required with gcc 4.6.0 with -O2 on x86_64 at least. It was reported that gcc 4.8.0 did inline these functions though. Also reinstate the bit vector for the common case, to further improve performance. Benchmark results for both aspects of this change are: $ yes abcdfeg | head -n1MB > big-file $ for c in orig inline inline-array; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == orig == real 0m0.088s user 0m0.081s sys 0m0.007s == inline == real 0m0.070s user 0m0.060s sys 0m0.009s == inline-array == real 0m0.049s user 0m0.044s sys 0m0.005s * src/cut.c (set_fields): Set up the printable_field bit vector for performance, but only when it's appropriate. I.E. not when either --output-delimeter or huge ranges are specified. (next_item): Ensure it's inlined and avoid unnecessary processing. (print_kth): Ensure it's inlined and add a branch for the fast path. Related to http://bugs.gnu.org/13127 2013-04-29T16:54:39Z Cojocaru Alexandru xojoc@gmx.com 2013-04-28T02:03:45Z urn:sha1:ef9db5735a401f60eb5b4a18a365bf1ece525053 print_kth() is the central function of cut used to determine if an item is to be output or not, so simplify it by moving some logic outside. Benchmark results for this change are: $ yes abcdfeg | head -n1MB > big-file $ for c in orig split; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == orig == real 0m0.111s user 0m0.108s sys 0m0.002s == split == real 0m0.088s user 0m0.081s sys 0m0.007s * src/cut.c (print_kth): Refactor a branch to outside the function. Related to http://bugs.gnu.org/13127 2013-04-29T16:54:27Z Cojocaru Alexandru xojoc@gmx.com 2012-12-09T09:43:10Z urn:sha1:3e466ad05181d95057e6612ff11059c91396cd0e The current implementation of cut, uses a bit array, an array of `struct range_pair's, and (when --output-delimiter is specified) a hash_table. The new implementation will use only an array of `struct range_pair's. The old implementation is memory inefficient because: 1. When -b with a big num is specified, it allocates a lot of memory for `printable_field'. 2. When --output-delimiter is specified, it will allocate 31 buckets. Even if only a few ranges are specified. Note CPU overhead is increased to determine if an item is to be printed, as shown by: $ yes abcdfeg | head -n1MB > big-file $ for c in with-bitarray without-bitarray; do src/cut-$c 2>/dev/null echo -ne "\n== $c ==" time src/cut-$c -b1,3 big-file > /dev/null done == with-bitarray == real 0m0.084s user 0m0.078s sys 0m0.006s == without-bitarray == real 0m0.111s user 0m0.108s sys 0m0.002s Subsequent patches will reduce this overhead. * src/cut.c (set_fields): Set and initialize RP instead of printable_field. * src/cut.c (is_range_start_index): Use CURRENT_RP rather than a hash. * tests/misc/cut.pl: Check if `eol_range_start' is set correctly. * tests/misc/cut-huge-range.sh: Rename from cut-huge-to-eol-range.sh, and add a test to verify large amounts of mem aren't allocated. Fixes http://bugs.gnu.org/13127 2013-02-04T13:55:01Z Pádraig Brady P@draigBrady.com 2013-02-04T11:39:20Z urn:sha1:be7932e863de07c4c7e4fc3c1db3eb6d04ba9af5 Fixes the issue introduced in unreleased commit v8.20-60-gec48bea. * src/cut.c (set_fields): Don't access the bit array if we've an open ended range that's outside any finite range. * tests/misc/cut.pl: Add tests for this case. Reported by Marcel Böhme in http://bugs.gnu.org/13627 2013-01-26T02:31:59Z Pádraig Brady P@draigBrady.com 2013-01-22T01:34:07Z urn:sha1:d302aed182d625281fba87aca3cb513882aa967b * src/cut.c (cut_fields): Handle the edge case where '\n' is the delimiter, which could be used for example to suppress the last line if it doesn't contain a '\n'. * test/misc/cut.pl: Add tests for this edge case. 2013-01-26T02:31:53Z Pádraig Brady P@draigBrady.com 2013-01-21T15:37:37Z urn:sha1:51ce0bf8440e18de34586003b1a5d7f568fbf636 Previously line N+1 was inspected before line N was fully output, which causes output ordering issues at the terminal or delays from intermittent sources like tail -f. * src/cut.c (cut_fields): Adjust so that we record the previous output character so we can use that info to determine wether to output a '\n' or not. * tests/misc/cut.pl: Add tests to ensure existing functionality isn't broken. * NEWS: Mention the fix. Fixes bug http://bugs.gnu.org/13498 2013-01-23T00:03:38Z Bernhard Voelker mail@bernhard-voelker.de 2013-01-23T00:03:38Z urn:sha1:4eaadb47430f21aee83eb920378be01945845a9a Each program with at least one long option which is marked as 'required_argument' and which has also a short option for that option, should print a note about mandatory arguments. Define that well-known note centrally and use it rather than literal printf/fputs, and add it where it was missing. * src/system.h (emit_mandatory_arg_note): Add new function. * src/cp.c (usage): Use it rather than literal printf/fputs. * src/csplit.c, src/cut.c, src/date.c, src/df.c, src/du.c: * src/expand.c, src/fmt.c, src/fold.c, src/head.c, src/install.c: * src/kill.c, src/ln.c, src/ls.c, src/mkdir.c, src/mkfifo.c: * src/mknod.c, src/mv.c, src/nl.c, src/od.c, src/paste.c: * src/pr.c, src/ptx.c, src/shred.c, src/shuf.c, src/sort.c: * src/split.c, src/stdbuf.c, src/tac.c, src/tail.c, src/timeout.c: * src/touch.c, src/truncate.c, src/unexpand.c, src/uniq.c: Likewise. * src/base64.c (usage): Add call of the above new function because at least one long option has a required argument. * src/basename.c, src/chcon.c, src/date.c, src/env.c: * src/nice.c, src/runcon.c, src/seq.c, src/stat.c, src/stty.c: Likewise. 2013-01-01T03:51:20Z Jim Meyering jim@meyering.net 2013-01-01T02:54:51Z urn:sha1:77da73c75432f3c5b4beebae7b0797a1e33160bc Run "make update-copyright", but then also run this, perl -pi -e 's/2\d\d\d-//' tests/sample-test to make that one script use the single most recent year number.