coreutils/src/wc.c, branch v9.0

maint: prefer rawmemchr to memchr when easy

2021-09-15T22:08:28Z

* bootstrap.conf (gnulib_modules): Add rawmemchr. * src/csplit.c: Include idx.h. * src/csplit.c (record_line_starts): * src/head.c (elide_tail_lines_pipe): * src/shuf.c (next_line): * src/split.c (lines_split): * src/tail.c (pipe_lines): * src/wc.c (wc_lines): Prefer rawmemchr to memchr when rawmemchr is easy. * src/csplit.c (load_buffer): * src/head.c (struct linebuffer): Make room for a 1-byte sentinel.

wc: add --debug to diagnose which implementation used

2021-05-02T19:51:37Z

* src/wc.c: (main): Handle the new --debug option. Only call avx2_supported if needed. (avx2_supported): Diagnose various failures and attempts. * NEWS: Mention the new wc improvement and --debug option.

wc: use avx2 optimization when counting only lines

2021-05-02T19:51:37Z

Use cpuid to detect CPU support for avx2 instructions. Performance was seen to improve by 5x for a file with only newlines, while the performance for a file with no such characters is unchanged. * configure.ac [USE_AVX2_WC_LINECOUNT]: A new conditional, set when __get_cpuid_count() and avx2 compiler intrinsics are supported. * src/wc.c (avx2_supported): A new function using __get_cpuid_count() to determine if avx2 instructions are supported. (wc_lines): A new function refactored from wc(), which implements the standard line counting logic, and provides the fallback implementation for when avx2 is not supported. * src/wc_avx2.c: A new module to implement using avx2 intrinsics. * src/local.mk: Reference the new module. Note we build as a separate lib so that it can be portably built with separate -mavx2 etc. flags.

doc: clarify what's counted by wc

2021-04-11T18:23:13Z

* src/wc.c (usage): State that only printable characters are considered when counting words. This also disambiguates wether we're talking about bytes or characters in this context. * doc/coreutils.texi (wc invocation): Likewise. Also clarify that --characters counts valid locale aware characters, and that --lines does not count a trailing "line" unless it ends with a newline character. Fixes https://bugs.gnu.org/47702

maint: use "char const " rather than "const char "

2021-04-11T17:33:45Z

* cfg.mk (sc_prohibit-const-char): Add a new syntax-check to enforce this style. * *.[ch]: sed -i 's/const char \*/char const */g'

maint: update all copyright year number ranges

2021-01-01T16:36:09Z

Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year.

maint: update all copyright year number ranges

2020-01-01T14:16:56Z

wc: treat non breaking space as a word separator

2019-02-26T06:49:44Z

* src/wc.c (iswnbspace): A new function to match characters in this class. (isnbspace): Likewise for single byte charsets. (main): Initialize posixly_correct from the environment, to allow disabling honoring NBSP in non C locales. (wc): Call is[w]nbspace() along with is[w]space. * bootstrap.conf: Ensure btowc is available. * tests/misc/wc-nbsp.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the change in behavior.

maint: update all copyright year number ranges

2019-01-01T23:51:48Z

wc: optimize processing of ASCII in multi byte locales

2018-06-21T04:00:39Z

===== Benchmark setup (on GNU/Linux) ==== $ yes áááááááááááááááááááá | head -n100000 > mbc.txt $ yes 12345678901234567890 | head -n100000 > num.txt ===== Before ==== $ time src/wc -Lm < mbc.txt real 0m0.186s $ time src/wc -m < mbc.txt real 0m0.186s $ time src/wc -Lm < num.txt real 0m0.055s $ time src/wc -m < num.txt real 0m0.056s ==== After ==== $ time src/wc -Lm < mbc.txt real 0m0.196s $ time src/wc -m < mbc.txt real 0m0.173s $ time src/wc -Lm < num.txt real 0m0.031s $ time src/wc -m < num.txt real 0m0.028s * src/wc.c (wc): Only call wide variant functions like iswprint() and wcwidth() for non is_basic() characters. I.E. non ISO C "basic character set" characters. This is especially significant on OSX where wcwidth() is very expensive (about 10x in tests). * NEWS: Mention the improvement. Suggested by Eric Fischer.

coreutils/src/wc.c, branch v9.0

maint: prefer rawmemchr to memchr when easy

wc: add --debug to diagnose which implementation used

wc: use avx2 optimization when counting only lines

doc: clarify what's counted by wc

maint: use "char const *" rather than "const char *"

maint: update all copyright year number ranges

maint: update all copyright year number ranges

wc: treat non breaking space as a word separator

maint: update all copyright year number ranges

wc: optimize processing of ASCII in multi byte locales

maint: use "char const " rather than "const char "