git/utf8.h, branch v2.18.2

Sync with Git 2.17.1

2018-05-29T08:10:05Z

* maint: (25 commits) Git 2.17.1 Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 fsck: complain when .gitmodules is a symlink index-pack: check .gitmodules files with --strict unpack-objects: call fsck_finish() after fscking objects fsck: call fsck_finish() after fscking objects fsck: check .gitmodules content fsck: handle promisor objects in .gitmodules check fsck: detect gitmodules files fsck: actually fsck blob data fsck: simplify ".git" check index-pack: make fsck error message more specific verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant ...

is_hfs_dotgit: match other .git files

2018-05-22T03:50:11Z

Both verify_path() and fsck match ".git", ".GIT", and other variants specific to HFS+. Let's allow matching other special files like ".gitmodules", which we'll later use to enforce extra restrictions via verify_path() and fsck. Signed-off-by: Jeff King

utf8: add function to detect a missing UTF-16/32 BOM

2018-04-16T02:40:56Z

If the endianness is not defined in the encoding name, then let's be strict and require a BOM to avoid any encoding confusion. The is_missing_required_utf_bom() function returns true if a required BOM is missing. The Unicode standard instructs to assume big-endian if there in no BOM for UTF-16/32 [1][2]. However, the W3C/WHATWG encoding standard used in HTML5 recommends to assume little-endian to "deal with deployed content" [3]. Strictly requiring a BOM seems to be the safest option for content in Git. This function is used in a subsequent commit. [1] http://unicode.org/faq/utf_bom.html#gen6 [2] http://www.unicode.org/versions/Unicode10.0.0/ch03.pdf Section 3.10, D98, page 132 [3] https://encoding.spec.whatwg.org/#utf-16le Signed-off-by: Lars Schneider Signed-off-by: Junio C Hamano

utf8: add function to detect prohibited UTF-16/32 BOM

2018-04-16T02:40:56Z

Whenever a data stream is declared to be UTF-16BE, UTF-16LE, UTF-32BE or UTF-32LE a BOM must not be used [1]. The function returns true if this is the case. This function is used in a subsequent commit. [1] http://unicode.org/faq/utf_bom.html#bom10 Signed-off-by: Lars Schneider Signed-off-by: Junio C Hamano

typofix: assorted typofixes in comments, documentation and messages

2016-05-06T20:16:37Z

Many instances of duplicate words (e.g. "the the path") and a few typoes are fixed, originally in multiple patches. wildmatch: fix duplicate words of "the" t: fix duplicate words of "output" transport-helper: fix duplicate words of "read" Git.pm: fix duplicate words of "return" path: fix duplicate words of "look" pack-protocol.txt: fix duplicate words of "the" precompose-utf8: fix typo of "sequences" split-index: fix typo worktree.c: fix typo remote-ext: fix typo utf8: fix duplicate words of "the" git-cvsserver: fix duplicate words Signed-off-by: Li Peng Signed-off-by: Junio C Hamano

utf8: add function to align a string into given strbuf

2015-09-17T17:02:48Z

Add strbuf_utf8_align() which will align a given string into a strbuf as per given align_type and width. If the width is greater than the string length then no alignment is performed. Helped-by: Eric Sunshine Mentored-by: Christian Couder Mentored-by: Matthieu Moy Signed-off-by: Karthik Nayak Signed-off-by: Junio C Hamano

Merge branch 'es/utf8-stupid-compiler-workaround'

2015-06-24T19:21:46Z

A compilation workaround. * es/utf8-stupid-compiler-workaround: utf8: NO_ICONV: silence uninitialized variable warning

utf8: NO_ICONV: silence uninitialized variable warning

2015-06-05T22:36:35Z

The last argument of reencode_string_len() is an 'int *' which is assigned the length of the converted string. When NO_ICONV is defined, however, reencode_string_len() is stubbed out by the macro: #define reencode_string_len(a,b,c,d,e) NULL which never assigns a value to the final argument. When called like this: int n; char *s = reencode_string_len(..., &n); if (s) do_something(s, n); some compilers complain that 'n' is used uninitialized within the conditional. Signed-off-by: Eric Sunshine Reviewed-by: Jeff King Signed-off-by: Junio C Hamano

utf8-bom: introduce skip_utf8_bom() helper

2015-04-16T18:35:06Z

With the recent change to ignore the UTF8 BOM at the beginning of .gitignore files, we now have two codepaths that do such a skipping (the other one is for reading the configuration files). Introduce utf8_bom[] constant string and skip_utf8_bom() helper and teach .gitignore code how to use it. Signed-off-by: Junio C Hamano

utf8: add is_hfs_dotgit() helper

2014-12-17T19:04:39Z

We do not allow paths with a ".git" component to be added to the index, as that would mean repository contents could overwrite our repository files. However, asking "is this path the same as .git" is not as simple as strcmp() on some filesystems. HFS+'s case-folding does more than just fold uppercase into lowercase (which we already handle with strcasecmp). It may also skip past certain "ignored" Unicode code points, so that (for example) ".gi\u200ct" is mapped ot ".git". The full list of folds can be found in the tables at: https://www.opensource.apple.com/source/xnu/xnu-1504.15.3/bsd/hfs/hfscommon/Unicode/UCStringCompareData.h Implementing a full "is this path the same as that path" comparison would require us importing the whole set of tables. However, what we want to do is much simpler: we only care about checking ".git". We know that 'G' is the only thing that folds to 'g', and so on, so we really only need to deal with the set of ignored code points, which is much smaller. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano