git/git-p4.py, branch v2.37.2

Merge branch 'kk/p4-client-name-encoding-fix' into maint

2022-08-11T04:52:33Z

"git p4" did not handle non-ASCII client name well, which has been corrected. source: * kk/p4-client-name-encoding-fix: git-p4: refactoring of p4CmdList() git-p4: fix bug with encoding of p4 client name

git-p4: refactoring of p4CmdList()

2022-07-21T16:47:06Z

The function p4CmdList executes a Perforce command and decodes the marshalled python dictionary. Special care has to be taken for certain dictionary values which contain non-unicode characters. The old handling contained separate hacks for each of the corresponding dictionary keys. This commit tries to refactor the coding to handle the special cases uniformely. Signed-off-by: Kilian Kilger Signed-off-by: Junio C Hamano

git-p4: fix CR LF handling for utf16 files

2022-07-20T18:38:06Z

Perforce silently replaces LF with CR LF for "utf16" files if the client is a native Windows client. Since git's autocrlf logic does not undo this transformation for UTF-16 encoded files, git-p4 replaces CR LF with LF during the sync if the file type "utf16" is detected and the Perforce client platform indicates that this conversion is performed. Windows only runs on little-endian architectures, therefore the encoding of the byte stream received from the Perforce client is UTF-16-LE and the relevant byte sequence is 0D 00 0A 00. Signed-off-by: Moritz Baumann Signed-off-by: Junio C Hamano

git-p4: fix bug with encoding of p4 client name

2022-07-08T14:59:12Z

The Perforce client name can contain arbitrary characters which do not decode to UTF-8. Use the fallback strategy implemented in metadata_stream_to_writable_bytes() also for the client name. Signed-off-by: Kilian Kilger Reviewed-by: Tao Klerks Signed-off-by: Junio C Hamano

Merge branch 'tk/p4-metadata-coding-strategies'

2022-05-20T22:27:00Z

"git p4" updates. * tk/p4-metadata-coding-strategies: git-p4: improve encoding handling to support inconsistent encodings

Merge branch 'kf/p4-multiple-remotes'

2022-05-20T22:26:55Z

"git p4" update. * kf/p4-multiple-remotes: git-p4: fix issue with multiple perforce remotes

Merge branch 'tk/p4-with-explicity-sync'

2022-05-20T22:26:55Z

"git p4" update. * tk/p4-with-explicity-sync: git-p4: support explicit sync of arbitrary existing git-p4 refs

Merge branch 'tk/p4-utf8-bom'

2022-05-20T22:26:54Z

"git p4" update. * tk/p4-utf8-bom: git-p4: preserve utf8 BOM when importing from p4 to git

git-p4: improve encoding handling to support inconsistent encodings

2022-05-04T17:30:01Z

git-p4 is designed to run correctly under python2.7 and python3, but its functional behavior wrt importing user-entered text differs across these environments: Under python2, git-p4 "naively" writes the Perforce bytestream into git metadata (and does not set an "encoding" header on the commits); this means that any non-utf-8 byte sequences end up creating invalidly-encoded commit metadata in git. Under python3, git-p4 attempts to decode the Perforce bytestream as utf-8 data, and fails badly (with an unhelpful error) when non-utf-8 data is encountered. Perforce clients (especially p4v) encourage user entry of changelist descriptions (and user full names) in OS-local encoding, and store the resulting bytestream to the server unmodified - such that different clients can end up creating mutually-unintelligible messages. The most common inconsistency, in many Perforce environments, is likely to be utf-8 (typical in linux) vs cp-1252 (typical in windows). Make the changelist-description- and user-fullname-handling code python-runtime-agnostic, introducing three "strategies" selectable via config: - 'passthrough', behaving as previously under python2, - 'strict', behaving as previously under python3, and - 'fallback', favoring utf-8 but supporting a secondary encoding when utf-8 decoding fails, and finally escaping high-range bytes if the decoding with the secondary encoding also fails. Keep the python2 default behavior as-is ('legacy' strategy), but switch the python3 default strategy to 'fallback' with default fallback encoding 'cp1252'. Also include tests exercising these encoding strategies, documentation for the new config, and improve the user-facing error messages when decoding does fail. Signed-off-by: Tao Klerks Signed-off-by: Junio C Hamano

git-p4: preserve utf8 BOM when importing from p4 to git

2022-04-06T19:59:58Z

Perforce has a file type "utf8" which represents a text file with explicit BOM. utf8-encoded files *without* BOM are stored as regular file type "text". The "utf8" file type behaves like text in all but one important way: it is stored, internally, without the leading 3 BOM bytes. git-p4 has historically imported utf8-with-BOM files (files stored, in Perforce, as type "utf8") the same way as regular text files - losing the BOM in the process. Under most circumstances this issue has little functional impact, as most systems consider the BOM to be optional and redundant, but this *is* a correctness failure, and can have lead to practical issues for example when BOMs are explicitly included in test files, for example in a file encoding test suite. Fix the handling of utf8-with-BOM files when importing changes from p4 to git, and introduce a test that checks it is working correctly. Signed-off-by: Tao Klerks Signed-off-by: Junio C Hamano