git/compat/fsmonitor, branch v2.37.2

Merge branch 'jh/builtin-fsmonitor-part3'

2022-06-10T22:04:15Z

More fsmonitor--daemon. * jh/builtin-fsmonitor-part3: (30 commits) t7527: improve implicit shutdown testing in fsmonitor--daemon fsmonitor--daemon: allow --super-prefix argument t7527: test Unicode NFC/NFD handling on MacOS t/lib-unicode-nfc-nfd: helper prereqs for testing unicode nfc/nfd t/helper/hexdump: add helper to print hexdump of stdin fsmonitor: on macOS also emit NFC spelling for NFD pathname t7527: test FSMonitor on case insensitive+preserving file system fsmonitor: never set CE_FSMONITOR_VALID on submodules t/perf/p7527: add perf test for builtin FSMonitor t7527: FSMonitor tests for directory moves fsmonitor: optimize processing of directory events fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed fsm-health-win32: force shutdown daemon if worktree root moves fsm-health-win32: add polling framework to monitor daemon health fsmonitor--daemon: stub in health thread fsmonitor--daemon: rename listener thread related variables fsmonitor--daemon: prepare for adding health thread fsmonitor--daemon: cd out of worktree root fsm-listen-darwin: ignore FSEvents caused by xattr changes on macOS unpack-trees: initialize fsmonitor_has_run_once in o->result ...

fsmonitor: on macOS also emit NFC spelling for NFD pathname

2022-05-26T22:59:27Z

Emit NFC or NFC and NFD spellings of pathnames on macOS. MacOS is Unicode composition insensitive, so NFC and NFD spellings are treated as aliases and collide. While the spelling of pathnames in filesystem events depends upon the underlying filesystem, such as APFS, HFS+ or FAT32, the OS enforces such collisions regardless of filesystem. Teach the daemon to always report the NFC spelling and to report the NFD spelling when stored in that format on the disk. This is slightly more general than "core.precomposeUnicode". Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed

2022-05-26T22:59:27Z

Teach the listener thread to shutdown the daemon if the spelling of the worktree root directory changes. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsm-health-win32: force shutdown daemon if worktree root moves

2022-05-26T22:59:27Z

Force shutdown fsmonitor daemon if the worktree root directory is moved, renamed, or deleted. Use Windows low-level GetFileInformationByHandle() to get and compare the Windows system unique ID for the directory with a cached version when we started up. This lets us detect the case where someone renames the directory that we are watching and then creates a new directory with the original pathname. This is important because we are listening to a named pipe for requests and they are stored in the Named Pipe File System (NPFS) which a kernel-resident pseudo filesystem not associated with the actual NTFS directory. For example, if the daemon was watching "~/foo/", it would have a directory-watch handle on that directory and a named-pipe handle for "//./pipe/...foo". Moving the directory to "~/bar/" does not invalidate the directory handle. (So the daemon would actually be watching "~/bar" but listening on "//./pipe/...foo". If the user then does "git init ~/foo" and causes another daemon to start, the first daemon will still have ownership of the pipe and the second daemon instance will fail to start. "git status" clients in "~/foo" will ask "//./pipe/...foo" about changes and the first daemon instance will tell them about "~/bar". This commit causes the first daemon to shutdown if the system unique ID for "~/foo" changes (changes from what it was when the daemon started). Shutdown occurs after a periodic poll. After the first daemon exits and releases the lock on the named pipe, subsequent Git commands may cause another daemon to be started on "~/foo". Similarly, a subsequent Git command may cause another daemon to be started on "~/bar". Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsm-health-win32: add polling framework to monitor daemon health

2022-05-26T22:59:27Z

Extend the Windows version of the "health" thread to periodically inspect the system and shutdown if warranted. This commit updates the thread's wait loop to use a timeout and defines a (currently empty) table of functions to poll the system. A later commit will add functions to the table to actually inspect the system. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsmonitor--daemon: stub in health thread

2022-05-26T22:59:27Z

Create another thread to watch over the daemon process and automatically shut it down if necessary. This commit creates the basic framework for a "health" thread to monitor the daemon and/or the file system. Later commits will add platform-specific code to do the actual work. The "health" thread is intended to monitor conditions that would be difficult to track inside the IPC thread pool and/or the file system listener threads. For example, when there are file system events outside of the watched worktree root or if we want to have an idle-timeout auto-shutdown feature. This commit creates the health thread itself, defines the thread-proc and sets up the thread's event loop. It integrates this new thread into the existing IPC and Listener thread models. This commit defines the API to the platform-specific code where all of the monitoring will actually happen. The platform-specific code for MacOS is just stubs. Meaning that the health thread will immediately exit on MacOS, but that is OK and expected. Future work can define MacOS-specific monitoring. The platform-specific code for Windows sets up enough of the WaitForMultipleObjects() machinery to watch for system and/or custom events. Currently, the set of wait handles only includes our custom shutdown event (sent from our other theads). Later commits in this series will extend the set of wait handles to monitor other conditions. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsmonitor--daemon: rename listener thread related variables

2022-05-26T22:59:26Z

Rename platform-specific listener thread related variables and data types as we prepare to add another backend thread type. [] `struct fsmonitor_daemon_backend_data` becomes `struct fsm_listen_data` [] `state->backend_data` becomes `state->listen_data` [] `state->error_code` becomes `state->listen_error_code` Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsmonitor--daemon: cd out of worktree root

2022-05-26T22:59:26Z

Teach the fsmonitor--daemon to CD outside of the worktree before starting up. The common Git startup mechanism causes the CWD of the daemon process to be in the root of the worktree. On Windows, this causes the daemon process to hold a locked handle on the CWD and prevents other processes from moving or deleting the worktree while the daemon is running. CD to HOME before entering main event loops. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsm-listen-darwin: ignore FSEvents caused by xattr changes on macOS

2022-05-26T22:59:26Z

Ignore FSEvents resulting from `xattr` changes. Git does not care about xattr's or changes to xattr's, so don't waste time collecting these events in the daemon nor transmitting them to clients. Various security tools add xattrs to files and/or directories, such as to mark them as having been downloaded. We should ignore these events since it doesn't affect the content of the file/directory or the normal meta-data that Git cares about. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano

fsmonitor-settings: NTFS and FAT32 on MacOS are incompatible

2022-05-26T22:59:26Z

On MacOS mark repos on NTFS or FAT32 volumes as incompatible. The builtin FSMonitor used Unix domain sockets on MacOS for IPC with clients. These sockets are kept in the .git directory. Unix sockets are not supported by NTFS and FAT32, so the daemon cannot start up. Test for this during our compatibility checking so that client commands do not keep trying to start the daemon. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano