git/read-cache.c, branch v1.6.3

checkout bugfix: use stat.mtime instead of stat.ctime in two places

2009-03-15T19:56:40Z

Commit e1afca4fd "write_index(): update index_state->timestamp after flushing to disk" on 2009-02-23 used stat.ctime to record the timestamp of the index-file. This is wrong, so fix this and use the correct stat.mtime timestamp instead. Commit 110c46a909 "Not all systems use st_[cm]tim field for ns resolution file timestamp" on 2009-03-08, has a similar bug for the builtin-fetch-pack.c file. Signed-off-by: Kjetil Barvik Signed-off-by: Junio C Hamano

Not all systems use st_[cm]tim field for ns resolution file timestamp

2009-03-08T21:04:39Z

Some codepaths do not still use the ST_[CM]TIME_NSEC() pair of macros introduced by the previous commit but assumes all systems use st_mtim and st_ctim fields in "struct stat" to record nanosecond resolution part of the file timestamps. Signed-off-by: Junio C Hamano

Record ns-timestamps if possible, but do not use it without USE_NSEC

2009-03-08T04:25:16Z

Traditionally, the lack of USE_NSEC meant "do not record nor use the nanosecond resolution part of the file timestamps". To avoid problems on filesystems that lose the ns part when the metadata is flushed to the disk and then later read back in, disabling USE_NSEC has been a good idea in general. If you are on a filesystem without such an issue, it does not hurt to read and store them in the cached stat data in the index entries even if your git is compiled without USE_NSEC. The index left with such a version of git can be read by git compiled with USE_NSEC and it can make use of the nanosecond part to optimize the check to see if the path on the filesystem hsa been modified since we last looked at. Signed-off-by: Junio C Hamano

write_index(): update index_state->timestamp after flushing to disk

2009-02-24T02:04:20Z

Since this timestamp is used to check for racy-clean files, it is important to keep it uptodate. For the 'git checkout' command without the '-q' option, this make a huge difference. Before, each and every file which was updated, was racy-clean after the call to unpack_trees() and write_index() but before the GIT process ended. And because of the call to show_local_changes() in builtin-checkout.c, we ended up reading those files back into memory, doing a SHA1 to check if the files was really different from the index. And, of course, no file was different. With this fix, 'git checkout' without the '-q' option should now be almost as fast as with the '-q' option, but not quite, as we still do some few lstat(2) calls more without the '-q' option. Below is some average numbers for 10 checkout's to v2.6.27 and 10 to v2.6.25 of the Linux kernel, to show the difference: before (git version 1.6.2.rc1.256.g58a87): 7.860 user 2.427 sys 19.465 real 52.8% CPU faults: 0 major 95331 minor after: 6.184 user 2.160 sys 17.619 real 47.4% CPU faults: 0 major 38994 minor Signed-off-by: Kjetil Barvik Signed-off-by: Junio C Hamano

make USE_NSEC work as expected

2009-02-20T05:39:48Z

Since the filesystem ext4 is now defined as stable in Linux v2.6.28, and ext4 supports nanonsecond resolution timestamps natively, it is time to make USE_NSEC work as expected. This will make racy git situations less likely to happen. For 'git checkout' this means it will be less likely that we have to open, read the contents of the file into RAM, and check if file is really modified or not. The result sould be a litle less used CPU time, less pagefaults and a litle faster program, at least for 'git checkout'. Since the number of possible racy git situations would increase when disks gets faster, this patch would be more and more helpfull as times go by. For a fast Solid State Disk, this patch should be helpfull. Note that, when file operations starts to take less than 1 nanosecond, one would again start to get more racy git situations. For more info on racy git, see Documentation/technical/racy-git.txt For more info on ext4, see http://kernelnewbies.org/Ext4 Signed-off-by: Kjetil Barvik Signed-off-by: Junio C Hamano

check_updates(): effective removal of cache entries marked CE_REMOVE

2009-02-19T01:11:21Z

Below is oprofile output from GIT command 'git chekcout -q my-v2.6.25' (move from tag v2.6.27 to tag v2.6.25 of the Linux kernel): CPU: Core 2, speed 1999.95 MHz (estimated) Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 20000 Counted INST_RETIRED_ANY_P events (number of instructions retired) with a unit mask of 0x00 (No unit mask) count 20000 CPU_CLK_UNHALT...|INST_RETIRED:2...| samples| %| samples| %| ------------------------------------ 409247 100.000 342878 100.000 git CPU_CLK_UNHALT...|INST_RETIRED:2...| samples| %| samples| %| ------------------------------------ 260476 63.6476 257843 75.1996 libz.so.1.2.3 100876 24.6492 64378 18.7758 kernel-2.6.28.4_2.vmlinux 30850 7.5382 7874 2.2964 libc-2.9.so 14775 3.6103 8390 2.4469 git 2020 0.4936 4325 1.2614 libcrypto.so.0.9.8 191 0.0467 32 0.0093 libpthread-2.9.so 58 0.0142 36 0.0105 ld-2.9.so 1 2.4e-04 0 0 libldap-2.3.so.0.2.31 Detail list of the top 20 function entries (libz counted in one blob): CPU_CLK_UNHALTED INST_RETIRED_ANY_P samples % samples % image name symbol name 260476 63.6862 257843 75.2725 libz.so.1.2.3 /lib/libz.so.1.2.3 16587 4.0555 3636 1.0615 libc-2.9.so memcpy 7710 1.8851 277 0.0809 libc-2.9.so memmove 3679 0.8995 1108 0.3235 kernel-2.6.28.4_2.vmlinux d_validate 3546 0.8670 2607 0.7611 kernel-2.6.28.4_2.vmlinux __getblk 3174 0.7760 1813 0.5293 libc-2.9.so _int_malloc 2396 0.5858 3681 1.0746 kernel-2.6.28.4_2.vmlinux copy_to_user 2270 0.5550 2528 0.7380 kernel-2.6.28.4_2.vmlinux __link_path_walk 2205 0.5391 1797 0.5246 kernel-2.6.28.4_2.vmlinux ext4_mark_iloc_dirty 2103 0.5142 1203 0.3512 kernel-2.6.28.4_2.vmlinux find_first_zero_bit 2077 0.5078 997 0.2911 kernel-2.6.28.4_2.vmlinux do_get_write_access 2070 0.5061 514 0.1501 git cache_name_compare 2043 0.4995 1501 0.4382 kernel-2.6.28.4_2.vmlinux rcu_irq_exit 2022 0.4944 1732 0.5056 kernel-2.6.28.4_2.vmlinux __ext4_get_inode_loc 2020 0.4939 4325 1.2626 libcrypto.so.0.9.8 /usr/lib/libcrypto.so.0.9.8 1965 0.4804 1384 0.4040 git patch_delta 1708 0.4176 984 0.2873 kernel-2.6.28.4_2.vmlinux rcu_sched_grace_period 1682 0.4112 727 0.2122 kernel-2.6.28.4_2.vmlinux sysfs_slab_alias 1659 0.4056 290 0.0847 git find_pack_entry_one 1480 0.3619 1307 0.3816 kernel-2.6.28.4_2.vmlinux ext4_writepage_trans_blocks Notice the memmove line, where the CPU did 7710 / 277 = 27.8 cycles per instruction, and compared to the total cycles spent inside the source code of GIT for this command, all the memmove() calls translates to (7710 * 100) / 14775 = 52.2% of this. Retesting with a GIT program compiled for gcov usage, I found out that the memmove() calls came from remove_index_entry_at() in read-cache.c, where we have: memmove(istate->cache + pos, istate->cache + pos + 1, (istate->cache_nr - pos) * sizeof(struct cache_entry *)); remove_index_entry_at() is called 4902 times from check_updates() in unpack-trees.c, and each time called we move each cache_entry pointers (from the removed one) one step to the left. Since we have 28828 entries in the cache this time, and if we on average move half of them each time, we in total move approximately 4902 * 0.5 * 28828 * 4 = 282 629 712 bytes, or twice this amount if each pointer is 8 bytes (64 bit). OK, is seems that the function check_updates() is called 28 times, so the estimated guess above had been more correct if check_updates() had been called only once, but the point is: we get lots of bytes moved. To fix this, and use an O(N) algorithm instead, where N is the number of cache_entries, we delete/remove all entries in one loop through all entries. From a retest, the new remove_marked_cache_entries() from the patch below, ended up with the following output line from oprofile: 46 0.0105 15 0.0041 git remove_marked_cache_entries If we can trust the numbers from oprofile in this case, we saved approximately ((7710 - 46) * 20000) / (2 * 1000 * 1000 * 1000) = 0.077 seconds CPU time with this fix for this particular test. And notice that now the CPU did only 46 / 15 = 3.1 cycles/instruction. Signed-off-by: Kjetil Barvik Acked-by: Linus Torvalds Signed-off-by: Junio C Hamano

add -u: do not fail to resolve a path as deleted

2009-01-29T01:29:33Z

After you resolve a conflicted merge to remove the path, "git add -u" failed to record the removal. Instead it errored out by saying that the removed path is not found in the work tree, but that is what the user already knows, and the wanted to record the removal as the resolution, so the error does not make sense. Signed-off-by: Junio C Hamano

Make 'ce_compare_link()' use the new 'strbuf_readlink()'

2008-12-17T21:36:34Z

This simplifies the code, and also makes ce_compare_link now able to handle filesystems with odd 'st_size' return values for symlinks. Signed-off-by: Linus Torvalds Signed-off-by: Junio C Hamano

read-cache.c: typofix in comment

2008-12-08T03:08:23Z

Signed-off-by: Junio C Hamano

git add --intent-to-add: do not let an empty blob be committed by accident

2008-12-01T01:59:19Z

Writing a tree out of an index with an "intent to add" entry is blocked. This implies that you cannot "git commit" from such a state; however you can still do "git commit -a" or "git commit $that_path". Signed-off-by: Junio C Hamano