git/perl, branch v2.2.2

git-svn: use SVN::Ra::get_dir2 when possible

2014-10-31T10:34:54Z

This avoids the following failure with normal "get_dir" on newer versions of SVN (tested with SVN 1.8.8-1ubuntu3.1): Incorrect parameters given: Could not convert '%ld' into a number get_dir2 also has the potential to be more efficient by requesting less data. ref: <1414636504.45506.YahooMailBasic@web172304.mail.ir2.yahoo.com> ref: <1414722617.89476.YahooMailBasic@web172305.mail.ir2.yahoo.com> Signed-off-by: Eric Wong Cc: Hin-Tak Leung

git-svn: add space after "W:" prefix in warning

2014-10-30T08:31:28Z

And minor reformatting while we're in the area. Signed-off-by: Eric Wong

git-svn: (cleanup) remove editor param passing

2014-10-30T08:29:43Z

Neither find_extra_svk_parents or find_extra_svn_parents ever used the `$ed' parameter. Signed-off-by: Eric Wong

git-svn: prepare SVN::Ra config pieces once

2014-10-29T19:59:25Z

Memoizing these initialization functions saves some memory for long fetches which require scanning many unwanted revisions before any wanted revisions happen. Signed-off-by: Eric Wong

Git.pm: add specified name to tempfile template

2014-10-29T19:59:23Z

This should help me track down errors in git-svn more easily: write .git/Git_XXXXXX: Bad file descriptor at /usr/lib/perl5/SVN/Ra.pm line 623 Signed-off-by: Eric Wong

git-svn: disable _rev_list memoization

2014-10-27T01:39:39Z

This memoization appears unneeded as the check_cherry_pick2 cache is in front of it does enough. With this change applied, importing from local svn+ssh and http copies of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2]. This 60M measurement is only for the git-svn Perl process itself and does not include memory used by git subprocesses accessing large packs (subprocess memory usage _is_ measured by my time(1) tool). Before this change, an import took longer (2:20) on svn+ssh:// but git-svn used around 240MB during the imports. Worse yet, git-svn ballooned to over 400M when writing out the cache to the filesystem. I also tried removing memoization for `has_no_changes', too, but a local copy of the R repository(*) was not close to finishing within 10 hours on my system. [1] http://svn.r-project.org/R [2] file:// repos causes libsvn to use more memory internally Signed-off-by: Eric Wong Cc: Hin-Tak Leung

git-svn: save a little memory as fetch progresses

2014-10-25T07:56:33Z

There is no reason to keep entries in the %revs hash after we're done processing a revision, so allow entries become freed as processing continues. Signed-off-by: Eric Wong

git-svn: remove unnecessary DESTROY override

2014-10-25T07:56:28Z

This override was probably never necessary, but most likely a no-op as it does not appear to do anything in SVN::Ra itself. Signed-off-by: Eric Wong

git-svn: reload RA every log-window-size

2014-10-24T22:56:06Z

Despite attempting to use local memory pools everywhere we can, (including our call to SVN::Ra::do_update and all subsequent reporter calls), there does not appear to be a way to force the Git::SVN::Fetcher callbacks to use a pool other than the per-SVN::Ra pool. Git::SVN::Fetcher ends up using the main RA pool which grows monotonically in size for the lifetime of the RA object. Thus the only way to free that memory appears to be to destroy and recreate the RA connection for at every --log-window-size interval. This reduces memory usage over the course of fetching 10K revisions using a test repository created with the script at the end of this commit message. As reported by time(1) on my x86-64 system: before: 54024k after: 28680k Unfortunately, there remains some yet-to-be-tracked-down slow memory growth which would be evident as the `nr' parameter increases in the repository generation script: -----------------------------8<------------------------------ set -e tmp=$(mktemp -d svntestrepo-XXXXXXXX) svnadmin create "$tmp" repo=file://"$(cd $tmp && pwd)" svn co "$repo" "$tmp/wd" cd "$tmp/wd" if ! test -f a then > a svn add a svn commit -m 'A' fi nr=10000 while test $nr -gt 0 do echo $nr > a svn commit -q -m A nr=$((nr - 1)) done echo "repository created in $repo" -----------------------------8<------------------------------ Signed-off-by: Eric Wong

git-svn: remove mergeinfo rev caching

2014-10-24T22:55:43Z

This should further reduce memory usage from the new mergeinfo speedups without hurting performance too much, assuming reasonable latency to the SVN server. Cc: Hin-Tak Leung Suggested-by: Jakob Stoklund Olesen Signed-off-by: Eric Wong