tests: fix races in and standardize the tail tests

* tests/tail-2/F-vs-missing.sh: Use standard "fastpoll" options
(-s.1 --max-unchanged-stats=1) to speedup the non-inotify case.
Add the non-inotify case to the test. `wait` on the background
tail process to terminate which should avoid the need for the
non standard `retry_delay_ cleanup ...` on NFS.
* tests/tail-2/F-vs-rename.sh: Remove 'out' at the start of the loop,
to avoid a race in checking its contents.  Also ensure 'a' & 'b'
files are present before the tail process starts.  Use the standard
"fastpoll" options as above.
* tests/tail-2/f-vs-rename.sh: Likewise.
* tests/tail-2/append-only.sh: Use more standard variable names.
* tests/tail-2/flush-initial.sh: Use "fastpoll" options for
non-inotify platforms.  Also `wait` on the background tail to avoid
stray processes and file cleanup issues on NFS.
* tests/tail-2/inotify-hash-abuse.sh: Always run non-inotify case.
Use "fastpoll" options.  Use a more standard retry_delay_ instead
of a hardcoded sleep loop.  Add a `wait` on the background tail.
* tests/tail-2/inotify-hash-abuse2.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Wait just on the
specific tail $pid needed.
* tests/tail-2/inotify-rotate.sh: Use "fastpoll" options.
* tests/tail-2/pid.sh: Use standard variable names.
Add a `wait` on the background tails.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/tail-2/tail-n0f.sh: Likewise.
* tests/tail-2/retry.sh: Use "fastpoll" options.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.  Speedup by using sub second
parameters to timeout(1).  Improve the part ensuring that
-F never follows a renamed file.
* tests/tail-2/infloop-1.sh: Remove invalid test.  tail(1) was not
being passed the --pid=$yes_pid option, retry_delay_ wasn't used
to avoid races, and yes could write huge files before being killed.
* tests/local.mk: Remove the invalid test reference.
* tests/tail-2/assert-2.sh: Rewrite using retry_delay_().  Since
no longer hardcoding large delays, remove the VERY_EXPENSIVE tag.
* tests/tail-2/assert.sh: Likewise.

1 files changed, 37 insertions, 31 deletions

diff --git a/tests/tail-2/assert.sh b/tests/tail-2/assert.sh
index faf7e32d0..906e364b5 100755
--- a/tests/tail-2/assert.sh
+++ b/tests/tail-2/assert.sh
@@ -26,36 +26,42 @@
 . "${srcdir=.}/tests/init.sh"; path_prepend_ ./src
 print_ver_ tail
 
-# Not "expensive" per se, but sleeping for so long is annoying.
-very_expensive_
-
-ok='ok ok ok'
-
-touch a foo
-tail --follow=name a foo > err 2>&1 &
-tail_pid=$!
-# Arrange for the tail process to die after 12 seconds.
-(sleep 12; kill $tail_pid) &
-
-echo sleeping for 7 seconds...
-
-# Give the backgrounded 'tail' a chance to start before removing foo.
-# Otherwise, without --retry, tail wouldn't try to open 'foo' again.
-sleep 1
-
-rm -f foo
-sleep 6
-echo $ok > f
-mv f foo
-
-# echo waiting....
-wait
-
-case "$(cat err)" in
-  *$ok) ;;
-  *) fail=1;;
-esac
-
-test $fail = 1 && cat err
+check_tail_output()
+{
+  local delay="$1"
+  grep "$tail_re" out ||
+    { sleep $delay; return 1; }
+}
+
+# Speedup the non inotify case
+fastpoll='-s.1 --max-unchanged-stats=1'
+
+
+for mode in '' '---disable-inotify'; do
+  rm -f a foo out
+  touch a foo || framework_failure_
+
+  tail $mode --follow=name $fastpoll a foo > out 2>&1 & pid=$!
+
+  # Wait up to 12.7s for tail to start.
+  echo x > a || framework_failure_
+  tail_re='^x$' retry_delay_ check_tail_output .1 7 ||
+    { cat out; fail=1; }
+
+  # Wait 12.7s for this diagnostic:
+  # tail: foo: No such file or directory
+  rm foo || framework_failure_
+  tail_re='No such file' retry_delay_ check_tail_output .1 7 ||
+    { cat out; fail=1; }
+
+  # Wait up to 12.7s for tail to notice new foo file
+  ok='ok ok ok'
+  echo "$ok" > foo || framework_failure_
+  tail_re="^$ok$" retry_delay_ check_tail_output .1 7 ||
+    { echo "$0: foo: unexpected delay?"; cat out; fail=1; }
+
+  kill $pid
+  wait $pid
+done
 
 Exit $fail