autoconf: testsuite Invocation

1 
1 19.3 Running `testsuite' Scripts
1 ================================
1 
1 Autotest test suites support the following options:
1 
1 `--help'
1 `-h'
1      Display the list of options and exit successfully.
1 
1 `--version'
1 `-V'
1      Display the version of the test suite and exit successfully.
1 
1 `--directory=DIR'
1 `-C DIR'
1      Change the current directory to DIR before creating any files.
1      Useful for running the testsuite in a subdirectory from a top-level
1      Makefile.
1 
1 `--jobs[=N]'
1 `-j[N]'
1      Run N tests in parallel, if possible.  If N is not given, run all
1      given tests in parallel.  Note that there should be no space
1      before the argument to `-j', as `-j NUMBER' denotes the separate
1      arguments `-j' and `NUMBER', see below.
1 
1      In parallel mode, the standard input device of the testsuite
1      script is not available to commands inside a test group.
1      Furthermore, banner lines are not printed, and the summary line
1      for each test group is output after the test group completes.
1      Summary lines may appear unordered.  If verbose and trace output
1      are enabled (see below), they may appear intermixed from
1      concurrently running tests.
1 
1      Parallel mode requires the `mkfifo' command to work, and will be
1      silently disabled otherwise.
1 
1 `--clean'
1 `-c'
1      Remove all the files the test suite might have created and exit.
1      Meant for `clean' Make targets.
1 
1 `--list'
1 `-l'
1      List all the tests (or only the selection), including their
1      possible keywords.
1 
1 
1    By default all tests are performed (or described with `--list')
1 silently in the default environment, but the environment, set of tests,
1 and verbosity level can be tuned:
1 
1 `VARIABLE=VALUE'
1      Set the environment VARIABLE to VALUE.  Use this rather than
1      `FOO=foo ./testsuite' as debugging scripts would then run in a
1      different environment.
1 
1      The variable `AUTOTEST_PATH' specifies the testing path to prepend
1      to `PATH'.  Relative directory names (not starting with `/') are
1      considered to be relative to the top level of the package being
1      built.  All directories are made absolute, first starting from the
1      top level _build_ tree, then from the _source_ tree.  For instance
1      `./testsuite AUTOTEST_PATH=tests:bin' for a `/src/foo-1.0' source
1      package built in `/tmp/foo' results in
1      `/tmp/foo/tests:/tmp/foo/bin' and then
1      `/src/foo-1.0/tests:/src/foo-1.0/bin' being prepended to `PATH'.
1 
1 `NUMBER'
1 `NUMBER-NUMBER'
1 `NUMBER-'
1 `-NUMBER'
1      Add the corresponding test groups, with obvious semantics, to the
1      selection.
1 
1 `--keywords=KEYWORDS'
1 `-k KEYWORDS'
1      Add to the selection the test groups with title or keywords
1      (arguments to `AT_SETUP' or `AT_KEYWORDS') that match _all_
1      keywords of the comma separated list KEYWORDS, case-insensitively.
1      Use `!' immediately before the keyword to invert the selection for
1      this keyword.  By default, the keywords match whole words; enclose
1      them in `.*' to also match parts of words.
1 
1      For example, running
1 
1           ./testsuite -k 'autoupdate,.*FUNC.*'
1 
1      selects all tests tagged `autoupdate' _and_ with tags containing
1      `FUNC' (as in `AC_CHECK_FUNC', `AC_FUNC_ALLOCA', etc.), while
1 
1           ./testsuite -k '!autoupdate' -k '.*FUNC.*'
1 
1      selects all tests not tagged `autoupdate' _or_ with tags
1      containing `FUNC'.
1 
1 `--errexit'
1 `-e'
1      If any test fails, immediately abort testing.  This implies
1      `--debug': post test group clean up, and top-level logging are
1      inhibited.  This option is meant for the full test suite, it is
1      not really useful for generated debugging scripts.  If the
1      testsuite is run in parallel mode using `--jobs', then
1      concurrently running tests will finish before exiting.
1 
1 `--verbose'
1 `-v'
1      Force more verbosity in the detailed output of what is being done.
1      This is the default for debugging scripts.
1 
1 `--color'
1 `--color[=never|auto|always]'
1      Enable colored test results.  Without an argument, or with
1      `always', test results will be colored.  With `never', color mode
1      is turned off.  Otherwise, if either the macro `AT_COLOR_TESTS' is
1      used by the testsuite author, or the argument `auto' is given,
1      then test results are colored if standard output is connected to a
1      terminal.
1 
1 `--debug'
1 `-d'
1      Do not remove the files after a test group was performed--but they
1      are still removed _before_, therefore using this option is sane
1      when running several test groups.  Create debugging scripts.  Do
1      not overwrite the top-level log (in order to preserve a supposedly
1      existing full log file).  This is the default for debugging
1      scripts, but it can also be useful to debug the testsuite itself.
1 
1 `--recheck'
1      Add to the selection all test groups that failed or passed
1      unexpectedly during the last non-debugging test run.
1 
1 `--trace'
1 `-x'
1      Trigger shell tracing of the test groups.
1 
1    Besides these options accepted by every Autotest testsuite, the
1 testsuite author might have added package-specific options via the
1 Testsuites::); refer to `testsuite --help' and the package
1 documentation for details.
1