autoconf: testsuite Invocation
1
1 19.3 Running `testsuite' Scripts
1 ================================
1
1 Autotest test suites support the following options:
1
1 `--help'
1 `-h'
1 Display the list of options and exit successfully.
1
1 `--version'
1 `-V'
1 Display the version of the test suite and exit successfully.
1
1 `--directory=DIR'
1 `-C DIR'
1 Change the current directory to DIR before creating any files.
1 Useful for running the testsuite in a subdirectory from a top-level
1 Makefile.
1
1 `--jobs[=N]'
1 `-j[N]'
1 Run N tests in parallel, if possible. If N is not given, run all
1 given tests in parallel. Note that there should be no space
1 before the argument to `-j', as `-j NUMBER' denotes the separate
1 arguments `-j' and `NUMBER', see below.
1
1 In parallel mode, the standard input device of the testsuite
1 script is not available to commands inside a test group.
1 Furthermore, banner lines are not printed, and the summary line
1 for each test group is output after the test group completes.
1 Summary lines may appear unordered. If verbose and trace output
1 are enabled (see below), they may appear intermixed from
1 concurrently running tests.
1
1 Parallel mode requires the `mkfifo' command to work, and will be
1 silently disabled otherwise.
1
1 `--clean'
1 `-c'
1 Remove all the files the test suite might have created and exit.
1 Meant for `clean' Make targets.
1
1 `--list'
1 `-l'
1 List all the tests (or only the selection), including their
1 possible keywords.
1
1
1 By default all tests are performed (or described with `--list')
1 silently in the default environment, but the environment, set of tests,
1 and verbosity level can be tuned:
1
1 `VARIABLE=VALUE'
1 Set the environment VARIABLE to VALUE. Use this rather than
1 `FOO=foo ./testsuite' as debugging scripts would then run in a
1 different environment.
1
1 The variable `AUTOTEST_PATH' specifies the testing path to prepend
1 to `PATH'. Relative directory names (not starting with `/') are
1 considered to be relative to the top level of the package being
1 built. All directories are made absolute, first starting from the
1 top level _build_ tree, then from the _source_ tree. For instance
1 `./testsuite AUTOTEST_PATH=tests:bin' for a `/src/foo-1.0' source
1 package built in `/tmp/foo' results in
1 `/tmp/foo/tests:/tmp/foo/bin' and then
1 `/src/foo-1.0/tests:/src/foo-1.0/bin' being prepended to `PATH'.
1
1 `NUMBER'
1 `NUMBER-NUMBER'
1 `NUMBER-'
1 `-NUMBER'
1 Add the corresponding test groups, with obvious semantics, to the
1 selection.
1
1 `--keywords=KEYWORDS'
1 `-k KEYWORDS'
1 Add to the selection the test groups with title or keywords
1 (arguments to `AT_SETUP' or `AT_KEYWORDS') that match _all_
1 keywords of the comma separated list KEYWORDS, case-insensitively.
1 Use `!' immediately before the keyword to invert the selection for
1 this keyword. By default, the keywords match whole words; enclose
1 them in `.*' to also match parts of words.
1
1 For example, running
1
1 ./testsuite -k 'autoupdate,.*FUNC.*'
1
1 selects all tests tagged `autoupdate' _and_ with tags containing
1 `FUNC' (as in `AC_CHECK_FUNC', `AC_FUNC_ALLOCA', etc.), while
1
1 ./testsuite -k '!autoupdate' -k '.*FUNC.*'
1
1 selects all tests not tagged `autoupdate' _or_ with tags
1 containing `FUNC'.
1
1 `--errexit'
1 `-e'
1 If any test fails, immediately abort testing. This implies
1 `--debug': post test group clean up, and top-level logging are
1 inhibited. This option is meant for the full test suite, it is
1 not really useful for generated debugging scripts. If the
1 testsuite is run in parallel mode using `--jobs', then
1 concurrently running tests will finish before exiting.
1
1 `--verbose'
1 `-v'
1 Force more verbosity in the detailed output of what is being done.
1 This is the default for debugging scripts.
1
1 `--color'
1 `--color[=never|auto|always]'
1 Enable colored test results. Without an argument, or with
1 `always', test results will be colored. With `never', color mode
1 is turned off. Otherwise, if either the macro `AT_COLOR_TESTS' is
1 used by the testsuite author, or the argument `auto' is given,
1 then test results are colored if standard output is connected to a
1 terminal.
1
1 `--debug'
1 `-d'
1 Do not remove the files after a test group was performed--but they
1 are still removed _before_, therefore using this option is sane
1 when running several test groups. Create debugging scripts. Do
1 not overwrite the top-level log (in order to preserve a supposedly
1 existing full log file). This is the default for debugging
1 scripts, but it can also be useful to debug the testsuite itself.
1
1 `--recheck'
1 Add to the selection all test groups that failed or passed
1 unexpectedly during the last non-debugging test run.
1
1 `--trace'
1 `-x'
1 Trigger shell tracing of the test groups.
1
1 Besides these options accepted by every Autotest testsuite, the
1 testsuite author might have added package-specific options via the
1 Testsuites::); refer to `testsuite --help' and the package
1 documentation for details.
1