automake: Command-line arguments for test drivers
1
1 15.3.3.1 Command-line arguments for test drivers
1 ................................................
1
1 A custom driver can rely on various command-line options and arguments
1 being passed to it automatically by the Automake-generated test harness.
1 It is _mandatory_ that it understands all of them (even if the exact
1 interpretation of the associated semantics can legitimately change
1 between a test driver and another, and even be a no-op in some drivers).
1
1 Here is the list of options:
1
1 ‘--test-name=NAME’
1 The name of the test, with VPATH prefix (if any) removed. This can
1 have a suffix and a directory component (as in e.g.,
1 ‘sub/foo.test’), and is mostly meant to be used in console reports
11 about testsuite advancements and results (⇒Testsuite progress
output).
1 ‘--log-file=PATH.log’
11 The ‘.log’ file the test driver must create (⇒Basics of test
metadata). If it has a directory component (as in e.g.,
1 ‘sub/foo.log’), the test harness will ensure that such directory
1 exists _before_ the test driver is called.
1 ‘--trs-file=PATH.trs’
11 The ‘.trs’ file the test driver must create (⇒Basics of test
metadata). If it has a directory component (as in e.g.,
1 ‘sub/foo.trs’), the test harness will ensure that such directory
1 exists _before_ the test driver is called.
1 ‘--color-tests={yes|no}’
11 Whether the console output should be colorized or not (⇒Simple
tests and color-tests, to learn when this option gets activated
1 and when it doesn’t).
1 ‘--expect-failure={yes|no}’
1 Whether the tested program is expected to fail.
1 ‘--enable-hard-errors={yes|no}’
1 Whether “hard errors” in the tested program should be treated
1 differently from normal failures or not (the default should be
1 ‘yes’). The exact meaning of “hard error” is highly dependent from
1 the test protocols or conventions in use.
1 ‘--’
1 Explicitly terminate the list of options.
1
1 The first non-option argument passed to the test driver is the program
1 to be run, and all the following ones are command-line options and
1 arguments for this program.
1
1 Note that the exact semantics attached to the ‘--color-tests’,
1 ‘--expect-failure’ and ‘--enable-hard-errors’ options are left up to the
1 individual test drivers. Still, having a behaviour compatible or at
1 least similar to that provided by the default driver is advised, as that
1 would offer a better consistency and a more pleasant user experience.
1