automake: Command-line arguments for test drivers

1 
1 15.3.3.1 Command-line arguments for test drivers
1 ................................................
1 
1 A custom driver can rely on various command-line options and arguments
1 being passed to it automatically by the Automake-generated test harness.
1 It is _mandatory_ that it understands all of them (even if the exact
1 interpretation of the associated semantics can legitimately change
1 between a test driver and another, and even be a no-op in some drivers).
1 
1 Here is the list of options:
1 
1 ‘--test-name=NAME’
1      The name of the test, with VPATH prefix (if any) removed.  This can
1      have a suffix and a directory component (as in e.g.,
1      ‘sub/foo.test’), and is mostly meant to be used in console reports
11      about testsuite advancements and results (⇒Testsuite progress
      output).
1 ‘--log-file=PATH.log’
11      The ‘.log’ file the test driver must create (⇒Basics of test
      metadata).  If it has a directory component (as in e.g.,
1      ‘sub/foo.log’), the test harness will ensure that such directory
1      exists _before_ the test driver is called.
1 ‘--trs-file=PATH.trs’
11      The ‘.trs’ file the test driver must create (⇒Basics of test
      metadata).  If it has a directory component (as in e.g.,
1      ‘sub/foo.trs’), the test harness will ensure that such directory
1      exists _before_ the test driver is called.
1 ‘--color-tests={yes|no}’
11      Whether the console output should be colorized or not (⇒Simple
      tests and color-tests, to learn when this option gets activated
1      and when it doesn’t).
1 ‘--expect-failure={yes|no}’
1      Whether the tested program is expected to fail.
1 ‘--enable-hard-errors={yes|no}’
1      Whether “hard errors” in the tested program should be treated
1      differently from normal failures or not (the default should be
1      ‘yes’).  The exact meaning of “hard error” is highly dependent from
1      the test protocols or conventions in use.
1 ‘--’
1      Explicitly terminate the list of options.
1 
1 The first non-option argument passed to the test driver is the program
1 to be run, and all the following ones are command-line options and
1 arguments for this program.
1 
1    Note that the exact semantics attached to the ‘--color-tests’,
1 ‘--expect-failure’ and ‘--enable-hard-errors’ options are left up to the
1 individual test drivers.  Still, having a behaviour compatible or at
1 least similar to that provided by the default driver is advised, as that
1 would offer a better consistency and a more pleasant user experience.
1