autoconf: Using Autotest
1
1 19 Generating Test Suites with Autotest
1 ***************************************
1
1 *N.B.: This section describes a feature which is still
1 stabilizing. Although we believe that Autotest is useful as-is, this
1 documentation describes an interface which might change in the future:
1 do not depend upon Autotest without subscribing to the Autoconf mailing
1 lists.*
1
1 It is paradoxical that portable projects depend on nonportable tools
1 to run their test suite. Autoconf by itself is the paragon of this
1 problem: although it aims at perfectly portability, up to 2.13 its test
1 suite was using DejaGNU, a rich and complex testing framework, but
1 which is far from being standard on Posix systems. Worse yet, it was
1 likely to be missing on the most fragile platforms, the very platforms
1 that are most likely to torture Autoconf and exhibit deficiencies.
1
1 To circumvent this problem, many package maintainers have developed
1 their own testing framework, based on simple shell scripts whose sole
1 outputs are exit status values describing whether the test succeeded.
1 Most of these tests share common patterns, and this can result in lots
1 of duplicated code and tedious maintenance.
1
1 Following exactly the same reasoning that yielded to the inception of
1 Autoconf, Autotest provides a test suite generation framework, based on
1 M4 macros building a portable shell script. The suite itself is
1 equipped with automatic logging and tracing facilities which greatly
1 diminish the interaction with bug reporters, and simple timing reports.
1
1 Autoconf itself has been using Autotest for years, and we do attest
1 that it has considerably improved the strength of the test suite and the
1 quality of bug reports. Other projects are known to use some generation
1 of Autotest, such as Bison, Free Recode, Free Wdiff, GNU Tar, each of
1 them with different needs, and this usage has validated Autotest as a
1 general testing framework.
1
1 Nonetheless, compared to DejaGNU, Autotest is inadequate for
1 interactive tool testing, which is probably its main limitation.
1
Menu