autoconf: Making testsuite Scripts

1 
1 19.4 Making `testsuite' Scripts
1 ===============================
1 
1 For putting Autotest into movement, you need some configuration and
1 makefile machinery.  We recommend, at least if your package uses deep or
1 shallow hierarchies, that you use `tests/' as the name of the directory
1 holding all your tests and their makefile.  Here is a check list of
1 things to do.
1 
1    - Make sure to create the file `package.m4', which defines the
1      identity of the package.  It must define `AT_PACKAGE_STRING', the
1      full signature of the package, and `AT_PACKAGE_BUGREPORT', the
1      address to which bug reports should be sent.  For sake of
1      completeness, we suggest that you also define `AT_PACKAGE_NAME',
1      `AT_PACKAGE_TARNAME', `AT_PACKAGE_VERSION', and `AT_PACKAGE_URL'.
1      ⇒Initializing configure, for a description of these
1      variables.  Be sure to distribute `package.m4' and to put it into
1      the source hierarchy: the test suite ought to be shipped!  See
1      below for an example `Makefile' excerpt.
1 
1    - Invoke `AC_CONFIG_TESTDIR'.
1 
1       -- Macro: AC_CONFIG_TESTDIR (DIRECTORY, [TEST-PATH = `directory'])
1           An Autotest test suite is to be configured in DIRECTORY.  This
1           macro causes `DIRECTORY/atconfig' to be created by
1           `config.status' and sets the default `AUTOTEST_PATH' to
1           TEST-PATH (⇒testsuite Invocation).
1 
1    - Still within `configure.ac', as appropriate, ensure that some
1      `AC_CONFIG_FILES' command includes substitution for
1      `tests/atlocal'.
1 
1    - The appropriate `Makefile' should be modified so the validation in
1      your package is triggered by `make check'.  An example is provided
1      below.
1 
1    With Automake, here is a minimal example for inclusion in
1 `tests/Makefile.am', in order to link `make check' with a validation
1 suite.
1 
1      # The `:;' works around a Bash 3.2 bug when the output is not writable.
1      $(srcdir)/package.m4: $(top_srcdir)/configure.ac
1              :;{ \
1                echo '# Signature of the current package.' && \
1                echo 'm4_define([AT_PACKAGE_NAME],' && \
1                echo '  [$(PACKAGE_NAME)])' && \
1                echo 'm4_define([AT_PACKAGE_TARNAME],' && \
1                echo '  [$(PACKAGE_TARNAME)])' && \
1                echo 'm4_define([AT_PACKAGE_VERSION],' && \
1                echo '  [$(PACKAGE_VERSION)])' && \
1                echo 'm4_define([AT_PACKAGE_STRING],' && \
1                echo '  [$(PACKAGE_STRING)])' && \
1                echo 'm4_define([AT_PACKAGE_BUGREPORT],' && \
1                echo '  [$(PACKAGE_BUGREPORT)])'; \
1                echo 'm4_define([AT_PACKAGE_URL],' && \
1                echo '  [$(PACKAGE_URL)])'; \
1              } >'$(srcdir)/package.m4'
1 
1      EXTRA_DIST = testsuite.at $(srcdir)/package.m4 $(TESTSUITE) atlocal.in
1      TESTSUITE = $(srcdir)/testsuite
1 
1      check-local: atconfig atlocal $(TESTSUITE)
1              $(SHELL) '$(TESTSUITE)' $(TESTSUITEFLAGS)
1 
1      installcheck-local: atconfig atlocal $(TESTSUITE)
1              $(SHELL) '$(TESTSUITE)' AUTOTEST_PATH='$(bindir)' \
1                $(TESTSUITEFLAGS)
1 
1      clean-local:
1              test ! -f '$(TESTSUITE)' || \
1               $(SHELL) '$(TESTSUITE)' --clean
1 
1      AUTOM4TE = $(SHELL) $(srcdir)/build-aux/missing --run autom4te
1      AUTOTEST = $(AUTOM4TE) --language=autotest
1      $(TESTSUITE): $(srcdir)/testsuite.at $(srcdir)/package.m4
1              $(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
1              mv $@.tmp $@
1 
1    Note that the built testsuite is distributed; this is necessary
1 because users might not have Autoconf installed, and thus would not be
1 able to rebuild it.  Likewise, the use of `missing' provides the user
1 with a nicer error message if they modify a source file to the
1 testsuite, and accidentally trigger the rebuild rules.
1 
1    You might want to list explicitly the dependencies, i.e., the list of
1 the files `testsuite.at' includes.
1 
1    If you don't use Automake, you should include the above example in
1 `tests/Makefile.in', along with additional lines inspired from the
1 following:
1 
1      subdir = tests
1      PACKAGE_NAME = @PACKAGE_NAME@
1      PACKAGE_TARNAME = @PACKAGE_TARNAME@
1      PACKAGE_VERSION = @PACKAGE_VERSION@
1      PACKAGE_STRING = @PACKAGE_STRING@
1      PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@
1      PACKAGE_URL = @PACKAGE_URL@
1 
1      atconfig: $(top_builddir)/config.status
1              cd $(top_builddir) && \
1                 $(SHELL) ./config.status $(subdir)/$@
1 
1      atlocal: $(srcdir)/atlocal.in $(top_builddir)/config.status
1              cd $(top_builddir) && \
1                 $(SHELL) ./config.status $(subdir)/$@
1 
1 and manage to have `$(EXTRA_DIST)' distributed.  You will also want to
1 distribute the file `build-aux/missing' from the Automake project; a
1 copy of this file resides in the Autoconf source tree.
1 
1    With all this in place, and if you have not initialized
1 `TESTSUITEFLAGS' within your makefile, you can fine-tune test suite
1 execution with this variable, for example:
1 
1      make check TESTSUITEFLAGS='-v -d -x 75 -k AC_PROG_CC CFLAGS=-g'
1