[Home]Build Testing Framework

BOOST WIKI | RecentChanges | Preferences | Page List | Links List

Vladimir Prus has contributed a Python-based testing framework for the build system. I would like to check it in and begin using it ASAP.

The Build.Build testing framework provides for setting up directory structure, invoking the build system on it, and examining results.

Requirements. To use the testing framework, you'd need Python 2.1. Also, it should be possible to invoke Boost.Build by typing "jam" on the command line. This means that BOOST_BUILD_PATH and BOOST_BUILD_INSTALLATION should be set appropriately.

Basic operations. You start by invoking build_system_test.py script like this:

 ./build_system_test.py --toolset=gcc
It scans the current directory for zip files and expects each one to contains file "test.py" on the top level. That file describes tests (details below). Once tests contained in zip are known, for each test zip is extracted to a temporary directory, next test is run, and the directory is removed.

File "test.py" defines tests by defining classes derived from boost_build_system.tester and containing methods which names start with "test". For example:

 class Sanity_test(boost_build_system.py):
 def test(self):
It should be noted that test.py is not imported, but is "exec"d, so you need not write any import statements to access boost_build_system.tester (and import won't work now anyway :-) )

Here is the complete list of methods defined in boost_build_system.tester that are usefull in writing tests. Note that filenames always use "/" as separator, regardless of the current platform. They are lso relative to temporary directory into which the zip file was unpacked.
  1. set_content(self, name, new_content)
  2. touch(self, name)
  3. rebuild(self, subdir="", extra_options="", commands_only=0) -- runs jam in the specified subdir and passing it 'extra_options'. Sets self.command to all the commands that jam wanted to execute. Also sets internal attributes so that expect* functions work. If commands_only is 1, only list of commands is collected, but nothing is actually run
  4. run(self, exe): runs specified exe and sets self.status and self.output
The following methods examine difference made to the directory during last 'rebuild' invocation:
  1. expect_addition(self, names, dir="") 'names' can be a string or a list of string. 'dir' if given, specifies common prefix that is appended to all the names. If any of the files was not added, it is considered a test failure.
  2. ignore_addition(self, wildcard) -- names matching wildcard are ignored.
  3. expect_removal(self, names, dir="") -- similar to expect_addition
  4. expect_touch(self, names, dir="") -- similar to expect_addition
  5. expect_nothing_more(self) -- tests that all the changes are expected -- i.e. that for all changes either expect* method ignore_addition with matching wildcard was called.

Different toolset support. At the moment, support for different toolsets in in inital state.
  1. Toolset specification. You should either set TOOLS environment variable to a toolset name, or explicitly give it using "--toolset=yfc" option.
  2. Using toolset information in tests.
    1. If test class defines class attribute "toolsets", which should be list of string, then the test will be run only for toolsets present in that list.
    2. Test can use global variable "toolset" in any way it likes.

Knows problems:
  1. import statement in tests does not work

BOOST WIKI | RecentChanges | Preferences | Page List | Links List
Edit text of this page | View other revisions
Last edited November 6, 2001 8:24 am (diff)
Disclaimer: This site not officially maintained by Boost Developers