R: Definition and execution of RUnit test suites.

runTestSuite {RUnit} | R Documentation

## Definition and execution of RUnit test suites.

### Description

`runTestSuite` is the central function of the RUnit package. Given one or more test suites it identifies and sources specified test code files one after another and executes all specified test functions defined therein. This is done sequentially for suites, test code files and test functions. During the execution information about the test function calls including the possible occurrence of failures or errors is recorded and returned at the end of the test run. The return object can then be used to create a test protocol of various formats.

`runTestFile` is just a convenience function for executing the tests in a single test file.

`defineTestSuite` is a helper function to define a test suite. See below for a precise definition of a test suite.

`isValidTestSuite` checks if an object defines a valid test suite.

### Usage

    defineTestSuite(name, dirs, testFileRegexp = "^runit.+\\.[rR]$",
                    testFuncRegexp = "^test.+",
                    rngKind = "Marsaglia-Multicarry",
                    rngNormalKind = "Kinderman-Ramage")
    isValidTestSuite(testSuite)
    runTestSuite(testSuites, useOwnErrorHandler = TRUE,
                 verbose = getOption("RUnit")$verbose,
                 gcBeforeTest = FALSE)
    runTestFile(absFileName, useOwnErrorHandler = TRUE,
                testFuncRegexp = "^test.+",
                rngKind = "Marsaglia-Multicarry",
                rngNormalKind = "Kinderman-Ramage",
                verbose = getOption("RUnit")$verbose,
                gcBeforeTest = FALSE)

### Arguments

`name` | The name of the test suite.
`dirs` | Vector of absolute directory names where to look for test files.
`testFileRegexp` | Regular expression for matching test files.
`testFuncRegexp` | Regular expression for matching test functions.
`rngKind` | name of an available RNG (see `RNGkind` for possible options).
`rngNormalKind` | name of a valid rnorm RNG version (see `RNGkind` for possible options).
`testSuite` | A single object of class test suite.
`testSuites` | A single object of class test suite or a list of test suite objects.
`useOwnErrorHandler` | If `TRUE` the RUnit framework installs its own error handler during test case execution (but reinstalls the original handler before it returns). If `FALSE` the error handler is not touched by RUnit but then the test protocol does not contain any call stacks in the case of errors.
`verbose` | level of verbosity of output log messages, 0: omits begin/end comments for each test function. Queried from global options set for RUnit at package load.
`absFileName` | Absolute file name of a test function.
`gcBeforeTest` | Run garbage collector before executing a test for more precise test timing. Enabling this option makes the tests run longer, especially when testing many small tests. By default GC is disabled (since 0.4.32).

### Details

The basic idea of the RUnit test framework is to declare a certain set of functions to be test functions and report the results of their execution. The test functions must not take any parameter nor return anything such that their execution can be automatised.

The specification which functions are taken as test functions is contained in an object of class `RUnitTestSuite` which is a list with the following elements.

name

  A simple character string. The name of a test suite is mainly used to create a well structure test protocol.

dirs

  A character vector containing the absolute names of all directories where to look for test files.

testFileRegexp

  A regular expression specifying the test files. All files in the test directories whose names match this regular expression are taken as test files. Order of file names will be alphabetical but depending on the used locale.

testFuncRegexp

  A regular expression specifying the test functions. All functions defined in the test files whose names match this regular expression are used as test functions. Order of test functions will be alphabetical.

After the RUnit framework has sequentially executed all test suites it returns all data collected during the test run as an object of class `RUnitTestData`. This is a (deeply nested) list with one list element for each executed test suite. Each of these executed test suite lists contains the following elements:

nTestFunc

  The number of test functions executed in the test suite.

nErr

  The number of errors that occurred during the execution.

nFail

  The number of failures that occurred during the execution.

dirs

  The test directories of the test suite.

testFileRegexp

  The regular expression for identifying the test files of the test suite.

testFuncRegexp

  The regular expression for identifying the test functions of the test suite.

sourceFileResults

  A list containing the results for each separate test file of the test suite.

The `sourceFileResults` list just mentioned contains one element for each specified test function in the source file. This element is a list with the following entries:

kind

  Character string with one of `success`, `error` or `failure` describing the outcome of the test function.

msg

  the error message in case of an error or failure and `NULL` for a successfully executed test function.

time

  The duration (measured in seconds) of the successful execution of a test function and `NULL` in the case of an error or failure. When running with `gcBeforeTest` option set to `FALSE` (default since 0.4.32), the timing of the tests might be misleading when garbage collector has to reclaim memory allocated by a previous test.

traceBack

  The full trace back as a character vector in the case of an error and `NULL` otherwise.

To further control test case execution it is possible to define two parameterless function `.setUp` and `.tearDown` in each test file. `.setUp()` is executed directly before and `.tearDown()` directly after each test function execution.

Quite often, it is useful to base test cases on random numbers. To make this procedure reproducible, the function `runTestSuite` sets the random number generator to the default setting `RNGkind(kind="Marsaglia-Multicarry", normal.kind="Kinderman-Ramage")` before sourcing each test file (note that this default has been chosen due to historical reasons and differs from the current R default). This default can be overwritten by configuring the random number generator at the beginning of a test file. This setting, however, is valid only inside its own source file and gets overwritten when the next test file is sourced.

### Value

`runTestSuite` and `runTestFile` both return an object of class RUnitTestData.

`defineTestSuite` returns an object of class `RUnitTestSuite`.

### Author(s)

Thomas KÃ¶nig, Klaus JÃ¼nemann & Matthias Burger

### See Also

`checkTrue` and friends for writing test cases. `printTextProtocol` and `printHTMLProtocol` for printing the test protocol. See RUnit-options for global options controlling log out.

### Examples

    ## run some test suite
    myTestSuite <- defineTestSuite("RUnit Example",
                                   system.file("examples", package = "RUnit"),
                                   testFileRegexp = "correctTestCase.r")
    testResult <- runTestSuite(myTestSuite)

    ##  same but without the logger being involved
    ## source(file.path(system.file("examples", package = "RUnit"),
    ##                  "correctTestCase.r"))
    ## test.correctTestCase()

    ## prints detailed text protocol
    ## to standard out:
    printTextProtocol(testResult, showDetails = TRUE)

    ##  use current default RNGs
    myTestSuite1 <- defineTestSuite("RUnit Example",
                                   system.file("examples", package = "RUnit"),
                                   testFileRegexp = "correctTestCase.r",
                                   rngKind = "Mersenne-Twister",
                                   rngNormalKind = "Inversion")

    testResult1 <- runTestSuite(myTestSuite)

    ##  for single test files, e.g. outside a package context
    testResult2 <- runTestFile(file.path(system.file("examples",
                                                     package = "RUnit"),
                                         "correctTestCase.r"))
    printTextProtocol(testResult2, showDetails = TRUE)

[Package RUnit version 0.4.32 Index]