%% LyX 2.0.0 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[twoside,english]{article}
\usepackage{mathpazo}
\renewcommand{\sfdefault}{lmss}
\renewcommand{\ttdefault}{lmtt}
\usepackage[T1]{fontenc}
\usepackage[utf8x]{inputenc}
\usepackage{listings}
\usepackage[a4paper]{geometry}
\geometry{verbose,tmargin=3cm,bmargin=4.5cm,lmargin=2.5cm,rmargin=3cm}
\usepackage{fancyhdr}
\pagestyle{fancy}
\usepackage{color}
\definecolor{shadecolor}{rgb}{1, 0, 0}
\usepackage{babel}
\usepackage{framed}
\usepackage{url}
\usepackage{graphicx}
\usepackage[unicode=true,
 bookmarks=true,bookmarksnumbered=false,bookmarksopen=false,
 breaklinks=true,pdfborder={0 0 1},backref=false,colorlinks=true]
 {hyperref}
\hypersetup{pdftitle={svUnit - A framework for unit testing in R},
 pdfauthor={Philippe Grosjean},
 pdfsubject={Test units using the svUnit R package},
 pdfkeywords={test unit, extreme programming, code refactoring, quality insurance}}
\usepackage{breakurl}

\makeatletter
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Textclass specific LaTeX commands.
\usepackage[noae]{Sweave}
\usepackage[noae]{Sweave}
\newenvironment{lyxlist}[1]
{\begin{list}{}
{\settowidth{\labelwidth}{#1}
 \setlength{\leftmargin}{\labelwidth}
 \addtolength{\leftmargin}{\labelsep}
 \renewcommand{\makelabel}[1]{##1\hfil}}}
{\end{list}}
\newenvironment{lyxcode}
{\par\begin{list}{}{
\setlength{\rightmargin}{\leftmargin}
\setlength{\listparindent}{0pt}% needed for AMS classes
\raggedright
\setlength{\itemsep}{0pt}
\setlength{\parsep}{0pt}
\normalfont\ttfamily}%
 \item[]}
{\end{list}}
\providecommand{\R}{\textsf{R}}
\providecommand{\file}[1]{`\textsf{#1}'}
\providecommand{\command}{\bgroup\@noligs\@cmdx}
\def\@cmdx#1{{\normalfont\texttt{#1}}\egroup}
\providecommand{\pkg}[1]{{\fontseries{b}\selectfont #1}}
{\catcode`\ =\active\global\def\code{\bgroup\@noligs\catcode`\ =\active\let \codespace\@codex}}
\def\codespace{{ }}
\def\@codex#1{\normalfont\texttt{#1}\egroup}
\providecommand{\class}[1]{\texttt{\textquotedbl{}#1\textquotedbl{}}}
\providecommand{\SciViews}[1]{\textsf{\textsl{S}\lower.08em\hbox{ci}\textsl{\kern-.04emV}\kern-.02em\lower.08em\hbox{iews}\@#1}}
\providecommand{\var}[1]{{\normalfont\textsl{#1}}}
\providecommand{\widget}[1]{`\textsf{\textsc{#1}}'}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% User specified LaTeX commands.
% \VignetteIndexEntry{svUnit - A framework for unit testing in R}
%\VignettePackage{svUnit}

\makeatother

\begin{document}

\title{svUnit - A framework for unit testing in R }


\date{Version 0.7-3, 2010-09-05}


\author{Philippe Grosjean (phgrosjean@sciviews.org)}

\maketitle

\section{Introduction}

Unit testing (see \url{http://en.wikipedia.org/wiki/Unit_test}) is
an approach successfully used to develop software, and to ease code
refactoring for keeping bugs to the minimum. It is also the insurance
that the software is doing the right calculation (quality insurance).
Basically, a test just checks if the code is running and is producing
the correct answer/behavior in a given situation. As such, unit tests
are build in \R{} package production because all examples in documentation
files, and perhaps, test code in \file{/tests} subdirectory are run
during the checking of a package (\command{R CMD check <pkg>}). However,
the \R{} approach lacks a certain number of features to allow optimal
use of unit tests as in extreme programming (test first – code second): 
\begin{itemize}
\item Tests are related to package compilation and cannot easily be run
independently (for instance, for functions developed separately).
\item Once a test fails, the checking process is interrupted. Thus one has
to correct the bug and launch package checking again... and perhaps
get caught by the next bug. It is a long and painful process.
\item There is no way to choose one or several tests selectively: all are
run or not (depending on command line options) during package checking.
\item It is very hard, or near to impossible to program in \R{} in a test
driven development (\emph{write tests first}) with the standard tools
(\url{http://en.wikipedia.org/wiki/Test- driven_development}). 
\item Consequently, the {}`\emph{test-code-simplify}' cycle is not easily
accessible yet to \R{} programmer, because of the lack of an interactive
and flexible testing mechanism providing immediate, or quasi immediate
feedback about changes made.
\item We would like also to emphasize that test suites are not only useful
to check code, they can also be used to check data, or the pertinence
of analyses.
\end{itemize}

\subsection{Unit testing in R without svUnit}

Besides the \textquotedbl{}regular\textquotedbl{} testing mechanism
of \R{} packages, one can find the \pkg{RUnit} package on CRAN (\url{http://cran.r-project.org}).
Another package used to provide an alternate implementation of test
unit: \pkg{butler}, but it is not maintained any more and has given
up in favor of \pkg{RUnit}. \pkg{RUnit} implements the following
features: 
\begin{itemize}
\item \textbf{Assertions,} \code{checkEquals()}, \code{checkEqualsNumeric()},
\code{checkIdentical()} and \code{checkTrue()} and negative tests
(tests that check error conditions, \code{checkException()}).
\item Assertions are grouped into \R{} functions to form one \textbf{test
function} that runs a series of related individual tests. It is easy
to temporarily inactivate one or more tests by commenting lines in
the function. To avoid forgetting tests that are commented out later
on, there is special function, named \code{DEACTIVATED()} that tags
the test with a reminder for your deactivated items (i.e., the reminder
is written in the test log).
\item A series of test functions (whose name typically start with \code{test....})
are collected together in a source-able \R{} code file (name starting
with \code{runit....}) on disk. This file is called a \textbf{test
unit}.
\item A \textbf{test suite} (object \class{RUnitTestSuite}) is a special
object defining a battery of tests It points to one or several directories
containing test units. A test suite is defined by \code{defineTestSuite()}.
\item One or more test suites can be run by calling \code{runTestSuite()}.
There is a shortcut to define and run a test suite constituted by
only one test unit by using the function \code{runTestFile()}. Once
the test is run, a \class{RUnitTestData} object is created that contains
all the information collected from the various tests run.
\item One can print a synthetic report (how many test units, test functions,
number of errors, fails and deactivated item), or get a more extensive
\code{summary()} of the tests with indication about which ones failed
or produced errors. The function \code{printTextProtocol()} does
the same, while \code{printHTMLProtocol()} produces a report in HTML
format.
\item \pkg{RUnit} contains also functions to determine which code is run
in the original function when tested, in order to detect the parts
of the code not covered by the test suite (code coverage function
\code{inspect()} and function \code{tracker()}).
\end{itemize}
As complete and nice as \pkg{RUnit} is, there is no tools to integrate
the test suite in a given development environment (IDE) or graphical
user interface (GUI), as far as we know. In particular, there is no
real-time reporting mechanism used to easy the \emph{test-code-simplify}
cycle. The way tests are implemented and run is left to the user,
but the implementation suggests that the authors of \pkg{RUnit} mainly
target batch execution of the tests (for instance, nightly check of
code in a server), rather that real-time interaction with the tests.

There is also no integration with the \textquotedbl{}regular\textquotedbl{}
\command{R CMD check} mechanism of \R{} in \pkg{RUnit}. There is
an embryo of organization of these tests units to make them compatible
with \command{R CMD check} on the R Wiki (\url{http://wiki.r-project.org/rwiki/doku.php?id=developers:runit}).
This approach works well only on Linux/Unix systems, but needs to
be adapted for Windows. 


\subsection{Unit testing framework for R with svUnit}

Our initial goal was to implement a GUI layer on top of \pkg{RUnit},
and to integrate test units as smoothly as possible in a code editor,
as well as, making tests easily accessible and fully compatible with
\command{R CMD check} on all platforms supported by \R{}. Ultimately,
the test suite should be easy to create, to use interactively, and
should be able to test functions in a complex set of \R{} packages.

However, we encountered several difficulties while trying to enhance
\pkg{RUnit} mechanism. When we started to work on this project, \pkg{RUnit}
(version 0.4-17) did not allow to subclass its objects. Moreover,
its \class{RUnitTestData} object is optimized for quick testing,
but not at all for easy reviewing of its content: it is a list of
lists of lists,... requiring embedded for loop and \code{lapply()}
/ \code{sapply()} procedures to extract some content. Finally, the
concept of test units as source-able files on disk is a nice idea,
but it is too rigid for quick writing test cases for objects not associated
(yet) with an \R{} packages.

We did a first implementation of the \pkg{RUnit} GUI based on these
objects, before realizing that it is really not designed for such
an use. So, we decide to write a completely different unit testing
framework in \R{} : \pkg{svUnit}, but we make it test code compatible
with \pkg{RUnit} (i.e., the engine and objects used are totally different,
but the test code run in \pkg{RUnit} or \pkg{svUnit} is interchangeable).

Finally, \pkg{svUnit} is also designed to be integrated in the \SciViews{}
GUI (\url{http://www.sciviews.org/SciViews-K}), on top of Komodo
Edit or IDE (\url{http://www.activestate.com/komodo_edit}), and to
approach extreme programming practices with automatic code testing
while you write it. A rather simple interface is provided to link
and pilot \pkg{svUnit} from any GUI/IDE, and the Komodo Edit/IDE
implementation could be used as an example to program similar integration
panels for other \R{} GUIs. \pkg{svUnit} also formats its report
with \emph{creole wiki} syntax. It is directly readable, but it can
also be displayed in a much nicer way using any wiki engine compatible
with the creole wiki language. It is thus rather easy to write test
reports in wiki servers, possibly through nightly automatic process
for your code, if you like.

This vignette is a guided tour of \pkg{svUnit}, showing its features
and the various ways you can use it to test your \R{} code.


\section{Installation}

The \pkg{svUnit} package is available on CRAN (\url{http://cran.r-project.org}),
and its latest development version is also available on R-Forge (\url{http://sciviews.r-forge.r-project.org/}).
You can install it with%
\footnote{Install the development version with \code{install.packages(\textquotedbl{}svUnit\textquotedbl{}, repos = \textquotedbl{}http://r-forge.r-project.org\textquotedbl{})}.%
}:

\SweaveOpts{keep.source = TRUE}

<<echo = FALSE, results = hide>>=
options(useFancyQuotes = TRUE)
@

<<eval = FALSE>>=
install.packages("svUnit")
@

This package has no dependence other than \R{} $\geq$ 1.9.0. However,
if you would like to use its interactive mode in a GUI editor, you
must also install Komodo Edit or Komodo IDE and \SciViews{}-K. The
procedure is explained here: \url{http://www.sciviews.org/SciViews-K}.

Once the \pkg{svUnit} package is installed, you can check it is working
correctly on your machine with the following example code:

<<>>=
library(svUnit)
Square <- function (x) return(x^2)
test(Square) <- function () {
    checkEqualsNumeric(9, Square(3))
    checkEqualsNumeric(10, Square(3))   # This intentionally fails
    checkEqualsNumeric(9, SSSquare(3))  # This raises error
    checkEqualsNumeric(c(1, 4, 9), Square(1:3))
    checkException(Square("xx"))
}
clearLog()
(runTest(Square))
@

\begin{framed}%
Although test unit code is compatible with both \pkg{svUnit} and
\pkg{RUnit}, do not load both packages in \R{} memory at the same
time, or you will badly mix incompatible code!\end{framed}


\section{Overview of svUnit}

You ensure that code you write in your \R{} functions does the expected
work by defining a battery of tests that will compare the output of
your code with reference values. In \pkg{svUnit}, the simplest way
to define such a battery of tests is by attaching it to functions
loaded in \R{} memory%
\footnote{In fact, you can attach svUnit tests to any kind of \R{} object,
not only function. This could be useful to test S3/S4 objects, or
even, datasets. %
}. Of course, you can also define batteries of tests that are independent
of any \R{} object, or that check several of them together (so called,
\emph{integration tests}). Here is a couple of examples:

<<>>=
library(svUnit)
## Create two R functions that include their own test cases
Square <- function (x) return(x^2)
test(Square) <- function () {
    checkEqualsNumeric(9, Square(3))
    checkEqualsNumeric(c(4, 9), Square(2:3))
    checkException(Square("xx"))
}

Cube <- function (x) return(x^3)
test(Cube) <- function () {
    checkEqualsNumeric(27, Cube(3))
    checkEqualsNumeric(c(8, 28), Cube(2:3))
    checkException(Cube("xx"))
}

## Add a separate test case
test_Integrate <- svTest(function () {
    checkTrue(1 < 2, "check1")
    v <- c(1, 2, 3)  # The reference
    w <- 1:3         # The value to compare to the reference
    checkEquals(v, w)
})
@

When you run a test in \pkg{svUnit}, it logs its results in a centralized
logger. The idea is to get a central repository for tests that you
can manipulate as you like (print, summarize, convert, search, display
in a GUI, etc.). If you want to start new tests, you should first
clean this logger by \code{clearLog()}. At any time, the logger is
accessible by \code{Log()}, and a summary of its content is displayed
using \code{summary(Log()}). So, to run test for your \code{Square()}
function as well as your \var{test\_Integrate} integration test,
you simply do the following:

<<>>=
clearLog()
runTest(Square)
runTest(test_Integrate)
Log()
@

In this report, you see that all your tests succeed. Note that \pkg{svUnit}
is making the distinction between a test that \textbf{fails} (the
code is run correctly, but the result is different from what was expected)
and code that raises \textbf{error} (it was not possible to run the
test because its code is incorrect, or for some other reasons). Note
also that the function \code{checkException()} is designed to explicitly
test code that should \code{stop()} in \R{}%
\footnote{checkException() can also track warning() messages with this little
trick: first convert all warnings into errors with owarn <- options(warn = 2)\$warn,
run the code that should generate a warning inside checkException(),
and then restore default warning behavior with options(warn = owarn).%
}. So, if that test does not raises an exception, it is considered
to have failed. This is useful to check that your functions correctly
trap wrong arguments, for instance, like in \code{checkException(Square(\textquotedbl{}xx\textquotedbl{}))}
here above (a character string is provided where a numerical value
is expected).

Now, let's look what happens if we test the \code{Cube()} function
without clearing the logger:

<<>>=
runTest(Cube)
Log()
@

We note this:
\begin{enumerate}
\item When a test succeeds, nothing is printed by default (result is returned
invisibly). But when a test fails, or raises errors, the guilty test(s)
results are printed. We expected \code{c(8, 28)} (made intentionally
wrong for the sake of the demonstration) from \code{Cube(2:3)} in
\code{checkEqualsNumeric(c(8, 28), Cube(2:3))}, and (of course),
we got \code{c(8, 27)}. This test shows how \pkg{svUnit} presents
test failures.
\item The results of the tests on \code{Cube()} are added to the previous
report. So, it is possible to build rather easily reports that summarize
tests on several objects, by adding test results in the logger sequentially.
\pkg{svUnit} does this naturally and transparently. Starting a new
report is equally simple: just use \code{clearLog()}...
\end{enumerate}

\subsection{Assertions in svUnit}

The most basic item in a test suite is an \textbf{assertion} represented
by a \code{checkXXX()} function in \pkg{svUnit}/\pkg{RUnit}. Five
such functions are currently defined:
\begin{lyxlist}{00.00.0000}
\item [{\code{checkEquals(current, target)}}] determines if data in \code{target}
is the same as data in \code{current}.
\item [{\code{checkEqualsNumeric(current, target)}}] does the same but
allows for a better comparison for numbers (variation allowed within
a tolerance window).
\item [{\code{checkIdentical(current, target)}}] checks whether two \R{}
objects are strictly identical.
\item [{\code{checkTrue(expr)}}] only succeed if \code{expr} is \code{TRUE}.
Note a difference in \pkg{svUnit} and \pkg{RUnit} (at least, in
its version 0.4-17): the \pkg{RUnit} function is not vectorized and
\code{expr} must return a single atomic logical value. The corresponding
\pkg{svUnit} function also accepts a vector of logical values. In
this case, all elements of the vector must be \code{TRUE} for the
test to succeed. When you make sure that \code{expr} always returns
a single logical value (for instance by using \code{all(expr)}),
both functions should be compatible.
\item [{\code{checkException(expr)}}] verifies that a given code raises
an exception (in \R{}, it means that a line of code with \code{stop()}
is executed).
\item [{\code{DEACTIVATED()}}] makes sure that all tests following this
instruction (in a test function, see next paragraph) are deactivated,
and inserts a reminder in the logger about the fact that some tests
are deactivated in this suite.
\end{lyxlist}
For all these functions, you have an additional optional argument
\code{msg =} where you can provide a (short) message to print in
front of each text in the report. These functions return invisibly:
\code{TRUE} if the test succeeds, or \code{FALSE} if it fails (code
is executed correctly, but does not pass the test), and \code{NA}
if there was an error (the \R{} code of the test was not executed
correctly). Moreover, these functions record the results, the context
of the test and the timing in a logger (object \class{svSuiteData}
inheriting from \class{environment}) called \var{.Log} and located
in the user’s workspace. So, executing a series of assertions and
getting a report is simply done as (in its simplest form, you can
use the various \code{checkXXX()} functions directly at the command
line):

<<>>=
clearLog()
checkEqualsNumeric(1, log(exp(1)))
checkException(log("a"))
checkTrue(1 == 2)
Log()
@

As you can see, the \code{checkXXX()} functions work hand in hand
with the test logger (the \code{checkXXX()} functions also return
the result of the test invisibly, so, you can also assign it to a
variable if you like). These function are mainly used for their side-effect
of adding an entry to the logger.

The last command \code{Log()} prints the content of the logger. You
see how a report is printed, with a first part being a short summary
by categories (assertions run at the command line are placed automatically
in the \code{eval} category: there is no better context known for
them. Usually, those assertions should be placed in test functions,
or in test units, as we will see later in this manual, and the category
will reflect this organization). A detailed report on the tests that
failed or raised an error is also printed at the end of the report. 

Of course, the same report is much easier to manipulate from within
the graphical tree in the Komodo’s \textbf{R Unit} tab, but the simple
text report in \R{} has the advantage of being independent from any
GUI, and from Komodo. It can also be generated in batch mode. Last,
but not least, it uses a general Wiki formatting called creole wiki
(\url{http://www.wikicreole.org/wiki/Creole1.0}). Figure \ref{fig:wikireport}
illustrates the way the same report looks like in DokuWiki with the
creole plugin (\url{http://www.wikicreole.org/wiki/DokuWiki}) installed.
Note the convenient table of content that lists here a clickable list
of all tests run. From this point, it is relatively easy to define
nightly cron task jobs on a server to run a script that executes these
tests and update a wiki page (look at your particular wiki engine
documentation to determine how you can access wiki pages on the command
line).

\begin{figure}

\centering{}

\includegraphics{svUnit_wikiReport}

\caption{\label{fig:wikireport}a \protect\pkg{svUnit} test report as it appears
when inserted in a wiki page (DokuWiki engine with the creole plugin
installed). Note the summary of results at the top left of the page,
and the clickable table of contents with detailed entries to easily
navigate to the test log you are consulting). Timing of the test is
also clearly indicated, since it is a complementary but important
information (if a test succeeds, but calculation is way too long,
it is good to know it)!}
\end{figure}





\subsection{Manipulating the logger data}

The \pkg{svUnit} package provides a series of functions to manipulate
the logger from the command line, in particular, \code{stats()},
\code{summary()}, \code{metadata()} and \code{ls()}:

<<>>=
## Clear test exclusion list for running all test suites
options(svUnit.excludeList = NULL)
## Clear the logger
clearLog()
## Run all currently defined tests
runTest(svSuiteList(), name = "AllTests")
## Get some statistics
stats(Log())[, 1:3]
## A slightly different presentation than with print
summary(Log())
## Metadata collected on the machine where tests are run
metadata(Log())
## List content of the log
ls(Log())
@

As you can see, \code{ls()} lists all components recorded in the
test suite. Each component is a \class{svTestData} object inheriting
from \class{data.frame}, and it can be easily accessed through the
\code{\$} operator. There are, of course similar methods defined
for those \class{svTestData} objects, like \code{print()}, \code{summary()}
and \code{stats()}:

<<>>=
myTest <- Log()$testCube
class(myTest)
myTest
summary(myTest)
stats(myTest)
@

As the logger inherits from \class{environment}, you can manage individual
test data the same way as objects in any other environment. For instance,
if you want to delete a particular test data without touching to the
rest, you can use:

<<>>=
ls(Log())
rm(test_R, envir = Log())
ls(Log())
@

As we will see in the following section, \pkg{svUnit} proposes several
means to organize individual assertions in modules: \textbf{test functions},
\textbf{test units} and \textbf{test suites}. This organization is
inspired from \pkg{RUnit}, but with additional ways of using tests
in interactive sessions (for instance, the ability to attach a test
to the objects to be tested).


\subsection{Test function}

The first organization level for grouping assertions together is the
\textbf{test function}. A test function is a function without arguments
whose name must start with \code{test}. It typically contains a series
of assertions applied to one object, method, or function to be checked
(this is not obligatory, assertions are not restricted to one object,
but good practices strongly suggest such a restriction). Here is an
example:

<<>>=
test_function <- function () {
    checkTrue(1 < 2, "check1")
    v <- c(1, 2, 3)  # The reference
    w <- 1:3         # The object to compare to the reference
    checkEqualsNumeric(v, w)
}
## Turn this function into a test
test_function <- as.svTest(test_function)
is.svTest(test_function)
@

A test function should be made a special object called \class{svTest},
so that \pkg{svUnit} can recognize it. This \class{svTest} object,
is allowed to live on its own (for instance, in the user's workspace,
or anywhere you like). It can be defined in a \R{} script, be saved
in a \file{.RData} file, etc... Note that this is very different
from \pkg{RUnit} where test must always be located in a unit test
file on disk). In \pkg{svUnit} (not \pkg{RUnit}), you run such a
test simply by using \code{runTest()}, which returns the results
invisibly and add it to the logger:

<<>>=
clearLog()
runTest(test_function)
Log()
@

Now, a test function is most likely designed to test an \R{} object.
The \pkg{svUnit} package also provides facilities to attach the test
function to the object to be tested. Hence, the test cases and the
tested object conveniently form a single entity that one can manipulate,
copy, save, reload, etc. with all the usual tools in \R{}. This association
is simply made using \code{test(myobj) <-}:

<<>>=
## A very simple function
Square <- function (x) return(x^2)

## A test case to associate with the Square() function
test(Square) <- function () {
    checkEqualsNumeric(9, Square(3))
    checkEqualsNumeric(c(1, 4, 9), Square(1:3))
    checkException(Square("xx"))
}
is.test(Square)  # Does this object contain tests?
@

One can retrieve the test associated with the object by using:

<<>>=
test(Square)
@

And of course, running the test associated with an object is as easy
as:

<<>>=
runTest(Square)
Log()  # Remember we didn't clear the log!
@

Now that you master test functions, you will discover how you can
group them in logical \textbf{units}, and associate them to \R{}
packages.


\subsection{Test units}

A \textbf{unit} is a coherent piece of software that can be tested
separately from the rest. Typically, a \R{} package is a structured
way to compile and distribute such code units in \R{}. Hence, we
need a mean to organize tests related to this {}``unit'' conveniently.

Since a package can contain several functions, data frames, or other
objects, our unit should collect together individual test functions
related to each of these objects that compose our package. Also, the
test unit should accommodate the well-define organization of a package,
and should integrate in the already existing testing features of \R{},
in particular, \command{R CMD check}. In both \pkg{RUnit}, and \pkg{svUnit},
one can define such test units, and they are made code compatible
between the two implementations.

A test unit is a source-able text file that contains one or more test
functions, plus possibly \code{.setUp()} and \code{.tearDown()}
functions (see the online help for further information on these special
functions). In \pkg{RUnit}, you must write such test unit files from
scratch. With \pkg{svUnit}, you can {}``promote'' one or several
test functions (associated to other objects, or {}``living'' alone
as separate \class{svTest} objects) by using \code{makeUnit()}.
Here is how you promote the test associated with our \code{Square()}
function to a simple test unit containing only one test function:

<<eval = FALSE>>=
# Create a test unit on disk and view its content
unit <- makeUnit(Square)
file.show(unit, delete.file = TRUE)
@

You got the following file whose name must start with \file{runit},
with an \file{.R} extension (\file{runit{*}.R}), and located by
default in the temporary directory of \R{}. Specify another directory
with the \code{dir =} argument of\code{ makeUnit()} for a more permanent
record of this test unit file. Note also that \code{.setUp()} and
\code{.tearDown()} functions are constructed automatically for you.
They specify the context of these tests. This context is used, for
instance, by the GUI in Komodo Edit/IDE to locate the test function
and the code being tested.
\begin{lyxcode}
\#\#~Test~unit~'Square'



.setUp~<-

function~()~\{

~~~~\#\#~Specific~actions~for~svUnit:~prepare~context

~~~~if~(\textquotedbl{}package:svUnit\textquotedbl{}~\%in\%~search())~\{

~~~~~~~~.Log~<-~Log()~\#\#~Make~sure~.Log~is~created

~~~~~~~~.Log\$..Unit~<-~\textquotedbl{}/tmp/RtmpBoZnId/runitSquare.R\textquotedbl{}

~~~~~~~~.Log\$..File~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..Obj~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..Tag~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..Msg~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~rm(..Test,~envir~=~.Log)

~~~~\}

\}



.tearDown~<-

function~()~\{

~~~~\#\#~Specific~actions~for~svUnit:~clean~up~context

~~~~if~(\textquotedbl{}package:svUnit\textquotedbl{}~\%in\%~search())~\{

~~~~~~~~.Log\$..Unit~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..File~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..Obj~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..Tag~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~.Log\$..Msg~<-~\textquotedbl{}\textquotedbl{}

~~~~~~~~rm(..Test,~envir~=~.Log)

~~~~\}

\}



\textquotedbl{}testSquare\textquotedbl{}~<-

function()~\{

~~~~checkEqualsNumeric(9,~Square(3))

~~~~checkEqualsNumeric(c(1,~4,~9),~Square(1:3))

~~~~checkException(Square(\textquotedbl{}xx\textquotedbl{}))

\}
\end{lyxcode}
Compatibility of these test unit files between \pkg{RUnit} and \pkg{svUnit}
was a major concern in the design of \pkg{svUnit}. Consequently,
code specific to \pkg{svUnit} (for managing the context of the tests)
is embedded in a \code{if ({}``package:svUnit'' \%in\% search())}
construct. That way, if \pkg{svUnit} is not loaded in memory, this
code is not executed. \emph{Note that you should avoid loading in
memory both }\pkg{svUnit}\emph{ and }\pkg{RUnit}\emph{ at the same
time! If you do so, you will most likely crash your tests.}

You will see further that it is possible to write much more complex
test units with the same \code{makeUnit()} function. But for the
moment, let’s discuss a little bit how such test units should be organized
in \R{} package.

If you intend to associate test units to your \R{} package, you should
respect the following conventions:
\begin{itemize}
\item Name your test units \file{runit{*}.R}.
\item Place them in the \file{/inst/unitTests} subdirectory of the package
sources, or in one of its subdirectories. If you place them in a subdirectory
of \file{/inst/unitTests}, then you define secondary unit tests for
(optional) detailed testing of specific item in your package. Always
keep in mind that all \file{runit{*}.R} files in a directory will
be run one after the other. So, if you want to make subgroups you
would like to dissociate them, and locate them is separate subdirectories.
\item When the package will be compiled, all these test units will be located
in \file{/unitTests} in the compiled/installed version of your \R{}
package.
\end{itemize}
If you respect these conventions, \pkg{svUnit} knows where package
unit tests are located and will be able to find and run them quite
easily. See, for instance, the examples in the \pkg{svUnit} package.

So, with test units associated to packages, you have a very convenient
way to run these tests, including from the Komodo GUI. With just a
little bit more coding you can also include these test units in the
\command{R CMD check} process of your packages. You do this by means
of examples in a help page (we prefer to use \textbf{examples}, instead
of \file{/tests} in the \command{R CMD check} process, because examples
offer a more flexible way to run tests and you can also run them in
interactive sessions through the \code{example()} \R{} function,
which is not the case for code located in the \file{/tests} subdirectory
of your package). Here is what you do to associate some or all of
your unit tests to the \command{R CMD check} process (illustrated
with the case of the \pkg{svUnit} package itself):
\begin{itemize}
\item Define a \file{.Rd} help file in the \file{/man} subdirectory called
\file{unitTests.<mypackage>.Rd} where <mypackage> is the name of
your package (or whatever name you prefer).
\item Fill the \file{.Rd} file, making sure that you define an alias as
\file{unitTests.<mypackage>}. Also place a little bit of information
telling how users can run your test in an interactive session.
\item The important part of this file is, of course, the \code{\textbackslash{}examples\{\}}
section. You must first clear the log, then run each test, and then,
call the \code{errorLog()} function. That function looks if one or
more tests failed or raised an error. In this case, it stops execution
of the example and causes a dump of the test log in the \command{R CMD check}
process. That way, providing that you have the \pkg{svUnit} package
installed in the machine where you run \command{R CMD check}, your
test units will be included nicely in the checking process of your
packages, that is, they will run silently each time you check your
package if no error occurs, but will produce a detailed report in
case of problems.
\item Here is how your \file{.Rd} file should looks like (example of the
\pkg{svUnit} package):\end{itemize}
\begin{lyxcode}
\textbackslash{}name\{unitTests.svUnit\}

\textbackslash{}alias\{unitTests.svUnit\}



\textbackslash{}title\{~Unit~tests~for~the~package~svUnit~\}



\textbackslash{}description\{~Performs~unit~tests~defined~in~this

~~package~by~running~\textbackslash{}code\{example(unitTests.svUnit)\}.

~~Tests~are~in~\textbackslash{}code\{runit{*}.R\}~files~Located~in~the

~~'/unitTests'~subdirectory~or~one~of~its

~~subdirectories~('/inst/unitTests'~and~subdirectories

~~in~package~sources).

\}



\textbackslash{}author\{Philippe~Grosjean

~~(\textbackslash{}email\{phgrosjean@sciviews.org\})\}~



\textbackslash{}examples\{

if~(require(svUnit))~\{~

~~~~clearLog()

~~~~runTest(svSuite(\textquotedbl{}package:svUnit\textquotedbl{}),~\textquotedbl{}svUnit\textquotedbl{})

~~~~\#\#~Possibly~run~other~tests~here...

~~~~errorLog()

\}

\}



\textbackslash{}keyword\{utilities\}~
\end{lyxcode}
Note, however, that if the package \pkg{svUnit} is not available
on the computer where you run \command{R CMD check}, your tests are
silently ignored (\code{require()} issues a warning, but that does
not prevent the checking process to continue). This is an intended
feature in order to allow compilation of your package without requiring
\pkg{svUnit}. Hence, dependence to \pkg{svUnit} is less strict and
also allows you to check your tests using \pkg{RUnit} (but you have
to write a dedicated function for that). Still to keep such a less
strict dependence on \pkg{svUnit}, you should add \code{svUnit}
in the \code{Suggests:} field in the \file{DESCRIPTION} file of
your package, not in \code{Depends:} or \code{Imports:} fields (except
if you use \pkg{svUnit} for other purposes that testing your package
using the mechanism discussed here, of course.

Also, this approach, associated with examples, provides a very convenient
and easy way to test a package from the command line in an interactive
session by running:

<<>>=
example(unitTests.svUnit)
@

In the present case, the \code{errorLog()} instruction in the examples
returns nothing, because all tests succeed. If there is an error somewhere,
you will see it printed at the end of this example.


\subsection{Test suites: collections of test functions and units}

The highest level of organization of your tests is the \textbf{test
suite}. A test suite is an unordered collection of test functions
and test units. You can select test units associated with \R{} package
in a very convenient way: just specify \code{package:myPkg} and all
test units in the \file{/unitTests} subdirectory of the package \pkg{myPkg}
will be included (\pkg{svUnit} does all the required work to map
these to actual directories where the test unit files are located).
Also, if you specify \code{package:myPkg (subgroup)}, you will include
the test units defined in \file{/unitTests/subgroup} in the package
\pkg{myPkg}. Of course, you will be able to also add test units defined
in custom directories, outside of \R{} packages (for instance for
integration of \emph{harness tests} that check cross-packages, or
multi-packages features of your application).

Test functions associated to your test suite receive a special treatment.
Unlike \code{runTest()} applied to a single test function, or to
an object that has an associated test function, these tests are not
run from the version loaded in memory. Instead, they are first collected
together in a test unit file on disk (located in the \R{} temporary
directory, by default), and run from there. Hence, building a more
complex test unit file by collecting together several test functions
is just a question of constructing a test suite, and then, applying
the \code{makeUnit()} function to this \class{svSuite} object.

Before we apply all this, you should also know the existence of one
more function: \code{svSuiteList()}. This function lists all test
units and test functions available in your system at a given time.
So, you don’t need to manually create lists of components. You are
better to list them automatically. Of course, this function has a
lot of arguments for listing only test units in packages, only test
functions, specifying where (in which environment) the test functions
are located, adding custom directories where to look for test units,
etc, etc. See the online help of this function for the description
of all these arguments. One argument is particularly important: \code{excludeList =}.
This argument defines one or several regular expressions that are
used as filters to hide items from the list. This is required, since
you will certainly not want to run again and again, let’s say, the
example tests associated with the \pkg{svUnit} package (\pkg{svUnit}
must be loaded in memory to run the tests, so its tests examples will
always be listed by \code{svSuiteList()}, ... unless you define an
adequate filter expression that will exclude them from your list)!
As the default argument suggests it, the regular expression for list
exclusion could also be recorded in \code{options(svUnit.excludeList = ...)}.
Here is how it works:

<<>>=
## Reset default exclusion list
options(svUnit.excludeList = c("package:sv", "package:RUnit"))
## List all currently available tests 
svSuiteList()
@

Thus, every entry matching the regular expressions \code{package:sv}
and \code{package:RUnit} are excluded from the list. The entries
\code{package:svUnit} and \code{package:svUnit (VirtualClass} match
first pattern and are thus excluded. Now, let's clear the exclusion
list to see what happens:

<<>>=
## Clear exclusion list
options(svUnit.excludeList = NULL)
svSuiteList()
@

The test units associated with the package \pkg{svUnit} are now listed.
You have noticed that \code{svSuiteList()} can also find automatically
\class{svTest} objects, as well as tests attached to objects in the
user's workspace. You can create a suite by collecting all these items
together very easily:

<<>>=
(mySuite <- svSuiteList())
@

Now let’s make a test unit using tests collected in this suite:

<<eval = FALSE>>=
myUnit <- makeUnit(mySuite, name = "ExampleTests")
file.show(myUnit, delete.file = TRUE)
@

This produces a file named \file{runitExampleTests.R} located (by
default) in the \R{} temporary directory, and which collects together
all tests in the user's workspace (either as \class{svTest} objects,
or as tests attached to other objects), plus tests suites in packages
that are \textbf{not} in the exclusion list. Running all tests in
your suite is very simple. You still use \code{runTest()} as usual,
but this time, you apply it to your \class{svSuite} object:

<<>>=
clearLog()
runTest(mySuite)
summary(Log())
@

There are many other tools to manipulate \class{svSuite} objects
in the \pkg{svUnit} package, including functions to define the content
of the suite manually (see online help).


\section{Using svUnit with SciViews Komodo}

If you use the SciViews Komodo GUI (see \url{http://www.sciviews.org/SciViews-K}),
you can integrate \pkg{svUnit} tests in this IDE and display reports
in a convenient hierarchical tree presentation (Fig. \ref{fig:KomodoRUnit}).
If \R{} is started from within Komodo Edit or IDE (with the \SciViews{-K}
and \SciViews{-K Unit} plugins installed), then, loading the \pkg{svUnit}
package in \R{} automatically installs the \textbf{R Unit} side panel
in Komodo at right. Its use should be straightforward:

\begin{figure}

\centering{}

\includegraphics{svUnit_KomodoRUnit}

\caption{\label{fig:KomodoRUnit}Komodo Edit with \protect\SciViews{-K} and
\protect\SciViews{-K Unit} after running tests. At right: the \textbf{R
Unit} panel that display (at the top) the list of available tests,
and test units where you can select the ones to run, and at the bottom,
a tree with the results from last tests run. The stripe at the very
top is green if all tests succeed, and red (as here), if at least
one tests failed or raised an error.}


\end{figure}

\begin{itemize}
\item Select the tests you want to run in the top part,
\item Click the \widget{Run} button each time you want to refresh the test
tree,
\item Browse the tree for failures or errors (the color at the top of the
panel immediately indicates if there is a problem somewhere: green
-> everything is fine, red -> there is a problem).
\item If you have failures or errors, move the mouse on top of the corresponding
item in the tree, and you got more information displayed in a tooltip,
\item Click on an item to open the test unit at that place in a buffer in
Komodo.
\end{itemize}
The \widget{Auto} mode, when activated, sources \R{} files currently
edited in Komodo whenever you save them, and then, refreshes the test
report tree. This mode allows you to run automatically your tests
in the background while you edit your \R{} code!

\begin{framed}%
If you want to implement such a side panel in another GUI, make sure
to look at the \code{koUnit\_XXX()} functions in the \pkg{svUnit}
package. These functions allow to control the GUI in Komodo remotely
from within \R{}, and similar functions should not be too difficult
to implement for other GUIs. \end{framed}
\begin{thebibliography}{References}
\bibitem{Grosjean}Grosjean, Ph., 2003. SciViews: an object-oriented
abstraction layer to design GUIs on top of various calculation kernels
{[}online: \url{http://www.ci.tuwien.ac.at/Conferences/DSC-2003}{]}

\bibitem{IEEE}IEEE Standards Boards, 1993. IEEE standard for software
unit testing. ANSI/IEEE Std 1008-1987. 24 pp.

\bibitem{Ihaka}Ihaka R. \& R. Gentleman, 1996. R: a language for
data analysis and graphics. \emph{J. Comput. Graphic. Stat.}, \textbf{5}:299-314.

\bibitem{Jeffries}Jeffries, R., 2006. Extreme programming, web site
at: \url{http://www.xprogramming.com}.

\bibitem{Konig}König, T., K. Jünemann \& M. Burger, 2007. RUnit –
A unit test framework for R. Vignette of the package RUnit available
on CRAN. 11 pp.

\bibitem{Rcore}R Development Core Team, 2008. R: A language and environment
for statistical computing. {[}online: \url{http://www.R-project.org}{]}.\end{thebibliography}

\end{document}
NA
NA
NA
