R: Sub-millisecond accurate timing of expression evaluation.

microbenchmark {microbenchmark} | R Documentation

## Sub-millisecond accurate timing of expression evaluation.

### Description

`microbenchmark` serves as a more accurate replacement of the often seen `system.time(replicate(1000, expr))` expression. It tries hard to accurately measure only the time it takes to evaluate `expr`. To achieved this, the sub-millisecond (supposedly nanosecond) accurate timing functions most modern operating systems provide are used. Additionally all evaluations of the expressions are done in C code to minimize any overhead.

### Usage

    microbenchmark(..., list = NULL, times = 100L, unit, check = NULL,
      control = list(), setup = NULL)

### Arguments

`...` | Expressions to benchmark.
`list` | List of unevaluated expression to benchmark.
`times` | Number of times to evaluate the expression.
`unit` | Default unit used in `summary` and `print`.
`check` | Function to check if the expressions are equal. By default `NULL` which omits the check.
`control` | List of control arguments. See Details.
`setup` | An unevaluated expression to be run (untimed) before each benchmark expression.

### Details

This function is only meant for micro-benchmarking small pieces of source code and to compare their relative performance characteristics. You should generally avoid benchmarking larger chunks of your code using this function. Instead, try using the R profiler to detect hot spots and consider rewriting them in C/C++ or FORTRAN.

The `control` list can contain the following entries:

order

  the order in which the expressions are evaluated. “random” (the default) randomizes the execution order, “inorder” executes each expression in order and “block” executes all repetitions of each expression as one block.

warmup

  the number of warm-up iterations performed before the actual benchmark. These are used to estimate the timing overhead as well as spinning up the processor from any sleep or idle states it might be in. The default value is 2.

### Value

Object of class ‘microbenchmark’, a data frame with columns `expr` and `time`. `expr` contains the deparsed expression as passed to `microbenchmark` or the name of the argument if the expression was passed as a named argument. `time` is the measured execution time of the expression in nanoseconds. The order of the observations in the data frame is the order in which they were executed.

### Note

Depending on the underlying operating system, different methods are used for timing. On Windows the `QueryPerformanceCounter` interface is used to measure the time passed. For Linux the `clock_gettime` API is used and on Solaris the `gethrtime` function. Finally on MacOS X the, undocumented, `mach_absolute_time` function is used to avoid a dependency on the CoreServices Framework.

Before evaluating each expression `times` times, the overhead of calling the timing functions and the C function call overhead are estimated. This estimated overhead is subtracted from each measured evaluation time. Should the resulting timing be negative, a warning is thrown and the respective value is replaced by `0`. If the timing is zero, a warning is raised. Should all evaluations result in one of the two error conditions described above, an error is raised.

One platform on which the clock resolution is known to be too low to measure short runtimes with the required precision is Oracle® Solaris on some SPARC® hardware. Reports of other platforms with similar problems are welcome. Please contact the package maintainer.

### Author(s)

Olaf Mersmann

### See Also

`print.microbenchmark` to display and `boxplot.microbenchmark` or `autoplot.microbenchmark` to plot the results.

### Examples

    ## Measure the time it takes to dispatch a simple function call
    ## compared to simply evaluating the constant \code{NULL}
    f <- function() NULL
    res <- microbenchmark(NULL, f(), times=1000L)

    ## Print results:
    print(res)

    ## Plot results:
    boxplot(res)

    ## Pretty plot:
    if (requireNamespace("ggplot2")) {
      ggplot2::autoplot(res)
    }

    ## Example check usage
    my_check <- function(values) {
      all(sapply(values[-1], function(x) identical(values[[1]], x)))
    }

    f <- function(a, b)
      2 + 2

    a <- 2
    ## Check passes
    microbenchmark(2 + 2, 2 + a, f(2, a), f(2, 2), check=my_check)
    ## Not run:
    a <- 3
    ## Check fails
    microbenchmark(2 + 2, 2 + a, f(2, a), f(2, 2), check=my_check)

    ## End(Not run)
    ## Example setup usage
    set.seed(21)
    x <- rnorm(10)
    microbenchmark(x, rnorm(10), check=my_check, setup=set.seed(21))
    ## Will fail without setup
    ## Not run:
    microbenchmark(x, rnorm(10), check=my_check)

    ## End(Not run)

[Package microbenchmark version 1.4-6 Index]