R: Benchmark a series of functions

mark {bench} | R Documentation

## Benchmark a series of functions

### Description

Benchmark a list of quoted expressions. Each expression will always run at least twice, once to measure the memory allocation and store results and one or more times to measure timing.

### Usage

    mark(
      ...,
      min_time = 0.5,
      iterations = NULL,
      min_iterations = 1,
      max_iterations = 10000,
      check = TRUE,
      memory = capabilities("profmem"),
      filter_gc = TRUE,
      relative = FALSE,
      time_unit = NULL,
      exprs = NULL,
      env = parent.frame()
    )

### Arguments

`...` | Expressions to benchmark, if named the `expression` column will be the name, otherwise it will be the deparsed expression.
`min_time` | The minimum number of seconds to run each expression, set to `Inf` to always run `max_iterations` times instead.
`iterations` | If not `NULL`, the default, run each expression for exactly this number of iterations. This overrides both `min_iterations` and `max_iterations`.
`min_iterations` | Each expression will be evaluated a minimum of `min_iterations` times.
`max_iterations` | Each expression will be evaluated a maximum of `max_iterations` times.
`check` | Check if results are consistent. If `TRUE`, checking is done with `all.equal()`, if `FALSE` checking is disabled and results are not stored. If `check` is a function that function will be called with each pair of results to determine consistency.
`memory` | If `TRUE` (the default when R is compiled with memory profiling), track memory allocations using. If `FALSE` disable memory tracking.
`filter_gc` | If `TRUE` remove iterations that contained at least one garbage collection before summarizing. If `TRUE` but an expression had a garbage collection in every iteration, filtering is disabled, with a warning.
`relative` | If `TRUE` all summaries are computed relative to the minimum execution time rather than absolute time.
`time_unit` | If `NULL` the times are reported in a human readable fashion depending on each value. If one of 'ns', 'us', 'ms', 's', 'm', 'h', 'd', 'w' the time units are instead expressed as nanoseconds, microseconds, milliseconds, seconds, hours, minutes, days or weeks respectively.
`exprs` | A list of quoted expressions. If supplied overrides expressions defined in `...`.
`env` | The environment which to evaluate the expressions

### Value

A tibble with the additional summary columns. The following summary columns are computed

- `expression` - `bench_expr` The deparsed expression that was evaluated (or its name if one was provided).

- `min` - `bench_time` The minimum execution time.

- `median` - `bench_time` The sample median of execution time.

- `itr/sec` - `double` The estimated number of executions performed per second.

- `mem_alloc` - `bench_bytes` Total amount of memory allocated by R while running the expression. Memory allocated outside the R heap, e.g. by `malloc()` or `new` directly is not tracked, take care to avoid misinterpreting the results if running code that may do this.

- `gc/sec` - `double` The number of garbage collections per second.

- `n_itr` - `integer` Total number of iterations after filtering garbage collections (if `filter_gc == TRUE`).

- `n_gc` - `double` Total number of garbage collections performed over all iterations. This is a psudo-measure of the pressure on the garbage collector, if it varies greatly between to alternatives generally the one with fewer collections will cause fewer allocation in real usage.

- `total_time` - `bench_time` The total time to perform the benchmarks.

- `result` - `list` A list column of the object(s) returned by the evaluated expression(s).

- `memory` - `list` A list column with results from `Rprofmem()`.

- `time` - `list` A list column of `bench_time` vectors for each evaluated expression.

- `gc` - `list` A list column with tibbles containing the level of garbage collection (0-2, columns) for each iteration (rows).

### See Also

`press()` to run benchmarks across a grid of parameters.

### Examples

    dat <- data.frame(x = runif(100, 1, 1000), y=runif(10, 1, 1000))
    mark(
      min_time = .1,

      dat[dat$x > 500, ],
      dat[which(dat$x > 500), ],
      subset(dat, x > 500))

[Package bench version 1.1.1 Index]