Apply a Function over a List or Vector via Futures

Description

Apply a Function over a List or Vector via Futures

Usage

future_eapply(env, FUN, ..., all.names = FALSE, USE.NAMES = TRUE)

future_lapply(X, FUN, ..., future.globals = TRUE, future.packages = NULL,
  future.seed = FALSE, future.lazy = FALSE, future.scheduling = 1)

future_replicate(n, expr, simplify = "array", future.seed = TRUE, ...)

future_sapply(X, FUN, ..., simplify = TRUE, USE.NAMES = TRUE)

future_tapply(X, INDEX, FUN = NULL, ..., default = NA, simplify = TRUE)

future_vapply(X, FUN, FUN.VALUE, ..., USE.NAMES = TRUE)

Arguments

Value

A named (unless USE.NAMES = FALSE) list. See base::eapply() for details.

For future_lapply(), a list with same length and names as X. See base::lapply() for details.

future_replicate() is a wrapper around future_sapply() and return simplified object according to the simplify argument. See base::replicate() for details. Since future_replicate() usually involves random number generation (RNG), it uses future.seed = TRUE by default in order produce sound random numbers regardless of future backend and number of background workers used.

For future_sapply(), a vector with same length and names as X. See base::sapply() for details.

future_tapply() returns an array with mode "list", unless simplify = TRUE (default) and FUN returns a scalar, in which case the mode of the array is the same as the returned scalars. See base::tapply() for details.

For future_vapply(), a vector with same length and names as X. See base::vapply() for details.

Global variables

Argument future.globals may be used to control how globals should be handled similarly how the globals argument is used with future(). Since all function calls use the same set of globals, this function can do any gathering of globals upfront (once), which is more efficient than if it would be done for each future independently. If TRUE, NULL or not is specified (default), then globals are automatically identified and gathered. If a character vector of names is specified, then those globals are gathered. If a named list, then those globals are used as is. In all cases, FUN and any ... arguments are automatically passed as globals to each future created as they are always needed.

Reproducible random number generation (RNG)

Unless future.seed = FALSE, this function guarantees to generate the exact same sequence of random numbers given the same initial seed / RNG state - this regardless of type of futures, scheduling ("chunking") strategy, and number of workers.

RNG reproducibility is achieved by pregenerating the random seeds for all iterations (over X) by using L'Ecuyer-CMRG RNG streams. In each iteration, these seeds are set before calling FUN(X[[ii]], ...). Note, for large length(X) this may introduce a large overhead. As input (future.seed), a fixed seed (integer) may be given, either as a full L'Ecuyer-CMRG RNG seed (vector of 1+6 integers) or as a seed generating such a full L'Ecuyer-CMRG seed. If future.seed = TRUE, then .Random.seed is returned if it holds a L'Ecuyer-CMRG RNG seed, otherwise one is created randomly. If future.seed = NA, a L'Ecuyer-CMRG RNG seed is randomly created. If none of the function calls FUN(X[[ii]], ...) uses random number generation, then future.seed = FALSE may be used.

In addition to the above, it is possible to specify a pre-generated sequence of RNG seeds as a list such that length(future.seed) == length(X) and where each element is an integer seed vector that can be assigned to .Random.seed. Use this alternative with caution. Note that as.list(seq_along(X)) is not a valid set of such .Random.seed values.

In all cases but future.seed = FALSE, the RNG state of the calling R processes after this function returns is guaranteed to be "forwarded one step" from the RNG state that was before the call and in the same way regardless of future.seed, future.scheduling and future strategy used. This is done in order to guarantee that an R script calling future_lapply() multiple times should be numerically reproducible given the same initial seed.

Author(s)

The implementations of future_replicate(), future_sapply(), and future_tapply() are adopted from the source code of the corresponding base R functions, which are licensed under GPL (>= 2) with 'The R Core Team' as the copyright holder.

Examples

## Regardless of the future plan, the number of workers, and
## where they are, the random numbers produced are identical

plan(multiprocess)
y1 <- future_lapply(1:5, FUN = rnorm, future.seed = 0xBEEF)
str(y1)

plan(sequential)
y2 <- future_lapply(1:5, FUN = rnorm, future.seed = 0xBEEF)
str(y2)

stopifnot(all.equal(y1, y2))