R: Parallelized 'lapply'

hpc {tm} | R Documentation

## Parallelized ‘lapply’

### Description

Parallelize applying a function over a list or vector according to the registered parallelization engine.

### Usage

    tm_parLapply(X, FUN, ...)
    tm_parLapply_engine(new)

### Arguments

`X` | A vector (atomic or list), or other objects suitable for the engine in use.
`FUN` | the function to be applied to each element of `X`.
`...` | optional arguments to `FUN`.
`new` | an object inheriting from class `cluster` as created by `makeCluster()` from package parallel, or a function with formals `X`, `FUN` and `...`, or `NULL` corresponding to the default of using no parallelization engine.

### Details

Parallelization can be employed to speed up some of the embarrassingly parallel computations performed in package tm, specifically `tm_index()`, `tm_map()` on a non-lazy-mapped `VCorpus`, and `TermDocumentMatrix()` on a `VCorpus` or `PCorpus`. Functions `tm_parLapply()` and `tm_parLapply_engine()` can be used to customize parallelization according to the available resources.

`tm_parLapply_engine()` is used for getting (with no arguments) or setting (with argument `new`) the parallelization engine employed (see below for examples).

If an engine is set to an object inheriting from class `cluster`, `tm_parLapply()` calls `parLapply()` with this cluster and the given arguments. If set to a function, `tm_parLapply()` calls the function with the given arguments. Otherwise, it simply calls `lapply()`.

Hence, to achieve parallelization via `parLapply()` and a default cluster registered via `setDefaultCluster()`, one can use

      tm_parLapply_engine(function(X, FUN, ...)
          parallel::parLapply(NULL, X, FUN, ...))

or re-register the cluster, say `cl`, using

      tm_parLapply_engine(cl)

(note that there is no mechanism for programmatically getting the registered default cluster). Using

      tm_parLapply_engine(function(X, FUN, ...)
          parallel::parLapplyLB(NULL, X, FUN, ...))

or

      tm_parLapply_engine(function(X, FUN, ...)
          parallel::parLapplyLB(cl, X, FUN, ...))

gives load-balancing parallelization with the registered default or given cluster, respectively. To achieve parallelization via forking (on Unix-alike platforms), one can use the above with clusters created by `makeForkCluster()`, or use

      tm_parLapply_engine(parallel::mclapply)

or

      tm_parLapply_engine(function(X, FUN, ...)
          parallel::mclapply(X, FUN, ..., mc.cores = n))

to use `mclapply()` with the default or given number `n` of cores.

### Value

A list the length of `X`, with the result of applying `FUN` together with the `...` arguments to each element of `X`.

### See Also

`makeCluster()`, `parLapply()`, `parLapplyLB()`, and `mclapply()`.

[Package tm version 0.7-1 Index]