R: Use a multi-core cluster

cluster {raster} | R Documentation

## Use a multi-core cluster

### Description

`beginCluster` creates, and `endCluster` deletes a 'snow' cluster object. This object can be used for multi-core computing with those 'raster' functions that support it.

`beginCluster` determines the number of nodes (cores) that are available and uses all of them (unless the argument `n` is used).

NOTE: beginCluster may fail when the package 'nws' is installed. You can fix that by removing the 'nws' package, or by setting the cluster type manually, e.g. `beginCluster(type="SOCK")`

endCluster closes the cluster and removes the object.

The use of the cluster is automatic in these functions: `projectRaster`, `resample` and in `extract` when using polygons.

`clusterR` is a flexible interface for using cluster with other functions. This function only works with functions that have a Raster* object as first argument and that operate on a cell by cell basis (i.e., there is no effect of neigboring cells) and return an object with the same number of cells as the input raster object. The first argument of the function called must be a Raster* object. There can only be one Raster* object argument. For example, it works with `calc` and it also works with `overlay` as long as you provide a single RasterStack or RasterBrick as the first argument.

This function is particularly useful to speed up computations in functions like predict, interpolate, and perhaps calc.

Among other functions, it does _not_ work with merge, crop, mosaic, (dis)aggregate, resample, projectRaster, focal, distance, buffer, direction. But note that projectRaster has a build-in capacity for clustering that is automatically used if beginCluster() has been called.

### Usage

    beginCluster(n, type='SOCK', nice, exclude)
    endCluster()
    clusterR(x, fun, args=NULL, export=NULL, filename='', cl=NULL, m=2, ...)

### Arguments

`n` | Integer. The number of nodes to be used (optional)
`type` | Character. The cluster type to be used
`nice` | Integer. To set the prioirty for the workers, between -20 and 20 (UNIX like platforms only)
`exclude` | Character. Packages to exclude from loading on the nodes (because they may fail there) but are required/loaded on the master
`x` | Raster* object
`fun` | function that takes `x` as its first argument
`args` | list with the arguments for the function (excluding `x`, which should always be the first argument
`export` | character. Vector of variable names to export to the cluster nodes such that the are visible to fun (e.g. a parameter that is not passed as an argument)
`filename` | character. Output filename (optional)
`cl` | cluster object (do not use it if beginCluster() has been called
`m` | tuning parameter to determine how many blocks should be used. The number is rounded and multiplied with the number of nodes.
`...` | additional arguments as for `writeRaster`

### Value

beginCluster and endCluster: None. The side effect is to create or delete a cluster object.

clusterR: as for the function called with argument `fun`

### Note

If you want to write your own cluster-enabled functions see `getCluster, returnCluster`, and the vignette about writing functions.

### Author(s)

Matteo Mattiuzzi and Robert J. Hijmans

### Examples

    ## Not run:
    # set up the cluster object for parallel computing
    beginCluster()

    r <- raster()
    values(r) <- 1:ncell(r)

    x <- clusterR(r, sqrt, verbose=T)

    f1 <- function(x) calc(x, sqrt)
    y <- clusterR(r, f1)

    s <- stack(r, r*2, r*3)
    f2 <- function(d,e,f) (d + e) / (f * param)
    param <- 122
    ov <- clusterR(s, overlay, args=list(fun=f2), export='param')

    pts <- matrix(c(0,0, 45,45), ncol=2, byrow=T)
    d <- clusterR(r, distanceFromPoints, args=list(xy=pts))

    values(r) <- runif(ncell(r))
    m <- c(0, 0.25, 1,  0.25, 0.5, 2,  0.5, 1, 3)
    m <- matrix(m, ncol=3, byrow=TRUE)
    rc1 <- clusterR(r, reclassify, args=list(rcl=m, right=FALSE),
                   filename=rasterTmpFile(), datatype='INT2S', overwrite=TRUE)

    # equivalent to:
    rc2 <- reclassify(r, rcl=m, right=FALSE, filename=rasterTmpFile(), datatype='INT2S', overwrite=TRUE)

    # example with the calc function
    a <- 10
    f3 <- function(x) sum(x)+a

    z1 <- clusterR(s, calc, args=list(fun=f3), export='a')

    # for some raster functions that use another function as an argument
    # you can write your own parallel function instead of using clusterR
    # get cluster object created with beginCluster
    cl <- getCluster()

    library(parallel)
    clusterExport(cl, "a")
    z2 <- calc(s, fun=function(x){ parApply(cl, x, 1, f3)} )
    # set flag that cluster is available again
    returnCluster()
    #

    # done with cluster object
    endCluster()

    ## End(Not run)

[Package raster version 3.0-7 Index]