R: Apply (Smoothed) Rectified Linear Transformation

step_relu {recipes} | R Documentation

## Apply (Smoothed) Rectified Linear Transformation

### Description

`step_relu` creates a specification of a recipe step that will apply the rectified linear or softplus transformations to numeric data. The transformed data is added as new columns to the data matrix.

### Usage

    step_relu(recipe, ..., role = "predictor", trained = FALSE, shift = 0,
      reverse = FALSE, smooth = FALSE, prefix = "right_relu_", skip = FALSE)

### Arguments

`recipe` | A recipe object. The step will be added to the sequence of operations for this recipe.
`...` | One or more selector functions to choose which variables are affected by the step. See `selections()` for more details.
`role` | Not used by this step since no new variables are created.
`trained` | A logical to indicate if the quantities for preprocessing have been estimated.
`shift` | A numeric value dictating a translation to apply to the data.
`reverse` | A logical to indicate if the left hinge should be used as opposed to the right hinge.
`smooth` | A logical indicating if the softplus function, a smooth appromixation to the rectified linear transformation, should be used.
`prefix` | A prefix for generated column names, default to "right_relu_" when right hinge transformation and "left_relu_" for reversed/left hinge transformations.
`skip` | A logical. Should the step be skipped when the recipe is baked by `bake.recipe()`? While all operations are baked when `prep.recipe()` is run, some operations may not be able to be conducted on new data (e.g. processing the outcome variable(s)). Care should be taken when using `skip = TRUE` as it may affect the computations for subsequent operations

### Details

The rectified linear transformation is calculated as

max(0, x - c)

and is also known as the ReLu or right hinge function. If `reverse` is true, then the transformation is reflected about the y-axis, like so:

max(0, c - x)

Setting the `smooth` option to true will instead calculate a smooth approximation to ReLu according to

ln(1 + e^(x - c)

The `reverse` argument may also be applied to this transformation.

### Value

An updated version of `recipe` with the new step added to the sequence of existing steps (if any).

### Connection to MARS

The rectified linear transformation is used in the Multivariate Adaptive Regression Splines as a basis function to fit piecewise linear functions to data in a strategy similar to that employeed in tree based models. The transformation is a popular choice as an activation function in many neural networks, which could then be seen as a stacked generalization of MARS when making use of ReLu activations. The hinge function also appears in the loss function of Support Vector Machines, where it penalizes residuals only if they are within a certain margin of the decision boundary.

### See Also

`recipe()` `prep.recipe()` `bake.recipe()`

### Examples

    data(biomass)

    biomass_tr <- biomass[biomass$dataset == "Training",]
    biomass_te <- biomass[biomass$dataset == "Testing",]

    rec <- recipe(HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur,
                  data = biomass_tr)

    transformed_te <- rec %>%
      step_relu(carbon, shift = 40) %>%
      prep(biomass_tr) %>%
      bake(biomass_te)

    transformed_te

[Package recipes version 0.1.2 Index]