R: Feature Tranformation - RFormula (Estimator)

ft_r_formula {sparklyr} | R Documentation

## Feature Tranformation – RFormula (Estimator)

### Description

Implements the transforms required for fitting a dataset against an R model formula. Currently we support a limited subset of the R operators, including `~`, `.`, `:`, `+`, and `-`. Also see the R formula docs here: http://stat.ethz.ch/R-manual/R-patched/library/stats/html/formula.html

### Usage

    ft_r_formula(x, formula, features_col = "features", label_col = "label",
      force_index_label = FALSE, dataset = NULL,
      uid = random_string("r_formula_"), ...)

### Arguments

`x` | A `spark_connection`, `ml_pipeline`, or a `tbl_spark`.
`formula` | R formula as a character string or a formula. Formula objects are converted to character strings directly and the environment is not captured.
`features_col` | Features column name, as a length-one character vector. The column should be single vector column of numeric values. Usually this column is output by `ft_r_formula`.
`label_col` | Label column name. The column should be a numeric column. Usually this column is output by `ft_r_formula`.
`force_index_label` | (Spark 2.1.0+) Force to index label whether it is numeric or string type. Usually we index label only when it is string type. If the formula was used by classification algorithms, we can force to index label even it is numeric type by setting this param with true. Default: `FALSE`.
`dataset` | (Optional) A `tbl_spark`. If provided, eagerly fit the (estimator) feature "transformer" against `dataset`. See details.
`uid` | A character string used to uniquely identify the feature transformer.
`...` | Optional arguments; currently unused.

### Details

The basic operators in the formula are:

- ~ separate target and terms

- + concat terms, "+ 0" means removing intercept

- - remove a term, "- 1" means removing intercept

- : interaction (multiplication for numeric values, or binarized categorical values)

- . all columns except target

Suppose a and b are double columns, we use the following simple examples to illustrate the effect of RFormula:

- `y ~ a + b` means model `y ~ w0 + w1 * a + w2 * b` where `w0` is the intercept and `w1, w2` are coefficients.

- `y ~ a + b + a:b - 1` means model `y ~ w1 * a + w2 * b + w3 * a * b` where `w1, w2, w3` are coefficients.

RFormula produces a vector column of features and a double or string column of label. Like when formulas are used in R for linear regression, string input columns will be one-hot encoded, and numeric columns will be cast to doubles. If the label column is of type string, it will be first transformed to double with StringIndexer. If the label column does not exist in the DataFrame, the output label column will be created from the specified response variable in the formula.

When `dataset` is provided for an estimator transformer, the function internally calls `ml_fit()` against `dataset`. Hence, the methods for `spark_connection` and `ml_pipeline` will then return a `ml_transformer` and a `ml_pipeline` with a `ml_transformer` appended, respectively. When `x` is a `tbl_spark`, the estimator will be fit against `dataset` before transforming `x`.

When `dataset` is not specified, the constructor returns a `ml_estimator`, and, in the case where `x` is a `tbl_spark`, the estimator fits against `x` then to obtain a transformer, which is then immediately used to transform `x`.

### Value

The object returned depends on the class of `x`.

- `spark_connection`: When `x` is a `spark_connection`, the function returns a `ml_transformer`, a `ml_estimator`, or one of their subclasses. The object contains a pointer to a Spark `Transformer` or `Estimator` object and can be used to compose `Pipeline` objects.

- `ml_pipeline`: When `x` is a `ml_pipeline`, the function returns a `ml_pipeline` with the transformer or estimator appended to the pipeline.

- `tbl_spark`: When `x` is a `tbl_spark`, a transformer is constructed then immediately applied to the input `tbl_spark`, returning a `tbl_spark`

### See Also

See http://spark.apache.org/docs/latest/ml-features.html for more information on the set of transformations available for DataFrame columns in Spark.

Other feature transformers: `ft_binarizer`, `ft_bucketizer`, `ft_chisq_selector`, `ft_count_vectorizer`, `ft_dct`, `ft_elementwise_product`, `ft_feature_hasher`, `ft_hashing_tf`, `ft_idf`, `ft_imputer`, `ft_index_to_string`, `ft_interaction`, `ft_lsh`, `ft_max_abs_scaler`, `ft_min_max_scaler`, `ft_ngram`, `ft_normalizer`, `ft_one_hot_encoder`, `ft_pca`, `ft_polynomial_expansion`, `ft_quantile_discretizer`, `ft_regex_tokenizer`, `ft_sql_transformer`, `ft_standard_scaler`, `ft_stop_words_remover`, `ft_string_indexer`, `ft_tokenizer`, `ft_vector_assembler`, `ft_vector_indexer`, `ft_vector_slicer`, `ft_word2vec`

[Package sparklyr version 0.8.0 Index]