R: Feature Tranformation - PCA (Estimator)

ft_pca {sparklyr} | R Documentation

## Feature Tranformation – PCA (Estimator)

### Description

PCA trains a model to project vectors to a lower dimensional space of the top k principal components.

### Usage

    ft_pca(x, input_col, output_col, k, dataset = NULL,
      uid = random_string("pca_"), ...)

    ml_pca(x, features = tbl_vars(x), k = length(features), pc_prefix = "PC",
      ...)

### Arguments

`x` | A `spark_connection`, `ml_pipeline`, or a `tbl_spark`.
`input_col` | The name of the input column.
`output_col` | The name of the output column.
`k` | The number of principal components
`dataset` | (Optional) A `tbl_spark`. If provided, eagerly fit the (estimator) feature "transformer" against `dataset`. See details.
`uid` | A character string used to uniquely identify the feature transformer.
`...` | Optional arguments; currently unused.
`features` | The columns to use in the principal components analysis. Defaults to all columns in `x`.
`pc_prefix` | Length-one character vector used to prepend names of components.

### Details

When `dataset` is provided for an estimator transformer, the function internally calls `ml_fit()` against `dataset`. Hence, the methods for `spark_connection` and `ml_pipeline` will then return a `ml_transformer` and a `ml_pipeline` with a `ml_transformer` appended, respectively. When `x` is a `tbl_spark`, the estimator will be fit against `dataset` before transforming `x`.

When `dataset` is not specified, the constructor returns a `ml_estimator`, and, in the case where `x` is a `tbl_spark`, the estimator fits against `x` then to obtain a transformer, which is then immediately used to transform `x`.

`ml_pca()` is a wrapper around `ft_pca()` that returns a `ml_model`.

### Value

The object returned depends on the class of `x`.

- `spark_connection`: When `x` is a `spark_connection`, the function returns a `ml_transformer`, a `ml_estimator`, or one of their subclasses. The object contains a pointer to a Spark `Transformer` or `Estimator` object and can be used to compose `Pipeline` objects.

- `ml_pipeline`: When `x` is a `ml_pipeline`, the function returns a `ml_pipeline` with the transformer or estimator appended to the pipeline.

- `tbl_spark`: When `x` is a `tbl_spark`, a transformer is constructed then immediately applied to the input `tbl_spark`, returning a `tbl_spark`

### See Also

See http://spark.apache.org/docs/latest/ml-features.html for more information on the set of transformations available for DataFrame columns in Spark.

Other feature transformers: `ft_binarizer`, `ft_bucketizer`, `ft_chisq_selector`, `ft_count_vectorizer`, `ft_dct`, `ft_elementwise_product`, `ft_feature_hasher`, `ft_hashing_tf`, `ft_idf`, `ft_imputer`, `ft_index_to_string`, `ft_interaction`, `ft_lsh`, `ft_max_abs_scaler`, `ft_min_max_scaler`, `ft_ngram`, `ft_normalizer`, `ft_one_hot_encoder`, `ft_polynomial_expansion`, `ft_quantile_discretizer`, `ft_r_formula`, `ft_regex_tokenizer`, `ft_sql_transformer`, `ft_standard_scaler`, `ft_stop_words_remover`, `ft_string_indexer`, `ft_tokenizer`, `ft_vector_assembler`, `ft_vector_indexer`, `ft_vector_slicer`, `ft_word2vec`

### Examples

    ## Not run:
    library(dplyr)

    sc <- spark_connect(master = "local")
    iris_tbl <- sdf_copy_to(sc, iris, name = "iris_tbl", overwrite = TRUE)

    iris_tbl %>%
      select(-Species) %>%
      ml_pca(k = 2)

    ## End(Not run)

[Package sparklyr version 0.8.2 Index]