R: Automatic group-wise Anomaly Detection by STL Decomposition

tk_anomaly_diagnostics {timetk} | R Documentation

## Automatic group-wise Anomaly Detection by STL Decomposition

### Description

`tk_anomaly_diagnostics()` is the preprocessor for `plot_anomaly_diagnostics()`. It performs automatic anomaly detection for one or more time series groups.

### Usage

    tk_anomaly_diagnostics(
      .data,
      .date_var,
      .value,
      .frequency = "auto",
      .trend = "auto",
      .alpha = 0.05,
      .max_anomalies = 0.2,
      .message = TRUE
    )

    ## S3 method for class 'data.frame'
    tk_anomaly_diagnostics(
      .data,
      .date_var,
      .value,
      .frequency = "auto",
      .trend = "auto",
      .alpha = 0.05,
      .max_anomalies = 0.2,
      .message = TRUE
    )

### Arguments

`.data` | A `tibble` or `data.frame` with a time-based column
`.date_var` | A column containing either date or date-time values
`.value` | A column containing numeric values
`.frequency` | Controls the seasonal adjustment (removal of seasonality). Input can be either "auto", a time-based definition (e.g. "2 weeks"), or a numeric number of observations per frequency (e.g. 10). Refer to `tk_get_frequency()`.
`.trend` | Controls the trend component. For STL, trend controls the sensitivity of the LOESS smoother, which is used to remove the remainder. Refer to `tk_get_trend()`.
`.alpha` | Controls the width of the "normal" range. Lower values are more conservative while higher values are less prone to incorrectly classifying "normal" observations.
`.max_anomalies` | The maximum percent of anomalies permitted to be identified.
`.message` | A boolean. If `TRUE`, will output information related to automatic frequency and trend selection (if applicable).

### Details

The `tk_anomaly_diagnostics()` method for anomaly detection that implements a 2-step process to detect outliers in time series.

Step 1: Detrend & Remove Seasonality using STL Decomposition

The decomposition separates the "season" and "trend" components from the "observed" values leaving the "remainder" for anomaly detection.

The user can control two parameters: frequency and trend.

1. `.frequency`: Adjusts the "season" component that is removed from the "observed" values.

2. `.trend`: Adjusts the trend window (t.window parameter from `stats::stl()` that is used.

The user may supply both `.frequency` and `.trend` as time-based durations (e.g. "6 weeks") or numeric values (e.g. 180) or "auto", which predetermines the frequency and/or trend based on the scale of the time series using the `tk_time_scale_template()`.

Step 2: Anomaly Detection

Once "trend" and "season" (seasonality) is removed, anomaly detection is performed on the "remainder". Anomalies are identified, and boundaries (recomposed_l1 and recomposed_l2) are determined.

The Anomaly Detection Method uses an inner quartile range (IQR) of +/-25 the median.

IQR Adjustment, alpha parameter

With the default `alpha = 0.05`, the limits are established by expanding the 25/75 baseline by an IQR Factor of 3 (3X). The IQR Factor = 0.15 / alpha (hence 3X with alpha = 0.05):

- To increase the IQR Factor controlling the limits, decrease the alpha, which makes it more difficult to be an outlier.

- Increase alpha to make it easier to be an outlier.

- The IQR outlier detection method is used in `forecast::tsoutliers()`.

- A similar outlier detection method is used by Twitter's `AnomalyDetection` package.

- Both Twitter and Forecast tsoutliers methods have been implemented in Business Science's `anomalize` package.

### Value

A `tibble` or `data.frame` with STL Decomposition Features (observed, season, trend, remainder, seasadj) and Anomaly Features (remainder_l1, remainder_l2, anomaly, recomposed_l1, and recomposed_l2)

### References

1. CLEVELAND, R. B., CLEVELAND, W. S., MCRAE, J. E., AND TERPENNING, I. STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics, Vol. 6, No. 1 (1990), pp. 3-73.

2. Owen S. Vallis, Jordan Hochenbaum and Arun Kejariwal (2014). A Novel Technique for Long-Term Anomaly Detection in the Cloud. Twitter Inc.

### See Also

- `plot_anomaly_diagnostics()`: Visual anomaly detection

### Examples

    library(dplyr)
    library(timetk)

    walmart_sales_weekly %>%
        filter(id %in% c("1_1", "1_3")) %>%
        group_by(id) %>%
        tk_anomaly_diagnostics(Date, Weekly_Sales)

[Package timetk version 2.6.1 Index]