PPC-loo {bayesplot}R Documentation

LOO predictive checks

Description

Leave-One-Out (LOO) predictive checks. See the Plot Descriptions section below for details.

Usage

ppc_loo_pit(y, yrep, lw, pit, compare = c("uniform", "normal"), ...,
  size = 2, alpha = 1)

ppc_loo_intervals(y, yrep, lw, intervals = NULL, ..., prob = 0.9,
  size = 1, fatten = 3, order = c("index", "median"))

ppc_loo_ribbon(y, yrep, lw, intervals = NULL, ..., prob = 0.9,
  alpha = 0.33, size = 0.25)

Arguments

y

A vector of observations. See Details.

yrep

An S by N matrix of draws from the posterior predictive distribution, where S is the size of the posterior sample (or subset of the posterior sample used to generate yrep) and N is the number of observations (the length of y). The columns of yrep should be in the same order as the data points in y for the plots to make sense. See Details for additional instructions.

lw

A matrix of (smoothed) log weights with the same dimensions as yrep. See the psislw function in the loo package, which returns smoothed weights that can be used to specify lw.

pit

For ppc_loo_pit, optionally a vector of precomputed PIT values that can be specified instead of y, yrep, and lw (these are all ignored if pit is specified). If not specified the PIT values are computed internally before plotting.

compare

For ppc_loo_pit, a string that can be either "uniform" or "normal". If "uniform" (the default) the Q-Q plot compares computed PIT values to the standard uniform distribution. If compare="normal", the Q-Q plot compares standardized PIT values to the standard normal distribution.

...

Currently unused.

alpha, size, fatten

Arguments passed to code geoms to control plot aesthetics. For ppc_loo_pit, size and alpha are passed to geom_point. For ppc_loo_intervals, size and fatten are passed to geom_pointrange. For ppc_loo_ribbon, alpha and size are passed to geom_ribbon.

intervals

For ppc_loo_intervals and ppc_loo_ribbon, optionally a matrix of precomputed LOO predictive intervals intervals with that can be specified instead of yrep and lw (these are both ignored if intervals is specified). If not specified the intervals are computed internally before plotting. If specified, intervals must be a matrix with number of rows equal to the number of data points and three columns in the following order: the first for the lower bound of the interval, the second for median (50%), and the third for the interval upper bound (column names are ignored).

prob

A value between 0 and 1 indicating the desired probability mass to include in the intervals. The default is 0.9.

order

For ppc_loo_intervals A string indicating how to arrange the plotted intervals. The default ("index") is to plot them in the order of the observations. The alternative ("median") arranges them by median value from smallest (left) to largest (right).

Value

A ggplot object that can be further customized using the ggplot2 package.

Plot Descriptions

ppc_loo_pit

The calibration of marginal predictions can be assessed using probability integral transformation (PIT) checks. LOO improves the check by avoiding the double use of data. See the section on marginal predictive checks in Gelman et al. (2013, p. 152–153). The default LOO PIT predictive check is a quantile-quantile (Q-Q) plot comparing the LOO PITs to the standard uniform distribution. Comparing to a uniform distribution is not good for extreme probabilities close to 0 and 1, so it can be useful to set the compare argument to "normal", which will produce a Q-Q plot comparing standardized PIT values to the standard normal distribution. This can be helpful to see the calibration better for the extreme values.

ppc_loo_intervals, ppc_loo_ribbon

Similar to ppc_intervals and ppc_ribbon but the intervals are for the LOO predictive distribution.

References

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2013). Bayesian Data Analysis. Chapman & Hall/CRC Press, London, third edition. (p. 152–153)

Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. 27(5), 1413–1432. doi:10.1007/s11222-016-9696-4. arXiv preprint: http://arxiv.org/abs/1507.04544/

See Also

Other PPCs: PPC-discrete, PPC-distributions, PPC-errors, PPC-intervals, PPC-overview, PPC-scatterplots, PPC-test-statistics

Examples


## Not run: 
library(rstanarm)
library(loo)

head(radon)
fit <- stan_lmer(log_radon ~ floor + log_uranium + floor:log_uranium
                   + (1 + floor | county), data = radon, cores = 2)
y <- radon$log_radon
yrep <- posterior_predict(fit)
psis <- psislw(-log_lik(fit), cores = 2)

# marginal predictive check using LOO probability integral transform
color_scheme_set("orange")
ppc_loo_pit(y, yrep, lw = psis$lw_smooth)
ppc_loo_pit(y, yrep, lw = psis$lw_smooth, compare = "normal")

# loo predictive intervals vs observations
sel <- 800:900
ppc_loo_intervals(y[sel], yrep[, sel], psis$lw_smooth[, sel],
                  prob = 0.9, size = 0.5)

color_scheme_set("gray")
ppc_loo_intervals(y[sel], yrep[, sel], psis$lw_smooth[, sel],
                  order = "median", prob = 0.8, size = 0.5)

## End(Not run)
[Package bayesplot version 1.4.0 Index]