R: Cross-validation methods for fitted unmarked models and fit...

crossVal {unmarked} | R Documentation

## Cross-validation methods for fitted unmarked models and fit lists

### Description

Test predictive accuracy of fitted models using several cross-validation approaches. The dataset is divided by site only into folds or testing and training datasets (i.e., encounter histories within sites are never split up).

### Usage

    ## S4 method for signature 'unmarkedFit'
    crossVal(
      object, method=c("Kfold","holdout","leaveOneOut"),
      folds=10, holdoutPct=0.25, statistic=RMSE_MAE, parallel=FALSE, ...)
    ## S4 method for signature 'unmarkedFitList'
    crossVal(
      object, method=c("Kfold","holdout","leaveOneOut"),
      folds=10, holdoutPct=0.25, statistic=RMSE_MAE, parallel=FALSE,
      sort = c("none", "increasing", "decreasing"), ...)

### Arguments

`object` | A fitted model inheriting class `unmarkedFit` or a list of fitted models with class `unmarkedFitList`
`method` | Cross validation method to use as string. Valid options are `"Kfold"`, `"holdout"`, or `"leaveOneOut"`
`folds` | Number of folds to use for k-fold cross validation
`holdoutPct` | Proportion of dataset (value between 0-1) to use as the "holdout" or "test" set, for the holdout method
`statistic` | Function that calculates statistics for each fold. The function must take an `unmarkedFit` object as the first argument and return a named numeric vector with statistic value(s). The default function `RMSE_MAE` returns root-mean-square error and mean absolute error. See `unmarked:::RMSE_MAE` for an example of correct statistic function structure.
`parallel` | If `TRUE`, run folds in parallel. This may speed up cross-validation if the unmarked model takes a long time to fit or you have a large number of sites and are using leave-one-out cross-validation.
`sort` | If doing cross-validation on a `fitList`, you can optionally sort the resulting table(s) of statistic values for each model.
`...` | Other arguments passed to the statistic function.

### Value

`unmarkedCrossVal` or `unmarkedCrossValList` object containing calculated statistic values for each fold.

### Author(s)

Ken Kellner contact@kenkellner.com

### See Also

`fitList`, `unmarkedFit`

### Examples

    ## Not run:
    #Get data
    data(frogs)
    pferUMF <- unmarkedFrameOccu(pfer.bin)
    siteCovs(pferUMF) <- data.frame(sitevar1 = rnorm(numSites(pferUMF)))
    obsCovs(pferUMF) <- data.frame(obsvar1 = rnorm(numSites(pferUMF) * obsNum(pferUMF)))

    #Fit occupancy model
    fm <- occu(~ obsvar1 ~ 1, pferUMF)

    #k-fold cross validation with 10 folds
    (kfold = crossVal(fm, method="Kfold", folds=10))

    #holdout method with 25
    (holdout = crossVal(fm,method='holdout', holdoutPct=0.25))

    #Leave-one-out method
    (leave = crossVal(fm, method='leaveOneOut'))

    #Fit a second model and combine into a fitList
    fm2 <- occu(~1 ~1, pferUMF)
    fl <- fitList(fm2,fm)

    #Cross-validation for all fits in fitList using holdout method
    (cvlist <- crossVal(fl, method='holdout'))

    ## End(Not run)

[Package unmarked version 1.0.0 Index]