R: Fast Cross-Validation via Sequential Testing

CVST-package {CVST} | R Documentation

## Fast Cross-Validation via Sequential Testing

### Description

The fast cross-validation via sequential testing (CVST) procedure is an improved cross-validation procedure which uses non-parametric testing coupled with sequential analysis to determine the best parameter set on linearly increasing subsets of the data. By eliminating under-performing candidates quickly and keeping promising candidates as long as possible, the method speeds up the computation while preserving the capability of a full cross-validation. Additionally to the CVST the package contains an implementation of the ordinary k-fold cross-validation with a flexible and powerful set of helper objects and methods to handle the overall model selection process. The implementations of the Cochran's Q test with permutations and the sequential testing framework of Wald are generic and can therefore also be used in other contexts.

### Details

Package: | CVST
Type: | Package
Title: | Fast Cross-Validation via Sequential Testing
Version: | 0.2-2
Date: | 2018-05-26
Depends: | kernlab,Matrix
Author: | Tammo Krueger, Mikio Braun
Maintainer: | Tammo Krueger <tammokrueger@googlemail.com>
Description: | The fast cross-validation via sequential testing (CVST) procedure is an improved cross-validation procedure which uses non-parametric testing coupled with sequential analysis to determine the best parameter set on linearly increasing subsets of the data. By eliminating under-performing candidates quickly and keeping promising candidates as long as possible, the method speeds up the computation while preserving the capability of a full cross-validation. Additionally to the CVST the package contains an implementation of the ordinary k-fold cross-validation with a flexible and powerful set of helper objects and methods to handle the overall model selection process. The implementations of the Cochran's Q test with permutations and the sequential testing framework of Wald are generic and can therefore also be used in other contexts.
License: | GPL (>=2.0)

Index of help topics:

    CV                      Perform a k-fold Cross-validation
    CVST-package            Fast Cross-Validation via Sequential Testing
    cochranq.test           Cochran's Q Test with Permutation
    constructCVSTModel      Setup for a CVST Run.
    constructData           Construction and Handling of 'CVST.data'
                            Objects
    constructLearner        Construction of Specific Learners for CVST
    constructParams         Construct a Grid of Parameters
    constructSequentialTest
                            Construct and Handle Sequential Tests.
    fastCV                  The Fast Cross-Validation via Sequential
                            Testing (CVST) Procedure
    noisyDonoho             Generate Donoho's Toy Data Sets
    noisySine               Regression and Classification Toy Data Set

### Author(s)

Tammo Krueger, Mikio Braun

Maintainer: Tammo Krueger <tammokrueger@googlemail.com>

### References

Tammo Krueger, Danny Panknin, and Mikio Braun. Fast cross-validation via sequential testing. Journal of Machine Learning Research 16 (2015) 1103-1155. URL http://jmlr.org/papers/volume16/krueger15a/krueger15a.pdf.

Abraham Wald. Sequential Analysis. Wiley, 1947.

W. G. Cochran. The comparison of percentages in matched samples. Biometrika, 37 (3-4):256–266, 1950.

M. Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32 (200):675–701, 1937.

### Examples

    ns = noisySine(100)
    svm = constructSVMLearner()
    params = constructParams(kernel="rbfdot", sigma=10^(-3:3), nu=c(0.05, 0.1, 0.2, 0.3))
    opt = fastCV(ns, svm, params, constructCVSTModel())

[Package CVST version 0.2-2 Index]