R: Compute the confidence interval of arbitrary coordinates

ci.coords {pROC} | R Documentation

## Compute the confidence interval of arbitrary coordinates

### Description

This function computes the confidence interval (CI) of the coordinates of a ROC curves with the `coords` function. By default, the 95% CI are computed with 2000 stratified bootstrap replicates.

### Usage

    # ci.coords(...)
    ## S3 method for class 'roc'
    ci.coords(roc, x,
    input=c("threshold", "specificity", "sensitivity"),
    ret=c("threshold", "specificity", "sensitivity"),
    best.method=c("youden", "closest.topleft"), best.weights=c(1, 0.5),
    best.policy = c("stop", "omit", "random"),
    conf.level=0.95, boot.n=2000,
    boot.stratified=TRUE,
    progress=getOption("pROCProgress")$name, ...)
    ## S3 method for class 'formula'
    ci.coords(formula, data, ...)
    ## S3 method for class 'smooth.roc'
    ci.coords(smooth.roc, x,
    input=c("specificity", "sensitivity"), ret=c("specificity", "sensitivity"),
    best.method=c("youden", "closest.topleft"), best.weights=c(1, 0.5),
    best.policy = c("stop", "omit", "random"),
    conf.level=0.95, boot.n=2000,
    boot.stratified=TRUE,
    progress=getOption("pROCProgress")$name, ...)
    ## Default S3 method:
    ci.coords(response, predictor, ...)

### Arguments

`roc, smooth.roc` | a “roc” object from the `roc` function, or a “smooth.roc” object from the `smooth` function.
`response, predictor` | arguments for the `roc` function.
`formula, data` | a formula (and possibly a data object) of type response~predictor for the `roc` function.
`x, input, ret, best.method, best.weights` | Arguments passed to `coords`. See there for more details. The only difference is on the `x` argument which cannot be “all” or “local maximas”.
`best.policy` | The policy follow when multiple “best” thresholds are returned by `coords`. “stop” will abort the processing with `stop` (default), “omit” will ignore the sample (as in `NA`) and “random” will select one of the threshold randomly.
`conf.level` | the width of the confidence interval as [0,1], never in percent. Default: 0.95, resulting in a 95% CI.
`boot.n` | the number of bootstrap replicates. Default: 2000.
`boot.stratified` | should the bootstrap be stratified (default, same number of cases/controls in each replicate than in the original sample) or not.
`progress` | the name of progress bar to display. Typically “none”, “win”, “tk” or “text” (see the `name` argument to `create_progress_bar` for more information), but a list as returned by `create_progress_bar` is also accepted. See also the “Progress bars” section of this package's documentation.
`...` | further arguments passed to or from other methods, especially arguments for `roc` and `ci.coords.roc` when calling `ci.coords.default` or `ci.coords.formula`. Arguments for `txtProgressBar` (only `char` and `style`) if applicable.

### Details

`ci.coords.formula` and `ci.coords.default` are convenience methods that build the ROC curve (with the `roc` function) before calling `ci.coords.roc`. You can pass them arguments for both `roc` and `ci.coords.roc`. Simply use `ci.coords` that will dispatch to the correct method.

This function creates `boot.n` bootstrap replicate of the ROC curve, and evaluates the coordinates specified by the `x`, `input`, `ret`, `best.method` and `best.weights` arguments. Then it computes the confidence interval as the percentiles given by `conf.level`.

For more details about the bootstrap, see the Bootstrap section in this package's documentation.

### Value

A matrix of class “ci.coords”, “ci” and “matrix” (in this order), with the confidence intervals of the CI. The matrix has 3 columns (lower bound, median and upper bound) and as many rows as `x` * `ret` were requested. Rows are sorted by `x` and then by `ret` and named as “input x: return”.

Additionally, the list has the following attributes:

`conf.level` | the width of the CI, in fraction.
`boot.n` | the number of bootstrap replicates.
`boot.stratified` | whether or not the bootstrapping was stratified.
`roc` | the object of class “roc” that was used to compute the CI.

### Warnings

If `boot.stratified=FALSE` and the sample has a large imbalance between cases and controls, it could happen that one or more of the replicates contains no case or control observation, producing a `NA` area. The warning “NA value(s) produced during bootstrap were ignored.” will be issued and the observation will be ignored. If you have a large imbalance in your sample, it could be safer to keep `boot.stratified=TRUE`.

This warning will also be displayed if you chose `best.policy = "omit"` and a ROC curve with multiple “best” threshold was generated during at least one of the replicates.

### References

James Carpenter and John Bithell (2000) “Bootstrap condence intervals: when, which, what? A practical guide for medical statisticians”. Statistics in Medicine 19, 1141–1164. DOI: 10.1002/(SICI)1097-0258(20000515)19:9<1141::AID-SIM479>3.0.CO;2-F.

Tom Fawcett (2006) “An introduction to ROC analysis”. Pattern Recognition Letters 27, 861–874. DOI: 10.1016/j.patrec.2005.10.010.

Hadley Wickham (2011) “The Split-Apply-Combine Strategy for Data Analysis”. Journal of Statistical Software, 40, 1–29. URL: www.jstatsoft.org/v40/i01.

### See Also

`roc`, `coords`, `ci`

CRAN package plyr, employed in this function.

### Examples

    data(aSAH)

    ## Not run:
    # Syntax (response, predictor):
    ci.coords(aSAH$outcome, aSAH$s100b, x="best", input = "threshold",
              ret=c("specificity", "ppv", "tp"))

    # With a roc object:
    rocobj <- roc(aSAH$outcome, aSAH$s100b)
    ci.coords(rocobj, x=0.9, input = "sensitivity", ret="specificity")
    ci.coords(rocobj, x=0.9, input = "sensitivity", ret=c("specificity", "ppv", "tp"))
    ci.coords(rocobj, x=c(0.1, 0.5, 0.9), input = "sensitivity", ret="specificity")
    ci.coords(rocobj, x=c(0.1, 0.5, 0.9), input = "sensitivity", ret=c("specificity", "ppv", "tp"))

    # With a smoothed roc:
    rocobj <- roc(aSAH$outcome, aSAH$s100b)
    ci.coords(smooth(rocobj), x=0.9, input = "sensitivity", ret=c("specificity", "ppv", "tp"))

    # Return everything we can:
    rets <- c("threshold", "specificity", "sensitivity", "accuracy", "tn", "tp", "fn", "fp", "npv",
              "ppv", "1-specificity", "1-sensitivity", "1-accuracy", "1-npv", "1-ppv")
    ci.coords(rocobj, x="best", input = "threshold", ret=rets)

    ## End(Not run)

[Package pROC version 1.12.1 Index]