R: Performance measures.

measures {mlr} | R Documentation

## Performance measures.

### Description

A performance measure is evaluated after a single train/predict step and returns a single number to assess the quality of the prediction (or maybe only the model, think AIC). The measure itself knows whether it wants to be minimized or maximized and for what tasks it is applicable.

All supported measures can be found by listMeasures or as a table in the tutorial appendix: https://mlr.mlr-org.com/articles/tutorial/measures.html.

If you want a measure for a misclassification cost matrix, look at makeCostMeasure. If you want to implement your own measure, look at makeMeasure.

Most measures can directly be accessed via the function named after the scheme measureX (e.g. measureSSE).

For clustering measures, we compact the predicted cluster IDs such that they form a continuous series starting with 1. If this is not the case, some of the measures will generate warnings.

Some measure have parameters. Their defaults are set in the constructor makeMeasure and can be overwritten using setMeasurePars.

### Usage

    featperc

    timetrain

    timepredict

    timeboth

    sse

    measureSSE(truth, response)

    mse

    measureMSE(truth, response)

    rmse

    measureRMSE(truth, response)

    medse

    measureMEDSE(truth, response)

    sae

    measureSAE(truth, response)

    mae

    measureMAE(truth, response)

    medae

    measureMEDAE(truth, response)

    rsq

    measureRSQ(truth, response)

    expvar

    measureEXPVAR(truth, response)

    arsq

    rrse

    measureRRSE(truth, response)

    rae

    measureRAE(truth, response)

    mape

    measureMAPE(truth, response)

    msle

    measureMSLE(truth, response)

    rmsle

    measureRMSLE(truth, response)

    kendalltau

    measureKendallTau(truth, response)

    spearmanrho

    measureSpearmanRho(truth, response)

    mmce

    measureMMCE(truth, response)

    acc

    measureACC(truth, response)

    ber

    measureBER(truth, response)

    multiclass.aunu

    measureAUNU(probabilities, truth)

    multiclass.aunp

    measureAUNP(probabilities, truth)

    multiclass.au1u

    measureAU1U(probabilities, truth)

    multiclass.au1p

    measureAU1P(probabilities, truth)

    multiclass.brier

    measureMulticlassBrier(probabilities, truth)

    logloss

    measureLogloss(probabilities, truth)

    ssr

    measureSSR(probabilities, truth)

    qsr

    measureQSR(probabilities, truth)

    lsr

    measureLSR(probabilities, truth)

    kappa

    measureKAPPA(truth, response)

    wkappa

    measureWKAPPA(truth, response)

    auc

    measureAUC(probabilities, truth, negative, positive)

    brier

    measureBrier(probabilities, truth, negative, positive)

    brier.scaled

    measureBrierScaled(probabilities, truth, negative, positive)

    bac

    measureBAC(truth, response)

    tp

    measureTP(truth, response, positive)

    tn

    measureTN(truth, response, negative)

    fp

    measureFP(truth, response, positive)

    fn

    measureFN(truth, response, negative)

    tpr

    measureTPR(truth, response, positive)

    tnr

    measureTNR(truth, response, negative)

    fpr

    measureFPR(truth, response, negative, positive)

    fnr

    measureFNR(truth, response, negative, positive)

    ppv

    measurePPV(truth, response, positive, probabilities = NULL)

    npv

    measureNPV(truth, response, negative)

    fdr

    measureFDR(truth, response, positive)

    mcc

    measureMCC(truth, response, negative, positive)

    f1

    measureF1(truth, response, positive)

    gmean

    measureGMEAN(truth, response, negative, positive)

    gpr

    measureGPR(truth, response, positive)

    multilabel.hamloss

    measureMultilabelHamloss(truth, response)

    multilabel.subset01

    measureMultilabelSubset01(truth, response)

    multilabel.f1

    measureMultilabelF1(truth, response)

    multilabel.acc

    measureMultilabelACC(truth, response)

    multilabel.ppv

    measureMultilabelPPV(truth, response)

    multilabel.tpr

    measureMultilabelTPR(truth, response)

    cindex

    cindex.uno

    iauc.uno

    ibrier

    meancosts

    mcp

    db

    dunn

    G1

    G2

    silhouette

### Arguments

`truth` | (factor) Vector of the true class.
`response` | (factor) Vector of the predicted class.
`probabilities` | (numeric | matrix) a) For purely binary classification measures: The predicted probabilities for the positive class as a numeric vector. b) For multiclass classification measures: The predicted probabilities for all classes, always as a numeric matrix, where columns are named with class labels.
`negative` | (`character(1)`) The name of the negative class.
`positive` | (`character(1)`) The name of the positive class.

### Format

none

### References

He, H. & Garcia, E. A. (2009) Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9. pp. 1263-1284.

H. Uno et al. On the C-statistics for Evaluating Overall Adequacy of Risk Prediction Procedures with Censored Survival Data Statistics in medicine. 2011;30(10):1105-1117. https://doi.org/10.1002/sim.4154.

H. Uno et al. Evaluating Prediction Rules for T-Year Survivors with Censored Regression Models Journal of the American Statistical Association 102, no. 478 (2007): 527-37. https://www.jstor.org/stable/27639883.

### See Also

Other performance: `ConfusionMatrix`, `calculateConfusionMatrix`, `calculateROCMeasures`, `estimateRelativeOverfitting`, `makeCostMeasure`, `makeCustomResampledMeasure`, `makeMeasure`, `performance`, `setAggregation`, `setMeasurePars`

[Package mlr version 2.16.0 Index]