makeCustomResampledMeasure {mlr}R Documentation

Construct your own resampled performance measure.

Description

Construct your own performance measure, used after resampling. Note that individual training / test set performance values will be set to 'NA', you only calculate an aggregated value. If you can define a function that makes sense for every single training / test set, implement your own [Measure].

Usage

makeCustomResampledMeasure(measure.id, aggregation.id, minimize = TRUE,
  properties = character(0L), fun, extra.args = list(), best = NULL,
  worst = NULL, measure.name = measure.id,
  aggregation.name = aggregation.id, note = "")

Arguments

measure.id

('character(1)')
Short name of measure.

aggregation.id

('character(1)')
Short name of aggregation.

minimize

('logical(1)')
Should the measure be minimized? Default is 'TRUE'.

properties

([character])
Set of measure properties. For a list of values see [Measure]. Default is 'character(0)'.

fun

('function(task, group, pred, extra.args)')
Calculates performance value from [ResamplePrediction] object. For rare cases you can also use the task, the grouping or the extra arguments 'extra.args'.

'task' ([Task])

The task.

'group' ([factor])

Grouping of resampling iterations. This encodes whether specific iterations 'belong together' (e.g. repeated CV).

'pred' ([Prediction])

Prediction object.

'extra.args' ([list])

See below.

extra.args

([list])
List of extra arguments which will always be passed to 'fun'. Default is empty list.

best

('numeric(1)')
Best obtainable value for measure. Default is -'Inf' or 'Inf', depending on 'minimize'.

worst

('numeric(1)')
Worst obtainable value for measure. Default is 'Inf' or -'Inf', depending on 'minimize'.

measure.name

('character(1)')
Long name of measure. Default is 'measure.id'.

aggregation.name

('character(1)')
Long name of the aggregation. Default is 'aggregation.id'.

note

([character])
Description and additional notes for the measure. Default is “”.

Value

Measure.

See Also

Other performance: ConfusionMatrix, calculateConfusionMatrix, calculateROCMeasures, estimateRelativeOverfitting, makeCostMeasure, makeMeasure, measures, performance, setAggregation, setMeasurePars

[Package mlr version 2.16.0 Index]