R: Reformat input data for performance evaluation calculation

mmdata {precrec} | R Documentation

## Reformat input data for performance evaluation calculation

### Description

The `mmdata` function takes predicted scores and labels and returns an `mdat` object. The `evalmod` function takes an `mdat` object as input data to calculate evaluation measures.

### Usage

    mmdata(scores, labels, modnames = NULL, dsids = NULL, posclass = NULL,
      na_worst = TRUE, ties_method = "equiv", expd_first = NULL,
      mode = "rocprc", nfold_df = NULL, score_cols = NULL, lab_col = NULL,
      fold_col = NULL, ...)

### Arguments

`scores` | A numeric dataset of predicted scores. It can be a vector, a matrix, an array, a data frame, or a list. The `join_scores` function can be useful to make scores with multiple datasets.
`labels` | A numeric, character, logical, or factor dataset of observed labels. It can be a vector, a matrix, an array, a data frame, or a list. The `join_labels` function can be useful to make labels with multiple datasets.
`modnames` | A character vector for the names of the models. The `evalmod` function automatically generates default names as "m1", "m2", "m3", and so on when it is `NULL`.
`dsids` | A numeric vector for test dataset IDs. The `evalmod` function automatically generates the default ID as `1` when it is `NULL`.
`posclass` | A scalar value to specify the label of positives in `labels`. It must be the same data type as `labels`. For example, `posclass = -1` changes the positive label from `1` to `-1` when `labels` contains `1` and `-1`. The positive label will be automatically detected when `posclass` is `NULL`.
`na_worst` | A Boolean value for controlling the treatment of NAs in `scores`. TRUE All NAs are treated as the worst scores FALSE All NAs are treated as the best scores
`ties_method` | A string for controlling ties in `scores`. "equiv" Ties are equivalently ranked "first" Ties are ranked in an increasing order as appeared "random" Ties are ranked in random order
`expd_first` | A string to indicate which of the two variables - model names or test dataset IDs should be expanded first when they are automatically generated. "modnames" Model names are expanded first. For example, The `mmdata` function generates `modnames` as `c("m1", "m2")` and `dsids` as `c(1, 1)` when two vectors are passed as input, and `modnames` and `dsids` are unspecified. "dsids" Test dataset IDs are expanded first. For example, The `mmdata` function generates `modnames` as `c("m1", "m1")` and `dsids` as `c(1, 2)` when two vectors are passed as input, and `modnames` and `dsids` are unspecified.
`mode` | A string that specifies the types of evaluation measures that the `evalmod` function calculates. "rocprc" ROC and Precision-Recall curves "prcroc" Same as above "basic" Normalized ranks vs. accuracy, error rate, specificity, sensitivity, precision, Matthews correlation coefficient, and F-score. "aucroc" Fast AUC(ROC) calculation with the U statistic
`nfold_df` | A data frame that contains at least one score column, label and fold columns.
`score_cols` | A character/numeric vector that specifies score columns of `nfold_df`.
`lab_col` | A number/string that specifies the label column of `nfold_df`.
`fold_col` | A number/string that specifies the fold column of `nfold_df`.
`...` | Not used by this method.

### Value

The `mmdata` function returns an `mdat` object that contains formatted labels and score ranks. The object can be used as input data for the `evalmod` function.

### See Also

`evalmod` for calculation evaluation measures. `join_scores` and `join_labels` for formatting scores and labels with multiple datasets. `format_nfold` for creating n-fold cross validation dataset from data frame.

### Examples

    ##################################################
    ### Single model & single test dataset
    ###

    ## Load a dataset with 10 positives and 10 negatives
    data(P10N10)

    ## Generate mdat object
    ssmdat1 <- mmdata(P10N10$scores, P10N10$labels)
    ssmdat1
    ssmdat2 <- mmdata(1:8, sample(c(0, 1), 8, replace = TRUE))
    ssmdat2

    ##################################################
    ### Multiple models & single test dataset
    ###

    ## Create sample datasets with 100 positives and 100 negatives
    samps <- create_sim_samples(1, 100, 100, "all")

    ## Multiple models & single test dataset
    msmdat1 <- mmdata(samps[["scores"]], samps[["labels"]],
                      modnames = samps[["modnames"]])
    msmdat1

    ## Use join_scores and join_labels
    s1 <- c(1, 2, 3, 4)
    s2 <- c(5, 6, 7, 8)
    scores <- join_scores(s1, s2)

    l1 <- c(1, 0, 1, 1)
    l2 <- c(1, 0, 1, 1)
    labels <- join_labels(l1, l2)

    msmdat2 <- mmdata(scores, labels, modnames = c("ms1", "ms2"))
    msmdat2

    ##################################################
    ### Single model & multiple test datasets
    ###

    ## Create sample datasets with 100 positives and 100 negatives
    samps <- create_sim_samples(10, 100, 100, "good_er")

    ## Single model & multiple test datasets
    smmdat <- mmdata(samps[["scores"]], samps[["labels"]],
                     modnames = samps[["modnames"]],
                     dsids = samps[["dsids"]])
    smmdat

    ##################################################
    ### Multiple models & multiple test datasets
    ###

    ## Create sample datasets with 100 positives and 100 negatives
    samps <- create_sim_samples(10, 100, 100, "all")

    ## Multiple models & multiple test datasets
    mmmdat <- mmdata(samps[["scores"]], samps[["labels"]],
                     modnames = samps[["modnames"]],
                     dsids = samps[["dsids"]])
    mmmdat

    ##################################################
    ### N-fold cross validation datasets
    ###

    ## Load test data
    data(M2N50F5)
    head(M2N50F5)

    ## Speficy nessesary columns to create mdat
    cvdat1 <- mmdata(nfold_df = M2N50F5, score_cols = c(1, 2),
                     lab_col = 3, fold_col = 4,
                     modnames = c("m1", "m2"), dsids = 1:5)
    cvdat1

    ## Use column names
    cvdat2 <- mmdata(nfold_df = M2N50F5, score_cols = c("score1", "score2"),
                     lab_col = "label", fold_col = "fold",
                     modnames = c("m1", "m2"), dsids = 1:5)
    cvdat2

[Package precrec version 0.9.1 Index]