R: Compare term usage across 2 document groups using the...

dtm_chisq {udpipe} | R Documentation

## Compare term usage across 2 document groups using the Chi-square Test for Count Data

### Description

Perform a `chisq.test` to compare if groups of documents have more prevalence of specific terms.
The function looks to each term in the document term matrix and applies a `chisq.test` comparing the frequency of occurrence of each term compared to the other terms in the document group.

### Usage

    dtm_chisq(dtm, groups, correct = TRUE, ...)

### Arguments

`dtm` | a document term matrix: an object returned by `document_term_matrix`
`groups` | a logical vector with 2 groups (TRUE / FALSE) where the size of the `groups` vector is the same as the number of rows of `dtm` and where element i corresponds row i of `dtm`
`correct` | passed on to `chisq.test`
`...` | further arguments passed on to `chisq.test`

### Value

a data.frame with columns term, chisq, p.value, freq, freq_true, freq_false indicating for each term in the `dtm`, how frequently it occurs in each group, the Chi-Square value and it's corresponding p-value.

### Examples

    data(brussels_reviews_anno)
    ##
    ## Which nouns occur in text containing the term 'centre'
    ##
    x <- subset(brussels_reviews_anno, xpos == "NN" & language == "fr")
    x <- x[, c("doc_id", "lemma")]
    x <- document_term_frequencies(x)
    dtm <- document_term_matrix(x)
    relevant <- dtm_chisq(dtm, groups = dtm[, "centre"] > 0)
    head(relevant, 10)

    ##
    ## Which adjectives occur in text containing the term 'hote'
    ##
    x <- subset(brussels_reviews_anno, xpos == "JJ" & language == "fr")
    x <- x[, c("doc_id", "lemma")]
    x <- document_term_frequencies(x)
    dtm <- document_term_matrix(x)

    group <- subset(brussels_reviews_anno, lemma %in% "hote")
    group <- rownames(dtm) %in% group$doc_id
    relevant <- dtm_chisq(dtm, groups = group)
    head(relevant, 10)

    ## Not run:
    # do not show scientific notation of the p-values
    options(scipen = 100)
    head(relevant, 10)

    ## End(Not run)

[Package udpipe version 0.8.6 Index]