R: Estimating Entropy From Observed Counts

entropy {entropy} | R Documentation

## Estimating Entropy From Observed Counts

### Description

`entropy` estimates the Shannon entropy H of the random variable Y from the corresponding observed counts `y`.

`freqs` estimates bin frequencies from the counts `y`.

### Usage

    entropy(y, lambda.freqs, method=c("ML", "MM", "Jeffreys", "Laplace", "SG",
        "minimax", "CS", "NSB", "shrink"), unit=c("log", "log2", "log10"), verbose=TRUE, ...)
    freqs(y, lambda.freqs, method=c("ML", "MM", "Jeffreys", "Laplace", "SG",
        "minimax", "CS", "NSB", "shrink"), verbose=TRUE)

### Arguments

`y` | vector of counts.
`method` | the method employed to estimate entropy (see Details).
`unit` | the unit in which entropy is measured. The default is "nats" (natural units). For computing entropy in "bits" set `unit="log2"`.
`lambda.freqs` | shrinkage intensity (for "shrink" option).
`verbose` | verbose option (for "shrink" option).
`...` | option passed on to `entropy.NSB`.

### Details

The `entropy` function allows to estimate entropy from observed counts by a variety of methods:

- `method="ML"`:maximum likelihood, see `entropy.empirical`

- `method="MM"`:bias-corrected maximum likelihood, see `entropy.MillerMadow`

- `method="Jeffreys"`:`entropy.Dirichlet` with `a=1/2`

- `method="Laplace"`:`entropy.Dirichlet` with `a=1`

- `method="SG"`:`entropy.Dirichlet` with `a=a=1/length(y)`

- `method="minimax"`:`entropy.Dirichlet` with `a=sqrt(sum(y))/length(y`

- `method="CS"`:see `entropy.ChaoShen`

- `method="NSB"`:see `entropy.NSB`

- `method="shrink"`:see `entropy.shrink`

The `freqs` function estimates the underlying bin frequencies. Note that estimated frequencies are not available for `method="MM"`, `method="CS"` and `method="NSB"`. In these instances a vector containing NAs is returned.

### Value

`entropy` returns an estimate of the Shannon entropy.

`freqs` returns a vector with estimated bin frequencies (if available).

### Author(s)

Korbinian Strimmer (http://strimmerlab.org).

### See Also

`entropy-package`, `discretize`.

### Examples

    # load entropy library
    library("entropy")

    # observed counts for each bin
    y = c(4, 2, 3, 0, 2, 4, 0, 0, 2, 1, 1)

    entropy(y, method="ML")
    entropy(y, method="MM")
    entropy(y, method="Jeffreys")
    entropy(y, method="Laplace")
    entropy(y, method="SG")
    entropy(y, method="minimax")
    entropy(y, method="CS")
    #entropy(y, method="NSB")
    entropy(y, method="shrink")

[Package entropy version 1.2.1 Index]