R: Plug-In Estimator of the Kullback-Leibler divergence and of...

KL.plugin {entropy} | R Documentation

## Plug-In Estimator of the Kullback-Leibler divergence and of the Chi-Squared Statistic

### Description

`KL.plugin` computes the Kullback-Leiber (KL) divergence from random variable X_1 to X_2. The corresponding probability mass functions are given by `freqs1` and `freqs2`, and the expectation is computed over `freqs1`.

`chi2.plugin` computes the chi-squared statistic between an observed X_1 and an expected X_2, where `freqs1` and `freqs2` are the corresponding probability mass functions.

### Usage

    KL.plugin(freqs1, freqs2, unit=c("log", "log2", "log10"))
    chi2.plugin(freqs1, freqs2, unit=c("log", "log2", "log10"))

### Arguments

`freqs1` | bin frequencies for variable X_1.
`freqs2` | bin frequencies for variable X_2.
`unit` | the unit in which entropy is measured. The default is "nats" (natural units). For computing entropy in "bits" set `unit="log2"`.

### Details

Kullback-Leibler divergence between the from random variable X_1 to X_2 is given as E_{X_1} \log (f(x_1)/f(x_2)) .

The chi-squared statistic is given ∑ (f(x_1)-f(x_2))^2/f(x_2) . It can also be seen as a second-order accurate approximation of twice the KL divergence.

Note that both the KL divergence and the chi-squared statistic are not symmetric in X_1 and X_2.

### Value

`KL.plugin` returns the KL divergence.

`chi2.plugin` returns the chi-squared statistic.

### Author(s)

Korbinian Strimmer (http://strimmerlab.org).

### See Also

`KL.Dirichlet`, `KL.shrink`, `KL.empirical`, `mi.plugin`, `discretize2d`.

### Examples

    # load entropy library
    library("entropy")

    # probabilities for two random variables
    freqs1 = c(1/5, 1/5, 3/5)
    freqs2 = c(1/10, 4/10, 1/2)

    # KL divergence from X1 to X2
    KL.plugin(freqs1, freqs2)

    # and corresponding (half) chi-squared statistic
    0.5*chi2.plugin(freqs1, freqs2)

[Package entropy version 1.2.1 Index]