| KL.plugin {entropy} | R Documentation |
KL.plugin computes the Kullback-Leiber (KL) divergence from random variable X_1
to X_2. The corresponding probability mass functions are given by freqs1 and freqs2, and the expectation is computed over freqs1.
chi2.plugin computes the chi-squared statistic between an observed X_1 and an expected X_2, where freqs1 and freqs2 are the corresponding probability mass functions.
KL.plugin(freqs1, freqs2, unit=c("log", "log2", "log10"))
chi2.plugin(freqs1, freqs2, unit=c("log", "log2", "log10"))
freqs1 |
bin frequencies for variable X_1. |
freqs2 |
bin frequencies for variable X_2. |
unit |
the unit in which entropy is measured.
The default is "nats" (natural units). For
computing entropy in "bits" set |
Kullback-Leibler divergence between the from random variable X_1 to X_2 is given as E_{X_1} \log (f(x_1)/f(x_2)) .
The chi-squared statistic is given ∑ (f(x_1)-f(x_2))^2/f(x_2) . It can also be seen as a second-order accurate approximation of twice the KL divergence.
Note that both the KL divergence and the chi-squared statistic are not symmetric in X_1 and X_2.
KL.plugin returns the KL divergence.
chi2.plugin returns the chi-squared statistic.
Korbinian Strimmer (http://strimmerlab.org).
KL.Dirichlet, KL.shrink, KL.empirical, mi.plugin, discretize2d.
# load entropy library
library("entropy")
# probabilities for two random variables
freqs1 = c(1/5, 1/5, 3/5)
freqs2 = c(1/10, 4/10, 1/2)
# KL divergence from X1 to X2
KL.plugin(freqs1, freqs2)
# and corresponding (half) chi-squared statistic
0.5*chi2.plugin(freqs1, freqs2)