R: R Interface to NSB Entropy Estimator

entropy.NSB {entropy} | R Documentation

## R Interface to NSB Entropy Estimator

### Description

`entropy.NSB` estimates the Shannon entropy H of the random variable Y from the corresponding observed counts `y` using the method of Nemenman, Shafee and Bialek (2002).

Note that this function is an R interface to the "nsb-entropy" program. Hence, this needs to be installed separately from http://nsb-entropy.sourceforge.net/.

### Usage

    entropy.NSB(y, unit=c("log", "log2", "log10"), CMD="nsb-entropy")

### Arguments

`y` | vector of counts.
`unit` | the unit in which entropy is measured. The default is "nats" (natural units). For computing entropy in "bits" set `unit="log2"`.
`CMD` | path to the "nsb-entropy" executable.

### Details

The NSB estimator is due to Nemenman, Shafee and Bialek (2002). It is a Dirichlet-multinomial entropy estimator, with a hierarchical prior over the Dirichlet pseudocount parameters.

Note that the NSB estimator is not a plug-in estimator, hence there are no explicit underlying bin frequencies.

### Value

`entropy.NSB` returns an estimate of the Shannon entropy.

### Author(s)

Jean Hausser.

### References

Nemenman, I., F. Shafee, and W. Bialek. 2002. Entropy and inference, revisited. In: Dietterich, T., S. Becker, Z. Gharamani, eds. Advances in Neural Information Processing Systems 14: 471-478. Cambridge (Massachusetts): MIT Press.

### See Also

`entropy`, `entropy.shrink`, `entropy.Dirichlet`, `entropy.ChaoShen`.

### Examples

    # load entropy library
    library("entropy")

    # observed counts for each bin
    y = c(4, 2, 3, 0, 2, 4, 0, 0, 2, 1, 1)

    ## Not run:
    # estimate entropy using the NSB method
    entropy.NSB(y) # 2.187774

    ## End(Not run)

    # compare to empirical estimate
    entropy.empirical(y)

[Package entropy version 1.2.1 Index]