Package: tokenizers
Type: Package
Title: A Consistent Interface to Tokenize Natural Language Text
Version: 0.1.4
Description: Convert natural language text into tokens. The tokenizers have a
    consistent interface and are compatible with Unicode, thanks to being built
    on the 'stringi' package. Includes tokenizers for shingled n-grams, skip
    n-grams, words, word stems, sentences, paragraphs, characters, lines, and
    regular expressions.
License: MIT + file LICENSE
LazyData: yes
Authors@R: c(person("Lincoln", "Mullen", role = c("aut", "cre"),
        email = "lincoln@lincolnmullen.com"),
        person("Dmitriy", "Selivanov", role = c("ctb"),
        email = "selivanov.dmitriy@gmail.com"))
URL: https://github.com/ropensci/tokenizers
BugReports: https://github.com/ropensci/tokenizers/issues
RoxygenNote: 5.0.1
Depends: R (>= 3.1.3)
Imports: stringi (>= 1.0.1), Rcpp (>= 0.12.3), SnowballC (>= 0.5.1)
LinkingTo: Rcpp
Suggests: testthat, covr, knitr, rmarkdown
VignetteBuilder: knitr
NeedsCompilation: yes
Packaged: 2016-08-29 19:15:04 UTC; lmullen
Author: Lincoln Mullen [aut, cre],
  Dmitriy Selivanov [ctb]
Maintainer: Lincoln Mullen <lincoln@lincolnmullen.com>
Repository: CRAN
Date/Publication: 2016-08-29 22:59:29
Built: R 3.4.1; x86_64-w64-mingw32; 2018-08-02 17:55:20 UTC; windows
Archs: x64
