R: Tokenising, Lemmatising, Tagging and Dependency Parsing...

udpipe_annotate {udpipe} | R Documentation

## Tokenising, Lemmatising, Tagging and Dependency Parsing Annotation of raw text

### Description

Tokenising, Lemmatising, Tagging and Dependency Parsing Annotation of raw text

### Usage

    udpipe_annotate(
      object,
      x,
      doc_id = paste("doc", seq_along(x), sep = ""),
      tokenizer = "tokenizer",
      tagger = c("default", "none"),
      parser = c("default", "none"),
      trace = FALSE,
      ...
    )

### Arguments

`object` | an object of class `udpipe_model` as returned by `udpipe_load_model`
`x` | a character vector in UTF-8 encoding where each element of the character vector contains text which you like to tokenize, tag and perform dependency parsing.
`doc_id` | an identifier of a document with the same length as `x`. This should be a character vector. `doc_id[i]` corresponds to `x[i]`.
`tokenizer` | a character string of length 1, which is either 'tokenizer' (default udpipe tokenisation) or a character string with more complex tokenisation options as specified in https://ufal.mff.cuni.cz/udpipe/1/users-manual in which case `tokenizer` should be a character string where the options are put after each other using the semicolon as separation.
`tagger` | a character string of length 1, which is either 'default' (default udpipe POS tagging and lemmatisation) or 'none' (no POS tagging and lemmatisation needed) or a character string with more complex tagging options as specified in https://ufal.mff.cuni.cz/udpipe/1/users-manual in which case `tagger` should be a character string where the options are put after each other using the semicolon as separation.
`parser` | a character string of length 1, which is either 'default' (default udpipe dependency parsing) or 'none' (no dependency parsing needed) or a character string with more complex parsing options as specified in https://ufal.mff.cuni.cz/udpipe/1/users-manual in which case `parser` should be a character string where the options are put after each other using the semicolon as separation.
`trace` | A non-negative integer indicating to show progress on the annotation. If positive it prints out a message before each `trace` number of elements of `x` for which annotation is to be executed, allowing you to see how much of the text is already annotated. Defaults to FALSE (no progress shown).
`...` | currently not used

### Value

a list with 3 elements

- x: The `x` character vector with text.

- conllu: A character vector of length 1 containing the annotated result of the annotation flow in CONLL-U format. This format is explained at https://universaldependencies.org/format.html

- error: A vector with the same length of `x` containing possible errors when annotating `x`

### References

https://ufal.mff.cuni.cz/udpipe, https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-2364, https://universaldependencies.org/format.html

### See Also

`udpipe_load_model`, `as.data.frame.udpipe_connlu`

### Examples

    model    <- udpipe_download_model(language = "dutch-lassysmall")
    if(!model$download_failed){
    ud_dutch <- udpipe_load_model(model$file_model)

    ## Tokenise, Tag and Dependency Parsing Annotation. Output is in CONLL-U format.
    txt <- c("Dus. Godvermehoeren met pus in alle puisten,
      zei die schele van Van Bukburg en hij had nog gelijk ook.
      Er was toen dat liedje van tietenkonttieten kont tieten kontkontkont,
      maar dat hoefden we geenseens niet te zingen.
      Je kunt zeggen wat je wil van al die gesluierde poezenpas maar d'r kwam wel
      een vleeswarenwinkel onder te voorschijn van heb je me daar nou.

      En zo gaat het maar door.",
      "Wat die ransaap van een academici nou weer in z'n botte pan heb gehaald mag
      Joost in m'n schoen gooien, maar feit staat boven water dat het een gore
      vieze vuile ransaap is.")
    x <- udpipe_annotate(ud_dutch, x = txt)
    cat(x$conllu)
    as.data.frame(x)

    ## Only tokenisation
    x <- udpipe_annotate(ud_dutch, x = txt, tagger = "none", parser = "none")
    as.data.frame(x)

    ## Only tokenisation and POS tagging + lemmatisation, no dependency parsing
    x <- udpipe_annotate(ud_dutch, x = txt, tagger = "default", parser = "none")
    as.data.frame(x)

    ## Only tokenisation and dependency parsing, no POS tagging nor lemmatisation
    x <- udpipe_annotate(ud_dutch, x = txt, tagger = "none", parser = "default")
    as.data.frame(x)

    ## Provide doc_id for joining and identification purpose
    x <- udpipe_annotate(ud_dutch, x = txt, doc_id = c("id1", "feedbackabc"),
                         tagger = "none", parser = "none", trace = TRUE)
    as.data.frame(x)

    ## Mark on encodings: if your data is not in UTF-8 encoding, make sure you convert it to UTF-8
    ## This can be done using iconv as follows for example
    udpipe_annotate(ud_dutch, x = iconv('Ik drink melk bij mijn koffie.', to = "UTF-8"))
    }

    ## cleanup for CRAN only - you probably want to keep your model if you have downloaded it
    if(file.exists(model$file_model)) file.remove(model$file_model)

[Package udpipe version 0.8.6 Index]