R: Read a CSV or other delimited file with Arrow

read_delim_arrow {arrow}

R Documentation

Read a CSV or other delimited file with Arrow

Description

These functions uses the Arrow C++ CSV reader to read into a data.frame. Arrow C++ options have been mapped to argument names that follow those of readr::read_delim(), and col_select was inspired by vroom::vroom().

Usage

read_delim_arrow(
  file,
  delim = ",",
  quote = "\"",
  escape_double = TRUE,
  escape_backslash = FALSE,
  col_names = TRUE,
  col_select = NULL,
  na = c("", "NA"),
  quoted_na = TRUE,
  skip_empty_rows = TRUE,
  skip = 0L,
  parse_options = NULL,
  convert_options = NULL,
  read_options = NULL,
  as_data_frame = TRUE
)

read_csv_arrow(
  file,
  quote = "\"",
  escape_double = TRUE,
  escape_backslash = FALSE,
  col_names = TRUE,
  col_select = NULL,
  na = c("", "NA"),
  quoted_na = TRUE,
  skip_empty_rows = TRUE,
  skip = 0L,
  parse_options = NULL,
  convert_options = NULL,
  read_options = NULL,
  as_data_frame = TRUE
)

read_tsv_arrow(
  file,
  quote = "\"",
  escape_double = TRUE,
  escape_backslash = FALSE,
  col_names = TRUE,
  col_select = NULL,
  na = c("", "NA"),
  quoted_na = TRUE,
  skip_empty_rows = TRUE,
  skip = 0L,
  parse_options = NULL,
  convert_options = NULL,
  read_options = NULL,
  as_data_frame = TRUE
)

Arguments

`file`	A character file name, `raw` vector, or an Arrow input stream
`delim`	Single character used to separate fields within a record.
`quote`	Single character used to quote strings.
`escape_double`	Does the file escape quotes by doubling them? i.e. If this option is `TRUE`, the value `""""` represents a single quote, `\"`.
`escape_backslash`	Does the file use backslashes to escape special characters? This is more general than `escape_double` as backslashes can be used to escape the delimiter character, the quote character, or to add special characters like `\\n`.
`col_names`	If `TRUE`, the first row of the input will be used as the column names and will not be included in the data frame. If `FALSE`, column names will be generated by Arrow, starting with "f0", "f1", ..., "fN". Alternatively, you can specify a character vector of column names.
`col_select`	A character vector of column names to keep, as in the "select" argument to `data.table::fread()`, or a tidy selection specification of columns, as used in `dplyr::select()`.
`na`	A character vector of strings to interpret as missing values.
`quoted_na`	Should missing values inside quotes be treated as missing values (the default) or strings. (Note that this is different from the the Arrow C++ default for the corresponding convert option, `strings_can_be_null`.)
`skip_empty_rows`	Should blank rows be ignored altogether? If `TRUE`, blank rows will not be represented at all. If `FALSE`, they will be filled with missings.
`skip`	Number of lines to skip before reading data.
`parse_options`	see file reader options. If given, this overrides any parsing options provided in other arguments (e.g. `delim`, `quote`, etc.).
`convert_options`	see file reader options
`read_options`	see file reader options
`as_data_frame`	Should the function return a `data.frame` (default) or an Arrow Table?

Details

read_csv_arrow() and read_tsv_arrow() are wrappers around read_delim_arrow() that specify a delimiter.

Note that not all readr options are currently implemented here. Please file an issue if you encounter one that arrow should support.

If you need to control Arrow-specific reader parameters that don't have an equivalent in readr::read_csv(), you can either provide them in the parse_options, convert_options, or read_options arguments, or you can use CsvTableReader directly for lower-level access.

Value

A data.frame, or a Table if as_data_frame = FALSE.

Examples


  tf <- tempfile()
  on.exit(unlink(tf))
  write.csv(iris, file = tf)
  df <- read_csv_arrow(tf)
  dim(df)
  # Can select columns
  df <- read_csv_arrow(tf, col_select = starts_with("Sepal"))

[Package arrow version 0.17.1 Index]