R: Map a function over a file by chunks

chunk.map {iotools}

R Documentation

Map a function over a file by chunks

Description

A wrapper around the core iotools functions to easily apply a function over chunks of a large file. Results can be either written to a file or returned as an internal list.

Usage

chunk.map(input, output = NULL, formatter = .default.formatter,
          FUN, key.sep = NULL, max.line = 65536L,
          max.size = 33554432L, output.sep = ",", output.nsep = "\t",
          output.keys = FALSE, skip = 0L, ...)

Arguments

`input`	an input connection or character vector describing a local file.
`output`	an optional output connection or character vector describing a local file. If `NULL`, the results are returned internally as a list.
`formatter`	a function that takes raw input and produces the input given to `FUN`
`FUN`	a user provided function to map over the chunks. The result of FUN is either wrapper in a list item, when `output` is `NULL`, or written to the output file using `as.output`
`key.sep`	optional key separator given to `chunk.reader`
`max.line`	maximum number of lines given to `chunk.reader`
`max.size`	maximum size of a block as given to `read.chunk`
`output.sep`	single character giving the field separator in the output.
`output.nsep`	single character giving the key separator in the output.
`output.keys`	logical. Whether as.output should interpret row names as keys.
`skip`	integer giving the number of lines to strip off the input before reading. Useful when the input contains a row a column headers
`...`	additional parameters to pass to `FUN`

Value

A list of results when output is NULL; otherwise no output is returned.

Author(s)

Taylor Arnold

[Package iotools version 0.3-3 Index]