tweakstreet-logo
  • Enter search terms

CSV Input

Read rows from a CSV file

Processing

For every input row, the step opens a CSV file and writes the file’s records into the row stream.

Settings

Name Type Description
Filesystem
 dict

The filesystem to use.

If the filesystem is nil, the local filesystem is used.

Evaluated for each input row
File
 string

The path of the CSV file to read.

Relative paths are interpreted as relative to the flow file.

Evaluated for each input row
Compression
 string

Decompresses file using the selected algorithm.

Compression Typical file endings
gzip .z or .gz
bzip2 .bz2 or .bzip2

Evaluated for each input row
Charset
 string

The character encoding of the file.

Evaluated for each input row
Skip Lines
 long

How many lines to skip in the file.

Use this to skip non-csv lines before the actual csv header and data.

Evaluated for each input row
Row Limit
 long

Limit data lines read from the file.

  • when nil, no limit is applied
  • when >=0, this number is the maximum number of data lines read from the file
  • any other value results in an error

Evaluated for each input row
Separator
 string

Character separating CSV fields.

Commonly this is a comma, semicolon, or tab character.

Evaluated for each input row
Enclosed by
 string

Character optionally enclosing CSV fields.

Commonly this is a singlequote or doublequote character.

Evaluated for each input row
Header Line
 boolean

If checked, the first line of the file is interpreted as containing field names. In this case read fields can use both field names and positional indexes in field mappings.

If not checked, the first line of the file is interpreted like a regular data line. In this case read fields can use only positional indexes in field mappings.

Evaluated for each input row

Read fields

Defines how fields are extracted from source fields.

Evaluated for each input row

Type
Data type of the field. Field values are implicitly cast to this type.
Name
Name of the field.
Mapping
Index used to access the data in the source record.
  • Use an integer number n to access the n-th field.
    Fields are indexed starting at 0.
  • Use a string to access a source field by name.
    This is only possible if there is a header line containing field names.
Conversion
Optional function parsing the source field from string to its target type.
  • Use nil to leave the source data as it is. It will be cast to the target type.
  • Supply a function (string x) -> any, the source field value is passed in as x and the return value is used for the output field.

Results

Name Type Description
record
 dict current csv row as a dict of strings
values
 list current csv row as a list of strings
On this page