tweakstreet-logo
  • Enter search terms

Find Files

Looks for files and returns a list of all found files

Processing

For each input row, the step recursively searches for files and folders inside a given base path.

Settings

Name Type Description
Filesystem
 dict

The filesystem to use.

If the filesystem is nil, the local filesystem is used.

Evaluated for each input row
Base Path
 string

The base path to start looking for files.

Relative paths are interpreted as relative to the flow file.

Evaluated for each input row
Match Files
 boolean

If true, files are matched, if false files are not included in the result.

Evaluated for each input row
Match Folders
 boolean

If true, folders are matched, if false folders are not included in the result.

Evaluated for each input row
Pattern
 string

The pattern to match. Only matching files are included in the result.

The full path of each file system entry is tested.

A pattern can take the following forms:

Type Interpretation
string Regular expression. Matching files are included in the result.
When using the local filesystem, PathMatcher syntax can be used.
  • .*\.xml$ matches file names ending in .xml
  • glob:**/*.csv matches file names ending in csv (using path matcher glob syntax)
nil All found files are included in the result.

Evaluated for each input row
Filter
 function

Function of the form (string path, dict attributes) -> boolean returning whether to include the given path in the result.

The function is invoked with each found file after a successful pattern match, and is intended to filter out files based on additional criteria like file size etc.

If nil, all matching files are included in the result.

Evaluated for each input row
Max Depth
 long

The maximum depth when descending into folders.

If nil, no limit is applied. If the limit is 0, only the base path is considered.

Evaluated for each input row
On Error
 string

Determines the course of action when an I/O error happens during directory traversal:

  • error: throw an error
  • warn: write a warning message to the log and continue
  • ignore: ignore any errors and continue

Evaluated for each input row

Results

Name Type Description
files
 list A list of dicts containing attributes of the found files.

The following attributes are always provided:

Key Type Value
name string file name
path string absolute path
parent string absolute path to parent directory, can be nil for fs root
size long size of file in bytes, can be nil if path is not a file
is_file boolean true if path points to a regular file, false otherwise
is_dir boolean true if path points to a directory, false otherwise
is_symlink boolean true if path is a symbolic link
is_other boolean true if path is not a file or directory (could be a pipe, socket, etc.)
mtime datetime time of last modification, can be nil if file system does not track mtime
ctime datetime time of creation, can be nil if file system does not track ctime

Additional attributes might be present if provided by the underlying filesystem.
paths
 list A list of strings containing the absolute paths of found files.
On this page