tweakstreet-logo
  • Enter search terms

XML Input

Parses XML and puts elements in rows

Processing

For each input row, the step parses an XML payload and generates parsing events as rows.

The step supports low-level parsing events such as begin element and end element. In addition the step supports tree events, which generate a DOM-style data structure for a sub-tree in the XML.

Settings

Name Type Description
General
XML Source
 string

Source of XML Text.

  • file - read from file on filesystem
  • setting - use text in XML setting

Evaluated for each input row
Filesystem
 dict

Only used if XML Source is set to file

The filesystem to use.

If the filesystem is nil, the local filesystem is used.

Evaluated for each input row
File
 string

Only used if XML Source is set to file

The path of the XML file to read.

Relative paths are interpreted as relative to the flow file.

Evaluated for each input row
XML
 string

Only used if XML Source is set to setting

The XML text to parse.

Evaluated for each input row
Generate Events for
 list

A list governing which tags to generate events for - a whitelist.

If empty or nil events are generated for all tags.

If non-empty, the list contains matcher strings. At least one of the strings in the list must match a tag in order for the tag to generate events.

Matching of matcher string m and a tag is performed as follows:

  • if m is equal to the tag’s local tag name, it matches
  • if m is equal to the tag’s namespace URI, it matches
  • m does not match the tag otherwise

Evaluated for each input row
Omit Events for
 list

A list governing which tags to not generate events for - a blacklist.

Only applied if a tag has successfully passed the ‘Generate Events for’ whitelist.

If empty or nil no tag is excluded.

If non-empty, the list contains matcher strings. If any of the strings in the list match a tag it does not generate events.

Matching of matcher string m and a tag is performed as follows:

  • if m is equal to the tag’s local tag name, it matches
  • if m is equal to the tag’s namespace URI, it matches
  • m does not match the tag otherwise

Evaluated for each input row
Trim text nodes
 boolean

If true, all text node content is trimmed of whitespace on both ends.

Evaluated for each input row
Sub-Trees
Extract Sub-Trees for
 list

A list governing which tags to generate sub-trees for - a whitelist.

Only applied if a tag has successfully passed the ‘Generate Events for’ whitelist and ‘Omit Events for’ blacklist.

If empty or nil no sub-trees are generated.

If non-empty, the list contains matcher strings. At least one of the strings in the list must match a tag in order for the tag to generate a sub-tree.

Matching of matcher string m and a tag is performed as follows:

  • if m is equal to the tag’s local tag name, it matches
  • if m is equal to the tag’s namespace URI, it matches
  • m does not match the tag otherwise

When a sub-tree is generated, it parses the entire content of the element and places it into a tree structure. The contents of a sub-tree are always complete. Contents of a sub-tree are not subject to whitelists/blacklist inclusion/exclusion rules.

Evaluated for each input row
Tree Element Placement
 string

Governs how element names are generated:

  • index by name - child tags are indexed by name into parents, if more than one child of the same name exists, a list is created
  • list of children - all child nodes are placed in a list at :children

Evaluated for each input row
Lists for
 list

Only used if ‘Tree Element Placement’ is set to index by name

A list governing which tags to always place into lists, even if there is only one child.

If non-empty, the list contains matcher strings. At least one of the strings in the list must match a tag in order for the tag to always be placed into a list, even if there is only one instance.

Matching of matcher string m and a tag is performed as follows:

  • if m is equal to the tag’s local tag name, it matches
  • if m is equal to the tag’s namespace URI, it matches
  • m does not match the tag otherwise

Example use:

When working with a structure like:

  <library>
    <book></book>
    <book></book>
  </library>

it is convenient to always put book tags into lists. Doing so ensures that both single-book and multi-book libraries always parse to a structure like this:

{:name 'library' :book [{...}, ...]}

Otherwise single-book libraries will generate a structure just containing the single book item directly under :book

{:name 'library' :book {...}}

Evaluated for each input row
Output Structure
Element Names
 string

Governs how element names are generated:

  • name - local name of element is used
  • namespace_uri:name - namespace uri - if applicable - is prepended to local element names
  • namespace_prefix:name - namespace prefix - if present - is prepended to local element names

Evaluated for each input row
Attribute Names
 string

Governs how attribute names are generated:

  • name - local name of attribute is used
  • namespace_uri:name - namespace uri - if applicable - is prepended to local attribute names
  • namespace_prefix:name - namespace prefix - if present - is prepended to local attribute names

Evaluated for each input row
Attribute Placement
 string

Governs how element names are generated:

  • index by name - attributes are indexed by name into parents
  • index by @name - attributes are indexed by name prefixed with '@' into parents
  • dict of attributes - attributes are placed in a dict at key :attributes

Evaluated for each input row
Text Nodes
 string

Governs how text nodes are processed:

  • attach to element - text content is placed in :text node
  • generate event - text content is not attached to elements, it generates a separate event
  • skip - text content is ignored

This setting does not apply to tree events. Sub-Trees always contain text information in :text keys attached to elements.

This setting is intended for recovering text information from documents with rich-text structure such as <b>hello <i>world</i></b>.

Evaluated for each input row
Misc Events
Start Document
 string

Governs whether to generate a start document event:

  • generate event - a start document event is generated
  • skip - no start document event is generated

Evaluated for each input row
End Document
 string

Governs whether to generate an end document event:

  • generate event - an end document event is generated
  • skip - no end document event is generated

Evaluated for each input row
End Element
 string

Governs whether to generate end element events:

  • generate event - end element events are generated
  • skip - no end element events are generated

Evaluated for each input row

Results

Name Type Description
event_type
 string

The type of event emitted:

  • start_document
  • end_document
  • start_element
  • end_element
  • text
  • tree
event
 dict

A dict with structure dependent on event type

Event Type Value
start_document {}
end_document {}
start_element {:name <name of element>, ... } additional keys depend on configuration of how attributes and text nodes are indexed in
end_element identical to start_element
text {:text string}
tree {:name <name of element>, :tree {...}} tree structure depends on configuration of how attributes and children are indexed in
parent
 dict The parent element’s start_element value or nil if parent element did not generate a start_element event.
ancestors
 list All parent elements' start_element values or [] if no parent elements generated start_element events. The closest parent is at the beginning of the list.
On this page