tweakstreet-logo
  • Enter search terms

Python

Calls a Python function

Processing

For every input row the step passes a value into a Python function and makes the function’s return value available in the ret result value.

Prerequisites

You need a local installation of Python 3.5 or higher.

Python module

The module must define a process(input) function. This function is called for every input row and the configured input value is passed in. The return value is made available as the ret result.

If process(input) raises an exception, the step generates an error with the corresponding error message.

Data Conversion

Data going into Python is converted as follows:

Tweakstreet Type Python Type
nil None
string string
long int
double float
decimal decimal
boolean bool
datetime datetime
binary bytes
list list
dict dict

Data returned by Python into Tweakstreet is converted as follows:

Python Type Tweakstreet Type
None nil
string string
int long or decimal for large values
float double
decimal decimal
bool boolean
datetime datetime
bytes / bytearray binary
list list
dict dict

Other Python Objects

Object types not mentioned in the above table are pickled and converted to Tweakstreet values on a best-effort basis reflecting the structure of their pickled form.

Lifecycle

The Python module can optionally define an init() function. This function is called just before the first row is processed. It is useful for doing one time setup like opening files, connecting to external services, etc.

If init() raises an exception, the optional shutdown() function is called immediately and the step generates an error. The call to process(input) is not made. Any subsequent row will trigger another call to init(), trying to initialize the module.

The module can optionally define a shutdown() function. This function is called after the last row was processed. It is useful for doing cleanup like flushing buffers, closing files, disconnecting cleanly from external services, etc.

If shutdown() raises an exception it is logged but otherwise ignored.

A Python module declaring all lifecycle functions looks like this:

def init():
  print("init")

def process(input):
  print("processing row")

def shutdown():
  print("shutdown")

Settings

Name Type Description
General
Input
 any

The value that is passed as an argument in the call to process(input)

Evaluated for each input row
Source

How the Python module is specified:

  • inline
    The Python module source is specified directly in the step configuration.

  • file
    The Python module is an external file.

Specified at design time
Script
 string

If source is inline, this is the source of the Python module used to process data.

Evaluated for each input row
Script File
 string

If source is file, this is the path to the Python file used to process data.

Evaluated for each input row
Data Exchange
Datetime Encoding
 string

How datetime values are passed to Python:

  • naive
    Timezone information is omitted. The Python datetime is naive.

  • naive at UTC
    The Python datetime is naive. It holds the local time at UTC.

  • datetime.timezone
    The Python datetime is aware. It contains a fixed offset from UTC as implemented by datetime.timezone in the Python standard library.

  • pytz.timezone
    The Python datetime is aware. It contains a political timezone as implemented by pytz. Your Python installation needs the pytz package installed. For fixed- offsets zones this mode may still produce datetimes localized by datetime.timezone.

Evaluated once when the step initializes
Default Timezone
 string

The timezone used for naive datetime values returned by Python.

Evaluated once when the step initializes
Python
Python Executable
 string

The Python executable used to run the module. Useful if you have multiple Python environments.

If this is nil or empty, Tweakstret runs python and expects to find it on PATH.

  • Specify an executable name - such as python3 - to look for a different binary name on PATH
  • Specify an absolute path to use a specific Python binary
  • Specify a relative path starting with . to point to a Python binary relative to the flow file

Evaluated once when the step initializes

Results

Name Type Description
ret
 any The return value received by calling process(input)

Notes

Tweakstreet runs a Python process, opens a local socket and exchanges data with the Python process using that socket. If you see communication problems between Tweakstreet and Python in the log, make sure no firewall rules block Tweakstreet and Python from connecting to each other.

On this page