A Transformer is an abstraction of direct data processing. It consumes a DataFrame and produces a DataFrame.

Transformers can be executed using a Transform operation.

Transformer usage diagram Transformer usage diagram


A Tokenize is an operation that outputs a transformed DataFrame on its left output port and a StringTokenizer (a Transformer) on its right output port. Passing a Transformer to a Transform operation allows to perform the StringTokenizer on another DataFrame.

transformer example