Transformation by Example (TBE) enables you to build recipe objects by mapping example output values for source values. then interprets the differences between the inputs and outputs to determine the transformation required to map them.
TBE leverages pattern-based matching and predictive transformation to derive transformations. When you provide explicit mappings of input value to output, the mapping is passed through predictive transformation to determine the best possible matching pattern.
Tip: TBE simplifies the process of defining patterns to match all values in your source column. Since you know and can specify the exact desired output, you can leave the details of defining the pattern or patterns required to match input to output to the product.
Transformation by Example works well in the following use cases:
Your data has special-case exceptions that must be transformed.
Tip: You can use this feature as a final cleanup for other transformations. If you have a transformation that handles 90% of the cases in a column, you can use this transformation to handle the remainder.
When a TBE step is added to your recipe, the number of individual changes can be many megabytes of data. Instead of storing these objects within the recipe definition, they are stored as a set of artifacts in the artifact storage database and referenced from the recipe.
These artifacts must be stored in a for the step to be editable and exportable.
NOTE: If the artifact storage service is disabled, this feature is unusable.
artifact.datafile is included as part of the export. This file must be imported with the flow definition, or the TBE step in the imported flow is broken. For more information, see Export Flow.
You cannot use multi-value inputs, such as Arrays or Objects, or use the feature to create them.
Tip: If you have Array or Object input columns, convert them to String type before using TBE.
Arithmetic operations or other numeric functions are not supported.
This feature can be enabled and disabled through the Settings page in the Admin console.
Locate the following setting:
Set this value to
In column-by-example transformations, you create a new column from an existing one by mapping input to output values.
For more information, see Create Column by Example.
Column-by-example also works on Datetime columns. When you use a Datetime column as your input, you specify the output values in the date/time format that you wish to use. That input value and all similarly formatted inputs should be converted to the output format. You can then specify additional example outputs for input values in a different format to standardize all of the values in the output column.
NOTE: For Datetime formatting to work properly, the input column must be specified as Datetime data type.
For string-based inputs, the following options in may assist in performing the same functions that you are trying to do in TBE:
|Extract Transform||You can use the extract transform to retrieve sub-strings from a column and insert into a new column.|
supports a variety of string manipulation functions, which can be used to gather data from a string.