Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

After you select the transformation to apply, all relevant parameters can be configured through selection or type-ahead fields, so that you can choose from only the elements that are appropriate for the selected transformation.

Tip

Tip: When you add, edit, or delete a recipe step, your changes to the recipe are automatically saved.

To open the Transform Builder, begin creating a step through one of the following methods:

...


  • Metadata. You can insert special strings that evaluate to references of your dataset's metadata. For more information, see Source Metadata References.

...

  • Multiple: Select one or more discrete columns from the drop-down list.
  • All: Select all columns in the dataset.
  • Range: Specify a start column and ending column. All columns inclusive are selected.
  • Advanced: Specify the columns using a comma-separated list. You can combine multiple and range options under Advanced. Ranges of columns can be specified using the tilde (~) character. Example:

    Code Block
    Store_Nbr, Item_Nbr, WM_Week~POS_Cost


...

For some transforms, you can specify patterns to identify conditions or elements of the data on which to take action. These matching patterns can be specified using one of the following types. 

Pattern TypeDescriptionExample
Literal value An exact string or value.

The following matches on the exact value between the quotes:

  'This is what I want to match.'

d-s-

lang

item

item

pattern
pattern

D s product
supports a variety of macro-like pattern identifiers, which can be used in place of more complex regular expressions.

The following matches when two digits appear at the beginning of a value:

`{start}{digit} {digit}`

Regular expression pattern

Regular expressions are a standard method of describing matching patterns.

Info

NOTE: The syntax of regular expressions can be complex and can lead to unexpected results if they are improperly specified. Regex is considered a developer-level skill.


The following matches on all numerical values from 0 to 99:

/^\d$|^\d\d$/

For more information on pattern-based matching, see Text Matching.

Flow parameter: You can also insert a flow parameter into your pattern-based inputs in the Transform Builder. To reference a flow parameter, click the Parameterize icon above any field that accepts pattern-based inputs.

...

.

...

Delimiter Groups

In the Transform Builder, transforms that require delimiter are organized into delimiter groups, so that you specify only the elements of a pattern that work together. Delimiter groups apply to the following transforms:

...

Delimiter groups are listed below. 

Delimiter groupDescription
On delimiterTransformation is applied based on a specific literal or pattern.
Between delimitersTransformation is applied on database between two literal or pattern-based delimiters. Details are below.
On multiple delimiters

Transformation is applied based on a sequence of delimiters. An individual pattern can be a string literal, 

d-s-

lang

item
itempattern
, or regular expression, and the sequence can contain combinations of these pattern types.

Between positionsTransformation is applied based on a starting index position and an ending index position. Index positions start from 0 on the left side of any cell value.
On positionsTransformation is applied based on a sequence of listed index positions. Index positions start from 0 on the left side of any cell value.
At regular intervalTransformation is applied at every nth position. Index positions start from 0 on the left side of any cell value.

For more information on the underlying syntax for delimiter groups, see Pattern Clause Position Matching.

...

Each delimiter can either include or exclude the matching value:

Transform Builder optionInclude as part of transformInclude/Exclude
Start delimiterfalseExcludes sub-pattern
Start delimitertrueIncludes sub-pattern
End delimiterfalseExcludes sub-pattern
End delimitertrueIncludes sub-pattern

Condition

condition is an expression that yields a true or false value. A condition may include all of the elements of a formula. This value determines whether the transformation is applied to the evaluated row.

...

A number of transforms support the following parameters.

D s ordering

Group parameter: For transforms that aggregate data, such as pivot or window, you can specify the column by which you wish to group the computed aggregations. In the following example, all values in the POS_Sales column are summed up for each value in the Store_Nbr column.

...

D trans
p03Valuesum(Sales)
Typestep
p01NameRow labels
p01ValueStore_Nbr
p02NameColumn labels
p02ValuecontractDate
p03NameValues
SearchTermPivot columns
The output can always be ordered using the sort transform. See Sort Transform.New Column Name parameter: For transforms that generate new columns, such as derive and extract, you can optionally specify the name of the new column, which saves adding a step to rename it. In the following example, the values of colA and colB are summed and written to the new column colC:

...

After you have added a step, you can modify it as needed. In the Recipe panel, select the Pencil icon next to the recipe step. The step is displayed for editing in the Transform Builder. D s alsoinCQLtruelabel(label = "transformation_ui") OR (label = "transform_ui")