Settype Transform

Tips:

• When a column is set to a data type, all values in the column are validated against the new type, which might change the number of mismatched values. Some cleanup might be required. Some operations might cause the data type to be re-validated automatically.
• It might be easier to set type using the column's drop-down. Selections of data type from the column drop-down are turned into recipe steps using the settype transform.
• If you encounter a significant number of mismatches after you change the data type, you might find it helpful to change or revert the type to String. All data can be interpreted as a String or a list of string values. The transforms and functions for manipulating String data might be easier to use to clean up mismatched data before changing the data type to the preferred one.
• Row values that do not match the new data type might be turned to null values during job execution.

You can also specify any custom data types that have been defined.

Basic Usage

Single-column example:

settype col: Score type: 'Integer'

Output: Changes the data type for the Score column to Integer. 

Multi-column example:

settype col: Score,studentId type: 'Integer'

Output: Changes the data type for the Score and studentId columns to Integer. 

Syntax and Parameters

settype col:col1,col2 type:'string_literal'

For more information on syntax standards, see Language Documentation Syntax Notes.

col

Identifies the column(s) to which to apply the transform. You can specify one or more columns.

Usage Notes:

type

settype col: zips type:'Zipcode'

Output: Changes the data type of the zips column to Zip Code data type. All values are validated as U.S. Zip code. 

Usage Notes:

Examples

Example - Simple settype with date values

Source:

Here is a list of activities listed by date. Note the variation in date values, including what is clearly an invalid date. Here is the source data:

myDate, myAction
4/4/2016,Woke up at 6:30
4-4-2016,Got ready
9-9-9999,Drove kids to school
4-4-2016, Commuted to work

Transform:

When this data is imported into the Transformer page, there are couple of immediate issues: no column headings and blank rows at the bottom. These two transforms fix that:

header

delete row: ISMISSING([myDate])

For the invalid date, you can infer from the rows around it that it should be from the same date. You can make the following change to fix it:

replace col: myDate on: `9-9-9999` with: '4-4-2016' global: true

Now that the dates look fairly consistent, you can set the data type of the column to a matching Datetime format:

settype col: myDate type: 'Datetime','mm-dd-yy','mm*dd*yyyy'

Note the syntax above for specifying Datetime types. In addition to the Datetime keyword, you must specify the format type, followed by the variation of that format. 

Results:

Example - Use merge and settype to clean up numeric data that should be treated as other data types