EXAMPLE - Type Functions

This example illustrates how various type checking functions can be applied to your data.

• ISVALID - Returns true if the input matches the specified data type. See VALID Function.
• ISMISMATCHED - Returns true if the input does not match the specified data type. See ISMISMATCHED Function.
• ISMISSING - Returns true if the input value is missing. See ISMISSING Function.
• ISNULL - Returns true if the input value is null. See ISNULL Function.
• NULL - Generates a null value. See NULL Function.

Source:

Some source values that should match the State and Integer data types:

Transform:

You can test for invalid values for State using the following:

derive type:single value: ISMISMATCHED (State, 'State')

You can test for valid matches for Qty using the following:

derive type:single value: (ISVALID (Qty, 'Integer') && (Qty > 0)) as:'valid_Qty'

The first transform flags rows 4 and 6 as mismatched.

The second transform flags as valid all rows where the Qty column is a valid integer that is greater than zero.

The following transform tests for the presence of missing values in either column:

derive type:single value: (ISMISSING(State) || ISMISSING(Qty)) as:'missing_State_Qty'

After re-organizing the columns using the move transform, the dataset should now look like the following:

Since the data does not contain null values, the following transform generates null values based on the preceding criteria:

derive type:single value: ((mismatched_State == 'true') || (valid_Qty == 'false') || (missing_State_Qty == 'true')) ? NULL() : 'ok' as:'status'

You can then use the ISNULL check to remove the rows that fail the above test:

delete row: ISNULL('status')

Results:

Based on the above tests, the output dataset contains one row: