As needed, you can deploy custom data types into the , in which type validation is performed against regular expressions that you specify. This method is most useful for validating against patterns, as opposed to specific values.

Custom Types Location

On the server hosting the , type definitions are stored in the following directory:

/opt/trifacta/node_modules/jsdata/type-packs/trifacta

This directory is referenced as $CUSTOM_TYPE_DIR in the steps below.

Directory contents:

• The dictionaries sub-directory contains user-defined dictionaries.

  NOTE: Please use the user interface to interact with your dictionaries. See Custom Type Dialog.

  
  

• The types sub-directory contains individual custom data type definitions, each in a separate file. 

• The manifest.json file contains a JSON manifest of all of standard and custom types in the system.

Examples

Each custom data type is created and stored in a separate file. The following example file contains a regular expression method for validating data against the set of days of the week:

{
  "name": "DayOfWeek",
  "prettyName": "Day of Week",
  "category" : "Date/Time",
  "defaultProbability": 1E-15,
  "testCase": {
    "stripWhitespace": true,
    "regexes": [
      "^(monday|tuesday|wednesday|thursday|friday|saturday|sunday)$",
      "^(mon|tue|wed|thu|fri|sat|sun)$"
    ],
    "probability": 0.001
  }
} 

Parameters:

Defining probabilities

For your custom type, the probability values are used to determine the likelihood that matching values indicate that the entire column is of the custom data type.

• The defaultProbability value specifies the baseline probability that a match between a value and one of the regular expressions indicates that the column is the specified type. On a logarithmic scale, values are typically 1E-15 to 1E-20.
• When a value is matched to one of the regular expressions, the probability value is used to increment the baseline probability that the next matching value is of the specified type. This value should also be expressed on a logarithmic scale (e.g. 0.001).
• In this manner, a higher number of matching values increases the probability that the type is also a match to the custom type.

Probabilities become important primarily if you are creating a custom type that is a subset of an existing type. For example, the Email Address custom type is a subset of String type. So, matches for the patterns expressed in the Email Address definition should register a higher probability value than the same incremental for the String type definition.

Add custom types to manifest

To the $CUSTOM_TYPE_DIR/manifest.json file, you must add the filenames of any custom types that you have created and stored in the types directory:

{
  "types": ["bodies-of-water.json", "dayofweek.json"],
  "dictionaries": ["oceans", "seas"]
} 

Enable custom types

Steps:

1. 

2. Locate the following property:
   
   

   "feature.enableCustomTypes": true,

   
   

3. To enable use of your custom data types in the , locate and edit enabledSemanticTypes property.
   
   

   NOTE: Add your entries to the items that are already present in enabledSemanticTypes. Do not delete and replace entries.

   
   

   NOTE: Do not use this parameter to attempt to remove specific data types. Removal of the default types is not supported.

   
   

   "webapp.enabledSemanticTypes": [ "<CustomTypeName1>", "<CustomTypeName2>", "<CustomTypeNameN>" ]

   where:

   • <CustomTypeName1> corresponds to the internal name value for your custom data type.

   
   
   

4. Save your changes and restart the platform.

Enable custom types

To enable use of your custom data types in the , locate and edit enabledSemanticTypes property.

"webapp.enabledSemanticTypes": [
    "<CustomTypeName1>",
    "<CustomTypeName2>",
    "<CustomTypeNameN>"
]

where:

• <CustomTypeName1> corresponds to the internal name value for your custom data type.

Register your custom types

To add your custom types to the , run the following command from the js-data directory: 

node bin/load-types --manifest ${PATH_TO_MANIFEST_FILE} 

Restart platform

Restart services. See Start and Stop the Platform.

Check for the availability of your types in the column drop-down. See Column Menus.