Overview of Operationalization
Operationalization refers to a general class of platform features that enable repeated application of Designer Cloud Powered by Trifacta Enterprise Edition on production data. Operationalization features broaden the scope of wrangled data, simplify job execution, and enable these processes on a repeated or scheduled basis.
In the following sections, you can review short summaries of specific features and explore more detailed information on them. These features can be applied to individual flows to simplify job execution.
Parameterization
Parameterization enables you to specify parameters that capture variability in your data source paths or names. For example, you can parameterize the names of folders in your filepaths to capture files within multiple folders. Or, you can parameterize your inputs to capture datasets named within a specific time range. Nested folders of data can be parameterized, too.
Parameter types:
dataset parameters: Parameterize the input paths to your data, allowing you to process data in parallel files and tables through the same flow.
output parameters: Parameterize the output paths for your results.
flow parameters: Define parameters that can be applied in your flows, including recipe steps.
Tip
You can apply overrides to any parameter at the flow level. These parameter override values are applied to any parameter that is referenced within the flow for any supported parameter type.
Parameter formats:
Note
Some of the following may not be available in your product edition.
Parameter Type | Description |
|---|---|
Pattern | Use regular expressions or Wrangle in your paths or queries to sources to capture a broader set of inputs. |
Wildcard | Replace parts of your paths or queries with wildcards. |
Datetime | You can specify parameterized Datetime values in one of the supported formats. |
Variable | Variable values can be specified as overrides during import, job execution, and output. |
Parameterization is available for the following:
File systems
Input | Output |
|---|---|
Date/time Pattern Variable | Timestamp Variable |
Relational sources
Input | Output |
|---|---|
Timestamp Variable | Timestamp Variable |
Note
For relational data, parameterization is applied to custom SQL queries used to import the data. For more information, see Enable Custom SQL Query.
For more information, see Overview of Parameterization.
Scheduling
The scheduling feature enables you to schedule the execution of individual flows on a specified frequency. Frequencies can be specified through the Trifacta Application through a simple interface or, if needed, in a modified form of cron syntax.
Tip
Scheduling is often used with parameterization to fully automate data preparation processes in Designer Cloud Powered by Trifacta Enterprise Edition.
For more information, see Overview of Scheduling.
Job Monitoring
After a job has been launched, detailed monitoring permits you to track the progress of your job during all phases of execution. Status, job stats, inputs, outputs and a flow snapshot are available through the Trifacta Application. For more information, see Overview of Job Monitoring.
Email Notifications
After a job has completed, you can send email notifications to stakeholders based on the success or failure of the job.
Note
This feature must be enabled. See Workspace Settings Page.
These notifications are defined within Flow View. See Email Notifications Page.
Webhooks
Webhook notifications let you define outgoing HTTP messages to any REST API. The message form and body can be customized to include job execution metadata. For more information, see Create Flow Webhook Task.
Deployment Manager
The Deployment Manager is a separate environment that can be enabled for the execution of production flows under limited access. Flows in development are exported from your default (Dev) instance and then imported to the Production instance, the Deployment Manager, where you can configure the periodic execution of the flow. For more information, see Overview of Deployment Manager.
Plans
A plan is a sequence of tasks executed in the Designer Cloud Powered by Trifacta platform. Plans can be used to build more complex sequences of transformation jobs, including external messaging, and much more. For more information, see Overview of Orchestration.