Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0682

D toc


Operationalization refers to a general class of platform features that enable repeated application of 

D s product
 on production data. Whether deployed in a single flow or across all flows in your environment, operationalization features broaden the scope of wrangled data, simplify job execution, and enable these processes on a repeated or scheduled basis.

In the following sections, you can review short summaries of specific features and explore more detailed information on them.

Single Flow Operations

These features can be applied to individual flows to simplify job execution.


Parameterization enables you to specify parameters that capture variability in your data source paths or names. For example, you can parameterize the names of folders in your filepaths to capture files within multiple folders. Or, you can parameterize your inputs to capture datasets named within a specific time range. Nested folders of data can be parameterized, too.

Parameter types:


NOTE: Some of the following may not be available in your product edition.

Parameter TypeDescription

Use regular expressions or

D s item
in your paths or queries to sources to capture a broader set of inputs.

WildcardReplace parts of your paths or queries with wildcards.
DatetimeYou can specify parameterized Datetime values in one of the supported formats.
VariableVariable values can be specified as overrides during import, job execution, and output.

Parameterization is available for the following:

File systems








Relational sources








NOTE: For relational data, parameterization is applied to custom SQL queries used to import the data. For more information, see Enable Custom SQL Query.

For more information, see Overview of Parameterization.


The scheduling feature, also known as Automator, enables you to schedule the execution of individual flows on a specified frequency. Frequencies can be specified through the 

D s webapp
 through a simple interface or, if needed, in a modified form of cron syntax. 


Tip: Automator  is often used with parameterization to fully automate data preparation processes in

D s product

For more information, see Overview of Automator.

Job Monitoring

After a job has been launched, detailed monitoring permits you to track the progress of your job during all phases of execution. Status, job stats, inputs, outputs and a flow snapshot are available through the

D s webapp
. For more information, see Overview of Job Monitoring.


Orchestration is a set of functionality that supports the automated execution of jobs across multiple flows. These jobs could be external processes, other flows, or even HTTP requests.


Webhook notifications let you define outgoing HTTP messages to any REST API. The message form and body can be customized to include job execution metadata. For more information, see Create Flow Webhook Task.

Deployment Manager

The Deployment Manager is a separate environment that can be enabled for the execution of production flows under limited access. Flows in development are exported from your default (Dev) instance and then imported to the Production instance, the Deployment Manager, where you can configure the periodic execution of the flow. For more information, see Overview of Deployment Manager.