Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


  • Long or complex recipes
    • Consider breaking recipes into smaller steps. You can change recipes together.
  • Number of columns in your data
    • The entire width of a dataset must be represented in the sample. 
    • Delete unnecessary columns early in your recipe.


      Tip: If your data is sourced in relational systems, you can apply optimizations to your imported datasets to pre-filter out columns in your dataset before they are ingested into the system. See Flow Optimization Settings Dialog.

      You can also use custom SQL statements to collect only the columns that are needed from source tables. See Create Dataset with SQL.

  • Complexity of transformations
    • Transformations that blend datasets (join and union) or that perform complex transformations on your dataset (aggregate, window, pivot, etc.) can be expensive to process. 
    • If your recipe contains too many of them, it can negatively impact job processing. Consider breaking these across multiple recipes instead.


If there are issues with job execution that cannot be resolved by reviewing the job log, workspace administrators can download a support bundle, which contains additional log information from the platform. For more information, see Support Bundle Contents.

Job Heartbeat and Timeouts

You can configure the how the

D s webapp
checks on job execution. 

D s config


These settings configure heartbeat checks for all jobs initiated from the

D s webapp


When enabled, any job that was running when a restart is executed is explicitly marked as Failed.

Default is false.


This setting in minutes defines the maximum permitted time that a job is allowed to run without returning a valid heartbeat check.

Default is 60  minutes.


This setting in minutes defines the maximum permitted time that a job is allowed to run.

The default is 2880  minutes (48 hours).


Interval between heartbeat checks.

Default is 60000  milliseconds (1 minute).

D s photon

D s photon
jobs are executed on the same node as the 
D s platform
and therefore do not need heartbeat checks. Maximum permitted times for these jobs are defined as follows.


When enabled, 

D s photon
jobs are permitted to timeout based on the following setting.


This setting in minutes defines the maximum time that a

D s photon
job is permitted to run.

Running Environments

D children