A running environment is the set of services that are used to execute a job.
- A job can include tasks to do the following:
- Ingest data
- Transform data
- Profile data
- Sample data
- Generate results
- A running environment can be hosted on the Trifacta node or across a cluster that is connected to the product.
Hosted on the Trifacta node, Trifacta Photon is an in-memory running environment designed for high performance on small- to medium-sized jobs.
Trifacta Photon may require enablement in your project or workspace:
- For more information, see Workspace Settings Page.
Amazon Elastic Map Reduce (EMR) is a managed-cluster data platform for processing large volumes of disparate sources of data. This scalable platform is used for running jobs from Designer Cloud powered by Trifacta Enterprise Edition and can handle data processing tasks of any size.
- The Designer Cloud application must be connected to an EMR cluster. For more information, see Configure for EMR.
- If you are accessing AWS resources using IAM roles, those roles must contain policies to run jobs on EMR. For more information, see Required AWS Account Permissions.
Snowflake provides cloud-based data storage and analytics as a service. If all of your source datasets and outputs are in Snowflake locations and other conditions are met, then the entire execution of the transformations can occur in Snowflake. For more information, see https://www.snowflake.com.
For datasets and outputs that are hosted in Snowflake, you can configure the Designer Cloud application to perform the transformation steps of your job in Snowflake. In this manner, no data needs to be transferred to and from the data warehouse, and performance should be significantly better.
Tip: For jobs that are executed in Snowflake, you can optionally enable the execution of the visual profile in Snowflake, too. Some limitations may apply. This option is enabled for individual flows. For more information, see Flow Optimization Settings Dialog.
NOTE: Snowflake is not a running environment that you explicitly select or specify as part of a job. If all of the requirements are met, then the job is executed in Snowflake. For more information on limitations, see Overview of Job Execution.
For more information, see Snowflake Running Environment.
Other Running Environments
Depending on your deployment of Designer Cloud powered by Trifacta Enterprise Edition, additional running environments may be available. For more information, see Running Environment Options.
This page has no comments.