Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

D toc

Excerpt

This section provides overview information on how to configure the running environments accessible from your deployment of the

D s webapp
.

A running environment is the set of services that are used to execute a job.

  • A job can include tasks to do the following:
    • Ingest data
    • Transform data
    • Profile data
    • Sample data
    • Generate results
  • A running environment can be hosted on the 
    D s node
     or across a cluster that is connected to the product.

D s photon

Hosted on the 

D s node
D s photon
 is an in-memory running environment designed for high performance on small- to medium-sized jobs. 
Configuration:

D s photon
 may require enablement in your project or workspace:




EMR

Amazon Elastic Map Reduce (EMR) is a managed-cluster data platform for processing large volumes of disparate sources of data. This scalable platform is used for running jobs from

D s product
and can handle data processing tasks of any size.

Configuration:

  • EMR is enabled by default and requires no additional configuration.
  • If you are accessing AWS resources using IAM roles, those roles must contain policies to run jobs on EMR. For more information, see Required AWS Account Permissions.