...

• User Interface: Depending on the product, users interact through one of the available interfaces through a secure connection with the platform. 

• D s item
  item Services

  : The following services interact with flow and dataset metadata in the

  D s item
  item database

  , a proprietary database.
  • Compiler: The

    D s lang

     compiler renders the steps created or selected by the user into a sequence of commands that can be executed against the specified target. See Wrangle Execution Layers.
  • UDF: As needed, developers can create user-defined functions (UDFs), which can be made available to end-users through the UI. For more information, see User-Defined Functions.
  • Scheduling: The Scheduling service enables management of asynchronous execution of jobs on a cron-based schedule. . For more information, see Overview of Scheduling.
  • Machine Learning: The executed

    D s lang

     recipes are analyzed for patterns, which provide more focused recommendations to end-users in the environment. 
  • Metadata Server: This server manages the read/write operations of the

    D s item
    item database

    .
• Workload Management: This layer, which includes job execution and monitoring functions, is managed by Batch Job Runner, a proprietary service of the

  D s platform

  . 
• Execution Environment: The

  D s platform

   can execute

  D s lang

  recipes through a variety of running environments.
• Storage: The unified storage layer provides brokerage services for read/write operations against HDFS, Hive, S3, and other clustered storage.