Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r100

...

Excerpt

In the Job History page, you can track the status of all of your flow and sample jobs and plan runs.

  • Flow jobs: These jobs execute transformations on your source data to generate outputs.  

  • Sample jobs:
  •  
  • For more information on sampling jobs, see Sample Jobs Page.
  • Plan runs: These jobs execute a sequence of tasks, which can include flow jobs. For more information, see Plan Runs Page.

...

  • Run by me: Jobs that you have initiated for the filtered time period are listed here. 
  • All jobs: All jobs that were initiated for assets that you own are listed here. For example, if a collaborator has run a job on a flow that an asset that you own, you can review its status here. 

...

  • Filter by status: Click one of the tabs to filter the display to show only the listings for the selected job status.
  • Filter by type and date: Click the Funnel icon to filter the list of jobs by source of execution, date range, or both. See below.
  • Search: Enter text in the search field to filter the listed jobs by job ID, flow nameasset name, or dataset name. 

Context menu:

...

Each job listed in the Job History page is a grouping of related jobs acting on the same recipe and dataset(s). Each of these these jobgroups breaks down into one or more of the following job types.

Tip

Tip: To review the status of individual jobs within a jobgroup, hover over the icons in the Status column for the jobgroup.

...

  • Pre-ingest SQL: These jobs are SQL scripts that execute before the source data is ingested to the platform.

  • Transform: These These jobs perform transformations on imported datasets based on the recipe transformations from which the job was launched.
  • Profile: If enabled as part of the job definition, a Profile job generates a visual summary of the results of your transformation transform job. 
    • Profiling jobs may take longer than transformation jobstransform jobs.
    • Even when selected, profiling jobs may not appear in the Job History page. In some cases, a profiling job may be folded into a transform job transform job for optimization reasons.

      Info

      NOTE: When the profiling job is run as part of the transform job, there is no listing for profiling in the mouse-over popup.

    • See Job Details Page.
  • Publish: Depending on multiple factors, your job may include a second Publish job that occurs after the Transform transform job. For example, job groups can include internal Publish jobs for writing results to the designated location in the base storage layer.

    Publishing can also be executed as a separate, post-execution job. As needed, job results can be published from their target location to another location or data store. These jobs are tracked separately as Publish jobs and can be launched from the Job Details page. 

  • Ingest: For larger datasets from some relational connections, the platform transfers the data from the source to the default storage layer for faster processing. These ingest jobs occur before any transform or profiling takes place.

...

  • All jobs: All jobs that you have initiated or were run on assets that you own.
  • Completed: Job has successfully executed.

Info

NOTE: Invalid steps in a recipe are skipped, and it's still possible for the job to be executed successfully.

Info

NOTE: A warning icon may indicate that recipe errors were detected during the

...

transform phase. You can hover over the icon for more information.


  • Failed: job failed to complete. 

Info

NOTE: You can re-run a failed job

...

. If you have since modified the recipe, those changes are applied during the second run. 

  • Publish Failed: Some failed jobs may be listed under this status, which means that the publishing step of the configured job failed to complete. 
  • Canceled:  Job was canceled.


  • Running: Job is in progress.
  • Queued: Job has been queued for execution.

...

An administrator can change the default number of days that are displayed. For more information, see Dataprep Project Settings Page.

D s also
inCQLtrue
label((label = "job_ui") OR (label = "job"))