Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

D toc
In
D s product
rtrue
, you

Excerpt

You can use the Job Details page to explore details about successful or failed jobs, including outputs,

...

dependency graph, and other metadata. Download results to your local desktop or, if enabled, explore a visual profile of the data in the results for further iteration on your recipe.

Info

NOTE: If the job is executed in an environment other than

D s photon
, the job is queued for execution in the environment. Jobs executed on a remote cluster may incur additional overhead to spin up execution nodes, which is typically within 10-15 seconds. During job execution, the
D s platform
rtrue
observes the job in progress and reports progress as needed back into the application. The
D s platform
 does not control the execution of the job.

Page options:


  • Cancel job: Click this button to cancel your job while it is still in progress. This button may not appear until the queued job has been submitted to the running environment.

    Info

    NOTE: This option may not be available for all running environments. Job cancellation is not supported in high-availability deployments.

  • Download results: Download the generated results to your local desktop.
  • Delete job: Delete the job and its results.

    Warning

    Deleting a job cannot be undone.

  • Download profile as PDF: If visual profiling was enabled for the job, you can download the profile in PDF format.
  • Download profile as JSON: If visual profiling was enabled for the job, you can download a JSON representation of the profile to your desktop.

...

Overview Tab

In the Overview tab, you can review the job status, its sources, and the details of the job run.

Info

NOTE: If your job failed, you may be prompted with an error message indicating a job ID that differs from the listed one. This job ID refers to the sub-job that is part of the job listed in the Job summary.


Image RemovedImage Added

D caption
Overview tab

You can review a snapshot of the results of your job.

Output Data:

The output data section displays a preview of the generated output of your job.

Info

NOTE: This section is not displayed if the job fails.

You can also perform the following:

  • View: If it is present, you can click the View link to view the job results in the datastore where they were written. 

    Info

    NOTE: The View link may not be available for all jobs.

  • Download : If it is present, click the  Download link to download the generated job results to your local desktop.

  • View details: Click View details to view the generated results in the side bar. See the Output Destinations below.

Completed Stages:

This panel provides information on the progress and completion status of each stage of the job execution. 

If present, you can click the Show Warnings link to see any warnings pertaining to recipe errors, including the relevant step number.To review the recipe and dependencies in your job,

...

click View steps and dependencies. See the Dependencies tab below.

  • If you chose to profile results of your job, click Go to View profile to  to review. See Profile tab below.
    • A visual profile provides a graphical snapshot of the results of a successful transformation job for the entire dataset and individual columns in the dataset. 
    • For more information on enabling a visual profile job, see Run Job Page.
    • For more information, see Overview of Visual Profiling.


Publish:

You can also review the outputs generated as a result of your job. 

...

 To review and export any of the generated results, click View

...

all. See Outputs Destinations tab below.

Job summary:

  • Job ID: Unique identifier for the job

  • Job status: Current status of the job:
    • Queued:  Job has been queued for execution.
    • Running: Job is in progress.
    • Completed: Job has successfully executed.

      Info

      NOTE:  Invalid steps in a recipe are skipped, and it's still possible for the job to be executed successfully.

      Canceled: Job was canceled by user

      .

       

    • Failed: Job failed to complete. 

      Info

      NOTE:  You can re-run a failed job from the Transformer page. If you have since modified the recipe, those changes are applied during the second run. See Transformer Page.

  • Flow: Name of the flow from which the job was executed. Click the link to open the flow. See Flow View Page. 

  • Output: Name of the output object that was used to define the generated results. Click the link to open the output. See Flow View Page. 

Execution summary:

  • Job type: The method by which the job was executed:
    • Manual - Job was executed through the application interface.

User: The user who launched the job

Environment: Where applicable, the running environment where the job was executed is displayed.

Start time: Timestamp for when processing began on the job. This value may not correspond to when the job was queued for execution.

Finish time: Timestamp for when processing ended on the job, successful or not

Last update: Timestamp for when the job was last updated

Duration: Elapsed time of job execution

Output Destinations Tab

If the job has successfully completed, you can review the set of generated outputs and export results.

...

For each output, you can do the following:

View details:

...

 View details about the generated output in the side bar.

Download result:

...

 Download the generated output to your local desktop. 

Info

NOTE: Some file formats may not be downloadable to your desktop. See below.

Create imported dataset: Use the generated output to create a new imported dataset for use

...

. See below.

Info

NOTE: This option is not available for all file formats.

Direct file download

Click one of the provided links to download the file through your browser to your local desktop.

...

Optionally, you can turn your generated results into new datasets for datasets for immediate use in the

d-s-productplatform
rtrue
. For the generated output, select select Create imported dataset from  from its context menu.


Info

NOTE: When you create a new dataset from your job results, the file or files that were written to the designated output location are used as the source. Depending on your backend datastore permissions are configured, this location may not be accessible to other users.

...

.


Profile Tab

Review the visual profile of your generated results in the Profile tab. Visual profiling can assist in identifying issues in your dataset that require further attention, including outlier values. 

Info

NOTE: This tab appears only if you selected to profile results in your job definition. See Run Job Page. 


D caption
Profile tab

Download as PDF: Download your visual profile and results of your data quality rules on the entire dataset as a PDF file

Download as JSON: Download your visual profile as a JSON file.

In particular, you should pay attention to the mismatched values and missing values counts, which identify the approximate percentage of affected values across the entire dataset. For more information, see Overview of Visual Profiling. 

Info

NOTE:  The computational cost of generating exact visual profiling measurements on large datasets in interactive visual profiles severely impacts performance. As a result, visual profiles across an entire dataset represent statistically significant approximations.

Info

NOTE:

D s product
treats null values as missing values. Imported values that are null are generated as missing values in job results (represented in the gray bar).

...

 


Tip

Tip: Mouse over the color bars to see counts of values in the category.

...

Depending on the data type of the column, varying information is displayed. For more information, see Column Statistics Reference. 

Tip

Tip: You should review the type information for each column, which is indicated by the icon to the left of the column.

...

Dependency graph Tab

In this tab, you can review a simplified representation of the flow from which what was executed for the job was executed. This flow view displays only the recipes and datasets assets that contributed to the generated results.

Tip

Tip: To open the full flowsource asset, you can click its name in the upper-left corner.

 


Image RemovedImage Added

D caption
Dependencies tabDependency graph tab

Zoom menu:

You can zoom the dependency graph canvas to display areas of interest. 

The zoom control options are available at the top-right corner of the dependency graph canvas. The following are the available zoom options:

Tip

Tip: You can use the keyboard shortcuts listed in the zoom options menu to make quick adjustments to the zoom level.

Zoom in: Zoom in 10% on the canvas to focus on greater detail.

Zoom out: Zoom out 10% from the canvas to see more of it.

Zoom to fit: Change the zoom level to fit all of the objects onto the screen.

25%, 50%, or 100%: Change the zoom level to one of the preset levels.

Recipe actions:

Download recipe: Download the text of the recipe in in 

D s lang
.

Display Display 

D s lang
/natural language: Toggle display of the recipe in raw language or in readable language.

Limitations:

  • You can select only recipes in the graph.
  • Context controls and menus are not available.


D s also
inCQLtrue
label((label = "job_ui") OR (label = "job"))