supports detailed monitoring of a job throughout each phase of its execution.  

Limitations:

Monitoring Phases

These phases apply to ingest and publishing jobs. Information on them is surfaced in the application.

Connect

In the Connect phases,  uses the specified connection for the flow to connect to the source of the job. 

NOTE: Errors in this phase typically involve issues in the connection definition or in the network configuration or availability.


Request

After the platform has been able to connect to the datastore, the Request phase entails the submission of the request to the datastore for the assets. For example, for JDBC-based datasets, this phase covers the SQL query of the database through the response that the query was successfully executed.

NOTE: Errors in this phase typically reflect errors in the SQL query, which can include renaming or moving of assets in the datastore.

Transfer

This phase covers the transfer of assets from the datastore to the platform.

NOTE: Errors in this phase typically indicate issued with permissions.

Prepare

NOTE: This phase applies to publishing jobs only.

Depending on the destination, the Prepare phase includes the creation of temporary tables, generation of manifest files, and the fetching of extra connections for parallel data transfer.

Process

After the data has been transferred to the platform, this phase covers the processing of cleanup after data transfer, including the dropping of temporary tables or copying data within the instance.

Enable

The base feature is enabled by default.

Monitoring Jobs in the Application

When the base feature is enabled, you can monitor jobs in the following locations.

Flow View

Import

Import Data:

For long-loading datasets, you can track the progress of the import through the Import Data page as you specify the import. See Import Data Page.

Library:

After specifying the import, if the data is continuing to be ingested, you can track progress through the Library page. See Library Page.

Dataset Details Page:

In the Dataset Details page, you can monitor the ingest progress. Hover over the Status link.

Dataset Details Page - Job Monitoring

Job Details Page