Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Through the Flow View page, you can access and manage all objects in your flow. For each imported dataset, recipe, or other object in your flow, you can perform a variety of actions to effectively manage flow development and job execution through a single page.

Warning

If you have enabled deployment management, avoid making changes in Flow View on a Production instance of the platform.

  • Scheduling executions through Flow View in a Prod environment is not supported. Job executions must be executed through the APIs. See API Endpoints.
  • Some Flow View options may not be available in a Prod environment.
  • You should apply changes to your flow in the Dev instance and then re-deploy to Production. For more information, see Overview of Deployment Management.
Info

NOTE: If the displayed flow has been shared with you, some options are not available.

D caption
Flow View page

The imported datasets in the flow or reference datasets added to the flow are listed on the left side of the screen. Associated with each dataset can be one or more recipes, which are used to transform the source data.

...

When a user is given access to a flow, all of the following actions are available to that user, except for editing details and deleting the flow.

Make a copy: Create a copy of the flow for another user.

 

Info

NOTE: The copied flow is independent of the source flow, but the original source datasets are connected.

Export: (Available to flow owner only)Export the flow for archive or transfer. For more information, see Export Flow.

...

  • ReplaceReplace the dataset with a different dataset or reference dataset.
  • Add new Recipe: Add a new recipe for the object. If a recipe already exists for it, this new recipe is created as a branch in the flow.
  • Edit name and description: (Available to flow owner only) Change the name and description for the object.
  • Remove structure: (If applicable) Remove the initial parsing structure. When the structure is removed:
    1. The dataset is converted to a raw dataset. A raw dataset is the source data converted into a flat file format.
    2. All steps to shape the dataset are removed. You must break up columns in manual steps in any recipe created from the object.
    3. See View for Raw Datasets below.
  • Remove from Flow: Remove the dataset from the flow.
    • All dependent flows, outputs, and references are not removed from the flow. You can replace the source for these objects as needed.

      Info

      NOTE: References to the deleted dataset in other flows remain broken until the dataset is replaced.

 

View for Datasets with Parameters

...

D caption
Parameters Panel in Flow View
The above information is useful for reviewing parameters and specifying overrides at execution time.

...

When you select a dataset with parameters in Flow View, you can review the parameters that have been specified for the selected dataset in the right panel.

Image Added

D caption
typefigure
Parameters tab in Flow View
Actions:

  • To edit the parameters for the dataset, select Edit parameters... from the context menu in the right panel.

 

View for Recipes

For each recipe, you can review or edit its steps or create new recipes altogether. You can also create references to the recipe, modify outputs, and create new recipes off of the recipe.

...

  • Size: Total count of columns and data types in the dataset.

 

Target tab

When a target has been assigned for this recipe, you can review its schema information in the Target tab. This tab appears only after a target has been assigned to the recipe.

...

  • Position: Left-to-right position of the column in the target.
  • Name: Name of the column in the target.
  • Type:  
    D s item
    itemdata type
     of the column in the target.

 

View for Outputs

Associated with each recipe is one or more outputs, which are publishing destinations. Through outputs, you can execute and track jobs for the related recipe.

...

Info

NOTE: Flow collaborators cannot modify publishing destinations.

Actions:

 

  • Run Job: Click Run Job to queue for immediate execution a job for the manual destinations. You can track the progress and results of this task through the Jobs tab. 

  • Delete Output: Remove this output from the flow. This operation cannot be undone.
    • Removing an output does not remove the jobs associated with the output. You can continue working with those executed jobs.See Jobs Page

       

  • Edit: Click this link to modify the selected destination's properties.

 

  • Download CLI Script: Select to export the necessary package of files for use by the
    D s item
    itemcommand line interface
     , which enables command-line automation of your data wrangling operations. See CLI for Jobs.

 

Jobs tab

D caption
typefigure
Jobs tab
Each entry in the Jobs tab identifies a job that has been queued for execution. You can track the progress, success, or failure of execution. When a job has finished execution you can review the results. Click the link to the job. For more information, see Job Results Page.Actions:

  • View Results: Click to view the results of your completed job.

    For more information, see Job Results Page.

     

     

  • View steps and dependencies: View steps of the recipe being executed and any dependencies referenced in the recipe.
  • Export Results: Click to export or publish the results from your completed job.

    For more information, see Export Results Window.

     

     

  • Cancel job: Select to cancel a job that is currently being executed.
  • Download Logs: Download the logs for the job. If the job is in progress, log information is likely to be incomplete.

 

View for References

When you select a recipe, you can choose to create a reference dataset off of that recipe. A reference dataset is a dataset that is a reference to the output generated from a recipe contained in another flow. Whenever the upstream recipe and its output data are changed, the results are automatically inherited through the reference to the reference dataset. 

...

  • Add new Recipe: Add a new recipe for the object. If a recipe already exists for it, this new recipe is created as a branch in the flow.
  • Edit name and description: (Available to flow owner only) Change the name and description for the object.
  • Remove from Flow: Remove the dataset from the flow. All dependent flows, outputs, and references are removed from the flow as well.

 

View for Connections

For flows that require connections to source data, you can review the details of the connection, whether you created it or it was shared with you.

D caption
typefigure
Connections view
Key Fields:

  • Connection Type: For more information, see Connection Types .
  • Owner: User that owns the connection. This user can modify connection properties.
  • Server information: You can review information about the source to which the connection links.
  • Shared:
    • Private - Connection is available for use only for specified users of the platform.
    • Public - Connection is available for all users.
    • For more information, see Share Connection Window.

...

See Share Connection Window.

 

View for Reference Datasets

...