Page tree

 

Support | BlogContact Us | 844.332.2821

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

D toc

Through the Flow View page, you can access and manage all objects in your flow. For your currently selected each imported dataset, recipe, or wrangled dataset in your flow, you can review its datasets and history information and make changes to it. Select a flow or create a new one from the Flows Page.Image Removedperform a variety of actions to effectively manage flow development and job execution through a single page.

...

Make a copy

...

Image Added

D caption
typefigure
Flows Details Page
For each dataset listed in the page:

  • Solid line connections: Other datasets that are used in the flow and are part of the same flow.
  • Dotted line connections: Other datasets that are used in the flow but belong to a different flow.   

Columns:

  • Name: Name of the wrangled datasets in the flow, including their underlying imported datasets.
  • Last Job: Status of the last job execution on the dataset.

    Tip

    Tip: Hover over the status to see additional details in the job card.

  • Source: For imported datasets, this field identifies the datastore where the source is located.

    Info

    NOTE: Imported datasets are not modified by

    D s product
    .

  • Last Updated: Timestamp for the last time the wrangled dataset was modified or the imported dataset was accessed.

Page Actions:

  • Click the name of a dataset to transform it. See Transformer Page.
  • Expand the caret next to a dataset to see its input dataset.
  • Search: Enter a string of text in the search bar. Matches are displayed as you type.
  • Sort: Click a column header to resort the displayed list of datasets.
  • Details: Click this button to review details on the dataset, including its sources and the jobs executed against it. See Dataset Details Page.

Flow Actions:

For the entire flow, the following actions are available in the drop-down on the right side of the screen. 

...

Edit Flow

...

Flow View page
The imported datasets in the flow or wrangled datasets added to the flow are listed on the left side of the screen. Associated with each imported dataset can be one or more recipes, which are used to transform the source data into wrangled datasets.

  • These objects are connected together by lines flowing between them, which show the relationships between the objects of the flow.
  • For any dataset, any objects on which it depends are displayed to the left of the object on one of the flowing lines leading from the dataset. In the above example, the POS-01 dataset is dependent on all of the objects in the flow, while the REF_CAL dataset is only dependent on its recipe and the REF_CAL.txt imported dataset.

    Tip

    Tip: When you run a job from a wrangled dataset, all of the recipes steps for the preceding datasets are executed as part of the job, and only the results of the terminal wrangled dataset are generated.

  • For more information on these objects, see Object Overview.

Select an object from your flow to open an object-specific panel on the right side of the screen.

Actions:

ActionDescription
Add DatasetsAdd new datasets to the flow. See Add Datasets to Flow below.
Make a copyCreate a copy of the flow. The copied flow is owned by the user who copied it.
Edit name and description...Change the name and description of the flow.
Delete

Delete the flow.

Warning

Deleting a flow removes all wrangled datasets that are contained in the flow. If copies of these datasets exist in other flows, they are not touched. Imported datasets are not deleted by this action.

Add Datasets to Flow

From the Flow View page, you can add imported or wrangled datasets to your flow. These datasets are added as independent objects in the flow and can be joined, unioned, or referenced by other datasets in the flow.

Image Added

D caption
Add datasets to current flow

  1. Search for or select the dataset to add.
    1. Use the page view controls to browse for other datasets, or select the appropriate tab to filter the list to Wrangled or Imported datasets.
    2. To import new datasets from external sources, click Import Datasets. See Import Dataset Page.
  2. When you have made your selections, click Add.
  3. The dataset is added as a new object in flow view.

    Info

    NOTE: For imported datasets that do not have a published schema, such as (CSV, TXT, LOG, or JSON files), a recipe, including steps for inferring structure, and a wrangled dataset are automatically created as part of the process.

View for Imported Datasets

When you select an imported dataset, you can preview the data contained in it, swap the source object, and more from the right-side panel.

Image Added

D caption
Imported Dataset view
Key Fields:

FieldDescription
Data Preview

In the Data Preview window, you can see a small section of the data that is contained in the imported dataset. This window can be useful for verifying that you are looking at the proper data.

Tip

Tip: You can select and copy data from this preview window.

TypeIndicates where the data is sourced or the type of file.
File SizeSize in KB.
LocationPath to the location of the file.
Used InCount of flows where the imported dataset is used, including the current one.

 

Actions:

ActionDescription
Swap

Swap out the current source for a new source for the imported dataset.

Info

NOTE: If the swapped-in source does not have the same schema as the original source, recipe steps in the current flow and any flow that uses the imported dataset may be broken.

For more information, see Dataset Browser.

Add new RecipeCreate a new recipe and wrangled dataset from the imported dataset. This recipe and dataset combination is independent of the original one.
Edit name and description...Change the name and description of the imported dataset.
Remove

Remove the imported dataset from the flow.

Info

NOTE: Any recipe and wrangled dataset using the imported dataset are also removed. In the Remove Dataset dialog, click Details to review the imported dataset.

More detailsSee Dataset Details Page.

View for Recipes

For each recipe, you can review or edit its steps or create new recipes altogether.

Image Added

D caption
Recipe view
Key Fields:

FieldDescription
Steps PreviewPreview the first steps in the recipe.
StepsTotal count of the steps in the recipe.

Actions:

 

...

Delete Flow flow and all of its datasetsDeleting a flow When you delete a flow, you delete all datasets in it. You can move the datasets out of the flow first.
ActionDescription
Edit RecipeOpen the recipe and begin editing. See Transformer Page.
Make a CopyCreate a copy of the recipe and a new wrangled dataset. The copied recipe is owned by the user who copied it.
Move...Move the recipe to a different flow, or create a new flow to contain it.
Delete

Delete the

recipe.

Warning

This step cannot be undone.

Dataset Actions:

For individual datasets, you can perform a number of actions. Hover over the dataset to reveal the following options:

...

Image Removed

...

Image Removed

...

Image Removed

...

Image Removed

...

Image Removed

...

See dataView the data in the wrangled dataset. See View for Wrangled Datasets below.

View for Wrangled Datasets

Image Added

D caption
Wrangled Dataset view
Key Fields:

FieldDescription
Data Preview

In the Data Preview window, you can see a small section of the data that is contained in the wrangled dataset. This window can be useful for verifying that you are looking at the proper data.

Tip

Tip: You can select and copy data from this preview window.

SizeCount of columns and data types in the wrangled dataset.
Used InCount of flows where the dataset is used.
RanCount of jobs launched for the wrangled dataset.
Last RanTimestamp for when the job last ran.

 

Actions:

ActionDescription
Edit RecipeEdit the recipe of the wrangled dataset. See Transformer Page.
Run Job

Launch a job for the wrangled dataset, its recipes, and all preceding datasets.

Info

NOTE: This feature is not available in

D s product
productpe
.

Add new RecipeCreate a new recipe and wrangled dataset from the wrangled dataset. This recipe and dataset combination is independent of the original one.
Edit name and description...Change the name and description for the wrangled dataset.
See recipeView the steps of the recipe associated with the wrangled dataset. See View for Recipes above.
More detailsReview details on the flows where the dataset is used.

View for Referenced Datasets

A referenced dataset is a wrangled dataset that is added to a flow from another flow.

Info

NOTE: A referenced dataset is a read-only object in the flow where it is referenced.

To add a referenced dataset, click Add Datasets from the main Flow View page and select one from a different flow. See Add Datasets to Flow above.

Image Added

Key Fields:

FieldDescription
SizeNumber of columns and data types in the referenced dataset.
Source FlowFlow that contains the dataset. Click the link to open the Flow View page for that dataset.

Actions:

ActionDescription
Add new RecipeCreate a new recipe and wrangled dataset from the referenced dataset. This recipe and dataset combination is independent of the original one.
Remove...Remove the referenced dataset from the flow. The source dataset in the other flow is untouched.