Through the Flow View page, you can access and manage all objects in your flow. For your currently selected each imported dataset, recipe, or wrangled dataset in your flow, you can review its datasets and history information and make changes to it. Select a flow or create a new one from the Flows Page.perform a variety of actions to effectively manage flow development and job execution through a single page.
Make a copy
Flows Details Page
- Solid line connections: Other datasets that are used in the flow and are part of the same flow.
- Dotted line connections: Other datasets that are used in the flow but belong to a different flow.
- Name: Name of the wrangled datasets in the flow, including their underlying imported datasets.
Last Job: Status of the last job execution on the dataset.
Tip: Hover over the status to see additional details in the job card.
Source: For imported datasets, this field identifies the datastore where the source is located.
NOTE: Imported datasets are not modified by
D s product
- Last Updated: Timestamp for the last time the wrangled dataset was modified or the imported dataset was accessed.
- Click the name of a dataset to transform it. See Transformer Page.
- Expand the caret next to a dataset to see its input dataset.
- Search: Enter a string of text in the search bar. Matches are displayed as you type.
- Sort: Click a column header to resort the displayed list of datasets.
- Details: Click this button to review details on the dataset, including its sources and the jobs executed against it. See Dataset Details Page.
For the entire flow, the following actions are available in the drop-down on the right side of the screen.
Flow View page
- These objects are connected together by lines flowing between them, which show the relationships between the objects of the flow.
For any dataset, any objects on which it depends are displayed to the left of the object on one of the flowing lines leading from the dataset. In the above example, the
POS-01dataset is dependent on all of the objects in the flow, while the
REF_CALdataset is only dependent on its recipe and the
Tip: When you run a job from a wrangled dataset, all of the recipes steps for the preceding datasets are executed as part of the job, and only the results of the terminal wrangled dataset are generated.
- For more information on these objects, see Object Overview.
Select an object from your flow to open an object-specific panel on the right side of the screen.
|Add Datasets||Add new datasets to the flow. See Add Datasets to Flow below.|
|Make a copy||Create a copy of the flow. The copied flow is owned by the user who copied it.|
|Edit name and description...||Change the name and description of the flow.|
Delete the flow.
Add Datasets to Flow
From the Flow View page, you can add imported or wrangled datasets to your flow. These datasets are added as independent objects in the flow and can be joined, unioned, or referenced by other datasets in the flow.
Add datasets to current flow
- Search for or select the dataset to add.
- Use the page view controls to browse for other datasets, or select the appropriate tab to filter the list to Wrangled or Imported datasets.
- To import new datasets from external sources, click Import Datasets. See Import Dataset Page.
- When you have made your selections, click Add.
The dataset is added as a new object in flow view.
NOTE: For imported datasets that do not have a published schema, such as (CSV, TXT, LOG, or JSON files), a recipe, including steps for inferring structure, and a wrangled dataset are automatically created as part of the process.
View for Imported Datasets
When you select an imported dataset, you can preview the data contained in it, swap the source object, and more from the right-side panel.
Imported Dataset view
In the Data Preview window, you can see a small section of the data that is contained in the imported dataset. This window can be useful for verifying that you are looking at the proper data.
|Type||Indicates where the data is sourced or the type of file.|
|File Size||Size in KB.|
|Location||Path to the location of the file.|
|Used In||Count of flows where the imported dataset is used, including the current one.|
Swap out the current source for a new source for the imported dataset.
For more information, see Dataset Browser.
|Add new Recipe||Create a new recipe and wrangled dataset from the imported dataset. This recipe and dataset combination is independent of the original one.|
|Edit name and description...||Change the name and description of the imported dataset.|
Remove the imported dataset from the flow.
|More details||See Dataset Details Page.|
View for Recipes
For each recipe, you can review or edit its steps or create new recipes altogether.
|Steps Preview||Preview the first steps in the recipe.|
|Steps||Total count of the steps in the recipe.|
|Edit Recipe||Open the recipe and begin editing. See Transformer Page.|
|Make a Copy||Create a copy of the recipe and a new wrangled dataset. The copied recipe is owned by the user who copied it.|
|Move...||Move the recipe to a different flow, or create a new flow to contain it.|
|Delete|| flow and all of its datasets|
|Deleting a flow |
| When you delete a flow, you delete all datasets in it. You can move the datasets out of the flow first.|
For individual datasets, you can perform a number of actions. Hover over the dataset to reveal the following options:
|See data||View the data in the wrangled dataset. See View for Wrangled Datasets below.|
View for Wrangled Datasets
Wrangled Dataset view
In the Data Preview window, you can see a small section of the data that is contained in the wrangled dataset. This window can be useful for verifying that you are looking at the proper data.
|Size||Count of columns and data types in the wrangled dataset.|
|Used In||Count of flows where the dataset is used.|
|Ran||Count of jobs launched for the wrangled dataset.|
|Last Ran||Timestamp for when the job last ran.|
|Edit Recipe||Edit the recipe of the wrangled dataset. See Transformer Page.|
Launch a job for the wrangled dataset, its recipes, and all preceding datasets.
|Add new Recipe||Create a new recipe and wrangled dataset from the wrangled dataset. This recipe and dataset combination is independent of the original one.|
|Edit name and description...||Change the name and description for the wrangled dataset.|
|See recipe||View the steps of the recipe associated with the wrangled dataset. See View for Recipes above.|
|More details||Review details on the flows where the dataset is used.|
View for Referenced Datasets
A referenced dataset is a wrangled dataset that is added to a flow from another flow.
NOTE: A referenced dataset is a read-only object in the flow where it is referenced.
To add a referenced dataset, click Add Datasets from the main Flow View page and select one from a different flow. See Add Datasets to Flow above.
|Size||Number of columns and data types in the referenced dataset.|
|Source Flow||Flow that contains the dataset. Click the link to open the Flow View page for that dataset.|
|Add new Recipe||Create a new recipe and wrangled dataset from the referenced dataset. This recipe and dataset combination is independent of the original one.|
|Remove...||Remove the referenced dataset from the flow. The source dataset in the other flow is untouched.|