NOTE: Access to this page in the application and privileges on its related objects is governed by roles in your workspace. For more information, please contact your workspace administrator.
If you have enabled Deployment Manager, avoid making changes in Flow View on a Production instance of the platform.
- Scheduling executions through Flow View in a Prod environment is not supported. Job executions must be executed through the APIs. See API Workflow - Deploy a Flow.
- Some Flow View options may not be available in a Prod environment.
- You should apply changes to your flow in the Dev instance and then re-deploy to Production. For more information, see Overview of Deployment Manager.
NOTE: If the displayed flow has been shared with you, some options are not available.
Flow View organization:
Flow View is organized into the following basic areas:
|Flow menu||At the top of the screen, you can access a menu of options, which enable adding objects to your flow, configuring aspects of your flow, and and other flow management tasks. See "Flow Menu" below.|
|Flow canvas||The primary workspace of Flow View is the flow canvas, where you build and organize the objects in your flow. See "Flow Canvas" below.|
|Context panel||When you select one or more objects in your flow, the object details and relevant context menu options are available in the right-hand panel. See "Context Panel" below.|
From the menubar at the top of Flow View, the following options are available:
Search icon: Click this icon to search for objects in your flow. See Flow Search Panel.
Details icon: Click to toggle display of the details panel. This setting is saved for the individual user.
Zoom menu: Flow View attempts to zoom the canvas to display as much of the flow as possible. As needed, you can change the level to zoom in or zoom out on the canvas.
Tip: You can access these zoom controls through the context menu for the flow canvas.
- Zoom in: Zoom in 10% on the canvas to focus on greater detail.
- Zoom out: Zoom out 10% from the canvas to see more of it.
- Zoom to fit: Change the zoom level to fit the objects of your flow.
- Zoom to selection: Zoom to center the selected object on the canvas.
- 25%, 50%, or 100%: Change the zoom level to one of the preset levels.
Add Datasets: Click to add new datasets to the flow. See "Add Datasets to Flow" below.
Flow context menu
Schedule Flow: To add a scheduled execution of the recipes in your flow:
- Define the scheduled time and interval of execution at the flow level. See Add Schedule Dialog.
- After the schedule has been created, you can review, edit, or delete the schedule through the Clock icon.
- The time of the next scheduled run is displayed in the time zone of the local browser.
- Define the scheduled destinations for each recipe through its output object. These destinations are targets for the scheduled job. See View for Outputs below.
Configure webhook tasks: You can define tasks to update third-party applications of the results of jobs executed from this flow. For more information, see Create Flow Webhook Task.
NOTE: Webhooks may need to be enabled in your environment. For more information, see Workspace Settings Page.
Share Flow: Collaborate with others on the same flow.
You can also send a copy to other users for separate work.
NOTE: When a flow containing one or more connections is shared, its connections are also shared. By default, credentials are included. If the sharing of credentials has been disabled, the new users must provide their own credentials for the shared connection. See Configure Sharing.
See Share Flow Dialog.
When a user is given access to a flow, all of the following actions are available to that user, except for editing details and deleting the flow.
Manage parameters: Create and manage recipe parameters, as well as specify overrides for them. See Manage Parameters Dialog.Manage email notifications: Configure types of jobs that generate success or failure emails and who receives the messages. See Manage Flow Notifications Dialog.
NOTE: This feature uses an SMTP email server to send messages. For more information on configuring the server, see Enable SMTP Email Server Integration.
NOTE: This feature may need to be enabled in your environment. For more information, see Workspace Settings Page.
- The copied flow is independent of the source flow.
- Optionally, you can duplicate the datasets from the original flow in the copy.
Duplicating datasets has some implications on shared flows. See Overview of Sharing.
NOTE: For flows using parameterized datasets, you should duplicate the datasets, which creates separate copies of parameters and their values in the new flow. If datasets are not copied, then parameter changes in the copied flow modify the values in the source flow.
Export Flow: (Available to flow owner only) Export the flow for archive or transfer. For more information, see Export Flow.
Manage advanced settings: You can configure some advanced or technical aspects of your flow and executions of its jobs. For more information, see Manage Flow Advanced Settings Dialog.
Move to: Move the flow to a new or existing folder. See Manage Flows with Folders.
Edit Flow name and description: (Available to flow owner only) Change the name and description of the flow.
Delete Flow: (Available to flow owner only) Delete the flow.
Deleting a flow cannot be undone.
Deleting a flow removes all recipes that are contained in the flow. If copies of these objects exist in other flows, they are not touched. Imported datasets are not deleted by this action.
Add Datasets to Flow
From the Flow View page, you can add data through the following objects:
- Imported datasets - data sourced from outside the platform
Reference datasets - dataset objects that are created from the output of a recipe in the current flow or another flow
NOTE: For long-loading relational sources, you can track progress of the load. While the data is loading, some recipe and flow options are disabled.
For more information on enabling long-loading, see Configure JDBC Ingestion.
- Search for or select the dataset to add.
- Use the page view controls to browse for other datasets, or select the appropriate tab to filter the list to a specific type of object.
- To import new datasets from external sources, click Import Datasets. See Import Data Page.
- When you have made your selections, click Add.
The object or objects are added as a new object in flow view.
For large relational datasets, you can monitor the import process through the Flow View page.
NOTE: This feature may require enablement in your deployment. See Configure JDBC Ingestion.
For more information, see Overview of Job Monitoring.
The central workspace of Flow View, the canvas is where you add and arrange the objects in your flow. In the canvas, you can select one or more objects at a time, drag them to reposition them on screen, and zoom in or out to focus on your current area of development.
When you add an object to the canvas, an icon representing it is added to the flow canvas. This object can be repositioned as needed.
Tip: The relative position of objects on the flow view canvas is preserved between screen updates. On refresh, the window on the canvas is repositioned based on the leftmost object on the canvas to focus on the flow to other objects from that one.
- If you create an object from another object, such as an output from a recipe, an arrow connects the recipe to the output.
For any object, the objects on which it depends are displayed to the left of the original object, and there is a line from the preceding objects to the original object.
Tip: When you run a job for a recipe, all of the recipes steps for the preceding datasets are executed as part of the job, and only the results of the terminal dataset are generated.
NOTE: Objects marked with a red dot indicate a problem with the object's configuration. Please select the object to begin investigating the error. Error information may be displayed in the right panel.
- Click the icon for an object to select it.
- Object details for the specific type of object are displayed in the context panel.
- Right-click the object to open its context menu.
- To select multiple objects:
- Click and drag over a set of objects.
- To select a discrete set of objects, press
COMMAND+ click the objects.
- When you select multiple objects, the objects are listed in the context panel with options that are applicable to all of the objects. Some objects, like notes, do not have a context panel.
- Right-click to display the context menu of options that are applicable to the selected objects.
- See "Context Panel" below.
- To move an object, click and drag it to a new location. Any arrows connecting to the object are repositioned as well.
- To move multiple objects, select them and then drag them to a new location.
Canvas context menu
When you right-click an empty part of the canvas, the following options are available:
Select All: Select all objects on the canvas. Options that are relevant to all of the objects are displayed on the context panel.
Tip: You can drag these objects to reposition them together.
- Add note: Add a note with text or emojis to the canvas. See "Canvas notes" below.
- Zoom: Zoom in or zoom out on the canvas as needed.
- Fit to screen: This option zooms the canvas to display all of the flow's objects on your screen at the same time.
- See "Flow Menu" above for details.
As you develop your flow, you can add notes where helpful to the canvas. For example, if you are collaborating with another user, you can leave status information about objects that are still in progress.
Tip: You can click and drag notes like other objects on the canvas, so they can be repositioned with the related flow object.
From a note's context menu, you can edit or delete the note.
In the flow canvas, you work with the following types of objects:
- Imported datasets
- Unstructured datasets
- Reference datasets
- Datasets with parameters
To begin working with data:
- Click Add datasets. Locate your source data and import it into your flow. See "Add Datasets to Flow" above.
In Flow View, select the imported dataset on the flow canvas. Then, in the context panel, click Add new recipe. A new, empty recipe is associated with the dataset.
Tip: Double-click an imported dataset to see a preview of it. Some datasets cannot be previewed.
- To open in the Transformer page, click the recipe and select Edit Recipe. See Transformer Page.
When created, these objects are connected together by lines flowing between them, which show the relationships between the objects in the flow.
A recipe is a set of steps to transform source data into the results you desire.
Tip: Double-click any recipe to edit it. See Transformer Page.
A recipe can be created from the following objects:
- An imported dataset, as described above.
- A reference dataset is an object that has been pulled into a flow from another flow. See "View for Reference Datasets" below.
- Another recipe. You can chain together recipes. For example, you may have a set of steps that you always apply at the beginning of transforming a specific type of feed. This recipe can be added into each flow as the first recipe chained to an imported dataset of that feed type.
The following objects can be created off of a recipe:
- An output object is a set of publishing targets for which you can execute jobs.
- A reference object is a reference to one of your flow's recipes that can be used in another flow. When a reference object is created, the target flow receives the output of the executed recipe.
- In the target flow, this object appears as a reference dataset.
- When a reference dataset is used in a flow, the target flow receives the output of the executed recipe.
For more information on these objects, see Object Overview.
Select an object from your flow to open an object-specific panel on the right side of the screen.
- When multiple objects are selected, the displayed details and options apply to all of the objects.
- If the selected objects are of different types, the available options are limited.
Tip: You can right-click any object in Flow View to see the list of available actions that appear when you select it and choose from the right panel.
Depending on the type of object or objects that you select in the canvas, the view in the context panel changes:
Tip: The object on the canvas and the context panel display the same set of context menu options. For more information on these options, see the links below.
View for Connections
When you create a dataset that is imported through a connection, a reference to the connection is added to your flow canvas. See View for Connections.
View for Imported Datasets
View for Dataset with Parameters
For datasets with parameters, the context panel and available options are different from non-parameterized imported datasets. See View for Dataset with Parameters.
View for Unstructured Datasets
If detecting structure has been disabled for your imported dataset, the structure detecting steps are broken out into the first steps of a recipe that is auto-generated for you. You can review and modify them within the recipe. See View for Unstructured Datasets.
View for Recipes
See View for Recipes.
View for Outputs
Outputs are created from recipes. See View for Outputs.
View for References
References are created from recipes. See View for References.
View for Reference Datasets
A reference dataset is an imported dataset that is sourced from the reference created from another recipe in another flow. See View for Reference Datasets.
This page has no comments.