Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0810

D toc
D s ed
rtrue
editionsawsent,awspro,awspr

Excerpt

Within a flow, you can create and manage flow parameters, including specifying override values. From the flow menu, select Parameters.

  • flow parameter is a reference token that can be invoked from within the flow. A flow parameter can be:
    • A string value
    • D s item
      itemparameter
    • A regular expression
  • Where it is invoked, the default value for the parameter is applied or, if an override value has been set, the override value is applied.
  • For more information, see Overview of Parameterization.

You specify flow parameters at the flow level. They can be invoked in any recipe within the flow.

Tip

Tip: Override values apply to all parameters in the flow that share the same name, even if they are output object parameters.


Tip

Tip: Flow parameters can be inherited from upstream flows. For example, if you create a reference dataset that references flow parameters from its flow, those parameters are passed to the downstream flow. While you cannot change the default value for the downstream instance of the parameter, you can apply override values for all recipes in the downstream flow.

Parameters Tab

In the Parameters tab, you can manage the flow parameters in your flow.

Info

NOTE: Non-flow parameters cannot be edited or deleted from the Parameters tab. These parameters must be modified in the interface for the object to which they apply. Overrides can be applied to these types of parameters through the Overrides tab.

When reviewing parameters for your plan, you may notice that a parameter has multiple values:

  • Parameter values that do not conflict are ok. For example, if the parameter is applied to an imported dataset and to an output object which are attached to different recipes, these values are not in conflict.
  • If the parameter values are in conflict, then a warning icon is displayed. For example, if a parameter is applied to an imported dataset and to an output object attached to the same recipe, then the conflicting parameter values must be overridden to fix them.
Info

NOTE: When a parameter has conflicting values from objects in a plan task, you must fix them by applying a single override value. If these values are not fixed, then the plan fails to execute.



D caption
Parameters tab
  • To create a new parameter, click Add parameter.
    • Specify the Name parameter. 

      Info

      NOTE: Name values are case-sensitive. After saving a parameter, you cannot change its name.

    • Specify the default Value for the parameter. Examples by parameter type:

      • String literal:

        Code Block
        This is my string.
        Info

        NOTE: Flow parameter values that are literal values are String values. You can convert them to other data types after they have been referenced in your recipe.

      • D s lang
        itempattern
        : Patterns can be used to find matches in your recipes. The following pattern matches for two consecutive digits:

        Code Block
        `{digit}{digit}`

        For more information on 

        D s lang
        itempatterns
        , see Text Matching.

      • Regular expression: These patterns also can be used for finding matches. The following pattern matches for two consecutive digits:

        Code Block
        /[0-9][0-9]/

        Regular expressions are a standard method for identifying patterns in data. The syntax is based on RE2 and PCRE regular expressions.

    • Click Save.
  • To edit a parameter's default value, hover over its entry, and click the Pencil icon.
  • To delete a parameter, hover over its entry and click the Trash icon.

    Warning

    Deleting a parameter cannot be undone. When you delete a parameter, all recipe steps that reference it are broken.

    Tip

    Tip: If you accidentally delete a flow parameter, you can recreate it with the same case-sensitive name. All references to it should work again.

After you create a parameter, you can insert it into your recipes using the following type of reference:

Code Block
${myRecipeParameterName}

For more information, see Create Flow Parameter.

Overrides Tab

In the Overrides tab, you can apply override values to any parameters referenced in your flow, including:

  • Flow parameters
  • output object parameters
  • Dataset parameters
  • Parameters that are available through reference datasets
Info

NOTE: The parameters listed in this tab have override values applied to them. Parameters that are using their default values in this flow are not listed.


D caption
Overrides tab

All of the available parameter overrides are listed. 

  • To apply an override to all references of a parameter within the flow, click Add override.
    • Select the name of the parameter.
    • Enter its override value. Click Save.
  • To edit a parameter override, hover over its entry and click the Pencil icon. Enter a new value. 

    Info

    NOTE: The override value is applied to all subsequent operations in the platform. When a job is submitted to the job queue, any overrides are applied at that time. Changes to override values do not affect jobs that are already in flight.

  • To delete an override, hover over its entry and click the Trash icon.

A note on upgraded parameters

If you have upgraded from a version of the product before Release 7.1, any parameters that were defined in the previous version appear grayed out in the Parameters tab.

  • The Parameters tab is used for defining new flow parameters, which is a new type of parameter. These parameters can be referenced inside your recipe steps. See above.

    Info

    NOTE: Parameters that are not of flow parameter type cannot be edited in the Parameters tab. You can apply override values to these parameters through the Overrides tab.

  • The Overrides tab is used to specify overrides to the default values for parameters, including parameters defined outside of Flow View. 
    • If you add an override using the name of the parameter that was upgraded, the override is applied, based on the matching name.

Manage Parameters for Plans

You can also specify parameter overrides at the plan level. For the recipes in your plan, you can pass in parameter overrides values, which are used during plan runs.

D s ed
editionsawsent,awspro,awspr

D caption
Manage Parameters Dialog for plans

In the Manage Parameters dialog, you can review the parameters that are defined for the flows that are part of your plan.

Info

NOTE: You cannot create parameters through Plan View. You must create the parameters within the flows that are part of your plan. After they are created, they are available for overrides through Plan View.

Info

NOTE: In a plan, parameters and their overrides apply only to the flow task from where they are sourced. They do not apply to other flow tasks. They do not apply back to the source flows.

Tip

Tip: To pass a flow parameter value from one recipe to another, you can insert the parameter value in a column in a recipe, export the results as a reference dataset, and then ingest that reference into another flow.


Columns:

  • Parameter: The name of the parameter
  • From: The source flow in your plan for the parameter
  • Created in: The object in the source flow where the parameter is defined
  • Value: The current value for the parameter
Tip

Tip: To override the inherited value for the parameter, click the Pencil icon. Then, click Save. Whenever the flow is executed as part of this current plan, the override value is used to replace the parameter value and any override that are defined within the flow.