Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.



NOTE: Avoid creating datasets with parameters where individual files or tables have differing schemas. Either import these sources separately and then correct in the application before performing a union on the datasets, or make corrections in the source application to standardize the schemas.


When working with datasets with parameters, it may be useful to do the following if you expect the underlying datasets to be less than 100% consistent with each other. 


For example, suppose you have two environments: Dev and Prod. You can create an environment parameter called env.sourceBucketName to store the name of the bucket from which all data in the workspace or project is imported.

Environment NameSource Bucket NameEnvironment Parameter Value

Code Block
$env.sourceBucketName = 'MyCo_Dev'


Code Block
$env.sourceBucketName = 'MyCo_Prod'

For more information, see Overview of Parameterization


In the screen above, you can see an example of pattern-based parameterization. In this case, you are trying to parameterize the two digits after the value: POS-r.

Include nested folders


NOTE: You cannot create multiple wildcard parameters when the Include nested folders option is selected. When this option is selected for the dataset, only one wildcard parameter is supported.

When you create a wildcard or pattern-based parameter, you have the option to scan any nested folders for matching sources.