...
| Info |
|---|
NOTE: Avoid creating datasets with parameters where individual files or tables have differing schemas. Either import these sources separately and then correct in the application before performing a union on the datasets, or make corrections in the source application to standardize the schemas. |
When working with datasets with parameters, it may be useful to do the following if you expect the underlying datasets to be less than 100% consistent with each other.
...
For example, suppose you have two environments: Dev and Prod. You can create an environment parameter called env.sourceBucketName to store the name of the bucket from which all data in the workspace or project is imported.
| Environment Name | Source Bucket Name | Environment Parameter Value | ||
|---|---|---|---|---|
| Dev | MyCo_Dev |
| ||
| Prod | MyCo_Prod |
|
For more information, see Overview of Parameterization.
...
In the screen above, you can see an example of pattern-based parameterization. In this case, you are trying to parameterize the two digits after the value: POS-r.
Include nested folders
| Info |
|---|
NOTE: You cannot create multiple wildcard parameters when the Include nested folders option is selected. When this option is selected for the dataset, only one wildcard parameter is supported. |
When you create a wildcard or pattern-based parameter, you have the option to scan any nested folders for matching sources.
...
