You can create a dataset from one or more files stored in ADLS.
You can parameterize your input paths to import source files as part of the same imported dataset. For more information, see Overview of Parameterization.
When you select a folder in ADLS to create your dataset, you select all files in the folder to be included. Notes:
- This option selects all files in all sub-folders. If your sub-folders contain separate datasets, you should be more specific in your folder selection.
- All files used in a single dataset must be of the same format and have the same structure. For example, you cannot mix and match CSV and JSON files if you are reading from a single directory.
- When a folder is selected from ADLS, the following file types are ignored:
*_FAILEDfiles, which may be present if the folder has been populated by HDI.
- If you have stored files in ADLS that begin with an underscore (
_), these files cannot be read during batch transformation and are ignored. Please rename these files through ADLS so that they do not begin with an underscore.