NOTE: For file-based sources,
File and path limitations:
- The colon character (
:) cannot appear in a filename or a file path.
- Filenames cannot begin with special characters like dot (
.) or underscore(
- Input file or table paths can have a maximum length of 1024 characters.
1. Connect to sources
During import, the
|D s webapp|
- Compressed files are recognized and can be imported based on their file extensions.
- Filenames that do not have an extension are treated as TXT files.
|D s product|
Tip: You can drag and drop files from your desktop to to upload them.
NOTE: You can upload a file up to 1 GB in size.
NOTE: When you upload an updated version of a previously uploaded file, the new file is stored as a separate upload altogether. In your flow, you must swap out the old dataset to point to the new one.
When a dataset has been selected, the following fields appear on the right side of the screen. Modify as needed:
- Dataset Name: This name appears in the interface.
- Dataset Description: You may add an optional description that provides additional detail about the dataset. This information is visible in some areas of the interface.
Tip: Click the Eye icon to inspect the contents of the dataset prior to importing.
Tip: You can select a single dataset or multiple datasets for import.
You can modify settings used during import for individual files. In the card edit any additional or optional settings for an individual dataset, click Edit Settings.
NOTE: In some cases, there may be discrepancies between row counts in the previewed data versus the data grid after the dataset has been imported, due to rounding in row counts performed in the preview.
Per-file encoding: By default,
|D s product|
|D s product|
NOTE: In some cases, imported files are not properly parsed due to issues with encryption types or encryption keys in the source datastore. For more information, please contact your datastore administrator.
For a list of supported encoding types, see Supported File Encoding Types.
Detect structure: By default,
|D s product|
- Unless you have specific problems with the initial structure, you should leave the Detect structure setting enabled. Recipes created from these imported datasets automatically include the structuring as the first, hidden steps. These steps are not available for editing, although you can remove them through the Recipe panel. See Recipe Panel.
- When detecting structure is disabled, imported datasets whose schema has not been detected are labeled, unstructured datasets. When recipes are created for these unstructured datasets, the structuring steps are added into the recipe and can be edited as needed.
- For more information, see Initial Parsing Steps.
Remove special characters from column names: When selected, characters that are not alphanumeric or underscores are stripped, and space characters are converted to underscores.
For more information, see Sanitize Column Names.
Selecting Column Headers: (file only) You can apply the column headers to your datasets during import. Select the required option from the drop-down list:
- Infer Header: (default) When selected, the
infers the header based on the data in the import.
D s webapp
Use first row as header: When selected, the first row is used as the column headers.
No Header: When selected, the inference is ignored and column headers are defined using generic names with no headers.
If replacing a file:
- If you replace a dataset in a flow and select the Use first row as header option, then the existing header row labels are updated with the new headers.
- Subsequent steps in a pre-existing recipe may be broken if the headers are changed by a replaced file.
. Perform the following:
- Click Edit Settings from the card for an individual dataset in the right panel. The dialog box is displayed.
- In the dialog box, select the required options and modify the settings.
- File Import Settings: For more information, see File Import Settings.
4. Import selections
Tip: If present, you can click the Add to new flow checkbox, which adds the imported datasets to an
- Click Continue. The dataset is imported.
- A recipe is created for it, added to a new flow, and loaded in the Transformer page for wrangling. See Transformer Page.
To import the selected datasets, click Continue.
To begin transforming one of these datasets in Flow View, select it. From its context menu, select Add new recipe. Select the recipe. In the context panel on the right, select Edit Recipe. See Transformer Page.
- To remove a dataset from import, click the X in the dataset card.
|D s also|