Import Data Page

Contents:

Expand all Collapse all

NOTE: Designer Cloud Educational is a free product with limitations on its features. Some features in the documentation do not apply to this product edition. See Product Limitations.

Contents:

Through the Import Data page, you can upload datasets or select datasets from sources that are stored on connected datastores. From the Library page, click Import Data.

Figure: Import Data page

General Limitations

NOTE: For file-based sources, Designer Cloud Educational expects that each row of data in the import file is terminated with a consistent newline character, including the last one in the file.

For single files lacking this final newline character, the final record may be dropped.

NOTE: An imported dataset requires about 15 rows to properly infer column data types and the row, if any, to use for column headers.

File and path limitations:

The colon character ( :) cannot appear in a filename or a file path.
Filenames cannot begin with special characters like dot ( .) or underscore( _).

Basic Workflow

1. Connect to sources

NOTE: Compressed files are recognized and can be imported based on their file extensions.

Upload: Designer Cloud Educational can also load files from your local file system.

Tip: You can drag and drop files from your desktop to to upload them.

NOTE: You can upload a file up to 1 GB in size.

NOTE: When you upload an updated version of a previously uploaded file, the new file is stored as a separate upload altogether. In your flow, you must swap out the old dataset to point to the new one.

For more information on the supported input formats, see Supported File Formats.

2. Add datasets

When you have found your source directory or file:

You can hover over the name of a file to preview its contents.

NOTE: Preview may not be available for some sources, such as Parquet.
Click the Plus icon next to the directory or filename to add it as a dataset.

Tip: You can import multiple datasets at the same time. See below.
Excel files: Click the Plus icon next to the parent workbook to add all of the worksheets as a single dataset, or you can add individual sheets as individual datasets. See Import Excel Data.

3. Configure selections

When a dataset has been selected, the following fields appear on the right side of the screen. Modify as needed:

Dataset Name: This name appears in the interface.

Dataset Description: You may add an optional description that provides additional detail about the dataset. This information is visible in some areas of the interface.

Tip: Click the Eye icon to inspect the contents of the dataset prior to importing.

You can select a single dataset or multiple datasets for import.

Edit settings

You can modify settings used during import for individual files. In the card for an individual dataset, click Edit Settings.

NOTE: In some cases, there may be discrepancies between row counts in the previewed data versus the data grid after the dataset has been imported, due to rounding in row counts performed in the preview.

Per-file encoding: By default, Designer Cloud Educational applies a specified encoding type on the imported the file. In some cases, the data preview panel may contain garbled data, due to a mismatch in encodings. In the Data Preview dialog, you can select a different encoding for the file. When the correct encoding is selected, the preview displays the data as expected.

NOTE: Assessing the file encoding type based on parsing an input file is not an accurate method. Instead, Designer Cloud Educational assumes that the file is encoded in the default encoding. If it is not, you should change the encoding type for the file.

NOTE: In some cases, imported files are not properly parsed due to issues with encryption types or encryption keys in the source datastore. For more information, please contact your datastore administrator.

For a list of supported encoding types, see Supported File Encoding Types.

Detect structure: By default, Designer Cloud Educational attempts to interpret the structure of your data during import. This structuring attempts to apply an initial tabular structure to the dataset.

Unless you have specific problems with the initial structure, you should leave the Detect structure setting enabled. Recipes created from these imported datasets automatically include the structuring as the first, hidden steps. These steps are not available for editing, although you can remove them through the Recipe panel. See Recipe Panel.
When detecting structure is disabled, imported datasets whose schema has not been detected are labeled, unstructured datasets. When recipes are created for these unstructured datasets, the structuring steps are added into the recipe and can be edited as needed.
For more information, see Initial Parsing Steps.

Remove special characters from column names: When selected, characters that are not alphanumeric or underscores are stripped, and space characters are converted to underscores.

For more information, see Sanitize Column Names.

Selecting Column Headers: (file only) You can apply the column headers to your datasets during import. Select the required option from the drop-down list:

Infer Header: (default) When selected, the Designer Cloud application infers the header based on the data in the import.
Use first row as header: When selected, the first row is used as the column headers.
No Header: When selected, the inference is ignored and column headers are defined using generic names with no headers.

If replacing a file:

If you replace a dataset in a flow and select the Use first row as header option, then the existing header row labels are updated with the new headers.
Subsequent steps in a pre-existing recipe may be broken if the headers are changed by a replaced file.

Tip: After the dataset is imported, you can rename columns manually or using any row in the dataset. For more information, see Rename Columns.

4. Import selections

Single dataset

If you have selected a single dataset for import:

Tip: If present, you can click the Add to new flow checkbox, which adds the imported datasets to an Untitled flow. For more information, see Flow View Page.

Click Continue. The dataset is imported.
A recipe is created for it, added to a new flow, and loaded in the Transformer page for wrangling. See Transformer Page.

Multiple datasets

You can import multiple datasets from multiple sources at the same time. In the Import Data page, continue selecting sources, and additional dataset cards are added to the right panel.

NOTE: If you are importing from multiple files at the same time, the files are not necessarily read in a regular or predictable order.

NOTE: When you import a dataset with parameters from multiple files, only the first matching file is displayed in the right panel.

In the right panel, you can see a preview of each dataset and make changes as needed.

Figure: Import Multiple Datasets

If you have selected multiple datasets for import: