1. Enabled S3 Integration: The

   D s platform

   has been configured to integrate with your S3 instance.For more information, see S3 Access.

2. Creating Datasets from S3 Files: You can read in source data stored in S3. An imported dataset may be a single S3 file or a folder of identically structured files. See Reading from Sources in S3 below.
3. Reading Datasets: When creating a dataset, you can pull your data from a source in S3. See Creating Datasets below.

4. Writing Results: After a job has been executed, you can write the results back to S3.

In the

Before You Begin Using S3

• Access: If you are using system-wide permissions, your administrator must configure access parameters for S3 locations. If you are using per-user permissions, this requirement does not apply. See S3 Access.

  Warning
  Avoid using /trifacta/uploads for reading and writing data. This directory is used by the D s webapp .

• Your administrator should provide a writeable home output directory for you. This directory location is available through your user profile. See Storage Config Page.

Secure Access

Your administrator can grant access on a per-user basis or for the entire

The

Storing Data in S3

Your administrator should provide raw data or locations and access for storing raw data within S3. All

• Users should know where shared data is located and where personal data can be saved without interfering with or confusing other users.
• The

  D s webapp

  stores the results of each job in a separate folder in S3.

Reading from Sources in S3

You can create an imported dataset from one or more files stored in S3.

Wildcards:

You can parameterize your input paths to import source files as part of the same imported dataset. For more information, see Overview of Parameterization.

Folder selection:

When you select a folder in S3 to create your dataset, you select all files in the folder to be included.

Notes:

• This option selects all files in all sub-folders and bundles them into a single dataset. If your sub-folders contain separate datasets, you should be more specific in your folder selection.
• All files used in a single imported dataset must be of the same format and have the same structure. For example, you cannot mix and match CSV and JSON files if you are reading from a single directory.

When a folder is selected from S3, the following file types are ignored:

• *_SUCCESS and *_FAILED files, which may be present if the folder has been populated by the running environment.

Creating Datasets

When creating a dataset, you can choose to read data in from a source stored from S3 or local file.

• S3 sources are not moved or changed.
• Local file sources are uploaded to /trifacta/uploads where they remain and are not changed.

Data may be individual files or all of the files in a folder. In the Import Data page, click the S3 tab. See Import Data Page.

Writing Results

When you run a job, you can specify the S3 bucket and file path where the generated results are written. By default, the output is generated in your default bucket and default output home directory.

• Each set of results must be stored in a separate folder within your S3 output home directory.
• For more information on your output home directory, see Storage Config Page.

Creating a new dataset from results

As part of writing results, you can choose to create a new dataset, so that you can chain together data wrangling tasks.

Purging Files

Other than temporary files, the