Getting Started with Cloud Dataprep

Contents:

Cloud Dataprep by TRIFACTA® INC. enables you to rapidly transform disparate datasets of any size into usable data for the entire enterprise. Ingest, explore, and transform your data through a leading-edge interface, reducing the time to prepare your data from weeks to minutes.  Cloud Dataprep by TRIFACTA INC. is integrated with the Google Cloud Platform and operated by partner  Trifacta.

Applicable Product Editions

These setup instructions apply to the following editions of the product:

1. Cloud Dataprep Premium by TRIFACTA INC.

   NOTE: If you are purchasing Cloud Dataprep Premium by TRIFACTA INC., you must contact Trifacta Support before you purchase the product.

2. Cloud Dataprep Standard by TRIFACTA INC.
   
3. Cloud Dataprep Legacy by TRIFACTA INC.

For more information, see Product Editions.

For more information on available plans, see https://www.trifacta.com/pricing/cloud-dataprep/.

Support packages

For  Cloud Dataprep Premium by TRIFACTA INC. and  Cloud Dataprep Standard by TRIFACTA INC.,  Trifacta offers a range of support packages. For more information, please contact  Trifacta Support.

Pre-requisites

Before you begin, please review the following pre-requisites.

Set up a project

To use either product edition, you must have the following already set up in the Google Cloud Platform.

1. Create or set up a Google Cloud project. In the Cloud Console, on the project selector page, select or create a Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to the project selector page

2. Enable billing on that project. Please verify that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

3. Enable services: In your project, enable the following services:
   1. Cloud Dataflow
   2. BigQuery 
   3. Cloud Storage APIs. See Enable the APIs.

Set up your storage bucket

On  Google Cloud Storage, you must have a bucket set up for use with your project.

1. In the Cloud Console, navigate to the Cloud Storage Browser page. See https://console.cloud.google.com/storage/browser.
2. Click Create bucket.
3. In the Create bucket dialog, specify the following attributes:
   • A unique bucket name. For more information on bucket name requirements, see https://cloud.google.com/storage/docs/bucket-naming#requirements.
   • A storage class. See https://cloud.google.com/storage/docs/storage-classes.
   • A location where bucket data will be stored.
4. Click Create.

Set up your staging bucket

By default, when you enable  Cloud Dataprep by TRIFACTA INC. in a project, a Google Cloud Storage staging bucket for Cloud Dataflow use is automatically created for you in a U.S. region. This staging bucket is used for staging assets for use on Cloud Dataflow jobs and is required for use with the product. If you do have permissions to create a storage bucket in the U.S, you do not need to create a storage bucket for staging and can skip to the next section. 

A bucket can be created from:

• Google Console: https://cloud.google.com/storage/docs/creating-buckets
• Google CLI: https://cloud.google.com/storage/docs/gsutil

The staging bucket can be changed:

• During enablement of the product in a project. You can select a different staging bucket as needed. 
• After the product has been enabled, individual users can configure the bucket to use for staging of their assets. See User Profile Page.

Premium-only requirements

Whitelist the IP address range of the Trifacta Service

If you are connecting to relational sources, you must whitelist the IP address range of the Trifacta service for your database instances.  The IP address range of the Trifacta service are the following:

34.68.114.64/28

For more information, please contact  Trifacta Support.

Purchase and enable through the Google Marketplace

After you have completed the above steps, please proceed through the Google Marketplace to complete your purchase. Your purchase covers:

• Basic entitlement
• Licensing for each Google Cloud projects

For more information, see https://console.cloud.google.com/marketplace/details/trifacta/cloud-dataprep-editions.

Setup

After the product has been licensed for your project, please complete the following steps for your account.

Required additional permissions for  Cloud Dataprep Premium by TRIFACTA INC.

Cloud Dataprep Premium by TRIFACTA INC. requires special permissions to use the project. For more information, see Create IAM Role for Dataprep. 

Enable in the project

1. In the Google Cloud Console, select the project in which you wish to enable  Cloud Dataprep by TRIFACTA INC..
2. Open the product. See https://console.cloud.google.com/dataprep.
3. As the project owner, you must enable access to project data for Google and  Trifacta. 

Login

Each user of the project must do the following:

1. In the Google Cloud Console, select the project in which you wish to enable  Cloud Dataprep by TRIFACTA INC..
2. Open the product. See https://console.cloud.google.com/dataprep.
3. Accept the terms of service. 
4. Select a  Google Cloud Storage bucket to use with the product. For more information, see Enable or Disable Dataprep.
5. The Trifacta application is displayed.
   1. The first time you login, you can immediately upload a dataset and begin transforming it. For more information, see Import Basics.
   2. On subsequent logins, the Home page is displayed:

Figure: Home page

Project settings

The project owner should review the settings for your project. See Project Settings Page.

Set up directories

Each user must configure the directories on  Google Cloud Storage for use with the product. You can change the directories that are used for uploads, job runs, and temp storage. 

1. In the left nav bar, select the User icon. 
2. In the User menu, select Preferences.
   
3. The User Profile page is displayed. 
   
4. As needed, you can change the Upload, Job Run, and Temp directories in your bucket. To save your changes, click Done. 

For more information, see User Profile Page.

Test

If you have completed the above steps, you should verify operations.

Verify operations

Before inviting other users, you should run a simple job through the product. 

Prepare Your Sample Dataset

To complete this test, you should locate or create a simple dataset. Your dataset should be created in the format that you wish to test.

Characteristics:

• Two or more columns. 
• If there are specific data types that you would like to test, please be sure to include them in the dataset.
• A minimum of 25 rows is required for best results of type inference.
• Ideally, your dataset is a single file or sheet. 

Verification Steps

Steps:

1. Login to the application.For Cloud Dataprep by TRIFACTA INC. editions, your login is your gmail address.

2. In the application menu bar, click Library.

   Tip: When you login for the first time, you can immediately import a dataset to begin transforming it.

3. Click Import Data. See Import Data Page.
   1. Select the connection where the dataset is stored. For datasets stored on your local desktop, click Upload.
   2. Select the dataset.
   3. Click Continue.
4. The initial sample of the dataset is opened in the Transformer page, where you can edit your recipe to transform the dataset.
   
   1. In the Transformer page, some steps are automatically added to the recipe for you. So, you can run the job immediately.
   2. You can add additional steps if desired. See Transformer Page.
5. Click Run. 
   

   1. If options are presented, select the defaults.

   2. To generate results in other formats or output locations, click Add Publishing Destination. Configure the output formats and locations. 
   3. To test dataset profiling, click the Profile Results checkbox. Note that profiling runs as a separate job and may take considerably longer. 

   4. See Run Job Page.

6. When the job completes, you should see a success message under the Jobs tab in the Flow View page. 
   
   1. Troubleshooting: Either the Transform job or the Profiling job may break. To localize the problem, try re-running a job by deselecting the broken job type or running the job on a different running environment (if available). You can also download the log files to try to identify the problem. See Job Details Page.
7. Click View Results from the context menu for the job listing. In the Job Details page, you can see a visual profile of the generated results. See Job Details Page.
8. In the Output Destinations tab, click a link to download the results to your local desktop. 
9. Load these results into a local application to verify that the content looks ok.

Verify IP address whitelisting

If you have whitelisted the Trifacta service IP addresses for your database server, you can create a connection to the database from inside the Trifacta application. If you are able to successfully read data into the application from your database, then the whitelist has been specified correctly.

For more information, see Connection Types.

Invite Users

You can invite other people to join your project at this time.

For more information, see https://cloud.google.com/iam/docs/quickstart.

Example Flows

When a new workspace is created, the first user is provided a set of example flows. These flows are intended to teach by example and illustrate many recommended practices for building your own flows. For more information on example flows, see Workflow Basics.

Resources

The following resources can assist users in getting started with wrangling.

• For a quick start of  Cloud Dataprep by TRIFACTA INC. products, see Quickstart for Dataprep.
• Check out the Trifacta Community: https://community.trifacta.com
  
  • Try the free Wrangler certification course. See https://community.trifacta.com/s/certification.
• For a basic summary of each step of the wrangling process, see Workflow Basics.

Access documentation: To access the full customer documentation, from the left nav bar, select Help menu > Documentation.

Additional Setup

Depending on your environment, the following additional configuration steps may be required. 

• To access BigQuery datasets in other projects, see Access Cross-Project BigQuery Datasets.
• To access  Google Cloud Storage datasets in other projects, see Access Cross-Project Cloud Storage Buckets.