Install for AWS

This install process applies to installing  Designer Cloud Enterprise Edition on an AWS infrastructure that you manage. 

AWS Marketplace deployments:

Scenario Description

• Installation of  Designer Cloud Enterprise Edition on an EC2 server in AWS
• Installation of Alteryx databases on AWS
• Integration with a supported EMR cluster.
• Base storage layer and backend datastore of S3

For more information on deployment scenarios, see Supported Deployment Scenarios for AWS.

Product Limitations

The following limitations apply to installations of  Designer Cloud Enterprise Edition on AWS:

• No support for Hive integration
• No support for secure impersonation or Kerberos
• No support for high availability and failover
• Job cancellation is not supported on EMR.
• When publishing single files to S3, you cannot apply an append publishing action.

Pre-requisites

Desktop Requirements

• All desktop users of the platform should have a supported version of Google Chrome installed on their desktops.
  • For more information. see Desktop Requirements.
  • If a supported browser is not available within your enterprise, desktop users can install the Alteryx enterprise application as a separate application. For more information, see Install for Wrangler Enterprise Application.
• All desktop users must be able to connect to the EC2 instance through the enterprise infrastructure.

AWS Pre-requisites

Depending on which of the following AWS components you are deploying, additional pre-requisites and limitations may apply. Please review these sections as well.

• Configure for EMR
• Enable S3 Access
• Create Redshift Connections

Prep

Before you begin, please verify that you have completed the following:

1. Review Planning Guide: Please review and verify Install Preparation and sub-topics.
   1. Limitations: For more information on limitations of this scenario, see Product Limitations in the Install Preparation area.

2. Read: Please read this entire document before you create the EMR cluster or install the Designer Cloud Powered by Trifacta platform.

3. Acquire Assets: Acquire the installation package for your operating system and your license key. For more information, contact Alteryx Support.
   1. If you are completing the installation without Internet access, you must also acquire the offline versions of the system dependencies. See Install Dependencies without Internet Access.
4. VPC: Enable and deploy a working AWS VPC.

5. S3: Enable and deploy an AWS S3 bucket to use as the base storage layer for the platform. In the bucket, the platform stores metadata in the following location:

   <S3_bucket_name>/trifacta

   See https://s3.console.aws.amazon.com/s3/home.

6. IAM Policies: Create IAM policies for access to the S3 bucket. Required permissions are the following: 

   • The system account or individual user accounts must have full permissions for the S3 bucket:

     Delete*, Get*, List*, Put*, Replicate*, Restore*

   • These policies must apply to the bucket and its contents. Example:

     "arn:aws:s3:::my-trifacta-bucket-name"
     "arn:aws:s3:::my-trifacta-bucket-name/*"

   • See https://console.aws.amazon.com/iam/home#/policies
7. EC2 instance role: Create an EC2 instance role for your S3 bucket policy. See https://console.aws.amazon.com/iam/home#/roles.
8. EC2 instance: Deploy an AWS EC2 with SELinux where the Alteryx software can be installed.

   1. The required set of ports must be enabled for listening. See System Ports.

   2. This node should be dedicated for Alteryx use.

      NOTE: The EC2 node must meet the system requirements. For more information, see System Requirements.

9. EMR cluster: An existing EMR cluster is required. 
   

   1. Cluster sizing: Before you begin, you should allocate sufficient resources for sizing the cluster. For guidance, please contact your Alteryx representative.

   2. See Deploy the Cluster below.
10. Databases:
    
    1. The platform utilizes a set of databases that must be accessed from the Alteryx node. Databases are installed as part of the workflow described later.
    2. For more information on the supported databases and versions, see System Requirements.
    3. For more information on database installation requirements, see Install Databases.
    4. If installing databases on Amazon RDS an admin account to RDS is required. For more information, see Install Databases on Amazon RDS.

AWS Information

Before you begin installation, please acquire the following information from AWS:

• EMR:
  • AWS region for the EMR cluster, if it exists.
  • ID for EMR cluster, if it exists
    • If you are creating an EMR cluster as part of this process, please retain the ID.
    • The EMR cluster must allow access from the Alteryx node. This configuration is described later.
• Subnet: Subnet within your virtual private cloud (VPC) where you want to launch the Designer Cloud Powered by Trifacta platform.
  • This subnet should be in the same VPC as the EMR cluster.
  • Subnet can be private or public.
  • If it is private and it cannot access the Internet, additional configuration is required. See below.
• S3:
  • Name of the S3 bucket that the platform can use

  • Path to resources on the S3 bucket

• EC2: 
  • Instance type for the Alteryx node

Internet access

From AWS, the Designer Cloud Powered by Trifacta platform requires Internet access for the following services:

• AWS S3
• Key Management System [KMS] (if sse-kms server side encryption is enabled)
• Secure Token Service [STS] (if temporary credential provider is used)
• EMR (if integration with EMR cluster is enabled)

Deploy the Cluster

In your AWS infrastructure, you must deploy a supported version of EMR across a recommended number of nodes to support the expected data volumes of your Alteryx jobs.

• For more information on suggested sizing, see Sizing Guidelines in the Install Preparation area.

For more information on the supported EMR distributions, see  Supported Deployment Scenarios for AWS.

When you configure the platform to integrate with the cluster, you must acquire some information about the cluster resources. For more information on the set of information to collect, see Pre-Install Checklist in the Install Preparation area.

Deploy the EC2 Node

An EC2 node of the cluster must be deployed to host the Designer Cloud Powered by Trifacta platform software. For more information on the requirements of this node, see System Requirements. 

When you configure the platform to integrate with the cluster, you must acquire some information about the cluster resources. For more information on the set of information to collect, see Pre-Install Checklist in the Install Preparation area.

Here are some guidelines for deploying the EC2 cluster from the EC2 cluster:

1. Instance size: Select the instance size.
2. Network: Configure the VPC, subnet, firewall and other configuration settings necessary to communicate with the instance. 
3. Auto-assigned Public IP: You must create a public IP to access the Designer Cloud Powered by Trifacta platform.
4. EC2 role: Select the EC2 role that you created.

5. Local storage: Select a local EBS volume. The default volume includes 100GB storage.

   NOTE: The local storage environment contains the Alteryx databases, the product installation, and its log files. No source data is ever stored within the product.

6. Security group: Use a security group that exposes access to port 3005, which is the default port for the platform. 
7. Create an AWS key-pair for access: This key is used to provide SSH access to the platform, which may be required for some admin tasks.

8. Save your changes.

Install Workflow

Configure for EMR

Please complete the following configuration to enable access to your pre-existing EMR cluster from the Designer Cloud Powered by Trifacta platform.

IAM and Security Group updates

You must make changes to your IAM and Security Group changes to enable the Alteryx instance to communicate with your existing EMR cluster and your EMR cluster to read/write to the Alteryx data bucket. Below are the requirements and suggested implementation details. Please adapt these suggestions to fit your environment as long as the requirements are satisfied. 

For additional documentation around these changes:

• https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-roles.html 
• https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-additional-sec-groups.html

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Action": [
                "elasticmapreduce:DescribeStep",
                "elasticmapreduce:ListBootstrapActions",
                "elasticmapreduce:ListClusters",
                "elasticmapreduce:DescribeCluster",
                "elasticmapreduce:AddJobFlowSteps",
                "elasticmapreduce:DescribeJobFlows",
                "elasticmapreduce:ListInstanceGroups"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Action": [
                "elasticmapreduce:Describe*",
                "elasticmapreduce:List*",
                "s3:ListAllMyBuckets",
                "ec2:Describe*"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "s3:PutObject",
                "s3:ListBucket",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR-TRIFACTA-BUCKET",
                "arn:aws:s3:::YOUR-TRIFACTA-BUCKET/*"
            ],
            "Effect": "Allow"
        }
    ]
}

        {
            "Action": [
                "s3:HeadBucket",
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR-TRIFACTA-BUCKET",
                "arn:aws:s3:::YOUR-TRIFACTA-BUCKET/*"
            ],
            "Effect": "Allow"
        },

Additional configuration must be applied within the platform. These steps are described later.

Additional Configuration for AWS Installs

Apply license key to EC2 node

Steps:

1. Acquire the license.json license key file that was provided to you by your Alteryx representative.

2. Transfer the license key file to the EC2 node that is hosting the Designer Cloud Powered by Trifacta platform. Navigate to the directory where you stored it.

3. Make the Alteryx user the owner of the file:

   sudo chown trifacta:trifacta license.json

4. Make sure that the Alteryx user  has read permissions on the file:

   sudo chmod 644 license.json

5. Copy the license key file to the proper location:

   cp license.json /opt/trifacta/license/

Launch the platform

For more information on how to launch the platform, see Start and Stop the Platform.

When the instance is spinning up for the first time, performance may be slow. When the instance is up, navigate to the following:

http://<public_hostname>:3005

When the login screen appears, enter the default admin credentials provided to you.

Configure for EMR clusters

The following steps apply to configure the platform to integrate with the EMR cluster:

1. From the application menu, select the Settings menu. Then, click Settings > Admin Settings. 
2. In the Admin Settings page, you can configure many aspects of the platform, including user management tasks, and perform restarts to apply the changes.

   1. In the Search bar, enter the following:

      aws.s3.bucket.name

   2. Set the value of this setting to be S3 bucket name.

3. Check the following setting. Verify that it is set to 2.3.0:

   "spark.version": "2.3.0",

4. The following setting must be specified.

   "aws.mode":"system",

   You can set the above value to either of the following:

   aws.mode value	Description
   system	Set the mode to system to enable use of EC2 instance-based authentication for access.
   user	Set the mode to user to utilize user-based credentials. This mode requires additional configuration.

   Details on the above configuration are described later.

5. Set the following parameter to true, which instructs the Designer Cloud application to run jobs on the integrated EMR cluster:

   "webapp.runinEMR" = true,

6. In the Admin Settings page, locate the External Service Settings section. 

7. In the Admin Settings page, locate the External Service Settings section.

   1. AWS EMR Cluster ID: Paste the value for the EMR Cluster ID for the cluster to which the platform is connecting.

   2. AWS Region: Enter the region where your EMR cluster is located.
   3. Resource Bucket: Enter the name of the S3 bucket to use.
   4. Resource Path: you should use something like EMRLOGS.

8. Click Save underneath the External Service Settings section.

Set base storage layer

The platform requires that one backend datastore be configured as the base storage layer. This base storage layer is used for storing uploaded data and writing results and profiles. 

See Set Base Storage Layer.

Verify Operations

Prepare Your Sample Dataset

To complete this test, you should locate or create a simple dataset. Your dataset should be created in the format that you wish to test.

Characteristics:

• Two or more columns. 
• If there are specific data types that you would like to test, please be sure to include them in the dataset.
• A minimum of 25 rows is required for best results of type inference.
• Ideally, your dataset is a single file or sheet. 

Store Your Dataset

If you are testing an integration, you should store your dataset in the datastore with which the product is integrated.

• You may need to create a connection between the platform and the datastore.
• Read and write permissions must be enabled for the connecting user to the datastore.
• For more information, see Connections Page.

Verification Steps

Steps:

1. Login to the application. See Login.
2. In the application menu bar, click Library.
3. Click Import Data. See Import Data Page.
   1. Select the connection where the dataset is stored. For datasets stored on your local desktop, click Upload.
   2. Select the dataset.
   3. In the right panel, click the Add Dataset to a Flow checkbox. Enter a name for the new flow.
   4. Click Import and Add to Flow.

   5. Troubleshooting:   At this point, you have read access to your datastore from the platform. If not, please check the logs, permissions, and your Alteryx® configuration.

4. In the left menu bar, click the Flows icon. Flows page, open the flow you just created. See Flows Page.
5. In the Flows page, click the dataset you just imported. Click Add new Recipe.
6. Select the recipe. Click Edit Recipe.
7. The initial sample of the dataset is opened in the Transformer page, where you can edit your recipe to transform the dataset.
   
   1. In the Transformer page, some steps are automatically added to the recipe for you. So, you can run the job immediately.
   2. You can add additional steps if desired. See Transformer Page.
8. Click Run Job. 

   1. If options are presented, select the defaults.

   2. To generate results in other formats or output locations, click Add Publishing Destination. Configure the output formats and locations. 
   3. To test dataset profiling, click the Profile Results checkbox. Note that profiling runs as a separate job and may take considerably longer. 
   4. See Run Job Page.

   5. Troubleshooting: Later, you can re-run this job on a different running environment. Some formats are not available across all running environments.

9. When the job completes, you should see a success message under the Jobs tab in the Flow View page. 
   
   1. Troubleshooting: Either the Transform job or the Profiling job may break. To localize the problem, try re-running a job by deselecting the broken job type or running the job on a different running environment (if available). You can also download the log files to try to identify the problem. See Job Details Page.
10. Click View Results from the context menu for the job listing. In the Job Details page, you can see a visual profile of the generated results. See Job Details Page.
11. In the Output Destinations tab, click a link to download the results to your local desktop. 
12. Load these results into a local application to verify that the content looks ok.

Documentation

You can access complete product documentation online and in PDF format. From within the Designer Cloud application, select Help menu > Product Docs.

Next Steps

After you have accessed the documentation, the following topics are relevant to AWS enterprise infrastructure deployments.

Please review them in order.

Upgrade

For more information on upgrading your  Designer Cloud Enterprise Edition on AWS, please contact Alteryx Customer Success and Services.