Page tree

 

Contents:


This documentation applies to installation from a supported Marketplace. Please use the installation instructions provided with your deployment.


If you are installing or upgrading a Marketplace deployment, please use the available PDF content. You must use the install and configuration PDF available through the Marketplace listing.

This guide steps through the requirements and process for upgrading  Trifacta® Data Preparation for Amazon Redshift and S3 through the AWS Marketplace. 

Upgrade

Process preview

Upgrading to the latest version of Trifacta Data Preparation for Amazon Redshift and S3 via the AWS Marketplace will terminate your existing instance and bring up a new instance with the latest software in its place. Please follow the instructions here carefully.

Your Trifacta instance is deployed as part of a Cloudformation Stack. When you upgrade this stack, Cloudformation informs you of the resources it plans on modifying and then manages the modifications. If there is any problem doing these modifications, Cloudformation rolls back its changes. Your existing instance is not deleted until the new one has been brought up successfully.

Upgrade flow:

  1. Upgrade prep: Review some items that you should know before upgrading Trifacta Data Preparation for Amazon Redshift and S3.
  2. Backup: Before performing the Cloudformation Stack upgrade, you perform two different backups.
  3. Upgrade: You perform the Cloudformation Stack upgrade which also makes the required changes to your environment, including bringing up a new instance which has the latest version of the software.
  4. Restore: You restore your backups onto the new version of the software.
  5. Post-Upgrade Fixes: You make any changes necessary to get up and running again.

Supported paths

This upgrade process supports upgrade for the following versions:

Source VersionTarget Upgrade Version

Trifacta Data Preparation for Amazon Redshift and S3 5.x

Trifacta Data Preparation for Amazon Redshift and S3 6.0.x

If you are upgrading from a version that is earlier than the supported Source Version listed above for this upgrade process, please use the links below to acquire the AMI(s) and documentation to upgrade to the earliest supported Source Version. Then, return to these instructions to complete the process.

Your VersionTarget VersionAMIDocumentation

Trifacta Data Preparation for Amazon Redshift and S3 4.2.x

Trifacta Data Preparation for Amazon Redshift and S3 5.0.x

Please see the AWS Marketplace listing for the product. The AMI is accessible from there.Trifacta Install Guide for AWS Marketplace v5.0

 

Upgrade Prep

  • This process requires broad permissions on your AWS account. If you do not have Administrator access, you may encounter errors when Cloudformation tries to modify AWS objects, like IAM or Security Groups.
  • If you have made additional tweaks to the default installation, these changes are likely to be lost. Please review and note any changes, so you can replicate them if necessary.
    • IAM role or policy changes
    • Changes on the Trifacta Server OS itself
    • SSL certificates and configuration
    • The IP address of your Trifacta Server is changing. This change requires a DNS update after the upgrade is complete.

Backup

We'll make two backups: one of the Trifacta software and one of the entire EC2 Instance.

  1. SSH to your current Marketplace instance. Example:

    ssh -i MyKey.pem centos@TrifactaServer.MyCompany.net
  2. Switch to Root user on the server:

    sudo su
  3. Stop the Trifacta platform on your current Marketplace instance:

    service trifacta stop
  4. Run the Trifacta backup script:

    /opt/trifacta/bin/setup-utils/trifacta-backup-config-and-db.sh

     

    1. When the script is complete, the output identifies the location of the backup. Example:

      /opt/trifacta-backups/trifacta-backup-5.0+126.20171217124021.a8ed455-20180514213601.tgz
  5. Store the backup in a safe location. You should copy it to either S3 or to your local computer via SCP.

    1. To copy the backup to the S3 bucket used by your installation, you can use this example:

      aws s3 cp /opt/trifacta-backups/trifacta-backup-5.0+126.20171217124021.a8ed455-20180514213601.tgz s3://<my-trifacta-s3-bucket>/trifacta-backups/
    2. If you choose to use SCP, please note that the AMI does not allow root login. You must copy the files to the CentOS user's home directory and modify any permissions to allow the CentOS user to read them. Then, you can use SCP to copy the files. Example:

      scp -i <my-key.pem> centos@<my-server-ip>:./example-file.txt ./
  6. You should also take a snapshot of the EBS volume backing your EC2 instance.
    1. This step is not needed to restore the product, but it can be useful if you have additional files or configurations to replicate to your new Trifacta instance.

Perform the Cloudformation Stack upgrade

In this section, you upgrade the Cloudformation Stack. This upgrade launches the latest version of the Trifacta software and performs any necessary adjustments to your existing resources.

Steps:

  1. Upgrade the Cloudformation Stack.

  2. Visit the Marketplace listing page for your product.

  3. Under View Usage Instructions, expand the View CloudFormation Template section.
  4. Right-click the Download CloudFormation Template link and copy its URL.
  5. In your AWS Console, go to CloudFormation.
  6. Select your Trifacta Stack.
  7. Click Update.
  8. Select Replace Current Template.
  9. Select Amazon S3 URL, and paste the link to the template in the textbox.

  10. Click Next and review the parameters, which should be inherited from your existing Stack.

  11. Wait until the CloudFormation Stack indicates that the upgrade is finished.
  12. Access the Trifacta software via the URL in the Outputs tab of CloudFormation.
  13. When you reach the the Login page, the software has started, and you can continue with the following sections.

Restore your backup onto the new instance

  1. Connect to your new Trifacta instance via SSH.

    NOTE: The IP address has changed.

  2. Switch to Root user on the Trifacta instance.

    sudo su
    cd
  3. The Trifacta software starts automatically on boot. Stop it before continuing:

    service trifacta stop
  4. Download the latest copy of the restore script to pick up additional fixes that have been made.

    NOTE: This step is required.


    curl --output trifacta-restore-from-backup.sh https://raw.githubusercontent.com/trifacta/trifacta-utils/release/6.0/trifacta-restore-from-backup.sh
    
    mv trifacta-restore-from-backup.sh /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh
    
    chown trifacta:trifacta /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh
    chmod 775 /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh
  5. Download the backup from your storage location and extract its contents. Example:

    mkdir -p /root/trifacta-restore-files
    cd /root/trifacta-restore-files
    aws s3 cp s3://<my-trifacta-s3-bucket>/trifacta-backups/<my-backup-file.tgz> .
    tar xzf <my-backup-file.tgz>
  6. Execute the restore script. Pass in the path to your unzipped backup as a parameter. Example:

    /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh -r /root/trifacta-restore-files/trifacta-backup-5.1+126.20171217124021.a8ed455-20180514213601
  7. Start up the product:

    service trifacta start
  8. Verify that the product is working as expected by running jobs.

Appendix - Post-Upgrade Fixups

Upgrade from Release 5.1 or earlier

Photon scaling factor has been removed

Applies if: You modified the Photon scaling properties in your pre-upgrade environment.

In Release 6.0, the Photon scaling factor parameter (photon.loadScalingFactor) has been removed. As part of this change, the following parameters are automatically set to new values as part of the upgrade. The new values are listed below:

"webapp.client.loadLimit": 10485760,
"webapp.client.maxResultsBytes": 41943040,

NOTE: If you had not modified either of the above values previously, then no action is required. If you had changed these values before upgrading, the settings are set to the new default values above.

Update Data Service properties

Applies if: You modified the Data Service classpath in your pre-upgrade environment.

After you have upgraded, the Data Service fails to start.

In Release 6.0.0, some configuration files related to the Data Service were relocated, so the classpath values pointing to these files need to be updated. 

Steps:


  1. To apply this configuration change, login as an administrator to the Trifacta node. Then, edit trifacta-conf.json. Some of these settings may not be available through the Admin Settings Page. For more information, see Platform Configuration Methods.
  2. Locate the  data-service.classpath setting. Change the class path value to point to the correct directory:

    /opt/trifacta/conf/data-service
  3. Locate the webapp.connectivity.kerberosDelegateConfigPath setting. If you are enabling Kerberos-based SSO for relational connections, please add the following value to the path:

    "%(topOfTree)s/services/data-service/build/conf/kerberosdelegate.config"

    For more information, see Enable SSO for Relational Connections.

  4. Save the file.

SSO signout updates

Applies if: You have enabled SSO for AD/LDAP and have noticed that logout is not working.

After upgrading to this release, signing out of the Trifacta application may not work when SSO is enabled. This issue applies to the reverse proxy method of SSO for AD/LDAP. 

NOTE: Beginning in Release 6.0, a platform-native method of SSO is available. This new method is recommended.

Some properties related to the reverse proxy must be updated. Please complete the following:

Steps:

  1. Login to the Trifacta node.
  2. Edit the following file:

    /opt/trifacta/pkg3p/src/tripache/conf/conf.d/trifacta.conf
  3. Add the following rule for the /unauthorized path:

  4. Modify the redirection for /sign-out from / to /unauthorized. Remove the rewrite rule:
  5. Save the file and restart the platform.

Troubleshooting - disambiguating email accounts

Applies if: Applies to all upgrades to Release 6.0.1 and later.

In Release 6.0.1, the Trifacta platform permitted case-sensitive email addresses. So for purposes of creating user accounts, the following could be different userIds in the platform. Pre-upgrade, the People table might look like the following:

| <Id> | <Email> | other columns |
| 1 | foobar@trifacta.com | * |
| 2 | FOOBAR@trifacta.com | * |
| 3 | FooBar@trifacta.com | * |

Beginning in Release 6.0.2, all email addresses (userIds) are case-insensitive, so the above distinctions are no longer permitted in the platform. 

As of Release 6.0.2, all email addresses are converted to lower-case. As part of the upgrade to Release 6.0.2, any email addresses that are case-insensitive matches (foobar and FOOBAR) are disambiguated. After upgrade the People table might look like the following:

| <Id> | <Email> | other columns |
| 1 | foobar@trifacta.com_duplicate_1 | * |
| 2 | foobar@trifacta.com_duplicate_2 | * |
| 3 | foobar@trifacta.com | * |

Notes:

  • The email address with the highest Id value in the People table is assumed to be the original user account.
  • The format for email addresses is: 

    <orig_userId>_duplicate_<row_id>

    where <row_id> is the row in the table where the duplicate was detected.

After all migrations have completed, you should review the migration logs. Search the logs for the following: all-emails-for-people-to-lower-case.

A set of users without duplicates has the following entry:

== 20181107131647-all-emails-for-people-to-lower-case: migrating =======
== 20181107131647-all-emails-for-people-to-lower-case: migrated (0.201s)

Entries like the following indicate that duplicate addresses were found for separate accounts. The new duplicate Ids are listed as part of the message:

== 20181107131647-all-emails-for-people-to-lower-case: migrating =======
warn: List of duplicated emails: foobar@trifacta.com_duplicate_1, foobar@trifacta.com_duplicate_2
== 20181107131647-all-emails-for-people-to-lower-case: migrated (0.201s)

NOTE: The above log entry indicates that there are duplicated user accounts.

Suggested approach:

  1. Change ownership on all flows in secondary accounts to the primary account.

  2. Delete secondary accounts.

Documentation

You can access complete product documentation online and in PDF format. From within the product, select Help menu > Product Docs.

This page has no comments.