Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

D toc

D s install marketplace

This guide steps through the requirements and process for upgrading 

D s product
productee
rtrue
 through the AWS Marketplace. 

...

Warning

Upgrading to the latest version of

d-s-product
productee
via the AWS Marketplace terminates your existing instance and creates a new instance with the latest software in its place. Please follow these instructions carefully.

...

  • If there is any problem doing these modifications, CloudFormation automatically rolls back its changes.
  • Your existing instance is preserved until the new one has been brought up successfully.

OverviewUpgrade flow:

  1. Review the Upgrade Prep before beginning.
  2. Perform two different backups.
  3. Perform the Cloudformation Stack upgrade, which makes the required changes to your environment, including bringing up a new instance with the latest version of
    d-s-product
    productee
    .
  4. Restore our your backups onto that the new version of the product.
  5. Perform any required changes to complete the upgrade.

...

Your VersionTarget VersionAMIDocumentation

D s product
productee
4.2.x

D s product
productee
5.0.x

Please see the AWS Marketplace listing for the product. The AMI is accessible from there.Trifacta Install Guide for AWS Marketplace with EMR v5.0

Upgrade Prep

  • You need to must copy your license file back onto the server after the upgrade. You should either back it up or copy the contents of /opt/trifacta/license/license.json to restore it after the upgrade.
  • This process requires broad permissions on your AWS account. If you do not have Administrator access, you may encounter errors when Cloudformation tries to modify AWS objects like IAM or Security Groups.
  • The EMR cluster is replaced with a new one featuring autoscaling groups and configurable sizing.
  • If you have made additional tweaks to the default installation, these changes are likely to be lost. Please review and note any changes, so you can replicate them after upgrade:
    • IAM role or policy changes
    • Changes on the
      D s server
      OS itself
    • SSL certificates and configuration
    • The IP address of your
      D s server
      will change. This change requires a DNS update after the upgrade is complete.
    • Your
      D s item
      itemlicense file

Back up your existing
D s item
iteminstance

Backup

This process creates two backups: one of the

D s item
itemsoftware
and one of the entire EC2 Instance.

  1. SSH to your current Marketplace instance. Example:

    Code Block
    ssh -i MyKey.pem centos@TrifactaServer.MyCompany.net


  2. Switch to root Root user on the server:

    Code Block
    sudo su


  3. Stop the 

    D s platform
     on on your current Marketplace instance:

    Code Block
    service trifacta stop


  4. Run the

    D s item
    itembackup script
    :

    Code Block
    /opt/trifacta/bin/setup-utils/trifacta-backup-config-and-db.sh


    1. When the script is complete, the output identifies the location of the backup. Example:

      Code Block
      /opt/trifacta-backups/trifacta-backup-5.0+126.20171217124021.a8ed455-20180514213601.tgz


  5. Store the backup in a safe location. You should copy it to either S3 or to your local computer via SCP.

    1. To copy the backup to the S3 bucket used by your installation, you can use this example:

      Code Block
      aws s3 cp /opt/trifacta-backups/trifacta-backup-5.0+126.20171217124021.a8ed455-20180514213601.tgz s3://<my-trifacta-s3-bucket>/trifacta-backups/


    2. If you choose to use SCP, please note that the AMI does not allow root login. You must copy the files you want to the CentOS user's home directory and perform modify any required permission changes permissions to allow the CentOS user to read them. After that, you can use SCP to copy the files. Example:

      Code Block
      On the server: chown centos:centos mybackupfile.tgz
      On your workstation: scp -i <my-key.pem> centos@<my-server-ip>:./example-file.txt ./


  6. You should also take a snapshot of the EBS volume backing your EC2 instance.
    1. This backup is not necessary to restore the
      D s platform
      , but it can be useful if you find that you had additional files or configurations to replicate on your new
      D s item
      iteminstance
      .

...

Please complete the following steps to upgrade the Cloudformation Stack. These steps create an instance of the latest version of

D s product
productee
and then performs any necessary adjustments to your existing resources.

Steps:

  1. Visit the Trifacta Marketplace listing page for your product.

  2. Under View Usage Instructions, expand the View CloudFormation Template section.
  3. Right-click Download CloudFormation Template. Copy its URL.
  4. In your AWS Console, go to CloudFormation.
  5. Select your Trifacta Stack
    D s item
    itemStack
    . Click Update.
  6. Select Replace Current Template.
  7. Select Amazon S3 URL, and paste the link to the template in the textbox.
  8. Click Next to review the parameters. In the latest version of the CloudFormation template we have updated the EMR cluster with some new features:

    1. Configurable autoscaling groups
    2. Configurable instance sizes
    3. Defaults are provided, but feel free to modify these values as necessary.
  9. Wait until the CloudFormation Stack indicates it's upgradedthe upgrade is finished.

Restore your backup onto the new instance

  1. Connect to your new

    D s item
    iteminstance
    via SSH.

    1. If you receive an SSH fingerprint warning, it is expected when connecting to a new instance.

    2. To suppress this warning, remove the relevant entry from the following file: ~/.ssh/known_hosts.
  2. Switch to root Root user on the

    D s item
    iteminstance
    :

    Code Block
    sudo su
    cd


  3. Restore your license file or create a new license file with the contents you copied earlier. Review and update the permissions and ownership appropriately:

    Code Block
    chown trifacta:trifacta /opt/trifacta/license/license.json
    chmod 755 /opt/trifacta/license/license.json


  4. Download the latest copy of the restore script to pick up additional fixes that have been made.

    Info

    NOTE: This step is required.


    Code Block
    curl --output trifacta-restore-from-backup.sh https://raw.githubusercontent.com/trifacta/trifacta-utils/release/6.0/trifacta-restore-from-backup.sh
    
    mv trifacta-restore-from-backup.sh /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh
    
    chown trifacta:trifacta /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh
    chmod 775 /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh


  5. Download the backup from your storage location and extract its contents. Example:

    Code Block
    mkdir -p /root/trifacta-restore-files
    cd /root/trifacta-restore-files
    aws s3 cp s3://<my-trifacta-s3-bucket>/trifacta-backups/<my-backup-file.tgz> .
    tar xzf <my-backup-file.tgz>


  6. Execute the restore script. Pass in the path to your unzipped backup as a parameter. Example:

    Code Block
    /opt/trifacta/bin/setup-utils/trifacta-restore-from-backup.sh -r /root/trifacta-restore-files/trifacta-backup-5.1+126.20171217124021.a8ed455-20180514213601


  7. Start up the platform:

    Code Block
    service trifacta start


  8. Login to the
    D s webapp
    .
  9. In the menu, navigate to Settings > Admin Settings.

    1. In the External Service Settings area, update your EMR cluster ID. This value is available from the Outputs tab.
    2. Under the Platform Settings area, search for spark.version. Verify that it is 2.3.0.
  10. In some circumstances, old EMR support .jars files are not correctly overwritten by the
    D s item
    itemsoftware
    , leading to EMR job failures. These should be remove to ensure that there are no problems:
    1. In the AWS S3 console, navigate to the following:

      Code Block
      <Resource Bucket>/<Resource Path>/trifacta/libs/


    2. These are located in the External Service Settings section.
    3. You should see about 6 files.
      Select all the files within this folder and delete them.
    4. When a job is next run in the
      D s platform
      , these files are replaced.
  11. Verify that the product is working as expected by running jobs.

...

  1. D s config
    methodt
  2. Locate the  data-service.classpath setting. Change the class path value to point to the correct directory:

    Code Block
    /opt/trifacta/conf/data-service


  3. Locate the webapp.connectivity.kerberosDelegateConfigPath setting. If you are enabling Kerberos-based SSO for relational connections, please add the following value to the path:

    Code Block
    "%(topOfTree)s/services/data-service/build/conf/kerberosdelegate.config"

    For more information, see see Enable SSO for Relational Connections.

  4. Save the file.

...

Update MySQL JAR for Configuration Service

Info

Applies if: You have installed the

D s item
itemdatabases
on MySQL.

When upgrading to Release 6.0.x, there is a known issue in which the MySQL driver JAR is not properly installed for the new Configuration Service. This causes a No suitable driver found error for the trifactaconfigurationservice.

The fix is to apply copy the MySQL driver to the correct location for Configuration Service in Release 6.0.x.

...

  1. Login to the 
    D s node
    .
  2. Locate the MySQL driver. A version of it should be available in one of the following locations:

    Code Block
    /opt/trifacta/services/batch-job-runner/build/install/batch-job-runner/lib/mysql-connector-java-6.0.6.jar
    /opt/trifacta/services/scheduling-service/server/build/install/scheduling-service/lib/mysql-connector-java-6.0.6.jar
    /opt/trifacta/services/time-based-trigger-service/server/build/install/time-based-trigger-service/lib/mysql-connector-java-6.0.6.jar


  3. Relocate the driver to the following location:

    Code Block
    /opt/trifacta/services/configuration-service/build/install/configuration-service/lib/mysql-connector-java-6.0.6.jar


For more information on MySQL installation, see see Install Databases for MySQL.

SSO signout updates

Info

Applies if: You have enabled SSO for AD/LDAP and have noticed that logout is not working.

...

Some properties related to the reverse proxy must be updated. Please complete the following:

Steps:

  1. Login to the 
    d-s-item
    item
    node
    .
  2. Edit the following file:

    Code Block
    /opt/trifacta/pkg3p/src/tripache/conf/conf.d/trifacta.conf


  3. Add the following rule for the /unauthorized path:
    Image Removed Image Added

  4. Modify the redirection for /sign-out from / to /unauthorized. Remove the rewrite rule:
    Image Removed Image Added
  5. Save the file and restart the platform.

...

  • The email address with the highest Id value in the People table is assumed to be the original user account.
  • The format for email addresses is: 

    Code Block
    <orig_userId>_duplicate_<row_id>

    where <row_id> is the row in the table where the duplicate was detected.

After all migrations have completed, you should review the migration logs. Search the logs for the following:  all-emails-for-people-to-lower-case .

A set of users without duplicates has the following entry:

...

For more information on this known issue, see the Troubleshooting section in in Configure for Spark.

Documentation

You can access complete product documentation online and in PDF format. From within the product, select Help menu > Product Docs.

...