View Source

Release 6.8

December 6, 2019

Welcome to Release 6.8 of . This release introduces several key features around operationalizing the platform across the enterprise. Enterprise stakeholders can now receive email notifications when recurring jobs have succeeded or failed, updating data consumers outside of the platform. This release also introduces a generalized webhook interface, which facilitates push notifications to applications such as Slack when jobs have completed. When jobs fail, users can download a much richer support bundle containing configuration files, script files, and a specified set of log files.

Macros have been expanded to now be export- and import-ready across environments. In support of this feature, the Wrangle Exchange is now available through the , where you can download macros created by others and import them for your own use. Like macros, you can now export and import flows across product editions and release (Release 6.8 or later only).

In the application, you can now use shortcut keys to navigate around the workspace and the Transformer page. And support for the Firefox browser has arrived. Read on for more goodness added with this release.

What's New

Install:

Support for ADLS Gen2 blob storage. See Enable ADLS Gen2 Access.

Workspace:

Individual users can now enable or disable keyboard shortcuts in the workspace or Transformer page. See User Profile Page.
Configure locale settings at the workspace or user level. See Locale Settings.
You can optionally duplicate the datasets from a source flow when you create a copy of it. See Flow View Page.
Create a copy of your imported dataset. See Library Page.

Browser:

Support for Firefox browser.

For supported versions, see Desktop Requirements.

Project Management:

Support for export and import of macros. See Macros Page.
- For more information on macros, see Overview of Macros.
Download and use macros available through the Wrangle Exchange. See https://www.trifacta.com/blog/crowdsourcing-macros-trifacta-wrangle-exchange/.

Operationalization:

Create webhook notifications for third-party platforms based on results of your job executions. See Create Flow Webhook Task.
Enable and configure email notifications based on the success or failure of job executions.
NOTE: This feature requires access to an SMTP server. See Enable SMTP Email Server Integration.
- For more information on enabling, see Workspace Admin Page.
  - Individual users can opt out of receiving email messages or can configure use of a different email address. See Email Notifications Page.
- For more information on enabling emails for individual flows, see Manage Flow Notifications Dialog.

Supportability:

Download logs bundle on job success or failure now contains extensive configuration information to assist in debugging. For more information, see Configure Support Bundling.

Connectivity:

Support for integration with EMR 5.8 - 5.27. For more information, see Configure for EMR.

Connect to SFTP servers to read data and write datasets. See Create SFTP Connections.

Create connections to Databricks Tables.
NOTE: This connection is supported only when the is connected to an Azure Databricks cluster.
For more information, see Create Databricks Tables Connections.

Support for using non-default database for your Snowflake stage.
- Support for ingest from read-only Snowflake databases.
- See Enable Snowflake Connections.

Import:

As of Release 6.8, you can import an exported flow into any edition or release after the build number of the export. See Import Flow.
Improved monitoring of long-loading relational sources. See Import Data Page.
NOTE: This feature must be enabled. See Configure JDBC Ingestion.

Transformer Page:

Select columns, functions applied to your source, and constants to replace your current dataset. See Select.
Improved Date/Time format selection. See Choose Datetime Format Dialog.
Tip: Datetime formats in card suggestions now factor in the user's locale settings for greater relevance.
Improved matching logic and performance when matching columns through RapidTarget.
- Align column based on the data contained in them, in addition to column name.
- This feature is enabled by default. For more information, see Overview of RapidTarget.
Improvements to the Search panel enable faster discovery of transformations, functions, and other objects. See Search Panel.

Job execution:

By default, the permits up to four jobs from the same flow to be executed at the same time. If needed, you can configure the application to execute jobs from the same flow one at a time. See Configure Application Limits.

If you enabled visual profiling for your job, you can download a JSON version of the visual profile. See Job Details Page.

Support for instance pooling in Azure Databricks. See Configure for Azure Databricks.

Language:

New trigonometry and statistical functions. See Changes to the Language.

API:

Apply overrides at time of job execution via API.
Define import mapping rules for your deployments that use relational sources or publish to relational targets.
Export and import macro definitions.
See Changes to the APIs.

Changes in System Behavior

Browser Support Policy:

For supported browsers, at the time of release, the latest stable version and the two previous stable versions are supported.

NOTE: Stable browser versions released after a given release of will NOT be supported for any prior version of . A best effort will be made to support newer versions released during the support lifecycle of the release.

For more information, see Desktop Requirements.

Install:

NOTE: In the next release of after Release 6.8, support for installation on CentOS/RHEL 6.x and Ubuntu 14.04 will be deprecated. You should upgrade the to a supported version of CentOS/RHEL 7.x or Ubuntu 16.04. Before performing the upgrade, please perform a full backup of the and its databases. See Backup and Recovery.

Support for Spark 2.1 has been deprecated. Please upgrade to a supported version of Spark.
- Support for EMR 5.6 and eMR 5.7 has also been deprecated. Please upgrade to a supported version of EMR.
- For more information, see Product Support Matrix.
To simplify the installation distribution, the Hadoop dependencies for the recommended version only are included in the software download. For the dependencies for other supported Hadoop distributions, you must download them from the and install them on the . See Install Hadoop Dependencies.
has been upgraded to use Python 3. This instance of Python has no dependencies on any Python version external to the .

Import/Export:

Flows can now be exported and imported across products and versions of products. See Changes to the Object Model.

CLI and v3 endpoints (Release 6.4):

NOTE: Do not attempt to connect to the using any version of the CLI or the v3 endpoints. They are no longer supported and unlikely to work.

In Release 6.4:

The Command Line Interface (CLI) was deprecated. Customers must use the v4 API endpoints instead.
The v3 versions of the API endpoints were deprecated. Customers must use the v4 API endpoints instead.
Developer content was provided to assist in migrating to the v4 API endpoints.
For more information on acquiring this content, please contact .

Key Bug Fixes

Ticket	Description
TD-40348	When loading a recipe in an imported flow that references an imported Excel dataset, Transformer page displays Input validation failed: (Cannot read property 'filter' of undefined) error, and the screen is blank.
TD-42080	Cannot run flow or deployment that contains more than 10 recipe jobs

New Known Issues

Ticket Description

TD-45923 Publishing a compressed Snappy file to SFTP fails.

TD-45922 You cannot publish TDE format to SFTP destinations.

TD-45492

Publishing to Databricks Tables fails on ADLS Gen1 in user mode.

TD-45273

Artifact Storage Service fails to start on HDP 3.1.

Workaround: The Artifact Storage Service can reference the HDP 2.6 Hadoop bundle JAR.

Locate the following property:
"artifact-storage-service.classpath"
Replace this value:
:%(topOfTree)s/%(hadoopBundleJar)s

With the following:

:%(topOfTree)s/conf/hadoop-site/:%(topOfTree)s/hadoop-deps/hdp-2.6/build/libs/hdp-2.6-bundle.jar

Save changes and restart the platform.

TD-45122

API: re-running job using only the wrangleDataset identifier fails even if the original job succeeds when writeSettings were specified.

Workaround: Use a full jobGroups job specification each time that you run a job.

See API JobGroups Create v4.

TD-44429

Cannot publish outputs to relational targets, receiving Encountered error while processing stream.

Workaround: This issue may be caused by the trifacta service account not having write and execute permissions to the /tmp directory on the . If so, you can do either of the following:

Enable write and execute permissions for the account on /tmp.
Create a new temporary account and provide the service account write and execute permissions to it. Then, add the following to data-service.jvmOptions:
-Dorg.xerial.snappy.tempdir=/new/directory/with/writeexecuteaccess

TD-44427

Cannot publish dataset containing duplicate rows to Teradata. Error message:

Caused by: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.10.00.14] [Error -2802] [SQLState 23000] Duplicate row error in abc_trifacta.tmp_218768523.
 at

Workaround: This is a known limitation on Teradata. For more information on this limitation, see Enable Teradata Connections.