Release Notes for Trifacta Wrangler Pro

Contents:

March 22, 2021

Release 8.2

What's New

• Re-organized user account, preferences, and storage settings to streamline the setup process. See Preferences Page.

API:

• You can now transfer ownership of assets created in Trifacta Wrangler Pro between users, based on their user identifiers or email addresses. For more information, see Changes to the APIs.

Connectivity:

Early Preview (read-only) connections available with this release:

• Amazon Athena
• Amazon DynamoDB
• Greenplum
• HubSpot
• MariaDB
• Microsoft Dynamics 365 Sales
• ServiceNow
• Smartsheet
• For more information, see Early Preview Connection Types.

Plan metadata references:

Use metadata values from other tasks and from the plan itself in your HTTP task definitions.

• For more information, see Create HTTP Task.
• For more information, see Plan Metadata References.

Spark Execution Properties Settings:

Advanced Spark parameters have been newly added to the Spark running environment. Use these parameters to override the global Spark settings for individual jobs. For more information, see Spark Execution Properties Settings.

Improve Accessibility of Job Results:

The Jobs tabs have been enhanced to display the list of latest and the previous jobs that have been executed for the selected output.

For more information, see View for Outputs.

Sample Jobs Page:

You can monitor the status of all sample jobs that you have generated. Project administrators can access all sample jobs in the workspace. For more information, see Sample Jobs Page.

Changes in System Behavior

None.

Key Fixes

New Known Issues

None.

December 21, 2020

Release 7.10

What's New

Import:

• The maximum permitted size of a file uploaded through Trifacta application has been increased from 100 MB to 1 GB.

Plan View:

• Import and Export Plans: You can import and export plans from one environment, workspace, or projects to others.

For more information, see Export Plan.

For more information, see Import Plan.

• Share Plans: Share plans with one or more users to work together on the same plan. For more information, see Share a Plan.
• Email notifications: Send email notifications to plan owners and collaborators based on the status of execution of plans. For more information, see Manage Plan Notifications Dialog.

Photon:

Introducing Trifacta Photon, an in-memory running environment for running jobs. Embedded in the Trifacta Wrangler Pro, Trifacta Photon is fast and best-suited for small- to medium-sized jobs.

• When you choose to run a job, you can now choose to run a job on Trifacta Photon.
• Trifacta Wrangler Pro specifies the most appropriate running environment as the default selection based on the size of your job.
• For more information, see Run Job Page.

Sampling:

• Trifacta Photon also enables quick scan sampling. A quick scan sample generates an appropriate selection of rows from the dataset from which to perform the sample. These samples are faster to generate. For more information, see Overview of Sampling.
• For more information on generating samples, see Samples Panel.

Sharing:

• Define permissions on individual objects when they are shared.
  
  

  NOTE: Fine-grained sharing permissions apply to flows and connections only.

  For more information, see Changes to User Management.

Authentication:

• Support for authentication to third-party datastores using OAuth2. For more information, see Enable OAuth 2.0 Authentication.
• Support for using OAuth2 for Snowflake connections. For more information, see OAuth 2.0 for Snowflake.

Connectivity:

• Improved Salesforce connection type.
  
  

  For more information, see Create Salesforce Connections.

API:

• Experimental feature: Export Python Pandas code to generate the transformation steps required to produce a defined output object.
  

• NOTE: This feature can be changed or removed from the platform at any time without notice. Do not deploy it in a production environment.
  
• For more information, see API Workflow - Wrangle Output to Python.

Changes in System Behavior

New public bucket for product assets

In prior releases, the product referenced the following S3 bucket, which contained demo assets:

trifacta-public-datasets

Beginning in this release, please enable access through your IAM role to the following buckets:

trifacta-public-datasets
3fac-data-public
aws-saas-datasets

For more information, see Getting Started with Trifacta Wrangler Pro.

Key Bug Fixes

New Known Issues

July 20, 2020April 15, 2020

Release 7.5Release 7.1

What's NewWhat's New

New Flow View is now generally available:API Documentation:

• Drag and drop to reposition objects on the Flow View canvas, and zoom in and out to focus on areas of development.

  API reference documentation is now available directly through the application. This release includes more supported endpoints and documented options. To access, select Help menu > API Documentation.

  Workflow documentation is available with the product documentation. For more information, see API Reference. 

• Perform joins and unions between objects on the Flow View canvas. 
• Search for flow objects by name or by type.

• Annotate the canvas with notes.

  Tip: The relative position of objects on the flow view canvas is preserved between screen updates. On refresh, the window on the canvas is repositioned based on the leftmost object on the canvas to focus on the flow to other objects from that one.

  NOTE: Classic Flow View is no longer available.

  See Flow View Page .

Import:

• For long-loading relational datasets, you can monitor the ingest process through Flow View as you continue your work.

  For more information, see Flow View Page.

• Improved performance when browsing databases for tables to import.

  Tip: Performance improvements are due to limiting the volume of table metadata that is imported when paging through available tables. This metadata can be retrieved when you hover over a table in the database browser.

  For more information, see Database Browser.

• Logical and physical optimizations when reading from relational sources during job execution, which includes column pruning push-down among other enhancements. 

  NOTE: This feature may need to be enabled in your workspace. See Workspace Settings Page.

  This feature applies to the following relational connections in this release:

  • Create Oracle Connections

  • Create SQL Server Connections

  • Create PostgreSQL Connections

  • Create Redshift Connections

  • Create Snowflake Connections

Transformer Page:

• Collaborative suggestions allow users within a workspace to receive suggestions based on the transformations that have been recently created by themselves or by all members of the workspace. As more users generate transformations, the relevance of these suggestions to the data in the workspace continues to improve.

  • This feature must be enabled for individual users or across the workspace. See Workspace Settings Page.

  • When the feature is enabled, individual users can choose to not share their transformation data. See User Profile Page.

  • For more information, see Overview of Predictive Transformation.

• Create and edit flow parameters and their overrides while editing your recipe. See Transformer Page.
  • For more information on editing flow parameters, see Manage Parameters Dialog.

Plans:

• Monitor status of all of your plan runs and drill into details. See Plan Runs Page.

Authorization:

• Workspace administrators can now create and assign roles to govern access to types of objects in the workspace. For more information, see Changes to User Management. 

Language:

• New approximation functions for median, percentile, and quartile based on a very fast algorithm:
  • APPROXIMATEMEDIAN Function
  • APPROXIMATEPERCENTILE Function
  • APPROXIMATEQUARTILE Function
• New functions to encode and decode base64 strings.
  • BASE64ENCODE Function
  • BASE64DECODE Function

Changes in System Behavior

Schema information is retained:

• When schematized datasources are ingested, schema information is now retained for publication of job results.

  NOTE: In prior releases, you may have set column data types manually because this schema information was lost during the ingest process. You may need to remove these manual steps from your recipe. For more information, see Improvements to the Type System.

Enhanced PII masking:

• For social security numbers and credit card numbers, the methods by which these values are determined for purposes of masking sensitive Personally Identifiable Information (PII) has been expanded and improved. For more information, see Improvements to the Type System.

Key Fixes

None.

New Known Issues

Import:

• Ingestion of large relational datasets is no longer a blocking operation. Track progress of large-scale ingestion in Flow View and the Library page. 
  • See Flow View Page.
  • See Import Data Page.

Flow View:

• Create flow parameters that you can reference in the recipes of your flow. 

  NOTE: For this release, flow parameters can be applied into your recipes only.

  • As needed, you can apply overrides to the parameters in your flow or to downstream flows. 

    NOTE: Flow parameters do not apply to datasets or output objects, which have their own parameters. However, if you specify an override at the flow level, any parameters within the flow that use the same name receive the override value, including output object parameters and datasets with parameters.

  • See Manage Parameters Dialog.
  • For more information on parameters, see Overview of Parameterization.

Transformation:

• Faster and improved method of surfacing transform suggestions based on machine learning.

Language:

Some functions now accept Datetime values: Beginning in this release, the following functions now accept Datetime values as inputs. When a Datetime value is input, output values are in unixtime format.

• MIN Function
• MINIF Function
• ROLLINGMIN Function
• MAX Function
• MAXIF Function
• ROLLINGMAX Function
• MODE Function
• MODEIF Function
• ROLLINGMODE Function
• KTHLARGEST Function
• KTHLARGESTIF Function
• ROLLINGKTHLARGEST Function
• KTHLARGESTUNIQUE Function
• KTHLARGESTUNIQUEIF Function
• ROLLINGKTHLARGESTUNIQUE Function

Date calculation functions:

Work day functions:

Time zone conversion functions:

Changes in System Behavior

Authorization:

• All Trifacta admin users are now workspace admins. 

• All workspace admins now have access to all user-created objects within the workspace.

  NOTE: Workspace admin access is at the owner level to each user-created object in the workspace. Under some conditions, workspace admins may have access to source datasets and generated results. See Workspace Admin Permissions.

Key Bug Fixes

None.

New Known Issues

March 17, 2020

Release 7.0

Beginning in this release, you can access publicly available APIs of the Trifacta platform to run and monitor jobs. Numerous bug fixes and performance improvements. 

What's New

APIs:

Import:

• Ingestion of large relational datasets is no longer a blocking operation. 
  • Track progress of large-scale ingestion in Flow View and the Library page. 
    • See Flow View Page.
    • See Import Data Page.
• Import tabular data from PDF documents. 
  
  
  NOTE: This feature is in Beta release.
• See Import PDF Data.

Transformer Page:

• Improved performance when loading the Transformer page and when navigating between the Flow View and Transformer pages.
• Join steps are now created in a larger window for more workspace. See Join Window.

Changes in System Behavior

None.

Key Bug Fixes

None.

New Known Issues

None.

February 19, 2020

Release 6.11

This release introduces in-app chat and content suggestions within the application. Give it a try!

What's New

In-app chat: 

Language:

• New functions to parse values against specific data types:

  Function Name	Description
  PARSEBOOL Function	Evaluates a String input against the Boolean datatype. If the input matches, the function outputs a Boolean value. Input can be a literal, a column of values, or a function returning String values.
  PARSEFLOAT Function	Evaluates a String input against the Decimal datatype. If the input matches, the function outputs a Decimal value. Input can be a literal, a column of values, or a function returning String values.
  PARSEINT Function	Evaluates a String input against the Integer datatype. If the input matches, the function outputs an Integer value. Input can be a literal, a column of values, or a function returning String values.

Changes in System Behavior

None.

Key Fixes

None.

New Known Issues

None.

January 23, 2020

Release 6.10

This release delivers enhancements to job monitoring, visual profiles that are downloadable in PDF format, and a useful feature to kickoff any support inquiries.

What's New

T roubleshooting:

• Encounter an issue with the product? Select Help > Report issue to collect session information for  Trifacta Support. See Report Issue.

Flow View:

• Monitor job progress through each phase in the Jobs panel. See Flow View Page.

Job Details:

• When visual profiling is enabled for a job, you can now download your visual profile in PDF format. See Job Details Page. 

Documentation:

• New content on the getting started with sampling. See Sampling Basics.
  • Feature overview: Overview of Sampling
  • Best practices: https://community.trifacta.com/s/article/Best-Practices-Managing-Samples-in-Complex-Flows

Changes in System Behavior

Sharing:

• The Send a Copy feature is no longer available in the product. As a workaround, you can make a copy of the flow and share it. See Flow View Page.

Key Bug Fixes

None.

New Known Issues

None.

December 17, 2019

Release 6.9

This release introduces new statistical functions, including co-efficient and covariance calculations for enhanced analysis. A redesigned User menu facilitates easier management of user and workspace settings.

What's New

Transformer Page:

• New column selection UI simplifies choosing columns in your transformations. See Transform Builder. 

Workspace:

• Redesigned Settings and Help menus. See Home Page.

• • User settings are now modified through Preferences. See Preferences Page.
  • Administrators now have a dedicated admin area. See Admin Console.

Language:

• New statistical functions. 

  Function Name	Description
  MEDIAN Function	Computes the median from all row values in a column or group. Input column can be of Integer or Decimal.
  PERCENTILE Function	Computes a specified percentile across all row values in a column or group. Input column can be of Integer or Decimal.
  QUARTILE Function	Computes a specified quartile across all row values in a column or group. Input column can be of Integer or Decimal.
  CORREL Function	Computes the correlation coefficient between two columns. Source values can be of Integer or Decimal type.
  COVAR Function	Computes the covariance between two columns using the population method. Source values can be of Integer or Decimal type.
  COVARSAMP Function	Computes the covariance between two columns using the sample method. Source values can be of Integer or Decimal type.

Changes in System Behavior

Language:

All MODE functions return the lowest value in a set of values if there is a tie in the evaluation.

In prior releases, these functions returned a null value in the event of a tie for most common value among the evaluated ones.

• See MODE Function.
• See MODEIF Function.
• See LISTMODE Function.
• See ROLLINGMODE Function.

Key Bug Fixes

None.

New Known Issues

None.

November 13, 2019

Release 6.8

This release features enhancements to the user experience in the workspace, in the form of keyboard shortcuts and user-specific locale settings. You can also receive email notifications based on the outcome of your job executions. Flows and macros can be exported and imported into different editions and releases of the product. Read on!

What's New

Workspace:

• Individual users can now enable or disable keyboard shortcuts in the workspace or Transformer page. See User Profile Page.
• Configure locale settings at the workspace or user level. See Locale Settings.
• You can optionally duplicate the datasets from a source flow when you create a copy of it. See Flow View Page.
• Create a copy of your imported dataset. See Library Page.

Transformer Page:

• Create a new table from selected columns, functions applied to your source, and constants to replace your current dataset. See Select.

• Improved Date/Time format selection. See Choose Datetime Format Dialog.

  Tip: Datetime formats in card suggestions now factor in the user's locale settings for greater relevance.

• Improved matching logic and performance when matching columns through RapidTarget. See Overview of RapidTarget.

Job execution:

• If you enabled visual profiling for your job, you can download a JSON version of the visual profile. See Job Details Page.

• Enable and configure email notifications based on the success or failure of job executions. 

  • For more information on enabling, see Workspace Settings Page.
    • Individual users can opt out of receiving email messages or can configure use of a different email address. See Email Notifications Page.
  • For more information on enabling emails for individual flows, see Manage Flow Notifications Dialog.

Language:

This release introduces the sampling method of calculating statistical functions. The following are now available:

Changes in System Behavior

Import/Export:

• Flows can now be exported and imported across products and versions of products. See Export Flow.

Key Bug Fixes

New Known Issues

None.

October 9, 2019

Release 6.7

This release introduces two key management features. You can now export and import macros into other instances of  Trifacta Wrangler Pro, which enables broader adoption of your data wrangling practices. You can also create webhook notifications to be sent to a third-party application based on the success or failure of your job. 

What's New

Browser:

• Support for Firefox browser. 
  
  NOTE: This feature is in Beta release.
  
  For supported versions, see Desktop Requirements.

Project Management:

• Support for export and import of macros. See Macros Page.
  • For more information on macros, see Overview of Macros.

Operationalization:

• Create webhook notifications for third-party platforms based on results of your job executions. See Create Flow Webhook Task.

Connectivity:

• Support for ingest from read-only Snowflake databases. See Create Snowflake Connections.

Changes in System Behavior

None.

Key Bug Fixes

None.

New Known Issues

None.

September 11, 2019

Release 6.6

This release introduces new functions to the language and improvements in search for transformations.

What's New

Transformer Page:

• Improvements to the Search panel enable faster discovery of transformations, functions, and other objects. See Search Panel.

Language:

Changes in System Behavior

None.

Key Bug Fixes

New Known Issues

None.

August 14, 2019

Release 6.5

This release introduces new functions and fixes a number off issues.

What's New

Import:

• Scan nested folders to import datasets with parameters. See Create Dataset with Parameters.

Language:

Connectivity:

• Support for using non-default database for your Snowflake stage. See Create Snowflake Connections.

Changes in System Behavior

None.

Key Bug Fixes

New Known Issues

None.

July 18, 2019

Release 6.4

This release introduces macros, which enable reuse and parameterization of common sequences of recipe steps.

What's New

Reuse:

• Introducing user-defined macros, which enable saving and reusing sequences of steps. For more information, see Overview of Macros.

Connectivity:

• Support for direct publishing to Tableau Server. For more information, see Run Job Page.

• Support for Snowflake database connections. See Create Snowflake Connections.

Transformation:

• New cluster view for transforming by example. See Transformation by Example Page.

Job Execution:

• Support for detailed job monitoring for ingest and publishing jobs. See Overview of Job Monitoring.
• Parameterize output paths and table and file names. See Run Job Page.

Wrangle:

• Broader support for metadata references: For Excel files, $filepath references now return the location of the source Excel file. Sheet names are appended to the end of the reference. See Source Metadata References .

Changes in System Behavior

Documentation:

• In prior releases, the documentation listed UTF32-BE and UTF32-LE as supported file formats. These formats are not supported. Documentation has been updated to correct this error. See Supported File Encoding Types. 

Key Bug Fixes

New Known Issues

None.

June 17, 2019

Release 6.3

This release features a brand-new way to develop transformations via example. Map source column values to your entered values, and the platform can infer similar pattern-based matches in your column of data. Also, multi-step operations are now available in the Recipe panel. For administrators: you can now review the list of all available schedules in the workspace.  

What's New

• Transform by example output values for a column of values. See Overview of TBE.

Transform:

• The redesigned Recipe panel enables multi-step operations and more robust copy and paste actions. See Recipe Panel.

Connectivity:

• Publishing (writeback) is now supported for relational connections. The following connection types are natively supported for publishing to relational systems.

• • Oracle Data Type Conversions
  • Postgres Data Type Conversions
  • SQL Server Data Type Conversions

Admin:

• Administrators can review, enable, disable, and delete schedules through the application. See Schedules Page.

Language:

• PARSEDATE Function

Changes to System Behavior

Key Bug Fixes

May 30, 2019

This release fixes a few bugs. 

What's New

None.

Changes in System Behavior

None.

Key Bug Fixes

New Known Issues

None.

May 15, 2019

Release 6.2

This release includes new functions and a new method for locating datasets and recipes to join in.

What's New

Transformer Page:

• Browse current flow for datasets or recipes to join into the current recipe. See Join Window.

Language:

• New functions:
  
  

  Function Name	Description
  RANK Function	Computes the rank of an ordered set of value within groups. Tie values are assigned the same rank, and the next ranking is incremented by the number of tie values.
  DENSERANK Function	Computes the rank of an ordered set of value within groups. Tie values are assigned the same rank, and the next ranking is incremented by 1.

Connectivity:

• Enhanced import process for Excel files. See Import Excel Data.

Changes to System Behavior

None.

Key Bug Fixes

None.

New Known Issues

None.

April 10, 2019

Release 6.1

This release greatly expands the set of supported JDBC-based connections. Also, it introduces new selection-specific operations, including replacement of individual cell values, as well as bug fixes.

What's New

Connectivity:

• Improved ingestion performance for JDBC connections. For more information on available connections, see Connection Types.

Wrangling:

• Replace specific cell values. See Replace Cell Values.

Changes to System Behavior

None.

Key Bug Fixes

None.

New Known Issues

None.

March 4, 2019

Release 6.0

This release of  Trifacta® Wrangler Pro introduces key features around job monitoring and column management, including multi-select and copy and paste of columns and column values. A new Job Details page captures more detailed information about job execution and enables more detailed monitoring of in-progress jobs. 

What's New

Wrangling:

• In data grid, you can select multiple columns before receiving suggestions and performing transformations on them. For more information, see Data Grid Panel.
  • New Selection Details panel enables selection of values and groups of values within a selected column. See Selection Details Panel.
• Copy and paste columns and column values through the column menus. see Copy and Paste Columns.
• Support for importing files in Parquet format. See Supported File Formats.

Connectivity:

• Read/write support for Redshift. 

  For more information, see Create Redshift Connections.

• Read support for the following relational connections. 
  

  NOTE: This feature is in Beta release.
  • Configuration specific to a connection type may be required:
    • Oracle
    • SQL Server. See Create SQL Server Connections.
    • PostgreSQL
    • DB2
    • Salesforce. See Create Salesforce Connections.
  • See Create Connection Window.
• Support for publishing to Tableau Server.
  
  NOTE: This feature is in Beta release.
  
  See Create Tableau Server Connections.

Monitoring:

• Monitor the status of in-progress jobs through the new Job Details page. See Job Details Page. 

Wrangle :

• New functions:

  Function Name	Description
  ARRAYINDEXOF Function	Computes the index at which a specified element is first found within an array. Indexing is left to right.
  ARRAYRIGHTINDEXOF Function	Computes the index at which a specified element is first found within an array, when searching right to left. Returned value is based on left-to-right indexing.
  ARRAYSLICE Function	Returns an array containing a slice of the input array, as determined by starting and ending index parameters.
  ARRAYMERGEELEMENTS Function	Merges the elements of an array in left to right order into a string. Values are optionally delimited by a provided delimiter.

• The LISTUNIQUE function is now the UNIQUE Function.
• All LIST* functions now accept Array type only inputs.

Changes to System Behavior

None.

Key Bug Fixes

None.

New Known Issues

February 5, 2019

What's New

Wrangling:

• Standardize column values through a simple new interface. See Standardize Page.
  • For more information on other methods of standardization, see Overview of Standardization. 

Workspace:

• New functions:

  The following functions can now be applied directly to arrays to derive meaningful statistics about them. 

  Function	Description
  ARRAYSORT Function	Sorts array values in the specified column, array literal, or function that returns an array in ascending or descending order.
  TRANSLITERATE Function	Transliterates Asian script characters from one script form to another. The string can be specified as a column reference or a string literal.

• Hide/unhide or wipeout workspace users. See Workspace Users Page.

Changes to System Behavior

None.

Errata

In prior releases, the product and documentation stated that the platform implemented a version of regular expressions based on Javascript syntax. This is incorrect.

The Trifacta platform implements a version of regular expressions based off of RE2 and PCRE regular expressions.

Key Fixes

None.

New Known Issues

January 10, 2019

What's New

Language:

• New functions to compute statistics on arrays.
• Track file-based lineage using $filepath and $sourcerownumber references. See Source Metadata References.

Jobs:

• Filter list of jobs by source of job execution or by date range. See Jobs Page.

Changes to System Behavior

None.

Key Fixes

None.

New Known Issues

None.

December 4, 2018

This is the initial release of Trifacta® Wrangler Pro.