These release notes apply to the following product tiers of :

Tip: You can see your product tier in the . Select Resources menu > About Cloud Dataprep.

For more information, see Product Editions.

For release notes from previous releases, see Earlier Releases of Dataprep by Trifacta.

January 11, 2023

Release 9.7

What's New

Schema validation:

Schema validation is now supported for CSV, TSV, and TXT files. 

NOTE: Detect structure must be enabled on a file-based imported dataset.

For more information, see Overview of Schema Management.

running environment:

Project administrators can define default execution settings that are passed to for jobs executed within the project. For more information, see Dataflow Execution Settings Page.

NOTE: By default, project users are permitted to override these settings in their account preferences or in individual jobs. As needed, project administrators can disable use of these overrides, which means all project users use the same  execution settings. For more information, see Dataprep Project Settings Page.

NOTE: Execution settings that were previously specified for output objects are not affected by this change. 

Dataprep In-VPC Execution: 

Tip: This feature was formerly known as Run Dataprep in Your VPC.

Support for running conversion jobs in your VPC. This includes ingestion of data from binary sources such as PDF, Excel, and Google Sheets.

NOTE: To process Google Sheets data in your VPC, a public key and private key must be specified. The private key must be accessible within your VPC. Otherwise, these value are optional. For more information, see Dataprep In-VPC Execution

Tip: Service accounts are now used for execution of conversion jobs in your VPC.

NOTE: This capability is disabled by default when In-VPC execution is enabled. For more information on enabling this feature, please contact .

For more information, see Dataprep In-VPC Execution


Publishing:

You can publish   Objects and arrays of Objects as complex types in BigQuery. 

Import flow:

When importing flows in the , you can now remap connections used for pre- and post-execution SQL scripts. For more information, see Import Flow.

Changes in System Behavior

Previously, Arrays could be published to BigQuery only for primitive homogenous data types. Beginning in this release, Arrays can now be published as nested data.

NOTE: If you have previously enabled publication of Arrays as primitive types in BigQuery, this change may cause breakages in your data pipeliens.

For more information, see Improvements to the Type System.

Deprecated

None.

Key Bug Fixes

None.

New Known Issues

None.

November 10, 2022

Release 9.6

What's New

Connectivity:

Early Preview (read-only) connections available with this release:

Connectivity:

Job history:

Changes to the layout of history pages for flow jobs, sample jobs, and plan runs for an improved user experience. 

BigQuery Running Environment:

Support for merge (upsert) operations when jobs are executed in BigQuery for table- and file-based sources. See BigQuery Running Environment.

In-VPC execution: 

In-VPC execution now supports connection from the to an in-VPC data service instance, which enables testing connections, viewing table and schema information, and collecting initial samples from datasources hosted within your VPC. 

Tip: In-VPC execution is now supported for  product edition.


NOTE: This capability is disabled by default when In-VPC execution is enabled. For more information on enabling this feature, please contact .

For more information, see Dataprep In-VPC Execution

Documentation:

Over the next set of releases, a number of object types are being migrated from the to the underlying platform layer.

Tip: This migration of objects to the platform layer is being performed to enable broader access to these object types in the future. You can expect to see enhancements to these capabilities in the future.

These object types include the following:

Additionally, the following capabilities are moving to the platform level:

Over the next few releases, documentation for these objects is migrating to the new Platform area. For more information, see Platform.

Changes in System Behavior

Flow collaborators can now edit custom SQL

Collaborators on a flow who have the flow editor permission can now edit any custom SQL used in importing datasets into the flow. 

Enable / Disable Data Grid

You can enable or disable the data grid in the Transformer page. When the data grid is disabled, you may not be able to edit some recipe steps. For steps that you can edit, select Preview to see the effects of the step on the data. When you select Preview, the data grid is re-enabled. 

NOTE: This feature can be enabled or disabled by an administrator.  

Deprecated

None.

Key Bug Fixes

TicketDescription
TD-74742BigQuery SQL execution fails with run for too long  error.

New Known Issues

None.

October 17, 2022

Release 9.5

What's New

Check out the updated resources in the left nav bar and on the Home page! 

Language:

In-VPC execution:

NOTE: In-VPC job execution requires additional configuration. See Dataprep In-VPC Execution


Parameterization:

Flow parameters now support a new type. Selector type parameters allow you to specify a list of permitted values for the parameter, which ensures data integrity throughout the transformation process. See Create Flow Parameter.

Schema refresh supports Excel, PDF, and Google Sheets:

You can now refresh the schemas from Excel, PDF or Google Sheets datasources converted during ingest. Schema refresh enables you to identify changes to the columns in your dataset. 

Import flow:

During the import flow process in the , you can now remap connections and environment parameters in the flow to corresponding objects in the new project or workspace through drop-down menus. For more information, see Import Flow

Connectivity:

Early Preview (read-only) connections available with this release:

Connectivity:

Billing:

Support for direct deposit payments from your US Bank account.

For more information, see Plans and Billing Page.

Asset transfer:

Transfer ownership of your individual assets to another user. See Transfer Asset Ownership.

NOTE: Administrators can transfer ownership of any user's asset to another user. Administrators can also transfer ownership of all of a user's assets. See User Details Page.

Job History:

Administrators can configure the default number of days of jobs to display in the Job History page. Default setting is 180  days.

Tip: Individual users can filter the list of jobs further, as needed.

For more information, see Dataprep Project Settings Page.

Transformer page:

For improved performance, you can now edit your recipes without loading a sample in the data grid. 

NOTE: This feature must be enabled in your environment.

For more information, see Dataprep Project Settings Page.

Changes in System Behavior

New pricing and packaging model

Release 9.5 introduces a new model for what is packaged in Enterprise, Professional, and Starter product editions and pricing for each edition. 

For more information on differences between product editions in the new model, please visit .

For additional questions, please contact .

Subscription management through Sales

Beginning in Release 9.5, changes to your product edition are managed through Sales. In the , click Contact Sales to reach out. For more information, see Start a Subscription.

API access moving to Enterprise only

Beginning in Release 9.5, all new or renewed subscriptions have access to public API endpoints on the Enterprise product edition only. For example, new or renewed subscriptions for the Professional product edition do not have access to API endpoints.

Existing customers that currently have access to API endpoints for non-Enterprise product editions can continue to use their available endpoints until their subscription expires. To use API endpoints after renewal, you must upgrade to the Enterprise product edition. 

For more information on differences between product editions in the new model, please visit .

For additional questions, please contact .

Quickbooks Online

The connection type has been re-enabled. See Quickbooks Online Connections.

Deprecated

None.

Key Bug Fixes

TicketDescription
TD-69813

array type columns in datasets that were imported before Release 9.2 are still published as String type.

New Known Issues

None.

August 17, 2022

Release 9.4

What's New

JavaScript User Defined Functions:

Import:

Broader service account usage:

For jobs executed within your enterprise VPC, you can now configure the use of service accounts for most jobs. 

Tip: Service accounts can replace the use of user credentials for most types of in-VPC batch processing, which provides a higher level of security and is recommended by Google.

NOTE: Workload Identity must be enabled to run jobs in your GKE cluster, which is required for In-VPC job execution. For more information on configuring Workload Identity, see Dataprep In-VPC Execution.

For more information on service accounts, see Google Service Account Management.

Connectivity:

Early Preview (read-only) connections available with this release:

Connectivity:

Support for creating connections to .

NOTE: This connection type is disabled by default. For more information on enabling this connection type, please contact, .


Connectivity:

Support for creating OAuth 2.0 connections to single-tenant and multi-tenant

Schema refresh supports JSON:

You can now refresh the schemas from JSON datasource converted during ingest. Schema refresh enables you to identify changes to the columns in your dataset. 

Email notifications enhancements:

Tip: Each email notification includes a summary of Data Quality rules (rules that were run and the success/failures of those rules) and the profile details (valid, mismatched, missing) when a job is completed successfully. In the email, click View job to view the details of the job. See Email Notifications Page.

Changes in System Behavior

Email notifications enabled by default:

NOTE: The settings for email notifications based on success or failure of jobs or plan runs have been enabled at the project or workspace level and at the flow level.

This change means that the user who executes a job and others who have access to the flow receive, by default, an email notification whenever a job executes for flows where email notification settings have never been modified. 

If needed, these new default settings can be modified.

Deprecated

None.

Key Bug Fixes

None.

New Known Issues

None.

June 21, 2022

Release 9.3

What's New

Private data processing:

You can execute  jobs within your enterprise's virtual private cloud (VPC). 

In-VPC execution must be enabled by an administrator. 

NOTE: This feature requires additional configuration of the through the gcloud command line tools.

For more information, see Dataprep In-VPC Execution.

Private data processing:

Jobs related to ingesting, sampling, and publishing data for relational databases can now be executed within your enterprise's virtual private cloud (VPC).

In-VPC execution must be enabled by an administrator.

NOTE: This feature requires additional configuration of the through the gcloud command line tools.

For more information, see Dataprep In-VPC Execution.

Expandable left nav bar:

The new left navigation bar can be expanded to display full-text options for each menu item. Collapse it to reclaim the screen area. Available options remain consistent. See Home Page.

Configure range joins:

Specify ranges of key values in your joins.

NOTE: This feature may need to be enabled by an administrator. See Dataprep Project Settings Page.

For more information, see  Configure Range Join .

Billing:

Edit credit card and billing information and review billing history and invoices.

For more information, see Plans and Billing Page.

Connectivity:

Early Preview (read-only) connections available with this release:

Connectivity:

Transformer page:

Improved performance of the Transformer page through asynchronous loading of initial samples.

Changes in System Behavior

Generate an initial sample:

When generating an initial sample from a set of files in a directory, the maximum number of files that can be read is now limited to 10 files by default. For more information on changing the maximum number, see Dataprep Project Settings Page.

Quickbooks Online connections are disabled:

This feature has been disabled due to technical issues. It will be re-enabled when these issues are resolved in a future release.

Deprecated

None.

Key Bug Fixes

None.

New Known Issues

None.

Earlier Releases

For release notes from previous releases, see Earlier Releases of Dataprep by Trifacta.