These release notes apply to the following product tiers of :

Tip: You can see your product tier in the . Select Help menu > About Cloud Dataprep.

For more information, see Product Editions.

For release notes from previous releases, see Earlier Releases of Cloud Dataprep.

December 7, 2021

Release 8.10

What's New

User management:

Introducing user and role management. In the Admin console in the , you can enable and disable user access and determine access levels to individual object types, such as flows, connections, and plans. 

For more information, see Referrals Page.

Connectivity:

Session Management:

You can view the current and recent sessions for your account in the . As needed, you can revoke any unfamiliar devices or sessions. For more information, see Sessions Page.

Changes in System Behavior

Data type mismatches can now be written out in CSV format:

Beginning in this release, for CSV outputs mismatched values are written as regular values by default. In prior releases, mismatched values were written as null values in CSV outputs.

See Improvements to the Type System.

Ingestion:

Maximum permitted record length has been increased from 1 MB to 20 MB. For more information, see Working with JSON v2.

Split transformation:

When splitting a column based on positions, the positions no longer need to be listed in numeric order. See Changes to the Language.

Deprecated

None.

Key Bug Fixes

None.

New Known Issues

TicketDescription
TD-66185

Flatten transformation cannot handle multi-character delimiters.

Workaround: When a column of arrays is flattened using the running environment, multi-character String delimiters are not supported. As a workaround, you can create a regular expression delimiter, as in the following, which uses either left bracket or right bracket as the delimiter:

/[|]/




November 23, 2021

Release 8.9

What's New

Refer and Earn:

Beginning in this release, for every new sign-up you refer, you get a reward of your choice.

Self-serve upgrades from your free trial

Through the trial expiration page, you can review and select the preferred plan that suits you. Provide the required card details through the application and subscribe to your preferred plan.

For more information, see Start a Subscription.

BigQuery Running Environment:

Beginning in this release, imported datasets created with customer SQL are supported for execution in the BigQuery running environment. For more information, see BigQuery Running Environment.

Connectivity:

Early Preview (read-only) connections available with this release:

Plans:

Collaboration:

You can view the list of collaborators and their corresponding avatars on shareable objects, such as Flows, Plans, and Connections pages.

Sampling:

Changes in System Behavior

None.

Deprecated

None.

Key Bug Fixes

TicketDescription
TD-65502Datasets from parameters are improperly being permitted to be referenced in recipes and returns an error during job execution.

New Known Issues

None.

October 12, 2021

Release 8.8

What's New

Project Usage:

:

Changes

Cancellation of jobs is temporarily disabled:

In previous releases, you could cancel in-progress jobs for flow and sampling jobs through the . As of this release, canceling of job types, such as sampling, transformation, and profiling jobs, is temporarily disabled.

NOTE: This change applies to all types of jobs executed across all running environments, including BigQuery. For plan runs, some jobs, such as flow tasks, may continue to completion before the plan is canceled.


Tip: For jobs, you can still cancel them through the interface in .

Job cancellation may be re-enabled in the future.

Billing:

Charges for your project and user usage of  are applied to your account based on the UTC (Greenwich) time zone. However, Google Marketplace tracks and reports usage based on the Pacific (U.S. West Coast) time zone, so some discrepancies in reporting have been observed.

Beginning at the end of October 2021, these discrepancies will be addressed. The daily reporting interval will be changed to start and end at midnight Pacific time to match how Google Marketplace reports. However, the usage tracking will remain based on the UTC time zone.

NOTE: During the year, UTC time may mean:

  • Pacific time zone is UTC-07:00 during daylight savings time.
  • Pacific time zone is UTC-08:00 during standard time.

vCPU usage has been tracked on an hourly basis and will be unchanged.

For more information, see Usage Page.

Import:

Improvements have been made in how double quotes are handled in CSV files during import to align  with other systems that support CSV import. 


API:

BigQuery Running Environment:


When running jobs in BigQuery, some additional data types, functions, and transformations are now supported:


BigQuery Running Environment:


Deprecated

None.

Known Issues

None.

Fixes

TicketDescription
TD-64383

jobs that use custom SQL to query an authorized view may fail when the Service Account in use has access to the authorized view but no access to underlying BigQuery table.

September 15, 2021

Release 8.7

What's New

Templates:


From the Flows page, you can now access pre-configured templates directly from the templates gallery.

Tip: Click Templates in the Flows page. Select the template, and the template is opened in Flow View for you.

Browsers:

Plans:

Import data:

Sharing:


Publishing:

Recipe panel:

Changes

None.

Deprecated

API:

Known Issues

TicketDescription
TD-63517

Unpivoting a String column preserves null values in but converts them to empty strings in Photon. Running jobs on the different running environments generates different results.

Workaround:  After the unpivot step, you can add an Edit with formula step. Set the columns to all of the columns in the unpivot and add the following formula, which converts all missing values to null values:

if(ismissing($col),NULL(),$col)




Fixes

TicketDescription
TD-63564

Schedules created by a flow collaborator with editor access stop working if the collaborator is removed from the flow.

Collaborators with viewer access cannot create schedules.


August 16, 2021

Release 8.6

What's New

Template Gallery:

Tip:  You can start a trial account by selecting a pre-configured template from our templates gallery. See  www.trifacta.com/templates.  

Collaboration:

Connectivity:

Performance:

Better Handling of JSON files:

The  now supports the regularly formatted JSON files during import. You can now import flat JSON records contained in a single array object. With this, each array is treated as a single line and imported as a new row. For more information, see Working with JSON v2

Usage reporting:

Detailed reporting on vCPU and active users is now available in the .

NOTE:  Active user reporting may not be available until September 1, 2021 or later.

For more information, see Usage Page.

Changes

 machines:

Deprecated

None.

Known Issues

Fixes

July 20, 2021

Release 8.5

What's New

Tip: When you complete your or trial, you can choose to license a higher or lower tier product edition. For more information, see Product Editions.

Parameterization:

Schedules:

Flow View:

Job execution:

Resource usage:

Connectivity:

Contribute to the future direction of connectivity: Click I'm interested on a connection card to upvote adding the connection type to the . See Create Connection Window.

Connectivity:

Connectivity:

API:

Job execution:

You can choose to ignore the recipe errors before job execution and then review any errors in the recipe through the Job Details page.

Language:

Performance:

Resource usage:

Changes

None.

Deprecated

None.

Known Issues

None.

Fixes

June 7, 2021

Release 8.4

What's New

Template Gallery:

Connectivity:

Collaboration:


Support for delete actions on merge (upsert) operations in BigQuery:

When publishing to a BigQuery table, you can choose to update or, with this release, to delete matching records during a merge option. For more information, see BigQuery Table Settings.

Job execution:

You can choose to ignore the recipe errors before job execution and then review any errors in the recipe through the Job Details page.

Language:

Changes

limits on execution time

is an in-memory running environment that is hosted on the same node as , which allows for faster execution suitable for small- to medium-sized jobs.

NOTE: Jobs that are executed on may be limited to run for a maximum of 10 minutes, after which they fail with a timeout error. If your job fails due to this limit, please switch to running the job on .

can be enabled or disabled by a project administrator. For more information, see Dataprep Project Settings Page.

Execution of scheduled jobs on is not supported

In conjunction with the previous change, execution of scheduled jobs is not supported on . Since jobs are now limited to 10 minutes of execution time, scheduled jobs have been automatically migrated to execution on to provide better execution success. For more information, see Trifacta Photon Running Environment.

Deprecated

None.

Known Issues

Fixes

May 20, 2021

Release 8.3 - push 3

What's New

Connectivity:

Changes

enabled by default

is an in-memory running environment that is hosted on the same node as , which allows for faster execution suitable for small- to medium-sized jobs.

NOTE: Jobs executed in  are executed within the . Data is temporarily streamed to the during job execution and is not persisted.

Beginning in this release, is enabled by default. Users can choose to run jobs on .

NOTE: For , is enabled by default for new projects. For existing projects, a project administrator must still choose to enable it.

can be enabled or disabled by a project administrator. For more information, see Dataprep Project Settings Page.

Deprecated

None.

Known Issues

None.

Fixes

None.

May 10, 2021

Release 8.3

What's New

Running Environments:

Cancel Jobs in :

You can cancel  jobs directly from the product.

NOTE: In some cases, the product is unable to cancel the job from the application. In these cases, click View in Dataflow Job and from there you can cancel the job in progress .

Support for merge (upsert) operations in BigQuery:

When publishing to a BigQuery table, you can choose to write results using the merge option. When selected, you specify a primary key of fields and then decide how data is merged into the table. For more information, see BigQuery Table Settings.

Connectivity:

Job execution:

Introducing new filter pushdowns to optimize the performance of your flows during job execution. For more information, see Flow Optimization Settings Dialog.

Job results:

You can now preview job results and download them from the Overview tab of the Job details page. For more information, see Job Details Page.

Tip: You can also preview job results in Flow View. See View for Outputs.

Changes

Improved method of JSON import

Beginning in this release, the  now uses the conversion service to ingest JSON files during import. This improved method of ingestion can save significant time wrangling JSON into records.

NOTE: The new method of JSON import is enabled by default but can be disabled as needed.

For more information, see Working with JSON v2.

Flows that use imported datasets created using the old method continue to work without modification.

NOTE: It is likely that support for the v1 version of JSON import is deprecated in a future release. You should switch to using the new version as soon as possible. For more information on migrating your flows and datasets to use the new version, see Working with JSON v1.

Future work on support for JSON is targeted for the v2 version only.

Optionally, you can re-enable the old version, which is useful for migrating to the new version.

For more information on using the old version and migrating to the new version, see Working with JSON v1.

Deprecated

None.

Known Issues

Fixes

April 26, 2021

Release 8.2 push2

What's New

Upgrade: Trial customers can upgrade through the Admin console. See Admin Console.

This is the initial release of for the following product tiers:

Changes

None.

Deprecated

None.

Known Issues

None.

Fixes

None.

April 14, 2021

Release 8.2

This is the initial release of for the following product tiers:

What's New

Photon:

Introducing , an in-memory running environment for running jobs. Embedded in the , delivers improved performance in job execution and is best-suited for small- to medium-sized jobs.

NOTE: must be enabled by a project owner. For more information, see Dataprep Project Settings Page.

Quick scan sampling:

Preferences:

Connectivity:

Plan metadata references:

Use metadata values from other tasks and from the plan itself in your HTTP task definitions.


Improved accessibility of job results:

The Jobs tabs have been enhanced to display the list of latest and the previous jobs that have been executed for the selected output.

For more information, see View for Outputs.

Sample Jobs Page:

You can monitor the status of all sample jobs that you have generated. Project administrators can access all sample jobs in the workspace. For more information, see Sample Jobs Page.

Simplified output and destination experience:

From the Home Page, you can quickly redesign your output and destination experience. The step-by-step procedures enables you to create an improved and streamlined output creation experience. For more information, see Start with a Template.

Changes

Improved methods for disabling the product:

Project owners can choose to disable from within the product. For more information, see Enable or Disable Dataprep.

After the product has been disabled in a project, is placed in a hidden state for later purging. For more information on purging or restoring data, see Wipe Out Dataprep Data.

API:

The following API endpoints are scheduled for deprecation in a future release:

NOTE: Please avoid using the following endpoints.


/v4/connections/vendors
/v4/connections/credentialTypes
/v4/connections/:id/publish/info
/v4/connections/:id/import/info

These endpoints have little value for public use.

Deprecated

None.

Known Issues

Fixes


Earlier Releases

For release notes from previous releases, see Earlier Releases of Cloud Dataprep.