Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r085

...

For release notes from previous releases, see Earlier Releases of Cloud Dataprep.

July 20, 2021

Release 8.5

What's New

Tip

Tip: When you complete your

D s product
productgdpent
or
D s product
productgdppro
trial, you can choose to license a higher or lower tier product edition. For more information, see Product Editions.


Parameterization:

  • Create environment parameters to ensure that all users of the project or workspace use consistent references.

    Info

    NOTE: You must be a workspace administrator or project owner to create environment parameters.

    Info

    Tip: Environment parameters can be exported from one project or workspace and imported into another, so that these references are consistent across the enterprise.

  • Parameterize names of your storage buckets using environment parameters.

...

Schedules:

  • Project owners and workspace administrators can review, enable, disable, and delete schedules through the application.

    D s ed
    nottrue
    editionsgdpsta

    See Schedules Page.

Flow View:

Job execution:

Connectivity:

Tip

Contribute to the future direction of connectivity: Click I'm interested on a connection card to upvote adding the connection type to the

D s webapp
. See Create Connection Window.

  • Early Preview (read-only) connections available with this release:

    D s ed
    editionsgdpent,gdppro,gdppr
  • D s conntype
    typeapache_impala

...

Connectivity:

  • Connect to your relational database systems hosted on Cloud SQL. In the Connections page, click the Cloud SQL card for your connection type.
    D s ed
    rtrue
    editionsgdpent,gdppro,gdppr

    For more information, see Create Connection Window

...

Connectivity:

  • Read-only support for
    D s conntype
    typeTeradata
    connections. For more information, see Teradata Connections.

...

API:

  • Cancel in-progress

    D s dataflow
    jobs via API.

    D s ed
    editionsgdpent,gdppro,gdppr,gdpst

    See Changes to the APIs.

...

  • NUMVALUE function can be used to convert a String value formatted as a number into an Integer or Decimal value.
  • NUMFORMAT function now supports configurable grouping and decimal separators for localizing numeric values.
  • For more information, see Changes to the Language.


Performance:

  • Improved performance when browsing folders containing a large number of files on 
    D s storage

Changes

None.

Deprecated

None.

Known Issues

None.

Fixes

  • TD-62190: You may not be able to view the SQL that was used to execute a job within BigQuery. This issue is due to a regression in the new BigQuery console in which job identifiers containing dashes are not supported. A ticket has been filed with Google.

June 7, 2021

Release 8.4

What's New

Template Gallery:

  • Check out the new gallery of flow templates, which can be imported into your workspace. These templates are pre-configured to solve the most compelling loading and transformation use cases in the product. For more information, see www.trifacta.com/templates.
    • For more information on importing flows into your workspace, see Import Flow.
    • For more information on using a template in the product, see Start with a Template

Connectivity:

  • Early Preview (read-only) connections available with this release:

    D s ed
    editionsgdpent,gdppro,gdppr

...

Changes

D s photon
limits on execution time

...

In conjunction with the previous change, execution of scheduled jobs is not supported on

D s photon
. Since
D s photon
jobs are now limited to 10 minutes of execution time, scheduled jobs have been automatically migrated to execution on
D s dataflow
to provide better execution success. For more information, see Trifacta Photon Running Environment.

Deprecated

None.

Known Issues

  • TD-62190: You may not be able to view the SQL that was used to execute a job within BigQuery. This issue is due to a regression in the new BigQuery console in which job identifiers containing dashes are not supported. A ticket has been filed with Google.

Fixes

  • TD-60881:  Incorrect file path and missing file extension in the application for parameterized outputs
  • TD-60382: Date format M/d/yy is handled differently by PARSEDATE function on
    D s photon
    and Spark.

May 20, 2021

Release 8.3 - push 3

What's New

Connectivity:

  • Support for SFTP connections.

    D s ed
    editionsgdpent,gdppro,gdppr

    Info

    NOTE: This connection type is import only.

    For more information, see SFTP Connections.

Changes

D s photon
enabled by default

...

D s photon
can be enabled or disabled by a project administrator. For more information, see Dataprep Project Settings Page.

Deprecated

None.

Known Issues

None.

Fixes

None.

May 10, 2021

Release 8.3

What's New

Running Environments:

...

Tip

Tip: You can also preview job results in Flow View. See View for Outputs.

Changes

Improved method of JSON import

...

For more information on using the old version and migrating to the new version, see Working with JSON v1.

Deprecated

None.

Known Issues

  • TD-61478: Time-based data types are imported as String type from BigQuery sources.

Fixes

  • TD-60701: Most non-ASCII characters incorrectly represented in visual profile downloaded in PDF format.
  • TD-59854: Datetime column from Parquet file incorrectly inferred to the wrong data type on import.

April 26, 2021

Release 8.2 push2

What's New

Tip

Upgrade: Trial customers can upgrade through the Admin console. See Admin Console.

...

  • D s product
    productgdpent
  • D s product
    productgdppro
  • D s product
    productgdpsta

Changes

None.

Deprecated

None.

Known Issues

None.

Fixes

None.

April 14, 2021

Release 8.2

...

  • D s product
    productgdpent
  • D s product
    productgdppro
  • D s product
    productgdpsta

What's New

Photon:

Introducing

D s photon
, an in-memory running environment for running jobs. Embedded in the
D s product
,
D s photon
delivers improved performance in job execution and is best-suited for small- to medium-sized jobs.
D s ed
nottrue
editionsgdple

...

Plan metadata references:

D s ed
editionsgdpent,gdppro,gdppr

Use metadata values from other tasks and from the plan itself in your HTTP task definitions.

...

From the Home Page, you can quickly redesign your output and destination experience. The step-by-step procedures enables you to create an improved and streamlined output creation experience. For more information, see Start with a Template.

Changes

Improved methods for disabling the product:

...

These endpoints have little value for public use.

Deprecated

None.

Known Issues

  • TD-60701: Most non-ASCII characters incorrectly represented in visual profile downloaded in PDF format.

Fixes

  • TD-59236:  Use of percent sign (%) in file names causes Transformer page to crash during preview.
  • TD-59218:  BOM characters at the beginning of a file causing multiple headers to appear in Transformer Page.

...

March 16, 2021

Release 8.1

What's New

Connectivity:

  • Introducing Early Preview connections. In each release of cloud-based product editions, new connection types may be made available in read-only mode for users to begin exploring their datasets stored in the connected datastores.

    Info

    NOTE: Early Preview connection types are read-only and are subject to change before they may be made generally available.

    D s ed
    editionsgdppr
  • Early Preview connections available with this release:
    • Airtable
    • Cassandra
    • Freshdesk
    • Google Analytics
    • MailChimp

...

Results of data quality checks are now part of the visual profile PDF available with your job results. In the PDF, you can download the data quality results over the entire dataset .

D s ed
editionsgdppr


  • Visual profiling must be enabled for the job.
  • For more information, see Job Details Page.

...

For more information, see Macros Page.

Changes

Freed IP address ranges:

The following IP address range is the only one in use by the

D s item
itemService
:

...

The Preferences area of the

D s webapp
has been changed. For more information, see Changes to Configuration.

Deprecated

None.

Known Issues

  • TD-58523: Cannot import dataset with filename in Korean alphabet from HDFS.

    • Workaround: You can upload files with Korean characters from your desktop. You can also add a 1 to the end of the file on HDFS, and it can then be imported.

  • TD-55299: Imported datasets with encodings other than UTF-8 and line delimiters other than \n may generate empty outputs on Spark or

    D s dataflow
    running environments.

  • TD-51516: Input data containing BOM (byte order mark) characters may cause Spark or
    D s dataflow
    running environments to read data improperly and/or generate invalid results.

Fixes

  • TD-56170: The Test Connection button for some relational connection types does not perform a test authentication of user credentials.
  • TD-54440: Header sizes at intermediate nodes for JDBC queries cannot be larger than 16K.
    • Previously, the column names for JDBC data sources were passed as part of a header in a GET request. For very wide datasets, these GET requests often exceeded 16K in size, which represented a security risk.

...

  • You can enable the 
    D s webapp
     to apply SQL filter pushdowns to your relational datasources to remove unused rows before their data is imported for a job execution. This optimization can significantly improve performance as less data is transferred during the job run. For more information, see Flow Optimization Settings Dialog.
  • Optimizations that were applied during the job run now appear in the Job Details Page. See Job Details Page.

Changes

None.

Deprecated

None.

Known Issues

  • TD-56830: Receive malformed_query: enter a filter criterion when importing table from Salesforce.

    • D s ed
      editionsgdppr
      oneLinetrue
    • NOTE: Some Salesforce tables require mandatory filters when they are queried. Mandatory filters are not currently supported for Salesforce connections

...

  • TD-56170: The Test Connection button for some relational connection types does not perform a test authentication of user credentials.

    • Workaround: Append the following to your Connect String Options:

      Code Block
      ;ConnectOnOpen=true
    • This option forces the connection to validate user credentials as part of the connection. There may be a performance penalty when this option is used.

Fixes

None.

...

January 12, 2021

Release 7.10

...

Changes

IP address range whitelist:

...

  • NOTE: When enabled, users are listed according to their email addresses.
  • NOTE: For 
    D s product
    productgdple
     , this feature is disabled by default.
  • For more information on enabling or disabling this feature, see  Changes to Configuration.

Deprecated

None.

Known Issues

None.

Fixes

TD-53527:  When importing a dataset via API that is sourced from a BZIP file stored on S3, the columns may not be properly split when the platform is permitted to detect the structure.

...

Changes

Changes to permissions: The set of required and optional permissions has changed for 

D s product
productgdppr
.

...

  • The Optimizer service optimizes query execution against data sources to minimize use of
    D s product
    productgdp
     resources, reduce compute costs, and improve overall job execution time.
  • No configuration is required.
  • D s ed
    editionsgdppr, gpdst
    oneLinetrue
  • You can apply optimizations for individual flows. For more information, see Flow Optimization Settings Dialog.

Deprecated

None.

Known Issues

None.

Fixes

TD-53475:  Missing associated artifact error when importing a flow.

...

APIModify the source 

D s storage
 bucket and path for a defined imported dataset.

D s ed
editionsgdppr
oneLinetrue

For more information, see API Workflow - Swap Datasets.

Changes

JDBC connection pooling disabled: The ability to create connection pools for JDBC-based connections has been disabled. It is likely to be removed in a future release.

D s ed
editionsgdppr
oneLinetrue

Deprecated Parameter History Panel Feature: As a part of collaborative suggestions enhancement, the support for Parameter History panel is deprecated from the software. For more information on collaborative suggestions feature, see Overview of Predictive Transformation.

...

Salesforce connector disabled temporarily: In Release 7.8, the Salesforce connector has been disabled temporarily. In a future release, it will be replaced with an improved version of the Salesforce connector.

D s ed
editionsgdppr
oneLinetrue

Deprecated

None.

Known Issues

TD-55503: When you swap datasets via API, existing samples are not discarded. These samples are invalid.

  • Workaround: This issue does not occur if you swap datasets through the 
    D s webapp
    . If it does occur via API, you can collect a new sample manually. See Samples Panel.

Fixes

TD-53318: Cannot publish results to relational targets when flow name or output filename or table name contains a hyphen (e.g. my - filename.csv).

...

Release 7.6 push 3

Features

None.

Changes

Disabled Optimizer Service: In the September release, the Optimizer service was introduced, which enabled users to apply advanced physical and logical optimizations for flow and job executions. Recently, an issue was discovered, which has caused us to disable the service temporarily.

  • This issue affected a very small number of users who were using the new feature. Now that the feature is disabled, impacted users should experience impacts only to performance of flow and job executions. Performance should be similar to pre-release of the service.
  • The Optimizer service was disabled through a configuration change that did not require any service interruptions. Users should not experience any loss of functionality or availability due to the work to resolve this issue.
  • The 
    D s item
    itemEngineering team
     is actively working to resolve the issue. Thank you for patience. If you have further questions, please contact 
    D s support
    .

Deprecated

None.

Known Issues

None.

Fixes

None.

...

October 1, 2020

Release 7.6 push 3

Features

None.

Changes

Shared VPCs: Across all product editions, you can now run jobs through another project by specifying a full URL for the shared VPC.

  • Previously, this capability was only available for 
    D s product
    productgdppr
    . This restriction has been lifted. It is now available for 
    D s product
    productgdpst
     and 
    D s product
    productgdple
    , too.
  • For more information on applying the shared VPC to a job, see Dataflow Execution Settings .
  • For more information on applying a shared VPC to jobs in the project, see Execution Settings Page.

    Info

    NOTE: You must create new or replace output objects to use these shared VPC settings across your project.

Deprecated

None.

Known Issues

None.

Fixes

None.

...

September 21, 2020

Release 7.6

...

  • Additional connect string options and troubleshooting information has been included for specific relational connections.
  • D s ed
    editionsgdppr,gdpst
    oneLinetrue
  • For more information, see Connection Types.

Changes

None.

Deprecated

None.

Known Issues

None.

Fixes

TD-52559:  When publishing a single CSV file with headers using the append or overwrite publishing action, multiple instances of the header may be written in the output file.

...

Changes

Delete Table IAM permission no longer required

...

  • Please add that the following range of IP addresses to the whitelist for the 

    D s item
    itemservices
     for access to relational datasources in your enterprise

    Code Block
    34.68.114.64/28
  • D s ed
    editionsgdppr
    oneLinetrue
  • For more information, see Getting Started with Cloud Dataprep.

Deprecated

None.

Known Issues

TD-50942: If a flow is unshared with you, you cannot see or access the datasources for any jobs that you have already run on the flow. You can still access the job results.

Fixes

TD-49559:  Cannot select and apply custom data types through column Type menu.

...