Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0871

...

  • D s conntype
    typedatabricks
     Tables provides a JDBC-based interface for reading and writing datasets in ADLS or WASB. Using the underlying JDBC connection, you can access your ADLS or WASB data like a relational datastore, run jobs against it, and write results back to the datastore as JDBC tables. 
  • Your connection to
    D s conntype
    typedatabricks
     Tables leverages the SSO authentication that is native to
    D s conntype
    typedatabricks

Supported Environments:

Operation

D s product

AmazonMicrosoft Azure
ReadNot supportedSupportedSupported
WriteNot supportedSupportedSupported

Limitations

  • Ad-hoc publishing of generated results to
    D s conntype
    typedatabricks
     Tables is not supported.
  • Integration with Kerberos or secure impersonation is not supported.
  • Some table types and publishing actions are not supported. 
  • Access to external Hive metastores is not supported.

...

  1. D s config
  2. Locate the following parameter and set it to true:

    Code Block
    "feature.databricks.connection.enabled": true,
  3. To allow for direct publishing of job results to

    D s conntype
    typedatabricks
    tables from the Run Job page, you must enable the following parameters. For more information on these settings, see Databricks Tables Table Settings.

    ParameterDescription
    feature.databricks.enableDeltaTableWrites

    Set this value to true to enable users to choose to write generated results to

    D s conntype
    typedatabricks
    delta tables from the Run Job page.

    feature.databricks.enableExternalTableWrites

    Set this value to true to enable users to choose to write generated results to

    D s conntype
    typedatabricks
    external tables from the Run Job page.

  4. Save your changes and restart the platform.

...

  1. Create datasets by reading from Databricks Tables tables.
  2. Write data to Databricks Tables.
Table TypeSupportNotes
Databricks managed tablesRead/Write
Delta tablesRead/Write
Info

NOTE: Versioning and rollback of Delta tables is not supported within the

D s platform
. The latest version is always used. You must use external tools to manage versioning and rollback.

External tables

Read/Write

Info

NOTE: When writing to an external table the TRUNCATE and DROP publishing actions are not supported.

Databricks unmanaged tablesRead/Write

Delta Tables (managed and unmanaged tables)

Read/Write
Partitioned tablesRead

The underlying format for Databricks Tables is Parquet.

...

Supported Versions: n/a

Supported Environments:

Operation

D s product

AmazonMicrosoft Azure
ReadNot supportedSupportedSupported
WriteNot supportedSupportedSupported


Tip

Tip: It's easier to create a connection of this type through the UI. Typically, only one connection is needed.

...