Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0681

D toc

This section describes how you interact through the

D s platform
rtrue
 with your Azure Databricks Tables data.

Info

NOTE: Use of Azure Databricks Tables requires installation on Azure, integration with Azure Databricks, and an Azure Databricks connection. For more information, see Configure for Azure Databricks.


Uses of Databricks Tables

The

D s platform
 can use Databricks Tables for the following tasks:

  1. Create datasets by reading from Databricks Tables tables.
  2. Write data to Databricks Tables.
Table TypeSupport
Databricks managed tablesRead/Write
Databricks unmanaged tablesRead

Delta Tables (managed and unmanaged tables)

Read

The underlying format for Databricks Tables is Parquet.

Limitations

  • Access to external Hive metastores is not supported.
  • Ad-hoc publishing to Azure Databricks is not supported.
  • Creation of datasets with custom SQL is not supported.
  • Use of partitioned tables in Databricks Tables is not supported.

Before You Begin Using Databricks Tables

Storing Data in Databricks Tables

Info

NOTE: The

D s platform
does not modify source data in Databricks Tables. Datasets sourced from Databricks Tables are read without modification from their source locations.

Reading from Databricks Tables

You can create a

D s item
itemdataset
 from a table or view stored in Databricks Tables.

For more information on how data types are imported from Databricks Tables, see Databricks Tables Data Type Conversions.

Writing to Databricks Tables

You can write data back to Databricks Tables using one of the following methods:

  • Job results can be written directly to Databricks Tables as part of the normal job execution. 
    • Data is written as a managed table to DBFS in Parquet format.
    • Create a new publishing action to write to Databricks Tables.
    • See Run Job Page.
  • For more information on how data is converted to Databricks Tables, see Databricks Tables Data Type Conversions.

Ad-hoc Publishing to Databricks Tables

Not supported.