Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info

NOTE: If you are using Azure AD to integrate with an Azure Databricks cluster, the Azure AD secret value stored in azure.secret must begin with an alphanumeric character. This is a known issue.

  • Nested folders are not supported when running jobs from Azure Databricks.

    Info

    NOTE: Avoid including spaces in the paths to your ADLS sources. Spaces in the path value can cause errors during execution on Databricks.


  • When a job is started and no cluster is available, a cluster is initiated, which can take up to four minutes. If the job is canceled during cluster startup:
    • The job is terminated, and the cluster remains. 
    • The job is reported in the application as Failed, instead of Canceled.
  • Azure Databricks integration works with Spark 2.4.0 only.

    Info

    NOTE: The version of Spark for Azure Databricks must be applied to the platform configuration through the databricks.sparkVersion property. Details are provided later.


  • Azure Databricks integration does not work with Hive.
  • Use of partitioned tables in Azure Databricks is not supported.

...