...
- The
must be installed on Microsoft Azure.D s platform Nested folders are not supported when running jobs from Azure Databricks.
Info NOTE: Avoid including spaces in the paths to your ADLS sources. Spaces in the path value can cause errors during execution on Databricks.
- When a job is started and no cluster is available, a cluster is initiated, which can take up to four minutes. If the job is canceled during cluster startup:
- The job is terminated, and the cluster remains.
- The job is reported in the application as Failed, instead of Canceled.
Azure Databricks integration works with Spark 2.4.x only.
Info NOTE: The version of Spark for Azure Databricks must be applied to the platform configuration through the
databricks.sparkVersion
property. Details are provided later.- Azure Databricks integration does not work with Hive.
- Use of partitioned tables in Azure Databricks is not supported.
...