The can be configured to integrate with supported versions of Azure Databricks clusters to run jobs in Spark.
NOTE: Before you attempt to integrate, you should review the limitations around this integration. For more information, see Configure for Azure Databricks.
Configure the following parameters to enable job execution on the specified Azure Databricks cluster:
Defines if the platform runs jobs in Azure Databricks. Set this value to true.
Configure the following Azure Databricks-specific parameters:
NOTE: If you are using instance pooling on the cluster, additional configuration is required. See Configure for Azure Databricks.