The Trifacta platform can be configured to integrate with supported versions of Azure Databricks clusters to run jobs in Spark.
NOTE: Before you attempt to integrate, you should review the limitations around this integration. For more information, see Configure for Azure Databricks.
You can apply this change through the Admin Settings Page (recommended) or
trifacta-conf.json. For more information, see Platform Configuration Methods.
Configure the following parameters to enable job execution on the specified Azure Databricks cluster:
Parameter Description webapp.runInDatabricks
Defines if the platform runs jobs in Azure Databricks. Set this value to
webapp.runWithSparkSubmit For all Azure Databricks deployments, this value should be set to
Configure the following Azure Databricks-specific parameters:
Parameter Description databricks.serviceUrl URL to the Azure Databricks Service where Spark jobs will be run (Example: https://westus2.azuredatabricks.net)
NOTE: If you are using instance pooling on the cluster, additional configuration is required. See Configure for Azure Databricks.
- Save your changes and restart the platform.
This page has no comments.