...

Supported versions of Databricks

• Azure AWS Databricks 7.3 (Recommended)

• Azure AWS Databricks 5.5 LTS

Job Limits

...

Instance pooling reduces cluster node spin-up time by maintaining a set of idle and ready instances. The 

...

• All cluster nodes used by the 

  D s platform

   are taken from the pool. If the pool has an insufficient number of nodes, cluster creation fails.
  
• Each user must have access to the pool and must have at least the ATTACH_TO permission.
• Each user must have a personal access token from the same Azure AWS Databricks workspace. See Configure personal access token below.

...

1. Acquire your pool identifier or pool name from Azure AWS Databricks.
   
   

   Info
   NOTE: You can use either the Databricks pool identifier or pool name.If both poolId and poolName are specified, poolId is used first. If that fails to find a matching identifier, then the poolName value is checked.

   
   

   Tip
   Tip: If you specify a poolName value only, then you can run your Databricks jobs against the available clusters across multiple D s item item workspaces . This mechanism allows for better resource allocation and broader execution options.

   
   

2. D s config

3. Set either of the following parameters: 

   1. Set the following parameter to the Azure AWS Databricks pool identifier:

      Code Block
      "databricks.poolId": "<my_pool_id>",

      
      

   2. Or, you can set the following parameter to the Azure AWS Databricks pool name:

      Code Block
      "databricks.poolName": "<my_pool_name>",

      
      

4. Save your changes and restart the platform.

...