Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0821

...

  1. D s config
  2. Locate the following parameters and set their values according to the table below:

    Code Block
    "fileStorage.whitelist": ["abfss"],
    "fileStorage.defaultBaseUris": ["abfss://filesystem@storageaccount.dfs.core.windows.net/"],
    ParameterDescription
    filestorage.whitelist

    A comma-separated list of protocols that are permitted to read and write with ADLS Gen2 storage.

    Info

    NOTE: The protocol identifier "abfss" must be included in this list.

    filestorage.defaultBaseUris

    For each supported protocol, this param must contain a top-level path to the location where

    D s platform
    files can be stored. These files include uploads, samples, and temporary storage used during job execution.

    Info

    NOTE: A separate base URI is required for each supported protocol. You may only have one base URI for each protocol.

  3. Save your changes and restart the platform.

Configure access mode

Authentication to ADLS Gen2 storage is supported for system mode only.

NOTE: User mode, Azure AD, and Azure SSO are not supported for use with ADLS Gen2

In user mode, per-user access is governed by Azure AD SSO. Additional configuration is required. See below.

ModeDescription
System

All users authenticate to ADLS using a single system key/secret combination. This combination is specified in the following parameters, which you should have already defined:

  • azure.applicationId
  • azure.secret
  • azure.directoryId

These properties define the registered application in Azure Active Directory. System authentication mode uses the registered application identifier as the service principal for authentication to ADLS. All users have the same permissions in ADLS.

For more information on these settings, see Configure for Azure.

User
Info

System mode access

When access to ADLS Gen2 is requested, the platform uses the combination of Azure directory ID, Azure application ID, and Azure secret to complete access.

Steps:

Please verify the following steps to specify the ADLS access mode.

  1. D s config
  2. Verify that the following parameter to system:

    Code Block
    "azure.adlsgen2.mode": "system",
  3. Save your changes.

User mode access

In user mode, a user ID hash is generated from the Key Vault key/secret and the user's AD login. This hash is used to generate the access token, which is stored in the Key Vault.

Pre-requisites:

  • D s platform
     must be integrated with a Databricks 8.3 cluster. For more information, see Configure for Azure Databricks.
  • User mode access to ADLS requires Single Sign On (SSO) to be enabled for integration with Azure Active Directory. For more information, see Configure SSO for Azure AD.

Steps:

Please verify the following steps to specify the ADLS access mode.

  1. D s config
  2. Verify that Set the following parameter to systemuser:

    Code Block
    "azure.adlsgen2.mode": "systemuser",
  3. Save your changes.

Testing

...