Configure Azure

Create registered application

You must create a Azure Active Directory (AAD) application and grant it the desired access permissions, such as read/write access to the ADLS resource and read/write access to the Azure Key Vault secrets . This service principal is used by the

After you have registered, acquire the following information:

These properties are applied later in the configuration process.

Configure the Platform

Configure for HDI

If you are integrating the

Configure base storage layer

For Azure installations, you can set your base storage layer to be HDFS or WASB.

See Set Base Storage Layer.

Configure for Key Vault

For authentication purposes, the

Please complete the following sections to create and configure your Azure Key Vault.

Create a Key Vault resource in Azure

1. Log into the Azure portal.
2. Goto: https://portal.azure.com/#create/Microsoft.KeyVault
3. Complete the form for creating a new Key Vault resource:

   1. Name: Provide a reasonable name for the resource. Example:

      Code Block
      <clusterName>-<applicationName>-<group/organizationName>

   2. Location: Pick the location used by the HDI cluster.
   3. For other fields, add appropriate information based on your enterprise's preferences.
4. To create the resource, click Create.

Enable Key Vault access for the

D s platform

In the Azure portal, you must assign access policies for application principal of the

Steps:

1. In the Azure portal, select the Key Vault you created. Then, select Access Policies.
2. In the Access Policies window, select the

   D s item
   item registered application

   .
3. Click Add New.
4. For Secret permissions, select the following:
   1. Get
   2. Set
   3. Delete
5. Do not select any other options.
6. Click OK.

Create WASB access token

If you are enabling access to WASB, you must create this token within the Azure Portal.

For more information, see https://docs.microsoft.com/en-us/rest/api/storageservices/delegating-access-with-a-shared-access-signature.

Configure Key Vault key and secret for WASB

In the Key Vault, you can create key and secret pairs for use.

WASB: To enable access to the Key Vault, you must specify your key and secret values as follows:

Acquire shared access signature value:

In the Azure portal, please do the following:

1. Open your storage account.
2. Select Shared Access Signature.
3. Generate or view existing signatures.
4. For a new or existing signature, copy the SAS token value. Omit the leading question mark (?).
5. Paste this value into a text file for safekeeping.

Create a custom key:

To create a custom key and secret pair for WASB use by the

1. On an existing or newly created Azure Key Vault resource, click Secrets.
2. At the top of the menu, click Generate/Import.
3. In the Create a secret menu:
   1. Select Manual for upload options.

   2. Chose an appropriate name for the key.

      Info
      NOTE: Please retain the name of the key for later use, when it is applied through the D s platform as the sasTokenId value. Instructions are provided later.

   3. Paste the SAS token value for the key into the secret field.
   4. Click Create.

Configure Key Vault location

For ADLS or WASB, the location of the Azure Key Vault must be specified for the 

Steps:

1. Log in to the Azure portal.
2. Select the Key Vault resource.
3. Click Properties.
4. Locate the DNS Name field. Copy the field value.

This value is the location for the Key Vault. It must be applied in the

Steps:

1. D s config

2. Specify the URL in the following parameter:

   Code Block
   "azure.keyVaultURL": "<your key value URL>",

Apply SAS token identifier for WASB

If you are using WASB as your base storage layer, you must apply the SAS token value into the configuration of the

Steps:

1. D s config

2. Paste the value of the SAS Token for the key you created in the Key Vault as the following value:

   Code Block
   "azure.wasb.defaultStore.sasTokenId": "<your Sas Token Id>",

3. Save your changes.

Configure Secure Token Service

Access to the Key Vault requires use of the secure token service (STS) from the

Configure for SSO

If needed, you can integrate the

Configure for ADLS

Enable read-only or read-write access to ADLS. For more information, see Enable ADLS Access.

Configure for WASB

Enable read-only or read-write access to WASB. For more information on integrating with WASB, see Enable WASB Access.

Configure relational connections

If you are integrating

Create encryption key file

An encryption key file must be created on the

Create Hive connection

You can create a connection to the Hive instance on the HDI cluster with some modifications.

Natively, Azure supports high availability for HiveServer2 via Zookeeper. As a result, host and port information in the JDBC URL must be replaced with a Zookeeper quorum.

In addition to the other Hive connection properties, please specify the following values for the properties listed below:

zk1.cloudapp.net:2181,zk2.cloudapp.net:2181,zk3.cloudapp.net

/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Connections are created through the Connections page. See Connections Page.

For additional details on creating a conection to Hive, see Create Hive Connections.

A Hive connection can also be created using the above property substitutions via CLI or API.

• For details on values to use when creating via CLI or API, see Connection Types.
• See CLI for Connections.
• See API Connections Create v3.

Create Azure SQL DB connection

For more information, see Create SQL DB Connections.

Create Azure SQL DW connection

For more information, see Create SQL DW Connections.

Workaround for missing Python packages

After installation, the supervisord process may complain about some Python packages that are "missing."

These packages are present but lack the appropriate permissions. To enable the packages for use, please run the following on the

python_dir="/usr/local/lib/python2.7"
directories=$(find "$python_dir/dist-packages/" -maxdepth 2 -type d)
for d in $directories; do
  chmod 775 "${d}"
  chmod ugo+r "${d}"/*
done