This section describes how to configure the for integration with KMS system for Cloudera. It assumes that access to the cluster is gated by Sentry.

Before you begin, please verify the pre-requisites. See Configure for KMS.

Configure Hadoop Cluster


NOTE: These changes should be applied through the management console for the Hadoop cluster before pushing the client configuration files to the nodes of the cluster.

In the following sections:

  • - the userID accessing the cluster component
  • -the appropriate group of user accessing the cluster component

Enable HDFS Encryption

On the Cloudera cluster, you may enable HDFS encryption using a designated Java KeyStore. For more information, see  http://www.cloudera.com/documentation/enterprise/latest/topics/sg_hdfs_encryption_wizard.html?scroll=concept_n2p_5vq_vt#concept_fcq_phr_wt_unique_1.

Java KMS Configuration

Additional configuration for the Java KMS is required. See http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_kms.html. 

Java KeyStore KMS Configuration

In the kms-site.xml configuration file, please locate the following properties:

NOTE: If you have deployed Cloudera Manager for your cluster, do not modify these properties in the file. Make any modifications through the Cloudera Manager console.


<property>
  <name>hadoop.kms.authentication.kerberos.keytab</name>
  <value>${user.home}/kms.keytab</value>
</property>

In Cloudera Manager, you may wish to change the following safety value value. Navigate to KMS service > Configuration > Advanced > Key Management Server Proxy Advanced Configuration Snippet (Safety Valve) for kms-site.xml. Modify the following:

<property>
    <name>hadoop.kms.aggregation.delay.ms</name>
    <value>10000</value>
</property>

In the kms-site.xml file, insert the following properties, which are required properties for the Key Management Server Advanced Configuration safety value:

<property>
  <name>hadoop.kms.authentication.kerberos.principal</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.kms.proxyuser.[hadoop.user].groups</name>
  <value>[hadoop.group]</value>
</property>
<property>
  <name>hadoop.kms.proxyuser.[hadoop.user].hosts</name>
  <value>*</value>
</property>

HDFS Configuration

In httpfs-site.xml, please insert the following properties, which are the safety value for HttpFS Advanced Configuration:

<property>
  <name>httpfs.proxyuser.[hadoop.user].groups</name>
  <value>[hadoop.group]</value>
</property>
<property>
  <name>httpfs.proxyuser.[hadoop.user].hosts</name>
  <value>*</value>
</property>

Save the files.

Validate

After the configuration is complete, you can try to import a dataset from a source stored in a cluster location managed by KMS, assuming that any required authentication configuration has been completed. See Import Data Page.

For more information, see Configure Hadoop Authentication.