Configure for EC2 Role-Based Authentication

D toc

Excerpt

When you are running the

D s platform

on an EC2 instance, you can leverage your enterprise IAM roles to manage permissions on the instance for the

D s platform

. When this type of authentication is enabled,

D s item

item	administrators

can apply a role to the EC2 instance where the platform is running. That role's permissions apply to all users of the platform.

Before you begin, your IAM roles should be defined and attached to the associated EC2 instance.

Info
NOTE: The IAM instance role used for S3 access should have access to resources at the bucket level.

To enable role-based instance authentication, the following parameter must be enabled.

Code Block
"aws.mode": "system",

The following additional parameters must be specified:

Parameter	Description
`aws.credentialProvider`	Set this value to `instance`. IAM instance role is used for providing access.
`aws.hadoopFsUseSharedInstanceProvider`	Set this value to `true` for CDH. The class information is provided below.

Hortonworks:

Code Block
"com.amazonaws.auth.InstanceProfileCredentialsProvider",

Cloudera:

Code Block
"org.apache.hadoop.fs.s3a.SharedInstanceProfileCredentialsProvider"

In the future:

CDH is moving back to using the Instance class in a future release. For details, see https://issues.apache.org/jira/browse/HADOOP-14301.

To access S3 for storage, additional configuration for S3 may be required.

Info
NOTE: Do not configure the properties that apply to `user` mode.

Output sizing recommendations:

Single-file output: If you are generating a single file, you should try to keep its size under 1 GB.
Multi-part output: For multiple-file outputs, each part file should be under 1 GB in size.
For more information, see https://docs.aws.amazon.com/redshift/latest/dg/c_best-practices-use-multiple-files.html

Versions Compared