Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r092
Excerpt

After you have performed the base installation of the 

D s platform
rtrue
, please complete the following steps if you are integrating with a Hadoop cluster.

If the 

D s platform
 is being installed on an edge node of the cluster, you can create a symlink from a local directory to the source cluster files so that they are automatically updated as needed.

  1. Navigate to the following directory on the 

    D s node
    :

    Code Block
    cd /opt/trifacta/conf/hadoop-site
  2. Create a symlink for each of the Hadoop Client Configuration files referenced in the previous steps. Example:

    Code Block
    ln -s /etc/hadoop/conf/core-site.xml core-site.xml
  3. Repeat the above steps for each of the Hadoop Client Configuration files.

Version update for Hortonworks

If you are using Hortonworks, you must complete the following modification to the site configuration file that is hosted on the  

D s node

Info

NOTE: Before you begin, you must acquire the full version and build number of your Hortonworks distribution. On any of the Hadoop nodes, navigate to /usr/hdp. The version and build number is a directory in this location, named in the following form: A.B.C.D-XXXX.

 

In the 

D s item
itemdeployment
, edit the following file:

Code Block
/opt/trifacta/conf/hadoop-site/mapred-site.xml

 

Perform the following global search and replace:

  1. Search:

    Code Block
    ${hdp.version}
  2. Replace with your hard-coded version and build number:

    Code Block
    A.B.C.D-XXXX

Save the file. 

Restart the 

D s platform
.

Modify 
D s item
configuration
configuration
 changes

  1. D s config
    methodt

  2. HDFS: Change the host and port information for HDFS as needed. Please apply the port numbers for your distribution:

    Code Block
    "hdfs.namenode.host": "<namenode>",
    "hdfs.namenode.port": <hdfs_port_num>
    "hdfs.yarn.resourcemanager": {
    "hdfs.yarn.webappPort": 8088,
    "hdfs.yarn.adminPort": 8033,
    "hdfs.yarn.host": "<resourcemanager_host>",
    "hdfs.yarn.port": <resourcemanager_port>,
    "hdfs.yarn.schedulerPort": 8030


     

  3. Save your changes and restart the platform.

...