Configure Publishing to Cloudera Navigator

Additional Configuration for SSL

To enable communication over SSL with Cloudera Navigator, please complete the following steps in Cloudera Manager and on the Alteryx node.

Steps:

1. Launch Cloudera Manager.
2. Select MGMT.
3. Click Configuration.
4. Click Scope > Navigator Metadata Server Category > Security.
5. Set Enable TLS/SSL for Navigator Metadata Server to true.

6. Set TLS/SSL for Navigator Metadata Server to the path where the Java keystore was created. The following is the default path:
   

   /opt/cm_keystore.jks

    

7. Set TLS/SSL Keystore File Password to the password to the Java keystore.
8. Set TLS/SSL Keystore Key Password to the password to the certificate.
9. Restart the MGMT service.
10. The JKS file that you created must be transferred to an accessible location on the Alteryx node.
11. Login to the application. 
12. You can apply this change through the Admin Settings Page (recommended) or trifacta-conf.json. For more information, see Platform Configuration Methods.
    
13. Configure the following properties:
    

     

    Property	Description
    "clouderaNavigator.https.enabled"	Set this value to true to enable HTTPS communications with Navigator.
    "clouderaNavigator.https.trustStore.type"	The format of the Java keystore file. Set this value to jks.
    "clouderaNavigator.https.trustStore.location"	The absolute path on the Alteryx node to the Java keystore file. In the previous example, this value was the following: /opt/cm_keystore.jks
    "clouderaNavigator.https.trustStore.password"	The password to the Java keystore file

14. Change the clouderaNavigator.baseURL value to use HTTPS.
15. Save your changes and restart the platform. See Start and Stop the Platform.

Configure Custom Source Types

In Cloudera Navigator, every listed entity is associated with a source. A source is a specified resource that is part of each job listing. Example sources include HDFS namenodes and Hive metadata servers. Each source has a specified source type. For more information on source types, see https://github.com/cloudera/navigator-sdk/blob/master/model/src/main/java/com/cloudera/nav/sdk/model/SourceType.java.

For each source type:

• If you have only one source for a type, no further configuration is required. 

• If you have multiple source types, you must specify your custom sources. For example, if you have multiple HDFS clusters, you must specify them in your custom sources. See below.

  NOTE: If you have multiple sources for a single type and have not completed the following configuration, the Designer Cloud Powered by Trifacta platform publication job to Cloudera Navigator fails. In the job log, you can review the source and the source type identifiers that caused the failure.

Steps:

1. To list all of your Cloudera Navigator source types, visit the following URL:
   
   

   http://<navigator_instance_url>:<port_number>/api/v<apiVersion>/entities?query=type%3Asource

   where:
   
   

   Property	Description
   <navigator_instance_url>	The URL of your instance of Cloudera Navigator
   <port_number>	The port number of your instance of Cloudera Navigator. Default value is 7187.
   <apiVersion>	The version number of the API in use.

2. Example: cURL with jq:

   curl "http://username:password@cloudera-navigator-host:7187/api/v13/entities?query=type%3Asource" | jq '[.[] | {type: .sourceType, identity: .identity}]'

3. From the returned list, you can determine the source Id's to use for each source type. 
4. You can apply this change through the Admin Settings Page (recommended) or trifacta-conf.json. For more information, see Platform Configuration Methods.

5. Locate the following settings. 
   
   

   NOTE: Leave these settings empty if you intend to use the default settings from Navigator.

   
   

   Setting	Description
   clouderaNavigator.customsources.YARN	Navigator identifier for the YARN resource manager to use
   clouderaNavigator.customsources.SPARK	Navigator identifier for the Spark instance to use
   clouderaNavigator.customsources.S3	Navigator identifier for the S3 bucket to use
   clouderaNavigator.customsources.HIVE	Navigator identifier for the Hive metadata server to use
   clouderaNavigator.customsources.HDFS	Navigator identifier for the HDFS cluster to use

6. You can specify the cluster to use by listing source type and mapping pairs under customSources. If you have multiple sources of a single type, this step disambiguates between them for the Designer Cloud Powered by Trifacta platform.

   1. First you must acquire the identifiers to use from Cloudera Navigator via API. 
      1. Navigator endpoint: /v13/entities?query=type%3Asource
      2. Request method: GET

      3. Example response:
         
         

         [
           {
             "type": "S3",
             "identity": "9"
           },
           {
             "type": "YARN",
             "identity": "6"
           },
           {
             "type": "SPARK",
             "identity": "5"
           },
           {
             "type": "CLUSTER",
             "identity": "1"
           },
           {
             "type": "HDFS",
             "identity": "10"
           },
           {
             "type": "HDFS",
             "identity": "11"
           },
           {
             "type": "HDFS",
             "identity": "12"
           }
         ]

   2. In the above example response are three HDFS clusters. Below, one of these clusters havs been specified by Id (11) to be used. 

      {
          ...,
          "clouderaNavigator": {
            ...,
            "customSources": {
              "HDFS": "11"
            }
          },
          ...
      }

7. Save your changes and restart the platform.
8. Verify your mappings by running a job on the named source. See below.

Validate

Steps:

1. If you haven't done so already, restart the platform to apply the configuration changes. See Start and Stop the Platform.
2. Run a job.
3. When the job completes, open the job through the Jobs page. See Jobs Page.

4. Acquire the jobGroup Id for the job. It is the final value in the URL. In the following example, the jobGroup Id is 3:
   
   

   http://example.com:3005/jobs/3

5. Login to Navigator. Search for the following string:
   
   

   trifacta.<jobGroupId>

   NOTE: It may take up to 30 minutes for results to be published to Navigator.

6. When you see one or more entries, such as the following, the job has been successfully published:

   trifacta.14.wrangle.29
   trifacta.14.filewriter.30
   trifacta.14.filewriter.31

7. The above entries indicate the individual jobs within the job group that have been completed.