API Workflow - Publish Results

Overview

After you have run a job to generate results, you can publish those results to different targets as needed. This section describes how to automate those publishing steps through the APIs.

In the application, you can publish after generating results. See Publishing Dialog.

Basic Workflow

1. Create connections to each target to which you wish to publish. Connections must support write operations. 
2. Specify a job whose output meets the requirements for the target. 
3. Run the job.
4. When the job completes, publish the results to the target(s). 

Step - Create Connections

For each target, you must have access to create a connection to it. After a connection is created, it can be reused, so you may find it easier to create them through the application.

• Some connections can be created via API. For more information, see  https://api.trifacta.com/ee/9.2/index.html#operation/createConnection

• Other connections must be created through the application. Links to instructions are provided below. 

Redshift connection

• Required Output Format: Avro
• Example Id: 2
• Create via API: N
• Doc Link: Amazon Redshift Connections

Other Requirements:

• Requires S3 set as the base storage layer. See Set Base Storage Layer.

Hive connection

• Required Output Format: Avro
• Example Id: 1
• Create via API: Y
• Doc Link: Hive Connections
• Other Requirements:
  • Requires integration with a Hadoop cluster.

Tableau Server connection

• Required Output Format: HYPER

• Example Id: 3
• Create via API: Y
• Doc Link: Tableau Server Connections
• Other Requirements:
  • None.

SQL DW connection

• Required Output Format: Parquet
• Example Id: 4
• Create via API: N
• Doc Link: Microsoft SQL Data Warehouse Connections
• Other Requirements:
  • Available only on Azure deployments. See Configure for Azure.

Step - Run Job

Before you publish results to a different datastore, you must generate results and store them in HDFS.

In the examples below, the following example data is assumed:

For more information on running a job, see  https://api.trifacta.com/ee/9.2/index.html#operation/runJobGroup
For more information on the publishing endpoint, see  https://api.trifacta.com/ee/9.2/index.html#operation/publishJobGroup

Step - Publish Results to Hive

The following uses the Avro results from the specified job (jobId = 2) to publish the results to the test_table table in the default Hive schema through connectionId=1.

Request:

{
  "connection": {
    "id": 1
  },
  "path": ["default"],
  "table": "test_table",
  "action": "create",
  "inputFormat": "avro"
}

Response:

{
    "jobgroupId":2,
    "reason":"JobStarted",
    "sessionId":"24862060-4fcd-11e8-8622-fda0fbf6f550"
}

Step - Publish Results to Tableau Server

The following uses the HYPER results from the specified job (jobId = 2) to publish the results to the test_table3 table in the default Tableau Server database through connectionId=3.

Request:

{
  "connection": {
    "id": 3
  },
  "path": ["default"],
  "table": "test_table3",
  "action": "createAndLoad",
  "inputFormat": "hyper"
}

Response:

{
    "jobgroupId":2,
    "reason":"JobStarted",
    "sessionId":"24862060-4fcd-11e8-8622-fda0fbf6f552"
}

Step - Publish Results to SQL DW

The following uses the Parquet results from the specified job (jobId = 2) to publish the results to the test_table4 table in the dbo SQL DW database through connectionId=4.

Request:

{
  "connection": {
    "id": 4
  },
  "path": ["dbo"],
  "table": "test_table4",
  "action": "createAndLoad",
  "inputFormat": "pqt"
}

Response:

{
    "jobgroupId": 2,
    "jobIds": 22,
    "reason": "JobStarted",
    "sessionId": "855f83a0-dc94-11e8-bd1a-f998d808020d"
}

{
  "connection": {
    "id": 2
  },
  "path": ["public"],
  "table": "test_table2",
  "action": "create",
  "inputFormat": "avro"
}

{
    "jobgroupId":2,
    "reason":"JobStarted",
    "sessionId":"fae64760-4fc4-11e8-8cba-0987061e4e16"
}