Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0682

...

For more information, see API Connections WrangledDatasets Get v4.

Step - Create outputObject

...

Code Block
curl -X GET \
  http://www.wrangle-dev.example.com/v4/connections \
  -H 'authorization: Basic <auth_token>' \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json'

D s terms
areacurl
For more information, see API Connections Get List v4.

 

Step - Create a Publication

You can create publications that publish table-based outputs through specified connections. In the following, a Hive table is written out to the default database through connectionId = 1. This publication is associated with the outputObject id=4.

Request:

Endpointhttp://www.wrangle-dev.example.com:3005/v4/publications
AuthenticationRequired
MethodPOST
Request Body
Code Block
{
    "path": [
        "default"
    ],
    "tableName": "myPublishedHiveTable",
    "targetType": "hive",
    "action": "create",
    "outputObject": {
        "id": 4
    },
    "connection": {
        "id": 1
    }
}

Response:

Status Code201 - Created
Response Body
Code Block
{
    "path": [
        "default"
    ],
    "id": 3,
    "tableName": "myPublishedHiveTable",
    "targetType": "hive",
    "action": "create",
    "updatedAt": "2018-11-13T01:25:39.698Z",
    "createdAt": "2018-11-13T01:25:39.698Z",
    "creator": {
        "id": 1
    },
    "updater": {
        "id": 1
    },
    "outputObject": {
        "id": 4
    },
    "connection": {
        "id": 1
    }
}

cURL example:

Code Block
curl -X POST \
  http://example.com:3005/v4/publications \
  -H 'authorization: Basic <auth_token>' \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json' \
  -d '{
    "path": [
        "default"
    ],
    "tableName": "myPublishedHiveTable",
    "targetType": "hive",
    "action": "create",
    "outputObject": {
        "id": 4
    },
    "connection": {
        "id": 1
    }
}' 

D s terms
areacurl

Tip

Checkpoint: You're done.

You have done the following:

  1. Created an output object:
    1. Embedded a writeSettings object to define an Avro output.
    2. Associated the outputObject with a recipe.
  2. Added another writeSettings object to the outputObject.
  3. Added a table-based publication object to the outputObject.

You can now generate results for these three different outputs whenever you run a job (create a jobgroup) for the associated recipe. 

Step - Apply Overrides

When you are publishing results to a relational source, you can optionally apply overrides to the job to redirect the output or change the action applied to the target table. For more information, see API Workflow - Run Job.