This section describes how to run a deployment using the APIs available in .

Pre-requisites

Before you begin, you should verify the following:

  1. Get authentication credentials. As part of each request, you must pass in authentication credentials to the 

    Tip: The recommended method is to use an access token, which can be generated from the . For more information, see Access Tokens Page.

    For more information, see API Authentication.

  2. Develop your flow. Before you deploy a flow through the Deployment Manager, you should build and test your flow in a development environment. See API Workflow - Develop a Flow.
  3. Deploy your flow. After you have developed a flow, you can deploy it. See API Workflow - Deploy a Flow.
  4. Acquire deployment identifier. In Deployment Manager, acquire the numeric value for the deployment. See Deployment Manager Page.

Step - Run Deployment

Through the APIs, you can run a deployment. When a deployment is executed:

Tip: When running a deployment, you cannot apply overrides through the API request. However, overrides can be specified through a different mechanism described later.


In the following example, the deployment identifier is 2.

Endpoint<protocol>://<platform_base_url>/v4/deployments/2/run
AuthenticationRequired
MethodPOST
Request Body

None.

Response Code201 - Created
Response Body
{
    "data": [
        {
            "reason": "JobStarted",
            "sessionId": "dd6a90e0-c353-11e7-ad4e-7f2dd2ae4621",
            "id": 33
        }
    ]
}

If the 201 response code is returned, then the deployment job has been queued for execution. 

Tip: Retain the id value in the response. In the above, 961247 is the internal identifier for the job group for the job. You will need this value to check on your job status.

For more information, see API Workflow - Deploy a Flow.

Checkpoint: You have queued your deployment job for execution.


Step - Monitoring Your Deployment Job

You can monitor the status of your deploy job through the following endpoint using the id value that was returned in the previous step:

<protocol>://<platform_base_url>/v4/jobGroups/<id>/status

For more information, see API Workflow - Run Job.


Step - Run Deployment with Overrides

When you run a deployment, you cannot apply overrides to the request. However, you can complete the following steps to apply overrides when you execute the jobs within the active release. In this workflow, you run jobs on the specific recipes of the active deployment, applying overrides as needed.

Suppose you are running the jobs for deployment id 2, and you want to apply some overrides.

NOTE: A deployment can trigger several different jobs within a single flow. In the following example, it is assumed that there is only one output.


Acquire the active outputs for the deployment

The first step is to acquire all of the active outputs for the deployment.

Endpoint<protocol>://<platform_base_url>/v4/deployments/2/activeoutputs
AuthenticationRequired
MethodGET
Request Body

None.

Response Code200 - OK
Response Body
{
    "data": [
        {
            "outputObjectId": 6,
            "flowNodeId": 27,
            "recipeName": "USDA_Farmers_Market_2014"
        }
    ]
}

in the above response:

Acquire output object information

The next step is to review the output object to determine what needs to be overridden. Since you are overriding a file-based publication, you can query directly for the writeSettings objects associated with the output object:

Endpoint<protocol>://<platform_base_url>/v4/outputObjects/6/writeSettings
AuthenticationRequired
MethodGET
Request Body

None.

Response Code200 - OK
Response Body
{
    "data": [
        {
            "delim": ",",
            "id": 17,
            "path": "hdfs://hadoop:50070/example/joe@example.com/USDA_Farmers_Market_2014.csv",
            "action": "create",
            "format": "csv",
            "compression": "none",
            "header": true,
            "asSingleFile": true,
            "prefix": null,
            "suffix": "_increment",
            "hasQuotes": true,
            "createdAt": "2019-11-05T18:26:31.972Z",
            "updatedAt": "2019-11-05T18:30:56.756Z",
            "creator": {
                "id": 2
            },
            "updater": {
                "id": 2
            },
            "outputObject": {
                "id": 6
            }
        },
        {
            "delim": ",",
            "id": 16,
            "path": "hdfs://hadoop:50070/example/joe@example.com/USDA_Farmers_Market_2014.json",
            "action": "create",
            "format": "json",
            "compression": "none",
            "header": false,
            "asSingleFile": false,
            "prefix": null,
            "suffix": "_increment",
            "hasQuotes": false,
            "createdAt": "2019-11-05T18:26:44.983Z",
            "updatedAt": "2019-11-05T18:30:56.743Z",
            "creator": {
                "id": 2
            },
            "updater": {
                "id": 2
            },
            "outputObject": {
                "id": 6
            }
        }
    ]
}

Run deployment with overrides

Now that you have access to the outputs generated by the deployment, you decide to override the following for each file:

From the activeOutputs endpoint, you retrieved the flowNodeId (27).

Endpoint<protocol>://<platform_base_url>/v4/jobGroups/
AuthenticationRequired
MethodPOST
Request Body
{
  "wrangledDataset": {
    "id": 27
  },
  "overrides": {
    "profiler": false,
    "writesettings": [
      {
        "path": "hdfs://hadoop:50070/example/joe@example.com/final/USDA_Farmers_Market.csv",
        "format": "csv",
        "action": "overwrite"
      },
      {
        "path": "hdfs://hadoop:50070/example/joe@example.com/final/USDA_Farmers_Market.json",
        "format": "json",
        "action": "overwrite"
      }
    ]
  },
  "ranfrom": null
}


Response Code201 - Created
Response Body
{
    "sessionId": "b29467c3-fc59-499e-aed6-d797746a86eb",
    "reason": "JobStarted",
    "jobGraph": {
        "vertices": [
            10,
            11,
            12
        ],
        "edges": [
            {
                "source": 10,
                "target": 11
            },
            {
                "source": 10,
                "target": 12
            }
        ]
    },
    "id": 4,
    "jobs": {
        "data": [
            {
                "id": 10
            },
            {
                "id": 11
            },
            {
                "id": 12
            }
        ]
    }
}

Checkpoint: Your job with overrides has been queued for execution.

You can use the job identifier (4) to monitor the job status.