Review the changes to the publicly available REST APIs for

Changes for Release 8.2

None.

Changes for Release 8.1

Customize relational connectors via API

Beginning in Release 8.1, you can customize aspects of each relational connection type available in your product edition through a set of APIs. Some terms:

• connection: In the

  D s webapp

  and via API, you can create and manage connections between the platform a specific datastore. A connection is the user-defined object that enables the connection to the datastore.

• connector: A connection interfaces with a connector, which is an underlying driver and its related configuration, that perform the actual connection. This configuration information includes runtime, publishing, and connection definitions.

  Tip
  Tip: All connections of the same type use the same underlying connector, including its configuration. Overrides that you apply to a connector apply to all current and new connections of that type in the workspace.

Get connector identifier

To use these API endpoints, you must acquire the connector identifier. This value is the vendor value for a connection of the type. You can acquire this value in one of two ways:

• Create a connection of the type in the

  D s webapp

  . Use the /v4/connections/:id endpoint with the GET method to acquire the connection information for your connection. Acquire the vendor value.

  D s api refdoclink
  operation/listConnections

• You may find the vendor values listed in the documentation. See Connection Types.

Get connector metadata information - defaults

The following endpoint returns the default metadata information for a specified connector type. This information is stored in the Connector Configuration Service database.

For the :connectorId value below, use the vendor value that you acquired above. For example, to acquire connector type definitions for MySQL connection type, use the value mysql.

operation/getConnectorDefaults

Get connector metadata information - current values

The following endpoint acquires the current metadata for a specified connector type, which include the default values with any applicable overrides applied to them.

operation/getConnectorConfig

Get connector metadata information - get overrides values

The following endpoint retrieves the overrides that have been applied to a specific connector.

operation/getConnectorOverrides

Create overrides for a connector

The following endpoint applies the specified value or values as overrides to the connector.

operation/updateConnectorOverrides

Delete overrides for a connector

The following endpoint deletes all override values for a specified connector.

operation/deleteConnectorOverrides

Changes for Release 8.0

Deprecation of prior asset transfer endpoint

The following endpoint has been available for transferring assets between users:

/v4/people/{to-userid:}/assetTransfer/{from-userid:}

This endpoint is not compatible with the forward direction of the platform.

For more information, see

operation/transferAssets

Changes for Release 7.7

credentialProvider no longer required for /v4/awsConfig

In prior releases, API requests to the /v4/awsConfig endpoint using the PUT method could receive the following error:

{
"exception":
{ "name": "ApiValidationFailed", "message": "Message does not adhere to API specification", "details": "'credentialProvider' field in request body must not be null" }}

The credentialProvider field is not required, and this requirement has been removed from the endpoint.

operation/updateAwsConfig

/v4/awsConfigs PUT method has been deprecated

In prior releases, the /v4/awsConfigs endpoint supported the use of a PUT method to modify AWS configuration objects.

This endpoint-method combination has been deprecated. The new PATCH method is the following:

{
    "role":"<my_iam_role_object_3>"
}

operation/updateAwsConfig

For more information on configuring AWS access objects, see API Workflow - Manage AWS Configurations.

Changes for Release 7.5

Flow sharing API now accepts user email addresses

Beginning in Release 7.5, the API has been enhanced to allow insertion of a user's email address as part of the request body. Below is an example request:

    "data": [
        {
            "email": "user@example.com",
            "role": "collaborator",
            "policy": "flow_editor"
        }
    ]
}

If the above returns a 201 - Success status message, then user@example.com has been given the role of collaborator, which uses the flow_editor policy, for flowId 402.

operation/shareFlow

Changes for Release 7.1

Introducing in-app API reference documentation

Beginning in Release 7.1, API reference documentation is available directly from the 

Key features:

• More endpoints and attributes are now available!
• Reference content comes directly from the codebase.
• Searchable and browsable

More content and features will be added to this new capability of the next few releases. 

Workflow documentation is still available in the product documentation. For more information on these workflows, see API Reference.

API changes for authorization

Release 7.1 introduces authorization, which provides finer-grained access controls to user-defined objects in the platform. For more information, see Changes to User Management.

As a result of authorization, the following changes have been applied to the listed API endpoints.

Connection permissions - POST

Change:

When setting permissions, the request body now must include the policyTag parameter:

"policyTag": "connection_viewer",

For more information, see the API reference documentation:

operation/createConnectionPermission

Connection permissions - GET

Change:

Response no longer includes connectionPermissions parameter.

Response does include the following new parameters:

"policyTag": "connection_viewer",
"workspaceAdmin": false,
"isCreatedBy": true,

For details on these parameters, please see the API Reference documentation:

operation/getConnectionPermissions

Connections - GET LIST

Change:

Response now includes an authorizationPermission field in the associatedPeople block for each connection:

{
    "data": [
        {
            "id": 14,
            ....
            "associatedPeople": {
                "data": [
                    {
                        "id": 1,
                        ...
                        "authorizationPermission": {
                            "policyName": null,
                            "policyTag": "workspace_admin",
                            "resourceOperationList": [
                                {
                                    "operations": [
                                        "execute",
                                        "delete",
                                        "update",
                                        "read",
                                        "create",
                                        "share"
                                    ],
                                    "policyTag": "plan_author",
                                    "resourceType": "plan"
                                },
                                {
                                    "operations": [
                                        "execute",
                                        "delete",
                                        "update",
                                        "read",
                                        "create",
                                        "share"
                                    ],
                                    "policyTag": "flow_author",
                                    "resourceType": "flow"
                                },

For details on these parameters, please see the API Reference documentation:

operation/listConnections

Connections - GET

Change:

Same change as previous.

For details on these parameters, please see the API Reference documentation:

operation/getConnection

People - GET LIST

Change:

Response now includes the following attributes:

• maximalPrivileges
• authorizationRoles

{
    "data": [
        {
            "id": 29,
            ....
           "maximalPrivileges": [
                {
                    "operations": [
                        "delete",
                        "update",
                        "read",
                        "create",
                        "share"
                    ],
                    "policyTag": "connection_author",
                    "resourceType": "connection"
                },
                {
                    "operations": [
                        "execute",
                        "delete",
                        "update",
                        "read",
                        "create",
                        "share"
                    ],
                    "policyTag": "plan_author",
                    "resourceType": "plan"
                },
                {
                    "operations": [
                        "execute",
                        "delete",
                        "update",
                        "read",
                        "create",
                        "share"
                    ],
                    "policyTag": "flow_author",
                    "resourceType": "flow"
                }
            ],
            "authorizationRoles": [
                {
                    "policyId": 13,
                    "name": "default",
                    "tag": null,
                    "workspaceId": 1,
                    "createdAt": "2020-04-22T07:50:04.502+0000",
                    "updatedAt": "2020-04-22T07:50:04.502+0000",
                    "resourceOperations": [
                        {
                            "operations": [
                                "execute",
                                "delete",
                                "update",
                                "read",
                                "create",
                                "share"
                            ],
                            "policyTag": "plan_author",
                            "resourceType": "plan"
                        },
                        {
                            "operations": [
                                "execute",
                                "delete",
                                "update",
                                "read",
                                "create",
                                "share"
                            ],
                            "policyTag": "flow_author",
                            "resourceType": "flow"
                        },
                        {
                            "operations": [
                                "delete",
                                "update",
                                "read",
                                "create",
                                "share"
                            ],
                            "policyTag": "connection_author",
                            "resourceType": "connection"
                        }
                    ]
                }
            ],

For details on these parameters, please see the API Reference documentation:

operation/listPeople

People - GET

Change:

Same change as previous.

For details on these parameters, please see the API Reference documentation:

operation/getPerson

Create Salesforce connection changes

Beginning in Release 7.1, the structure of a Salesforce connection object has been changed. In prior releases, a Salesforce connection could not be shared without also sharing credentials. These changes allow a Salesforce connection to be shared without requiring the sharing of credentials. 

Prior to Release 7.1, when creating a Salesforce connection using the APIs, the /v4/connections POST method required the insertion of a Salesforce security token as part of the params section of the request:

{
  "vendor": "salesforce",
  "vendorName": "salesforce",
  "type": "jdbc",
  "credentialType": "basic",
  "ssl": true,
  "name": "Salesforce",
  "description": "example_salesforce_connection",
  "disableTypeInference": true,
  "isGlobal": true,
  "credentialsShared": true,
  "host": "salesforce.example.com",
  "params": {
    "servername": "myServer",
    "securityToken": "string"
  },
  "credentials": [
    {
      "username": "string",
      "password": "string"
    }
  ]
}

Beginning in Release 7.1, the following changes have been applied to this request structure:

• Change credentialType to securityToken

• Move securityToken key from params to credentials area of the API request

• A host and port number value are not required.

{
  "vendor": "salesforce",
  "vendorName": "salesforce",
  "type": "jdbc",
  "credentialType": "securityToken",
  "ssl": true,
  "name": "Salesforce",
  "description": "example_salesforce_connection",
  "disableTypeInference": true,
  "isGlobal": true,
  "credentialsShared": true,
  "host": "salesforce.example.com",
  "params": {
    "servername": "myServer"
  },
  "credentials": [
    {
      "username": "string",
      "password": "string",
      "securityToken": "string"
    }
  ]
}

For more information:

• API: 

  D s api refdoclink
  operation/createConnection

• UI: Create Salesforce Connections

Apply overrides to file-based data sources when you run a job

You can now apply overrides to your file-based data sources when you run a job using the APIs. See API Workflow - Run Job.

Changes for Release 7.0

v3 endpoints are no longer available

The v3 version of the API endpoints are no longer available in the platform.

You must use the v4 versions. See API Reference.

Path to file-based imported datasets can be specified by URI

In Release 6.8 and earlier, you could create an imported dataset from a file-based source using a request like the following:

{
  "path": "/tri-hdfs/uploads/1/4aee9852-cf92-47a8-8c6a-9ff2adeb3b4a/POS-r02.txt",
  "type": "hdfs",
  "bucket": null,
  "name": "POS-r02b.txt",
  "description": "POS-r02 - copy"
}

Beginning in this release, you can specify the source using a single uri value:

{
  "uri": "hdfs:///tri-hdfs/uploads/1/4aee9852-cf92-47a8-8c6a-9ff2adeb3b4a/POS-r02.txt",
  "name": "POS-r02b.txt",
  "description": "POS-r02 - copy"
}

For more information, see 

operation/createImportedDataset

awsRoles GET method removed

Prior to Release 7.0, the platform supported the ability to retrieve awsRole objects based on awsRoleId value.

In Release 7.0 and later, this endpoint has been disabled in the product. 

operation/listAwsRoles

Connections endpoints simplified

Prior to Release 7.0, the request to create a connection looked liked the following:

{
    "connectParams": {
        "vendor": "sqlserver",
        "vendorName": "sqlserver",
        "host": "sqlserver.example.com",
        "port": "1433"
    },
    "host": "sqlserver.example.com",
    "port": 1433,
    "vendor": "sqlserver",
    "params": {
        "connectStrOpts": ""
    },
    "ssl": false,
    "vendorName": "sqlserver",
    "name": "sqlserver_test2",
    "description": "",
    "type": "jdbc",
    "isGlobal": false,
    "credentialType": "basic",
    "credentialsShared": true,
    "disableTypeInference": false,
    "credentials": [
        {
            "username": "<username>",
            "password": "<password>"
        }
    ]
}

In the above, the connectParams information is duplicated elsewhere in the request. 

Beginning in Release 7.0, the connectParams elements of the Connections endpoint have been removed. 

The above request now looks like the following:

{
    "host": "sqlserver.example.com",
    "port": 1433,
    "vendor": "sqlserver",
    "params": {
        "connectStrOpts": ""
    },
    "ssl": false,
    "vendorName": "sqlserver",
    "name": "sqlserver_test2",
    "description": "",
    "type": "jdbc",
    "isGlobal": false,
    "credentialType": "basic",
    "credentialsShared": true,
    "disableTypeInference": false,
    "credentials": [
        {
            "username": "<username>",
            "password": "<password>"
        }
    ]
}

Required changes:

• The params and name attributes are now required when creating a new connection. 
• If you have downstream scripts that utilize the connectParams objects, these must be updated to reference the corresponding attributes after upgrade to Release 7.0 or later.

For more information, see 

operation/getConnection

Publishing for a jobGroup has been simplified

After you have run a job, you can publish the results from the job using the following endpoint:

/v4/jobGroups/<id>/publish

Prior to Release 7.0, the request to submit to the endpoint looked like the following:

{
  "connection": {
    "id": 1
  },
  "path": ["default"],
  "table": "test_table",
  "action": "create",
  "inputFormat": "pqt",
  "flowNodeId": 27
}

However, the flowNodeId value is not necessary, as it can be inferred from the id value that is part of the endpoint URI. 

Beginning in Release 7.0, the request no longer requires the flowNodeId value:

{
  "connection": {
    "id": 1
  },
  "path": ["default"],
  "table": "test_table",
  "action": "create",
  "inputFormat": "pqt"
}

For more information, see 

operation/publishJobGroup

WASB URI format has changed

In Release 6.8 and earlier, the supported format for WASB URIs was the following:

wasbs://blobhost.blob.core.windows.net/container/path/to/file.csv

This format was not an Azure standard. Beginning in Release 7.0, the following standards-based format is supported:

wasbs://container@blobhost.blob.core.windows.net/path/to/file.csv

Saving Databricks Personal Access Token has different response

In Release 6.8 and earlier, when saving a Databricks Personal Access Token, the response looked like the following:

Beginning in Release 7.0, the response looks like the following:

{
  "message": "Databricks Personal Access Token has been stored for user <user_email>"
}

For more information, see

operation/saveDatabricksAccessToken

Documentation for awsRoles object corrected

In Release 6.8 and earlier, the following issues appeared in the documentation for the awsRoles object. These issues have been corrected in this release:

Inadequate permissions:

The POST and PUT methods for the awsRoles endpoint require either of the following user permissions:

• Workspace admin

• D s item
  item admin

Optional workspaceid/personid attributes:

For the POST and PUT methods for the awsRoles endpoint, you can optionally assign the role to a workspace or person.

See

operation/createAwsRole

See

operation/updateAwsRole

Changes for Release 6.8.1

Improvements to AssetTransfer API documentation

Before disabling or deleting a user's account, an admin may wish to transfer the user's assets to the ownership of a different user. For more information on how to transfer all assets via API, see 

operation/transferAssets

Changes for Release 6.8

Overrides to relational sources and targets

Through the APIs, you can now apply overrides to relational sources and targets during job execution or deployment import. 

jobGroups

When you are running a job, you can override the default publication settings for the job using overrides in the request. For more information, see API Workflow - Run Job.

Deployments

When you import a flow package into a deployment, you may need to remap the source and output of the flow to use production versions of your data. This capability has been present in the product for file-based sources and targets. Now, it's available for relational sources and targets. For more information, see Define Import Mapping Rules. 

New flow object definition

Release 6.8 introduces a new version of the flow object definition. This new version will support cross-product and cross-version import and export of flows in the future. For more information see Changes to the Object Model.

Export and import macros

Beginning in Release 6.8, you can export and import macro definitions via API.

• Export: 

  D s api refdoclink
  operation/getMacroPackage

• Import: 

  D s api refdoclink
  operation/importMacroPackage

Changes for Release 6.4

Request format for assigning connection permissions endpoint has changed

For this endpoint:

/v4/connections/<cid>/permissions/

where:

• <cid> is in the internal connection identifier.

The request payload format has changed.

Before Release 6.4, the request format was as follows:

[
  {
    "personId": 3,
    "role": "readOnly"
  }
]

Beginning in Release 6.4, the request format is as follows:

{
  "data": [
    {
      "person": {
        "id": 3
      },
      "role": "readOnly"
    }
  ]
}

For more information, see

operation/createConnectionPermission

v4 version of password reset request endpoint

To assist in migration from the command-line interface to using the APIs, a v4 version of an API endpoint has been made available to allow for administrators to generate password reset codes.

Changes to awsConfig object

In Release 6.0, the awsConfig object was introduced to enable the assignment of AWS configurations to individual users (per-user auth) via API. This version of the awsConfig object supported a mapping of a single IAM role to an awsConfig object.

Beginning in Release 6.4, per-user authentication now supports mapping of multiple possible IAM roles to an individual user's configuration. To enable this one-to-many mapping, the awsRoles object was introduced.

• An awsRoles object creates a one-to-one mapping between an IAM role and an awsConfig object.
• An awsConfig object can have multiple awsRoles assigned to it.

Changes to awsConfig object:

• The role field in the object has been replaced by activeRoleId, which maps to the active role for the configuration object.
• For each role reference in the awsConfig objects, a corresponding awsRole object has been created and mapped to it.

Beginning in Release 6.4, you can create, edit, and delete awsRoles objects, which can be used to map an AWS IAM role ARN to a specified AWSConfig object. You can map multiple awsRoles to a single awsConfig.

For more information, see API Workflow - Manage AWS Configurations.

Changes for Release 6.3

Assign AWSConfigs to a user at create time

Beginning in Release 6.3, you can assign an AWSConfig object to a user when you create the object. This shortcut reduces the number of REST calls that you need to make. 

Prior to Release 6.3, AWSConfig objects were assigned through the following endpoint. Example:

/v4/people/2/awsConfigs/6

See API Workflow - Manage AWS Configurations.

Changes for Release 6.0

Error in Release 6.0.x API docs

In Release 6.0 - Release 6.0.2, the online and PDF versions of the documentation referenced the following endpoint: API JobGroups Get Status v4. According to the docs, this endpoint was triggered in this manner:

/v4/jobGroups/<id>/status

This endpoint exists in v3 of the API endpoints. It does not exist in v4.

Instead, you should monitor the status field for the base GET endpoint for jobGroups. For more information, see

operation/getJobGroup

Planned End of Life of v3 API endpoints

In Release 6.0, the v3 API endpoints are supported.

You must migrate to using the v4 API endpoints before upgrading to the next release after Release 6.0. 

Changes for Release 5.9

Introducing Access Tokens

Each request to the API endpoints of the 

• A request could include clear-text username/password combinations. This method is not secure.
• A request could include a browser cookie. This method does not work for well for use cases outside of the browser (e.g. scripts).

Beginning in Release 5.9, API users can manage authentication using access tokens. These tokens obscure any personally identifiable information and represent a standards-based method of secure authentication.

The basic process works in the following manner:

1. API user requests generation of a new token.
   1. This initial request must contain a valid username and password.
   2. Request includes expiration.
   3. Token value is returned in the response.
2. The token value inserted into the Authorization header of each request to the platform.
3. User monitors current time and expiration time of the token. At any time, the user can request a new token to be generated using the same endpoint used in the initial request. 

Access tokens can be generated via API or the 

• API: For more information, see 

  D s api refdoclink
  operation/createApiAccessToken

• D s webapp

  : For more information, see Access Tokens Page.

For more information on API authentication methods, see 

section/Authentication

Changes for Release 5.1

None.

Changes for Release 5.0

Introducing v4 APIs

Release 5.0 signals the introduction of version 4 of the REST APIs. 

v4 conventions

The following conventions apply to v4 and later versions of the APIs:

• Parameter lists are consistently enveloped in the following manner:
  
  

  Code Block
  { "data": [ { ... } ] }

• Field names are in camelCase and are consistent with the resource name in the URL or with the embed URL parameter.

• From early API versions, foreign keys have been replaced with identifiers like the following:
  
  

  v3 and earlier	v4 and later
  Code Block "createdBy": 1,	Code Block "creator": { "id": 1 },
  Code Block "updatedBy": 2,	Code Block "updater": { "id": 2 },

• Publication endpoint references database differently. This change is to make the publishing endpoint for relational targets more flexible in the future. 

  v3 and earlier	v4 and later
  Code Block "database": "dbName",	Code Block "path": ["dbName"],

Changes for Release 4.2

Create Hive and Redshift connections via API

You can create connections of these types via API:

• Only one global Hive connection is still supported.
• You can create multiple Redshift connections.

WrangledDataset endpoints are still valid 

In Release 4.1.1 and earlier, the WrangledDataset endpoints enabled creation, modification, and deletion of a wrangled dataset object, which also created the associated recipe object.  

In Release 4.2, wrangled datasets have been removed from the application interface. However, the WrangledDataset API endpoints remain, although they now apply to the recipe directly.

The following endpoints and methods are still available:

For more information, see Changes to the Object Model.