...
To create an imported dataset, you must acquire the following information about the source. In the above example, the source is the
POS-r01.txt
file.- path
- type
- name
- description
- bucket (if a file stored on S3)
Construct the following request:
Endpoint http://www.example.com:3005/v3/importedDatasetimportedDatasets
Authentication Required Method POST
Request Body Code Block { "path": "/user/pos/POS-r01.txt", "type": "hdfs", "bucket": null, "name": "POS-r01.txt", "description": "POS-r01.txt" }
You should receive a
201 - Created
response with a response body similar to the following:Code Block { "id": 8, "size": "281032", "path": "/user/pos/POS-r01.txt", "isSharedWithAll": false, "type": "hdfs", "bucket": null, "isSchematized": false, "createdBy": 1, "updatedBy": 1, "updatedAt": "2017-02-08T18:38:56.640Z", "createdAt": "2017-02-08T18:38:56.560Z", "connectionId": null, "parsingScriptId": 14, "cpProject": null }
You must retain the
id
value so you can reference it when you create the recipe.Next, you create the recipe. Construct the following request:
Endpoint http://www.example.com:3005/v3/wrangledDataset
Authentication Required Method POST
Request Body Code Block { "name":"POS-r01", "importedDataset":{"id":8}, "flow":{"id":10} }
You should receive a
201 - Created
response with a response body similar to the following:Code Block { "id": 23, "flowId": 10, "scriptId": 24, "wrangled": true, "createdBy": 1, "updatedBy": 1, "updatedAt": "2017-02-08T20:28:06.067Z", "createdAt": "2017-02-08T20:28:06.067Z", "flowNodeId": null, "deleted_at": null, "activesampleId": null, "name": "POS-r01", "active": true }
- From the recipe, you must retain the value for the
id
. For more information, see API WrangledDatasets Create v3.
- Repeat the above steps for each of the source files that you are adding to your flow.
...
Endpoint | http://www.example.com:3005/v3/jobgroupjobGroups/<id>/status |
---|---|
Authentication | Required |
Method | GET |
Request Body | None. |
...