Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

To test your rule, perform a dry run of the import. See below.

Example - Remap a WASB location

For WASB sources, you can apply remapping rules during import. 

In this example, your import rule must remap the blob host, container, and file location:

Dev Blobhoststorage-wasb-account-dev.blob.core.windows.net
Dev Containercontainer-dev
Dev File Location/projs/work/orders.csv
Prod Blobhoststorage-wasb-account-prod.blob.core.windows.net
Prod Containercontainer-prod
Prod File Location/2003/transactions/orders.csv

You can use the following value import rules to change the blobhost, container, and file paths.

Info

NOTE: You can specify multiple rules in a single request. Rules are applied in the order that they are listed. Latter rules must factor the results of earlier rules.

host rule: This rule replaces the blobhost name to use with the new one: storage-wasb-account-prod.blob.core.windows.net.

userinfo rule: This rule replaces the container name to use with the new one: container-prod.

fileLocation rule: This rule performs a text substitution to replace the file path. This rule applies to both input and output object file paths.

Request:

Itemv4 APIs
API Endpoint


Code Block
/v4/deployments/4/valueImportRules


Method

PATCH

Request Body:


Code Block
[
  {"type":"host","on":"storage-wasb-account-dev.blob.core.windows.net","with":"storage-wasb-account-prod.blob.core.windows.net"},
  {"type":"userinfo","on":"container-dev","with":"container-prod"},
  {"type":"fileLocation","on":"/projs/work/orders.csv","with":"/2003/transactions/orders.csv"}
]


Response:

Itemv4 APIs
Status Code - Success200 - OK
Response Body

When the new rules are applied, all previously existing rules for the object in the deployment are deleted. The response body contains any rules that have been deleted as part of this request.

In the following example, there were no rules, so nothing was deleted:

Code Block
{
    "deleted": {
        "data": []
    }
}


Documentation

See

D s api refdoclink
operation/updateValueImportRules


To test your rule, perform a dry run of the import. See below.

Example - Remap an ADLS Gen2 location

For ADLs Gen2 sources, you can apply remapping rules during import. 

In this example, your import rule must remap the storage account, filesystem, and file location:

Dev Storage Accountstorage-adlsgen2-account-dev.blob.core.windows.net
Dev Filesystemfilesystem-dev
Dev File Location/projs/work/orders.csv
Prod Storage Accountstorage-adlsgen2-account-prod.blob.core.windows.net
Prod Filesystemfilesystem-prod
Prod File Location/2003/transactions/orders.csv

You can use the following value import rules to change the storage account, filesystem, and file paths.

Info

NOTE: You can specify multiple rules in a single request. Rules are applied in the order that they are listed. Latter rules must factor the results of earlier rules.

host rule: This rule replaces the storage account name to use with the new one: storage-adlsgen2-account-prod.blob.core.windows.net.

userinfo rule: This rule replaces the filesystem name to use with the new one: filesystem-prod.

fileLocation rule: This rule performs a text substitution to replace the file path. This rule applies to both input and output object file paths.

Request:

Itemv4 APIs
API Endpoint


Code Block
/v4/deployments/4/valueImportRules


Method

PATCH

Request Body:


Code Block
[
  {"type":"host","on":"storage-adlsgen2-account-dev.blob.core.windows.net","with":"storage-adlsgen2-account-prod.blob.core.windows.net"},
  {"type":"userinfo","on":"filesystem-dev","with":"filesystem-prod"},
  {"type":"fileLocation","on":"/projs/work/orders.csv","with":"/2003/transactions/orders.csv"}
]


Response:

Itemv4 APIs
Status Code - Success200 - OK
Response Body

When the new rules are applied, all previously existing rules for the object in the deployment are deleted. The response body contains any rules that have been deleted as part of this request.

In the following example, there were no rules, so nothing was deleted:

Code Block
{
    "deleted": {
        "data": []
    }
}


Documentation

See

D s api refdoclink
operation/updateValueImportRules


To test your rule, perform a dry run of the import. See below.

Example - Remap an ADLS Gen1 location

For ADLS Gen1 sources, you can apply remapping rules during import. 

In this example, your import rule must remap the Azure data lake store and file location:

Dev data storeadl://storage-adlsgen1-account.azuredatalakestore.net
Dev File Location/projs/work/orders.csv
Prod data storeadl://storage-adlsgen1-account-prod.azuredatalakestore.net
Prod File Location/2003/transactions/orders.csv

You can use the following value import rules to change the datastore and file paths.

Info

NOTE: You can specify multiple rules in a single request. Rules are applied in the order that they are listed. Latter rules must factor the results of earlier rules.

host rule: This rule replaces the datastore name to use with the new one: storage-adlsgen1-account-prod.azuredatalakestore.net.

fileLocation rule: This rule performs a text substitution to replace the file path. This rule applies to both input and output object file paths.

Request:

Itemv4 APIs
API Endpoint


Code Block
/v4/deployments/4/valueImportRules


Method

PATCH

Request Body:


Code Block
[
  {"type":"host","on":"storage-adlsgen1-account-dev.azuredatalakestore.net","with":"storage-adlsgen1-account-prod.azuredatalakestore.net"},
  {"type":"fileLocation","on":"/projs/work/orders.csv","with":"/2003/transactions/orders.csv"}
]


Response:

Itemv4 APIs
Status Code - Success200 - OK
Response Body

When the new rules are applied, all previously existing rules for the object in the deployment are deleted. The response body contains any rules that have been deleted as part of this request.

In the following example, there were no rules, so nothing was deleted:

Code Block
{
    "deleted": {
        "data": []
    }
}


Documentation

See

D s api refdoclink
operation/updateValueImportRules


To test your rule, perform a dry run of the import. See below.

Example - Remap a relational datasource

...