Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0871

D toc

Excerpt

This section describes how you interact through

...

the 

D s platform
rtrue
 with

...

D s conntype
rtrue
typesqldatawarehouse
.


Limitations

Limitations

  • D s conntype
    typesqldatawarehouse
     connections are available only if you have deployed the 
    D s platform
     onto Azure.
  • When using the SQL DW read-write connection, the The defined length of a table row cannot exceed 1 MB.

    Info

    NOTE: In this release, this connection cannot be created through the APIs. Please create connections of this type through the application.

Uses

...

of 
D s conntype
typesqldatawarehouse

The

D s platform
 can use SQL DW for
D s conntype
typesqldatawarehouse
 for the following tasks:

  1. Create datasets by reading from SQL DW tables
    D s conntype
    typesqldatawarehouse
     tables.
  2. Write to SQL DW tables

    D s conntype
    typesqldatawarehouse
     tables with your job results.

  3. Ad-hoc publication of data to SQL DW

    D s conntype
    typesqldatawarehouse
    .

Before You Begin

...

Using 
D s conntype
typesqldatawarehouse

  • Enable Access: Integration requires the following:

    • Installation of the 
      D s platform
       on Microsoft Azure.
    • Either ADL or WASB is supported as the base storage layer. For more information, see Set Base Storage Layer.
  • Read Access: Your SQL DW administrator

    D s conntype
    typesqldatawarehouse
     administrator must configure read permissions. Your administrator should provide a database for upload to your SQL DW datastore.

  • Write Access: You can write and publish jobs results to SQL DW

    D s conntype
    typesqldatawarehouse

Secure Access

SQL DW These connections require SSL access.

Storing Data

...

in 
D s conntype
typesqldatawarehouse

Your

D s conntype
typesqldatawarehouse
 administrator should provide database access for storing datasets. Users should know where shared data is located and where personal data can be saved without interfering with or confusing other users. 

Info

NOTE: The

D s platform
does not modify source data in SQL DW. Datasets sourced from SQL DW
D s conntype
typesqldatawarehouse
are read without modification from their source locations.

Reading

...

from 
D s conntype
typesqldatawarehouse

You can create a

D s item
itemdataset
 from a table stored in SQL DW
D s conntype
typesqldatawarehouse
.

For more information, see Database Browser.

Writing

...

to 
D s conntype
typesqldatawarehouse

You can write back data to SQL DW using

D s conntype
typesqldatawarehouse
 using one of the following methods:

Info

NOTE: Writing and publishing to SQL DW

D s conntype
typesqldatawarehouse
is not supported if Azure AD SSO has been enabled.

  • Job results can be written directly to SQL DW as
    D s conntype
    typesqldatawarehouse
     as part of the normal job execution. Create a new publishing action to write to SQL DW
    D s conntype
    typesqldatawarehouse
    . See Run Job Page.
  • As needed, you can publish results to SQL DW for previously executed jobs.

    Info

    NOTE: You cannot re-publish results to SQL DW if the original job published to SQL DW. However, if the dataset was transformed but publication to SQL DW failed, you can publish from the Publishing dialog.

    Info

    NOTE: To publish to SQL DW, the source results must be Parquet format.

    See Publishing Dialog

    D s conntype
    typesqldatawarehouse
     for previously executed jobs.

  • For more information on how data is converted to SQL DW
    D s conntype
    typesqldatawarehouse
    , see SQL DW Data Type Conversions.

...

  • No validation is performed for the connection and any required permissions during job execution. So, you can be permitted to launch your job even if you do not have sufficient connectivity or permissions to access the data. The corresponding publish job fails at runtime. 
  • No data validation is performed during writing and publication to SQL DW
    D s conntype
    typesqldatawarehouse
    . Your job fails if the schema for the
    D s item
    dataset
    dataset
    varies from the target schema.
  • Prior to publication, no validation is performed on whether a target is a table or a view, so the job that was launched fails at runtime.