Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r092

...

Excerpt

This section describes how you interact through the 

D s platform
rtrue
 with your SQL DW
D s conntype
rtrue
typesqldatawarehouse
data warehouse.

...


Limitations

  • Microsoft SQL DW connections are
  • D s conntype
    typesqldatawarehouse
     connections can interact with data stored as managed tables or external tables. 
  •  
    D s conntype
    typesqldatawarehouse
     connections can use dedicated or serverless SQL pools. 
  • For more information, see Microsoft SQL Data Warehouse Connections.

Limitations

  • D s conntype
    typesqldatawarehouse
     connections are available only if you have deployed the 
    D s platform
     onto Azure.
  • When using the SQL DW read-write connection, the The defined length of a table row cannot exceed 1 MB.

    Info

    NOTE: In this release, this connection cannot be created through the APIs. Please create connections of this type through the application.

Uses

...

of 
D s conntype
typesqldatawarehouse

The 

D s platform
 can use SQL DW for use 
D s conntype
typesqldatawarehouse
 for the following tasks:

  1. Create datasets by reading from SQL DW tables.Write to SQL DW tables from 
    D s conntype
    typesqldatawarehouse
     tables.
  2. Write to 

    D s conntype
    typesqldatawarehouse
     tables with your job results.

  3. Ad-hoc publication of data to SQL DWto 

    D s conntype
    typesqldatawarehouse
    .

Before You Begin

...

Using 
D s conntype
typesqldatawarehouse

  • Enable SQL DW Access: SQL DW integration Integration requires the following:

    • Installation of the 
      D s platform
       on Microsoft Azure.
    • Either ADL or WASB is supported as the base storage layer. For more information, see Set Base Storage Layer.
  • Read Access: Your SQL DW administrator must configure read permissions. Your administrator should provide a database for upload to your SQL DW datastore.

  • Write Access: You can write and publish jobs results to SQL DWto 

    D s conntype
    typesqldatawarehouse

Secure Access

SQL DW These connections require SSL access.

Storing Data

...

Your 

D s conntype
typesqldatawarehouse
 administrator should provide database access for storing datasets. Users should know where shared data is located and where personal data can be saved without interfering with or confusing other users. 

Info

NOTE: The

D s platform
does not modify source data in SQL DW. Datasets sourced from SQL DW
D s conntype
typesqldatawarehouse
 connections are read without modification from their source locations.

Reading

...

Data

You can create a 

D s item
itemdataset
 from a table stored in SQL DWmanaged or external table through 
D s conntype
typesqldatawarehouse
.

For more information, see Database Browser.

Writing

...

to 
D s conntype
typesqldatawarehouse

You can write back data to SQL DW using to 

D s conntype
typesqldatawarehouse
 using one of the following methods:

Info

NOTE: Writing and publishing to SQL DW

D s conntype
typesqldatawarehouse
is not supported if Azure AD SSO has been enabled.

  • Job results can be written directly to SQL DW as to 
    D s conntype
    typesqldatawarehouse
     as part of the normal job execution. Create a new publishing action to write to SQL DW. See Run Job Pageto 
    D s conntype
    typesqldatawarehouse
    . See Microsoft SQL Data Warehouse Table Settings.
  • As needed, you can publish results to SQL DW for previously executed jobs.

    Info

    NOTE: You cannot re-publish results to SQL DW if the original job published to SQL DW. However, if the dataset was transformed but publication to SQL DW failed, you can publish from the Publishing dialog.

    Info

    NOTE: To publish to SQL DW, the source results must be Parquet format.

    See Publishing Dialogto 

    D s conntype
    typesqldatawarehouse
     for previously executed jobs.

  • For more information on how data is converted

    to SQL DW

    to 

    D s conntype
    typesqldatawarehouse
    , see SQL DW Data Type Conversions.

Data Validation issues:

  • No validation is performed for the connection and any required permissions during job execution. So, you can be permitted to launch your job even if you do not have sufficient connectivity or permissions to access the data. The corresponding publish job fails at runtime. 
  • No data validation is performed during writing and publication to SQL DWto 
    D s conntype
    typesqldatawarehouse
    . Your job fails if the schema for the 
    D s item
    dataset
    dataset
     varies from the target schema.
  • Prior to publication, no validation is performed on whether a target is a table or a view, so the job that was launched fails at runtime.

...