D toc |
---|
This section describes how to integrate the
D s platform | ||
---|---|---|
|
- Snowflake provides a cloud-database datawarehouse designed for big data processing and analytics. For more information, see https://www.snowflake.com.
For more information on supported versions, see Connection Types.
Limitations
Info | |
---|---|
NOTE: This integration is supported only for deployments of
|
- SSO connections are not supported.
Pre-requisites
- If you do not provide a stage database, then the
must create one for you in the default database. In this default database, you must include a schema namedD s platform PUBLIC
. For more information, please see the Snowflake documentation.
Enable
When relational connections are enabled, this connection type is automatically available. For more information, see Enable Relational Connections.
Configure
To create a Snowflake connection, you must enable the following feature. The job manifest feature enables the creation of a manifest file to track the set of temporary files written to S3 before publication to Snowflake.
Info |
---|
NOTE: This feature must be enabled when the base storage layer is set to S3. Please verify the following. |
Steps:
D s config Locate the following parameter and set it to
true
:Code Block "feature.enableJobOutputManifest": true,
- Save your changes and restart the platform.
Create Stage
In Snowflake terminology, a stage is a database object that points to an external location on S3. It must be an external stage containing access credentials.
If a stage is used, it is typically the default bucket used on S3 for storage.
Info NOTE: For read-only connections to Snowflake, you must specify a Database for Stage. The connecting user must have write access to this database.
Tip Tip: You can specify a separate database to use for your stage.
If a stage is not specified, a temporary stage is created using the current user's AWS credentials.
Info NOTE: Without a defined stage, you must have write permissions to the database from which you import. This database is used to create the temporary stage.
For more information on stages, see https://docs.snowflake.net/manuals/sql-reference/sql/create-stage.html.
In the
D s platform |
---|
Create Snowflake Connection
For more information, see Create Snowflake Connections.
Testing
Steps:
After you create your connection, load a small dataset based on a table in the connected Snowflake database.
Info NOTE: For Snowflake connections, you must have write access to the database from which you are importing.
See Import Data Page.- Perform a few simple transformations to the data. Run the job. See Transformer Page.
- Verify the results.
For more information, see Verify Operations.