Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0682

D toc


D s platform
can be configured to read and write data from multiple environments at the same time. This page provides information on the supported options.


After you have configured the base storage layer and access and browsing capabilities, you cannot switch them for your

D s item


Base Storage Layer:

The base storage layer defines where job results are written by default.


NOTE: The base storage layer should be enabled and configured during initial installation. After the base storage layer has been configured, it cannot be switched to another environment.


Tip: The

D s platform
can enable connectivity to both S3 and HDFS at the same time. Note that webapp.storageProtocol=s3 should still be specified to write results to S3.

Access and Browse data - S3:

Optionally, you can enable access and the ability to browse your S3 datastore.

JDBC Sources:

Independent of these storage options, you can access database table data through JDBC datastores. See Enable Relational Connections.


Base Storage Layer: HDFS

Access and Browse data - S3: Off


The default configuration, this deployment should be used for most on-premise Hadoop environments. In this case, the

D s platform
only has access to HDFS and Hive as sources on a single Hadoop cluster.

Hybrid Hadoop-based Deployment

Base Storage Layer: HDFS

Access and Browse data - S3: On


This deployment is recommended for the following:

  • On-premises Hadoop clusters that require access to remote S3 data
  • Hadoop clusters hosted in the cloud that require access to remote S3 data and want to continue to use HDFS as an output location

In this scenario, the

D s platform
has access to HDFS and Hive data on the same cluster, as well as access to the remote S3 buckets that have been enabled for the platform.

  • HDFS remains the output location for all job results, profiles, and uploads.

Configuration for Storage Deployments

Base Storage Layer: Set Base Storage Layer

Storage Deployments: