Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0682

D toc

Before you begin installing 

D s product
rtrue
, please review the following checklist to verify that the applicable items are available and ready to deploy with the software.

Info

NOTE: Enablement of specific features or integration with external sources may have additional requirements listed in any referenced content. Please be sure to review the Details sections listed below.

Info

NOTE: If the version is listed as Default, the supported version is the one that is included with the supported distribution.


For more information on general limitations of your product, see Product Limitations.

Hosting Infrastructure

Container deployments

If you are deploying the 

D s node
 to a container, the following versions are supported by the 
D s platform
.

Docker

Supported Versions: 

  • Docker: 17.12 or higher. Docker version must be compatible with the following version(s) of Docker Compose.
  • Docker Compose: 1.24.1

Details: Install for Docker in the Install Guide


Platform Integrations

Cluster Integrations

Cluster types

Info

NOTE: Depending on your version of the following cluster platforms, specific versions of Spark may be required.

On-premises
Cluster typeSupported VersionsNotes, Limitations and Additional Doc
Cloudera
  • CDH 6.3 Recommended
  • CDH 6.2
  • CDH 6.1
  • CDH 5.16 Recommended
Info

NOTE: CDH 6.x requires use of Spark native libraries provided by the cluster. See Configure for Spark in the Configuration Guide.

Details: Supported Deployment Scenarios for Cloudera in the Install Guide

Hortonworks
  • HDP 3.1 Recommended
  • HDP 3.0
  • HDP 2.6
Info

NOTE: HDP 3.x requires use of Spark native libraries provided by the cluster. See Configure for Spark in the Configuration Guide.

Details: Supported Deployment Scenarios for Hortonworks in the Install Guide

AWS
Cluster typeSupported VersionsNotes, Limitations and Additional Doc
EMR
  • EMR 5.8 - 5.27

Details: Configure for EMR in the Configuration Guide

For more information, see AWS Integrations below.

Azure
Cluster typeSupported VersionsNotes, Limitations and Additional Doc
Azure Databricks
  • Azure Databricks 5.3 - 5.5 LTS

Details: Configure for Azure Databricks in the Configuration Guide

HDI
  • HDI 3.6
Details: Supported Deployment Scenarios for Azure in the Install Guide

For more information, see Azure Integrations below.

Cluster hardware

See Sizing Guidelines.

On-Premises integrations

Base storage layer options

The platform must be configured to integrate with a base storage layer. This layer is used for storage of uploads, samples, and job results. See Set Base Storage Layer in the Configuration Guide.

ItemSupported VersionsNotes, Limitations and Additional Doc
HDFSDefaultDetails: Configure for Hadoop in the Configuration Guide
S3n/a

Details: Enable S3 Access in the Configuration Guide

SSO Authentication methods

ItemSupported VersionsNotes, Limitations and Additional Doc
AD-LDAPn/aDetails: Configure SSO for AD-LDAP in the Configuration Guide
SAML2.0Details: Configure SSO for SAML in the Configuration Guide

Hadoop Integrations

ItemSupported VersionsNotes, Limitations and Additional Doc
Hive
  • Hive 1.x
  • Hive 2.x
Additional support requirements vary with the version of Hive. Details: Configure for Hive in the Configuration Guide
KMS

Default

Additional configuration is required depending on your deployed distribution of Hadoop. Details: Configure for KMS in the Configuration Guide

Sentry

Default


Ranger

Default



AWS Integrations

Base storage layer options

Details: Supported Deployment Scenarios for AWS in the Install Guide

ItemSupported VersionsNotes, Limitations and Additional Doc
S3n/a
HDFSDefault

SSO Authentication methods

Details: Configure for AWS in the Configuration Guide

ItemSupported VersionsNotes, Limitations and Additional Doc
AWS Key-Secretn/a
EC2 instance rolesn/a
IAM rolesn/a

Azure Integrations

Base storage layer options

ItemSupported VersionsNotes, Limitations and Additional Doc
ADLSn/aDetails: Enable ADLS Access in the Configuration Guide
WASBSn/aDetails: Enable WASB Access in the Configuration Guide

SSO Authentication methods

ItemSupported VersionsNotes, Limitations and Additional Doc
Azure ADn/aDetails: Configure SSO for Azure AD in the Configuration Guide

D s node

D s node
hardware

ItemMinimumRecommendedNotes, Limitations and Additional Doc
Number of Cores8 cores16 cores
RAM64 GB128 GB
Install disk space4 GB16 GB
Total free disk space

16 GB

/opt - 10 GB

/var - remainder

100 GB

/opt - 10 GB

/var - remainder


Details: System Requirements

D s node
software

ItemSupported VersionsNotes, Limitations and Additional Doc
Operating System

  • CentOS: 6.4 - 6.x, 7.1, 7.2, 7.4 - 7.6
  • RHEL: 6.4 - 6.x, 7.1, 7.2, 7.4 - 7.6
  • Ubuntu: 14.04 (Trusty), 16.04 (Xenial)

Info

NOTE: There are additional requirements for some of these operation system versions. See System Requirements.

JavaJava 1.8
NginX1.12.2
NodeJS10.13.0

Other requirements:

  • Edge node: Platform must be installed on an edge node of the cluster.
  • Root access: Required for installation
  • SSL access: Access to the platform can be limited to SSL only. See Install SSL Certificate in the Install Guide.
  • Internet access: If the
    D s node
    is not connected to the Internet, you must acquire additional software packages for the installation process. See Install Dependencies without Internet Access in the Install Guide.

See System Requirements.

D s item
itemdatabases

The

D s platform
requires multiple databases to store object metadata and job information. Supported databases:

ItemSupported VersionsNotes, Limitations and Additional Doc
PostgreSQL

9.6


MySQL5.7 CommunityDetails: System Requirements

See Install Databases in the Databases Guide.

Desktops

For more information on the following requirements, see Desktop Requirements.

Desktop hardware

ItemSupported VersionsNotes, Limitations and Additional Doc
CPU
  • Intel Pentium 4 or newer
  • AMD Opteron

RAM
  • 8 GB recommended
  • 4 GB minimum

Screen1280 x 720 pixels or better
ConnectionDSL or better

Desktop browsers

Info

NOTE: Stable browser versions released after a given release of

D s product
will NOT be supported for any prior version of
D s product
.  A best effort will be made to support newer versions released during the support lifecycle of the release.

ItemSupported VersionsNotes, Limitations and Additional Doc
Google Chrome
v.76 - v.78,

and any stable version that is released prior to the next release of

D s product
.

Mozilla Firefox

v.68 - v.70,

and any stable version that is released prior to the next release of

D s product
.

D beta

Desktop application

For

D s product
productee
, the
D s deskapp
can be deployed to desktops that do not have a supported browser installed on them, provided that they meet the following requirements:

ItemSupported VersionsNotes, Limitations and Additional Doc
RAM8 GB minimum
Operating SystemWindows 7 (Service Pack 1), 8 or 10 (64-bit only)
PackagesVisual C++ for Visual Studio 2015.Download: https://www.microsoft.com/en-us/download/details.aspx?id=48145

Connectivity

For more information, see Connection Types in the User Guide.