Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r0711

...

Info

NOTE: Depending on your version of the following cluster platforms, specific versions of Spark may be required.

On-premises
Cluster typeSupported VersionsNotes, Limitations and Additional Doc
Cloudera
  • CDH 6.3 Recommended
  • CDH 6.2
  • CDH 6.1
  • CDH 5.16 Recommended
Info

NOTE: CDH 6.x requires use of Spark native libraries provided by the cluster. See Configure for Spark in the Configuration Guide.

Details: Supported Deployment Scenarios for Cloudera in the Install Guide

Hortonworks
  • HDP 3.1 Recommended
  • HDP 3.0
  • HDP 2.6
Info

NOTE: HDP 3.x requires use of Spark native libraries provided by the cluster. See Configure for Spark in the Configuration Guide.

Details: Supported Deployment Scenarios for Hortonworks in the Install Guide

AWS
Cluster typeSupported VersionsNotes, Limitations and Additional Doc
EMR
  • EMR 5.13 - 5.29.0

EMR 5.28.0 is not supported, due to Spark compatibility issues. Please use 5.28.1 or later.

Details: Configure for EMR in the Configuration Guide

For more information, see AWS Integrations below.

Azure
Cluster typeSupported VersionsNotes, Limitations and Additional Doc
Azure Databricks
  • Azure Databricks 6.x
  • Azure Databricks 5.5 LTS

Details: Configure for Azure Databricks in the Configuration Guide

HDI
  • HDI 3.6
Details: Supported Deployment Scenarios for Azure in the Install Guide

For more information, see Azure Integrations below.

...

The platform must be configured to integrate with a base storage layer. This layer is used for storage of uploads, samples, and job results. See Set Base Storage Layer in the Configuration Guide.

ItemSupported VersionsNotes, Limitations and Additional Doc
HDFSDefaultDetails: Configure for Hadoop in the Configuration Guide
S3n/a

Details: Enable S3 Access in the Configuration Guide

SSO Authentication methods

ItemSupported VersionsNotes, Limitations and Additional Doc
AD-LDAPn/aDetails: Configure SSO for AD-LDAP in the Configuration Guide
SAML2.0Details: Configure SSO for SAML in the Configuration Guide

Hadoop Integrations

ItemSupported VersionsNotes, Limitations and Additional Doc
Hive
  • Hive 1.x
  • Hive 2.x
  • Hive 3.x (limited)
Additional support requirements vary with the version of Hive. Details: Configure for Hive in the Configuration Guide
KMS

Default

Additional configuration is required depending on your deployed distribution of Hadoop. Details: Configure for KMS in the Configuration Guide

Sentry

Default


Ranger

Default



AWS Integrations

Base storage layer options

Details: Supported Deployment Scenarios for AWS in the Install Guide

ItemSupported VersionsNotes, Limitations and Additional Doc
S3n/a
HDFSDefault

SSO Authentication methods

Details: Configure for AWS in the Configuration Guide

ItemSupported VersionsNotes, Limitations and Additional Doc
AWS Key-Secretn/a
EC2 instance rolesn/a
IAM rolesn/a

Azure Integrations

Base storage layer options

ItemSupported VersionsNotes, Limitations and Additional Doc
ADLS Gen2n/aDetails: Enable ADLS Gen2 Access in the Configuration Guide
ADLS Gen1n/aDetails: Enable ADLS Gen1 Access in the Configuration Guide
WASBSn/aDetails: Enable WASB Access in the Configuration Guide

SSO Authentication methods

ItemSupported VersionsNotes, Limitations and Additional Doc
Azure ADn/aDetails: Configure SSO for Azure AD in the Configuration Guide

...

D s node

D s node
hardware

ItemMinimumRecommendedNotes, Limitations and Additional Doc
Number of Cores8 cores, x86_6416 cores, x86_64
RAM64 GB128 GB
Install disk space4 GB16 GB
Total free disk space

16 GB

/opt - 10 GB

/var - remainder

100 GB

/opt - 10 GB

/var - remainder


Details: System Requirements

D s node
software

ItemSupported VersionsNotes, Limitations and Additional Doc
Operating System
  • CentOS: 7.1, 7.2, 7.4 - 7.7, 8.1

    Info

    NOTE: MySQL 5.7 Community is not supported on CentOS/RHEL 8.1.

  • RHEL: 7.1, 7.2, 7.4 - 7.7, 8.1
  • Ubuntu: 16.04 (Xenial), 18.04 (Bionic Beaver)
Info

NOTE: There are additional requirements for some of these operation system versions. See System Requirements.

JavaJava 1.8
NginX1.12.2
NodeJS12.16.1

Other requirements:

  • Edge node: Platform must be installed on an edge node of the cluster.
  • Root access: Required for installation
  • SSL access: Access to the platform can be limited to SSL only. See Install SSL Certificate in the Install Guide.
  • Internet access: If the
    D s node
    is not connected to the Internet, you must acquire additional software packages for the installation process. See Install Dependencies without Internet Access in the Install Guide.

...

The

D s platform
requires multiple databases to store object metadata and job information. Supported databases:

ItemSupported VersionsNotes, Limitations and Additional Doc
PostgreSQL

12.3

Info

NOTE: PostgreSQL 12.3 is supported on supported versions of CentOS/RHEL 7 only.

9.6


MySQL

5.7 Community

Info

NOTE: MySQL 5.7 Community is not supported on CentOS/RHEL 8.1.

Details: System Requirements

See Install Databases in the Databases Guide.

...

For more information on the following requirements, see Desktop Requirements.

Desktop hardware

ItemSupported VersionsNotes, Limitations and Additional Doc
CPU
  • Intel Pentium 4 or newer
  • AMD Opteron

RAM
  • 8 GB recommended
  • 4 GB minimum

Screen1280 x 720 pixels or better
ConnectionDSL or better

Desktop browsers

Info

NOTE: Stable browser versions released after a given release of

D s product
will NOT be supported for any prior version of
D s product
.  A best effort will be made to support newer versions released during the support lifecycle of the release.

ItemSupported VersionsNotes, Limitations and Additional Doc
Google Chrome
v.79 - v.81,

and any stable version that is released prior to the next release of

D s product
.

Mozilla Firefox

v.73 - v.75,

and any stable version that is released prior to the next release of

D s product
.

D beta

Desktop application

Warning

The

D s deskapp
is no longer available in the software distribution and has been deprecated. Please switch to a supported browser version.

A Release 6.8 version of the

D s deskapp
can be made available upon request. For more information, please contact
D s support
.

For

D s product
productee
, the
D s deskapp
can be deployed to desktops that do not have a supported browser installed on them, provided that they meet the following requirements:

ItemSupported VersionsNotes, Limitations and Additional Doc
RAM8 GB minimum
Operating SystemWindows 7 (Service Pack 1), 8 or 10 (64-bit only)
PackagesVisual C++ for Visual Studio 2015.Download: https://www.microsoft.com/en-us/download/details.aspx?id=48145

Connectivity

For more information, see Connection Types in the User Guide.