Skip to main content

SFTP Connections

You can create connections to SFTP servers to upload your datasets to the Cloud Portal.

Linux- and Windows-based SFTP servers are supported.

Jobs can be executed from SFTP sources on the following running environments:

  • Trifacta Photon

  • Spark on EMR

Read: Supported

Write: Not supported

Limitations

  • Read-only connection

  • Files and folders whose names begin with underscore (_) aren't visible.

  • Ingest of over 500 files through SFTP at one time isn't supported.

  • For private SFTP servers, you can't run jobs on Spark. These jobs must be run using Trifacta Photon.

  • Through SFTP connections, you can't run jobs on Avro or Parquet files or on parameterized datasets, from file types that require conversion such as JSON, PDF, and Excel.

    • Single Excel files can be imported through SFTP.

Prerequisites

  • Acquire user credentials to access the SFTP server. You can use username/password credentials or SSH keys. See below.

  • Verify that the credentials can access the proper locations on the server where your data is stored. Initial directory of the user account must be accessible.

SSH Keys

If preferred, you can use SSH keys to for authentication to the SFTP server.

Note

SSH keys must be private RSA keys. If you have OpenSSH keys, you can use the ssh-keygen utility to convert them to private RSA keys.

Allowlist SFTP server

If your SFTP server is private, you must add the Trifacta's IP to the allowlist of IPs that are permitted to communicate with the server. For more information, please see the documentation that is provided with your software distribution.

Create Connection

Create through application

You can create a SFTP connection through the Cloud Portal.

Steps:

  1. In the left nav bar, select the Connections icon. See Connections Page.

  2. In the Connections page, click Create Connection. See Create Connection Window.

  3. In the Create Connection window, click the SFTP connection card.

  4. Specify the properties for your SFTP server.

    Property

    Description

    Host

    The hostname of the FTP server to which you are connecting. Do not include any protocol identifier (sftp://).

    Port

    The port number to use to connect to the server. Default port number is 22.

    Credential Type

    Select one of the following:

    basic - authenticate via username and password

    SSH Key - authenticate via username and SSH key

    User Name

    The username to use to connect.

    Password

    (Basic credential type) The password associated with the username.

    SSH Key

    (SSH Key credential type) The SSH key that applies to the username.

    Test Connection

    Click this button to test the connection that you have specified.

    Default Directory

    Absolute path on the SFTP server where users of the connection can begin browsing.

    Block Size (Bytes)

    Fetch size in bytes for each read from the SFTP server.

    Note

    Raising this value may increase speed of read operations. However, if it is raised too high, resources can become overwhelmed, and the read can fail.

    Connection Name

    The name of the connection as you want it to appear in the application.

    Description

    This description is displayed in the application.

    For more information, see Create Connection Window.

  5. Click Save.

Create through APIs

  • Type: jdbc

  • Vendor: sftp

For more information, see : API Reference docs

In this section: