Configure for High Availability

Contents:

Expand all Collapse all

Our documentation site is moving!

For up-to-date documentation of release 8.7 of Self Managed Designer Cloud, please visit us at https://help.alteryx.com/SMC/r87/.

Contents:

This section contains additional configuration tasks for setting up high availability on the Alteryx® node.

The Designer Cloud Powered by Trifacta platform can also integrate with a highly available Hadoop cluster. For more information, see Enable Integration with Cluster High Availability.

Limitations

For more information on deploying high availability, see Install for High Availability.

Log File Setup

Since the platform nodes are mounting to the same directory, a configuration flag forces log files to be put into a subdirectory containing the hostname of the node from which the file was generated. The resulting directory structure:

/opt/trifacta/
   --- conf
   --- bin
   --- logs/
       --- node1/webapp.log
       --- node1/data-service.log
       --- node1/spark-job-service.log
       --- node1/...
       --- node2/webapp.log
       --- node2/data-service.log
       --- node2/spark-job-service.log
       --- node2/...
       ---- ......

Job logs are already prefixed with the job-group id, so there is no need to prefix them with the hostname.
The Download Logs should work independent of the node serving the endpoint, so there is no hostname prefixing required for the job logs.

To enable separation of log files into separate directories by node, please complete the following:

Steps:

Login as admin to the server (main) node in your HA environment.
To apply this configuration change, login as an administrator to the Alteryx node. Then, edit trifacta-conf.json. For more information, see Platform Configuration Methods.

Set the following parameter to true:

"feature.highAvailability.separateLogFilesByName": false,

Save the file.
Before restarting the server, please complete any remaining configuration changes.

Disable restart from the Admin Settings Page

The preferred method of restarting the platform is to use the Restart button in the Admin Settings page. When a Alteryx node is part of a high availability environment, the node cannot be restarted through the Admin Settings page, which only restarts the node. Other nodes in the environment are not restarted, and any configuration changes are not propagated.

NOTE: In an HA environment, each node must be started, stopped, and restarted through the command line. See Start and Stop the Platform.

To disable use of the Restart button in the Admin Settings page, please complete the following steps.

Steps:

Login as admin to the server (main) node in your HA environment.
To apply this configuration change, login as an administrator to the Alteryx node. Then, edit trifacta-conf.json. For more information, see Platform Configuration Methods.

Set the following parameter to false:

"feature.highAvailability.enableRestartFromAdminUI": true,

Save the file.
Before restarting the server, please complete any remaining configuration changes.

NFS Mount Setup

Each node shares the following resources:

Alteryx databases
Directories shared via NFS mount:
```
/opt/trifacta/logs
/opt/trifacta/conf
```

Example NFS Mount

The following sections contain an example configuration for the NFS mount to be shared across all nodes.

NOTE: There are many ways to configure an NFS mount. The following example applies to CentOS 7. Please modify these commands for different operating systems and versions as needed.

Server-side Configuration

NFS service

Use the following command to install the required NFS packages on the server:

yum install nfs-utils nfs-utils-lib

Enable and start NFS services:

systemctl enable rpcbind
systemctl enable nfs-server
systemctl enable nfs-lock
systemctl enable nfs-idmap

systemctl start rpcbind
systemctl start nfs-server
systemctl start nfs-lock
systemctl start nfs-idmap

Shared directories

The following directories shared across all nodes:

/opt/trifacta/logs
/opt/trifacta/conf

To export the shared directories from the server, edit the following file:
```
/etc/exports
```

Append the following lines to the end of the file:

/opt/trifacta/conf 192.168.1.102(rw,sync,no_root_squash,no_all_squash)
/opt/trifacta/logs 192.168.1.102(rw,sync,no_root_squash,no_all_squash)

Property	Description
/opt/trifacta/conf /opt/trifacta/logs	Directories to share across NFS
`192.168.1.102`	IP address for client. NOTE: Insert the above commands once for each client IP address.
`rw`	Read/write permissions on the shared folder.
`sync`	Enables synchronizing shared directories.
`no_root_squash`	Enables root privilege.
`no_all_squash`	Enables user's authority.

Restart the nfs-server service:

systemctl restart nfs-server

Client-side Configuration

On each client node, install NFS packages:

yum install nfs-utils nfs-utils-lib

Enable and start the NFS services:

systemctl enable rpcbind
systemctl enable nfs-server
systemctl enable nfs-lock
systemctl enable nfs-idmap

systemctl start rpcbind
systemctl start nfs-server
systemctl start nfs-lock
systemctl start nfs-idmap

Mount NFS Shared Directories on Client

Mount the shared directories from the server to the client as shown below:

mount -t nfs 192.168.1.101:/opt/trifacta/logs /opt/trifacta/logs
mount -t nfs 192.168.1.101:/opt/trifacta/conf /opt/trifacta/conf

NOTE: Repeat the above on each client node.

If you encounter a Connection timed out error, please see the Troubleshooting section below.

Verifying NFS Shared Directories On Clients

Verify that the shared directories from the server are mounted:

mount | grep nfs

Example output:

rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
192.168.1.101:/opt/trifacta/logs on /opt/trifacta/logs type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.0.9.167,local_lock=none,addr=10.0.6.77)

Auto-mount NFS Shared Directories

The above manual steps must be repeated whenever the platform is rebooted.

To automatically mount the shared directories on reboot:

Edit the following file:
```
/etc/fstab
```

Add the following lines:

192.168.1.101:/opt/trifacta/logs /opt/trifacta/logs nfs rw,sync,hard,intr 0 0
192.168.1.101:/opt/trifacta/conf /opt/trifacta/conf nfs rw,sync,hard,intr 0 0

Reboot the client system and check the whether the NFS shared directories were mounted or not:
```
mount | grep nfs
```

Integrate with HAProxy Load Balancer

After you have installed the software on each node in your high availability environment and configured your NFS mount, please complete the following instructions to integrate each node with a load balancer.

NOTE: HAProxy is the recommended load balancer. These instructions apply to HAProxy. If you are using a different load balancer, please consult the documentation that came with your product.

Configuration example:

#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local1 notice

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    option                  log-health-checks
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           1s
    maxconn                 3000

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend  main *:8088
#    acl url_static       path_beg       -i /static /images /javascript /stylesheets
#    acl url_static       path_end       -i .jpg .gif .png .css .js
     mode http
#    use_backend static          if url_static
    default_backend             app

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
#backend static
#    balance     roundrobin
#    server      static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
    balance     roundrobin
    option httpchk GET /v2/ping
    server app1 192.168.1.101:3005 check
    server app2 192.168.1.102:3005 check

Key Settings:

Setting	Description
`balance`	Set this value to the method of load balancing. Round robin is recommended.
`option httpchk`	Please set the value to the following: GET /v2/ping
`server app1`	Set this value to the IP address of the server node.
`server app2` `server app3`	Please insert separate entries for the IP addresses of each client node.

Platform Start

To restart all nodes of the platform, please complete the following steps:

NOTE: When the platform is restarted in a high availability environment, all scheduled jobs that are in-progress or have been triggered during downtime may be assigned to a single node. To distribute across multiple nodes, restart all services except for the time-based service on each node. After all other services have restarted, start the time-based trigger service on each node.

Apply any configuration changes to trifacta-conf.json on the server. Since the configuration directory is now shared across all nodes, these changes are automatically applied to all client nodes.
Verify that all nodes are pointing to the same instances of the Alteryx databases.
Start the master node.

NOTE: Nodes in an HA environment must be restarted at the command line. See Start and Stop the Platform.
Wait for it to complete restart.
Start each client node one at a time.

NOTE: Do not start multiple nodes at the same time, which can cause migration errors.
When all nodes are restarted, you may continue with any necessary configuration steps through the (main) server node.

Additional Configuration

Enable Redis Sentinel

You can configure the Alteryx node to connect to a high availability Redis deployment using Redis Sentinel. For more information, see Configure for Redis.

Troubleshooting

When mounting the shared directories on the client, a Connection timeout error occurs

If you encounter the error:

mount.nfs: Connection timed out

This error may be due to the firewall blocking the NFS server.

Solution:

To access NFS shared directories from remote clients, the following ports in the NFS server iptables / firewall must be allowed:

rpcinfo -p
Sample output:

    program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  60985  status
    100024    1   tcp  54302  status
    100005    1   udp  20048  mountd
    100005    1   tcp  20048  mountd
    100005    2   udp  20048  mountd
    100005    2   tcp  20048  mountd
    100005    3   udp  20048  mountd
    100005    3   tcp  20048  mountd
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100227    3   tcp   2049  nfs_acl
    100003    3   udp   2049  nfs
    100003    4   udp   2049  nfs
    100227    3   udp   2049  nfs_acl
    100021    1   udp  46666  nlockmgr
    100021    3   udp  46666  nlockmgr
    100021    4   udp  46666  nlockmgr
    100021    1   tcp  42955  nlockmgr
    100021    3   tcp  42955  nlockmgr
    100021    4   tcp  42955  nlockmgr
    100011    1   udp    875  rquotad
    100011    2   udp    875  rquotad
    100011    1   tcp    875  rquotad
    100011    2   tcp    875  rquotad

Run the following commands on the NFS server:

firewall-cmd --permanent --add-port=111/tcp
firewall-cmd --permanent --add-port=54302/tcp
firewall-cmd --permanent --add-port=20048/tcp
firewall-cmd --permanent --add-port=2049/tcp
firewall-cmd --permanent --add-port=46666/tcp
firewall-cmd --permanent --add-port=42955/tcp
firewall-cmd --permanent --add-port=875/tcp

To apply the change, restart the firewall service:
```
firewall-cmd --reload
```

Mount the shared directories on each client with the following commands:

mount -t nfs 192.168.1.101:/opt/trifacta/logs /opt/trifacta/logs
mount -t nfs 192.168.1.101:/opt/trifacta/conf /opt/trifacta/conf

The NFS shared directories should mount without error.

Send Feedback

Page tree