Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

...

The

D s webapp
rtrue
can connect to a high-performance environment embedded in the
d-s-item
item
node
for  for execution of jobs against small- to medium-sized datasets, called the
d-s-
server
photon
 running environment
.

  • The The  
    d-s-serverphoton
     can  running environment can be selected in the Run Jobs page.

By default, the 

d-s-serverphoton
 running environment is enabled for new installations.

...

  • Faster execution times for transform and profiling jobs
  • Better consistency with typecasting done in Hadoop Spark jobs

This section provides information on how to enable and configure the

d-s-serverphoton
 running environment.

Info

NOTE: Some configuration is shared with the Photon in-browser engine

D s photon
clienttrue
client. For more information, see Configure Photon Client.


Known Limitations

Info

NOTE: For profiles executed in the

d-s-serverphoton
running environment, percentages for valid, missing, or mismatched column values may not add up to 100% due to rounding. See Overview of Visual Profiling.

Disable 
d-s-

...

photon
 Running Environment

The 

d-s-serverphoton
 is  running environment is enabled by default. Please complete the following configuration to disable the running environment.

...

  1. D s config
  2. To disable the 

    d-s-serverphoton
     running environment, apply the following configuration settings:

    Code Block
    "webapp.runInTrifactaServer": false,
    "feature.enableSamplingScanOptions": false,
    "feature.enableFirstRowsSample": false, 
  3. Do not change the following, which applies to the Photon web-client

    D s photon
    clienttrue
     client:

    Code Block
    "photon.enabled": true,
  4. Save your changes and restart the platform.

Example Configuration

The following configuration includes the default values.

...

Some of these values apply to the Photon web client

D s photon
clienttrue
 client. For more information, see Configure Photon Client.

ParameterDescription
cacheEnabledDebugging setting. Leave the default value.
numThreads

Maximum number of threads permitted to the Photon

D s photon
process. For recommended values, see Configure Photon Client.

enabled

This value should be set to true. For more information, see Configure Photon Client.

distroPath

Please verify that this property is set to the following value, which works for all operating system distributions:

Code Block
  "distroPath": "/photon/dist/centos6/photon",
traceExecutionDebugging setting. Leave the default value.
websocket.hostInternal parameter. Do not modify.
websocket.portInternal parameter. Do not modify.
mode

Set this value is wasm. For more information, see Configure Desktops.

Modify Limits

Runtime job timeout

By default, the 

D s platform
 imposes no limit on execution of a 
d-s-serverphoton
 job. As needed, you can enable and configure a limit.

...

  1. D s config

    Code Block
    "photon.timeoutEnabled": false,
    "photon.timeoutMinutes": 180,
    SettingDescription
    timeoutEnabledSet to false to disable job limiting. Set to true to enable the timeout specified below.
    timeoutMinutes

    Defines the number of minutes that a

    d-s-serverphoton
    job is permitted to run. Default value is 180 (three hours).

  2. Save your changes and restart the platform.

...

Job logs can be downloaded from the Job page. See Jobs Page.

d-s-

...

photon

...

 running environment memory timeout

To prevent crashes, the 

d-s-serverphoton
 imposes  running environment imposes a memory consumption limit for each job. If this memory timeout is exceeded, the job is automatically killed. As needed, you can disable this memory protection (not recommended) or change the memory threshold when jobs are killed.

...

  1. D s config

  2. Locate the following settings:

    Code Block
    "photon.memoryMonitorEnabled": false,
    "photon.memoryPercentageThresold": 60,
    SettingDescription
    memoryMonitorEnabledSet to false to disable memory monitoring. Set to true to enable the threshold specified below.
    memoryPercentageThreshold

    Defines the percentage of total available system memory that a

    d-s-serverphoton
    job process is permitted to consume. Default value is 60 (60%).

    Tip

    Tip: This threshold applies to individual jobs. If this threshold value is over 50%, it is possible for two concurrent

    d-s-serverphoton
    jobs to use more than the available memory, crash the server, and force a restart. You may wish to start by setting threshold values at a lower level.

  3. Save your changes and restart the platform.

...

Job logs can be downloaded from the Job page. See Jobs Page.

Batch FileSystem Access Timeout Settings

The default timeout settings for reading and writing of data from the client browser through Photon to through 

D s photon
 running environment to the 
d-s-servernode
 should work in most cases.

...

  1. D s config
  2. Locate the photon.extraCliArgs node.

  3. Add the following values to the extraCliArgs entry:

    Code Block
    "photon.extraCliArgs" : "-batch_vfs_read_timeout <300> -batch_vfs_write_timeout <300>"
    ArgumentDescription
    -batch_vfs_read_timeout

    Timeout limit in seconds of read operations from the datastore. Default value is 300 seconds (5 minutes).

    Tip

    Tip: Raising the value to 3600 seconds should be fine in most environments. Avoid setting this value above 7200 seconds (2 hours).

    -batch_vfs_write_timeout

    Timeout limit in seconds of write operations to the datastore. Default value is 300 seconds (5 minutes).

    Info

    NOTE: Do not modify unless specifically instructed by

    D s support
    .

  4. To reduce timeouts, raise the above settings.
  5. Save your changes and restart the platform.

Configure VFS Service

The

d-s-serverphoton
 interacts  running environment interacts with backend datastores through the VFS service.

...


For more information, see Configure VFS Service.

Use 
d-s-

...

photon
 Running Environment

When executing a job, select the Run on 

D s server
 option from the drop-down in the Run Job dialogthe Photon option.

Info

NOTE: Before you test, please be sure to complete all steps of Required Platform Configuration.