System Services and Logs

...

Filename Description

jobgroups/

Directory of logs for transformation jobs by Id.

Tip

Tip: If you are troubleshooting a failed job, please acquire the job logs from the Job Details page when you contact

D s support

. See Job Details Page.

jobs/ Directory of logs for other kinds of jobs, such as sampling or ingest, by Id.

nginx/ Temporary storage for nginx server. No log files are stored here.

artifact-storage-service.access.log Access logs for the artifact storage service.

artifact-storage-service.log Application logs for the artifact storage service.

batch-job-runner.access.log

Access logs for the batch job runner service.

Batch job runner service manages transformation jobs and scheduling. More information is below.

batch-job-runner.job-status.log Status information on batch job runner jobs.

batch-job-runner.log Application logs for the batch job runner service.

configuration-service.access.log

Access logs for the configuration service service.

Configuration service is used for managing configuration that can be changed at runtime for different workspaces and users. Some of these settings are available through the

D s webapp

. See Workspace Admin Page.

configuration-service.log Application logs for the configuration service.

conversion-service.access.log

Access logs for the conversion service.

Conversion service is used for converting from various inputs formats and to various output formats.

conversion-service.log Application logs for the conversion service.

data-service.access.log

Access logs for the data service.

Data service is used for interacting with relational sources. More information is below.

data-service.log Application logs for the data service.

job-metadata-service.access.log Access logs for the job metadata service.

job-metadata-service.log Application logs for the job metadata service.

migration-report.log Log containing information about specific migrations between releases.

migration.log Application logs for database migrations performed for the webapp service.

ml_service.access.log

Access logs for the ml (machine-learning) service.

Machine learning service is used for predictive interaction, suggestion ranking, pattern profiling, pattern suggestions, and collecting user action logs. More information is below.

ml_service.log Application logs for the ml (machine-learning) service.

nginx_service.log Application logs for the nginx service. More information is below.

protobuf-events.log Client events around column values, user selections, and script editing.

proxy_access.log Access logs for the nginx server.

proxy_error.log Error logs for the nginx server.

scheduling-service.access.log

Access logs for the scheduling service.

Scheduling service is used for scheduling jobs at a specific time. More information is below.

scheduling-service.log Application logs for the scheduling service.

secure-token-service.access.log}}

Access logs for the secure token service.

secure-token-service is used for securely storing tokens for some external services, such as Azure AD and Databricks.

secure-token-service.log

Application logs for the secure token service

spark-job-service.log

Application logs for the Spark job service.

Spark job service is used for interfacing with cluster-based Spark service to plan and execute Spark jobs. More information is below.

supervisord.log

Logs for the supervisord system service.

supervisord manages the starting, stopping, and restarting of services for the

D s platform

. More information is below.

time-based-trigger-service.access.log

Access logs for the time-based trigger service.

Time-based trigger service is used for managing the triggers for scheduled jobs. More information is below.

time-based-trigger-service.log Application logs for the time-based trigger service.

vfs-service.access.log

Access logs for the VFS service.

VFS service is used for managing loading of files from various supported datastores. More information is below.

vfs-service.log Application logs for the VFS service.

webapp.access.log

Access logs for the Webapp service.

Webapp service serves the

D s webapp

to users. More information is below.

webapp.log Application logs for the Webapp service. More information is below.

webapp.sql-error.log Error log for SQL issued from the Webapp service.

webworker.log

Event logs for webworkers running in browser clients.

Webworkers run in the background of a browser client's

D s item

item	session

and are used for predictive interaction, suggestion ranking, pattern profiling, pattern suggestions, and collecting user action logs.

...

Description: Service enables the execution of Java-based user-defined functions within a transform recipe.

Log File Can Help With

java-udf-service.log

Status of the service

Tip
Tip: You can pass through messages on errors through Logger to this log, which can assist in diagnosing issues. See Java UDFs.

Machine Learning Service

Description: ML service provides machine learning capabilities for the platform.

Log File	Can Help With
`ml_service.log`	This log is likely to contain information that is only useful if the ML service has crashed.

Nginx Service

Description: Ngnix is a proxy server embedded in the platform that serves the web application and other resources.

Log File	Can Help With
`nginx_service.log`	This log may be useful in identifying any warnings that occurred when the nginx services starts. The `nginx` server may contain log information for the server that provides HTTP access.

Proxy

Description: The proxy (nginx) service manages requests from the user interface to the other components of the platform.

Log File Can Help With

proxy_access.log

Shows any requests made through the nginx to the port used by the
D s platform
.

proxy_error.log

Contains any errors thrown by the nginx service when a request is made to the port used by the
D s platform
.

Scheduling Service

Description: Handles all metadata related to scheduling

Log File	Can Help With
`scheduling-service.log`	Schedule-related issues
`scheduling-service.access.log`	Gives information about accessed routes

Spark Job Service

Description: Service that manages jobs processed on Spark.

Log File	Can Help With
`spark-job-service.log`	Status of the service Debugging issues with Spark jobs See Configure for Spark.

Supervisord Server

Description: Process that starts, stops, and restarts services in the platform.

Log File	Can Help With
`supervisord.log`	Status information from platform services

Time-Based Trigger Service

Description: Handles all metadata related to the trigger service

Log File	Can Help With
`time-based-trigger-service.log`	schedules not triggering correctly
`time-based-trigger-service.access.log`	Gives information about accessed routes

VFS Service

Description: Loads data from the various filesystems supported by the platform, both in the front-end user interface and in batch mode when the

D s photon

running environment is enabled. For more information, see Running Environment Options.

Log File	Can Help With
`vfs-service.log`	Client connection issues Status issues with backend components Information on batch jobs that cannot be started

Webapp Service

Description: Loads data from the various filesystems supported by the platform in the front-end user interface.

Log File	Can Help With
`webapp.log`	Client connection issues Status issues with backend components Information on batch jobs that cannot be started

Additional logs

Job logs

The following sources of information may provide information related to job status and performance:

job log
spark log
cdf script
yarn application logs
(if log aggregation is enabled)
platform configuration file
(
D s triconf
path full
)
batch job runner log
spark service log
hadoop conf directory
(if accessible)

Page tree

Versions Compared

Old Version 3

New Version Current

Key