As needed, you can modify via parameter the logging levels for the following services of the platform. These settings should only be modified when you are debugging an issue. After the issue is resolved, you should set the logging level back to its original value.
For more information on these services, see System Services and Logs.
For more information on logging levels, see https://logging.apache.org/log4j/log4j-2.12.0/manual/customloglevels.html.
can access the logs through the
. See Admin Settings Page.
You can use the following settings to mask personal information in the log files. However, doing so may complicate debugging:
Logging Parameter and Default Value | Description | |
---|---|---|
| Set this value to | |
| When PII masking is enabled, this value can be used as a randomizing element for generating hashed values. |
By default, log files for each service is automatically rotated for you.
As needed, you can enable and configure log rotation for additional log files. When enabled, the following logs are subject to rotation:
proxy_access.log
proxy_error.log
batch-job-runner.access.log
data-service.access.log
ml_service.access.log
scheduling-service.access.log
time-based-trigger-service.access.log
NOTE: By default, these log files are rotated by supervisord. For more information on default values for stdout_logfile_maxbytes, stdout_logfile_backups, and stderr equivalents, see http://supervisord.org/configuration.html#program-x-section-values. |
The following logs are excluded from log rotation and are not rotated when this feature is enabled:
To enable log rotation, you must copy the default configuration file into the appropriate working directory and set permissions:
sudo cp /opt/trifacta/conf/trifacta-logrotate.conf /etc/logrotate.d/trifacta-logrotate.conf sudo chown root: /etc/logrotate.d/trifacta-logrotate.conf sudo chmod 644 /etc/logrotate.d/trifacta-logrotate.conf |
You can test the configuration using the following command:
sudo logrotate --debug /etc/logrotate.d/trifacta-logrotate.conf |
The following are the access logs:
Service | Applicable Log Files | Parameter |
---|---|---|
VFS Service | vfs-service.access.log | vfs-service.loggerOptions.format |
WebApp | webapp.access.log | webapp.loggerOptions.format |
You can configure the fields and format of them.
NOTE: This field configuration applies only the log files listed above. |
For each of the above parameters, you can use a token-based method of configuring the fields to include in a log entry in the listed logs. You can modify:
Entries are specified as space-delimited fields, each of which begins with a colon:
:field1 :field2 :field3 |
For more information on these log format options, see https://github.com/expressjs/morgan#tokens.
Application log files have the following structure:
2019-07-09T11:46:54.667Z - warn: [EXPRESS] received extra query parameter(s): limit,offset,numPageLinks [url=/v4/connections/2/query] [method=GET] [sid=9133aa58-9cbb-4e0a-bfb4-2c59c44ffe2d] [rid=e4319f35-82a1-4c0b-a2d3-2e75d224178a] |
The following fields are included in these files:
NOTE: These fields cannot be modified. |
Field | Description | |
---|---|---|
Timestamp | Date and timestamp in UTC when the event occurred
| |
Level | Logging level for the event: info , warning , error , or debug | |
url | URL of the endpoint that triggered the event. Hostname and port number are not included.
| |
method | HTTP method that was used to invoke the request | |
sid | The user's session identifier. Session Ids may be preserved across logins, but they may occasionally change if the user logs out of the application. | |
rid | The request identifier is unique to the specific request and is preserved across all services. |
By default, logs are in flat text format and exported as .log
files.
Some loggers enable output in JSON format.
NOTE: JSON format is not recommended for |
true
:"*.loggerOptions.json": true, |
Save your changes and restart the platform or service.
More examples are listed below.
The WebApp manages loading of data from the supported connections into the front-end web interface.
NOTE: After changing these settings, the platform must be restarted. |
Logging Parameter and Default Value | Description | |
---|---|---|
| When set to true , log messages are silent. | |
| Supported levels (in decreasing order of verbosity): silly , debug , info , warning , error | |
| If set to true , logging output is in JSON format. | |
| String containing list of fields to include in each log message. |
The Batch Job Runner service manages the execution of batch jobs on the backend running environment.
Logging Parameter and Default Value | Description | |
---|---|---|
| Supported levels (in decreasing order of verbosity): |
Loads data from the various filesystems supported by the platform, both in the front-end user interface and in batch mode when the running environment or
web client is enabled. Both are enabled by default. For more information, see Configure VFS Service.
Logging Parameter and Default Value | Description | |
---|---|---|
| When set to true , log messages are silent. | |
| Supported levels (in decreasing order of verbosity): silly , debug , info , warning , error | |
| If set to true , logging output is in JSON format. | |
| String containing list of fields to include in each log message. |
Service prepares queries against JDBC interfaces, using internal REST API calls.
Logging Parameter and Default Value | Description | |
---|---|---|
| Supported levels (in decreasing order of verbosity): trace , debug , info , warn , error , fatal , off |
For the ML service, logging level can be modified at the command line:
python cyclone.py —log-level <level> |
Supported levels: DEBUG
, INFO,
, WARNING
ERROR
, CRITICAL
Logging Parameter and Default Value | Description | |
---|---|---|
| If set to true , logging output is in JSON format. |
The JSData logging options do not apply to a specific service. Instead, they are used by various services to log activities related to and its interactions with various connections and running environments.
Logging Parameter and Default Value | Description | |
---|---|---|
| When set to true , log messages are silent. | |
| Supported levels (in decreasing order of verbosity): silly , debug , info , error , warn | |
| If set to true , logging output is in JSON format. |