This section contains miscellaneous configuration topics to enable minor features of the platform.
For more information, see Configure Application Limits.
When you publish a CSV file, by default the fields in the file are comma-separated. Optionally, you can configure a different field delimiter.
Tip: During publication, you can specify whether the output file includes double quote marks around each field. For more information, see Run Job Page. |
Steps:
To enable the feature, verify that the following parameter has been set to true
:
"feature.publishDelimiterQuoteOption.enabled": true, |
If you wish to specify the delimiter character using a Unicode value, please set the following to true
:
"feature.publishDelimiterQuoteOption.enableUnicodeCSVDelimiters": true, |
Search for the following parameter:
"webapp.outputCsvDelimiter" |
If you have enabled the Unicode CSV delimiters feature, you can enter Unicode characters in the following format:
\uXXXX |
where XXXX
is the Unicode value for the character.
NOTE: The delimiter value is used as the default value for each job. The default can be overridden on a per-job basis. For more information, see Run Job Page. |
You cannot upload a file with an unsupported or unlisted file extension. As needed, you can add extensions to the list of supported file formats, which enables the file to be uploaded.
.xml
and .XML
must both be listed.NOTE: Even if the file extension is added to this list, the platform may not be able to process the file. For more information, see Supported File Formats. |
NOTE: The |
Steps:
Locate the following parameter and add file extensions. In the example, two new extensions have been added to the list:
"webapp.client.allowedFileExtensions": [".log",".LOG"], |
The supports two methods of applying machine learning to the ranking of transformation suggestions in the
.
NOTE: The older, server-side method has been deprecated and should not be used. Please verify that the following parameters have been set as indicated below. |
Steps:
Set the following parameters to the listed values:
Parameter Name | Description | |
---|---|---|
feature.mlTransformSuggestions.suggestionRankingModel.enabled | Set this value to true . This parameter enables the newer method of ranking transform suggestions based on machine learning. This method is executed in the client and is therefore faster. | |
feature.mlTransformSuggestions.enabled | Set this value to
| |
feature.mlTransformSuggestions.delayThreshold |
|
For non-schematized outputs, enforces casting of all values to the annotated data type of the column by default. For example, if the output value is
-3.4
and the data type for the output column is Integer, the platform enforces Integer type casting and writes a null value instead.
This feature is managed by the following workspace setting: Schematized output
:
Parameter Setting | Description | Implications on Mismatched Values |
---|---|---|
true | (Default) All output values are forced to match the data type of the output columns. | Mismatched values are written as null values. |
false | All output values are written in their output form, regardless of the column's data type.
| Mismatched values are written as-is. |
Steps:
Set the following parameter accordingly:
"Schematized output" : true, |
By default, does not permit the deletion of jobs. As needed, you can enable deletion of jobs.
To enable this feature:
Steps:
Set the following parameter to true
:
"feature.enableJobDeletion" : false, |
When enabled, jobs can be deleted from Flow View page or from the Jobs page through the context menu.
By default, users are permitted to modify the paths for uploading datasets and outputting results. For more information, see Storage Config Page.
If needed, you can disable the ability for individual users to modify these paths.
Steps:
Set the following parameter to false
:
Allow the user to modify their paths |
For more information, see Workspace Settings Page.
By default, supports the application of parameters to imported datasets. Datetime, wildcard, or variable parameters can be used to operationalize execution of jobs on different versions of the same dataset. For example, if a new dataset is dropped in the same folder each day with a different date value in the filename, you can apply parameters to the imported dataset to ensure that all matching datasets in the directory can be executed. For more information, see Overview of Parameterization.
To disable this feature, please complete the following steps.
Steps:
Set the following parameter to false
:
"feature.parameterization.enabled" : false, |
Optionally, you can configure the platform to all users to upload images to be used as their profile pictures. See User Profile Page.
Steps:
Set the following parameter to true
:
"webapp.enableProfilePicture" : true, |
The flow sharing feature enables a user to share a flow with another user for collaboration. In this manner, a single source of truth can be retained as multiple users collaborate to transform the flow's datasets into the desired output.
Flow sharing is enabled by default. For more information on disabling, see Workspace Settings Page.
By default, the can display lineage highlighting of applicable steps for a selected column. When enabled, you can select Show related steps from a column's drop-down menu, and all recipe steps that reference the column are highlighted in the recipe panel.
Please complete the following steps to disable this feature.
Steps:
Set the webapp.enableColumnLineageScriptHighlighting
configuration flag to false
:
"webapp.enableColumnLineageScriptHighlighting": false, |
By default, the publishes job results with the access controls inherited from the default output home directory of the user who ran the job.
In some cases, these access controls may not work. For example, if the user is working on a project with multiple users, those other users may not be able to access the job results even though the output is written into a directory on the backend datastore to which they all have access.
As needed, you can change the to write job results into the target directory inheriting user access controls from the parent of the target directory. In this manner, permissions are inherited based on the target location.
NOTE: When this feature is enabled, job results are always written with permissions inherited from the parent directory of the target location. |
Steps:
Locate the following parameter and set it to true
:
"feature.copyFileToDestination": true, |
By default, you can create custom data types in . When a column is set to a custom data type, the values in the column are validated against the type specification.
After a custom data type has been created, it cannot be removed from the platform. |
For more information on creating custom data types using regular expression, see Create Custom Data Types Using RegEx.
NOTE: Use of regular expressions is considered a developer-level skill. |
To disable this feature, please complete the following steps.
Steps:
Set the following to false
:
"feature.enableCustomTypes": false, |