The  supports a single global file encoding type, which is set to UTF-8 by default. This file encoding type applies to all text files for the following operations:

NOTE: This setting applies only to text files. Binary types, such as Avro, are not affected by the global file encoding type.

NOTE: If you change this setting, datasets that were imported under the former encoding type are no longer valid. Instructions are provided below for updating them.

Supported File Encoding Types

For more information, see Supported File Encoding Types.

Configure Global File Encoding Type

  1. Set the following parameter to the appropriate file encoding type:

    "inputFileEncoding": "UTF-8",
  2. Save your changes and restart the platform.

NOTE: After you change the global encoding type, datasets that were imported under the old encoding type must be reloaded to the platform. For more information, see Update Sources.

Validate

After you have changed the global file encoding type, restart services. See Start and Stop the Platform.

You should try to create a dataset from source data of the selected encoding type.

Update Sources

After you have changed the global encoding type, datasets that were imported under the former encoding type are no longer valid. 

Steps:

  1. For each dataset imported under the old encoding type, upload a new version.
  2. For each recipe that used the old version of the imported dataset:
    1. Edit the recipe in the Transformer Page. 
    2. Swap the source from the old version to the new one. For more information, see Flow View Page.
  3. Repeat for each imported and recipe combination.