This section covers data type conversions between the and the Parquet file format.

NOTE: The listed in this page reflect the raw data type of the converted column. Depending on the contents of the column, the Transformer Page may re-infer a different data type, when a dataset using this type of source is loaded.

Import

NOTE:  does not support ingest of Parquet files with nested values, which can occur for Map or Object data types.

Parquet

Data Type

Notes
STRINGString 
INTInteger 
DECIMALDecimal 
DATEDatetime 
TIMEDatetime 
TIMESTAMPDatetime 
LISTArray 
MAPObject  

Limitations on import:

The Parquet data format supports the use of row groups for organizing chunks of data. This row grouping is helpful for processing across distributed systems. 

 places limitations on the volume of data that can be displayed in the browser. By default, these limits are set to 10 MB. 

If Parquet row groups are greater than 10 MB:

Other product functions work as expected with Parquet format.

Export

On export,  are exported to their corresponding Parquet types, with the following specific mappings: 

Parquet Data TypeNotes
BooleanBOOLEAN
IntegerINT64 
DecimalDOUBLE 
StringBYTE_ARRAY (STRING) 

The fallback data type on export is BYTE_ARRAY (STRING).