D toc |
---|
Excerpt | ||||
---|---|---|---|---|
This section contains information on the fie formats and compression schemes that are supported for input to and output of
|
Info |
---|
NOTE: To work with formats that are proprietary to a desktop application, such as Microsoft Excel, you do not need the supporting application installed on your desktop. |
...
General:
Code Block "/"
Seb browser:
Code Block "\"
Excel filenames:
Code Block "#","{","}"
Spark-based running environment:
Code Block "{", "*", "\"
Native Input File Formats
...
- CSV
JSON v1, including nested
Info NOTE: JSON files can be read natively but often require additional work to properly structure into tabular format. Depending on how the
is configured (v1 or v2), JSON files may require conversion before they are available for use in the application. See "Converted file formats" below.D s webapp Info NOTE:
requires that JSON files be submitted with one valid JSON object per line. Consistently malformed JSON objects or objects that overlap linebreaks might cause import to fail. See Initial Parsing Steps in the User GuideD s product - Plain Text
- LOG
- TSV
Parquet
Info NOTE: When working with datasets sourced from Parquet files, lineage information and the
$sourcerownumber
reference are not supported.Avro
Info NOTE: When working with datasets sourced from Avro files, lineage information and the
$sourcerownumber
reference are not supported.Google Sheets
D s ed r true editions gdpent,gdppro,gdpsta,gdppr,gdpst,awsent,awspro,awspr
See Import Google Sheets DataInfo NOTE: Individual users must enable access to their Google Drive. No data other than Google Sheets is read from Google Drive.
.
For more information on data is handled initially, see Initial Parsing Steps in the User Guide.
Converted file formats
Files of the following type are not read into the product in their native format. Instead, these file types are converted using the Conversion Service into a file format that is natively supported, stored in the base storage layer, and then ingested for use in the product.
...
Excel (XLS/XLSX)
Info NOTE: Other Excel-related formats, such as XLSM format, are not supported.
Tip Tip: You may import multiple worksheets from a single workbook at one time. See Import Excel Data in the User Guide.
Google Sheets
D s ed r true editions gdpent,gdppro,gdpsta,gdppr,gdpst,awsent,awspro,awspr Tip Tip: You may import multiple sheets from a single Google Sheet at one time. See Import Google Sheets Data in the User Guide.
PDF
- PDF is supported for import only.
- See Import PDF Data in the User Guide.
- JSON v2
Notes on JSON:
There are two methods of ingesting JSON files for use in the product.
...
D s product |
---|
- CSV
- JSON
Avro
- BigQuery Table
Compression Algorithms
...
File extension | Format name | Notes | ||
---|---|---|---|---|
.sz | Framing2 format | See: https://github.com/google/snappy/blob/master/framing_format.txt | ||
.snappy | Hadoop-snappy format | See: https://code.google.com/p/hadoop-snappy/
|
...
D s also | ||||
---|---|---|---|---|
|