Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

...

  • For JSON files, it is important to import them in unstructured format.
  • D s product
    rtrue
    requires that JSON files be submitted with one valid JSON object per line. 
    • Multi-line JSON import is not supported.
    • Consistently malformed JSON objects or objects that overlap linebreaks might cause import to fail.

Step 1: Split the rows. JSON data is initially split using Splitrows transformation.

Step 2: Unnest the rows. Then, the data must be broken out from the nested JSON structures into flat rows. Using the Unnest Objects transformation, the application attempts to render the JSON into consistently formatted rows. 

Info

NOTE: After initial parsing, you might need to apply the Unnest Objects transformation multiple times on individual columns to completely unnest the data.

Step 3: Delete source. If the data is successfully unnested, the source column is removed with a Delete Columns transformation.

Example recipe:

D trans
p03Value\"
Typestep
p01NameColumn
p01Valuecolumn1
p02NameSplit on
p02Value\n
p03NameQuote escape character
SearchTermSplit into rows

In the following, the values c1c3 identify the keys used to demarcate top-level nodes in the JSON source. These become individual column headers in the data grid.

D trans
p03Valuetrue
Typestep
p01NameColumn
p01Valuecolumn1
p02NamePaths to elements
p02Valuec1,c2,c3
p03NameRemove elements from original
SearchTermUnnest Objects into columns

If the above successfully executes, the source column is deleted:

...

...

For more information, see Working with JSON v2.

Known Issues

  • Some characters in imported datasets, such as NUL (ASCII character 0) characters, may cause problems with recognizing line breaks. If initial parsing is having trouble with line breaks, you may need to fix the issue in the source data prior to import, since the Splitrows transformation must be the first step in your recipe. 

...