Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r094

...

Excerpt

Version 2: This section describes how you can import JSON files into 

D s product
rtrue
, convert them to tabular format, wrangle them, and then export them back in the same JSON format.

The basic workflow task is described by way of example. In the example workflowtask, the JSON file must be imported into 

D s product
rtrue
, a new column must be inserted into the JSON, and the resulting JSON must be exported in the same structure.

...

This method of working with JSON is enabled by default.

Info

NOTE: When this feature is enabled, all JSON imported datasets created under the legacy method must be recreated to behave like v2 datasets with respect to conversion and schema management. Features developed in the future may not retroactively be supported in the v1 legacy mode. You should convert to using the v2 method.


...


You can choose to continue using the legacy method of working with JSON.

...

  • Each row is a complete JSON record containing keys and values.

    Tip

    Tip: Nested JSON, such as metrics above, can be inserted as part of a record. It can then be unnested within the application.

  • Each key's value must have a comma after it, except for the final key value in any row. 

    Info

    NOTE: The end of a JSON record is the right curly bracket (}). Commas are not added to the end of each line in this format.

...

Task

  1. Import the JSON file.

  2. Any nested data must be unnested within columns. Each level in the JSON hierarchy must be un-nested in a separate step.
  3. When all of the JSON data is in tabular form, perform any 
    D s lang
     transformations. 
  4. If you need to rebuild the loose JSON hierarchy, you must nest the lower levels of the JSON hierarchy back into their original form.
    1. If it is ok to write out flat JSON records, you can export without nesting the data again.
  5. Run the job, generating a JSON output. 

...