Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

D toc

D s transforms

Excerpt

Unpacks array data into separate rows for each value. This transform operates on a single column.

This transform does not reference keys in the array. If your array data contains keys, use the unnest transform. See Unnest Transform.

D s
snippetBasic

D code

flatten col: myArray

Output: Generates a separate row for each value in the array. Values of other columns in generated rows are copied from the source. 

D s
snippetParameters

D code

flatten: col: column_ref

TokenRequired?Data TypeDescription
flattenYtransformName of the transform
colYstringSource column name 

D s lang notes

col

Identifies the column to which to apply the transform. You can specify only one column.

D s
snippetusage

Required?Data Type
YesString (column name)


D s
snippetExamples

Example - Flatten an array

In this example, the source data includes an array of scores that need to broken out into separate rows.

Source:

LastNameFirstNameScores
AdamsAllen[81,87,83,79]
BurnsBonnie[98,94,92,85]
CannonChris[88,81,85,78]

Transformation:

When the data is imported, you might have to re-type the Scores column as an array:

D trans
RawWrangletrue
Typestep
WrangleTextsettype col: Scores type: 'Array'
p01NameColumns
p01ValueScores
p02NameNew type
p02ValueArray
SearchTermChange column data type

You can now flatten the Scores column data into separate rows:

D trans
RawWrangletrue
Typestep
WrangleTextflatten col: Scores
p01NameColumn
p01ValueScores
SearchTermExpand Array into rows

Results:

LastNameFirstNameScores
AdamsAllen81
AdamsAllen87
AdamsAllen83
AdamsAllen79
BurnsBonnie98
BurnsBonnie94
BurnsBonnie92
BurnsBonnie85
CannonChris88
CannonChris81
CannonChris85
CannonChris78

This example is extended below.

Example - Flatten and unnest together

While the above example nicely flattens out your data, there are two potential problems with the results:

  • There is no identifier for each test. For example, Allen Adams' score of 87 cannot be associated with the specific test on which he recorded the score.
  • There is no unique identifier for each row.

The following example addresses both of these issues. It also demonstrates differences between the  unnest and the flatten transform, including how you use  unnest to flatten array data based on specified keys.

Include Page
EXAMPLE - Flatten and Unnest Transforms
EXAMPLE - Flatten and Unnest Transforms

D s also
labelwrangle_transform_flatten