Unnest Your Data

Array values can be flattened into individual values in separate rows.

This section describes how to flatten the values in an Array into separate rows in your dataset.

Source:

In the following example dataset, students took the same test three times, and their scores were stored in any array in the Scores column.

Transformation:

When the data is imported, you might have to re-type the Scores column as an array:

Scores

Array

You can now flatten the Scores column data into separate rows:

Scores

Results:

You can also split out the individual values in an array into separate columns.

This section describes how to unnest the values in an Array into separate columns in your dataset.

Source:

In the following example dataset, students took the same test three times, and their scores were stored in any array in the Scores column.

Transformation:

When the data is imported, you might have to re-type the Scores column as an array:

Scores

Array

You can now unnest the Scores column data into separate columns:

Scores

[0]

[1]

[2]

[3]

true

true

In the above transformation:

Results:

The following example illustrates how flatten and unnest can be used together to reshape your data.

This example illustrates you to use the flatten and unnest transforms.

Source:

You have the following data on student test scores. Scores on individual scores are stored in the Scores array, and you need to be able to track each test on a uniquely identifiable row. This example has two goals:

Transformation:

When the data is imported from CSV format, you must add a header transform and remove the quotes from the Scores column:

Use row(s) as column names

Use a single row to name columns

1

colScores

'\"'

''

true

Validate test date: To begin, you might want to check to see if you have the proper number of test scores for each student. You can use the following transform to calculate the difference between the expected number of elements in the Scores array (4) and the actual number:

Single row formula

(4 - arraylen(Scores))

'numMissingTests'

When the transform is previewed, you can see in the sample dataset that all tests are included. You might or might not want to include this column in the final dataset, as you might identify missing tests when the recipe is run at scale.

Unique row identifier: The Scores array must be broken out into individual rows for each test. However, there is no unique identifier for the row to track individual tests. In theory, you could use the combination of LastName-FirstName-Scores values to do so, but if a student recorded the same score twice, your dataset has duplicate rows. In the following transform, you create a parallel array called Tests, which contains an index array for the number of values in the Scores column. Index values start at 0:

Single row formula

range(0,arraylen(Scores))

'Tests'

Also, we will want to create an identifier for the source row using the sourcerownumber function:

Single row formula

sourcerownumber()

'orderIndex'

One row for each student test: Your data should look like the following:

Now, you want to bring together the Tests and Scores arrays into a single nested array using the arrayzip function:

Single row formula

arrayzip([Tests,Scores])

Your dataset has been changed:

Use the following to unpack the nested array:

column1

Each test-score combination is now broken out into a separate row. The nested Test-Score combinations must be broken out into separate columns using the following:

column1

'[0]','[1]'

After you delete column1, which is no longer needed you should rename the two generated columns:

Manual rename

column_0

'TestNum'

Manual rename

column_1

'TestScore'

Unique row identifier: You can do one more step to create unique test identifiers, which identify the specific test for each student. The following uses the original row identifier OrderIndex as an identifier for the student and the TestNumber value to create the TestId column value:

Single row formula

(orderIndex * 10) + TestNum

'TestId'

The above are integer values. To make your identifiers look prettier, you might add the following:

'TestId00','TestId'

Extending: You might want to generate some summary statistical information on this dataset. For example, you might be interested in calculating each student's average test score. This step requires figuring out how to properly group the test values. In this case, you cannot group by the LastName value, and when executed at scale, there might be collisions between first names when this recipe is run at scale. So, you might need to create a kind of primary key using the following:

'LastName','FirstName'

'-'

'studentId'

You can now use this as a grouping parameter for your calculation:

Single row formula

average(TestScore)

studentId

'avg_TestScore'

Results:

After you delete unnecessary columns and move your columns around, the dataset should look like the following:

This example shows how you can unnest Object data into separate columns. The example contains vehicle identifiers, and the Properties column contains key-value pairs describing characteristics of each vehicle.

This example shows how you can unpack data nested in an Object into separate columns.

Source:

You have the following information on used cars. The VIN column contains vehicle identifiers, and the Properties column contains key-value pairs describing characteristics of each vehicle. You want to unpack this data into separate columns.

Transformation:

Add the following transformation, which identifies all of the key values in the column as beginning with alphabetical characters.

Properties

`{alpha}+`

`=`

','

Now that the Object of values has been created, you can use the unnest transform to unpack this mapped data. In the following, each key is specified, which results in separate columns headed by the named key:

extractkv_Properties

year

make

model

color

mileage

cost

Results:

When you delete the unnecessary Properties columns, the dataset now looks like the following:

This example shows how to extract values (for example, hashtag values) from a column and convert them into a column of arrays.

In this example, you extract one or more values from a source column and assemble them in an Array column.

Suppose you need to extract the hashtags from customer tweets to another column. In such cases, you can use the {hashtag} Alteryx pattern to extract all hashtag values from a customer's tweets into a new column.

Source:

The following dataset contains customer tweets across different locations.

Transformation:

The following transformation extracts the hashtag messages from customer tweets.

customer_tweets

`{hashtag}`

Hashtag tweets

Then, the source column can be deleted.

Results: