Page tree

NOTE:  Trifacta Wrangler is a free product with limitations on its features. Some features in the documentation do not apply to this product edition. See Product Limitations.

   

This example describes how to use rolling statistical functions.

Functions:

ItemDescription
ROLLINGAVERAGE Function Computes the rolling average of values forward or backward of the current row within the specified column.
ROLLINGMAX Function Computes the rolling maximum of values forward or backward of the current row within the specified column. Inputs can be Integer, Decimal, or Datetime.
ROLLINGSTDEV Function Computes the rolling standard deviation of values forward or backward of the current row within the specified column.
ROLLINGVAR Function Computes the rolling variance of values forward or backward of the current row within the specified column.
ROLLINGSTDEVSAMP Function Computes the rolling standard deviation of values forward or backward of the current row within the specified column using the sample statistical method.
ROLLINGVARSAMP Function Computes the rolling variance of values forward or backward of the current row within the specified column using the sample statistical method.

Also:

ItemDescription
MERGE Function Merges two or more columns of String type to generate output of String type. Optionally, you can insert a delimiter between the merged values.
ROUND Function Rounds input value to the nearest integer. Input can be an Integer, a Decimal, a column reference, or an expression. Optional second argument can be used to specify the number of digits to which to round.

Source:

In this example, the following data comes from times recorded at regular intervals during a three-lap race around a track. The source data is in cumulative time in seconds (time_sc). You can use ROLLING and other windowing functions to break down the data into more meaningful metrics.

lapquartertime_sc
100.000
1119.554
1239.785
1360.021
2080.950
21101.785
22121.005
23141.185
30162.008
31181.887
32200.945
33220.856

Transformation:

Primary key: Since the quarter information repeats every lap, there is no unique identifier for each row. The following steps create this identifier:

Transformation Name Change column data type
Parameter: Columns lap,quarter
Parameter: New type String

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula MERGE(['l',lap,'q',quarter])
Parameter: New column name 'splitId'

Get split times: Use the following transform to break down the splits for each quarter of the race:

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(time_sc - PREV(time_sc, 1), 3)
Parameter: Order rows by splitId
Parameter: New column name 'split_time_sc'

Compute rolling computations: You can use the following types of computations to provide rolling metrics on the current and three previous splits:

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROLLINGAVERAGE(split_time_sc, 3)
Parameter: Order rows by splitId
Parameter: New column name 'ravg'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROLLINGMAX(split_time_sc, 3)
Parameter: Order rows by splitId
Parameter: New column name 'rmax'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROLLINGMIN(split_time_sc, 3)
Parameter: Order rows by splitId
Parameter: New column name 'rmin'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGSTDEV(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rstdev'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGVAR(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rvar'

Compute rolling computations using sample method: These metrics compute the rolling STDEV and VAR on the current and three previous splits using the sample method:

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGSTDEVSAMP(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rstdev_samp'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGVARSAMP(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rvar_samp'

Results:

When the above transforms have been completed, the results look like the following:

lapquartersplitIdtime_scsplit_time_scrvar_samprstdev_samprvarrstdevrminrmaxravg
10l1q00 

     
11l1q120.09620.096

0020.09620.09620.096
12l1q240.5320.4340.2290.4790.0290.16920.09620.43420.265
13l1q361.03120.5010.1540.3920.0310.17720.09620.50120.344
20l2q081.08720.0560.3150.5610.0390.19820.05620.50120.272
21l2q1101.38320.2960.1420.3760.0290.1720.05620.50120.322
22l2q2122.09220.7090.6170.7860.0590.24220.05620.70920.39
23l2q3141.88619.7940.6210.7880.1130.33719.79420.70920.214
30l3q0162.58120.6950.5790.7610.1390.37319.79420.70920.373
31l3q1183.01820.4370.4430.6660.1380.37119.79420.70920.409
32l3q2203.49320.4750.5370.7330.1130.33619.79420.69520.35
33l3q3222.89319.40.5200.7210.2520.50219.420.69520.252

You can reduce the number of steps by applying a window transform such as the following:

Transformation Name Window
Parameter: Formula1 lap
Parameter: Formula2 rollingaverage(split_time_sc, 0, 3)
Parameter: Formula3 rollingmax(split_time_sc, 0, 3)
Parameter: Formula4 rollingmin(split_time_sc, 0, 3)
Parameter: Formula5 round(rollingstdev(split_time_sc, 0, 3), 3)
Parameter: Formula6 round(rollingvar(split_time_sc, 0, 3), 3)
Parameter: Formula7 round(rollingstdevsamp(split_time_sc, 0, 3), 3)
Parameter: Formula8 round(rollingvarsamp(split_time_sc, 0, 3), 3)
Parameter: Group by lap
Parameter: Order by lap

However, you must rename all of the generated windowX columns.

This page has no comments.