Page tree

Release 8.7.1


Contents:

   

Contents:


Computes the rolling average of values forward or backward of the current row within the specified column.
  • If an input value is missing or null, it is not factored in the computation. For example, for the first row in the dataset, the rolling average of previous values is the value in the first row.
  • The row from which to extract a value is determined by the order in which the rows are organized based on the order parameter. 

  • If you are working on a randomly generated sample of your dataset, the values that you see for this function might not correspond to the values that are generated on the full dataset during job execution.

  • The function takes a column name and two optional integer parameters that determine the window backward and forward of the current row.
    • The default integer parameter values are -1 and 0, which computes the rolling average from the current row back to the first row of the dataset.
  • This function works with the Window transform. See Window Transform.

For more information on a non-rolling version of this function, see AVERAGE Function.

Wrangle vs. SQL: This function is part of Wrangle, a proprietary data transformation language. Wrangle is not SQL. For more information, see Wrangle Language.

Basic Usage

Column example:

rollingaverage(myCol)


Output: 
Returns the rolling average of all values in the myCol column.

Rows before example:

rollingaverage(myNumber, 3)

Output: Returns the rolling average of the current row and the three previous row values in the myNumber column.

Rows before and after example:

rollingaverage(myNumber, 3, 2)


Output:
 Returns 
the rolling average of the three previous row values, the current row value, and the two rows after the current one in the myNumber column.     

Syntax and Arguments

rollingaverage(col_ref, rowsBefore_integer, rowsAfter_integer) order: order_col [group: group_col]


ArgumentRequired?Data TypeDescription
col_refYstringName of column whose values are applied to the function
rowsBefore_integerNintegerNumber of rows before the current one to include in the computation
rowsAfter_integerNintegerNumber of rows after the current one to include in the computation

For more information on the order and group parameters, see Window Transform.

For more information on syntax standards, see Language Documentation Syntax Notes.

col_ref

Name of the column whose values are used to compute the rolling average. 

  • Multiple columns and wildcards are not supported.

Usage Notes:

Required?Data TypeExample Value
YesString (column reference to Integer or Decimal values)myColumn

rowsBefore_integer, rowsAfter_integer

Integers representing the number of rows before or after the current one from which to compute the rolling average, including the current row. For example, if the first value is 5, the current row and the five rows before it are used in the computation. Negative values for rowsAfter_integer compute the rolling function from rows preceding the current one.

  • rowBefore=0 generates the current row value only.
  • rowBefore=-1  uses all rows preceding the current one.
  • If rowsAfter is not specified, then the value 0 is applied.
  • If a group parameter is applied, then these parameter values should be no more than the maximum number of rows in the groups.

Usage Notes:

Required?Data TypeExample Value
NoInteger4

Examples


Tip: For additional examples, see Common Tasks.

Example - Compute prior quarter values

The following dataset contains order information for the preceding 12 months. You want to compare the current month's average against the preceding quarter. 

Source:

DateAmount
12/31/15118
11/30/156
10/31/15443
9/30/15785
8/31/1577
7/31/15606
6/30/15421
5/31/15763
4/30/15305
3/31/15824
2/28/15135
1/31/15523

Transformation:

Using the ROLLINGAVERAGE function, you can generate a column containing the rolling average of the current month and the two previous months:

Transformation Name Window
Parameter: Formulas ROLLINGAVERAGE(Amount, 3, 0)
Parameter: Order by -Date

Note the sign of the second parameter and the order parameter. The sort is in the reverse order of the Date parameter, which preserves the current sort order. As a result, the second parameter, which identifies the number of rows to use in the calculation, must be positive to capture the previous months.

Technically, this computation does not capture the prior quarter, since it includes the current quarter as part of the computation. You can use the following column to capture the rolling average of the preceding month, which then becomes the true rolling average for the prior quarter. The window column refers to the name of the column generated from the previous step:

Transformation Name Window
Parameter: Formulas NEXT(window, 1)
Parameter: Order by -Date

Note that the order parameter must be preserved. This new column, window1, contains your prior quarter rolling average:

Transformation Name Rename columns
Parameter: Option Manual rename
Parameter: Column window1
Parameter: New column name 'Amount_PriorQtr'

You can reformat this numeric value:

Transformation Name Edit column with formula
Parameter: Columns Amount_PriorQtr
Parameter: Formula NUMFORMAT(Amount_PriorQtr, '###.00')

You can use the following transformation to calculate the net change. This formula computes the change as a percentage of the prior quarter and then formats it as a two-digit percentage.

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula NUMFORMAT(((Amount - Amount_PriorQtr) / Amount_PriorQtr) * 100, '##.##')
Parameter: New column name 'NetChangePct_PriorQtr'

Results:

NOTE: You might notice that there are computed values for Amount_PriorQtr for February and March. These values do not factor in a full three months because the data is not present. The January value does not exist since there is no data preceding it.

DateAmountAmount_PriorQtrNetChangePct_PriorQtr
12/31/15118411.33-71.31
11/30/156435.00-98.62
10/31/15443489.33-9.47
9/30/15785368.00113.32
8/31/1577596.67-87.1
7/31/15606496.3322.1
6/30/15421630.67-33.25
5/31/15763421.3381.09
4/30/15305494.00-38.26
3/31/15824329.00150.46
2/28/15135523.00-.74.19
1/31/15523  

Example - Rolling window functions

This example describes how to use the rolling computational functions:
  • ROLLINGSUM - computes a rolling sum from a window of rows before and after the current row. See ROLLINGSUM Function.
  • ROLLINGAVERAGE - computes a rolling average from a window of rows before and after the current row. See ROLLINGAVERAGE Function.
  • ROWNUMBER - computes the row number for each row, as determined by the ordering column. See ROWNUMBER Function.

The following dataset contains sales data over the final quarter of the year. 

Source:

DateSales
10/2/16200
10/9/16500
10/16/16350
10/23/16400
10/30/16190
11/6/16550
11/13/16610
11/20/16480
11/27/16660
12/4/16690
12/11/16810
12/18/16950
12/25/161020
1/1/17680


Transformation:

First, you want to maintain the row information as a separate column. Since data is ordered already by the Date column, you can use the following:

Transformation Name Window
Parameter: Formulas ROWNUMBER()
Parameter: Order by Date

Rename this column to rowId for week of quarter.

Now, you want to extract month and week information from the Date values. Deriving the month value:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula MONTH(Date)
Parameter: New column name 'Month'

Deriving the quarter value:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula (1 + FLOOR(((month-1)/3)))
Parameter: New column name 'QTR'

Deriving the week-of-quarter value:

Transformation Name Window
Parameter: Formulas ROWNUMBER()
Parameter: Group by QTR
Parameter: Order by Date

Rename this column WOQ (week of quarter).

Deriving the week-of-month value:

Transformation Name Window
Parameter: Formulas ROWNUMBER()
Parameter: Group by Month
Parameter: Order by Date

Rename this column WOM (week of month).

Now, you perform your rolling computations. Compute the running total of sales using the following:

Transformation Name Window
Parameter: Formulas ROLLINGSUM(Sales, -1, 0)
Parameter: Group by QTR
Parameter: Order by Date

The -1 parameter is used in the above computation to gather the rolling sum of all rows of data from the current one to the first one. Note that the use of the QTR column for grouping, which moves the value for the 01/01/2017 into its own computational bucket. This may or may not be preferred.

Rename this column QTD (quarter to-date). Now, generate a similar column to compute the rolling average of weekly sales for the quarter:

Transformation Name Window
Parameter: Formulas ROUND(ROLLINGAVERAGE(Sales, -1, 0))
Parameter: Group by QTR
Parameter: Order by Date

Since the ROLLINGAVERAGE function can compute fractional values, it is wrapped in the ROUND function for neatness. Rename this column avgWeekByQuarter.

Results:

When the unnecessary columns are dropped and some reordering is applied, your dataset should look like the following:

DateWOQSalesQTDavgWeekByQuarter
10/2/161200200200
10/9/162500700350
10/16/1633501050350
10/23/1644001450363
10/30/1651901640328
11/6/1665502190365
11/13/1676102800400
11/20/1684803280410
11/27/1696603940438
12/4/16106904630463
12/11/16118105440495
12/18/16129506390533
12/25/161310207410570
1/1/171680680680


Example - Rolling computations for racing splits


This example describes how to use the rolling computational functions:

  • ROLLINGAVERAGE - computes a rolling average from a window of rows before and after the current row. See ROLLINGAVERAGE Function.
  • ROLLINGMIN - computes a rolling minimum from a window of rows. See ROLLINGMIN Function.
  • ROLLINGMAX - computes a rolling maximum from a window of rows.  See ROLLINGMAX Function.
  • ROLLINGSTDEV - computes a rolling standard deviation from a window of rows. See ROLLINGSTDEV Function.
  • ROLLINGVAR - computes a rolling variance from a window of rows. See ROLLINGVAR Function.
  • ROLLINGSTDEVSAMP - computes a rolling standard deviation from a window of rows using the sample method of statistical calculation. See ROLLINGSTDEVSAMP Function.
  • ROLLINGVARSAMP - computes a rolling variance from a window of rows using the sample method of statistical calculation. See ROLLINGVARSAMP Function.

Source:

In this example, the following data comes from times recorded at regular intervals during a three-lap race around a track. The source data is in cumulative time in seconds (time_sc). You can use ROLLING and other windowing functions to break down the data into more meaningful metrics.

lapquartertime_sc
100.000
1119.554
1239.785
1360.021
2080.950
21101.785
22121.005
23141.185
30162.008
31181.887
32200.945
33220.856

Transformation:

Primary key: Since the quarter information repeats every lap, there is no unique identifier for each row. The following steps create this identifier:

Transformation Name Change column data type
Parameter: Columns lap,quarter
Parameter: New type String

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula MERGE(['l',lap,'q',quarter])
Parameter: New column name 'splitId'

Get split times: Use the following transform to break down the splits for each quarter of the race:

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(time_sc - PREV(time_sc, 1), 3)
Parameter: Order rows by splitId
Parameter: New column name 'split_time_sc'

Compute rolling computations: You can use the following types of computations to provide rolling metrics on the current and three previous splits:

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROLLINGAVERAGE(split_time_sc, 3)
Parameter: Order rows by splitId
Parameter: New column name 'ravg'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROLLINGMAX(split_time_sc, 3)
Parameter: Order rows by splitId
Parameter: New column name 'rmax'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROLLINGMIN(split_time_sc, 3)
Parameter: Order rows by splitId
Parameter: New column name 'rmin'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGSTDEV(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rstdev'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGVAR(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rvar'

Compute rolling computations using sample method: These metrics compute the rolling STDEV and VAR on the current and three previous splits using the sample method:

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGSTDEVSAMP(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rstdev_samp'

Transformation Name New formula
Parameter: Formula type Multiple row formula
Parameter: Formula ROUND(ROLLINGVARSAMP(split_time_sc, 3), 3)
Parameter: Order rows by splitId
Parameter: New column name 'rvar_samp'

Results:

When the above transforms have been completed, the results look like the following:

lapquartersplitIdtime_scsplit_time_scrvar_samprstdev_samprvarrstdevrminrmaxravg
10l1q00 

     
11l1q120.09620.096

0020.09620.09620.096
12l1q240.5320.4340.2290.4790.0290.16920.09620.43420.265
13l1q361.03120.5010.1540.3920.0310.17720.09620.50120.344
20l2q081.08720.0560.3150.5610.0390.19820.05620.50120.272
21l2q1101.38320.2960.1420.3760.0290.1720.05620.50120.322
22l2q2122.09220.7090.6170.7860.0590.24220.05620.70920.39
23l2q3141.88619.7940.6210.7880.1130.33719.79420.70920.214
30l3q0162.58120.6950.5790.7610.1390.37319.79420.70920.373
31l3q1183.01820.4370.4430.6660.1380.37119.79420.70920.409
32l3q2203.49320.4750.5370.7330.1130.33619.79420.69520.35
33l3q3222.89319.40.5200.7210.2520.50219.420.69520.252

You can reduce the number of steps by applying a window transform such as the following:

Transformation Name Window
Parameter: Formula1 lap
Parameter: Formula2 rollingaverage(split_time_sc, 0, 3)
Parameter: Formula3 rollingmax(split_time_sc, 0, 3)
Parameter: Formula4 rollingmin(split_time_sc, 0, 3)
Parameter: Formula5 round(rollingstdev(split_time_sc, 0, 3), 3)
Parameter: Formula6 round(rollingvar(split_time_sc, 0, 3), 3)
Parameter: Formula7 round(rollingstdevsamp(split_time_sc, 0, 3), 3)
Parameter: Formula8 round(rollingvarsamp(split_time_sc, 0, 3), 3)
Parameter: Group by lap
Parameter: Order by lap

However, you must rename all of the generated windowX columns.

 

See Also for ROLLINGAVERAGE Function:

This page has no comments.