Page tree

The cloud-based product Trifacta Free is available now! Register for your free account.

 

Contents:


Generates the standard deviation of values by group in a column that meet a specific condition.

NOTE: When added to a transform, this function is applied to the sample in the data grid. If you change your sample or run the job, the computed values for this function are updated. Transforms that change the number of rows in subsequent recipe steps do not affect the values computed for this step.

 Terms...

Relevant terms:

Term Description
Population Population statistical functions are computed from all possible values. See https://en.wikipedia.org/wiki/Statistical_population.
Sample

Sample-based statistical functions are computed from a subset or sample of all values. See https://en.wikipedia.org/wiki/Sampling_(statistics).

These function names include SAMP in their name.

NOTE: Statistical sampling has no relationship to the samples taken within the product. When statistical functions are computed during job execution, they are applied across the entire dataset. Sample method calculations are computed at that time.

For more information on how the platform calculates standard deviation, see STDEV Function.

Wrangle vs. SQL: This function is part of Wrangle , a proprietary data transformation language. Wrangle is not SQL. For more information, see Wrangle Language.

Basic Usage

stdevif(testScores, testScores > 0)

Output: Returns the standard deviation of the testScores column when the testScores value is greater than 0.

Syntax and Arguments

stdevif(col_ref, test_expression) [group:group_col_ref] [limit:limit_count]


ArgumentRequired?Data TypeDescription
col_refYstringReference to the column you wish to evaluate.
test_expressionYstringExpression that is evaluated. Must resolve to true or false

For more information on syntax standards, see Language Documentation Syntax Notes.

For more information on the group and limit parameters, see Pivot Transform.

col_ref

Name of the column whose values you wish to use in the calculation. Column must be a numeric (Integer or Decimal) type.

Usage Notes:

Required?Data TypeExample Value
YesString that corresponds to the name of the columnmyValues

test_expression

This parameter contains the expression to evaluate. This expression must resolve to a Boolean (true or false) value.

Usage Notes:

Required?Data TypeExample Value
YesString expression that evaluates to true or false(LastName == 'Mouse' && FirstName == 'Mickey')


Examples

Tip: For additional examples, see Common Tasks.

Example - Conditional Calculation Functions

This example illustrates how you can use the following conditional calculation functions to analyze weather data:
  • AVERAGEIF - Average of a set of values by group that meet a specified condition. See AVERAGEIF Function.
  • MINIF - Minimum of a set of values by group that meet a specified condition. See MINIF Function.
  • MAXIF - Maximum of a set of values by group that meet a specified condition. See MAXIF Function.
  • VARIF - Variance of a set of values by group that meet a specified condition. See VARIF Function.
  • STDEVIF - Standard deviation of a set of values by group that meet a specified condition. See STDEVIF Function.

Source:

Here is some example weather data:

datecityraintempwind
1/23/17Valleyville0.0012.86.7
1/23/17Center Town0.319.45.3
1/23/17Magic Mountain0.000.07.3
1/24/17Valleyville0.2517.23.3
1/24/17Center Town0.541.17.6
1/24/17Magic Mountain0.325.08.8
1/25/17Valleyville0.023.36.8
1/25/17Center Town0.833.35.1
1/25/17Magic Mountain0.59-1.76.4
1/26/17Valleyville1.0815.04.2
1/26/17Center Town0.966.17.6
1/26/17Magic Mountain0.77-3.93.0
1/27/17Valleyville1.007.22.8
1/27/17Center Town1.3220.00.2
1/27/17Magic Mountain0.775.65.2
1/28/17Valleyville0.12-6.15.1
1/28/17Center Town0.145.04.9
1/28/17Magic Mountain1.501.10.4
1/29/17Valleyville0.3613.37.3
1/29/17Center Town0.756.19.0
1/29/17Magic Mountain0.603.36.0


Transformation:

The following computes average temperature for rainy days by city:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula AVERAGEIF(temp, rain > 0)
Parameter: Group rows by city
Parameter: New column name 'avgTempWRain'

The following computes maximum wind for sub-zero days by city:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula MAXIF(wind,temp < 0)
Parameter: Group rows by city
Parameter: New column name 'maxWindSubZero'

This step calculates the minimum temp when the wind is less than 5 mph by city:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula MINIF(temp,wind<5)
Parameter: Group rows by city
Parameter: New column name 'minTempWind5'

This step computes the variance in temperature for rainy days by city:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula VARIF(temp,rain >0)
Parameter: Group rows by city
Parameter: New column name 'varTempWRain'

The following computes the standard deviation in rainfall for Center Town:

Transformation Name New formula
Parameter: Formula type Single row formula
Parameter: Formula STDEVIF(rain,city=='Center Town')
Parameter: Group rows by city
Parameter: New column name 'stDevRainCT'

You can use the following transforms to format the generated output. Note the $col placeholder value for the multi-column transforms:

Transformation Name Edit column with formula
Parameter: Columns stDevRainCenterTown,maxWindSubZero
Parameter: Formula numformat($col,'##.##')

Since the following rely on data that has only one significant digit, you should format them differently:

Transformation Name Edit column with formula
Parameter: Columns varTempWRain,avgTempWRain,minTempWind5
Parameter: Formula numformat($col,'##.#')

Results:

datecityraintempwindavgTempWRainmaxWindSubZerominTempWind5varTempWRainstDevRainCT
1/23/17Valleyville0.0012.86.78.35.17.263.80.37
1/23/17Center Town0.319.45.37.3 532.60.37
1/23/17Magic Mountain0.000.07.31.66.43-3.9120.37
1/24/17Valleyville0.2517.23.38.35.17.263.80.37
1/24/17Center Town0.541.17.67.3 532.60.37
1/24/17Magic Mountain0.325.08.81.66.43-3.9120.37
1/25/17Valleyville0.023.36.88.35.17.263.80.37
1/25/17Center Town0.833.35.17.3 532.60.37
1/25/17Magic Mountain0.59-1.76.41.66.43-3.9120.37
1/26/17Valleyville1.0815.04.28.35.17.263.80.37
1/26/17Center Town0.966.17.67.3 532.60.37
1/26/17Magic Mountain0.77-3.93.01.66.43-3.9120.37
1/27/17Valleyville1.007.22.88.35.17.263.80.37
1/27/17Center Town1.3220.00.27.3 532.60.37
1/27/17Magic Mountain0.775.65.21.66.43-3.9120.37
1/28/17Valleyville0.12-6.15.18.35.17.263.80.37
1/28/17Center Town0.145.04.97.3 532.60.37
1/28/17Magic Mountain1.501.10.41.66.43-3.9120.37
1/29/17Valleyville0.3613.37.38.35.17.263.80.37
1/29/17Center Town0.756.19.07.3 532.60.37
1/29/17Magic Mountain0.603.36.01.66.43-3.9120.37

 

This page has no comments.