Data Quality Rules Reference

Note

This feature may not be available in all product editions. For more information on available features, see Compare Editions.

This section contains reference information on the data quality rule types and input types that are available in Dataprep by Trifacta.

Data quality rules can be applied to your dataset through the Transformer page.
Input types identify the calculated metric types that can be used as inputs for a data quality rule.

Rule Types

name	description
Unique	Column values must be unique.
Implies	Source column values imply the values of a target column. For each unique source value, there should be exactly one implied target value.
Not Missing	Column values must not be missing. Null values and empty strings are not allowed.
Not Null	Column values must not be null. Empty strings are allowed.
Valid	Column values must be valid instances of a data type.
Match	Column values must match a pattern.
Not Match	Column values must not match a pattern.
Starts With	Column values must start with a pattern.
Ends With	Column values must end with a pattern.
Equal	Column values must equal a provided value.
Not Equal	Column values must not equal a provided value.
In Range	Column values must lie between provided minimum and maximum values.
Greater Than	Column values must be greater than a minimum value.
Less Than	Column values must be less than a maximum value.
In Set	Column values must be one of a set of acceptable values.
Not In Set	Column values must not be one of a set of unacceptable values.
Formula	Apply a custom data quality rule formula.

The following metric input types can be selected as the source of a data quality rule.

Note

These input types are available for selection from the Column drop-down.

Metric input types are supported for the following rules:

name	description
Average	The average column value.
Count Distinct	The number of unique column values.
Maximum	The maximum column value.
Minimum	The minimum column value.
Sum	The sum of column values.
Standard Deviation	The sample standard deviation of column values.
Variance	The sample variance of column values.
Count	The number of rows.
Correlation	The Pearson correlation coefficient between two columns.
Z-Score	The distance from the mean, in units of standard deviations.