Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

...

Tip

Tip: You can annotate your recipe with comments, such as: sample: random and then create a new sample at that location.

Sample Types

D s product
 currently supports the following sampling methods.

First rows samples

This sample is taken from the first set of rows in the transformed dataset based on the current cursor location in the recipe. The first N rows in the dataset are collected based on the recipe steps up to the configured sample size.

  • This sample may span multiple datasets and files, depending on how the recipe is constructed.
  • The first rows sample is different from the initial sample, which is gathered without reference to any recipe steps.  

These samples are fast to generate. These samples may load faster in the application than samples of other types.

Tip

Tip: If you have chained together multiple recipes, all steps in all linked recipes must be run to provide visual updates. If you are experiencing performance problems related to this kind of updating, you can select a recipe in the middle of the chain of recipes and switch it off the initial sample to a different sample. When invoked, the recipes from the preceding datasets do not need to be executed, which can improve performance.

Random samples

Random selection of a subset of rows in the dataset. These samples are comparatively fast to generate. You can apply quick scan or full scan to determine the scope of the sample.

Filter-based samples

Find specific values in one or more columns. For the matching set of values, a random sample is generated.

You must define your filter in the Filter textbox.

Anomaly-based samples

Find mismatched or missing data or both in one or more columns.

You specify one or more columns and whether the anomaly is:

  1. mismatched
  2. missing
  3. either of the above

Optionally, you can define an additional filter on other columns.

Stratified samples

Find all unique values within a column and create a sample that contains the unique values, up to the sample size limit. The distribution of the column values in the sample reflects the distribution of the column values in the dataset. Sampled values are sorted by frequency, relative to the specified column.

Optionally, you can apply a filter to this one.

Tip

Tip: Collecting samples containing all unique values can be useful if you are performing mapping transformations, such as values to columns. If your mapping contains too many unique values among your key-value pairs, you can try to delete all columns except the one containing key-value pairs in a step, collect the sample, add the mapping step, and then delete the step where all other columns are removed.

Cluster-based samples

Cluster sampling collects contiguous rows in the dataset that correspond to a random selection from the unique values in a column. All rows corresponding to the selected unique values appear in the sample, up to the maximum sample size. This sampling is useful for time-series analysis and advanced aggregations.

Optionally, you can apply an advanced filter to the columnmore information on sample types, see Sample Types.