Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

When working with datasets with parameters, it may be useful to do the following on a periodic basis. following if you expect the underlying datasets to be less than 100% consistent with each other.

  1. Recreate the dataset with parameters, except deselect the Detect Structure option during the import step. 
  2. In the Transformer page, collect a Random Sample using a Full Scan. This step attempts to gather data from multiple individual files, which may illuminate problems across the data.

    Tip

    Tip: If you suspect that there is a problem with a specific file or rows of data (e.g. from a specific date), you can create a static dataset from the file in question.

...