|D s webappd-toc|
After you have created the flow and the datasets within the flow and before applying recipe steps to change the data, create a duplicate of the flow. This becomes a snapshot of your original dataset. Since the imported datasets are not affected, the storage overhead for creating backups is relatively low. See Flow View Page.
Track Source Row Information
You can mark the original row numbers of your source data. In the first step in your recipe after initial parsing, add the following:
derive value:SOURCEROWNUMBER() as:'sourceRowNumber
NOTE: Source row information can become invalid if you perform multi-dataset operations such as lookups, unions, and joins. For more precise tracking of source information, you should consider creating multi-column keys, including the source row number information. For more information, see Generate Primary Keys.
Track Steps Affecting a Column
To see all of the steps in your current recipe that reference a specific column, select Show related steps... from the column menu.
All steps are highlighted in the Recipe panel.
NOTE: If another column is dependent on the selected column, all steps pertaining to that column are highlighted as well.
For more information, see Column Menus.
Track Column Value Changes
|D s product|
After you have completed your general setup steps of your transform, create a copy of the original column:
D code showNote true
derive value:String as:'String_orig'
- You now have a copy of the original column before any manipulations were applied to it.
Add any transforms to your recipe, including any that change the values of
String. In the example below, the following transform has been applied:
set col:String value:TRIM(String)
At the point in your recipe where you would like to test the column for changes, insert the following:
derive value:(String != String_orig) as:'String_changes'
String_changescolumn now contains
truevalues where the values in
Stringhave been changed from their original values (
To see just the values that are different, sort in descending order.
Tip: You can reposition this test anywhere in your recipe after you have created the
String_origcolumn. Edit the recipe step, copy all of the content in the step, and then delete the step. Insert a new step elsewhere in your recipe and paste the command back in. If you've renamed the column somewhere along the way, you must remove the original step and apply it again, too.
- Before you run your recipe, you may want to remove the tracking columns that you generated (
String_changesin our example).
Example tracking column changes
Track Row Changes
- Create a copy of the flow. In its name, identify that it is your original. See Flow View Page.
- In the other flow, create your recipes as normal.
- When done, you can add the following steps:
- Union the two datasets together.
- Sort them by a key column.
NOTE: This method may not work if your recipe includes joins or added or removed columns.
If the rows are exact duplicates, they are removed. The remaining rows contain data that has been changed.