If you know that you are dropping some rows and columns from your dataset, add these transform transformation steps early in your recipe. This reduction simplifies working with the content through the application and, at execution, speeds the processing of the remaining valid data. Since you may be executing your job multiple times before it is finalized, it should also speed your development process.
- To drop columns:
To delete rows: The following example removes all rows that lack a value for the
D trans p03Value delete matching rows Type step p01Name Condition p01Value Is missing p02Name Column p02Value id p03Name Action SearchTerm Filter rows
- To keep rows: The following example keeps all rows that lack a value in the
D trans p03Value keep matching rows Type step p01Name Condition p01Value Is missing p02Name Column p02Value id p03Name Action SearchTerm Filter rows
- See Filter Data.
Perform joins early
After you have filtered out unneeded rows and columns, join operations should be performed in your recipe.These steps bring together your data into a single consistent dataset. By doing them early in the process, you reduce the chance of having changes to your join keys impacting the results of your join operations. See Join Panel.