Page tree

 

Support | BlogContact Us | 844.332.2821

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

...

  1. Use your CSV file as the source for a new dataset within the flow containing the associated dataset.
  2. In your wrangled dataset, make sure that the first line of data is treated as the header. If not, add a header transform to your recipe.
  3. Open the other (source) dataset in the Transformer page. 
  4. From the Tools menu in the Transform Editor, select Union. See Transformer Page

  5. Create a union: 
    1. Include all columns from both datasets.
    2. Configure the step to perform the union by name, instead of by position.
    3. See Union Page.
  6. Add this step to your recipe. 
  7. You should see one row in the union recipe that contains the new data. 
  8. Sort your

    Sort your data by a key value (e.g. business_id).

  9. Determine an appropriate grouping parameter. This step is necessary to simplify the filling process when the job runs at scale. Ideally, you should choose a grouping column that contains a relative few number of values in it (e.g. region).

  10. Fill values in the data rows with metadata column values. For each metadata column, add the following command, done here for the source_system column of metadata. 

    D code

    window value: FILL(source_system, 1) order: business_id group:region

  11. Repeat the above step for each metadata column you want to insert.

    Tip

    Tip: Copy the first successful step with the FILL function. Paste it and modify it for each successive column.

  12. Drop the source metadata columns.
  13. Rename the window columns to use a more appropriate name.
  14. Delete the row containing the original metadata values.

...