Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

...

Tip

Tip: You can also use this technique to generate an out result set output of your source data, which is useful if you do not have read access to the source outside of

D s product
.

 

Steps:

  1. Create an imported dataset from your source. Add it to a flow. See Import Data Page.
    1. Depending on how raw you wish to review your data, you may choose to disable Detect Structure. For more information, see Initial Parsing Steps.
  2. In Flow view, create a recipe for your imported dataset. See Flow View Page.
  3. In Flow view, edit the newly created recipe. It is opened in the Transformer page. See Transformer Page.
  4. If needed, add a header step to your dataset. 
  5. Click Run Job.
  6. In the Run Job page, select the following options:
    1. Choose the default running environment.

    2. CSV format (you need at least one format to generate your dataset's profile).
    3. Select to profile results.
  7. Click Run Job.
  8. When the results are generated, click View Results.

  9. A profile of your dataset is displayed.

...

  1. In Flow View, select the recipe that was used to create the source profile. 
  2. Rename this recipe to something like, SourceData.
  3. Create an output off of this recipe. Click Run the job if you have not yet created the visual profile.  
  4. Select the recipe again. Now, click Add New Recipe.
  5. Edit this new recipe and build out your transformation steps. 
  6. Whenever you need to regenerate the profile for the source, select the SourceData recipe and select the output from it. Then, run a job off of for it. 

    Tip

    Tip: This technique is useful if you are replacing the source dataset with refreshed data on a periodic basis.

See Flow View Page.