Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version r097

D toc


D s product
allows you to bring together multiple datasets in a single recipe, so that you can transform broader and deeper datasets. This section covers several methods by which additional data can be integrated into your recipe.

Data to Combine

You can combine your current dataset and recipe with any of the following types of datasets:

  • Imported Datasets: You can combine your current dataset with other imported datasets in the current flow or imported datasets to which you have access.
  • Recipes: You can combine your dataset with the output of other recipes in your flow. When those recipes change, those changes are reflected downstream in your recipe. For example, you may wish to perform some elementary cleanup of a dataset before it is used in your recipe. So, you combine your current dataset with the output of the recipe that performs the cleanup on the other dataset.
  • Reference Datasets: For recipes in other datasets, you can create a reference dataset, which is a reference to the output of the recipe that can be used in other flows.

Append Datasets

You may need to add the rows of one dataset to your current dataset. You can use a union operation to bring in the rows of a second dataset into your current dataset. For example, if you have transactional data, such as orders or log events, you may wish to string these rows together in a single dataset.

See Union Similar Datasets.

Join Datasets

You can join together two datasets based on the values in a common column. For example, you may have a weeklyOrders dataset of thats include a customerId column. You might want to join this dataset with your Customers data. You can join in the Customers data based on matches in the customerId column of both datasets. 

A join operation combines datasets based on the values in one or more columns. There are multiple types of join. For more information, see Join Dataset.

Lookup data

You can also perform simple lookups from the column of one dataset into a selected column of another. Where matches are found, the values for all columns in the second dataset's row are added to the first dataset.


Tip: A lookup is basically a left join on a single column.

To perform a lookup, select the column whose values you wish to look up. From its column menu, select Lookup...

Combine Datasets with Parameterization

You can combine individual source files or tables together into a single imported dataset. For example, if you have sets of log files or transactions stored in separate files or tables in the same area, you can use a parameterized dataset to combine these datasets together. 

Combine Datasets with SQL

D s ed

For relational sources, you can combine data from multiple tables into a single dataset using custom SQL statements that you create. At import, these SQL statements are applied to the datastore, returning the customized dataset for use in 

D s product