Page tree

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next



NOTE: This list of limitations should not be considered complete.

General Limitations

Data Volume


D s webapp
 applies no fixed limits to the number of columns or rows that can be handled during transformation.


NOTE: During transformation,

D s product
is designed to process data volumes of any size.

However, some important considerations:

Soft row limits

  • The number of rows that you see within the 
    D s webapp
     in the currently selected sample is determined by:
    • Maximum permitted sample size stored on the base storage layer
    • Currently configured sample size for the current recipe

See Sampling


  • Sample sizes are 10 MB.


Soft column limits

  • Soft row limits do not affect the number of columns that are displayed. All available and visible columns are displayed. The number of rows may be affected by the number of columns. 


    Tip: Avoid creating and working with datasets that are wider than 1000 columns. Datasets that are wider than this recommendation may result in performance impacts in the

    D s webapp

  • The number of columns may be limited by:
    • Number of columns permitted in the source datastore. 
    • For SQL-based datastores, limits may be placed on the length of individual queries.


  • All values displayed or generated in the application are based on the currently displayed sample. 
    • Transforms that generate new data may not factor values that are not present in the current sample.
    • When the job is executed, transforms are applied across all rows and values in the source data.
    • Transforms that make changes based on data values, such as header and valuestocols, will still be configured according to sample data at the time of that the step was added, instead at execution time. For example, all of the values detected in the sample are used to determine the columns of a valuestocols transform step based on the selected sample when the step was added.
  • Random samples are derived from up to the first 1 GB of the source file. 
    • Data from later parts of a multi-part file may not be included in the sample.



NOTE: Some functions do not correctly account for multi-byte characters. Multi-byte metadata values may not be consistently managed.

Size Limits

Upload File Size Limits

  • Maximum upload size for a file is 1 GB.

Sample Size Limits

  • For the
    D s photon
    running environment, samples are limited to 10 MB.

Limitations for
D s product





D s product
is a free product that is not intended for production uses. High volume or complex transformations, such as lookups, unions, pivots, or joins may cause the application to crash. If you are encountering persistent issues with this free product, please contact your
D s item

  • Connectivity to external datastores is not supported.
    • Integrations with backend data storage is not supported. All files must be uploaded and downloaded from the application.
    • Connectivity to relational sources is not supported.
  • Results can be written to CSV or JSON format only.
    • Compression cannot be applied to outputs.
  • Total volume of storage is limited. See Data Usage Page.
  • The monthly count of jobs, compute time, and total volume of downloaded data is may be limited. See Data Usage Page.
  • Sharing is not supported.
  • Scheduling is not supported.No access to APIs.
  • Example flows are not available through the
    D s webapp
    . Example datasets may or may not be available, depending on when you first registered for the product.
  • The plans feature is not available.
  • Data quality rules cannot be created or applied to your recipes.
  • Sampling jobs cannot be canceled.
  • You cannot create or use parameters in the
    D s webapp
  • Use of SQL is not supported.
  • Administration features are not available. Your workspace can contain one and only one user.
  • No access to developer tools, including:
    • REST APIs
    • User-defined functions
    • code generation capabilities


Other Limitations

  • File Formats: Limitations may apply to individual file formats. See Supported File Formats.
  • Data Type Conversions: There are some limitations on how data types are converted during import or export/publication. See Type Conversions.