Improvements to the Type System

This section provides information on improvements to the Alteryx® type system.

Release 8.2

None.

Release 8.0

Data type inference and row split inference run on more data

When an dataset is imported into the Designer Cloud application, a larger volume of data is read from it for the following processes:

• Split row inference: Patterns in the data are used to determine the end of a row of data. When a larger volume of data is read, there should be more potential rows to review, resulting in better precision on where to split the data into separate rows in the application.
• Type inference: Patterns of data in the same column are used to determine the best Alteryx data type to assign to the imported dataset. A larger volume of data means that the application has more values for the same column from which to infer the appropriate data type.

If you notice unexpected changes in column data types or in row splits in your datasets:

1. Type inference: You should move your recipe panel cursor to the top of the dataset to see if you must reassign data types.
2. Split row inference: Create a new imported dataset, disabling type inference in the import settings. Check the splitrows transform to see if it is splitting the rows appropriately. For more information, see Import Data Page.

Release 7.5

PII - Improved matching for social security numbers

In prior releases, Personally Identifiable Information (PII) for social security numbers was identified based only on the length of values, which matched too broadly.

In this release, the constraints on matching of SSN values has been tightened when applied to PII. 

For more information, see Social Security Number Data Type.

For more information, see Data Type Validation Patterns.

PII - Improved and expanded matching for credit card numbers

In prior releases, PII for credit card numbers was identified base on 16-digit values. 

In this release, the matching constraints have been expanded to include 14-digit credit card values. 

Also, the constraints around valid 16-digit numbers have been improved with better recognition around values for different types of credit cards. In the following table, you can see lists of valid test numbers for different credit card types and can see how detection of these values has changed between releases.

For more information, see Credit Card Data Type.

For more information, see Data Type Validation Patterns.

Improved type inference for relational sources

In prior releases, when you generated outputs, the typecasting for the outputs was determined by the data types that were inferred by the application. So, if a column contained only "Yes" or "No" values, then the application is likely to have inferred the column data type as Boolean.

The above presented problems for relational sources for the following reasons:

1. Relational sources could include schema information, which should override what the application inferred. Type inferencing can optionally be disabled for schematized sources from with in the application. For more information on disabling type inference for a schematized source, see Import Data Page. 
2. For some relational sources, datasets are ingested into the backend datastore and stored there as CSV files. These CSV files are then used as the source for the imported datasets. In these cases, the original source schema information was lost, which meant that the application's type inference was applied. This applied to the following data sources:
   1. Snowflake
   2. Redshift
   3. SQL Datawarehouse
   4. Alation
   5. Waterline

Beginning in this release, the schemas from relational datasources that are ingested to the backend datastore are now used for generated outputs, unless the type was being forcibly set to something else during the recipe step. At the time of original import, the schema of the relational datasource is stored as part of the ingest process; this schema is stored separately. 

During publication, Designer Cloud maps its internal data types to the data types of the publishing target using an internal mapping per vendor. For more information, see Type Conversions.