Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from space DEV and version next

D toc

Excerpt

While transforming your data, you may need to change the data type of one or more columns.

For example, data of String type may be the easiest to manipulate. Since there are no mismatched values for String data type, you may wish to change a column's data type to this baseline type. 

  • Data types that you see in the Transformer page represent types that are understood by the product. 
  • When data is imported from a separate datastore, 
    D s product
     may apply internal data types to the data. These types may differ from the original data typing in the source. As needed, the inferring of data types can be disabled at the file, connection, or global level. For more information, see Disable Type Inference.
  • When data is published from the product to a separate datastore, these types may be mapped to different data types in the target. For more information, see Type Conversions.
Tip

Tip: You can use the Change Column Type transformation to override the data type inferred for a column. However, if a new transformation step is added, the column data type is re-inferred, which may override your specific typing. You should consider applying Change Column Type transformations as late as possible in your recipes.

For more information on the available data types, see Supported Data Types.

...

You can change a column's data type in one of the following ways:

Change from

...

column menus

You can change the data type for individual columns through the following column menus:

...

Tip

Tip: Both of the above methods become individual steps in your recipe.

Change

...

through Transform Builder 

You can change data type for multiple columns to a single data type, you a single column or multiple columns through the Transform Builder. You can use a transformation like the following, which changes the columns LastNameFirstName, and Address to String data type.

D trans
p03ValueAddress
Typeref
p01NameColumn 1
p01ValueLastName
p02NameColumn 2
p02ValueFirstName
p03NameColumn 3
p04ValueString
p04NameNew Type
SearchTermChange column type

Info

NOTE: You can lock the data type for columns to prevent it from being updated when the data is transformed in subsequent steps.

Info

NOTE: When specifying a data type by name, you must use the internal value for the data type. The value in the column menu is the display name for the type.

For more information, see Valid Data Type Strings.

...

Lock Data Type

You can lock a column's data type through the Transform Builder. When a column's data type is locked, the data type is no longer automatically checked and updated by the 

D s webapp

Tip

Tip: If you do not wish to have the data types modified, you can add a transformation to lock all of them in a single step. Details are below.

Via Transform Builder

  1. In the Search panel, enter lock column type
  2. From the Columns drop-down, select any one of the following options:
    1. MultipleSelect one or more columns from the drop-down list.
    2. RangeSpecify a start column and ending column. All columns inclusive are selected.
    3. All: Select all columns in the dataset.

      Info

      NOTE: This option locks all the column's data type.

    4. AdvancedSpecify the columns using a comma-separated list. You can combine multiple and range options under Advanced. Example:

      Code Block
      c1,c3,c5~c8
  3. Specify the other parameters.
  4. To add the step to your recipe, click Add

Example - lock a column's data type

This transformation locks the column data type:

D trans
Typeref
WrangleTextsettype col: myDate datetimeType: 'Datetime','mm-dd-yy hh:mm:ss','month*dd*yyyy*hh:MMaX' type: Datetime
p01NameColumns
p01ValueMultiple
p02NameColumn 1
p02ValueStore_Nbr, Whse_Name
p03NameNew Type
SearchTermlock column to current type

Example - lock the data types for all columns

This transformation locks the data types for all columns:

Tip

Tip: Many transformations support the Advanced option for column selection. You can specify column ranges, including all columns using the asterisk (*) wildcard. See the following.

D trans
Typeref
p01NameColumns
p01ValueAdvanced
p02NameColumn 1
p02Value*
p03NameNew Type
SearchTermlock column to current type

Unlock Data Type

You can unlock a column's data type by following any one of these methods:

Via Transform Builder

In the Transformer Builder, you can select unlock to the current type option to apply the unlock feature to one or more columns.

This transformation unlocks the column data type:

D trans
Typeref
WrangleTextsettype col: myDate datetimeType: 'Datetime','mm-dd-yy hh:mm:ss','month*dd*yyyy*hh:MMaX' type: Datetime
p01NameColumns
p01ValueMultiple
p02NameColumn 1
p02ValueStore_Nbr, Whse_Name
p03NameNew Type
SearchTermunlock column to current type

Via column menus

You can unlock the data type for individual columns through the following column menus:

  • To the left of the column name, you can click the icon and select Automatically update. The selected column is unlocked. 

Change Datetime Data Type

If you are changing a column's data type to Datetime, you must also select a format string to apply to the column. In

Via column menus

You can apply a Datetime data type through the column menus, the available options can be selected after you select Datetime. For more information, see Datetime Data Type.. When you choose the Datetime data type, you must apply a format for your Datetime values. For more information, see Choose Datetime Format Dialog.

Via Transform Builder

In the Transformer Builder, you can apply a specific transformation to format one or more columns to Datetime data type, using a specific format.

Tip

Tip: You can use the following transformation to change the format of a Datetime column.

This transformation looks like the following:

D trans
p03ValueDate/Time
Typeref
WrangleTextsettype col: myDate datetimeType: 'Datetime','mm-dd-yy hh:mm:ss','month*dd*yyyy*hh:MMaX' type: Datetime
p01NameColumns
p01ValueMultiple
p02NameColumn 1
p02ValuemyDate
p03NameNew Type
p04Valuemonth*dd*yyyy*hh:MMaX
p04NameDate/time Type
SearchTermChange column type

D s also
inCQLtrue
label(label = "data_type")