| D toc |
|---|
| Excerpt |
|---|
You might encounter problems with how data has been structured or formatted that you must fix prior to providing the content to your target system. You can use the methods in this section to locate problems with the content or data typing of your data. |
Locate mismatched values
When
| D s product | ||
|---|---|---|
|
...
In the data quality bar, mismatched values are identified in red:
| Tip |
|---|
Tip: Before you start performing transformations on your data based on mismatched values, you should verify the data type for these columns to ensure that they are correct. The type against which values are checked is displayed to the upper left of the data quality bar. Below, the data type is |
...
| D caption | ||
|---|---|---|
| ||
Mismatched values in red |
| Tip |
|---|
Tip: Before you start performing transformations on your data based on mismatched values, you should check the data type for these columns to ensure that they are correct. For more information, see Supported Data Types. |
| Tip |
|---|
| Tip: From the Transformer page, click the mismatched values in a column's data quality bar to see their count, highlight them in the rows of the data grid, and trigger a set of suggestions for your review. |
Mismatched values can be sourced from a variety of issues:
- Values may be miskeyed into the source system.
- The source system may introduce errors in output, particularly if the data is generated for export using a customized structure.
- Incorrect use of column delimiters may create offsets among fields in individual rows.
- Data may be badly structured across a set of rows.
- The column may be assigned the wrong data type.
| Tip |
|---|
Tip: When cleaning up bad data, you should look to work from bigger problems to smaller problems. If a higher percentage of a column's values have been categorized as mismatched data, it may indicate a wider problem with the data. In affected rows, verify if other columns' values are also mismatched. These rows should be reviewed and fixed first. When fixed, other mismatches may be fixed in other rows, too. |
To locate data:
| Info |
|---|
NOTE: Remember that you are working on a sample of your data. For small datasets, the Initial Data sample includes all rows of the dataset and is unsampled. |
- From the Transformer page, click the mismatched values in a column's data quality bar to see their count, highlight them in the rows of the data grid, and trigger a set of suggestions for your review.
To refine the data grid view, click the Show Only Affected Rows checkbox in the status bar at the bottom of the screen. Only the rows that are affected by the previewed transform are displayed.
Tip Tip: This step highlights specific values that are mismatched. You can take note of individual values.
- To locate a specific value, click the Filters icon on the right side of the screen. In the Rows tab, enter the specific value to locate. Rows containing this value are highlighted. Back in the data grid, you can select one of these highlighted values to be prompted for suggestions.
Methods for fixing mismatched data
...
| Tip |
|---|
Tip: You can also use the |
Bad data typing
| Tip | |
|---|---|
Tip: Particularly for dates, data is often easiest to manage as String data type.
|
...
| Tip |
|---|
Tip: If possible, you should review and refer to an available schema of your dataset, as generated from the source system. If the data has also been mis-typed in the source system, you should fix it there as well, so any future exports from that system show the correct type. |
| D s also | ||||
|---|---|---|---|---|
|
