Top causes of poor data quality and how to fix them - Technology

Multiple Data Systems

Different systems capturing the same data.

Over time your organisation may undertake a range of projects to solve a business issue or service a new business opportunity. It can be quite common for new systems to then appear organically that ultimately capture the same or similar data.

Having duplicate data from different systems can lead to inaccurate reporting and ultimately an incorrect decision if you are not using the most current data. It also often results in inefficiencies if you’re spending a lot of time either checking, consolidating or cleansing data before reporting or analysis.

Multiple Data Channels

The existence of multiple channels through which data is gathered and applied.

Multiple channels including face-to-face, telephone, internet and smart-phone applications can be used when gathering data.

While using multiple data channels may seem like a solid strategy to interact with your customers, problems can arise when these interactions are stored in different data repositories creating multiple sources of truth.

Data from multiple sources need to be integrated so your organisation can avoid storing duplicate data and achieve a single, unified source of truth.

Information Silos

Data is tightly held within different business units.

Information silos restrict the exchange or sharing of data as they are generally held tightly by an individual or team within an organisation.

For instance if a specific silo of information is only known to a specific area within your organisation, other areas may be using and applying incorrect data as they may be using data that is not current, or not using the data at all as they do not know of its existence. This may lead to inaccurate reporting and business decisions.

Poor Data Definitions

No common understanding of data.

For an organisation to be consistent with its data there needs to be a unified understanding of it. Inconsistency usually happens when definitions do not exist or if they are not shared across the organisation.

When there is inconsistency and lack of standardisation or the same data is applied differently, there’s also a higher likelihood of inconsistent reporting.

How many times have you heard of or seen multiple answers to the same questions because different people have used a different interpretation of a business term?

Old Systems

Older systems tend to be outdated and typically do not offer the amount of data validation and flexibility that newer systems do.

The issue with these legacy systems is they can lack the required workflows, structure and data validation that more modern systems have.

For example older systems may not be able to mandate certain business processes. They may also not be able to provide contemporary data validation or input options such as dropdowns and rely on free text fields. All of these can lead to invalid data, incorrect and missing data.

Rationalising Systems

Eliminating the use of data systems such as private databases and spreadsheets can be the first step in avoiding duplication and information silos.

Better Data Integration Between Systems

This allows data to be shared programmatically between systems. It enables increased sharing and better quality by allowing systems to have the most up to date data.

Applying A Master Source Of Truth For The Data

There are two options you can consider if you have multiple sources of the same type of data. If possible you could try to rationalise the systems so that there is only one. However that may be a hard task to accomplish.

An alternative is to identify the master source of the information and put in place a form of master data management. You can do this through a specific master data management solution or through data integration. Regardless of the method you choose, it’s important that everyone within an organisation recognises which is the master source of truth so they can confidently rely on that information.

Improving Data Validation Within Systems

Data validation is important if you want to ensure your data quality remains high. The first step you can take in this process is to add on-screen field validation so a user cannot enter incomplete or inaccurate data.

The simplest examples include address or email validation and phone number formats. Whether it’s incorrect data, fraudulent data or missing data, you have to work out what type of poor data you need to address and apply solutions based on that.

Defining And Applying Common Definitions

Standardising your common data definitions is an extremely important step towards avoiding poor quality data. When reporting on specific measures, make sure they are defined and agreed on by the organisation.

For example one definition that a lot of organisations struggle with is ‘what is a customer or client’. For any definition, make sure it is recognised and followed by the entire organisation and definitions are shared and accessible to everyone.

What’s Next?

Make sure to come back for the next article in this series where we will discuss FIVE MORE causes of poor quality data and how to address them.

Data Agility’s expertise in data analytics and information management ensures that organisations have confidence that their data is of the highest quality. If you want reliable data that can be used to make better business decisions, contact us today.