Click to learn more about author Farah Kim.

In a particularly intense weekly meeting, team leads from sales, marketing, IT, and finance were debating on the poor quality of data captured in the company’s new CRM system. Almost 40% of the data captured was defective – fake email addresses, missing information, invalid addresses, and so on. The data was important to a new business initiative, but the fact that it didn’t turn out well made everyone lose tempers. Fingers were pointed, processes were scrutinized, and business and IT leads were at lock horns. During all this, several key questions came to light:

• Who owns the data?
• Who is responsible for ensuring data is accurate before it is used?
• Who ensures Data Quality?
• Is Data Quality IT’s responsibility?

The answer to all these questions was quite evident: data and Data Quality is EVERYONE’s responsibility. The company owns the data. The teams working with data are responsible for ensuring their quality. But therein lies the problem.

The realization that data is everyone’s responsibility was not pleasant to this team. Although they all knew the answer, it was evident that accepting it would mean initiating a series of changes that the firm wasn’t prepared for. There would be an undoing of old processes, creating new processes, getting executive buy-ins, building new plans, adopting new methodologies, and removing the discord between IT and business users.

It was better for the teams to dust their hands off in the situation than to accept that they had to bring about a change. The result? Increasing conflict. Lack of operational efficiency and collaboration. Poor workplace culture. Poor customer retention. Poor ROI.

Companies, like the one discussed here, would rather silo data away and treat it as an IT process than consider it as a fundamental business component. To avoid having to deal with messy data themselves, business leaders delegate authority for managing Data Quality to the IT department. The IT department, in turn, gets frustrated when they are tasked with the burden of having to ensure each department gets data according to their business needs. IT teams spend their days performing mundane ETL (extract, transform, load) tasks instead of working on data security or Data Management. 

Is there any way to overcome this madness? Yes.

Is it easy? No.

But you still need to do it.

The reason?

Data does not impact IT infrastructure. Data impacts the business foundation itself. Poor data can lead to penalties, loss of reputation, loss of business credibility, loss of customers, loss of employees, and much more. The risks are too many, and those risks are not the burden of IT to bear alone.

Let’s understand this problem at a deeper level, but before that, here’s a quick summary of what we mean by data quality.

What is Data Quality and How Does it Impact Businesses?

Data Quality refers to the degree to which your data is:

• Accurate
• Consistent
• Complete
• Valid
• Timely
• Accessible
• Compliant

In simpler terms, if your data suffers from:

• Flawed and defected information (typos, spelling mistakes, character mistakes)
• Mismatched or irrelevant data
• Inconsistencies (phone numbers with or without dashes in between)
• Incomplete information (missing surnames, missing first names)
• Invalid information (emails, phone numbers, or physical addresses that are not valid)
• Outdated information (records that have not been updated for a while)
• Duplicated information (same records with differing or exact kind of information)
• Silos (disparate data sources where information is stored in different databases)

You have a data quality crisis requiring everyone in the business to play their part and treat data for what it is – the lifeline of an organization.

Moreover, you will need a Data Quality framework in place to control your data from becoming messy and unusable. The major functions of this framework would be:

• Data Cleaning: Making sure your data is free from typos, character errors, numeric or alphabetical errors, and so on.
• Data Matching: Combining data from multiple sources to get a consolidated customer view.
• Data Standardization: Ensuring that your data follow defined standards across the board. Minor issues like small casing for names, dashes between phone numbers, etc. can cause significant problems when this data needs to be used for marketing or branding purposes.

The fact that data quality is such an overwhelming problem makes it an organizational issue that all key stakeholders must work together to resolve.

Why Data Quality is NOT an IT Problem

Simply because of its involvement in the process. IT is directly involved in data management (user authorization, the process set up, etc.), data migration, and data storage (warehouses and databases, etc.). It is not directly involved at the point data was created and neither at the point when data is used by a business user for a business case.

Thomas. C. Redman explains this point well. Here’s an excerpt from his article in the Harvard Business Review:

“The two most important moments in a piece of data’s lifetime are the moment it is created and the moment it is used. Most of these moments don’t occur in IT. They occur in the trenches, when a salesperson signs up a new customer; in middle management, as a group struggles to understand and improve market share; in the analytics group, when a data scientist is seeking a new discovery in big data; and in an executive’s office, as she works the numbers to decide whether now is the time to add staff. The really interesting and important moments for data occur in the business, not in IT.”

Truer words have not been said.

Ok, so Who Should Be Responsible for Data Quality?

I reiterate – everyone.

The reason is quite simple.

Poor data impacts your business. Those customers who received the wrong emails are going to be furious. The deliveries that go out only to return because of an invalid address is a cost to your business. The important list of leads with invalid email addresses has caused you to lose significant ROI.

And to be honest, these mistakes sound basic and silly, but it’s a fact that companies overestimate the quality of their data and are under the impression that they have excellent data. In reality, it’s not uncommon for a company to have data so bad and messy, they end up making mistakes costing billions of dollars. A report shown below shows this well.

The irony is, most of these mistakes are performed by business users or consumers themselves rather than the IT team. For example, the making of a web form to collect information is a marketing tool, designed to meet the marketing teams’ data requirements. At this data creation point, the marketing team should ensure that their web forms are designed to prevent messy data from being entered. Anyone entering phone numbers must enter the complete country + city code, or the form cannot be submitted. Anyone entering an address without a proper ZIP code should be prompted. While this doesn’t ensure 100% clean data, it will still help the company in reducing the percentage of defective data.

Next up comes the challenge of duplicated data. If marketing, sales, and customer service all store the same customer record in their respective data silos, the firm can never match their data sets and get a consolidated overview of its customers. Again, this is not an IT problem, but a business problem where an outdated practice by core teams leads to messy, duplicated, and unusable data.

This is not to say that IT is not involved. Of course, the smooth operation of a data workflow is dependent on IT, but to assume that data quality is purely an IT function is to create operational bottlenecks.

How Do You Get Everyone to Be Responsible for Data Quality?

Well, it starts at the top.

A Data Quality issue must be recognized and acknowledged. Teams must get out of denial.

The CEO needs to initiate the Data Quality project, which will be overseen by the VP or the heads of involved departments and finally managed by a Chief Data Quality Manager.

This sounds easy but is also challenging, especially since it will require you to develop a strong business case highlighting the current problems with the company’s data and why a Data Quality initiative is the solution.

You will have to assess where the damage has been done the most and look at what tools or processes can be introduced to improve Data Quality.

You will need to implement a holistic Data Quality program that takes into account your company’s current problem with quality, your challenges, and the steps you need to take to overcome them.

Remember, this program is not something that can be performed on a whim by hiring developers or more IT staff. It will require in-depth planning from key stakeholders in the organization. It will need executive buy-ins on budget, on purchasing of tools, and hiring of data specialists. It will also require you to work with third-party consultants and Data Quality solutions providers. The point is not to have 100% flawless data, but to ensure that you have data that you can trust – data that will help your teams overcome conflicts, collaborate productively, and together, eliminate obstacles that stand in the way of your business’s success.