Click to learn more about author Kevin W. McCarthy.

Anyone who has worked in Data Management knows the struggles that IT endures. They typically have limited resources around people and infrastructure. They have protocols and regulations that they must adhere to when handling data that is constantly changing and evolving (hello, GDPR). And the backlog of requests from the business for queries, extracts, reports, and the like is a never-ending workload that can’t be completed quickly enough to keep pace with the instant-gratification world we live in. On top of that, if the data delivered to those business users is of poor quality, it’s just another crisis waiting to happen. It’s a tough, underappreciated world for the typical IT warrior.

But occasionally, help is on the horizon. One example of this help can come in the form of Data Quality Tools. While these tools are fairly common and widely adopted today, most businesses are not getting the value out of them they hoped or are not leveraging them properly. Unfortunately, the lack of appropriate usage leaves the business questioning the ultimate value of these tools. The hesitancy with IT tends to be a little deeper and controversial—are they trying to say I don’t keep an accurate database, or worse, that they can automate some (or all) of the work that I do every day? I’ve heard many of these questions before, and it’s interesting how much effort is spent by Data Quality vendors in justifying the ROI of these tools, but much less time is spent easing the minds of those in IT that these tools should be viewed as a blessing and not a curse.

First of all, IT needs to face a brutal fact: the quality of their data needs improvement. This is true for every DBA maintaining every database out there. In fact, according to Experian’s 2018 Global Data Management Benchmark Report, U.S. organizations believe a third (33%) of their customer and prospect data is inaccurate in some way. This is not a reflection of the IT department, but more a reflection of how quickly data is moving, changing, and being added or deleted from our organizations today. Without Data Quality Tools in place, it is virtually impossible to keep the pervasiveness of data discrepancies at bay. As a long-time Data Quality purveyor, I remember multiple prospect visits where IT would jump into the conversation stating that they don’t have any Data Quality issues. It took an embarrassingly short amount of time with a Data Profiling Tool to prove that statement incorrect in every case (much shorter than an analyst could run a bunch of SQL queries)! Though uncovering Data Quality issues ultimately plays to the benefit of the organization, unfortunately, it does not endear the use of Data Quality Tools with the IT department.

Secondly, Data Quality Tools are not a threat. They are not going to expose that IT is doing a bad job, since as stated before, everyone has bad data. They also are not a threat in taking over the jobs of IT, as between backlogged requests and more complex issues to tackle, they’ve still got plenty of work to do. What Data Quality Tools can do is help IT focus on the areas that need their expertise and experience, and leave the mundane tasks to automation. Some examples of this include address verification, matching software to bring together duplicate records, checking on standardization, etc. These are good examples of using the right tools and the right resources to focus in on the right efforts.

In my mind, the one legitimate concern for IT when it comes to Data Quality Tools is to ensure they pick the right tool. One common mistake is to purchase an IT-only set of tools. There are many Enterprise Data Management platforms that offer Data Quality Tools, but many of these are extremely complex and difficult to use. Ideally, IT wants to partner with the business in setting up a Data Quality practice within an organization.  If possible, you should let those with business knowledge, but limited technical expertise, use the Data Quality Tools themselves to manage the rules that drive the context and validity of the data they want to use in their critical decision making.

No one likes to admit they need help, but when it comes to Data Quality, IT needs to accept that they can’t do it alone. By using Data Quality software, they can become more valuable, in demonstrating the improvements that can be gained through higher-quality and more reliable data. Automated Data Quality processes will also free them up to spend their time on more critical items that deserve their attention, such as Big Data, IoT, Advanced Analytics, Kirk vs Picard, and so on. Once IT recognizes how much Data Quality Tools can bring to the table, they will come to love these tools and how much more they can accomplish.