Click to learn more about author Kevin W. McCarthy.

If you’ve read any of my previous articles, you know I’m a fan of pop culture. Recently, the pop culture gods must have heard my prayers by bringing back the early 90s paranormal juggernaut known as “The X-Files.” If you never saw “The X-Files” (congratulations on being young or having a life in the 90s), the show revolves around conspiracy theories and strange phenomena, like the existence of aliens, or Bigfoot, or some other mystical creature or power. What was great about The X-Files was the way each week’s episode would tell a story about something supernatural, and frame it in a way where you’d start to believe that this impossible concept may, in fact, be possible.  The main character even had a poster on his wall with the phrase, “I Want To Believe.”

Now “The X-Files” is back on the air, and the thought of urban legends being less fiction and more reality is ready to be accepted once again. In my line of work, Data Quality and Data Management, there’s no more elusive phantom than the concept of “trusted data.” Trusted data is a myth to most people. Like Atlantis or Nirvana, we’ve all heard of it and would love to get there someday, but any right-minded data management professional would say trusted data is just a fantasy.  But is it?…

Trusted data is the concept that users have faith in the information in their systems, and believe it to be accurate in their queries and reports, using this information to make impactful and well-considered decisions to influence the growth of their businesses.

For as long as data and databases have been around, people have lacked trust in their data. While this is sad for many reasons, it can result in some real consequences. Rather than making informed decisions, people use their ‘gut’ purely because they don’t trust the data. The results of this can be disastrous, as you can imagine. In our 2018 global Data Management benchmark report, we found that while 81 percent of organizations say they trust their data to make key business decisions, 69 percent say that inaccurate data continues to undermine customer experience efforts. If their data were truly trusted, it would be accurate and would help to improve, rather than undermine, customer experience.

Believe it or not, trusted data can be achieved. It takes a combination of tools, methodology, and willpower. The tools need to be accessible and collaborative, meaning the creation of trusted data is not just an IT function. It needs input from the business as well, to properly define and standardize data values, and to encourage business rules that put guardrails around the data on an ongoing basis. If it takes a programming degree to define a field as a numeric value with a range from 1 to 100 in your current Data Quality tool, then you’ve got some troubles on your journey to trusted data.

Trusted data is also more than individual business rules and transformation functions—it requires a prescribed methodology. In the same way that you stay healthy by getting enough sleep, eating right, exercising, etc., achieving trusted data requires the same holistic approach. There is a three-step approach to looking at data:

1. Profile Data: You want to start by examining the contents and gathering a baseline for what you have. Then you can define the business rules necessary to validate these data fields in the future. Examples of this include that the data only contains these values, should be greater than or less than some value, should never be null, etc.
2. Standardize Data: By implementing transformations that take any values outside of your defined norms and modifies that information accordingly, you can put your data into common formats (take vp, Vp, V Prez and make them all Vice President). You then can correct your information in basic ways, like with certain common spellings, and enrich it whenever possible with additional context.
3. Combine Records: Matching the data is one of the most important elements of Data Management. It is not just to remove duplication, but to get a complete view of the relationships, be it individuals and households, companies and contacts, or products and SKUs.

The final piece of the puzzle is willpower. Willpower refers to your ability to monitor these processes over time, and make sure you stay diligent in not letting bad data slip through the cracks. It’s easy to do a one-time cleanup of your data (actually, it’s still not that easy), but what is really hard is keeping it fit for purpose week after week, month after month, and year after year. This will take a dedication to Data Quality throughout your organization, but will also reap benefits for every department in the fact that they have now achieved what most thought was not possible: trusted data.

So keep an eye out for that random unicorn or tree nymph, because knowing it’s possible to actually achieve trusted data in your organization means almost anything is possible! I want to believe!