Click to learn more about authors Oksana Sokolovsky and Rohit Mahajan.
A few years ago, if I told you about “smart data discovery,” you may have thought I was just discussing the latest technology trend. Today, however, you’ve probably heard plenty about it. The technology continues to evolve and is proving to solve many customer use cases. In fact, it is increasingly being used to identify critical data relationships that data professionals can use to determine business outcomes.
CIOs and CDOs are in the middle of this consistently evolving, and accelerating, pace of business, driven by information technology. It’s what keeps them up at night, because they’re constantly faced with the challenge of getting the most value from the massive amounts of information they’re charged with managing. Business depends on the effective use of data, and if you are not able to quantify what you have, as well as where you have it, you run the risk of falling behind.
This is especially true when you consider the growing prevalence of global data privacy regulations. The European Union’s General Data Protection Regulation (GDPR) was merely the tip of the iceberg; in the U.S., the countdown has already begun toward the implementation of California’s Consumer Privacy Act (CCPA), with more than a dozen other states considering new laws of their own dealing with the kind of information companies may keep about their customers and how they are required to handle it.
In this ever-changing regulatory environment, manually tracking data simply is no longer good enough. Consider that most organizations today could be processing billions of transactions on a daily basis: How can you realistically expect data at this scale to be managed manually, predominately through human effort? This is why smart data discovery, which leverages artificial intelligence, is the foundation to help organizations identify relationships within their data and use that as a basis to provide insights that contribute to improved decision making and enabling regulatory compliance with the aforementioned data privacy laws.
Smart data discovery that leverages artificial intelligence (AI) is increasingly a must-have for many companies, but it now has the potential to go much further than that. So with that as a given, the ability to gain and maintain control over Personally Identifiable Information (PII) and sensitive data as it streams into firms’ data lakes and centers is key, regardless of whether it already resides “at rest” inside an existing data lake, or whether it’s new, “in-motion” PII being introduced at any particular moment. It’s not only important for compliance, but we’re seeing signs that it enables companies to take action and transact proactively with their customers in a more efficient manner.
Smart data discovery, for example, could be important for firms where data scientists are charged with making sense of the data. More and more we are seeing the data preparation tasks, which normally dominate much of the data scientists’ time, being freed up by automation through the use of technology like smart data discovery. This leaves data scientists free to focus on higher-value tasks. According to some estimates, 40 percent of all Data Science tasks will be automated by next year. And the people doing the work may not be trained experts; the growth of “citizen data scientists” has been cited by several industry analyst firms. The benefit, according to DATAVERSITY® blogger Kartik Patel, is that while a citizen data scientist may “not have the skills or training to be an analyst or a programmer, with the right tools, they are capable of generating reports, analyzing data, and sharing data to make decisions.”
A key takeaway: Businesses that deploy smart data discovery across all of their information – all of their sites – will be in a far better position to gain the insights they need to stay ahead of their competition.