by Angela Guess
Roy Wilds recently wrote in Information Management, “You think you know what’s in your data. But there may be a lot more there than you realize. The combination of big data and modern data science can empower you to ask questions in entirely new ways, and uncover answers locked away in your data to questions you hadn’t thought to ask. But how do you get at those insights? The answer has two parts. First, you need to move beyond traditional relational or Excel-based analytics and embrace the power of big data systems to wring useful information from hard-to-analyze sources. Traditional analytics are great for gleaning insights from data that can be represented in simple data models. But if you want to work with complex or “messier” data (like text, audio files and social media), or huge data sources (like genomic data, clinical images and decades-long studies), you’re going to want to harness the power of big data systems and distributed computing.”
Wilds goes on, “Next, you’ll need to use data science to wrangle your data to get more meaning out of it. When you do, you can start to connect the dots in surprising ways. You can move from descriptive analytics to predictive and prescriptive data models that transform your organization. We work with many customers who are new to big data. They don’t want to invest the time to become Hadoop experts, or learn all about Impala and Pig. But one thing they completely understand is how much information is locked up in text. Whether it’s transcripts of customer service calls, physician notes, comments on Twitter and Facebook, there are vast quantities of relevant data in documents. How do you do research on that information? How do you mine it to analyze for similarities? In many cases, you can’t, because you have to open those documents up one at a time. Big data with distributed programming frameworks like Apache Spark can help transform that mass of text (as well as all sorts of other complex data, like sensors and clinical images) into structures compatible with analytics.”
Photo credit: Flickr