People are using computers, smart phones, and laptops with increasing regularity. As a result, businesses are collecting and storing huge amounts of data, or Big Data. Acquiring insights from all this information is a challenge, but is also key to achieving a competitive advantage. Analytics solutions that find unstructured and structured data can help tremendously in gaining insights. This data can come from privately acquired sources, but can also come from public data available on the Internet. The ability to cross-reference private information about consumer preferences, and information from blogs, tweets, and data from social networks, can allow organizations to understand the needs of their customers, and to some extent, predict their future purchases.

At the DATAVERSITY® Enterprise Data World 2016 Conference, Kenneth Viciana hosted a panel discussing the future of Big Data. Viciana currently works for Equifax, leading a team responsible for their Enterprise Data Strategy. He tries to transform Big Data into insights his company can monetize. Viciana mentioned the goal for business is to predict what their customers want in the future. The panel included Robert J. Abate, an expert consultant for Intellisys, Inc., Kelle O’Neal, Founder & CEO of First San Francisco Partners, and Joe Caserta, CEO of Caserta Concepts. After introductions, the panel discussed the following topics:

• The Current State of Big Data Efforts
• Success Areas/Challenging Areas
• Predictions on the future of Big Data

The Current State of Big Data Efforts

During the discussion, Robert Abate said the use of Big Data is in a discovery type mode and is still in an emerging phase. Currently, using Big Data involves sifting through vast amounts of information, while searching for nuggets of gold. Abate described how Visualization Models have been steadily improving and explained the time value curve of information. (The time value curve states information has its highest value on the day it is created, and steadily loses value with the passage of time.)

Kelle O’ Neal stated, “We are learning more from our failures right now, then we are from our successes. It is important to realize we are learning from our failures, and that’s okay.” She said this because the field of Big Data is so new. O’ Neal is supporting a philosophy of experimentation that neither punishes nor shames for unexpected results. The field of Big Data is so new, any information or surprise result can be of value. At present, the “culture” of Data Scientists working on Big Data is quite small and, comparatively speaking, somewhat inexperienced as they explore and learn about this new way of thinking, this new paradigm.

Joe Caserta suggested startup companies have an easier time using Big Data. They haven’t the baggage of habits and customs that develop over time in the workplace. A start-up begins with a clean slate. An established company would have its people trained to work and respond in certain ways that have proven successful in the past. Caserta also said companies are starting to think in terms of “Data Ecosystems,” which includes Data Lakes (large amounts of raw data stored in its original format).

Success Areas/Challenging Areas

Abate stated Predictive Analytics would become popular for predicting how products will sell. He then went on to describe how they had been asked to predict how well a new Canon camera would sell. Tracking “the buzz” from the Internet, they discovered sales tracked exactly with the buzz, allowing them to predict the pattern of sales.

He also talked about the effects of weather and traffic, citing Big Data research done for Walmart which showed an assumption to be untrue. Walmart’s had made an assumption there was a profit from bumps in business just prior to a storm and just after. While this was true, it had not been understood there was a loss of business during the time of the storm that counteracted the bumps, creating an overall loss of business for that time period.

O’Neal used the 1984 “Where’s the beef?” television commercial, starring an elderly woman named Clara Peller, as an analogy of business owners wondering, “Where is what I just paid for?” O’ Neal is referring to the inexperience of Data Scientists as they attempt to find patterns in Big Data that can be profitable to their clients. She also observed organizations will often mistakenly treat Big Data and Data Warehouses as the same thing. (A Data Warehouse is a large amount of data gathered from a broad range of resources “within a company” that gets used as a guide for management decisions, while Big Data can be described as “very large” amounts of data which is analyzed to find associations, patterns, and trends, with a focus on human interactions and behavior).

Caserta brought up the need for paradigm shifts, noting some people have developed habits and ways of thinking that do not support Big Data research. The ways technology are now being used for data collection, and even the way data “gets” used, is all changing. Resistance to change, to the immersion and adoption of new methods, is a major problem for businesses. Caserta provided Lewin’s Force Field Analysis, during the discussion, as an example of the problems in promoting change.

A Sample of Lewin’s Force Field Analysis

Forces for Change                                                        Forces  Resisting Change

Global economics                                                       Period of time in present role
Intensity of competition                                            Status & perks of office under threat
Reduce Costs                                                               No apparent reason for proposed changes
New executive leadership                                          Lack of understanding of proposed changes
Speed of Technical Change                                        Fear of inability to cope with new role
Social Trends and changes                                         Concern over job security
Move to cross-functional teams

Predictions on the Future of Big Data

Joe Caserta predicts machines will recommend certain courses of action, and for more simple problems, the machine will act to solve the problem. He went on to say some of the more visionary companies are doing this today, and eventually all companies will do this. Caserta said Spark is becoming increasingly popular. (Apache Spark is an open source processing engine for Big Data, designed for speed and easy use.) He also predicted most businesses will soon be using the Cloud as a resource, rather than the clumsier, more expensive process of setting up and using their own servers.

Kelle O’Neal predicts Big Data will become more and more useful. She also brought up the issues of digital ethics and privacy. She stated, “we’ll need to learn to respect the owned data of individuals and other businesses. We’ll need to create the capability to allow individuals to maintain a degree of privacy.” She also spoke of the need to protect the client’s privacy. She asks the question, “Whose data is that?”

Robert Abate listed a number of predictions he believes will take place:

• Business decisions will be driven by Big Data.
• Immersive Information Visualization (the use multiple screens to use information) will become popular.
• Machine Learning will be used to predict product sales.
• Currently, Big Data algorithms are simplistic, and in the future will become more refined and complicated.
• The title CIO will come to mean Chief Infrastructure Officer.
• The Chief Data Officer will be as important as the Chief Financial Officer, because information will develop the same value as money.
• There will be an increase in data regulation, as privacy becomes more of a concern.
• An exploration and innovation field will evolve as a result of Big Data.
• Business data will come, more and more, from the Internet of Interconnected Data between businesses.

Register for the Enterprise Data World 2017 in Atlanta, Georgia Conference Today

Here is the video of the Enterprise Data World 2016 Presentation: