Many data scientists and analytics consultants complain they were hired to be a part of advanced analytics teams at corporations, but all they do is data analysis or reporting work. The bigger problem here is that the personnel leading these teams genuinely believe their problem statements are advanced in nature. And rightly so – as accessing and engineering data and reporting high-quality insights are in no way easy tasks.
Moreover, “advanced” is a relative word. And for corporations, a lot of time advancement means advancing from their current maturity level. I have seen companies set up advanced analytics practices, specifically to help them graduate to a more advanced state. This can be as simple as moving from Excel to a business intelligence tool like Tableau or PowerBI or automating manual activities leveraging VBA or RPA. The disconnect between business leaders and data science professionals leads to job dissatisfaction and, ultimately, attrition.
This begs the question: What is advanced analytics?
Five Types of Advanced Analytics Problems
Gartner defines advanced analytics as “autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence (BI), to discover deeper insights, make predictions, or generate recommendations.”
At my own company, we have hundreds of analytics projects at any given moment. Based on those experiences, I find that advanced analytics problems can be classified into five types.
Type 1: Convoluted Problems
Multi-level, multi-criterion, multi-objective business problems are the perfect recipe for convoluted problem statements. Data scientists typically need to apply a piecemeal problem-solving approach in these cases.
Example: Clients typically ask a consulting firm to leverage machine learning to help them predict revenue-at-risk or customer churn. A good consultant will break this problem into parts and solve for it.
- Who are the customers with a high likelihood of churn?
- When will they churn – next week? Next month? In three months? In six months?
- Why will they churn (the root cause or possible triggers)?
- What should be done to retain the most loyal and profitable customers?
This approach brings together multiple algorithms to a seemingly simple question and provides deep and actionable insights into the business.
Type 2: Multi-Disciplinary Problems
These are problems that require bringing together concepts from multiple disciplines such as statistics, economics, operations research, finance, etc. They require a strong collaboration between multiple subject matter experts.
Example: A Fortune 500 industrial firm’s production planning team brought on resources to develop a capacity determination tool that leveraged a combination of monte carlo simulations (statistics), discounted cash flow analysis (corporate finance), linear programming (operations research), and net present value and value-at-risk (risk management) modeling. The different algorithms were consolidated in the form of a framework, which then helped them make key CapEx decisions.
Type 3: Advanced/Complex Algorithms
These are cases where the problems may be straightforward, but the algorithms required to solve for them are very sophisticated. These are mostly artificial intelligence, machine learning, deep learning, and optimization algorithms. Some examples include random forests, support vector machines, neural networks, bayesian belief networks, and goal programming.
Examples:
- Predictive analytics algorithms like prophet, artificial neural networks, long short-term memory (or LSTM), gaussian processes for demand forecasting
- Anomaly detection algorithms like isolation forests, support vector machines, etc. for cybersecurity
- Collaborative filtering, matrix decomposition, deep neural networks for designing recommendation engines
- Decision trees, neural networks, and genetic algorithms for text mining
- Logistic regression, naïve Bayes, support vector machines, -nearest neighbors, and decision trees for classification (fraud/no fraud, churn/not churn, spam/not spam, etc.)
- Linear programming, markov chains, goal programming, etc. for optimization
Type 4: Predictive and Prescriptive Analytics Problems
Typically, analytics problems are classified as descriptive (what happened?), predictive (why did it happen and what will happen?), and prescriptive (what should happen?). The latter two could be classified as advanced analytics, not just because they require a lot of rigor and advanced algorithms (described above), but because they influence and drive decision-making.
Examples:
- Root cause analysis: What is causing my machine or equipment to break down?
- Driver analysis: What are the controllable and uncontrollable factors that drive sales in my stores?
- Optimization: What are the most time and fuel-efficient routes for my trucks?
Type 5: Research and Experimentation
There is a ton of research happening in the analytics and AI space and a lot of new algorithms have been developed recently. They have fueled a new wave of experimentation in the data science world bridging the gap between academics and industry. Companies have a healthy thirst for new and more effective ways of decision making, higher ROI, and faster and more accurate ways to solve existing problem statements.
Examples:
- Neural networks instead of time series for forecasting
- Logistic regression instead of decision trees for predicting churn
- Reinforcement learning instead of A/B testing for campaign optimization
In Conclusion
Over the next decade, advanced analytics will play a key role in any organization’s success. Analysts who want to work with these types of implementations need to help companies move from basic to more advanced forms of analytics. The rest will follow suit.
Enhanced decision-making driven by advanced analytics will help companies build sustainable competitive advantage. Analytics and business leaders need to understand what advanced analytics is and how to build capabilities around it.