Blueshift has “A fairly relentless pursuit of more and faster performance,” said Justin Stafford,

CEO and co-founder. Blueshift is an Australian company that provides Cloud-based integrated business planning solutions for companies in the Consumer Packaged Goods (CPG) space. Blueshift does this through a product called ‘ONE,’ which combines the complexity of business planning – including commercial planning, sales forecasting, and budgeting – with supply chain planning, into an integrated Cloud solution.

According to the company’s website, Blueshift ONE facilitates trust across business functions by tying together the demand forecast with the P & L forecast.

“When business functions trust one another through the common language of a ONE number forecast, it turns the [decision-making process] from reactive to proactive.”

Blueshift ONE is an integrated business planning tool that supports demand planning, trade promotion management and trade promotion optimization. Although their customers are primarily in Australia, New Zealand, and Asia, they are seeing some traction in US markets, Stafford said.

Blueshift’s customers range from “around the $50 million turnover mark” up through multibillion-dollar turnover. “We focus specifically on that set of customers, and the specific problem that we are solving.” CPG companies typically allocate 10 percent to 20 percent of total revenue for ‘discretionary trade spend:’ short-term promotions that drive sales.

“You can imagine that 10 percent to 20 percent of total revenue for a CPG company is quite a large number.” For example, a company that produces Fast Moving Consumer Goods (FMCG) might have a $300 million turnover, which would translate into spending $30-60 million dollars a year on promotional activity, he said, “And we can generally save them between 1 percent and 10 percent of that total bucket per year.”

Integrated business planning with Blueshift ONE allows customers to optimize price elasticity, provide

“better and faster visibility of profitability of promotional activity,” and the ability to forecast impact for the supply chain, so factories know “exactly how much and how far in advance to produce the stock” for any promotion.

Many companies are prevented from optimizing their promotions and experimenting with prices because they still use spreadsheets, a tool that’s “very slow and cumbersome,” he said. “But if you can make [planning] really fast and easy to experiment with, and [provide] model promotional scenarios and plans, then you end up with a more optimized program.”

He believes that the easier users find a solution, the more likely they are to rely on it because they see how easy it is to generate and execute optimized promotions. “We’ve seen that play out with a number of different case studies that we’ve done, and one customer saved as much as 11 percent of their discretionary spending the first year.”

Growth Pushes the Limit

As Blueshift started to grow, the increased strain on their analytic capability compromised their ability to create a consistent, reliable experience for their users. They started to hear from some of their power users that query performance on reports was inconsistent.

After attempting a series of scale-up configurations, they realized that scaling-up would not be enough to transcend inefficiencies. According to Stafford:

“Our solution is historically based on the Microsoft SQL Server stack, and as you learn to appreciate, it’s not shard-capable, and so really, there’s only so much you can scale a single piece of hardware up to get to a performance level that is satisfactory.”

Sharding is a type of database partitioning that separates very large databases the into smaller, faster, more easily managed parts called data shards. No matter how much optimization is done, he said, ultimately, “we are talking about small, incremental gains.” With the goal of improving exponentially, Stafford said they started to consider that there could be possibilities beyond their current system:

“We were saying to ourselves that there’s got to be something out there, with the technology these days, that can help us to get to 10x or 100x performance improvements, and also allow us to scale for larger customers in a way that is just not possible with our current setup.”

After a “fairly extensive analysis,” including successful proof of concept processes and external advice, the team chose MemSQL, “due to the distributed data sharding support, support of ANSI SQL, and fast benchmark results.”

MemSQL Solution

According to MemSQL:

“MemSQL 6 includes the ability to run Machine Learning algorithms in a distributed SQL environment, enhancements to online operations, and increases to query performance to deliver up to 80 times improvement from previous versions.”

To increase query processing performance, MemSQL leverages modern chip architectures to achieve a processing rate of “one billion rows per CPU core per second.” MemSQL also provides:

“Efficient query isolation for improved concurrency on large data volumes and thousands of users, and higher performance on encoded data for faster processing of financial, web, or sensor application workloads.”

Customers have the means to query raw, unaggregated data on a scale of billions of rows per request concurrently, for thousands of customers. Previously the scale required to offer this was untenable, but a “highly scalable distributed system balances data and queries across a cluster of industry-standard hardware for maximum performance, concurrency, and availability.”

A big part of Blueshift’s value proposition is their commitment to superior performance. “That’s one of the reasons which got us to MemSQL. The other is the ability to scale,” Stafford said. Their existing solution worked well on a small data set, where sharding isn’t essential:

“But once you get to thousands of SKUs and thousands of customers and hundreds of weeks and multiply it out to multi-millions of rows, then you end up with a situation where the performance starts to degrade,” he said.

“Our users will be making changes throughout the day that involve a lot of desegregation and re-aggregation, but to the user it’s transparent,” he remarked. The user applies a 10 percent discount, for example, on a set of products at a national level, and the process appears to the user to be straightforward. Underneath that, however, a complex pricing structure is applied at the state level, which might impact millions of rows. The pricing structure is brought through the calculation and reaggregated back up to national level to show the end result to the user right away.

“Ours is a fair bit of desegregation and a fair bit of heavy-lifting, set-based operation work in real time, in addition to just doing the Data Warehouse segregation, and MemSQL handles both of those quite well, whereas some of the others are not as optimized for large-scale updates,” he said.

The company says that the Blueshift ONE platform, powered by MemSQL, is now capable of delivering analytics on nearly 2 billion records with 60 measures in under 2 seconds. The fast performance gives the application a leg up on the competition as they grow their customer base and expand into new analytic capabilities that were previously not possible.

The Donkey and the Stallion

The ability for MemSQL to complete a query instantaneously is based on Data Warehouse type workloads, he said, and best results come from well-designed queries.

“I would consider something like a SQL Server as like a donkey, where it’s going to give you slow performance but it’s going to be reliable and predictable. It’s just going to give you that slow and steady approach whereas, I’d kind of liken MemSQL to a stallion where it’s going to be blazing fast but it’s going to be quite temperamental.”

Stafford is quick to clarify that MemSQL is not buggy, it’s just extremely fast. “If a query is well-designed in MemSQL, it’s going to be instantaneous, but if it’s not, it’s going to basically never complete,” about the quality of your queries. “Some updates may never finish because they are just not well-designed to the sharding nature of MemSQL and so some care needs to be taken into that.”

Immediate Results

Blueshift says that test results have been able to prove MemSQL to be 100x faster than their existing SQL Server implementation. During additional stress tests of MemSQL, the data set was expanded by 100x to mimic substantial data growth, and yet the analytics performed by MemSQL continued to blow past expectations.

Stafford is optimistic about the gains they expect to make as a result of the improvements they’ve been able to make to Blueshift ONE.

“There are some pretty big sales to be made in this space, and on the secondary side to that, when you have a solution that is like ours, you end up with improved forecast accuracy and the associated benefit of forecast accuracy and optimized inventory, so there’s a whole bunch of savings to be made there.”

Next Steps

Promotion Optimization Institute, an organization focused on collaboratively improving the promotion and distribution of consumer goods, does a yearly comparison of trade promotion (TPx) vendors entitled “POI TPx Vendor Panorama.” Blueshift’s rating in the 2017 report noted that the change to MemSQL “Sets the foundation for more functional improvements across the platform.”

Stafford is looking forward to TPO’s next Vendor Panorama so he can put the improved Blueshift ONE up against the competition and see how well it performs on high-volume data sets. “Performance of these solutions is always a hot topic amongst the CPG space,” and he’s fairly confident that Blueshift ONE will impress.

“We will have the competitive advantage in the space and our competitors won’t be able to even come close to the performance that we are talking about. Time will tell them that, but I see actually using our performance in that regard as a competitive advantage even if not benchmark, it will be something that we are raising a flag on because I know that is a hot topic in this space in terms of our customer base.”

 

Photo Credit: LeoWolfert/Shutterstock.com