Click to learn more about author David Segleau.

I recently spent more than a few hours trying to build and configure a DIY solution in the cloud for ad-hoc analytical queries. The exact pieces of software are not that important because all of the build-it-yourself-better-and-cheaper solutions start to look the same after a while. After about 10 hours of reading multiple how-to and step-by-step guides, installing software bundles, and configuring, securing, and starting up a storage repository, an ETL process, and a distributed query engine, I could finally start running some queries.

But if I had problems, especially performance or reliability issues, could I troubleshoot them, get some help, and resolve them quickly? No way! Not without trolling through a bunch of community forums, examining log files until my eyes bleed, reading many how-to guides, and trying to piece together how all of these packages actually communicate with each other. Nostalgically, I started longing for the days of the one-size-fits-everybody, big monolithic-product-in-a-box solution offered by the major software vendors (yes, I’m looking at you, Oracle).

But is that really my only choice? The DIY-better-and-cheaper solution that’s cobbled together from lots of multiple parts vs. the very expensive one-size-fits-everybody monolithic-product-in-a-box? Many pundits and software providers would say that it does really boil down to that one choice. I disagree.

The DIY technology providers (bless their little open source hearts) would say that their solution is the better option. Primarily because it’s open source (more noble and modifiable), more flexible (aka “lots and lots of ways to configure it”), and cost-effective (because it scales and performs better, has very low startup costs, and allows you to pull in/configure only what you need). They point to companies like Twitter, LinkedIn, Facebook, Amazon, Google, Apple, Expedia, Airbnb, Uber, etc. who have all used the DIY approach with fantastic results. The problem is that most small to mid-sized businesses do not have an army of technical talent that they can throw at the DIY model. Nor do they have the time or resources to spend working through the learning curve inherent in such an approach.

Actually, it’s pretty common knowledge in the industry that the 1st, and often 2nd, generation of DIY roll-your-own-technology-stack projects fail, and only the 3rd generation projects tend to succeed. Yes, I will admit that modern container technology helps with building and deploying DIY software configurations. But now, instead of going to 10 different places to get your software, you’re getting a palette of 10 pieces of software dropped on your doorstep in a container. You still have a lot of DIY work to get it to actually solve your technical problem.

On the other hand, the monolithic-product-in-a-box vendors will tell you that theirs is the better option because it’s comprehensive (has absolutely everything you could ever want and then some), is simple to use (just look at the encyclopedic documentation), and has an army of professional services and support staff to help you (for a fee, of course) if you should ever need it. It’s a nice sales pitch and looks simplistically attractive, but the licensing, operational, service, and management costs are often prohibitive for small to mid-sized businesses. After all, simplicity is nice, but cost control is even nicer.

So, is there a middle ground? I think that there is. I think the bundled/packaged technology service-on-a-platform providers that have arisen in the last few years offer the optimal solution for many enterprises. These vendors have grown up in the age of the cloud, during the big data/NoSQL/data lake/container explosion, and have personnel with a lot of open source, DIY experience — often having worked on open source projects directly.

These vendors (like my friends at Ahana.io, Dataworkz, and Couchbase, to name a few) take the best of what the open source, DIY providers have to offer and bundle that together as a platform or set of services that are easy to configure, secure, deploy, and use. They are providing the best of both worlds — the broader solution and ease-of-use approach of the product companies, with the only use-what-you-need approach of the DIY crowd. These vendors provide solutions that are better suited for most enterprises that don’t have bottomless technology resources (time, budget, or people) and that just need to “get the job done right, as soon as possible, without breaking the bank.”

So the next time you find yourself or your IT department, like I was, wallowing in the endless cycle of the DIY build-it-yourself-better-and-cheaper vs. the very expensive monolithic-product-in-a-box quandary, think about that middle option. Look at some of the newer bundled technology service providers. They may well have the “get it done now, get it done right, at the lowest startup cost possible” solution that you’re looking for.