“Seven years ago, the petabyte was pretty much the exclusive realm of very, very, very, large enterprises, like Citibank or Boeing,” said Shridar Subramanian, VP Product and Marketing at StorageCraft, but petabyte-sized data sets are now becoming more common with smaller companies. Over time, more companies in the mid-sized enterprise space, and more importantly, companies in the small and mid-sized business segment, are getting inundated with these petabyte-sized data sets.

One petabyte is one thousand terabytes, or one million gigabytes. StorageCraft President Douglas Brockett said, “the petabyte era will start barreling down on mid-sized organizations, too. What used to be an anomaly will start to become the norm for SMBs and mid-size organizations.”

Organizations are not equipped to deal with data storage at this scale, Subramanian said. They don’t have the infrastructure, their personnel are not equipped to safely manage data at this volume, and they are not equipped to make decisions about what to keep and what to protect. They wonder if their data needs to go in the cloud, and if so, how does that work?

Exponential Proliferation: Dawn of the Petabyte Era

An entity like a small police department, for example, has their officers using 4K body cams recording on a second-by-second basis, “and all of that data has to get stored somewhere,” he said. “This is happening pretty much across the board.” In school districts, in small medical offices, everything is now digitized, he said. “Not just that, but the MRIs, the mammograms—all of these things are pretty data intensive.”As the resolution of this data increases, the amount of data that gets stored increases as well.

The Problem of Storage

The traditional mindset is silo-based and silo-focused, which is manageable at a small scale, but creates problems at petabyte scale. “Within primary storage, they have a choice. Do I put them in HDD, or on SSDs, or do I put them in a hybrid infrastructure? And each one of these things is treated as a separate island,” remarked Subramanian.

If the decision is made that a cloud solution is best, then the struggle becomes where to put all the data. “Which cloud do I pick?” So not only is there concern about the amount of data, he said, decisions have to be made about where to store it, and if it ends up in a particular silo, how can it be recovered or managed?

The Weight of the Petabyte

Protection is a serious issue for all companies, but once a company reaches the petabyte realm, access and management become unwieldy, especially if the data is stored in multiple bifurcated clouds and silos. A multitude of regulations for data privacy and data retention are having an impact across the board, not just in the U.S. GDPR and regulations for the financial industry, as well as those that have been in effect longer, such as HIPAA in the healthcare industry, are now becoming a concern for many smaller and mid-sized enterprise companies. Knowing what data requires protection and what to keep locally vs. what can be kept in the cloud is a manageable issue on a smaller scale, but the volume of the data is forcing companies to struggle on several fronts, he said.

Access is of growing importance now that personal information is more tightly regulated. Companies now need to know where social security numbers are stored, for example, and how to search for, access, and protect them, he said. “Let’s say an employee leaves, and they walk off with certain documents, then I need to know what documents they walked out with.”

In response to this massive influx of data, companies are now wisely doing away with silos, in order to create a more converged infrastructure. This can contribute to a seamless environment across layers of Data Management so that data can move easily and be backed up easily, he said, and a converged structure provides an opportunity to look at which data really needs to be protected and how to go about tiering, etc.  

From Cloud to Hybrid

Subramanian said that initially, when cloud storage emerged, customers wanted to put 80 to 90 percent of their data in the cloud. Soon they realized that either the economics don’t work out for everything, or the latencies, or the access patterns and the need to access data didn’t work in the cloud environment for many of their applications. “So they have been moving to more of a hybrid infrastructure, which is a combination of a private cloud and a public cloud or an on-prem and a cloud infrastructure.”

Subramaniannotes that there is an increasing demand for useful insights and analytics out of the data that companies are amassing. “Some people call it Machine Learning, some people call it AI, but it is primarily getting insights from the data that I have as a company.”

StorageCraft

Despite the history of success with companies like Citibank, downscaled enterprise solutions “are not going to cut it,” he said, because solutions geared for big companies will be overly complex, over-engineered, or overly expensive.

StorageCraft answers this need with scalable solutions for small and mid-sized companies to address the challenges of massive data stores without creating data silos. Data can be stored in a hybrid on-prem and public cloud storage environment that provides access to analytics and insights as well as built-in data protection and backup. “It becomes pretty much a one-stop shop. Not only that, what we are building is something that is extremely simple to deploy, very simple to manage, and very easy to scale,” he said.

Many customers start with a single node and as the amount of data grows, nodes can be added on the fly without disruption and without having to create multiple silos. “And all of these data can actually be replicated to the cloud for disaster recovery.”

Channel Partners

A new feature Subramanian talked about was working with channel partners to better serve the changing needs of mid-sized enterprise businesses. “We don’t have to be doing everything. In fact, there is expertise out there that we really want to leverage and come up with partnerships as well as joint solutions.”

Traditionally, channel partners have either managed the whole process, or they sell a product and then they’re gone. He sees the trend moving to more of a hybrid model where companies serve as consultative solutions architects for them. “We are beginning to work with many channel partners who can help these small and mid-sized enterprise companies deploy solutions that can help them address these challenges.”

StorageCraft offers small and mid-sized companies a solution to the problems of the petabyte era, Subramanian said, “In a way that is consistent and completely aligned with where they are from a resource perspective, as well as from an IT infrastructure perspective.”

Image used under license from Shutterstock.com