The last thing a manager wants to think about before falling asleep at night, is whether the customer data archived yesterday will be there tomorrow. Who knows when a client will come back years after project completion and ask for the video created back then? A supervisor needs confidence that data is automatically archived in the cloud, daily, and that it persists, in an easily findable place.

At least one director, Mike McMullan, Director of Production at Crisp Video Group can breathe more easily today, knowing he can pull up any archived customer data from 2015 forward. In a recent DATAVERSITY® interview, McMullan described his journey, from his surprise at finding customer data in a recycling bin to having confidence in accessing any client’s data from 2015 to-date. As Crisp Video has scaled up to be “the largest fastest growing video-marketing firm in the legal space,” McMullan acknowledged that trusting in a robust and accessible data archive has not always been easy.

When McMullan started with Crisp Video in 2015, the company had a 50-terabyte Synology network-attached storage (NAS), that ran everything in production and backed up data to Amazon S3, in the cloud. The CEO approached him one day and asked if he would like to help manage the data. McMullan accepted and began moving completed “projects from hot-storage or production to cold storage in the NAS.” In less than two years — late 2016 to early 2017 — he saw that Crisp Video needed more storage. McMullan started looking for solutions.

Searching for a Storage Solution that Scales

Mike McMullan advocated for additional hardware that would give more storage: a full server rack for production. He got a Studio Network Solutions EVO and planned on doing backups the same way as before, archiving data on Amazon S3 through Synology.

While transitioning to Studio Network Solutions EVO, he found archived customer video in Synology’s recycling bin. Luckily it had been less than 30 days since deleting the older video from production to archives, and it could be saved. Also, a customer had not directly asked for it. But McMullan realized a larger issue, “backups were not running nightly” on Synology and this had been going on for some time.

He wanted to “set and forget” nightly backups and have confidence this process was running smoothly. While Synology had an easy to use built-in application, McMullan “had to know the nitty-gritty side of things,” to ensure data was properly archived. He did not have the bandwidth for this kind of detail. He considered management software, like CloudBerry, to ensure and “facilitate nightly backups to S3,” but thought the price point was too high. He took a step back with all his requirements, and looked for a better backup option based on price point, storage scalability, and ease of setting it up and forgetting about it.

Reaching out to a Reasonable, Reliable, and Robust Cloud-Storage Solution

Crisp Video and McMullan reached out to Backblaze B2 Cloud Storage solution. He chose Backblaze products due to many factors. The “price per gigabyte to upload and restore” archived data was very reasonable. McMullan also had a lot of faith in Backblaze when he saw its knowledge-sharing approach, while still being a professional data center.

McMullan sat down with Backblaze’s representative and signed up Crisp Video in late 2016. Both partnered to piece all Crisp Video’s hardware together, while helping Crisp Video plan a move to a new office in February 2017. Backblaze assured McMullan that the archived data would be backed up nightly successfully, from the day the new office opened and into the future.

Before opening day, McMullan set up the new hardware in the new office. He transferred everything to a new Studio Network Solutions server and then set up a separate server dedicated to “running and facilitating the backups,” along with the qBackup software application. From that day forward, Mike McMullan and Crisp Video have used Backblaze and qBackup. McMullan explained:

“For the price point, Backblaze and qBackup run flawlessly. I have confidence that we can restore files because we have had to restore files since then.”

Crisp Video’s confidence in Backblaze was first proved after successfully running a backup of 60 to 75 terabytes, covering a lot of footage. The video company then ran out of space on the old Synology drive. So, McMullan backed up everything onto Backblaze using a 300 MB fiber connection. This took a little time and was completed in June 2017. But McMullan had confidence.

In the meantime, McMullan ran partial backups to make sure some files were available, and Crisp Video knew they could pull files from storage if the need arose. Later that summer, Backblaze introduced Fireball, a service that migrates huge data sets from on-premise to its cloud server. McMullan ran this more “streamlined solution” to put everything on the Backblaze servers quicker. McMullan said:

“When we ran everything on Fireball, it worked. We also know that the restore process with Backblaze and qBackup has been great as well. As we have grown from those initial 50 to 70 terabytes to the 2,300 terabytes now, we know we can continue to rely on Backblaze to restore our data.”

Since moving to Backblaze, McMullan does not feel constrained by how much physical storage he has for the data. At the very least, he knows:

“If I archive one piece of footage from the production server, shot in January of 2018, I no longer need to worry about our client coming back and needing it. Now Crisp Video has packages that run for two-plus months to two-plus years. The archived video will be there from each of the fifty-ish terabytes shoots to the four-terabyte multi shoot file of the second annual conference in Alpharetta, Georgia.”

Even better to McMullan, a client who Crisp has not seen for a couple of years and who is not part of a larger Crisp storage package, can arrange to get their footage, because it will have been archived. Ever since Crisp started in the legal business in 2015, this archived information is accessible, regardless of whether the project has “closed.” Mike McMullan could sleep at night.

Advice in Archiving Data in Cold Storage

In reflecting on his experience, McMullan advised the following to others managing archived data:

• Always consider your initial backup in planning: This means thinking about what files will be backed-up (e.g., carbon copies), and whether the production server in use has a built-in backup solution. Be aware that some backup software, like CloudBerry and qBackup, do not map files to the same format when doing the backup. Think of zipping a directory. In these cases, the files have to be restored from the archived to the original formats.

• Plan on more than one backup: McMullan uses multiple backup options. He still uses the Synology NAS as a local, redundant backup. He recommends holding onto those other backups in case years from now another solution is needed, and thinking about how to pull as many files as possible from an older backup solution. McMullan suggests having local redundant and cloud backup solutions.

• Think carefully about partial backups: While partial backups can make some files readily available, they can cause some problems, notes McMullan. When the backup does not complete and gets canceled, it cannot be restored. “The backup has to complete 100 percent before accessing any of the archived data.” Upon restarting a backup to completion, all the files need to be there.

• Quarantine files that cannot be deleted due to a backup in process: When files are needed to complete a backup, quarantine these, McMullan advises. Make sure others do not touch or delete those files so the backup completes. Local redundant backups serve as a fail-safe.

• Ensure confidence in your solution: McMullan’s success comes from doing some housecleaning to make sure the backups run correctly. This chore is done more frequently at the start of a new solution to develop assurance. His assistant looks at the backups maybe once a month, to see that the nightly ones run correctly. Some of the Windows updates have hindered a backup on one random night or another. This is a very light touch given the price point, and based on the backup performance, he has the data to rely on Backblaze’s solution for Crisp Video.

Not all cloud archiving solutions are equal. Mike McMullan and Crisp Video’s story show how understanding archiving requirements and planning data backups as well as testing — at least initially — that backups complete as expected makes better business sense and delivers more confidence in archived data.

Image used under license from Shutterstock.com