IT Managers Start To Deal With The Problem Of Big Data Backup

It's How An IT Manager puts the pieces together that makes big data valuable
It’s How An IT Manager puts the pieces together that makes big data valuable
Image Credit

The era of big data has arrived. IT Managers everywhere are swimming in a sea of data and only now are they starting to get the tools, the IT manager skills, and the IT manager training that will allow them to make sense of what they have. It turns out that there is another problem that has arrived at the same time and right now there is not a clear answer to how best to deal with it: how to back up all of that data.

What’s The Big Deal Here?

Back in the old days, when the size of the data that your IT Manager job required you to manage was still something that could be dealt with, whenever your team received new data it just got added to your existing backup plan. A backup job was run sometime late in the night and you always knew that if something bad happened to your new data, then you could just restore it from the backup that you had created.

However, that’s all changed now that we’re living in the era of big data. The data sets that IT teams are being asked to manage are too large to be backed up. The petabytes of content that may consist of video, audio, and images is too large to be moved across your company’s network let alone be backed up.

Let’s all agree on something right off the bat: creating a disk based backup of your big data store is not something that is going to be practical to do. The cost of having to create a backup for your big data that would be virtually unlimited in size much like your big data itself would simply too expensive to do. It looks like another solution is going to be called for.

Solutions To Your Big Data Backup Problem

What really caught my attention is that the best potential solution to this problem just might be an old solution: tape. Back in the day before disk drives became so cheap, everything was stored on tapes. They were easy to get and cheap to use.

The reason that tape storage might be something that once again you should consider as you enter into the world of big data is that all data is not created the same. Much of the data that your IT team is responsible for is rarely, if ever, accessed. What this means is that to store it on expensive disk drives does not make any financial sense.

Instead, a better strategy is to store as much of your big data on disks as is possible. However, the older data that is not being accessed needs to be backed up and moved off to tape. By doing this you will have both protected your data and found a way to lower the cost of storing all of that data.

What All Of This Means For You

Like it or not, in addition to all of that IT team building that you are supposed to be doing, you are now responsible for managing a lot of big data. All of this data is valuable to the company and you are going to need to come up with a way to back it up. However, the traditional ways that we back up data won’t work for big data – it’s just too big.

A better way to go about solving this problem is to consider an old solution: tape. It turns out that taking the time to move your older data that is not being accessed very much off of expensive disk storage to a cheaper tape storage solution can solve two problems at the same time: backing up the data and reducing your data storage costs.

At the end of the day, the importance of information technology to your company means that you have the job of keeping the company’s big data store secure and backed up. Considering different ways to do this is an important part of your IT manager job. Reconsider what tape might be able to do for you and perhaps you will have a found a cost effective solution that everyone can live with.

– Dr. Jim Anderson
Blue Elephant Consulting –
Your Source For Real World IT Management Skills™

Question For You: Do you think that a tape storage solution should be a manual process or automated by robots?

Click here to get automatic updates when The Accidental IT Leader Blog is updated.

P.S.: Free subscriptions to The Accidental IT Leader Newsletter are now available. Learn what you need to know to do the job. Subscribe now: Click Here!

What We’ll Be Talking About Next Time

If you were to create a list of the buzz words that are filtering through the world of IT right now, “noSQL” would have to be at the top of your list. IT Managers everywhere have decided that they’ve had enough of traditional databases and the high vendor fees that come with them. Open source noSQL databases seem to appear to be the solution to all of their data processing needs. The problem is that deciding between using SQL and noSQL databases is something that has not been a part of our IT manager skills training…