If you were to create a list of the buzz words that are filtering through the world of IT right now, “noSQL” would have to be at the top of your list. IT Managers everywhere have decided that they’ve had enough of traditional databases and the high vendor fees that come with them. Open source noSQL databases seem to appear to be the solution to all of their data processing needs. The problem is that deciding between using SQL and noSQL databases is something that has not been a part of our IT manager skills training…
IT Manager’s ACID Problem
When we get presented with a new technology, it can be all too easy to start to think that it is a solution that we can apply to every problem that we are currently facing. noSQL is one such technology – even if using it has not been a part of our IT manager training. However, the key is to realize that not all data that the IT department has been asked to process is created the same.
The data that your IT team has traditionally been asked to process generally all looks the same. This is the data that we feed to the company’s mission critical systems. The acronym ACID has been created to describe this data:
- Atomic: each transaction is executed completely and can be rolled back if something goes wrong.
- Consistent: no transaction will be permitted to leave the database if for some reason it creates an inconsistency with the stored data.
- Isolated: each transaction does not affect another transaction
- Durable: before a transaction can be considered to be complete, it must first be recorded permanently in the database.
What IT Managers Need To Know About BASE
In the new world of “big data” in which we find ourselves, clearly not all data is going to meet the ACID criteria. This is where the door of opportunity opens for noSQL databases.
When we start to consider web and social media applications, we start to have to deal with data that is orders of magnitude larger than most standard corporate databases. This means that the developers on our IT teams need to become more flexible when dealing with this much information.
The data properties of this kind of new data workload have been captured in the acronym BASE. This stands for:
- Basically Available: just what it sounds like – the database is no longer required to be consistently real-time atomic.
- Soft-State: Database states are now permitted to change and expire instead of always having to have to be durable.
- Eventual Consistency: This flexibility is in contrast to a traditional ACID database’s requirement to provide stringent transactional consistency.
Which Database Is The Right One To Use?
I’m sorry about this; however, although it’s important to understand what kind of data set you are dealing with (ACID or BASE) , that’s not going to be enough to tell you which type of SQL / noSQL database you need to use for your next IT project.
noSQl will be the right database to use when you have BASE workloads that are clearly not ACID, when you have a great deal of data, and when you want to be able to run your database using commodity hardware and software.
There is one other point for you to consider when you are trying to decide between an SQL or noSQL solution. If you try to use an SQL database for an application that must deal with high-volume workloads that are delivered via the web, then you’re going to see your database collapse because of the overhead. Instead, in these situations go with a noSQL solution.
What All Of This Means For You
It’s never been easy to be an IT Manager (you know how hard that IT team building stuff is) and lately it seems as though even those things that we thought that we had under control, like databases, are undergoing significant changes. One of these changes is the arrival of no SQL databases – when should we use them?
It turns out that not all data sets are created equally. Data that can be classified as being ACID are well suited to being processed by a standard database. However, data that can be classified as being BASE would be better handled by a noSQL database. Additional issues such as the quantity of data needs to be taken into consideration also.
What this means for you as an IT manager is that what might have once been a fairly standard decision (“throw it into the database”), has now become yet another issue that you need to take a careful look at before making up your mind. Take the time to learn how to do this correctly and you’ll find that you are making the right decisions for your company.
– Dr. Jim Anderson
Blue Elephant Consulting –
Your Source For Real World IT Management Skills™
Question For You: Do you think that you should create a separate DB subteam to handle noSQL or should this be part of your existing IT team?
Click here to get automatic updates when The Accidental IT Leader Blog is updated.
P.S.: Free subscriptions to The Accidental IT Leader Newsletter are now available. Learn what you need to know to do the job. Subscribe now: Click Here!
What We’ll Be Talking About Next Time
When you decided that you wanted to go into IT, I’m willing to bet that you thought that this meant that you had closed the door on a career in what are traditionally called the business fields: sales and marketing. Well guess what, that door isn’t really closed, it’s actually open just a bit. This means that as a successful IT manager, in addition to your IT manager skills you’re also going to need to develop your marketing skills.