As IT managers, it’s our responsibility to use our IT manager skills and our IT teams to ensure that the company’s tools and IT systems are always up and operating correctly. If those tools are the primary way that the company makes money, then our job is doubly important. That’s why a recent system error that occurred on options exchanges run by IntercontinentalExchange (ICE) should capture our attention.
What Went Wrong
The world of modern financial markets is both complex and fast-paced. Thousands of transactions are executed every minute and the IT systems that execute those transactions need to be both reliable and accurate so that everyone who is using them trusts what they are doing.
That’s why on Tuesday, April 29th, 2014, when something went wrong it was such a big deal. IntercontinentalExchange operates two options exchanges. The first is the NYSE Arca Options exchange and the second is the NYSE Amex Options exchange. What appears to have happened is that during the first 13 minutes of trading activity on Tuesday, an internal system error occurred.
The result of this error is that later in the day, IntercontinentalExchange was forced to cancel nearly 20,000 trades. The impact of this is that 500,000 options contracts were affected. Just to make things even worse, IntercontinentalExchange took their time in informing the markets which trades were going to be undone. The NYSE Arca Options exchange rollbacks were announced at 1:22pm while the NYSE Amex Options rollbacks were announced at 3:42pm. The result of this is that traders spent most of the day unsure of where they stood.
The Arrival Of A “Kill Switch”
Clearly IntercontinentalExchange has experienced an IT problem. What this means is that there is an IT manager and an IT team somewhere that is scrambling to find out what went wrong and why it went wrong. I’m not sure if there is any IT manager training that’s going to help them get to the bottom of this issue quickly. However, there is a much bigger question to be answered here. How can this type of problem be prevented in the future?
Clearly IntercontinentalExchange has an IT process problem. It may be a lack of testing or monitoring, but no matter what they need to make changes to their process. However, the NYSE has made a proposal to take things just a bit further.
The NYSE believes that IT managers may never be able to prevent errors like this from happening. That’s why they are proposing the creation of what they are calling a “kill switch”. This is a piece of software that each firm could implement that would stop trading in a company’s stocks if some predetermined threshold was exceeded. Not a perfect solution, but one that might be able to prevent the need to have to undo 20,000 options trades in the future.
What All Of This Means For You
If there is one thing that IT managers are counted on to do, it’s to keep the company’s systems up and running. Over at IntercontinentalExchange they recently experienced a situation in which this was not the case – they had a significant system error.
What happened is that nearly 20,000 trades that had occurred on two options exchanges had to be canceled. An “internal system error” occurred and as the company looked into what had happened, they were unable to inform traders what trades were going to be canceled until late in the day. In order to deal with events like this, the NYSE is considering implementing a “kill switch” that would shut the market down if things start to go wrong.
As IT managers, on top of all of that IT team building that we’re supposed to be doing, it is our responsibility to make sure that significant system errors don’t happen in the first place. We have the tools and processes in place that we can use to ensure that we don’t have problems like this: testing, simulation, and monitoring. However, in the case of IntercontinentalExchange it appears as though these were not being used correctly. Make sure that you don’t make the same mistakes!
– Dr. Jim Anderson
Blue Elephant Consulting –
Your Source For Real World IT Management Skills™
Question For You: Do you think that the trades should have been cancelled or should they have been left in place?
P.S.: Free subscriptions to The Accidental IT Leader Newsletter are now available. Learn what you need to know to do the job. Subscribe now: Click Here!
What We’ll Be Talking About Next Time
In order for your IT team to accomplish all that is being asked of it, both you and your team are going to have to become innovative. As we are all probably quite aware of now, this is an easy thing to say, but it can be devilishly difficult to do no matter what IT manager skills we believe we have. The heart of the problem comes from not knowing where an IT manager needs to start your search when you go looking for the inspiration that often turns into innovation.