While Reuters says that Wednesday’s power outage led to less than one hour of downtime, users tell Inside Market Data that they suffered problems for much longer.
In its only comment on the outage, the vendor says, “Reuters had a temporary power outage lasting under an hour at our Hauppauge [NY] facility in the US last night (Wednesday). Full power was back on in under an hour, and our services are now fully restored.” The statement was made on Thursday.
But one source says the problems began at 2:28 pm EST on Wednesday, Aug. 23, and streaming updates were not back up until around 5:30 pm EST. In addition, all statistics, reference and calculated data and ticks for the last half a day were gone.
Another observer says the “all-clear” was given at 10:30 pm EST on Wednesday, but the first source says some exchange data continued to be affected on Thursday.
Multiple sources confirm that the outage was caused by problems with the UPS (uninterruptible power supply) at the Hauppauge data center. According to one person, there are two UPS switches, and when the yellow switch suffered a power cut, it attempted to switch to the red switch. However, the red UPS could not cope with the demand, and it, too, shut down.
Reuters apparently chose to move to street power rather than switch over to its Chicago plant, with two sources saying that it expected that this would get the Hauppauge facility up more quickly. However, one source says, “Customers still do not understand why there was no effective live backup.”
The outage affected Reuters’ IDN (Integrated Data Network), which delivers the Reuters Data Feed.
Although the power outage was limited to the Hauppauge center, customers outside the US were also affected because they did not receive North American data. However, one source says that their systems were restored more quickly.
The latest outage follows several well-publicized failures with Reuters facilities in 2004, after which CEO Tom Glocer announced the acceleration of business continuity plans (IMD, Oct. 25, 2004). However, after this week’s outage Reuters refuses to comment on any details or on its plans for preventing future such outages.
Additional coverage of this story will be provided in the next issue of IMD, published on Monday, Sept. 4.