Trading at the Singapore Stock Exchange (SGX) closed for three hours on Wednesday after its systems failed to cope with a voltage fluctuation caused by a lightning strike. An independent investigation will probe the Exchange's UPS and failover setup.
The voltage fluctuation lasted 0.1 second and did not cause major disruption to other businesses. SGX has two uninterruptible power supplies (UPSs) and "four layers of redundancy", but its its official statement says the internal power supply of both units "malfunctioned", and it could not switch over to its secondary site because of a possible data "mismatch".
Lightning strikes twice?
The Monetary Authority of Singapore (MAS) has demanded a full independent inquiry into the incident which sources estimate will have caused losses of around $1 million to businesses including SGX.
"We have instructed SGX to take immediate remedial action to prevent a recurrence," a MAS statement said. "MAS has also instructed SGX to conduct a thorough investigation to find out the root cause of the outage and submit its investigation findings to MAS. MAS will not hesitate to take appropriate regulatory and supervisory actions against SGX should lapses be uncovered in the investigations."
Lightning strikes are common in Singapore, and businesses are normally well prepared for power surges and fluctuations - and this one was comparatively minor according to the utility SP PowerGrid. The lightning strike, at 2.18pm, caused no interruption of the power grid, and most customers were not affected at all, SP says. SGX was out of action until Securities trading restarted at 5.15pm and Derivatives restarted at 7pm (see timeline below).
The SGX trading system, installed in 2011, was powered from two substations, and each feed had a UPS. The fluctuation triggered a switch to the UPSs, but the SGX statement says "the internal power source in both UPS systems malfunctioned". Such an unlikely event will trigger an investigation into the set-up and installation of the systems.
The investigation will also have to look into why SGX was not able to switch over to its secondary data center. The Exchange could not use the secondary site, apparently, because of fears that its data might not be up to date.
"The power supply disruption was transient in nature, bringing about a potential mismatch between the primary and secondary market order books," the SGX statement says. "As such, a cut-over to the secondary data centre from the primary data centre was not executed. A complete restart of both primary and secondary systems was necessary. This step was essential to ensure continued integrity of the markets."
The SGX trading system was protected by UPSs from Hitec, with Caterpillar diesel generators. Hitec made no comment on the incident ahead of the official enquiry.
Outages in Stock Exchanges are serious but not unknown, with the Australian Stock Exchange suffering more than once in the last five years.
New thinking about reliable and efficient data center power