We have just recovered from our worst-ever outage, at 25 hours and 9 minutes. Here is what happened.
Our principal database server went down at 09:16 Monday 22/09/14. We moved quickly to reboot it, but this failed. Concluding it was a hardware issue (it turned out to be a failed motherboard), we switched over to the backup and got Yacapaca working again around 11:30.
By 12:00 it was apparent that the backup servers were just (more…)
