Lessons From Recent Power Outages
andre.kesteloot at verizon.net
Wed Jul 18 16:32:45 CDT 2012
On 7/18/2012 17:13 PM, Terry Fox wrote:
> Energy Wise - 07/18/2012
> Um, isn’t blaming the grid kind of burying your head in the sand? If
> they are supposed to have such a fantastic and reliable Cloud,
> shouldn’t their cloud support include on-site backup power
> facilities? And, there should not be an “secret” or “unknown” closets
> of fiber that nobody knows about until Virginia Power/Pepco goes bye-bye.
> This reminds me of the story about the American University (or was it
> Howard?) facility that installed a new generator. When Pepco died,
> their generator promptly came on, ran for a few hours, then died.
> Somehow the pump that fills the day tank from the main tank was wired
> to Pepco instead of the generator, and the small day tank ran dry. Oops.
> I guess the Elastic got stretched too thin, and things did not
> computer anymore. All this because the virtual cloud got bumped
> offline by a few very real clouds.
> In the old days, this would be a true “teaching moment”, bordering on
> a firing offense. Being offline for an hour or two could be
> excusable, but for days?
> Terry, WB4JFI
I do not know whether you recall an earlier item I posted --> Poor
I like their use of the passive form after "unfortunately"...
> In April 2012, some of Amazon's servers went down. This was their
> explanation for what happened:
> /"At approximately 8:44PM PDT, there was a cable fault in the high
> voltage Utility power distribution system. Two Utility substations
> that feed the impacted Availability Zone went offline, causing the
> entire Availability Zone to fail over to generator power. All EC2
> instances and EBS volumes successfully transferred to back-up
> generator power._At 8:53PM PDT, one of the generators overheated and
> powered off because of a defective cooling fan._ At this point, the EC2
> instances and EBS volumes supported by this generator failed over to
> their secondary back-up power (which is provided by a completely
> separate power distribution circuit complete with additional generator
> capacity)._Unfortunately, one of the breakers on this particular
> back-up power distribution circuit was incorrectly configured to open
> at too low a power threshold and opened when the load transferred to
> this circuit._ After this circuit breaker opened at 8:57PM PDT, the
> affected instances and volumes were left without primary, back-up, or
> secondary back-up power."/
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Tacos