OVH flames scorched cloud customers with pledge to build data centre fire simulation lab
- Reference: 1616482872
- News link: https://www.theregister.co.uk/2021/03/23/ovh_fire_lab_restoration_update/
- Source link:
News of the lab came in a video update from OVH founder and chair Octave Klaba, who advised that some services in the company’s ruined Strasbourg data centre complex would return this week.
[1]
The SBG4 data centre, he said, will bring servers back online between Wednesday and Friday this week.
Klaba muddied the waters about the status of SBG1, of which he said in a Sunday [2]Tweet : “We don’t plan to restart SBG1. Ever.” The Register has a screenshot of the Tweet [3]here in case it is memory-holed.
[4]
In his new video, Klaba said after smoke was seen around SBG1 “we decided to close SBG1 and not restart for the moment.”
Cloud builders hoover up 60% of ALL servers sold in 2020 as enterprise bit barns left to sweat [5]READ MORE
What is clear is that servers from SBG1 will be moved to either SBG4, or to other OVH sites after they are cleaned.
Restoration of SBG3 is proceeding well, with public cloud infrastructure soon to be restored. The data centre hosted backups for around 30 percent of customers whose servers were destroyed along with the SBG2 data centre. Klaba said some of those backups will be restored this week.
A vSphere-based environment will also be back this week, but re-building virtual storage and networks will take time.
Update March,22 8pm
Here, the 3nd video with last updates about the fire in our DC in SBG2. [6]pic.twitter.com/sMGrCz2X4I — Octave Klaba (@olesovhcom) [7]March 22, 2021
With his update out of the way, Klaba quickly touched on root cause analysis and said it will be months before final findings are delivered.
While we all wait to learn if the [8]UPS maintenance worker fingered as a possible source of the fire is about to become a star of The Register ’s [9]Who, Me? column, Klaba promised that OVH will upgrade all of its data centres to make them more resilient.
He added that the company will open-source its water-cooling technology, then addressed the company’s fire suppression and extinguishing regimes. Klaba said all OVH data centers have them, but that the technology used varies depending on the room it is protecting.
[10]
OVH Strasbourg, post-fire. Click to enlarge
“It is not the same because the purpose is different in each room,” he said, explaining that server fires and electrical fires need different dousing tech.
Klaba then pledged to find out what works best, saying OVH will create a fire lab.
“I want to test how the fire is going in different rooms,” he said. “I want to find how to extinguish the fire in different rooms and share with the rest of the industry.”
[11]
“That’s for the future,” he concluded, as right now OVH is working “20x7” to bring customers back online. ®
Get our [12]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YFnKUZmy1qZXJGoKnwVofQAAABg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://twitter.com/olesovhcom/status/1373280258171019271
[3] https://regmedia.co.uk/2021/03/23/octave_klaba_sbg1_not_ever_restart_tweet.jpg
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YFnKUZmy1qZXJGoKnwVofQAAABg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2021/03/19/cloud_builders_hoover_up_60/
[6] https://t.co/sMGrCz2X4I
[7] https://twitter.com/olesovhcom/status/1374069154085490690?ref_src=twsrc%5Etfw
[8] https://www.theregister.com/2021/03/17/ovh_restoration_update/
[9] https://www.theregister.com/Tag/who-me
[10] https://regmedia.co.uk/2021/03/23/ovh_strasbourg_burned_shutterstock_editorial_only.jpg
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YFnKUZmy1qZXJGoKnwVofQAAABg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://whitepapers.theregister.com/
Re: 20x7
France, so: Yes, ideally. Two at least.
Re: 20x7
It's France, so 20 days a month, 7 months a year. The rest is vacation time.
Re: 20x7
It's France, so 20 days a month, 7 months a year. The rest is vacation time.
I am missing strike time in your calculation, it is France after all.
SBG1 data centre status in doubt again
If you look at the site layout, it would be awkward for SBG1 to remain while SBG2 is demo'd and rebuilt. As SBG1 looks half toast anyway, moving any remaining kit elsewhere seems sensible, even it it's into more containers north of SBG4. Of course they may still call that "SBG1"...
https://lafibre.info/images/ovh/202103_ovh_strasbourg_plan.jpg
https://lafibre.info/images/ovh/202103_incendie_ovh_strasbourg_3.jpg
"Of course they may still call that "SBG1"..."
Or maybe BBQ1...
A lab to model the effects of data centre fires
It seems to me that they already had one of those, but perhaps in retrospect it wasn't necessary to run live workloads on it for the model to be valid.
The backup is on top of the cabinet ...
My company discovered that's not a good idea after a data centre was wiped by the IRA back in the bad old days. Hilariously the reserve hardware was located in Belfast!
Backing up SBG2 to SBG3 is almost as bad. Do people never learn? If it hadn't been a fire it could have been a network or power setup that took out the whole Strasbourg complex. These days geographical separation of data should be more than 100 yards - Oops metres. Preferably in a different country.
The point is the easy bit having been wiped out in SBG2 is getting hardware running elsewhere - a VPS can be up and running in minutes and a few mouse clicks from a wide choice of providers. But if you have to wait WEEKS before you can get your hands on and start loading the backup - well what can one say? And if a customer finds the (presumably paid for) backup has gone up in smoke too then any sympathy one may have for OVH being the victim of a possible third party UPS error follows it up the chimney.
The French IT lawyer industry must be licking their lips - provided they don't host their billing system with OVH.
the company will open-source its water-cooling technology
Presumably so that the community can improve it to either prevent the fire starting, or to put it out once it does?
French cloud company OVH has pledged to create a lab to model the effects of data centre fires
Let me guess, they will run it in the cloud
.
.
.
of smoke
20x7
... is that because of a 4 hour lunch break?