OVH says burned data centre’s UPS, batteries, fuses in the hands of insurers and police
- Reference: 1615964167
- News link: https://www.theregister.co.uk/2021/03/17/ovh_restoration_update/
- Source link:
In a video shared on Tuesday, Klaba said the electrical equipment has been removed from site and will be examined by experts, as will video of the fire that [1]destroyed one of the company’s four data centres in Strasbourg, France. The founder and CTO added he will offer an update on both the root cause of the blaze and restoration progress on Friday.
[2]
For now, Klaba said OVH has adopted a new policy to back up all customer data, for free.
“Some customers do not understand what they bought,” he said, referring to confusion about OVH having its own backups of some services and no backups of other data. In future the cloudy concern will therefore just make customer backups a part of its products.
[3]
“This will change the standards of the industry and the market,” he said.
Tuesday's update contained good news and bad news for the three other data centres at the Strasbourg site, one of which – SBG1 – was heavily damaged in the blaze that took out SBG2. Another, SBG3, was partially hit, and the fourth, SBG4, was unscathed.
OVH rises to Europe data sovereignty challenge (and AWS) with tape-as-a-service [4]READ MORE
The good was that the power supply – all 20kV of it – and optical networking have been restored at the location. Replacement UPSes and electrical systems have been installed, and are a day away from validation. Water cooling works, high-level networks have been restored, and work on networks for individual data centre rooms will soon commence. 2,500 servers have been built, and OVH will have added 15,000 new machines by the end of March. Restoration of many services appears to be on track to commence next week.
The bad was that two racks in the SBG1 data centre – 61E and 62E – may take longer to restore than others. (Previous discussions of SBG1 suggested four other racks – 61Bxx, 61Cxx, 62Bxx and 62Cxx – were severely damaged by the fire.) Restoring SBG3, which has five floors, may also be delayed. The bottom three floors are all fine, apparently, but the fourth floor has “some issues” and the top floor has smoke damage.
“There are not so many servers in this room, but it has to be cleaned,” Klaba said. “We are working with the insurer to be allowed in.”
“Each server has to be checked,” he added. “Even if it seems to be clean, we will check it to not create additional issues.”
All other servers across SBG1 and SBG4 “should be restarted very quickly,” he said. Watch his whole video below. ®
[5]
Update 16 mars 8:30pm
Here, the 10min's video with last updates about the fire in our DC in SBG2. [6]pic.twitter.com/BPaa9pEBZk — Octave Klaba (@olesovhcom) [7]March 16, 2021
Get our [8]Tech Resources
[1] https://www.theregister.com/2021/03/15/ovh_restoration_roadmap/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YFHhTFO6VWBJRXGr4PxtfQAAAMo&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YFHhTFO6VWBJRXGr4PxtfQAAAMo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2021/01/20/ovhcloud-tape-storage-as-a-service/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YFHhTFO6VWBJRXGr4PxtfQAAAMo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://t.co/BPaa9pEBZk
[7] https://twitter.com/olesovhcom/status/1371934994751500288?ref_src=twsrc%5Etfw
[8] https://whitepapers.theregister.com/
Re: “Some customers do not understand what they bought”
Can I just say that the Pedants are reVolting!
Re: “Some customers do not understand what they bought”
20,000V ÷ 15,000 new servers = 1.3̇ volts per server which is really quite impressively low power consumption!
Re: “Some customers do not understand what they bought”
If the servers are wired in series to the high voltage supply, which is very unlikely, and would be an utterly dumb thing to do for all sorts of reasons.
Re: “Some customers do not understand what they bought”
Reminds me of the problem trying to work out which bulb in the Christmas lights has gone, and that was only for 40 of them :-)
Re: “Some customers do not understand what they bought”
I'm not an OVH customer but to be fair at least three others I've used don't backup your stuff by default, it's a service you pay for. Quite reasonable as you might choose to take care of it yourself... as I did.
They also don't push you to install in two DCs, that's an option you can choose as you may want to use a 2nd DC from another provider.
So, it seems to me that a lot of these providers assume you know what you're doing....... and it seems a lot of OVH customers didn't :-)
Re: “Some customers do not understand what they bought”
Pedantic note: The power unit is Watt, not Volt
Even more pedantic note: Watts is not necessarily Volts x Amps and this is [1]rather important if you're powering a data centre.
[1] https://download.schneider-electric.com/files?p_File_Name=VAVR-9EJ262_R1_EN.pdf&p_Doc_Ref=SPD_VAVR-9EJ262_EN
Re: “Some customers do not understand what they bought”
I know 'experts' who supply standby power-generators that really don't understand power factor and it's relationship with electrical power. They seem to have a locked-in reaction and that all loads have a power factor of 0.8, therefore you must have a bigger alternator. Which then runs at part-load.
Re: “Some customers do not understand what they bought”
Not in my experience. Customers tend to ignore the need for backups as unnecessary extra expense.
I've even had one tell me they took out the backup option (and cost) "because we thought you'd do it anyway".
Anyone knows what type of UPSes were involved?
Yesterday an UPS of mine after several power glitches in a short time because of the strong winds in the area failed, was stuck in battery mode, and started to smell badly.... was quickly disconnected, after OVH events.
20,000 Volts is a substantial grid connection. 20,000 watts is nearly possible through a modern domestic connection (230V gives 87A).So, maybe a little confusion here?
It is a rather confusing statement, I would assume the '20 000 volts' refers to an incoming HV connection from their national grid, which a local substation converts down to 400V three phase. maybe.
20kV
OVH says it has a 20kV power supply to its Strasbourg data center operation. It is substantial, yes.
C.
UPS in the data centre?
I'm surprised that the UPSes were in the same building as the servers. My previous office had UPSes, batteries & generators in a separate power house across the car park. If it had burned the office/data centre would have been OK.
Re: UPS in the data centre?
Just as well nobody dug a trench across the car park.
Re: UPS in the data centre?
UPS in the building is pretty standard design, and nothing wrong with that, providing you take the proper precautions. I suspect we may find out in this instance that something went wrong with those precautions being in place.
in the hands of police
I've been trying to get someone of authority to take a look at OVH for some time to no avail.
Perhaps all the abuse complaints from millions of victims came flooding in all at one time causing overload?
I also don't think that some of OVH's "customers" would appreciate having backups of what they host on there.
let it burn
“Some customers do not understand what they bought”
Could they have been misled by salespeople?
Nah.
Pedantic note: The power unit is Watt, not Volt.