News: 1746530951

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Brain-inspired neuromorphic computer SpiNNaker overheated when coolers lost their chill

(2025/05/06)


Exclusive The brain-inspired SpiNNaker machine at Manchester University suffered an overheating incident over the Easter weekend that will send a chill down the spines of datacenter administrators.

Brain-inspired chips promise ultra-efficient AI, so why aren't they everywhere? [1]READ MORE

According to Professor Steve Furber, now retired (although he told El Reg "SpiNNaker is still seen as my baby!"), a failure with the cooling on April 20 led to a rise in temperatures until the servers were manually shut down the following day.

The [2]SpiNNaker (Spiking Neural Network Architecture) project is all about simulating a brain by connecting hundreds of thousands of Arm cores. While a human brain presents a huge challenge, Furber, one of the designers of the original Arm processor, reckoned a mouse brain was possible.

During a talk earlier this month to celebrate the [3]40th anniversary of the switch-on of the first Arm processor , Furber told the audience that the hope was to model "one whole mouse" at the required level of detail.

Assuming the hardware survived its baking.

[4]

"SpiNNaker," he told The Register , "is hosted in the Kilburn Building, which was completed in 1972 as a purpose-built computer building and, as such, has a plant room that supplies chilled water as a utility to all the central machine rooms.

[5]

[6]

"The SpiNNaker room was built to house the machine in 2016 in what used to be the mechanical workshop, and is cooled by circulating hot air from the back of the cabinets through a plenum chamber into chillers at either end that blow the air through a cooling system using the building's chilled water."

The problem was with the chilled water supply. Furber said, "If the chilled water isn't actually chilled, the chiller fans are adding to the problem rather than helping solve it."

[7]

And so the temperatures began to rise inexorably. Without an automatic shutdown, the servers struggled on. Furber told us that he believed there was an automatic over-temperature shutdown on the individual SpiNNaker boards, and said, "This may have protected the SpiNNaker hardware from damage," but even with the hard-to-replace boards off, the network switches and power supplies remained powered up.

[8]808 lines of BBC BASIC and a dream: Arm architecture turns 40

[9]Intel's neuromorphic 'owl brain' swoops into Sandia labs

[10]SpiNNcloud Systems unveils Arm-based 'neuromorphic supercomputer'

[11]Tencent explores a future where HPC, quantum, cloud and edge have converged

The latter two component types suffered some damage, and without them, the SpiNNaker boards cannot all be tested, "so there may be more issues hidden behind the ones we know about."

Furber added, "We have had a few issues with the cooling system in the nine years that the machine has been operational, but these have not previously led to any damage." He reckoned that the long Easter weekend (in the UK, where Easter Friday and Easter Monday are both public holidays) might have contributed to the length of time it took to contain the temperature rise.

"We are looking into ways to fully automate the shutdown process in the future!"

As for the system's current state, Furber told us, "The machine is back up for internal users at around 80 percent of full capacity but still undergoing tests."

[12]

The good news is that the software is designed to work around partial hardware failures. The bad news is that replacing the failed parts will likely require further shutdowns. ®

Get our [13]Tech Resources



[1] https://www.theregister.com/2022/09/12/brain_chips_interview/

[2] https://www.theregister.com/2011/07/07/arm_project_spinnaker_super/

[3] https://www.theregister.com/2025/04/29/arm_40/

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/aiinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aBoyHU8kYPLX6harNY1iTwAAABU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/aiinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aBoyHU8kYPLX6harNY1iTwAAABU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/aiinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aBoyHU8kYPLX6harNY1iTwAAABU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/aiinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aBoyHU8kYPLX6harNY1iTwAAABU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[8] https://www.theregister.com/2025/04/29/arm_40/

[9] https://www.theregister.com/2024/04/17/intel_hala_point_neuromorphic_owl/

[10] https://www.theregister.com/2024/05/08/spinncloud_systems_unveils_armbased_neuromorphic/

[11] https://www.theregister.com/2024/01/29/tencent_huawei_tech_predictions/

[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/aiinfrastructuremonth&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aBoyHU8kYPLX6harNY1iTwAAABU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[13] https://whitepapers.theregister.com/



TRT

Perhaps they should have modelled Acomys russatus ? Able to operate in desert temperatures exceeding 42°C

Anonymous Coward

Had the same idea ... but I hear it's fully booked serving as [1]hairpiece to the Orange Incubus ... lotsa wasted heat to dissipate from that overagitated empty skullcap in the oblong orifice.

[1] https://www.biolib.cz/IMG/GAL/BIG/390595.jpg

kotenok2000

Or model a Portia genus spider? They are remarkable for their intelligence, and do that wih 100000 neurons. For comparsion mouse brain has about 70 million neurons.

https://en.wikipedia.org/wiki/Portia_(spider)#Intelligence

TRT

I had to refresh my memory of these little beasties. Remarkable things aren't they?!

And very modern.. according to Wikipedia they implement a "cryptic REST posture", which of course is de facto now when it comes to API security.

Auto-Slowdown/Shutdown Systems

An_Old_Dog

... were not fully implemented here because HVAC failures, brownouts, and UPS/line-conditioning failures never happen in the real world.

Re: Auto-Slowdown/Shutdown Systems

that one in the corner

Oh, be fair: it was two British Bank Holidays in a row, perfectly reasonable to expect cold weather, high winds and rain. Snow at Easter isn't unknown. They probably had emergency temperature control plans: an undergraduate poised to run in with a space-heater

Re: Auto-Slowdown/Shutdown Systems

HuBo

Yeah, and lucky this didn't happen a week later during the (could've been) end of times UK [1]heatwave apocalypse and resultant [2]Iberian peninsula synch fail blackout armageddon ! Especially if this mouse-brain neuromorph had been a GPU-driven computational fusion energy megaspace heater of doom, rather than power-sipping spike machinery ...

Safe to say we've averted a major [3]" New-Zealand China Syndrome " on this one, with Manchester core meltdown shooting cataclysmic jets of devastation, all over Wellington NZ. Missed it by that much!

I hope they get the remaining 20% of the Muridae -brain beastmachine back up soon, but things could have been just so much worse imho ...

[1] https://www.lbc.co.uk/weather/spring-heatwave-break-uk-april-may-records/

[2] https://www.theregister.com/2025/04/28/nationwide_power_outages_knock_spain/

[3] https://en.wikipedia.org/wiki/Nuclear_meltdown#China_syndrome

Re: Auto-Slowdown/Shutdown Systems

Red Ted

Snow at Easter isn't unknown.

In the UK it is statistically more likely that at Christmas.

and nothing of value lost

Reaps

seems to be the usual worthless Ai garbage

so even if it had burn't building to the ground nothing of value would have been lost.

Re: and nothing of value lost

Anonymous Coward

Says more about your knowledge than the Spinnaker project.

FWIW - spinnaker is a simulation of biological neurons.

If God had wanted you to go around nude, He would have given you bigger hands.