Researchers in Italy have developed a physical system to mimic properties of human brains that they hope will massively reduce the power costs of neural networks fundamental to AI development.

Successful approaches to neural networks have largely depended on software representations of brain synapses on top of a conventional stack of digital computing hardware and software.

However, [1]a paper published in Nature Materials this week shows that neural networks can be built using analogue computing based on a physical mesh of silver nanowires which, when viewed under an electron microscope, look rather appropriately like a plate of spaghetti.

[2]

Scanning electron microscopy image of a highly interconnected memristive nanowire network reservoir (scale bar, 2 μm) Image: Milano et al

Nodes between the wire are "memristive" in nature, which means their resistance depends on the voltage which has passed through them. The resistive switching mechanism at the nanowire junctions is modulated by the formation/rupture of a silver conductive path across the nanowire shell layer, under the action of the applied electric field.

Gianluca Milano, a post-doctoral researcher at Istituto Nazionale di Ricerca Metrologica in Torino, told The Register the main goal was to massively reduce the number of training parameters needed to get neural networks to make sense of input data.

[3]

"For your natural networks, the most expensive part, in terms of power costs, is training: usually you have thousands of parameters that you have to train. And this is the root of the [4]AI power consumption problem ," he said.

[5]

[6]

The researcher's response was to divide computation into two parts. In the first, the input is processed by means of short-term memory, which is taken care of by the physical reservoir, which does not have to be trained. Only long-term memory requires training in terms of "a fine-tuning of parameters," Milano said.

"In this sense, you can reduce the number of parameters that you have to train by a lot, and also you can simplify the hardware required," he said.

[7]

Using the approach, a 4x4 training input grid would only required three training parameters, rather than 16, the paper shows.

[8]University, Nvidia team teaches robots to get a grip with OpenAI's CLIP

[9]Intel offers Loihi 2 to boffins: A 7nm chip with more than 1m programmable neurons

[10]If it's going to rain within the next 90 mins, this very British AI system can warn you

[11]US drug watchdog green-lights first prostate-cancer-predicting AI software

The system was trained to recognise handwritten numerals from 0 to 9, with another benchmark being the prediction of the Mackey–Glass time series, originally developed to model the variation in the relative quantity of mature cells in the blood – and considered difficult to predict with "conventional machine-learning algorithms".

The study is part of a trend looking into neuromorphic computing, which takes direct inspiration from brain structures and physics, rather than modelling these processes on a conventional computing stack.

[12]

From the paper: Fully memristive reservoir computing implementation and spatio-temporal evolution of the nanowire network reservoir state (click to enlarge) Image: Milano et al

The principle demonstrated in the paper promises a reduction in power consumption of neural networks by several orders of magnitude.

Fellow report author Carlo Ricciardi, associate professor at Politecnico di Torino, said that in simulated neural networks, the energy cost was around 1 milli-joule per synaptic event. The energy required for each biological brain synaptic event was to the order of 10 to the -13 Joules. The research suggests systems might be built requiring 10 times that amount.

It's not quite the efficiency — and a long way from the scale — of the human brain. But perhaps an interesting step in the right direction. ®

Get our [13]Tech Resources



[1] https://www.nature.com/articles/s41563-021-01099-9

[2] https://regmedia.co.uk/2021/10/05/spag.png

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YVx2yDChwsfOkwq37ajJSAAAAJg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[4] https://www.theregister.com/2021/09/13/ai_environmental_cost/

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YVx2yDChwsfOkwq37ajJSAAAAJg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YVx2yDChwsfOkwq37ajJSAAAAJg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YVx2yDChwsfOkwq37ajJSAAAAJg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[8] https://www.theregister.com/2021/10/01/uw_nvidia_clip_model_paper/

[9] https://www.theregister.com/2021/10/01/artificial_brain_intel/

[10] https://www.theregister.com/2021/09/30/ai_model_rain_deepminnd/

[11] https://www.theregister.com/2021/09/30/fda_prostate_cancer/

[12] https://regmedia.co.uk/2021/10/05/fig3.png

[13] https://whitepapers.theregister.com/