Energy breakthrough needed to build AGI, says OpenAI boss Altman
- Reference: 1705901617
- News link: https://www.theregister.co.uk/2024/01/22/google_deepmind_train_alphageometry_system/
- Source link:
In a panel discussion with Bloomberg at Davos last week, he argued "There's no way to get there without a breakthrough." Altman is in favor of renewable energy sources like nuclear fusion, and he is motivated to keep investing in the technology. He personally invested $375 million into Helion Energy – a nuclear fusion startup that has signed a [1]deal to supply energy to Microsoft in the next few years.
AI models made up of billions of parameters require huge amounts of energy to train. OpenAI's old GPT-3 system reportedly consumed 936 megawatt hours (MWh), [2]according to AI company Numenta. The US Energy Information Administration [3]estimates that the average household consumes about 10.5 MWh per year. That means training GPT-3 consumed as much energy as about 90 households consume in a year.
[4]
Larger models will require even more energy. "We're not done with scaling [LLMs] – we still need to push up," Aiden Gomez, CEO of [5]Cohere , [6]declared during another discussion at Davos.
AlphaGeometry represents a breakthrough in AI reasoning
Google DeepMind researchers have trained an AI system to prove geometric theorems at nearly the same level achieved by human mathematics Olympiad gold medalists.
In a paper [7]published in Nature last week, the team DeepMind unveiled AlphaGeometry – a system made up of a language model and a symbolic deduction engine. The former generates potential mathematical strategies to solve a particular problem, while the latter attempts to deduce a final solution.
[8]
[9]
"With AlphaGeometry, we demonstrate AI's growing ability to reason logically, and to discover and verify new knowledge," [10]wrote co-authors Trieu Trinh and Thang Luong. "Solving Olympiad-level geometry problems is an important milestone in developing deep mathematical reasoning on the path towards more advanced and general AI systems."
Interestingly, the system was trained on 100 million samples of synthetic data, depicting random geometric diagrams. AlphaGeometry was tasked with learning all the relationships between the points and lines in shapes to figure out all the geometric proofs.
[11]
In a test benchmarking the system's performance, it managed to solve 25 out of 30 geometry questions from Olympiad competitions – given a few hours. For comparison, the average human gold medalist can solve about 25.9 of these in the same time.
Google DeepMind has released the code for its model [12]here .
[13]Google teases AlphaCode 2 – a code-generating AI revamped with Gemini
[14]Equinix to expand liquid cooling to 100 datacenters to quench your AI thirst
[15]Google AI chatbot more empathetic than real doctors in tests
[16]Amazon Ads rolls out generative AI for ad image composition
Medical AI chatbots may not democratize healthcare
The World Health Organization (WHO) isn't optimistic that medical AI systems will be good for poorer countries if they are built by orgs in wealthier nations that fail to train them on more diverse data.
Developers like [17]Google believe that AI can help those that have limited access to healthcare in the future. But officials at the WHO believe that the technology may not adequately serve them – especially if they aren't representative of the patients used to source clinical data to train these systems.
"The very last thing that we want to see happen as part of this leap forward with technology is the propagation or amplification of inequities and biases in the social fabric of countries around the world," declared Alain Labrique, the WHO's director for digital health and innovation, Nature [18]reported .
[19]
Labrique and his colleagues argued that the development of medical AI shouldn't be dominated by large tech businesses, and that their technologies should be audited by independent third parties before they are released. Developers are currently building models capable of automatically generating clinical notes from meetings, aiding doctors to diagnose diseases, and more.
Potential issues – such as different accents, languages, or medical histories that aren't in its training data – could potentially throw these systems off, leading to lower performance and poor patient outcomes.
Amazon rolls out experimental AI shopping assistant
Consumers can now quiz an AI chatbot about a particular item sold on Amazon with the online marketplace's mobile app.
The "Looking for specific info" tab, which used to show product reviews and answers to common questions, has been replaced with a large language model, Marketplace Pulse [20]first reported . The system seems to work by ingesting and summarizing information from the product's listing page.
Netizens can ask questions about the item they're interested in. The chatbot doesn't compare products or suggest alternatives. Nor can it carry out actions like adding items to shoppers' virtual carts or disclose pricing history. An Amazon spokesperson confirmed to CNBC that it was testing the chatbot.
"We're constantly inventing to help make customers' lives better and easier, and are currently testing a new feature powered by generative AI to improve shopping on Amazon by helping customers get answers to commonly asked product questions," [21]explained Maria Boschetti. Like all chatbots, Amazon's latest system is prone to hallucination – so take what it says with a pinch of salt.
Interestingly, the virtual shopping assistant's capabilities are quite open ended. It can reportedly write jokes, poems, or even generate code based on information on a product across multiple languages. ®
Get our [22]Tech Resources
[1] https://www.theregister.com/2023/05/22/microsoft_helion_fusion_deal/
[2] https://www.numenta.com/blog/2022/05/24/ai-is-harming-our-planet/
[3] https://www.eia.gov/energyexplained/use-of-energy/electricity-use-in-homes.php
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Za5K30QwggdJBRC2hUA7yQAAAFM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://www.theregister.com/2022/03/03/language_model_gpt3/
[6] https://www.weforum.org/agenda/2024/01/artificial-intelligence-ai-innovation-technology-davos-2024/
[7] https://www.nature.com/articles/s41586-023-06747-5
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Za5K30QwggdJBRC2hUA7yQAAAFM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Za5K30QwggdJBRC2hUA7yQAAAFM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Za5K30QwggdJBRC2hUA7yQAAAFM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://github.com/google-deepmind/alphageometry
[13] https://www.theregister.com/2023/12/07/alphacode_2_software/
[14] https://www.theregister.com/2023/12/13/equinix_liquid_cooling/
[15] https://www.theregister.com/2024/01/16/google_ai_chatbot_heathcare/
[16] https://www.theregister.com/2023/10/26/amazon_ads_generative_ai/
[17] https://www.theregister.com/2024/01/16/google_ai_chatbot_heathcare/
[18] https://www.nature.com/articles/d41586-024-00161-1
[19] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Za5K30QwggdJBRC2hUA7yQAAAFM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[20] https://www.marketplacepulse.com/articles/amazon-adds-an-ai-shopping-assistant
[21] https://www.cnbc.com/2024/01/16/amazon-launches-generative-ai-tool-to-answer-shoppers-questions.html
[22] https://whitepapers.theregister.com/
If we had an energy breakthrough and had access to 10x the energy we did today but clean without any CO2 or radioactive waste, but used that additional 900% energy not to make the lives of people better but to run LLMs in the misguided hope that somehow enough computing power will make them "intelligent" we deserve to have our nukes used against us and terminators built to kill the rest.
Doctor heal thyself
AI provide your own energy breakthrough!
Re: Doctor heal thyself
By my calculations, if we ignore the first law of thermodynamics, we can achieve an infinite-energy breakthrough. Even if the probability of such a thing happening is zero, 0 × ∞ = a nonzero value, so there is still a net gain.
Re: Doctor heal thyself
Yes, and if we ignore the laws of gravity* we could fly.
*or temporarily forget about them, say, by being surprised and distracted at the moment of falling
Re: Doctor heal thyself
"0 × ∞ = a nonzero value"
No, zero times *anything* is still zero. That's real maths for you. If you don't have an infinity, you just don't have it. Unless you've re-defined zero to equal a value other than zero, that is...?
Re: Doctor heal thyself
There'll be plenty of energy for AGIs once the humans are eliminated...
Energy breakthrough
AGI is the new Bitcoin, but with more Snake Oil.
Wrong way around!!!
Saying an energy breakthrough is needed for AGI is getting it completely backwards.... it assumes that the current statistical models are the way to build AGI, and just throwing more data and processing (and therefore more power) at it will get there.
"OpenAI's old GPT-3 system reportedly consumed 936 megawatt hours (MWh) [to train]"
A human brain consumes around 20W, and even assuming the "training time" is every living moment from birth to a wise old 80 years, is a total consumption of just over 14MWh. To arrive at a reasonably competent 20-year old trained brain consumes 3.5 MWh. So real human intelligence is not only far better than current-best LLMs, it is also more than 2 orders of magnitude less energy-intensive. Also interesting to note that humans do not need giant volumes of text to train, and indeed being able to memorise and regurgitate vast amounts of information is an exercise in memory, not intelligence.
What is needed is a completely different approach, not 'more-of-the-same'... and (in the spirit of being constructive not just criticising), I would note that what humans ARE being trained on is vast amounts of what is not only 'video' but "3-D" / depth-perceptive video combined with all of the internal sensory input that informs the brain about the current state of the body. As far as I can work out, most AI researchers are trying to build a 'superbrain-in-a-jar' kind of intelligence, but I think that it won't be possible to build that completely divorced from physical interaction with the real world.
This guy seems to have discovered the centre around which the rrest of the universe revolves and it's himself.
"in favor of renewable energy sources like nuclear fusion"
Fusion is definitely not "renewable energy"; it is taking energy locked up shortly after the Big Bang and dumping it into our environment. Free of chemical and radioactive pollution, yes; but renewable, no.
Wind, water, solar are renewable; they take energy out of the environment, and then release it back when we use it.
Re: "in favor of renewable energy sources like nuclear fusion"
Strictly speaking, the processes driving wind, solar and hydro are not renewable, unless you believe in a cyclic universe where the big bang starts with the remnants of the last gnab gib....
An energy breakthrough would be sorely needed, but "to build AGI" is pretty low on the list of reasons.