Google DeepMind touts AI model for 'better' global weather forecasting
(2024/12/05)
- Reference: 1733383634
- News link: https://www.theregister.co.uk/2024/12/05/google_deepmind_weather_model/
- Source link:
Google DeepMind researchers claim they’ve used machine learning to devise a model that can deliver better 15-day weather forecasts and requires only modest quantities of compute resources to produce its predictions.
The model, dubbed GenCast, provides a probabilistic ensemble weather forecast – a distribution of probable weather – rather than a [1]deterministic [2]numerical weather prediction (NWP) calculation that points to a specific outcome – at risk of error.
One of the paper’s co-authors, DeepMind researcher Ilan Price, told The Register : "GenCast is a machine learning-based weather model, which learns directly from historical weather data. This is in contrast to traditional models, which make forecasts by solving physics equations.”
[3]
"One limitation of these traditional models is that the equations they solve are only approximations of the atmospheric dynamics. GenCast is not limited to learning dynamics/patterns that are known exactly and can be written down in an equation.
It has the opportunity to learn more complex relationships and dynamics directly from the data
"Instead it has the opportunity to learn more complex relationships and dynamics directly from the data, and this allows GenCast to outperform traditional models."
In a [4]DeepMind write-up , Price and co-author Matthew Wilson rate the model “a critical advance in AI-based weather prediction that builds on our previous weather model [GraphCast], which was deterministic, and provided a single, best estimate of future weather. By contrast, a GenCast forecast comprises an ensemble of 50 or more predictions, each representing a possible weather trajectory."
[5]
[6]
There's more to this than Googlers boasting about their work on the company blog. GenCast is also the subject of a [7]a paper published in the journal Nature that argues the model "significantly outperforms the top operational ensemble NWP model, [8]ENS ."
The European Centre for Medium-Range Weather Forecasts ( [9]ECMWF ) produces ENS, an ensemble of 51 NWP-based forecasts that together represent a range of possibilities and is relied upon by 23 member states and 12 co-operating states in Europe.
[10]
According to the DeepMind paper, "GenCast … generates global 15-day ensemble forecasts at 0.25° resolution, which are more accurate than the top operational ensemble system, ENS of ECMWF."
The paper claims GenCast beat the ENS on 97.2 percent of the evaluated targets. It also does better than ENS in predicting the path of tropical cyclones, according to the researchers.
[11]AWS says AI could disrupt everything – and hopes it will do just that to Windows
[12]Amazon promises 4x faster AI silicon in 2025, turns Trainium2 loose on the net
[13]NASA's Solar Dynamics Observatory datacenter flooded, offline until 2025
[14]IBM sued again in storm over Weather Channel data sharing
However GenCast gets the job done, better forecasts are welcome as weather events have significant socio-economic consequences. According to [15]a report last month commissioned by the International Chamber of Commerce, "climate-related extreme weather events have cost the global economy more than $2 trillion over the past decade."
The DeepMind researchers’ work therefore has the potential to help mitigate the consequences of severe weather by assisting people and businesses better prepare for adverse conditions. It may also help with renewable energy planning, through improved wind-power forecasting, for example.
In other Google AI news... DeepMind has [16]put out Genie 2 , which can apparently generate 3D playable worlds from prompt images. People can use keyboard and mouse inputs to navigate, and AI agents can autonomously inhabit the digital domains.
Also, the Big G has put [17]Veo , its [18]Sora -like [19]video generating model , into the hands of testers within businesses in a private preview.
(OpenAI's hyped-up-but-not-shipped Sora may arrive this month; the super lab is tediously cryptically [20]promising 12 days of stuff big and small in some form or another.)
And it appears GenCast does so quite affordably. NWP compute costs [21]can be significant – potentially hundreds of thousands of dollars annually in compute and data egress charges, depending on the scenario. ENS ensemble forecasts at a resolution of 0.2° or 0.1°, "take hours on a supercomputer with tens of thousands of processors," observe Price and Wilson.
"It takes a single Google Cloud TPU v5 just 8 minutes to produce one 15-day forecast in GenCast’s ensemble, and every forecast in the ensemble can be generated simultaneously, in parallel," they say.
Google Cloud bills Cloud TPUv5 instances at around [22]$1-3 per chip-hour .
[23]
Asked to provide a more specific figure, Price told The Register , "We don't have a concrete comparison in terms of monetary cost or energy, but the amount of computation necessary differs by orders of magnitude."
Price and Wilson say that DeepMind has released the GenCast model code and weights to help accelerate research and development for the weather and climate community. And the release of real-time and historical forecasts from GenCast and prior models is planned. ®
Get our [24]Tech Resources
[1] https://www.worldclimateservice.com/2021/10/12/difference-between-deterministic-and-ensemble-forecasts/
[2] https://www.ncei.noaa.gov/products/weather-climate-models/numerical-weather-prediction
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://deepmind.google/discover/blog/gencast-predicts-weather-and-the-risks-of-extreme-conditions-with-sota-accuracy/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.nature.com/articles/s41586-024-08252-9
[8] https://www.ecmwf.int/en/forecasts/documentation-and-support/medium-range-forecasts
[9] https://www.ecmwf.int/en/forecasts/documentation-and-support/medium-range-forecasts
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2024/12/04/amazon_leans_into_ai/
[12] https://www.theregister.com/2024/12/03/amazon_ai_chip/
[13] https://www.theregister.com/2024/12/03/nasa_sdo_outage/
[14] https://www.theregister.com/2024/11/08/ibm_weather_channel_privacy/
[15] https://iccwbo.org/news-publications/policies-reports/new-report-extreme-weather-events-cost-economy-2-trillion-over-the-last-decade/
[16] https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
[17] https://deepmind.google/technologies/veo/
[18] https://www.theregister.com/2024/02/15/google_debuts_gemini_15_pro/
[19] https://www.theregister.com/2024/05/15/google_ai_developer/
[20] https://twitter.com/OpenAI/status/1864328928267259941
[21] https://journals.ametsoc.org/view/journals/atot/36/3/jtech-d-18-0142.1.xml
[22] https://cloud.google.com/tpu/pricing
[23] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[24] https://whitepapers.theregister.com/
The model, dubbed GenCast, provides a probabilistic ensemble weather forecast – a distribution of probable weather – rather than a [1]deterministic [2]numerical weather prediction (NWP) calculation that points to a specific outcome – at risk of error.
One of the paper’s co-authors, DeepMind researcher Ilan Price, told The Register : "GenCast is a machine learning-based weather model, which learns directly from historical weather data. This is in contrast to traditional models, which make forecasts by solving physics equations.”
[3]
"One limitation of these traditional models is that the equations they solve are only approximations of the atmospheric dynamics. GenCast is not limited to learning dynamics/patterns that are known exactly and can be written down in an equation.
It has the opportunity to learn more complex relationships and dynamics directly from the data
"Instead it has the opportunity to learn more complex relationships and dynamics directly from the data, and this allows GenCast to outperform traditional models."
In a [4]DeepMind write-up , Price and co-author Matthew Wilson rate the model “a critical advance in AI-based weather prediction that builds on our previous weather model [GraphCast], which was deterministic, and provided a single, best estimate of future weather. By contrast, a GenCast forecast comprises an ensemble of 50 or more predictions, each representing a possible weather trajectory."
[5]
[6]
There's more to this than Googlers boasting about their work on the company blog. GenCast is also the subject of a [7]a paper published in the journal Nature that argues the model "significantly outperforms the top operational ensemble NWP model, [8]ENS ."
The European Centre for Medium-Range Weather Forecasts ( [9]ECMWF ) produces ENS, an ensemble of 51 NWP-based forecasts that together represent a range of possibilities and is relied upon by 23 member states and 12 co-operating states in Europe.
[10]
According to the DeepMind paper, "GenCast … generates global 15-day ensemble forecasts at 0.25° resolution, which are more accurate than the top operational ensemble system, ENS of ECMWF."
The paper claims GenCast beat the ENS on 97.2 percent of the evaluated targets. It also does better than ENS in predicting the path of tropical cyclones, according to the researchers.
[11]AWS says AI could disrupt everything – and hopes it will do just that to Windows
[12]Amazon promises 4x faster AI silicon in 2025, turns Trainium2 loose on the net
[13]NASA's Solar Dynamics Observatory datacenter flooded, offline until 2025
[14]IBM sued again in storm over Weather Channel data sharing
However GenCast gets the job done, better forecasts are welcome as weather events have significant socio-economic consequences. According to [15]a report last month commissioned by the International Chamber of Commerce, "climate-related extreme weather events have cost the global economy more than $2 trillion over the past decade."
The DeepMind researchers’ work therefore has the potential to help mitigate the consequences of severe weather by assisting people and businesses better prepare for adverse conditions. It may also help with renewable energy planning, through improved wind-power forecasting, for example.
In other Google AI news... DeepMind has [16]put out Genie 2 , which can apparently generate 3D playable worlds from prompt images. People can use keyboard and mouse inputs to navigate, and AI agents can autonomously inhabit the digital domains.
Also, the Big G has put [17]Veo , its [18]Sora -like [19]video generating model , into the hands of testers within businesses in a private preview.
(OpenAI's hyped-up-but-not-shipped Sora may arrive this month; the super lab is tediously cryptically [20]promising 12 days of stuff big and small in some form or another.)
And it appears GenCast does so quite affordably. NWP compute costs [21]can be significant – potentially hundreds of thousands of dollars annually in compute and data egress charges, depending on the scenario. ENS ensemble forecasts at a resolution of 0.2° or 0.1°, "take hours on a supercomputer with tens of thousands of processors," observe Price and Wilson.
"It takes a single Google Cloud TPU v5 just 8 minutes to produce one 15-day forecast in GenCast’s ensemble, and every forecast in the ensemble can be generated simultaneously, in parallel," they say.
Google Cloud bills Cloud TPUv5 instances at around [22]$1-3 per chip-hour .
[23]
Asked to provide a more specific figure, Price told The Register , "We don't have a concrete comparison in terms of monetary cost or energy, but the amount of computation necessary differs by orders of magnitude."
Price and Wilson say that DeepMind has released the GenCast model code and weights to help accelerate research and development for the weather and climate community. And the release of real-time and historical forecasts from GenCast and prior models is planned. ®
Get our [24]Tech Resources
[1] https://www.worldclimateservice.com/2021/10/12/difference-between-deterministic-and-ensemble-forecasts/
[2] https://www.ncei.noaa.gov/products/weather-climate-models/numerical-weather-prediction
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://deepmind.google/discover/blog/gencast-predicts-weather-and-the-risks-of-extreme-conditions-with-sota-accuracy/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.nature.com/articles/s41586-024-08252-9
[8] https://www.ecmwf.int/en/forecasts/documentation-and-support/medium-range-forecasts
[9] https://www.ecmwf.int/en/forecasts/documentation-and-support/medium-range-forecasts
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2024/12/04/amazon_leans_into_ai/
[12] https://www.theregister.com/2024/12/03/amazon_ai_chip/
[13] https://www.theregister.com/2024/12/03/nasa_sdo_outage/
[14] https://www.theregister.com/2024/11/08/ibm_weather_channel_privacy/
[15] https://iccwbo.org/news-publications/policies-reports/new-report-extreme-weather-events-cost-economy-2-trillion-over-the-last-decade/
[16] https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
[17] https://deepmind.google/technologies/veo/
[18] https://www.theregister.com/2024/02/15/google_debuts_gemini_15_pro/
[19] https://www.theregister.com/2024/05/15/google_ai_developer/
[20] https://twitter.com/OpenAI/status/1864328928267259941
[21] https://journals.ametsoc.org/view/journals/atot/36/3/jtech-d-18-0142.1.xml
[22] https://cloud.google.com/tpu/pricing
[23] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1GH1SqfLBQIO550D_8BLwAAARU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[24] https://whitepapers.theregister.com/
Re: Past performance
Headley_Grange
I read somewhere that in the UK if you said that tomorrow's weather would be like today's then you'd be right 65% of the time.
Hang on a minute
sabroni
"By contrast, a GenCast forecast comprises an ensemble of 50 or more predictions, each representing a possible weather trajectory."
This is so much better than having one forecast, look at the choice!!!
Re: Hang on a minute
Richard 12
The existing forecasts also produce a large spread of plausible outcomes, with probabilities.
The forecast we see on the TV is the one believed "most probable". It is not the "only".
Re: Hang on a minute
Locomotion69
And the best short time forecast is just looking out of a window.
You will not believe the amount of people I see looking at their phone for the short time weather expectations, even sitting next to a window.
Re: Hang on a minute
Headley_Grange
The looking out of the window does depend very much on where you live. It's not recommended if you live in Scotland.
Re: Hang on a minute
Mage
Or in high buildings in Russia.
But really I don't trust this and it's likely to be dangerous sometimes.
See also IBM claims for Medical AI for "Watson" after "winning" Jeopardy. Turned out mostly only shared branding and didn't work.
A lot of these so called AI systems are doing pattern matching based on past human selection or results. This is ultimately doomed, especially if after a while there are no human experts to scrape the data from.
All the "AI" purveyors have been proven to mislead, hype or even lie. Even the terms used are a lie.
Cool
codejunky
Hopefully it succeeds. Could be a good product from the AI boom.
Past performance
is no guarantee of future results.