Open source AI makes modern PCs relevant, and subscriptions seem shabby
(2023/05/11)
- Reference: 1683786669
- News link: https://www.theregister.co.uk/2023/05/11/open_source_ai_makes_subscriptions_irrelevant/
- Source link:
Column This time last year the latest trend in computing became impossible to ignore: huge slabs of silicon with hundreds of billions of transistors – the inevitable consequence of another set of workarounds that kept Moore's Law from oblivion.
But slumping PC sales suggest we don't need these monster computers – and not just because of a sales shadow cast by COVID.
In the first half of 2022, corporate computing looked pretty much the same as it had for the last decade: basic office apps, team communication apps, and, for the creative class, a few rich media tools. Sure, gamers would always find a way to put those transistors to work, but the vast majority of hardware was already overpowered and underworked. Why waste transistors on solved problems?
[1]
Then the world changed. A year ago, OpenAI launched DALL-E, the first of the widely available generative AI tools – a "diffuser" that converts noise, a text prompt, and a massive database of weightings into images. It seemed almost like magic. Not long after, Midjourney offered much the same – though tuned to a decidedly '70s Prog Rock album cover aesthetic. It seemed as though demand for cloud computing would skyrocket as these tools found their way into products from Microsoft, Canva, Adobe and others.
[2]
[3]
Then the world changed again. In August, Stability AI introduced an open source database of diffuser weightings. At its start, Stable Diffusion demanded a state-of-the-art GPU, but the open source community soon found it could optimize the diffuser to run on, well, pretty much anything. It wouldn't necessarily be fast, but it would work – and it would scale up with your hardware.
Instead of demanding massive cloud resources, [4]these newer AI tools run locally . And if you purchased a monster computer they'd run at least as speedily as anything on offer from OpenAI or Midjourney – without a subscription.
[5]
The ever-excitable open source community driving Stable Diffusion created an impressive series of new diffuser weightings, each targeting a specific aesthetic. Stable Diffusion isn't merely as fast as anything offered by a commercial AI firm – it's both more useful and more extensible.
And then – yes, you guessed it – the world changed again. At the start of December, OpenAI's [6]ChatGPT completely rewrote our expectations for artificial intelligence, becoming the fastest web app to reach 100 million users. A large language model (LLM) powered by a "generative pre-trained transformer" – how many of us have forgotten that's what GPT stands for? – that trained its weightings on the vast troves of text available on the internet.
That training effort is estimated to have cost millions (possibly tens of millions) in Azure cloud computing resources. That cost of entry had been expected to be enough to keep competitors at bay – except perhaps for Google and Meta.
[7]
Until, yet again, the world changed. In March, Meta [8]released LLaMA – a much more compact and efficient language model, with a comparatively tiny database of weightings, yet with response quality approaching OpenAI's GPT-4.
With a model of only thirty billion parameters, LLaMA can comfortably sit in a PC with 32GB of RAM. Something very like ChatGPT – which runs on the Azure Cloud because of its massive database of weightings – can be run pretty much anywhere.
[9]It's time to reveal all recommendation algorithms – by law if necessary
[10]Once AI can create endless viral videos, good luck switching off social media
[11]Conversational AI tells us what we want to hear – a fib that the Web is reliable and friendly
[12]To preserve Earth's treasures, digital silence is golden
Meta's researchers offered their weightings to their academic peers, free to download. As LLaMA could run on their lab computers, researchers at Stanford immediately improved LLaMA through their new training technique called [13]Alpaca-Lora , which cut the cost of training an existing set of weightings from hundreds of thousands of dollars down to a few hundred dollars. They shared their code, too.
Just as DALL-E lost out to Stable Diffusion for usability and extensibility, ChatGPT looks to be losing another race, as researchers produce a range of models – such as Alpaca, [14]Vicuña , [15]Koala , and a menagerie of others – that train and re-train quickly and inexpensively.
They're improving far more rapidly than anyone expected. In part that's because they're training on many ChatGPT "conversations" that have been shared across sites like Reddit, and they can run well on most PCs. If you have a monster computer they run very well indeed.
The machines for which we couldn't dream up a use just a year ago have found their purpose: they're becoming the workhorses of all our generative AI tasks. They help us code, plan, write, draw, model, and much else besides.
And we won't be beholden to subscriptions to make these new tools work. Tt looks as though open source has already outpaced commercial development of both diffusers and transformers.
Open source AI has also reminded us of why the PC proliferated: by making it possible to bring home tools that were once only available in the office.
This won't close the door to commerce. If anything, it means that there's more scope for entrepreneurs to create new products, without worrying about whether they infringe on the business models underlying Google, Microsoft, Meta or anyone else. We're headed into a time of pervasive disruption in technology – and size doesn't seem to confer many advantages.
The monsters are on the loose. I reckon that's a good thing. ®
Get our [16]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2022/10/26/generative_ai_pcs_upgrades/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2022/12/03/in_brief_ai/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models
[9] https://www.theregister.com/2023/04/13/reveal_all_recommendation_algorithms/
[10] https://www.theregister.com/2023/03/08/generative_ai_viral_video/
[11] https://www.theregister.com/2023/02/08/ai_battle_microsoft_google/
[12] https://www.theregister.com/2022/09/14/digital_silence_is_golden/
[13] https://github.com/tloen/alpaca-lora
[14] https://lmsys.org/blog/2023-03-30-vicuna
[15] https://bair.berkeley.edu/blog/2023/04/03/koala
[16] https://whitepapers.theregister.com/
But slumping PC sales suggest we don't need these monster computers – and not just because of a sales shadow cast by COVID.
In the first half of 2022, corporate computing looked pretty much the same as it had for the last decade: basic office apps, team communication apps, and, for the creative class, a few rich media tools. Sure, gamers would always find a way to put those transistors to work, but the vast majority of hardware was already overpowered and underworked. Why waste transistors on solved problems?
[1]
Then the world changed. A year ago, OpenAI launched DALL-E, the first of the widely available generative AI tools – a "diffuser" that converts noise, a text prompt, and a massive database of weightings into images. It seemed almost like magic. Not long after, Midjourney offered much the same – though tuned to a decidedly '70s Prog Rock album cover aesthetic. It seemed as though demand for cloud computing would skyrocket as these tools found their way into products from Microsoft, Canva, Adobe and others.
[2]
[3]
Then the world changed again. In August, Stability AI introduced an open source database of diffuser weightings. At its start, Stable Diffusion demanded a state-of-the-art GPU, but the open source community soon found it could optimize the diffuser to run on, well, pretty much anything. It wouldn't necessarily be fast, but it would work – and it would scale up with your hardware.
Instead of demanding massive cloud resources, [4]these newer AI tools run locally . And if you purchased a monster computer they'd run at least as speedily as anything on offer from OpenAI or Midjourney – without a subscription.
[5]
The ever-excitable open source community driving Stable Diffusion created an impressive series of new diffuser weightings, each targeting a specific aesthetic. Stable Diffusion isn't merely as fast as anything offered by a commercial AI firm – it's both more useful and more extensible.
And then – yes, you guessed it – the world changed again. At the start of December, OpenAI's [6]ChatGPT completely rewrote our expectations for artificial intelligence, becoming the fastest web app to reach 100 million users. A large language model (LLM) powered by a "generative pre-trained transformer" – how many of us have forgotten that's what GPT stands for? – that trained its weightings on the vast troves of text available on the internet.
That training effort is estimated to have cost millions (possibly tens of millions) in Azure cloud computing resources. That cost of entry had been expected to be enough to keep competitors at bay – except perhaps for Google and Meta.
[7]
Until, yet again, the world changed. In March, Meta [8]released LLaMA – a much more compact and efficient language model, with a comparatively tiny database of weightings, yet with response quality approaching OpenAI's GPT-4.
With a model of only thirty billion parameters, LLaMA can comfortably sit in a PC with 32GB of RAM. Something very like ChatGPT – which runs on the Azure Cloud because of its massive database of weightings – can be run pretty much anywhere.
[9]It's time to reveal all recommendation algorithms – by law if necessary
[10]Once AI can create endless viral videos, good luck switching off social media
[11]Conversational AI tells us what we want to hear – a fib that the Web is reliable and friendly
[12]To preserve Earth's treasures, digital silence is golden
Meta's researchers offered their weightings to their academic peers, free to download. As LLaMA could run on their lab computers, researchers at Stanford immediately improved LLaMA through their new training technique called [13]Alpaca-Lora , which cut the cost of training an existing set of weightings from hundreds of thousands of dollars down to a few hundred dollars. They shared their code, too.
Just as DALL-E lost out to Stable Diffusion for usability and extensibility, ChatGPT looks to be losing another race, as researchers produce a range of models – such as Alpaca, [14]Vicuña , [15]Koala , and a menagerie of others – that train and re-train quickly and inexpensively.
They're improving far more rapidly than anyone expected. In part that's because they're training on many ChatGPT "conversations" that have been shared across sites like Reddit, and they can run well on most PCs. If you have a monster computer they run very well indeed.
The machines for which we couldn't dream up a use just a year ago have found their purpose: they're becoming the workhorses of all our generative AI tasks. They help us code, plan, write, draw, model, and much else besides.
And we won't be beholden to subscriptions to make these new tools work. Tt looks as though open source has already outpaced commercial development of both diffusers and transformers.
Open source AI has also reminded us of why the PC proliferated: by making it possible to bring home tools that were once only available in the office.
This won't close the door to commerce. If anything, it means that there's more scope for entrepreneurs to create new products, without worrying about whether they infringe on the business models underlying Google, Microsoft, Meta or anyone else. We're headed into a time of pervasive disruption in technology – and size doesn't seem to confer many advantages.
The monsters are on the loose. I reckon that's a good thing. ®
Get our [16]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2022/10/26/generative_ai_pcs_upgrades/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2022/12/03/in_brief_ai/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZFy8wdIUv-bpZUPTeR@nugAAAEY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models
[9] https://www.theregister.com/2023/04/13/reveal_all_recommendation_algorithms/
[10] https://www.theregister.com/2023/03/08/generative_ai_viral_video/
[11] https://www.theregister.com/2023/02/08/ai_battle_microsoft_google/
[12] https://www.theregister.com/2022/09/14/digital_silence_is_golden/
[13] https://github.com/tloen/alpaca-lora
[14] https://lmsys.org/blog/2023-03-30-vicuna
[15] https://bair.berkeley.edu/blog/2023/04/03/koala
[16] https://whitepapers.theregister.com/
Re: "ChatGPT looks to be losing another race"
Ken Hagan
They won't ask themselves that because the pricing will be arranged so that all the things MS want you to have are free (and "integral") with all the things that you wanted.
It's leveraging a monopoly in one area to acquire a monopoly in another area. It's illegal, but they have always got away with it in the past.
Can't happen fast enough
Ken Hagan
How long till we can have something like an Echo but all done locally so that you aren't spaffing your entire existence to some corporate data whore, and the damn thing eventually gets used to your accent and habits?
Not long, I'm guessing, and it will be FOSS that does it coz none of the corporates have an incentive.
Electrons being pushed
b0llchit
While it is very impressive, all the improvements and so, it should also be interesting to look at the cost.
All these monster machines are now running mostly idle at maybe 1...10% energy consumption. When a portion of these new monster machines starts to run these large models, then it is to be expected that the energy consumption will increase significantly. It is nice to say: "Computer, do my homework" and "Computer, email the neighbours a reminder to shut up.".
When we run an audio machine model on the local computer, then we are using a lot of energy. Add the local interactions with the newest language model to write that next "perfect" paragraph and email. You will be using more energy. A lot more than before.
Yes, yes, new devices will be less wasteful, but still, we are on track to more electrons being pushed, not fewer. How effective is it to run all those transistors to come up with the phrase: "We all knew, but couldn't resist.".
"ChatGPT looks to be losing another race"
Doesn't matter though. Borkzilla has managed to graft ChatGPT into Office and, soon, everything else it makes.
And that will be paid for by user subscriptions.
So it's just the Board of the Fortune 500 that will suddenly be asking themselves why they're paying for an inferior . . no, wait, it's from Borkzilla so they won't ask themselves anything.