Portable Large Language Models – not the iPhone 15 – are the future of the smartphone
- Reference: 1694590693
- News link: https://www.theregister.co.uk/2023/09/13/personal_ai_smartphone_future/
- Source link:
Before that phone I could justify grabbing Cupertino’s annual upgrade. These days, what do we get? The iPhone 15 [1]delivered USB-C, a better camera, and faster wireless charging. It's all nice, but not truly necessary for most users.
Yet smartphones are about to change for the better – thanks to the current wild streak of innovation around AI.
[2]
Pretty much everyone with a smartphone can already access the "Big Three" AI chatbots – OpenAI's ChatGPT, Microsoft's Bing Chat and Google's Bard – through an app or browser.
[3]
[4]
That works well enough. Yet alongside these "general purpose" AI chatbots, a subterranean effort – spearheaded by another of the behemoths of big tech – looks to be gaining the inside track.
Back in February, Meta AI Labs [5]released LLaMA – a large language model scaled down both in its training data set and in its number of parameters. Our still-rather-poorly-intuited understanding of how large language models work equates a greater number of parameters with greater capacity – GPT-4, for example, is thought to have a trilion or more parameters, though OpenAI is tight-lipped about those numbers.
[6]
Meta's LLaMA gets away with a paltry 70 billion and, in one version, just seven billion.
So is LLaMA only one two-thousandth as good as GPT-4? This is where it gets very interesting. Although LLaMA has never beaten GPT-4 head-to-head in any benchmarking, it's not bad – and in many circumstances, it's more than good enough.
LLaMA is open source-y in a kinda sorta very Meta-ish way, enabling a [7]field army of researchers to take the tools, the techniques and the training and improve them all, rapidly and dramatically. Within weeks, we saw [8]Alpaca , Vicuna and a menagerie of other large language models, each tweaked to be better than LLaMA - all the while drawing closer to GPT-4 in benchmarking.
[9]Large language models' surprise emergent behavior written off as 'a mirage'
[10]Artificial General Intelligence remains a distant dream despite LLM boom
[11]ChatGPT study suggests its LLMs are getting dumber at some tasks
[12]Hope for nerds! ChatGPT's still a below-average math student
When Meta AI Labs [13]released LLaMA2 in July – under a less Meta-centric license – thousands of AI coders set to work tuning it for a variety of use cases.
Not to be outdone, three weeks ago Meta AI Labs also did its own bit of fine tuning, [14]releasing Code LLaMA – tuned to provide code completions inline with an IDE, or simply to be fed code for analysis and repair. Within two days, a startup called [15]Phind had fine-tuned Code LLaMA into a large language model that beat GPT-4 – albeit at a single benchmark.
[16]
That's a first – and [17]a warning shot across the bow of OpenAI, Microsoft and Google. It seems these "tiny" large language models can be good enough, while also small enough that they don't have to run in an airplane-hangar-sized cloud computing facility where they consume vast resources of power and water. Instead, they can run on a laptop – even a smartphone.
That's not just theory. For months I've had the [18]MLC Chat app running on my iPhone 13. It runs the seven-billion-parameter model of LLaMA2 without much trouble. That mini-model is noticeably less bright than the LLaMA2 model that employs 13 billion parameters (which sits in a sweet spot between size and capability) – but my smartphone doesn't have enough RAM to hold that one.
Nor does the iPhone 15 - although Apple's spec sheets .
These personal large language models – running privately, on device, all the time – will soon be core features of smartphone operating systems. They'll suck in all your browsing data, activity and medical data, even financial data – all the data that today we hand off to the cloud to be used against us – and they will continuously improve themselves to represent more accurately our states of mind, body, and finances.
They'll consult, they'll encourage - and they'll warn. They won't replace the massive general purpose models – but neither will they leak all our most personal data to the cloud. Most smartphones already have enough CPU and GPU to run these personal large language models, but they need more RAM – the better to think with. With a bit more memory, our smartphones can grow wildly smarter. ®
Get our [19]Tech Resources
[1] https://www.theregister.com/2023/09/12/apple_announces_iphone_15_lineup/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZQGIQn@MRFec0upYotWxLAAAAQM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQGIQn@MRFec0upYotWxLAAAAQM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZQGIQn@MRFec0upYotWxLAAAAQM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/02/25/ai_in_brief/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZQGIQn@MRFec0upYotWxLAAAAQM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2023/03/08/meta_llama_ai_leak
[8] https://www.theregister.com/2023/03/21/stanford_ai_alpaca_taken_offline/
[9] https://www.theregister.com/2023/05/16/large_language_models_behavior/
[10] https://www.theregister.com/2023/07/04/agi_llm_distant_dream/
[11] https://www.theregister.com/2023/07/20/gpt4_chatgpt_performance/
[12] https://www.theregister.com/2023/08/25/chatgpt_outperforms_average_uni_students/
[13] https://www.theregister.com/2023/07/19/meta_llama_2/
[14] https://www.theregister.com/2023/08/25/meta_lets_code_llama_run/
[15] https://www.phind.com/blog/code-llama-beats-gpt4
[16] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZQGIQn@MRFec0upYotWxLAAAAQM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[17] https://www.theregister.com/2023/05/11/open_source_ai_makes_subscriptions_irrelevant/
[18] https://mlc.ai/mlc-llm/
[19] https://whitepapers.theregister.com/
Re: Sure, it's possible, but why would you want it?
A use case? How about multiple advisors are better than fewer! And humans, even expert humans, make silly mistakes.
I went to my doctor one day with chest pains - I was having a heart attack. He prescribed indigestion and sent me home. I was hospitalised that night. After I had recovered, I mentioned the incident to a nurse at the same clinic. She was shocked. She said the doc should have done an ECG with the symptoms I was experincing. She figured she would have done a better job on that day.
I agree. An LLM with access to my conversations with the doctor on the day may have asked me get a second opinion.
Re: Sure, it's possible, but why would you want it?
I'm sure all the support people who read the Reg are looking forward to callouts where they are constantly pestered by a semi-informed user armed with a LLM asking a barrage of pointless questions.
However, I too would be disappointed with any doctor who prescribed indigestion.
Re: Sure, it's possible, but why would you want it?
A simple flow chart is all that was needed there. An ECG and blood test is cheap compared to the risk of missing a cardiac problem.
Re: Sure, it's possible, but why would you want it?
You and I might not want it but HMG would love it. Who needs to weaken encryption when you can get the subject's own device to analyse what they're up to?
"neither will they leak all our most personal data to the cloud"
At this point in time, that sounds rather like wishful thinking to me.
It appears as if everything everyone is doing at the moment is geared toward siphoning my personal life to The Cloud TM . If those portable whatchamacallits are going to become pervasive, I'm willing to bet that they'll happily lap up everything they can and send it to the mothership ASAP.
It's the contrary that would surprise me.
I read the article twice, but I'm still stuck. Once my phone's got this PLLM thing on it, constantly running, sucking battery power and reducing available memory, what will it actually do for me? Like most of my friends (we're old) I use the phone to listen to my music (stuff on shelves in my house), text and mail people, find out when the next train is, find out where the next pub is and make the odd foray onto the web to answer questions - usually about 70s and 80s rock music. All I see in my future is a life blighted by annoying pop-ups recommending stuff that I'm not interested in. There'd better be an off switch.
I don't think that the tech companies want us running our own LLM AI locally on our devices? No they want them in the cloud so they can hoover up all our personal data, so I don't expect them to be pushing local AI any time soon.
Yes even Apple whose ads make it look like they are squeakily clean and don't collect personal data from iPhone users still vacuum up a large amount of it from iOS even if they aren't selling it to 3rd party ad companies, they are using it to sell you ads on places like the App store.
> Yes even Apple whose ads make it look like they are squeakily clean and don't collect personal data from iPhone users still vacuum up a large amount of it from iOS even if they aren't selling it to 3rd party ad companies,
First, turning off personalised ads on Apple OS' really is pretty easy, as long as you're capable of tapping a button: https://fossbytes.com/apple-data-collection-explained/
Second, Apple has continually moved away from the cloud for ML towards doing more on-device. It announced yesterday it's moved a bunch of voice recognition tasks for Siri from cloud to Apple Watch, because the processor on it now has a fast enough neural engine to support it. It's absolutely in its interests (and yours) to do more ML tasks on-device.
I'm guessing that you missed the part about Apple moving voice recognition - a core ML tech - from the cloud on to the *Watch*?
They're fully away that ML is moving on the device. Why do you think they have spent years building chips with Neural Engine into all their devices? The problem with shifting LLM's on to the device, though, is that they're incredibly power-hungry, and that means the more you use them, the more you can watch your battery slide towards zero.
Sure, it's possible, but why would you want it?
Just about every single "AI in the machine" story ever written has been a cautionary tale. From the Cirrius Cybernetics Corporation onwards. No one (besides, perhaps, Ian Banks, perhaps Asimov) has posited an AI sidekick actually doing something useful and productive, let alone not turning on the user. I have no use-case for such a thing. What would you do with it?