Grok-1 chatbot code released – open source or open Pandora's box?
(2024/03/18)
- Reference: 1710785715
- News link: https://www.theregister.co.uk/2024/03/18/grok_chatbot_code_released/
- Source link:
As promised, Elon Musk has released the model behind the xAI chatbot, Grok-1.
[1]Released under the Apache 2.0 license, the base model weights and network architecture are now available. The model has 314 billion parameters and needs hardware with enough GPU memory to run it. It is fine-tuned for applications such as natural language dialog, and represents the raw base model checkpoint from the pre-training phase, which concluded in October 2023.
Critics have pointed to relatively lackluster performance in benchmarks; while Grok is a big model, it does poorly from what some engineers have seen. "Considering how poor it is compared to other models, it really emphasises how important fine tuning is. Models with MUCH smaller parameter counts are outperforming it in many metrics," said one [2]poster on the Hacker News forums last night.
[3]
You might find that the smaller [4]Mistral performs just as well as Grok-1, for instance.
[5]
[6]
To put its size in [7]perspective , even at 314 billion parameters, it still has some catching up to do with OpenAI's GPT-4, which had 1.76 trillion params at last count.
Notably, unlike existing LLMs, which are trained on data with a cutoff point in time, Grok has access to real time data corpus of everyone's Xeets via X.com, which should make for some interesting experiments in the days to come, although as another commenter noted: "Twitter tweet data in itself is both highly idiosyncratic and short by design, which alone is not conducive towards training a LLM."
[8]
[9]Grok will be familiar to users of Musk's social media platform, X, and subscribers have been able to ask the chatbot questions and receive answers. [10]According to xAI , Grok was modeled after The Hitchhiker's Guide to the Galaxy. "It is intended to answer almost anything and, far harder, even suggest what questions to ask."
If a user flicks through a dog-eared copy of [11]The Hitchhiker's Guide to the Galaxy radio scripts , the following definition can be found lurking in Fit the Tenth: "The Hitchhiker's Guide to the Galaxy is an indispensable companion to all those who are keen to make sense of life in an infinitely complex and confusing universe, for though it cannot hope to be useful or informative on all matters, it does make the reassuring claim that where it is inaccurate, it is at least definitively inaccurate.
"In case of major discrepancy it is always reality that's got it wrong."
[12]
The release comes on the first anniversary of the launch of OpenAI's GPT-4 model, and Musk's legal spat with his former AI pals remains in the background. At the beginning of this month, [13]Musk sued OpenAI , alleging there was little open about the company, despite its name. OpenAI [14]responded by releasing a trove of emails, claiming Musk was fully aware of its plans and wanted it folded into Tesla.
[15]Top LLMs struggle to make accurate legal arguments
[16]Google launches Gemini AI systems, claims it's beating OpenAI and others - mostly
[17]Elon Musk's xAI wants $1B cash infusion in exchange for equity shares
[18]Musk thinks X marks the spot for Grok AI engine based on social network
Patrik Backman, general partner at OpenOcean, said of Grok-1 being released: "For once, Elon Musk is putting his principles into action. If you sue OpenAI for transforming into a profit-driven organization, you must be prepared to adhere to the same ideals."
What hasn't been released by xAI is also of note. The Grok-1 weights are out there, yet the data used for training is not available under the same license, prompting AI expert Gary Marcus to [19]quip : "PartlyOpenAI."
Open sourcing generative AI tools has proven controversial. Some developers worry that making the technology available risks abuse and others point to the inherent benefits of transparency.
Meta [20]shared – [21]sort of – its Llama 2 models last year, and other companies have followed suit. OpenAI, on the other hand, has most definitely not.
By opening up the weights behind Grok-1, Musk is attempting to plant a flag in the opposite camp to the proprietary world of OpenAI.
As for its ultimate performance, like everything Musk touches, it could go either way. ®
Get our [22]Tech Resources
[1] https://x.ai/blog/grok-os
[2] https://news.ycombinator.com/item?id=39737281
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2024/03/17/ai_pc_local_llm/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://medium.com/@mazzalucas42/decoding-grok-1-rust-powered-ai-benchmarks-and-the-uncharted-territory-of-real-time-insights-1e167cb9f289
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2023/11/06/ai_in_brief/
[10] https://x.ai/blog/grok
[11] https://search.worldcat.org/title/1274613046
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.theregister.com/2024/03/01/elon_musk_sues_openai_gpt4/
[14] https://www.theregister.com/2024/03/06/openai_musk_lawsuit/
[15] https://www.theregister.com/2024/01/10/top_large_language_models_struggle/
[16] https://www.theregister.com/2023/12/06/google_gemini_ai/
[17] https://www.theregister.com/2023/12/06/elon_musks_xai_is_seeking/
[18] https://www.theregister.com/2023/11/06/ai_in_brief/
[19] https://twitter.com/GaryMarcus/status/1769487304467562951
[20] https://www.theregister.com/2023/07/19/meta_llama_2/
[21] https://www.theregister.com/2023/07/21/llama_is_not_open_source/
[22] https://whitepapers.theregister.com/
[1]Released under the Apache 2.0 license, the base model weights and network architecture are now available. The model has 314 billion parameters and needs hardware with enough GPU memory to run it. It is fine-tuned for applications such as natural language dialog, and represents the raw base model checkpoint from the pre-training phase, which concluded in October 2023.
Critics have pointed to relatively lackluster performance in benchmarks; while Grok is a big model, it does poorly from what some engineers have seen. "Considering how poor it is compared to other models, it really emphasises how important fine tuning is. Models with MUCH smaller parameter counts are outperforming it in many metrics," said one [2]poster on the Hacker News forums last night.
[3]
You might find that the smaller [4]Mistral performs just as well as Grok-1, for instance.
[5]
[6]
To put its size in [7]perspective , even at 314 billion parameters, it still has some catching up to do with OpenAI's GPT-4, which had 1.76 trillion params at last count.
Notably, unlike existing LLMs, which are trained on data with a cutoff point in time, Grok has access to real time data corpus of everyone's Xeets via X.com, which should make for some interesting experiments in the days to come, although as another commenter noted: "Twitter tweet data in itself is both highly idiosyncratic and short by design, which alone is not conducive towards training a LLM."
[8]
[9]Grok will be familiar to users of Musk's social media platform, X, and subscribers have been able to ask the chatbot questions and receive answers. [10]According to xAI , Grok was modeled after The Hitchhiker's Guide to the Galaxy. "It is intended to answer almost anything and, far harder, even suggest what questions to ask."
If a user flicks through a dog-eared copy of [11]The Hitchhiker's Guide to the Galaxy radio scripts , the following definition can be found lurking in Fit the Tenth: "The Hitchhiker's Guide to the Galaxy is an indispensable companion to all those who are keen to make sense of life in an infinitely complex and confusing universe, for though it cannot hope to be useful or informative on all matters, it does make the reassuring claim that where it is inaccurate, it is at least definitively inaccurate.
"In case of major discrepancy it is always reality that's got it wrong."
[12]
The release comes on the first anniversary of the launch of OpenAI's GPT-4 model, and Musk's legal spat with his former AI pals remains in the background. At the beginning of this month, [13]Musk sued OpenAI , alleging there was little open about the company, despite its name. OpenAI [14]responded by releasing a trove of emails, claiming Musk was fully aware of its plans and wanted it folded into Tesla.
[15]Top LLMs struggle to make accurate legal arguments
[16]Google launches Gemini AI systems, claims it's beating OpenAI and others - mostly
[17]Elon Musk's xAI wants $1B cash infusion in exchange for equity shares
[18]Musk thinks X marks the spot for Grok AI engine based on social network
Patrik Backman, general partner at OpenOcean, said of Grok-1 being released: "For once, Elon Musk is putting his principles into action. If you sue OpenAI for transforming into a profit-driven organization, you must be prepared to adhere to the same ideals."
What hasn't been released by xAI is also of note. The Grok-1 weights are out there, yet the data used for training is not available under the same license, prompting AI expert Gary Marcus to [19]quip : "PartlyOpenAI."
Open sourcing generative AI tools has proven controversial. Some developers worry that making the technology available risks abuse and others point to the inherent benefits of transparency.
Meta [20]shared – [21]sort of – its Llama 2 models last year, and other companies have followed suit. OpenAI, on the other hand, has most definitely not.
By opening up the weights behind Grok-1, Musk is attempting to plant a flag in the opposite camp to the proprietary world of OpenAI.
As for its ultimate performance, like everything Musk touches, it could go either way. ®
Get our [22]Tech Resources
[1] https://x.ai/blog/grok-os
[2] https://news.ycombinator.com/item?id=39737281
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2024/03/17/ai_pc_local_llm/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://medium.com/@mazzalucas42/decoding-grok-1-rust-powered-ai-benchmarks-and-the-uncharted-territory-of-real-time-insights-1e167cb9f289
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2023/11/06/ai_in_brief/
[10] https://x.ai/blog/grok
[11] https://search.worldcat.org/title/1274613046
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfjHiFv6RYB9IAK2HkbRywAAANA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.theregister.com/2024/03/01/elon_musk_sues_openai_gpt4/
[14] https://www.theregister.com/2024/03/06/openai_musk_lawsuit/
[15] https://www.theregister.com/2024/01/10/top_large_language_models_struggle/
[16] https://www.theregister.com/2023/12/06/google_gemini_ai/
[17] https://www.theregister.com/2023/12/06/elon_musks_xai_is_seeking/
[18] https://www.theregister.com/2023/11/06/ai_in_brief/
[19] https://twitter.com/GaryMarcus/status/1769487304467562951
[20] https://www.theregister.com/2023/07/19/meta_llama_2/
[21] https://www.theregister.com/2023/07/21/llama_is_not_open_source/
[22] https://whitepapers.theregister.com/
Curiously Enough
The Oncoming Scorn
my edition of the The Hitchhiker's Guide to the Galaxy radio scripts that had the good fortune to fall through a time warp from a thousand years in the future into the clearance bin at WH Smiths & wound up in my possession around 1985 - 87 isn't dog eared at all.
Icon - What thieving bastard has taken my towel off the towel rail?
Hand me a shovel
Badbob
I’ve buried my copy of Hitchhikers in warm compost. I shall retrieve it forthwith.
Anonymous Coward
> needs hardware with enough GPU memory to run it.
How much? I can't find the requirements anywhere. I'm not even sure what the article is referring to specifically.
I don't see much use in it given we have the venerable LLaMA derivatives, but it might be fun to try out. I'm sure it can be useful somewhere.
may_i
If it has 314 billion parameters and you use 4 bit quantisation, you'll need around 150GB. With 16 bit quantisation, you're looking at something around 628GB.
Better get some shares in nVidia along with the required number crunching hardware!
No, it isn't fine tuned.
It is the raw checkpoint. The release clearly states it is not fine tuned for any purpose. I know you all hate Musk, but at least read the first paragraph!