Tesla's Dojo supercomputer is a billion-dollar bet to make AI better at driving than humans
- Reference: 1689944467
- News link: https://www.theregister.co.uk/2023/07/21/tesla_dojo_spending/
- Source link:
Dojo was first mentioned by CEO Elon Musk during a Tesla investor day in 2019. It was built specifically for training machine learning models needed for video processing and recognition to enable the vehicles to be self-driving.
During Tesla's Q2 earnings call this week, Musk said Tesla was not going to be "open loop" on its Dojo expenditure, but the sum involved would certainly be "north of a billion through the end of next year."
[1]
"In order to copy us, you would also need to spend billions of dollars on training compute," Musk claimed, saying that developing a reliable autonomous driving system is "one of the hottest problems ever."
[2]
[3]
"You need the data and you need the training computers, the things needed to actually achieve this at scale toward a generalized solution for autonomy."
Musk pointed out that training complex machine learning models needs huge volumes of data, the more the better, and this is what Tesla has access to, thanks to all the telemetry from its vehicles.
[4]
"With respect to Autopilot and Dojo, in order to build autonomy, we obviously need to train our neural net with data from millions of vehicles. This has been proven over and over again, the more training data you have, the better the results," he said.
"It barely works at 2 million [training examples]. At 3 million, it's like, wow, OK, we're seeing something. But then, you get to, like, 10 million training examples, it becomes incredible. So there's just no substitute for massive amount of data. And obviously, Tesla has more vehicles on the road collecting this data than all of the other companies combined. I think maybe even an order of magnitude," Musk claimed.
On the Dojo system itself, Musk said it was designed to significantly reduce the cost of neural net training, and has been "somewhat optimized" for the kind of training that Tesla requires, which is video training.
[5]
"We see a demand for really vast training resources. And we think we may reach in-house neural net training capability of 100 exaFLOPS by the end of next year," Musk claimed, which is quite a lot of compute power, to put it mildly.
[6]Tesla to license Full Self-Driving stack to other automakers, says Musk
[7]Tesla board members to return $735M in compensation settlement
[8]First of Tesla's 'bulletproof' Cybertrucks clunks off production line
[9]Tesla plots entry to Britain's stagnant energy market
Dojo is based largely on Tesla's [10]own technology , starting with the D1 chip that comprises 354 custom CPU cores. Twenty-five of these D1 chips are interlinked into a 5x5 array inside a "training tile" module, building up to the base Dojo V1 configuration featuring 53,100 D1 cores, according to our colleagues at [11]The Next Platform .
Musk believes that with all of the training data and a "high-efficiency inference computer" in the car, Tesla's autonomous driving system will soon make its vehicles not just as proficient as a human driver, but eventually much better. When? He didn't say and has [12]form in making grand claims .
"To date, over 300 million miles have been driven using FSD [Full Self-Driving] Beta. That 300-million-mile number is going to seem very small, very quickly. And FSD will go from being as good as a human to then being vastly better than a human. We see a clear path to full self-driving being 10 times safer than the average human driver," he claimed.
This is important, Musk explained, because "right now, I believe there's something in the order of a million automotive deaths per year. And if you're 10 times better than a human, that would still mean 100,000 deaths, So, it's like, we'd rather be a hundred times better, and we want to achieve as perfect a safety as possible."
Dojo is not the only supercomputer Tesla has for video training. The company also built a compute cluster equipped with [13]5,760 Nvidia A100 GPUs , but Musk said they simply couldn't get enough GPUs for the task.
"We'll actually take the hardware as fast as Nvidia will deliver it to us," he said, adding: "If they could deliver us enough GPUs, we might not need Dojo, but they can't because they've got so many customers." ®
Get our [14]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZLqrpdXOQsGAXAW7ZGX0ZwAAARY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZLqrpdXOQsGAXAW7ZGX0ZwAAARY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZLqrpdXOQsGAXAW7ZGX0ZwAAARY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZLqrpdXOQsGAXAW7ZGX0ZwAAARY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/hpc&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZLqrpdXOQsGAXAW7ZGX0ZwAAARY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2023/07/20/tesla_to_license_fsd_software/
[7] https://www.theregister.com/2023/07/18/tesla_excess_compensation/
[8] https://www.theregister.com/2023/07/17/first_tesla_cybertrunk/
[9] https://www.theregister.com/2023/07/12/tesla_electric_uk_entry/
[10] https://www.theregister.com/2022/08/24/tesla_supercomputer_dojo/
[11] https://www.nextplatform.com/2022/08/23/inside-teslas-innovative-and-homegrown-dojo-ai-supercomputer/
[12] https://www.theregister.com/2023/07/20/tesla_to_license_fsd_software/
[13] https://blogs.nvidia.com/blog/2021/06/22/tesla-av-training-supercomputer-nvidia-a100-gpus/
[14] https://whitepapers.theregister.com/
Re: GPU demand.
At least the used cards from the miners had video outputs.
When the neural net market crashes and these big systems are dismantled by the receiver, you'll be able to buy their cards dirt cheap - and do nothing useful with them :-(
"But then, you get to, like, 10 million training examples, it becomes incredible"
Great news.
Call me when it becomes reliable.
Oh, and useful.
Re: "But then, you get to, like, 10 million training examples, it becomes incredible"
...and can fit in a car. So they have a supercomputer that is an 'AI' and can drive. That's nice. How do they engineer that into something that is viable in a vehicle?
Re: "But then, you get to, like, 10 million training examples, it becomes incredible"
And on an on-board computer.
Edit: Already said by GruntyMcPugh
Re: "But then, you get to, like, 10 million training examples, it becomes incredible"
... and stops driving itself into the back of stopped fire engines and such at speed.
Re: "But then, you get to, like, 10 million training examples, it becomes incredible"
>>and stops driving itself into the back of stopped fire engines and such at speed.
These would seem to be the very negative examples needed to train the AI what not to do...
10 million crash records are what is likely needed in order to avoid crashes, not 10 million examples of uneventful trips.
Re: "But then, you get to, like, 10 million training examples, it becomes incredible"
Musk as ever seems to be as dumb as a brick.
10 million training examples or 100 million training examples or a zillion billion quintillion training examples are completely useless they are all graded. Or to put it another way, how good a driver will the AI be after being trained by watching how average drivers perform?
To train to be a good driver it needs to learn from examples of good driving which you are not going to get from the majority of Tesla drivers (or BMW drivers or Audi drivers or...).
Throwing a billion monkeys at a billion typewriters does not make an intelligent end-product.
And in this case, the monkeys aren't even sentient themselves, they're just mechanical automaton monkeys.
This is the same problem that we've had since the 60's. Neural networks, AI, etc. etc. etc. - and the answer is always "if only we had more computers, more computer time, and just left it running for longer processing more input, I'm sure that somehow it will magically become intelligent".
No. It won't. If it did, Google would have had the best AI in the world about 10-15 years ago. Or even Amazon.
Brute-force and ignorance is not the seed of intelligence.
> This is the same problem that we've had since the 60's. Neural networks, AI, etc. etc. etc. - and the answer is always "if only we had more computers, more computer time, and just left it running for longer processing more input, I'm sure that somehow it will magically become intelligent".
I'll agree that has been the attitude of those who want to USE IT NOW! You've had long enough NOW WE WANT TO MAKE MONEY (or at least use it for advertising). And, hey, look, it works: we "solved" chess by brute force[1].
Oh, and Hollywood. Hollywood and bad SF books ("Valentina: Soul in Sapphire"[2]) just *love* that trope.
Those actually doing the AI - not so much. The good ones want to see it done better, not just brute force. Although you can see the appeal for just going with the flow: bite your tongue and wait for the stock options to vest, just like every other buzzword peddler.
[1] okay, brute force applied to the best of the extant search strategies - literal "blindly try every option" would still be running and for a *long* time to come, but still just doesn't seem like that is how the humans do it.
[2] even in 1984 it was so wrong to read that, but maybe it'll redeem itself in the last chapter. Nope.
I thought the intended outcome was not to create intelligence, merely to create a saleable product to make the world's richest man even richer? And perhaps provide a platform for his ego and attention seeking behaviour.
Did his Muskiness..
...really just claim that if they throw enough data at a neural net, it will magically generalise????
Really???
But will it be clever enough
to allow Tesla to figure out how to make Right Hand Drive cars in the future?
https://www.fleetnews.co.uk/news/manufacturer-news/2023/05/12/tesla-cancels-orders-for-right-hand-drive-model-s-and-x-cars
GPU demand.
First it was crypto mining, now it's AI. Maybe one day gamers will be able to get some GPUs.
Apparently I can't choose a sarcastic Icon without telling the world who I am.