Anika Begay
Elon Musk doesn’t want Tesla to be just a car company. He wants Tesla to be an AI company, one that has figured out how to make cars drive themselves.
Crucial to that mission is Dojo, Tesla’s custom supercomputer designed to train its Fully Self-Driving (FSD) neural networks. FSD isn’t actually fully autonomous; it can perform some automated driving tasks, but it still requires an attentive human at the wheel. But Tesla thinks that with more data, more computing power, and more training, it can cross the threshold from near-self-driving to fully self-driving.
And this is where Dojo comes in.
Musk has been teasing Dojo for some time, but the executive has been ramping up discussions about the supercomputer throughout 2024. Dojo’s importance to Tesla may be existential: With electric vehicle sales declining, investors want reassurance that Tesla can achieve autonomy. Below is a timeline of Dojo mentions and promises.
2019
First mentions of Dojo
April 22 – At Tesla’s Autonomy Day, the automaker brought its AI team on stage to talk about Autopilot and Full Self-Driving, and the AI that powers both. The company shared information about Tesla’s custom chips, designed specifically for neural networks and self-driving cars.
During the event, Musk teases Dojo, revealing that it is a supercomputer for training artificial intelligence. He also notes that all Tesla cars produced at the time would have all the hardware needed for fully autonomous driving and would only need a software update.
2020
Musk begins Dojo tour
February 2 – Musk says Tesla will soon have more than a million connected vehicles worldwide, equipped with the sensors and processing power needed for fully autonomous driving, and touts the capabilities of Dojo.
“Dojo, our training supercomputer, will be able to process large amounts of video training data and efficiently handle hyperspace arrays with large numbers of parameters, lots of memory, and ultra-high bandwidth between cores. More on that later.”
August 14 – Musk reiterates Tesla’s plan to develop a neural network training computer called Dojo “to process really massive amounts of video data,” calling it “a beast.” He also says the first version of Dojo will be out “about a year away,” which would put its launch date around August 2021.
December 31 — Elon says Dojo isn’t necessary, but it will make autonomous driving better. “It’s not enough to be safer than human drivers, Autopilot ultimately needs to be more than 10x safer than human drivers.”
2021
Tesla Makes Dojo Official
August 19 – The automaker officially announced Dojo at Tesla’s first AI Day, an event designed to attract engineers to Tesla’s AI team. Tesla also showcased its D1 chip, which the automaker says it will use, along with Nvidia’s GPU, to power the Dojo supercomputer. Tesla notes that its AI cluster will house 3,000 D1 chips.
October 12 – Tesla releases A Dojo Technology whitepaper, “A Guide to Tesla Configurable Floating Point Formats and Arithmetic.” The whitepaper outlines a technical standard for a new type of binary floating point arithmetic used in deep learning neural networks that can be implemented “entirely in software, entirely in hardware, or any combination of software and hardware.”
2022
Tesla Reveals Dojo Progress
August 12 – Musk says Tesla will “phase in Dojo. It won’t need to buy as many incremental GPUs next year.”
September 30 – At Tesla’s second AI Day, the company reveals it has installed the first Dojo cabinet, testing 2.2 megawatts of load tests. Tesla says it has built one tile per day (made up of 25 D1 chips). Tesla shows Dojo on stage running a stable diffusion model to create an AI-generated image of a “Cybertruck on Mars.”
Importantly, the company has set a target date of completing a full Exapod cluster by Q1 2023, and says it plans to build a total of seven Exapods in Palo Alto.
2023
A risky bet‘
April 19 – Musk tells investors during Tesla’s first-quarter earnings call that Dojo “has the potential to be an order of magnitude improvement in training costs” and also “has the potential to become a marketable service that we could offer to other companies in the same way that Amazon Web Services offers web services.”
Musk also notes that he would “consider Dojo a risky bet,” but “a bet worth making.”
June 21 — The Tesla AI X account claims that the company’s neural networks are already in customer vehicles. The thread includes a timeline chart of Tesla’s current and projected computing power, which places Dojo production start in July 2023, though it’s unclear whether that refers to the D1 chips or the supercomputer itself. Musk says that Dojo was already online and running tasks in Tesla data centers that same day.
The company also predicts that Tesla’s computing power will reach the top five in the world by February 2024 (there is no indication that this has been successful) and that Tesla will reach 100 exaflops by October 2024.
July 19 – Tesla notes in its second-quarter earnings report that it has started production of the Dojo. Musk also says that Tesla plans to spend more than $1 billion on the Dojo through 2024.
September 6 – Musk posts on X that Tesla is limited by AI training compute, but that Nvidia and Dojo will solve that problem. He says that managing data from the roughly 160 billion video frames Tesla receives from its cars per day is extremely difficult.
2024
Scalability Plans
January 24 – During Tesla’s fourth-quarter and full-year earnings conference call, Musk again acknowledged that Dojo is a high-risk, high-reward project. He also said that Tesla was pursuing “the dual path of Nvidia and Dojo,” that “Dojo is working,” and that it’s “doing some training work.” He noted that Tesla is scaling it up and has “plans for Dojo 1.5, Dojo 2, Dojo 3, and whatnot.”
January 26 – Tesla has announced plans to spend $500 million to build a Dojo supercomputer in Buffalo. Musk then downplays the investment a bit, posting on X that while $500 million is a lot, it is “equivalent to only a $10k Nvidia H100 system. Tesla will spend more on Nvidia hardware this year. The stakes to be competitive in AI are at least several billion dollars a year at this point.”
April 30 – At TSMC’s North American Technology Symposium, the company said that its next-generation Dojo training tile, the D2, which places the entire Dojo tile on a single silicon wafer rather than connecting 25 chips to create one, is already in production, according to IEEE Spectrum.
May 20 – Moss notes that the rear end of the Giga Texas factory expansion will include the construction of “a super-dense, water-cooled supercomputer cluster.”
June 4th – A CNBC report reveals that Musk diverted thousands of Nvidia chips reserved for Tesla to X and xAI. After initially claiming the report was false, Musk publishes on X that Tesla had no location to send Nvidia chips to power them, due to ongoing construction of the Giga Texas South Extension, “so they would just sit in a warehouse.” He noted that the extension “will house 50,000 H100s for FSD training.”
He too messages:
“Of the roughly $10 billion in AI spending I said Tesla would incur this year, about half is internal, mostly the Tesla-designed AI inference computer and the sensors in all our cars, plus Dojo. For building the AI training superclusters, NVidia hardware is about 2/3 of the cost. My current best guess for Tesla’s NVidia purchases is $3-4 billion this year.”
July 1st – Musk reveals on X that current Tesla vehicles may not have the right hardware for the company’s next-generation AI model. He says that the roughly 5x increase in parameter count with next-generation AI “is very difficult to achieve without upgrading the vehicle’s inference computer.”
Nvidia Supply Challenges
July 23 – During Tesla’s second-quarter earnings conference call, Musk said that demand for Nvidia hardware is “so high that GPUs are often hard to come by.”
“I think that requires us to do a lot more work on Dojo to make sure we have the training capacity that we need,” Musk says. “And we see a path to be competitive with Nvidia with Dojo.”
A chart in Tesla’s investor deck predicts that Tesla’s AI training capacity will rise to about 90,000 H100-equivalent GPUs by the end of 2024, up from about 40,000 in June. Later that day on X, Musk posts that Dojo 1 will have “about 8k H100-equivalents of online training by the end of the year.” He also posts photos of the supercomputer, which appears to use the same refrigerator-like stainless steel exterior as Tesla’s Cybertrucks.
XXX
July 30 – The AI5 is about 18 months away from full-scale production, Musk says in a answer to a post by someone claiming to have started a club of “Tesla HW4/AI4 owners angry at being left behind when AI5 comes out.”
August 3 – Musk posts on X that he has gotten a tour of the “Tesla supercomputing cluster at Giga Texas (aka Cortex).” He notes that it would consist of about 100,000 Nvidia H100/H200 GPUs with “massive storage for FSD and Optimus video training.”
