Elon Musk's companies, Tesla and xAI, are investing heavily in AI training hardware, with an anticipated $10 billion worth of training compute capacity expected to be online by the end of 2024. This significant investment is a testament to the crucial role AI hardware plays in staying competitive in the AI industry.

In July, xAI began AI training using the Memphis Supercluster, a system that is set to integrate 100,000 liquid-cooled H100 GPUs. This system alone requires an enormous amount of power, drawing at least 150 MW, with the GPUs accounting for around 70 MW. The total cost of the system remains unknown, though the GPUs alone would cost around $2 billion if purchased at $20,000 per unit.

In late August, Tesla unveiled its Cortex AI cluster, equipped with 50,000 Nvidia H100 GPUs and 20,000 of Tesla's own Dojo AI wafer-sized chips. The Dojo cluster is crucial for training Tesla's full self-drive (FSD) capability, making it strategically vital for the company.

In early September, xAI began operating its Colossus supercomputer, which has already integrated 100,000 H100 GPUs and is expected to add 50,000 H100 and 50,000 H200 GPUs in the coming months. This supercomputer also costs billions.

While these investments are substantial, they may still fall short of the ambitious plan outlined by Elon Musk in April 2024, where he stated that Tesla alone would spend $10 billion on AI hardware this year. Despite the potential delay in meeting Musk's targets, the investments in AI hardware are a crucial step for Tesla and xAI to maintain their position in the AI industry.