Elon Musk and xAI Power Up Supercomputer with 100,000 NVIDIA H100 GPUs

On Monday morning (22), Elon Musk and xAI turned on their supercomputer equipped with 100,000 NVIDIA H100 GPUs in liquid cooling.

On social media, the businessman states that this is the most powerful artificial intelligence training cluster on the entire planet and thanks those involved in this great project.

Great work from the xAI, X, NVIDIA, and supporting companies team to power up the Memphis Supercluster training at 4:20 AM local time. With 100,000 liquid-cooled H100s in just one RDMA, it’s the most powerful AI training cluster in the world!

Elon Musk

Continuing his posts, Elon Musk also claims that xAI will be the most powerful artificial intelligence on the planet for some time – at least until December 2024.

This is a significant advantage in training the world’s most powerful AI across all metrics as of December this year.

Elon Musk

See more:

It’s important to note that each unit of the NVIDIA Hopper H100 80GB GPU costs between $30,000 and $40,000 – bringing Elon Musk’s costs to at least $3 billion just for the graphics cards that will keep his artificial intelligence running.

Considering that the startup xAI had raised a $6 billion valuation in May 2024, this represents between 50% and up to 67% of the money spent on GPUs alone.

Furthermore, Elon Musk guarantees that he will not stop and promises to invest $9 billion in the purchase of B200 “Blackwell” accelerators – which should represent another leap, both for his social network and for the entire AI market.

Training Grok 3

All this technological power is used to power Grok 3, an artificial intelligence that is being produced at xAI. Elon Musk’s AI is the only one in the world to have this many GPUs, surpassing the others in both numbers and performance.

In comparative terms, rumors related to OpenAI and ChatGPT indicate that the GPT-4 language, for example, supposedly used 40,000 NVIDIA A100 GPUs.

Fonte: TweakTown