According to one estimate, a cluster 100,000 H100 chips would need 150 megawatts of power. The largest supercomputer at the United States national laboratory, The Captain, on the other hand, requires 30 megawatts of power. Meta plans to spend up to $40 billion in capital this year to equip data centers and other infrastructure, which represents an increase of more than 42% compared to 2023. The company foresees even more dizzying growth in that spending next year. comes.
Everything goes up
Meta’s total operating costs have grown 9% this year. But overall sales (largely from ads) have increased more than 22%, leaving the company with wider margins and higher profits even as it invests billions of dollars in Llama’s efforts.
For its part, OpenAI, considered the current leader in the development of cutting-edge AI, is burning money despite charging developers for access to its models. What for now remains a non-profit company has stated that it is training GPT-5, a successor to the model currently powered by ChatGPT. OpenAI has said that GPT-5 will be larger than its predecessor, but has not said anything about the cluster computer that you are using for training. OpenAI has also said that in addition to scaling, GPT-5 will incorporate other innovations, such as a recently developed approach to reasoning.
Sam Altman, CEO of OpenAI, has stated that GPT-5 will represent “a significant leap forward” compared to its predecessor. Last week, Altman responded to a news report claiming that OpenAI’s next frontier model would launch in December, writing in X: “fake news out of control.”
On Tuesday, Google CEO Sundar Pichai stated that the latest version of the company’s Gemini family of generative AI models is in development.
open source AI
Meta’s open approach to AI has sometimes proven controversial. Some artificial intelligence experts worry that making significantly more powerful AI models freely available could be dangerous because it could help criminals launch cyberattacks or automate the design of chemical or biological weapons. Although Llama is tuned before release to restrict bad behavior, it is relatively trivial to remove these restrictions.
Zuckerberg remains optimistic about the open source strategy, even as Google and OpenAI push proprietary systems: “It seems pretty clear to me that open source will be the most cost-effective, customizable, reliable, efficient, and easy-to-use option available to developers.” “he said Wednesday. “And I’m proud that Llama is leading this way.”
Zuckerberg added that Llama 4’s new capabilities should be able to drive a broader range of features across all Meta services. Currently, the most outstanding offer based on Llama models is the chatbot similar to ChatGPT known as Meta AI, which is available on Facebook, Instagram, WhatsApp and other applications.
According to Zuckerberg, more than 500 million people use Meta AI every month. Over time, the company hopes to generate revenue through ads in the feature: “There will be a broader and broader set of queries that people use it for, and monetization opportunities will exist over time as we get there.” said Meta CFO Susan Li on Wednesday’s call. With the ability to earn revenue from ads, Meta might be able to subsidize Llama for everyone else.
Article originally published in WIRED. Adapted by Mauricio Serfatty Godoy.
#Meta #Trains #Generation #Llama #Worlds #Largest #GPU #Cluster
