Meta, the parent company of Facebook, Instagram, and WhatsApp, is planning to deploy an updated version of its own AI-focused custom chips into its data centers this year.
According to documents seen and reported on by Reuters, the company is hoping to reduce its reliance on Nvidia by deploying the second generation of chips designed in-house and first reported to be in development in 2023.
The chips, dubbed the Meta Training and Inference Accelerator (MTIA) chip, are based on 7nm nodes and provide 102 Tops of Integer (8-bit) accuracy computation or 51.2 teraflops of FP16 accuracy computation. The chips run at 800 megahertz and are about 370 millimeters square.
Meta was originally expected to roll out its in-house chips in 2022 but scrapped the plan after they failed to meet internal targets, with the shift from CPUs to GPUs for AI training forcing the company to redesign its data centers and cancel multiple projects. That caused Meta's capital expenditure costs to dip in Q3 2023 by $3bn compared to the year before.
Yesterday (February 1) Meta released its Q4 2023 earnings, which saw the company reporting revenue of $40bn for the three months to the end of December, 25 percent up on the previous year
Speaking to analysts after the results were posted, CEO Mark Zuckerberg spoke about the company’s plan to invest in AI and data center capacity as its need for compute capacity continues to grow.
Zuckerberg said there wasn’t a clear expectation for just how much compute power would be required to train AI models and run AI inference engines but noted “the trend has been that state-of-the-art large language models [LLMs] have been trained on roughly 10x the amount of compute each year.”
He added: “A major goal will be building the most popular and most advanced AI products and services. If we succeed, everyone who uses our services will have a world-class AI assistant to help get things done, every creator will have an AI that their community can engage with, every business will have an AI that their customers can interact with to buy goods and get support, and every developer will have a state-of-the-art open source model to build with.”
The company’s CFO Susan Li said in a statement to shareholders that the company expects to see spending growth driven by investments in AI and non-AI servers in addition to the company’s data centers.
Earlier this month, Zuckerberg told The Verge that Meta would operate "600,000 [Nvidia] H100 equivalents of compute” by the end of 2024, claiming that the company had built up the capacity to do this at a scale that may be larger than any other individual company.
