Apple is using Amazon's custom chips including Inferentia and Graviton, and is exploring the use of Trainium2 for pretraining its models.
Apple's senior director for AI and Machine Learning Benoit Dupin went on stage at AWS Re:Invent 2024 to talk about the company's use of Amazon's chips.
According to Dupin, when Apple needed to scale inference globally for search, it "did so by leveraging AWS services in more than 10 regions," adding that "more recently, we have started to use AWS solutions with Graviton and Inferentia."
Dupin stated that Apple has seen 40 percent efficiency gains by migrating its AWS instances from x86 to Graviton, and has been able to execute some of its search text features twice as efficiently after moving from Graviton4 to Inferentia2 instances earlier this year.
The company is further looking to AWS chips for its Apple Intelligence service that is powered by Apple's large language models (LLMs).
Apple Intelligence runs both locally on Apple devices, and on Apple's services with Private Cloud Compute - launched in June of this year.
To further Apple Intelligence, the company needed to scale out its infrastructure for model training. Dupin said: "AWS has been right there alongside us as we've scaled we work with AWS services across virtually all phases of our AI and ML lifecycle."
The company is in the "early stages" of evaluating Amazon's Trainium2 for this task, and according to Dupin is expecting to see up to 50 percent improvement in efficiency in pre-training with AWS.
While a strong endorsement of AWS, Apple is also a known customer of both Microsoft Azure and Google Cloud, with the company previously releasing a research paper showing that it had used Google's TPU chips for training Apple Intelligence.
Apple also develops its own chips - and is hoping to transition its Apple Intelligence servers from Apple's M2 Ultra chip to the new M4 series in 2025. Apple’s M4 chips contain 28 billion transistors and are made using TSMC’s 3nm process.
Amazon has also recently announced its first EC2 instances featuring the Trainium2 chips, and plans for Trainium3.
