The rise of Edge AI has largely been born out of increased demand for inference power available need the user, with organizations increasingly seeking to deploy AI algorithms and models onto local Edge devices.
At present, the AI chip landscape is largely dominated by a handful of vendors developing accelerators to support ever-growing workloads in the data center. However, as the Edge grows, opportunities lie for new companies to diverge from data center architecture.
SiMa (pronounced See-ma) is one such company that is currently making waves in the Edge AI market, hoping to carve a niche for itself in the space above lower powered devices like phone chips and below the significantly more powerful data center gear.
Founded in 2018 by former Groq COO Krishna Rangasayee, the San Jose, California-based company targets the market segment between 5W and 25W of energy usage. Thus far, it has developed an embedded Edge machine learning system-on-chip (MLSoC) that it says allows customers to run entire applications on a chip.
To date, the company has raised $270 million in funding and released the second generation of its offering in September 2024, with samples due to be available to customers in Q4 of this year.
While not all of SiMa’s customers have been made public, earlier this month it announced a partnership with manufacturing organization TRUMPF, which will use the chips to develop AI-powered lasers.
When DCD spoke with SiMa’s senior vice president of engineering and operations Gopal Hegde in early July 2024, he revealed that the second-generation chip based on TSMC’s 6nm process technology was due to be with the company in a matter of days.
Hegde is a chip industry veteran and serial entrepreneur, who joined SiMa four years ago after a stint at Marvell, which had acquired a startup he’d previously been working for.
He tells us that SiMa is specifically targeting the embedded Edge market – which the company describes as the layer that sits between the cloud and personal devices.
SiMa believes this segment is worth around $40bn currently, and within that market, the company is pursuing applications in healthcare, smart retail, autonomous vehicles, government, and robotics.
“AI has really taken off in the data center and cloud, but it took almost 10 years for that to happen,” Hegde says. He argues that this has also been the case with Edge AI, where a unique set of requirements has caused the industry to move relatively slowly.
Hegde identifies three main challenges: cost, ease of deployment, and a lack of expertise. Hegde says SiMa is different from other Edge AI companies in the market because in addition to attempting to address the issues around cost and expertise, its Palette software solution provides a no-code approach to Edge machine learning application development.
“[We are] actually focusing on the software infrastructure that is required to deploy AI and machine learning, that is the main difference between us and many other companies out there,” he says. “A lot of our competitors build excellent silicon and, in many cases, their silicon may be better than ours. But nobody has software similar to what we have and nobody's trying to solve the problem as comprehensively as we are in order to address our customer needs.”
These customer needs include the challenges posed by increasingly complex AI workloads, which Hegde says most chip vendors have reacted to by “throwing more hardware [at the workload] and hoping the problem goes away.”
Unfortunately, he says, that’s not an option for SiMa customers because they can’t deploy, for example, Nvidia’s upcoming 1kW Blackwell GPU at the Edge, as most devices deployed for Edge AI purposes have single to low double-digit power consumption. Of course, Nvidia also has its own Edge offerings, including the 40-60W Nvidia A2 GPU.
“We are not making the silicon more complex but we are improving the compute and we are improving the machine learning capability,” says Hegde. “With our second generation [chip], there’s twice as much compute compared to the first generation so it can support a lot more complex applications, and the way we solve the problem is with software.”
He added that for SiMa, the “key innovation” has a lot to do with how it develops its compiler toolchain software, which enables the company to run networks very efficiently without having to deploy more hardware.
He says this approach is in direct contrast with some of the leading chip companies such as Nvidia, Intel, and AMD who just “throw more GPU cores at the problem… or more expensive memory,” which ends up making the hardware more complex, more power-hungry, and more expensive. Instead, by deploying software, SiMa has a much more efficient way of scheduling and executing machine learning instructions in parallel.
“We're keeping our fingers crossed to bring up this chip performance per watt and we are seeing improvement over 50 percent,” Hegde says. “Compared to our previous generation on emulation platform, over the life of the product in the last two years, we’ve improved the performance by over 30 percent purely by making software tweaks to it.”
As a result, in the next 12 months, the company expects to see a roughly two times performance per watt improvement.
Competing with Nvidia
It’s almost impossible to talk about AI chips without mentioning Nvidia, and while Hegde says it's hard not to view the GPU giant as a competitor, simply because of its sheer dominance, ultimately, the two companies are targeting two very different customer bases, with the lowest power Nvidia solution offering the same power consumption as SiMa’s absolute top offering.
And while Nvidia has been setting MLPerf performance benchmarking records for cloud workloads, Hegde says the company’s performance doesn’t stack up when it comes to Edge performance.
In August 2023, SiMa made its debut MLPerf submission in the v3.1 round and went head-to-head with Nvidia’s Jetson Xavier NX kit (10-40W) in the closed Edge ResNet50 benchmarking test. SiMa was able to show better latency, power efficiency, and overall performance.
“When [Nvidia] runs Edge workloads, they actually don't do very well because they are not optimized for Edge,” Hegde says. “So we went to MLPerf basically to compete with them and over three submissions (SingleStream, MultiStream, and Offline), we actually beat them every time.”
Hegde says Nvidia no longer participates in the closed Edge category, instead focusing its efforts on other submissions where the company does continue to set records.
However, while SiMa might already have edged out Nvidia on its own turf, unlike some other Edge AI chip startups that have started mulling over a future entry into the data center, Hegde notes that’s not a path SiMa is considering.
“Our ambition is to be a key player in the embedded Edge market and we want to get there by actually addressing the three major problems that we talked about: cost, ease of use and deployment, and accelerating a complete end-to-end application.
“In all three cases, what we're doing is very different to what Nvidia is doing, and very different to all our competitors.”
"A lot of our competitors build excellent silicon and, in many cases, their silicon may be better than ours. But nobody has software similar to what we have and nobody's trying to solve the problem as comprehensively as we are in order to address our customer needs"
SiMa’s approach is in direct contrast with some of the leading chip companies such as Nvidia, Intel, and AMD who just throw more GPU cores at the problem… or more expensive memory.