Google DeepMind has published a research paper hitting back at criticism of its AI chip design system AlphaChip.
The Google AI arm originally announced its novel reinforcement learning method for designing chip layouts in 2020, and published a paper in Nature on it in 2021.
That paper claimed that AlphaChip was able to save thousands of hours of human effort for each new generation of Google TPU AI accelerator, and was already being used by the company to help design its tensor chips.
The work was open-sourced in 2022, and has since been used by Google for its Axion Arm-based CPUs as well as other internal Google chips that have yet to be announced
However, in 2023 two papers cast doubt on the success of the effort - one from Chang et al., and one from Igor Markov. Chang's paper said that they were unable to reproduce Google's methods, while Markov, a scientist at competitor Synopsys, published a meta-analysis that called the approach a 'false dawn.'
Markov said that the analysis "demonstrated that Google RL lags behind (i) human designers, (ii) a well-known algorithm (Simulated Annealing), and (iii) generally-available commercial software, while being slower; and in a 2023 open research contest, RL methods weren't in top 5." He also cited a Google whistleblower who was concerned about the paper.
As a result of the growing controversy, Nature added an editor's note on Google's paper and said that it was investigating the research. An independent expert who had reviewed Google’s paper retracted his Nature commentary article that had originally praised Google’s work.
Google claims that Chang's paper failed in multiple ways, including not pre-training and using substantially less compute. It also said that "Markov published baseless allegations of fraud," and added that an internal investigator tracked down the whistleblower who admitted that, while he suspected fraud, "he did not have evidence to support his suspicion of fraud."
The tech giant now says that Nature completed its investigation this April and "found entirely in our favor," removing the editor's note in September. That same month, MediaTek announced it would use AlphaChip as part of its chip development.
Markov republished and updated his analysis this month, adding "none of the major concerns about the Nature paper have been addressed." He said that AlphaChip was not open source as claimed: "Among other pieces, source code for simulated annealing is still missing, and additionally the Nature results cannot be reproduced without proprietary training data and test data."
In a blog post about AlphaChip, Google claimed that "AlphaChip has designed better chip layouts and provided more of the overall floorplan, accelerating the design cycle and yielding higher-performance chips.
"AlphaChip has triggered an explosion of work on AI for chip design, and has been extended to other critical stages of chip design, such as logic synthesis and macro selection."
Among the names listed in the original paper is Richard Ho, the former Google TPU lead that DCD exclusively reported moved to OpenAI in 2023.
