Research carried out by Google self-driving spin-out Waymo has found that autonomous vehicle (AV) scaling laws are similar to those of large-language models.

Waymo said that increasing training data and compute resources directly leads to better AV performance.

While this appears intuitive, this is not always the case in model development.

More parameters can sometimes make models degrade, because the model overfits the training data. Similarly, the model may have learned everything it can from the data types provided, so more data does not produce any new insights.

Scaling laws have led the AI data center boom, on the firm belief that more compute and data will lead to better multimodal language models. Some critics warn of a 'wall,' where the models top out and no amount of compute and data can improve things.

For now, that does not appear to be happening, although new data sources for LLMs are harder and harder to come by, and some diminishing returns have been claimed by industry observers.

For AVs, Waymo saw parallels with LLM development. "Like the case for LLMs, we observe that improvements in the cross-entropy loss follow a power-law as we scale the training compute, while jointly scaling the model and dataset sizes," Waymo researchers said in a newly published paper.

"However, unlike the case for LLMs, the optimal models for this task tend to be relatively smaller in size, while requiring significantly more data to train."

They add: "We believe that this finding, if it holds for similar robotic planning tasks, has important implications for data collection and the sizes of models that should be trained. Furthermore, the smaller sizes of these models result in lower latency, which implies that improvements in onboard system performance can be directly driven by scaling training dataset size and compute."

GPU maker Nvidia, which has benefited from the LLM boom and has a vested interest in pushing new model developments, claims that robotics will be the next major training workload and will require more data centers than current LLMs.

At present, Waymo has data covering 500,000 hours of driving, and runs its own driving simulations on its 'Carcraft' virtual world. The company operates in Phoenix, San Francisco, Los Angeles, and Austin, with plans to operate in 10 cities this year.

"Through these insights, researchers and developers of AV models can begin to know with certainty that enriching the quality and size of the data and models will deliver better performance," Waymo said.