This week Google announced it would provide a cloud computing performance evaluator called PerKit Benchmarker. The evaluation tool is hosted on the open-source collaboration site Github, and will allow users of the Google Cloud Platform, Amazon’s AWS, and Microsoft’s Azure to measure their current provider’s performance against industry-established benchmarks.

“For those of you developing applications on the cloud, performance is often a critical concern”, Google’s Cloud Platform Performance Team wrote in a blog. “It turns out that it’s surprisingly difficult to evaluate cloud offerings beyond just looking at price or feature charts”. Google said it then invited input from other cloud providers, industry analysts and academia – in all, more than 30 parties – to help develop the benchmarking framework. Industry contributors included, among others, CenturyLink, Microsoft, Intel, Rackspace, and ARM.

Metrics for peak performance

Google’s PerfKit includes an Explorer user interface to graphically display and interpret results. The benchmarking system will report on standard metrics of peak performance, end-to-end time to provision resources in the cloud, and provide “a transparent view of application throughput, latency, variance, and overhead” across the cloud platforms. 

“Once installed, PerfKit runs about 20 benchmark tests that look at everything from raw CPU performance to more complex database and network benchmarks”, TechCrunch noted in its evaluation of the new tool.

Google said that representatives from Stanford and MIT would lead quarterly review discussions among the stakeholder community regarding default benchmarks and settings. “PerfKit is a living benchmark framework, designed to evolve as cloud technology changes, always measuring the latest workloads so you can make informed decisions about what’s best for your infrastructure needs”, Goggle wrote in its blog. “As new design patterns, tools, and providers emerge, we’ll adapt PerfKit to keep it current.”