The market for hardware, software and services to support deployments of Apache Hadoop is going to grow at an average annual rate of nearly 55% between now and 2018, according to a newly published report by Transparency Market Research, an Albany, New York-based research and consulting firm.

The analysts said the Hadoop market would grow from its 2012 size of US$1.5bn to about $21bn in 2018, driven by “exponential” growth in the amount of unstructured data generated by private and public organizations.

Hadoop is the most popular open-source framework for processing a lot of data in parallel by a cluster of commodity servers. It was created in 2005 by an engineer at Yahoo. The framework uses a programming model called MapReduce, which originated at Google.

As organizations increasingly want to make use of all the data they have accumulated in their data centers, Hadoop is gaining momentum because it offers a relatively inexpensive way to crunch through massive data sets quickly.

However, there is a shortage of professionals skilled enough to help companies realize the value of the framework. For this reason, about half of the revenue from the Hadoop market in 2012 came from services, according to Transparency. Hardware had second-largest market share.

While the analysts expect services to continue to drive the biggest portion of the market through 2018, they say software's share will grow rapidly and will outgrow hardware in terms of revenue by 2017. Transparency explains it by continuous technological changes in the software market.

North America has so far seen the highest rate of adoption of Hadoop solutions, but Europe is considered an emerging Hadoop market and leading players, such as Cloudera and Hortonworks (both have Hadoop distributions and commercial products and services around them), have been focused on the region.

Cloudera currently has the largest revenue share in the Hadoop market, the analysts said. However, players like Hortonworks, MapR and Greenplum are expected to become powerful players over the long term.

The government sector is the largest consumer of Hadoop-related products and services, according to the report. It is followed by the the banking, financial services and insurance vertical.

The latter is followed by healthcare and life sciences and retail. The telecommunications sector is currently in initial stages of Hadoop implementation and will become a significant user around 2014.