The Apache Software Foundation has had a profound impact on the technology landscape. This non-profit enables developers from various, often competing, commercial organizations to collaborate on open source software. It has long been trusted, with visionary projects emerging from the academic world – such was the case with Apache Spark. We can thank the ASF for such everyday tools as the HTTP Server, Tomcat, Cassandra, Drill, Hive and Hadoop.

In total, the foundation now manages almost 280 top-level projects, plus 44 ‘podlings’ in incubation stage – and it does all this with an annual budget of just $1m. Despite the relative absence of money in ASF, the number of contributors to its projects has been growing steadily since it was incorporated in 1999.

Track record

The successful track record of the foundation has been attributed to its unique approach to governance, reliance on mailing lists, meritocracy and almost scientific rigor – all collectively known as ‘the Apache Way’.

The rules state, for example, that a business can’t be a member – only individuals are accepted. To be granted the right to commit source code to the project, a developer has to have a proven record of participation, before the matter is put to a vote by existing committers.

To get an insight into the current state of ASF, we asked Rich Bowen (aka Dr Bacchus), the executive vice president of the foundation. By day, Bowen is an open source evangelist at Red Hat, where he looks after the RDO Project. By night, he writes documentation for Apache projects and works with the community. The meeting took place in Budapest, where the ASF was holding two independent events, dividing its annual conference for the very first time into ApacheCon: Core, and ApacheCon: Big Data.

According to Bowen, the split is meant to reflect the range of projects within ASF. At the time of the very first ApacheCon, the foundation was managing just two or three projects. Today, it has arrived at a point where it can’t host enough sessions to cover everything it does. It’s no secret that the most active projects at Apache today by commit volume and download volume are ‘Big Data’ tools, so it made sense for these to get a separate event, although both are still hosted in succession at the same venue.

So what does Bowen think about the track record of the ASF? “If you look at just the web server market, we’re half of that. Hadoop is a very important player in the Big Data world. Lucene and Solr are very important in the search engine world. Tomcat is a major player in web applications.

Traffic Server

“The Apache Traffic Server – which hardly anyone has ever heard of – is a traffic accelerator and caching service used to deliver huge quantities of data on the web without ever being visible to anybody. So, yes, I’m very proud to be a part of the Apache Software Foundation. I think what we have done has changed the world.”

But running the non-profit presents its issues, chief among which is the need to increase its membership while keeping all the traits that make it unique. “When we were two projects, everybody knew what ‘the Apache Way’ was. As we started to bring on more and more projects, it became obvious that not everyone did,” Bowen explained: “Trying to maintain our culture while growing – you see this in any big company as well – is a huge challenge. You see projects that come in and want to be a snowflake, they want to do things their own way while benefiting from being part of the foundation.

“Holding that line and remembering why it’s important has been a real challenge over the years, as more and more people come into the foundation who don’t have that back story, that history.”

I think what we have done has changed the world

Rich Bowen

Bowen admits the ASF is not right for everyone, and developers who want to join need to know what they are getting themselves into. His advice to them is to spend a few months watching the mailing lists – in particular, the incubator mailing list.

“The incubator is the part of our organization where we take a project and we say: here’s how you become part of the foundation. We train them on the collaborative decision-making process as we see it.”

And then there’s the financial question. The foundation strives to provide software for the public good, so most of its work is done by volunteers. Out of its $1m budget, the ASF spends 75 percent on physical infrastructure.

Bowen explains that projects are indirectly funded by the companies that have a vested interest. For example, the Hadoop project enjoys resources donated by a dozen different companies to run continuous integration systems and new-builds.

“CloudStack is a great example of this. People in the media seem to believe that CloudStack is a Citrix thing. In reality, what you see in Citrix is a small player [in development] and most of the people involved in developing CloudStack are folks who use it in their own infrastructure, so they provide developers and resources.

“Citrix sells a product that’s based on this and that’s wonderful – we wouldn’t survive without their sponsorship of the foundation – but it’s all these people who rely on the software and so contribute back to it.”

Could it be better?

So if the ASF is so reliant on tradition, what, if anything, would Bowen want to change about it? “I would like to see more Africans involved in our projects. We have a lot of Asian participation, and we have nothing from Africa – and I’m African myself. I want to see more women participating. I want to see less of us, old white guys, defining who the new white guys are that join our projects. This is a big thing that is really important to me over the coming 20 years – that we fix that. It’s going to be difficult, because the people who are trying to fix it are the ones who caused the problem – but we are making slow progress.”

Critics of the foundation say that the Apache Way is old and inflexible. They say the rules are too strict and public voting slows down the development process. But they can’t ignore the impact of Hadoop, or the meteoric rise of Spark. Or the fact that despite occasional criticism, the ASF still attracts hundreds of new contributors and dozens of projects.

The Apache Way is still delivering results, and in the age of agile development, automation and daily builds, it remains faithful to its original orthodox open source model.

This article appeared in the November 2015 issue of DatacenterDynamics magazine