Despite hosting thousands of queries from developers about the latest and greatest tools and technologies in the digital space, developer Q&A platform Stack Overflow continues to maintain a fairly old school approach to its own infrastructure.
Speaking on Scott Hanselman’s Hanselminutes Podcast last week, Roberta Arcoverde, Director of Engineering at Stack Overflow, explained how the company has retained on-premise approach to its infrastructure in a single application its whole Q&A platform.
“We run on-prem, we run on our own data center, we haven't gone to the cloud. We also have a monolithic application so we have not breaking down into services or microservices,” Arcoverde said.
Previous posts on Stack Overflow suggest the company migrated from Oregon to a QTS data center in New York/New Jersey in 2010. QTS operates three sites in New Jersey in East Windsor, Jersey City, and Piscataway.
Stack Overflow has also previously made reference to a Fortrust data center in Denver; Iron Mountain acquired Fortrust in 2017.
Arcoverde added that the company runs a single .NET-based multi-tenant web app running across just nine web servers. That application handles around 6,000 requests per second, and around 2 billion requests per month. Stack Exchange, the Q&A technology underneath Stack Overflow running more than 175 sites including ServerFault, SuperUser, and Arqade, also sits on those nine machines.
“Those servers...they run that five to 10 percent capacity so we could, in theory, be running on a single web server. [But we wouldn't want to do that. We are designed for low latency. We cannot handle a lot of memory pressure, so we designed for low allocations everywhere. We try to avoid creating objects that will have to be collected and avoid memory pressure on those nine web servers so that we don't have to stall on garbage collections.”
“That's allowed us to grow and to stay where we are right now running on nine web servers for years. We haven't bought a new machine in two years at this point.”
When asked about moving to the cloud, Arcoverde said the cost and latency compared to the current set up isn’t worth the effort.
“We have, as an exercise, thought about [moving to the cloud] many many times. When I joined, we did this regular exercise where we would try to understand how much it would cost to run StackOverflow on the cloud, and it was just never worth it,” she said. “These days, when we think about the cloud, we are thinking less about the power that he will take and worry about latency. We have an infrastructure that has single hops between nodes and those hops are connected via 10 gigabytes network cables, that's an infrastructure that's very hard to mimic on the cloud. But we are constantly re-evaluating.”
Despite hosting endless questions relating to the latest technology trends, Arcoverde said Stack Overflow had largely eschewed the likes of Kubernetes and microservices because the company wasn’t experiencing the kinds of problems those tools were designed to solve.
“Why do you break down a monolith into microservices or services? Typically, because you want to scale to separate teams, you want to have multiple teams working on the same project without stepping on each other's toes, you want to have fast deploys for example. Fast deploys have never been a problem to us.”
She noted however, this may change in the future. Stack Overflow has grown to around 50 engineers, all working on the same code base.
“It's becoming trickier to onboard new engineers in this 14-year-old code base. Perhaps [in the future] we will find ourselves in a situation where actually, shouldn't we break down this specific module into a service perhaps and give it to a specific team and have them own it so that they don't need to understand the entire code base anymore. [But] we are not there yet, I don't think this is a problem that we are facing right now.”
“We are constantly re-evaluating and changing and there are a lot of conversations going on right now about what are the parts of the monolith that we should be breaking down now that we are growing and preparing for the next stages of growth? We are pragmatists, so if the time comes when we need to do that, that's certainly something that we would consider.”
