Early data centers were powered by masses of cables under the raised floor. To make the system easier to modify and maintain, and get rid of the skeins of cables that blocked airflow, the cables to the racks are now more usually condensed into a “bus bar” - a single solid conductor that runs overhead, carrying enough AC power to run all the equipment on a row of racks.

Racks are fed via “tap-off boxes” which simply attach to the bus bar, and connect down to the PDUs in the racks.

“AC buses are used pretty regularly for three-phase distribution,” says My Truong, field CTO at Equinix. He references Starline as “one of the most prevalent bus systems out there for three-phase power distribution in data centers.”

“Overhead bus bar systems eliminate the need for remote power panels, resulting in more usable space for server racks,” explained Marc Dawson, director for bus bars at Vertiv, in a blog. “Tap-off boxes can be placed at any point along the bus bar therefore eliminating miles of power whips.”

But what goes through the bus bar? Traditionally it has been alternating current. What if that could be replaced with DC?

DC distribution - a fad of the 2010s?

In the 2010s, a number of data centers were built which operated with DC power throughout. The goal was to eliminate the energy losses that took place when power was converted multiple times between AC for transmission, and DC for charging the UPS batteries and running the IT.

“The DC distribution ideas which first raised their head in the late noughties were based on the replacement of 'conventional' AC distribution arrangements within the building,” explains DCD Academy instructor Andrew Dewing.

Intel experimented with using 400V DC to distribute power around the data center, but the leading proponent of DC distribution was probably Validus DC Systems, a company founded by Rudy Kraus in 2006 and bought by ABB in 2011.

“Direct current is the native power resident in all power electronics,” Kraus told DCD in 2011: “Every CPU, memory chip, and disk drive consume direct current power. Alternating current was chosen as a power path based on criteria set 100 years ago, 50 years before power electronics existed.”

As Dewing explains: “The ideas that ABB tried to push utilized the idea of DC distribution from the UPS DC bus downwards to the load allowing the removal of the inverter stage of the UPS. The idea was that by distributing at DC you could remove a lot of transformers that might be used and save energy.”

Some flagship data centers around the world tried it out. “We built three centers of mention,” says Kraus, now chief commercial officer at VIE Technologies: “One at Syracuse University with IBM, one in Lenzburg [near Zurich, Switzerland] for HP, and one for JPMorgan Chase in New Jersey.”

Others were reported, including one in Ishikari City, Hokkaido, Japan.

Validus distributed power at 380V, and claimed that DC distribution could produce huge savings. Validus said 2.5MW data center using DC would use up to 25 percent less energy, saving $3 million a year at 2009 prices.

There would be big savings in the electrical distribution system, with equipment down by 50 percent, installation by 30 percent, and it would only need half the physical space. With those simplifications, it would also be more reliable, the company promised.

Other benefits include no longer having to synchronize power systems. With no cyclic variation in voltage, there are no harmonics, and no problem with devices getting out of synch.

“You could literally put anything you want that’s DC on that bus,” says Kraus. “You could do fuel cells, you could do DC UPS, you could do rectifiers, you could do batteries, you could do windmills. And all this stuff is very, very simple.”

With all these benefits, ABB said it had bought a "breakthrough" technology in 2011, but the industry was not convinced.

Safety first

Among the naysayers, the influential Facebook engineer James Hamilton cast doubt on the claims made: “The marketing material I’ve gone through in detail compare[s] excellent HVDC designs with very poor AC designs. Predictably the savings are around 30 percent. Unfortunately, the difference between good AC and bad AC designs is also around 30 percent.”

Hamilton reckoned the efficiency improvements might be as much as five percent, but the small market for DC systems would make the equipment more expensive equipment, along with “less equipment availability and longer delivery times, and somewhat more complex jurisdictional issues with permitting and other approvals taking longer in some regions.”

Another issue was safety.

“A big drawback with DC power distribution is that safety is much more expensive to achieve,” says Ed Wylie, of American Superconductor (AMSC).

He sees this as a general electrical principle. AC switches can be less bulky, because their current oscillates and regularly passes through zero, at which point it is easy to physically move the switch contact.

“With DC,” says Wylie, “the current might be maintained, and you burn out the switch fairly quickly. If you look at the equivalent rating of a DC switch, and an AC switch, one is monstrous, and the other is small and compact.”

In 2018, OmniOn’s Vito Savino was employed by ABB and saw Validus’ trial data center. “I went to a small data center in Syracuse University, New York, that was kind of the pilot that Validus had done,” he told DCD. “And it was appropriate for it to be in a lab environment.”

Like Wylie, Savino says DC distribution was held back by the lack of safety devices. “When you're running DC at a high voltage, over the SELV [Safety Extra Low Voltage] level of 72V, you can really hurt somebody if they happen to touch the wrong places.”

At that time, he says: “The protection devices that are supposed to kick in were not available. They weren't made yet for DC, and therefore it was an experiment, and it did not go very well. It was not well received because nobody wanted to be around that voltage. When we talk about DC voltages, 48V is a safe voltage, unless there’s a lot of current behind it.”

As Savino puts it: “When we think about it from a safety perspective, we want that 48V to be contained inside of the rack where only a technician would ever be able to see it and the closest they get to it is putting a breaker into the power distribution unit that is encapsulated in the cabinet.”

“There's a lot of human resistance,” he says, clarifying that he is talking about social reluctance to adopt the technology, rather than electrocution, and saying that such is justified.

Kraus says: “It's not a safety issue, in my opinion. If you look at the Navy, the Air Force, and the phone company. They are all using it.”

As it turned out, the idea also had limited application internationally, according to Dewing: “In the US electrical systems there were many AC voltage levels and, at the time, a more conservative approach to the use of isolation transformers, for example. Whilst the theory might have worked for the US it was never viable in other parts of the world where electrical distribution was much simpler with only two or three voltage levels (e.g. UK mostly has 132kV, 11kV, 400V).”

DC distribution in the building could also have struggled to deliver enough power says Dewing: “The voltage levels of DC suggested (circa 500-700V DC) would have necessitated very large cabling requirements in a world where space was at a premium.”

The new current wars

Kraus thinks these objections were born out of simple stubbornness: “The AC guys fought it. It was back to the Edison-Westinghouse days. There was a major fight between DC and AC.”

He’s referring to the 1890s, when Edison pushed DC distribution for the grid, but ultimately lost out to Westinghouse’s AC equipment, which Edison said was unsafe.

“As time goes, we're surrounded more and more every day with DC products,” he says, citing lights, electric cars, and more. “The fact of the matter is, I think I was right. I think that DC will continue to grow.”

In the end, ABB did not develop the Validus technology. “They had a switch of CEOs at that point, and decided not to do much with it,” says Kraus.

“Honestly, I don't think ABB learned much with the Validus acquisition,” says Savino. “It kind of just went away in terms of productizing IT and marketing it.”

DC distribution may have faded out of data centers after 2010, when it seemed too risky and unusual to run DC voltages through the building, But there have been some big changes since then.

Go Direct

Firstly, racks these days may be fitted with DC bus bars. Secondly, data centers may be able to draw their power from new sources like solar arrays or fuel cells, both of which deliver power in DC form.

Concerned about the reliability of the grid, or believing they can generate power better themselves, some operators are moving to microgrids with on-site power sources. These sources, including hydrogen fuel cells, and solar arrays, provide power as DC.

When data centers draw power in as DC, and use it to run DC racks, the AC stage looks like a smaller part of the system, and that’s making some people look again at the idea of skipping the AC part of the loop.

“A Bloom fuel cell is a really great example,” says Truong. “Those naturally produce 380V HVDC right out of the cell. But because we're so AC-centric in most localities, we DC-AC convert that to 480V AC three-phase, run it into the building, and then reconvert it back to DC at least once to go through UPS, and then reconvert it to AC to send it to the IT equipment!”

That 380V DC power could be sent straight to the IT with relatively little effort: “There are tricks you may not be aware of. A standard 240V power supply will probably conveniently ingest a 380V HVDC input with minimal friction. It's just not certified that way.”

He spells it out: “If we have power supplies that could naturally take 380V HVDC, there's nothing preventing us from passing that power directly out of a fuel cell, not taking that three, eight, or 10 percent conversion loss along the way, and having a relatively more efficient overall power distribution.”

It doesn’t just help energy efficiency, he says: “Bussing DC also has some nice other artifacts. Because it is a DC bus, you don't have to phase sync, you don’t have to do a bunch of AC antics to synchronize up supplies. So you could actually have a solar array feed into the bus and pick up some of the slack of the system with battery systems. There's a lot of opportunity for us to go rearchitect when we go to DC buses.”

Traditional AC power distribution simply repeats what has been done before without trying to understand the underlying principles, like a “cargo cult,” says Truong: “I like to say we cargo-cult it forward until the thing stops working for us. And I think we're at that inflection point in the industry, where we have cargo-culted AC forward as far as we can.”

It’s tough to make changes inside colocation and enterprise data centers, though. “At Equinix, we are very comfortable with AC distribution today,” says Truong. “To advocate for a DC distribution mechanism, we would have to have the customer come to us and say ‘we need 480-volt distribution, and we want to do it this way.’”

Hyperscaler providers own the design end-to-end, but others don’t have that luxury, he says: “How do we collaborate with the end user and recognize the benefit to doing this, versus our classic data center designs?”

Colocation customers won’t be willing to go to DC distribution until it looks like a safe option, backed by popular standards, and that is what Truong aims to do in his other job as chair of the Open19 group, now known as SSIA.

“For us to break the paradigm, we have to break a bunch of things simultaneously, and get customers comfortable with breaking it,” he says. ”But we need to show some value to them. And that is very hard for us right now.”

Rethinking in California

One man with the freedom to rethink things is long-time data center simplifier Yuval Bachar, the original founder of the Open19 group, now at California-based startup ECL

ECL has built a 1MW data center powered by on-site hydrogen fuel cells and is proposing to build more for customers wherever there is a supply of hydrogen.

Each facility will be built for one customer, and that customer can choose how radical they want to be. That’s because, Bachar says, the power distribution inside the facility will be the customer’s choice.

“We can actually push DC directly to the data center, and distribution inside the data center is based on customer requirements. For us, DC is better because we save stages of conversion. But if the customer wants us to actually use DC inside then we do the conversion for the customer. Our design is capable of doing either.”

Bachar is well aware of the history of DC, and the safety concerns around it. The ECL design can include transformers for conditioning, along with DC-to-DC converters: “We can actually drive any voltage of DC, we can drive AC, and we can drive a blend of them as well. You can have half DC, half AC if you want.”

For him, the key concern is how much current the design will require in the distribution bus bars that go through the building when rack densities get high. If it is distributed at a low voltage, then the amount of current required to supply multiple racks would be “challenging,” he says.

“The reason for that is, at the amount of power that we use per rack, we're going to have to drive into distribution system 1000s of amps at low voltage. You could be close to 10,000 to 15,000 amps. That requires pretty special copper distribution.”

He’s not wrong there. That much current will require carefully designed thick copper bus bars, and resistance will be a problem. So much so that at least one group of academics has suggested using high-temperature superconductors.

In the real world, with regular copper conductors, high currents mean “you're going to get heat, and you can get sparking,” says Bachar.

His answer is to distribute higher voltages, which means a more reasonable level of current, and then convert the voltage down at the racks.

“We're trying to put the voltage high enough to be safe,” he says. “For 1MW, we push at 800V, 750V, or 700V. That’s about 2000A, which is reasonable. To increase that five times or six times, that is becoming a nightmare.”

Pushing for standards

Pioneers like Truong and Bachar are happy to push boundaries, and sometimes things are easier than one might expect. For instance, Savino remembers that his company, OmniOn, was once part of General Electric (GE) and found that AC equipment worked on DC.

“GE is very big in AC distribution, and we learned that we could use their AC bus bar scheme,” he says. All we had to do was get a rating for it from UL [Underwriters’ Laboratories], at a DC level, and we could use that same bus bar for carrying DC.”

However, the broader market is not going to step outside of conventional practices until DC schemes are broadly backed by UL certificates and standards.

Truong thinks consistency might persuade more people to have a look at it. “How do we bring a consistent enough standard that decouples enough of the pieces that were historically not decoupled, and allows the various organizations to really trace advantages on the sustainability factor overall?”

Persuading people to adopt something new will depend on them seeing business value, Truong says - and he sees a promising way to deliver that: “Right now, our opportunity to show business value is really on sustainability. Frankly, I don't believe that most organizations are going to be able to meet their sustainability goals without radically changing the way they think about things.”

He also knows he has to persuade OEMs to accept a standardized approach: “OEMs are semi-reluctant to adopt this because they don't see a customer base for it. We have a bit of a loop to break through there, as well as getting OEMs to recognize that interoperable standards in server designs are actually to their best benefit. I think that they still see value in customized products, They'll probably take a little while to rethink their current product value.”

Back in 2011, when Amazon data center veteran James Hamilton dismissed DC distribution, he felt it might break through in the end: “The industry as a whole continues to learn, and I think there is a good chance that high voltage DC distribution will end up becoming a more common choice in modern data centers.”

He may turn out to be right.

Kraus is in semi-retirement, but he’s ready to get back into the industry if DC distribution makes a comeback: “We proved out some pretty interesting things. And I always believed the question was, how long would it take to come around again?”