As data centers around the world gear up for the influx of data produced by new AI-generated loads, attention must be paid both to the integrity and the capacity of connections, both within the data center and outside.
Therefore, the decision to plan and maintain those interconnections has never been more important. DCD spoke to Nicholas Cole, global enterprise and data center solution manager at Exfo, one of the world's premier designers and manufacturers of network testing equipment for data centers, to talk us through the importance of upgrading and maintaining those vital fiber optic links.
Whether for a new build or an infrastructure upgrade, operators must decide which technologies and applications the data center must support, such as IaaS, SaaS, or AIaaS, and at what scale. This decision impacts how the network infrastructure is designed, including the number of servers, scaling options, and data transmission technologies like Ethernet, fiber channel, and InfiniBand.
The network distance, bandwidth needs, and communication structure will determine whether singlemode or multimode fiber is appropriate, as well as the optimal fiber configuration – whether simplex, duplex, or parallel optics. Each configuration has its own set of advantages and trade-offs.
Cole elaborates on this, stating, “Singlemode fiber is the preferred choice for long-distance communication because it can transmit signals at significantly higher speeds and over much greater distances compared to multimode fiber. It offers nearly limitless bandwidth capacity, making it ideal for scaling up network capabilities. In contrast, multimode fiber is preferred for shorter distances, such as within data centers or campus environments, due to its capability to handle high data rates over these limited spans.
“When deciding on the fiber configuration – whether duplex, or parallel optics (base-8 or 16) – the choice should be informed by both the current needs and the potential future growth of the fiber infrastructure. One challenge for data center operators is deciding how much fiber to put into the cable. This decision largely hinges on whether the fiber plant will be built around duplex or parallel optical cabling."
Cole continues: “Duplex cabling requires high data rates to multiplex signals across wavelengths rather than individual fibers, which means more expensive transceivers. Some companies have opted for parallel optics. That means rather than multiplexing across wavelengths, they multiplex across fibers. If you go down that path, you need much more fiber infrastructure, because rather than single connections, you're now typically using groups of eight or sixteen.”
Whether to deploy parallel optical or duplex cabling depends on network speeds and the scale of operations. When you hear about the plants of the big hyperscalers having millions of fibers inside a building, it is because they multiplex across fibers and not wavelengths, and that tends to be the future-proofing direction that most large cloud operators are moving in as it allows them to adopt the latest and greatest transceiver technology.
“As you increase the data rates, it's safer to put in more fiber because it can multiplex across fibers rather than wavelengths.”
Even then, though, the future is outpacing design, as Cole adds: “Things are still moving forward and we are looking towards a base-16 fiber cabling infrastructure. It's not just a simple case of more active equipment, it's also what's physically available in the marketplace. That's why they need to think, ‘How quickly do I want to move if I want to be at the cutting edge of the industry? Ethernet technology and transceiver technology are not going to move as quickly as I want to move.”
AI workloads require even more cables and high-end transceivers, with individual nodes interconnected to become more than the sum of their parts.
“That’s where parallel optical fiber infrastructure puts you in a better position. AI means building what they call ‘pods’ or ‘clusters’ – a sort of ‘Lego brick’ of AI. It needs to be high speed so that the pods' interconnection becomes high-speed too, and that’s before any traffic leaves the data center. I'm hearing people talk about 400G, 800G, and even 1.6 Terabit transceivers in that containerized pod. Fortunately, that's completely independent of the legacy architecture, so it means upgrading just becomes a question of space.”
Of course, these days of ever more cabling and high complexity transceivers translate into every piece of passive cabling and every active transceiver needing to be tested on installation and proactively as problems arise – and Exfo has the solutions for that. It’s an approach that benefits the profitability of the data center and the planet in equal measure, as Cole explains:
“There's an increased importance on networking and lots of high-speed transceivers that fail; so they all need testing. Because they're high-speed, they’re going to be expensive. Yet, what the hyperscalers and big cloud providers have been doing is just throwing these transceivers, worth thousands of dollars, away if they suspect they're faulty. Our equipment will test and make sure where the fault lies before they just chuck a $2,000 transceiver in the bin. We are helping with the elimination of waste.”
Fortunately, data center components tend to be platform-agnostic these days, but this can present problems that make regular testing even more important throughout the facility's lifecycle.
“Generally speaking, the industry wants standardization. No one wants to get locked into a proprietary system. The need for interoperability is prevalent in the industry. But although everything should be interoperable, the reality is that everyone may interpret the specifications slightly differently,” he says.
“When you put them together, you may get issues, it may work but you may not get optimum performance. We can measure the performance and then identify the problematic components. We're not only measuring the links in the system, we measure system performance, but down to the individual components.”
At this point, we switch the question of AI on its head and look inward to what technology can do to make data centers work more efficiently. Cole points out that the true possibilities will only come as we see the fledgling sector develop, but already there are some clear opportunities.
“Some obvious things would be the telemetry and control systems monitoring what's happening inside the data center. There are sensors everywhere. That's what AI feeds on – data. Harnessing that means you could optimize things very easily and it’s going to help with the Edge for sure. AI and Edge go hand in hand because Edge is going to be difficult to staff. If you've got many smaller facilities in geographically widespread areas, then physically having people inside them is going to be difficult.”
A more overlooked aspect of the data revolution is 5G, which reduces latency whilst increasing speed and capacity, bringing more opportunities for portable data consumption and M2M communication. Although it has been around for more than five years, its true potential and impact are yet to be fully realized – but it’s coming.
“5G just means more data at higher rates, with higher density of cellular networks because of the high frequency and amount of data that's being transported across it. At the moment, the promise of 5G hasn't happened, but more things can be connected, and we'll see it through IoT inside buildings, for example.”
But the real difference will come from outside the home.
“Cars and vehicles are the best examples where we’ll see an impact. We're going to need to reduce latency, so there's going to have to be processing happening closer to the connected device, and there we're going to see Edge data centers and mini-processing units closer to the point where the data has been created. Then you'll have this multi-tiered networking world where you have different rings around a core data center environment. The impact on data centers won't happen until you connect something to that network and technology, and it's ramping up.”
In finishing, we ask Cole if he can see the demand for data tailing off any time soon. The answer is emphatic: “Today, there are around 20 billion connected devices. By 2030, one estimate suggests more than 100 billion, and wow, I can already see that in my life. I have two phones, two iPads, two laptops. I've got three gaming consoles. I've got lots of things connected – and younger people are connecting things even more quickly. The more things we connect, the more data we produce, and the more applications that will be invented.
“I think this is going to be an intense decade for data digitization and connectivity for sure.”
Indeed, for such an intense decade as Cole predicts, intelligent decisions will need to be made about how to ensure that facilities are equipped for today, tomorrow, and the years of exponential growth to come. Failure to invest in and maintain optical connections will mean the stragglers will be left behind.
More from Exfo
-
Sponsored Optical fiber and high-speed data transfer to power the AI revolution
Why artificial intelligence requires networking intelligence
-
Keeping up with the evolving data center: part 2 – best-practice testing for DCIs
Once you've adopted new interfaces, you need to make sure they keep working
-
Keeping up with the evolving data center: part 1 – emerging technologies and related challenges
Increasing traffic is driving faster adoption of new interconnection technologies