To demonstrate how data centers and the Cloud are organized in a hierarchy, one can perform a simple test using an email application. Simply click on a message that has been received within the last few hours, then open one that was delivered a week or two ago and compare the time taken for each operation.

In many cases the older email will take perceptibly longer to open. The explanation is that the data comprising each message is being stored in different locations. The newer one may for example be held in a localized micro data center at the edge of the network, physically closer in proximity to you. Whereas the older one will have been archived to a hyperscale facility many hundreds or even thousands of kilometers away.

Proximity matters

There are many positive attributes of hyperscale data centers, such as economies of scale, high availability and reliability. However, one might have to sacrifice a modicum of latency. By contrast, an Edge data center will not have the same footprint or sense of scale as its larger counterpart. It will however, benefit from similar levels of security and availability, in addition to enabling faster response times for critical applications than those hosted in the Cloud.

What’s interesting is how both the data center market and its network of connected infrastructure has evolved. Nowadays, many regional or hyperscale data centers contain important archived information, whilst providing large volumes of Cloud cover for space and services to many of today’s smart-device users. Colocation data centers are distributed more centrally, performing as the work horses for Cloud computing. Closest to where the production and processing of data takes place are Edge data centers, providing the fastest possible connectivity and lowest latency to customers.

The greatest and most rapid technological changes are being generated from the Edge of the network. These are being driven in part, by increasing complexity, the proliferation of embedded sensors enabling intelligence everywhere for an IoT future, and the requisite bandwidth and low latency infrastructure for data intensive applications.

Take, for example, the case of Autonomous Vehicles or driverless cars. Here is a technology that many are predicting will become a commercial reality in a very short time scale. But consider the data requirement; a typical self-driving car generates data at a rate of 750MB/ sec. That’s like 2666 people simultaneously using the Internet! Put two or three such cars on the same road at the same time and it’s fairly easy to see how rapidly data throughput will move into the GB/ sec range.

The very action of ‘normal driving’ requires skill, complexity and decision making. Therefore the compute power needed to replicate those processes on board each vehicle will have to match the human brain in both speed of decision and reaction to any unplanned consequences of incidents taking place both inside and outside the driving environment.

All of this decision making will inevitably produce an enormous amount of data, in addition to any information being transmitted via the surrounding network and localized Edge data centers.

As passengers, the question must be asked, would we really want all this data to be sent to a hyperscale facility hundreds of kilometers away for processing? And how quickly could we depend on the autonomous decisions being made, should it suffer latency or take more time for such important data to reach the critical point of use?

Faster infrastructure for faster networks

It is said that the introduction of 5G would increase the speed of data transmission and therefore has the potential to faster propel Edge facility growth, and with good reason. Information cannot be transmitted any faster, if the surrounding infrastructure however, remains clogged, outdated and unable to perform the most basic of functions.

Intel once estimated that a single Autonomous Vehicle will produce 4000GB of data per day, which leads you to the inevitable realization that not only will we need a better connected network of data centers capable of transmitting that critical data with minimal latency, but also larger storage arrays and compute power will be required on board, as we move towards a more service-driven, analytics-based predictive future.

If Autonomous Vehicles are also to be considered as Edge data centers, this adds an additional layer of complexity to the conversation. Again, a single outcome remains clear; it will be essential for data processing to occur as closely as possible to the point of action, so that latency and speed of decision making are kept to the absolute minimum, without sacrificing reliability and availability.

The data centers playing host to such applications will therefore have to combine the speed of response of the smallest most local data centers with the “five nines” reliability of the largest hyperscale facilities. This requires an integrated and better connected network of facilities – an approach to infrastructure which is quite literally the most critical component of the modern, digitized world.

Driverless cars may be among the most exciting of technologies driving the demands for ever more sophisticated Edge data centers, but there are many others such as drones, AR/ VR applications and so-on. However, data generation will be dominated by machine-to-machine applications. Already the Internet of Things comprises an estimated 10 billion connected devices; Cisco estimates that soon it may be 50 or even 100 billion.

There is a wave of new data gathering strength and the data center industry must adapt to meet its demands. Therefore, the three-tier model of hyperscale, centralized and localized data centers will remain in place, but the constitution of its individual parts, especially those at the Edge, will have to evolve with higher levels of sophistication and change dramatically, for the better.

Mike Hughes is president for UK & Ireland at Schneider Electric