Datacenter and networking topologies are being redrawn at the edge

As the next generation of digital infrastructure unfolds, there is a widespread expectation that a new tier of datacenters, or at least of compute and storage capacity, will need to be built to meet the latest demands of edge computing. The Internet of Things (IoT), distributed cloud and other edge use cases will require computing, routing, caching and storage, localized analytics, and some automation and policy management close to where there are users and 'things.' Some suppliers, ranging from Schneider Electric and Vertiv to Dell, Ericsson Huawei and Nokia, expect that microdatacenters will play a leading role in meeting distributed edge demand.

Others such as Google are less convinced and are focused on building networks and capacity one step back from the edge – to the 'near edge.' Some say that edge datacenters will be relatively niche, citing the main use case as CDNs because of their high data volumes. In most other cases, they say, data volumes and latency needs will not need a 'dedicated' edge datacenter. They say the buildout of new large and hyperscale datacenters, supported by reliable networks, will bring the edge much closer to users and 'things.'

Both arguments will likely bear out over time: the edge will almost certainly require many different datacenter types – and probably many of them. They will range from hyperscale cloud and large colocation facilities that are sited near or near enough to the point of use to support many applications, to new micro-modular datacenters at the edge, to smaller clusters of capacity that are not large or critical enough to even be described as datacenters.

The 451 Take

The connectivity paths and datacenters required for different edge use cases will vary, although patterns of similar architectures are likely (see graphical schemas below). We are bullish on the edge opportunity for micro-modular datacenters, including for integrated IT datacenter products via supplier partnerships (and with remote management services in the mix). At the same time, IoT gateways/systems are becoming increasingly sophisticated with more compute, storage and analytics. At the near edge, colocation providers are dominating – their focus is on developing properties into connectivity-rich multi-MW datacenters with a 15-plus-year investment timeline. This may change for top-tier metro markets in the next 5-10 years (or sooner) if cloud giants follow through on a strategy of building additional capacity or shift existing capacity to their own datacenters (along with new investments in dark fiber). It is unclear how far the cloud giants will reach into the 'true' edge where telcos will become either key players or key partners.

Where is the edge?

The location of the edge itself is largely defined at the highest level by the application and workload function of the compute. It can include the physical or virtual location of the following:

• Data: Where data is initially generated – including by sensors and other devices (things) – and processed, analyzed, consumed and stored.
• Connections: Carrier-neutral or carrier-specific connectivity services with cross connects, cloud exchanges or direct connects (including dark fiber to other datacenters). The physical point where data is integrated can also be considered the edge.
• Telecom gateways: Entry or access points to local/WANs, including fiber and cellular, as well as to cloud computing and other IT service environments on networks.

Distributed datacenters at the edge

The volume of traffic in some distributed edge datacenters will be very high, and many data sets and applications may be involved. Failures or congestion in networks may cause serious problems in machines, devices and user experience – driving requirements for flexible, agile networking approaches, such as software-defined networking. Two-tier, leaf-and-spine architectures inside the datacenter will be ideal for optimized throughput and redundancy because of the vastly increased traffic between different services.

To understand the different types of datacenters required at the edge, it is useful to differentiate between edge and near edge.

Edge datacenter functions

The true edge is sometimes defined as where near-real-time response and action is needed, measured in sub-2 milliseconds (low or ultra-low latency). Ideally, there would be no – or very few – network paths or 'hops' and very little or no use of shared communications infrastructure between the user/data collection point and the true edge point of processing/aggregation. (When 451 Research has asked enterprise customers what they mean by edge, answers have varied: Some say at the device; others at the first point of aggregation; and others at the point where CIO control, in the form of performance or security, etc., and ownership begins.)

In some cases, telecom gateways such as 4G and 5G base stations and towers will require dedicated datacenter capacity to be collocated very close by, most likely in the form of micro-modular datacenters. These will meet the needs of east-west traffic at the edge where fiber is not an economic or otherwise practical option, such as, for example, between end-user devices and services near each other.

Some edge applications, including process manufacturing and the teleoperation of vehicles, generate data that requires rapid response and action. They need platforms that transform the data streams into formats that can be processed by applications that analyze and act on the data in real time. Wireless networks to support this with management software, protocols and processing/storage are known as ultra-reliable low-latency communications. In these environments, compute and storage capacity is required very close to the point of data generation – these are true edge computing functions.

This true edge processing capability is also referred to as 'fog computing,' which involves performing the required analysis and taking the resultant action as closely as possible to the data – or things of IoT. (In other words, below the level of the centralized cloud).

To address this need, manufacturers such as Intel, HPE, Nokia, Dell EMC, Lenovo and Huawei, among others, sell local gateways that are essentially access routers with special interfaces (MODBUS, CANBUS, etc.) with varying degrees of compute and storage capability and that have the capability for VMs or containers to run applications. While this is a good and possibly the only viable solution for many environments, the scale of some potential IoT deployments will require hundreds or thousands of gateways, representing a considerable capital expense, in addition to the ongoing management and security of the gateways. In these cases, localized datacenter capacity will be needed that is one step back – the near edge.

Near-edge datacenter functions

The near edge can be broadly defined as the 'zone' where sub-millisecond latency, or low-single-digit latency, cannot (usually) be guaranteed across available networks, but performance and security has been architected to securely process, analyze, store and forward larger amounts of data – and possibly to connect to other applications and data sources – without going back to a centralized cloud service. Near-edge datacenters will likely be a mix of microdatacenters and much larger facilities, including enterprise, colocation and cloud datacenters that are sited deliberately or coincidentally near the user of the data.

Equinix, for example, positions its colocation and interconnection facilities as offering cost and management benefits over fully distributed edge environments (one or more aggregation points or devices at the interconnect versus hundreds or thousands of smaller devices to manage) and performance benefits over fully centralized cloud models. One of the notable moves by the company in this area is its Data Hub, which combines its interconnection and colocation service with tiered cloud-integrated storage to provide a potential aggregation layer for large amounts of IoT and other edge computing data. The service offers the opportunity for near-local storage, analysis and action of data, including data integration, as well as potential summarization of a large amount of data prior to backhaul to a centralized cloud.

Networking and datacenter topologies at the edge

Generally speaking, datacenter and network topologies to support low and medium latency and high traffic will vary. The deployment of 5G networks (likely to begin in 2019/2020 in pioneering countries such as South Korea and Japan) will reshape connectivity architectures for edge datacenters. In principle, 5G enables very good connectivity coverage from multiple gateways (cell towers) – if one gateway fails, the signal is picked up by another. With its 10Gbps design goal, one potential application of 5G is to replace the need for local high-bandwidth fiber to the edge, especially in some remote locations.

Ideally, 5G networking routes will be optimized, hopping intelligently between different 5G networks and cell towers. To optimize and utilize bandwidth more effectively, network management is increasingly being virtualized and separated from switches and devices. If visualized, these edge network and edge datacenter architectures may look something like the schema below, with true edge microdatacenters, and IoT and telecom gateways (at the very bottom) performing aggregation, control and analytics.

Although edge computing itself is not a specific technology, it will incorporate and attract a host of datacenter technologies and techniques that will make edge processing and storage a more effective deployment option. These include but are not limited to:

• Converged edge infrastructure incorporating IT, communications and facilities
• Prefabrication and micro-modular packaging
• New designs for embedding intelligence in infrastructure
• Software tools specifically designed to analyze, store, integrate and forward data
• Policy-based software designed for initiating automated action at the edge
• Remote and integrated facilities and IT management, including 'as a service'
• Distributed resiliency and availability tools