Our relentless demand for generative AI is driving the emergence of a new type of networking model: “edge networks.” Today, we are witnessing the global development of edge data centers located closer to the end user to meet GenAI’s demands for real-time responsiveness and low-latency performance. To put it in numbers: analyst firm IDC predicted that edge computing spending is expected to reach $232 billion in 2024, up 15.4% from 2023.
Edge data centers are small data centers and processing locations that are part of a distributed network of data centers. They are located close to the areas they serve, which significantly reduces latency and improves the performance of applications that require real-time data processing. This decentralized approach also helps balance loads, guarantees data flows in the event of an outage, and improves the overall resilience of a network.
Director of Technology and Operations, Neos Networks.
Supporting GenAI
The need for edge networks is clear. AI applications are data-intensive and compute-intensive. However, edge computing promises to overcome these technical challenges by enabling real-time decision making with lower latency, local data storage and processing, and less data transfer to the cloud. This is particularly pertinent when it comes to the need for localized data processing and inference.
As GenAI requires even faster processing, there will be many existing and new applications where networks will need to deliver ultra-low latency. The more time-sensitive the application, the more data will need to be stored and processed at the edge. Take AI inference (using an AI model to draw conclusions from new information or data) as an example.
Data processing at the edge can reduce the time it takes to get a result from a few seconds to a fraction of a second. Moreover, several other emerging use cases in the industry highlight why computing needs to be located close to the end user, be it in content generation applications such as ChatGPT, interactive customer service agents, immersive augmented reality experiences, smart healthcare, smart retail, and predictive maintenance. In these scenarios where every millisecond counts, the user will enjoy a higher quality experience if the computing is housed as close to them as possible.
The sustainability argument
A recent TechRadarPro article argued that we don’t have the power to handle the current boom in demand for data centers, due to AI. That’s why we need to build data centers outside of central locations at the edge. According to Goldman Sachs, a ChatGPT query requires almost 10 times more electricity to process than a Google search. Despite the inevitable increase in electricity expenditure due to GenAI, edge data centers offer the advantage of reducing network power consumption at central locations. By distributing the computing load across the network, power demand is distributed, not concentrated. By running applications at the edge, data can be processed and stored closer to end-user devices, rather than relying on data centers that are hundreds of miles away.
Investing in an AI-ready network
Investment in high-speed connectivity will enable network edge sites to be connected in a more practical and sustainable way. Fiber optic cables offer significantly lower latency and higher bandwidth than traditional copper cables, enabling faster data transfer speeds. High-speed fiber networks are easily scalable, so as data demand increases, additional bandwidth can be provided without significant changes to the infrastructure. Fiber networks also consume less energy than traditional infrastructure, contributing to lower operating costs and carbon footprint. With advances in pluggable optics technology, the same economic, sustainability, and technological benefits derived from fiber are now being delivered within the data center.
However, while projects such as the UK’s Project Gigabit and the US Broadband Equity, Access and Deployment (BEAD) programme are a necessary step in the right direction, governments must prioritise building out the edge of the network and better connecting data centres, not just expanding fibre to the home (FTTH).
The key to AI success
In the race to become AI leaders, countries must foster startups and define regulatory parameters. However, AI success will depend on a country’s fixed network infrastructure and its ability to transport significant amounts of data with little or no latency. If networks cannot cope with the influx of traffic generated by “always-on” large language models (LLMs), AI ambitions may fail.
That’s why AI strategies need to focus on the size, location, and quality of the underlying network infrastructure. Even as widespread investment in “traditional” data centers has skyrocketed around the world—such as Google’s new $1 billion data center in the U.K. announced earlier this year and Microsoft’s $5 billion investment in building data centers in Australia—less attention has been paid to edge data centers. To meet AI demands, data center construction must be complemented by edge construction.
A hybrid model?
To manage the rapid flow of information cost-effectively and sustainably, it will be essential to take a hybrid approach of strategically located data centers at the edge of the network, in combination with central data centers. This is particularly crucial for AI inference, where data flows go to the edge for processing and then back to central data centers for distribution. Time-critical applications will be better served near the edge of the network, but applications with large amounts of data and less time-critical data will be better served in central data centers.
With major companies like Microsoft setting ambitious goals to triple their data center capacity over the next year to deliver AI, we should also look at edge data centers as a considered part of the strategy, not only to meet the low-latency requirements of GenAI applications, but also to alleviate energy pressure on the core network.
We list the best AI tools.
This article was produced as part of TechRadarPro's Expert Insights channel, where we showcase the brightest and brightest minds in the tech industry today. The views expressed here are those of the author, and not necessarily those of TechRadarPro or Future plc. If you're interested in contributing, find out more here:
