AWS’ Matt Rehder deploys RNG flat networks: up to 40% more energy efficient, at scale

Amazon’s network breakthrough is not a software rewrite or a new chip. It is a topology change that AWS says makes datacenter networking up to a third faster and up to 40 percent more energy efficient than traditional hierarchical designs. And the really interesting part is why this took so long: the “random graph” networks academics proposed in 2012 were theoretically elegant, but they ran into scaling problems in routing complexity and cabling chaos.

In a recent interview, Matt Rehder, VP of global network engineering at AWS, explained that traditional datacenter networks are hierarchical, “sort of like an org chart,” where devices talk “up the chain of command” to reach other departments. Hierarchy does make routing simpler, but it also creates contention where data bottlenecks can form, while other parts of the network can sit underutilized. RNG, or Resilient Network Graphs, is AWS’ attempt to keep the performance and resilience benefits of a flatter graph, without inheriting the dead ends that previously blocked production deployments.

So what actually replaces the org-chart approach? AWS has built RNG as a mostly flat graph, but not “entirely random.” The project grew from academic ideas in random graph topology and an earlier design called Jellyfish, which depended on truly random graphs and called for removing routers from server racks and locating them centrally to simplify cabling. Rehder said Jellyfish increased latency between servers within a rack, and that academics had also run into another wall: production-ready random routing is hard because devices have limited memory space. “It requires much more complicated routing rules to figure out how to program every device - you can't just program every device to know who everyone is,” Rehder said. There is also the build reality of cabling, where randomness can turn into literal cable spaghetti, something you can demonstrate in a lab but struggle to scale across a real datacenter.

AWS’ internal execution was the difference. Rehder said the company has spent the past few years solving these problems, enabled by a “15-year history” of iteratively improving hardware and software ownership of its network. The approach eventually succeeded by mixing deterministic and randomized elements in its cabling. Internally, the effort was referred to as Penrose, because the original design involved Penrose tiles. As the project evolved, AWS settled on the name Resilient Network Graphs “to reflect the customer benefit,” including “a more resilient and performant network,” according to a company spokesperson.

Two specific pieces of the RNG system are doing the heavy lifting. RNG uses a routing algorithm called Spraypoint to identify node paths. And it uses an optical device called a Shufflebox for mixing connections between routers. Rehder described the Shufflebox as a “magic” component for making a random network feel structured in physical reality. In a random graph network, you do not get neat, hierarchical cable alignment. So AWS scrambles or randomizes fiber connections inside the Shufflebox, meaning the fiber ports you plug in are “scrambled around” and come out on a randomized port on the other side. The core idea is to preserve routing flexibility while making interconnects operationally manageable.

Behind the scenes, AWS also connected the dots between academia and execution. RNG took shape about three years ago when Seshadhri Comandur, an Amazon Scholar and professor at the University of California, Santa Cruz, answered an internal Slack message from Ratul Mahajan, another Amazon Scholar and a datacenter networking expert and professor at the University of Washington, who was looking for graph theory and routing expertise. With help from AWS principal applied scientist Giacomo Bernardi and other colleagues, AWS became the first company to deploy a flat datacenter network at scale.

Where AWS has put RNG also matters for interpreting the business stakes. RNG is AWS’ new network for core database servers. Rehder said the company’s UltraServer network is used for machine learning hardware because those workloads need full bandwidth. The pitch for core server networking is that oversubscription can be handled “more efficiently” because “everyone's not talking to each other at the same time.” In other words, this is not just about squeezing more speed from pipes. It is about spending less on the network infrastructure that makes those pipes possible.

AWS says it expects the technology will deliver better performance and reliability for Amazon customers, while also saving billions of dollars in hardware and reducing CO2 emissions. RNG has already been rolled out in Ireland, Germany, and Spain, and AWS plans to deploy it in the majority of company datacenters by the end of the year. For peers across cloud, colocation, and AI infrastructure, the strategic question is straightforward: will networking hardware and routing software evolve together like AWS is doing here, or will the industry keep treating networks as fixed infrastructure until performance constraints force expensive, incremental patches?