On-chip interconnects §

Why is RC delay the dominant physics on-die, when board-level traces are transmission lines? A wire must be treated as a transmission line only when its length is a significant fraction of the signal’s rise-time distance ($L \gtrsim t_\text{rise} \cdot v$ where $v$ is the propagation speed, roughly $c/2$ in silicon dioxide). On-die wires are µm to a few mm long with sub-100-ps rise times, so they behave as lumped RC circuits: no reflections, no impedance matching, no termination. The wire is a resistance in series with a capacitance to the substrate and to neighbouring wires.

The classic model is the Elmore delay for a distributed RC line of length $L$, with resistance $r$ and capacitance $c$ per unit length:

The quadratic dependence on length is the single most important fact on this page: doubling a wire’s length quadruples its delay. Three design consequences follow:

1. Repeaters. Long wires are split into $n$ segments separated by buffers, turning $L^2$ into $n \cdot (L/n)^2 = L^2/n$. This is why NoC routers exist at all: a 10 mm wire is unusable, but ten 1 mm segments with a register at each boundary are fine.
2. Wires do not scale. Transistor delay has improved by orders of magnitude per process generation; wire resistance and capacitance have not (copper resistivity rises with thinning, and capacitance is set by geometry). On modern chips, interconnect — not transistors — sets the critical path.
3. Crosstalk. The same capacitance that sets delay couples neighbouring wires; a bus bundles its wires so they switch together, and spacing or shielding protects critical signals.

A shared bus is the simplest interconnect topology. All masters (CPU, DMA, GPU) connect to the same address, data, and control lines. An arbiter (typically a priority encoder or round-robin) grants bus access to one master at a time; every other master tri-states its drivers.

Example. The canonical on-chip example is ARM’s AMBA family, which wires the peripherals of most ARM-based SoCs (and many others). AMBA AHB is a multi-master system bus supporting burst transfers and split transactions; AMBA APB is a simple, low-power, single-manager bus for slow peripherals (UARTs, timers, GPIO). A typical SoC hangs a fast AHB backbone off the CPU and bridges it to an APB segment for the slow periphery. AMBA AXI (IHI 0022) generalises this into point-to-point channels — see the crossbar section below.

The key trade-off: a shared bus is cheap — the wire count is $O(1)$ in the number of masters, since the bus width does not grow — but the aggregate bandwidth is fixed. Adding masters adds contention, not throughput.

Fairness. Under strict priority, a low-priority master can be starved indefinitely (a bounded-unfairness problem, distinct from priority inversion, which is when a low-priority holder of a resource blocks a high-priority requester). Round-robin or weighted fair queuing guarantees each master a bounded wait.

Try it yourself: the simulation below runs four masters against a strict-priority and a round-robin arbiter. Master 0 requests every cycle; watch its share of grants under each policy, and watch the starvation counter for the low-priority masters.

A crossbar (or crosspoint switch) provides a dedicated path between any input and any output. If there are $N$ inputs and $N$ outputs, the crossbar has $N^2$ crosspoints — one connection point where an input row crosses an output column, each a pass-gate plus its control logic. (A 2×2 switch is a small building block assembled from several crosspoints, not a crosspoint itself.) Each cycle the configured connections must form a matching: at most one connection per input and per output, which is the only way a crossbar “blocks”.

Example. A 4-core CPU with a 4×4 crossbar connecting four cores to four memory controllers. Each core can access its own memory controller simultaneously with the other cores accessing theirs — no contention. AMBA AXI is the standard channel-based protocol spoken over such point-to-point fabrics: five independent channels (read address/data, write address/data, write response) per manager–subordinate pair, which is what makes parallel paths useful.

The cost is quadratic in the number of ports: a 16×16 crossbar has 256 crosspoints, each with routing logic, and the wires to reach them grow too. This is why large SoCs move to ring or mesh topologies.

As core counts grew beyond ~16, the quadratic wire cost of a crossbar became prohibitive. A mesh NoC connects each router to its north, south, east, and west neighbours; a ring connects each node to exactly two neighbours. A packet traverses multiple hops, each adding one router latency (~1–2 cycles). The animation above walks a packet from router (0,0) to router (3,3) — six hops, the mesh’s diameter.

Examples. Intel’s client CPUs (Skylake through Coffee Lake) connect cores, cache slices, and the integrated GPU over a bidirectional ring; AMD’s Infinity Fabric in Ryzen/EPYC is likewise assembled from ring segments. For larger fabrics, Intel’s Xeon Phi (Knights Landing) used a 2-D mesh across its 72 cores. Coherent NoC protocols are standardised too: AMBA CHI (IHI 0051) and ACE define snoop, DVM, and data messages over such fabrics, so that a mesh can be cache-coherent out of the box.

The trade-off: a mesh scales linearly in wire count — an $N \times N$ 2-D mesh has $2N(N-1)$ bidirectional links ($N$ rows of $N-1$ horizontal links plus $N$ columns of $N-1$ vertical links; boundary nodes have fewer neighbours) — but latency grows with the diameter, $2(\sqrt{N}-1)$ hops for an $N \times N$ mesh of $N$ nodes. Virtual channels and wormhole routing mitigate head-of-line blocking.

A register file is not a bus in the traditional sense, but it is where all on-chip interconnect begins: the ALU reads two registers and writes one result per cycle, and the “interconnect” is the metal routing from the execution units to the register file, typically 10–100 µm. In the memory hierarchy it is the top level — and structurally it is not an SRAM array: SRAM optimises density with few ports, while a register file needs many ports, so it is built from flip-flop/latch bit-cells with dedicated access paths per port.

Example. The x86-64 architectural register file has 16 general-purpose registers (64 bits each). Out-of-order cores keep a much larger physical register file to support register renaming — how and why is covered in CPU Architecture; the interconnect-relevant fact is simply that many execution units must each read two registers and write one every cycle, so the port count, not the capacity, dominates the design.

The design constraint is access time: a register file must be readable and writable in a single clock cycle (~0.3 ns at 3 GHz). Each additional read port adds a full set of word-lines and bit-lines spanning the whole array, so area grows roughly quadratically in port count while capacity grows linearly. This limits the size to a few hundred entries; beyond that, the array becomes too large (and its wires too long, per the RC-delay section) to access in one cycle.

A SoC has 4 cores and 4 memory controllers. Every link is 64 bits wide and runs at 1 GHz, so one link carries 8 GB/s. Compare three fabrics:

(a) a single shared 64-bit bus at 1 GHz; (b) a 4×4 crossbar; (c) a 4×4 mesh where each memory controller hangs off one router and each hop adds 1 cycle of latency.

(i) What is the peak aggregate bandwidth of each fabric? (ii) All 4 cores each demand 8 GB/s from a distinct memory controller. Which fabrics sustain this? (iii) In the mesh, a core sits 3 hops from its controller. What latency does that add compared to the crossbar?