Paper 04: Scale-Free Networks — Why Systems Drift Toward a Few Services Everyone Depends On

I. Abstract #

Splitting software into small services promises that load spreads out and failures stay put. The Barabási-Albert network model says otherwise. When a network keeps adding members that attach to crowded spots, most services end up with a few links while a small group holds hundreds. Nobody designs that group; when one of them stalls, failure travels backward to everyone waiting on it. This paper covers the math, the backward spread, and three ways to live with it: asynchronous calls, hard resource partitions per caller, and cutting off early when that service slows.

II. The “Split It Up and Load Spreads Out” Promise vs. the Real Graph #

Architecture slides have one underrated strength: every box is the same size. Redraw a system that has run for a few years by who really calls whom, and the picture usually changes: a few boxes have arrows coming in from almost every other box. Those boxes are my subject.

Each box is a service: software on a server doing one job, such as login or payments. Each arrow is a dependency: one service must call another to finish its work. The number of arrows attached to a box is its link count.

Microservices means splitting a system into many small services that call each other over the network. It replaces the monolith: one block of software holding every feature. The promise: split things up, and load and risk spread out.

That promise pictures an Erdős-Rényi random network: every pair of services is equally likely to be linked. Then every service has close to the average link count, and “the average service” means something.

Real systems usually differ: new services do not link at random, so a few core services end up carrying the whole organization. Each is a single point of failure: if it breaks, everything breaks, the very thing microservices were meant to remove. Why? Two rules suffice.

III. Two Rules Are Enough to Create a Hub #

The Barabási-Albert model (1999) reproduces this with two rules:

1. The network keeps adding members. New services keep arriving.
2. New links favor crowded spots. A new service attaches to the services everyone else already uses.

Rule two is preferential attachment: the more links you have, the easier the next one. Its everyday version is the colleague who knows everything. Everyone who asked him recommends him to newcomers, so he gets asked even more, even if others are just as capable.

Suppose a food-delivery app has a login service, auth. Every team needs to know who is using the app, so each calls auth. Each time that happens, auth becomes the more obvious choice for the next team. No meeting ever decided “make auth a single point of failure”. There were just forty small meetings, each sensibly concluding: reuse what exists.

In graph terms, each service is a node, and a node with far more links than the rest is a hub.

Run the two rules long enough and link counts follow a power law¹: the more links, the rarer, but never rare enough to vanish, so a few giant hubs always exist. Here, ten times the links is about a thousand times rarer. Such a network is scale-free: no “typical size” represents its nodes.

So the average stops being representative. With 2 dependencies per new service, a 200-service network averages 4 links, and its largest service has about 30. At 20,000 services the average is still 4, but the largest has hundreds (382 in Section VI). Sizing machines by “the average service” fails exactly where it is most dangerous: at the hub.

Early arrivals also keep a lasting lead: in the simulation, service 100 ends with 28 links, service 10,000 with 3.

IV. From Network Model to Your System: The Mapping #

The table needs one more concept: a synchronous call. That is when service A calls service B and waits for the answer before doing anything else, like staying on the line during a phone call.

Age is not the only factor. The Bianconi-Barabási model (2001) adds fitness: each node’s own usefulness, multiplied into its chance of being picked. So auth, built later, still pulls dependencies faster than an older reporting service, since everyone must log in.

That is why splitting a God service usually fails. Splitting the code takes a few sprints; splitting the reason everyone needs it is harder, because afterward everyone still needs to log in.

How Does a Service Nobody Calls Get Its First Caller #

The original Barabási-Albert model ignores direction, but software dependencies have one: A calling B does not mean B calls A. Count only inbound links and a service that just went live scores 0, so under rule two nobody ever calls it.

Price (1976) fixes this by giving every node a small initial attractiveness². In a real system, that is whatever gets a service noticed before it is popular. It may be a line in the internal service directory, a demo, or solving a problem someone has today.

Together, these mechanisms pile callers onto a few nodes, and a hub forms without anyone deciding. What happens when it stalls?

V. When a Hub Stalls, Which Way Does Failure Travel #

A scale-free network shrugs off random failures, because a randomly failed node is usually small; hit the hub and the network falls apart. In the food-delivery app, a failed reviews service hides the reviews; a failed auth stops all orders.

The Hub’s Load Ceiling #

Every hub has a hard ceiling: how many requests it handles at once. A request is one message to a service, like the app asking auth “is this user logged in?”. Past that ceiling, requests queue or get rejected. The ceiling usually comes from a connection pool, or from the thread pool described below. A connection pool is a fixed-size set of database connections, opened in advance and shared, like a bank branch with a fixed number of counters.

Suppose auth handles at most 200 requests at once, and at the lunch peak each caller keeps about 5 requests waiting on it. 200 divided by 5 is 40: auth can handle 40 callers. Then service 41 goes live, say meal recommendations. No alert fires: the dashboard shows auth at 45% CPU, since requests waiting in line use almost no CPU.

Failure Runs Backward, Not Sideways #

If I had to guess which way failure spreads, I would say along the arrows, like water through a pipe. I would be off by exactly 180 degrees.

In Motter and Lai’s cascade model (2002), written for networks like power grids, a failed node’s load shifts to other nodes along new paths: failure spreads sideways. Microservices have no such detour, since no other service can do the login job. When the hub freezes, requests to it just wait.

A thread is a line of work running in parallel inside a server; a thread pool is a service’s fixed set of threads. When auth freezes, each request to the promotions service calls auth and holds a thread. Soon the whole pool is full, and the promotions service takes nothing more, not even work that needs no auth. Then its own callers run out of threads, and so on up the chain.

At the office, it is the afternoon the colleague who knows everything spends in meetings: whoever waits on him stalls, and so does whoever waits on them. The example breaks at one point: a person can say “I’ll answer this afternoon”, while a synchronous call holds the caller until its maximum wait time runs out.

So failure runs backward against the dependency direction. One name for this is synchronous backpressure. Backpressure is pressure pushed upstream: a slow receiver forces the sender to slow down, here through the waiting synchronous calls themselves. For a hub, the callers upstream are most of the system.

Section VI tests the chain’s first link: does a hub really appear on its own?

VI. Simulation: A Hub Appears Among 20,000 Services #

I simulated a Barabási-Albert network of 20,000 services, each new one bringing 2 dependencies, with nothing tuned to force the result. The network averages 4.0 links and 80% of services have 4 or fewer, yet the largest service has 382, nearly 100 times the mean. No line of code says “create a hub”; one appears anyway, and it is the first service added.

How to read the chart: both axes are log scale, each major tick 10 times the last; horizontal is link count, vertical is the share of services with that count. Orange is the data, dashed white the best-fit straight line. A power law shows as a straight line sloping down; its steepness here is 2.82 (a slope of −2.82), near the theoretical 3. Buckets are log-spaced, since even spacing shrinks the tail, where hubs live, to a few noisy points.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import numpy as np
import networkx as nx
import matplotlib.pyplot as plt

N, m = 20_000, 2          # 20,000 services, each new one bringing 2 dependencies
G = nx.barabasi_albert_graph(N, m, seed=42)
degrees = np.array([d for _, d in G.degree()])
print(f"Mean: {degrees.mean():.1f}, largest: {degrees.max()}, "
      f"share <= 4: {(degrees <= 4).mean():.0%}")

# Logarithmic binning so noise does not swallow the tail
bins = np.logspace(np.log10(degrees.min()), np.log10(degrees.max()), 25)
hist, edges = np.histogram(degrees, bins=bins, density=True)
centers = np.sqrt(edges[:-1] * edges[1:])
ok = hist > 0                       # drop empty bins before taking logs

# Fit a straight line on log-log axes: its slope is -gamma
slope, intercept = np.polyfit(np.log10(centers[ok]), np.log10(hist[ok]), 1)
print(f"Fitted exponent: {-slope:.2f}")

k = np.logspace(np.log10(centers[ok][0]), np.log10(degrees.max()), 100)
plt.style.use("dark_background")
fig, ax = plt.subplots(figsize=(10, 6), facecolor="#171717")
ax.set_facecolor("#171717")
ax.loglog(centers[ok], hist[ok], color="#F97316", lw=2, label="Empirical Data PDF")
ax.loglog(k, 10**intercept * k**slope, "--", color="#FAFAFA",
          label=f"Power Law Fit (γ={-slope:.2f})")
ax.set_xlabel("Degree (k)")
ax.set_ylabel("Probability Density P(k)")
ax.legend()
plt.show()

VII. Three Ways to Live With Hubs, and What Each Costs #

Hubs are an arithmetic consequence of preferential attachment, so the three measures below do not remove them; they route around them. Question for your engineering team: which service takes the most synchronous calls, and what is its load ceiling?

1. Move the hub into infrastructure with EDA and CQRS. EDA is an architecture where services announce “this just happened” through a message broker instead of calling and waiting. A message broker is intermediary software that receives, stores and forwards messages; Kafka and RabbitMQ are examples.

   CQRS, often paired with EDA, separates the write path from the read path so they stop competing for resources. The price is eventual consistency: data across services will match, but only after a delay.

   This sounds like tidying up but is really moving house: the hub does not vanish, it moves into the broker. The gain: a broker is built for this load; Kafka, for instance, appends to disk sequentially and holds a far higher ceiling than a business service. Senders drop off a message and move on, so no thread is held.

2. Isolate with bulkheads. A bulkhead is a wall dividing a ship’s hull into sealed compartments, so one breach does not sink the ship. In software, it splits the hub’s thread pool into fixed shares, one per calling service. A caller that fails and floods requests uses up only its share; the other 39 keep running.

3. Stop the backward spread with circuit breakers. A circuit breaker is a switch on the caller side. When the hub is too slow or keeps failing, the switch trips so the caller returns an error at once instead of waiting. It blocks the backward spread most directly, cutting exactly where threads get held.

VIII. References #

• Barabási, A.-L., & Albert, R. (1999). Emergence of Scaling in Random Networks.
• Bianconi, G., & Barabási, A.-L. (2001). Competition and multiscaling in evolving networks.
• Clauset, A., Shalizi, C. R., & Newman, M. E. J. (2009). Power-Law Distributions in Empirical Data.
• Motter, A. E., & Lai, Y.-C. (2002). Cascade-based attacks on complex networks.
• Price, D. d. S. (1976). A general theory of bibliometric and other cumulative advantage processes.