Topology — the drawing and the reality

Every networking textbook has a page drawing four shapes — bus, ring, star and mesh — with the advantages and disadvantages of each.

The catch is that every network you can buy today is cabled as a star. Homes, offices, data centres: the cable runs from every point to a box in the middle.

Those four shapes therefore do not describe how to run cable. They describe two things that are still true — how many cables you must buy and how many cable failures it survives — and both are computable.

There is one more thing textbooks often fail to separate: the shape of the cabling and the shape the data actually travels are two different shapes.

If you have never thought about this, start here

Look at the cable in your machine and ask one question — where does the other end go?

Almost always the answer is "to the box in the middle", whether that is a home router or a switch in a rack. That is a star, and it is a star for one reason — one broken cable affects one person.

Every other shape died on that single point, not on speed.

Four shapes, in two numbers

The four shapes side by side: bus uses one cable, ring uses as many as there are nodes, star the same but through a central device, and mesh uses n times n minus one over two

The first number is how many cables you run.

  nodes   full mesh   ring    star    bus
      5          10      5       5      1
     10          45     10      10      1
     50       1,225     50      50      1
    100       4,950    100     100      1
   1000     499,500  1,000   1,000      1

A full mesh needs n(n-1)/2 cables, which grows quadratically: 4,950 at a hundred nodes and nearly half a million at a thousand. That is why nobody builds a full mesh out of cable — not because it is a bad idea, but because you cannot pay for it.

The second number is resilience.

bus — one cable breaks and the whole segment dies.

ring — one break still leaves the other way round; two breaks are needed to partition it.

star — one break kills one node, but if the central device dies everything dies.

mesh — you must cut n-1 cables to detach a single node.

Bus lost on the first line. One shared cable means anyone who trips over it takes down the office, and finding the break is very hard because there is no central point.

The star won because it moves the fragility to one point you choose — a device in a rack, on a UPS, replaceable in ten minutes.

The cabling shape and the data shape

This is the part textbooks tend not to separate, and it is the most important.

Both pictures are cabled as a star, but with a hub in the middle data floods every port like a bus, while with a switch it goes only to the destination port

Cabling in a star says nothing about how data moves. What says that is the device in the middle.

  hub      takes in one port, sends out every port
           everyone hears everything, collisions across the whole segment
           cabled as a star, behaves as a bus

  switch   remembers which MAC sits on which port, sends there
           each port is its own collision domain
           cabled as a star, behaves as point-to-point links

Replacing a hub with a switch changes not one cable, yet it changes the logical topology from a bus into a set of separate links — and that did more for Ethernet's speed than raising the cable rate did.

The Ethernet article explains the CSMA/CD mechanism that exists because everyone shares one cable. Once a switch gives every port its own cable, that mechanism barely has to run at all.

When a star is not enough — and the loop that follows

A star has one obvious weakness: the middle. If that switch dies, everything attached to it dies.

The fix is more than one switch, connected to each other, which gives a star of stars. And when you add a spare cable so a failed device cannot cut the network in half, you get a loop automatically.

Two switches joined by two cables for resilience, forming a closed circuit in which broadcast frames circulate forever

A loop at layer two does not merely slow things down; it takes the whole segment out, because broadcast frames keep circulating with nothing to stop them — there is no TTL as there is in IP.

That is why STP exists, and once STP is running that spare cable is switched off, which is the origin of EtherChannel that lets you use both.

Those three articles line up because they are one problem — resilience requires spare paths, and spare paths create loops.

Mesh came back, but a different mesh

Mesh as cable-between-every-pair died on the cable count, but the idea returned in two places.

In data centres, in the shape called spine-leaf, which is not a full mesh: every lower switch connects to every upper switch, so all paths are the same length and there are several of them, using leaf x spine cables rather than n(n-1)/2.

In wireless, mesh is far cheaper because there is no cable at all. Adding a path means adding a point that can hear another, which changes the cost equation entirely.

And the internet itself is a partial mesh — nobody planned its shape. It grew from each party connecting to whoever they chose, which is why the switching article concludes that the winner is whatever needs the least agreement.

When it lies

"Our network is a star." True of the cabling, and it says nothing about the behaviour. You have to know what sits in the middle, and how many layers there are.

"Mesh is the most resilient." It genuinely is, and it is genuinely the most expensive: 4,950 cables at a hundred nodes. That figure is why nobody does it, not ignorance of its merits.

"Bus died because it was slow." Bus died because one cable break took everything down and the break was hard to find. Speed was secondary.

"Adding a spare cable makes it safer." Only if something manages the loop. Without STP, running a spare cable is manufacturing an outage.

"The diagram is what exists." Diagrams are usually drawn at installation and never updated. Believe what you can measure from the equipment, not a picture drawn three years ago.

Real cases from real work

Case 1 — finding the real topology without the documentation

Situation You inherit a network with no documentation, or documentation that does not match.

Command Ask the equipment instead of the documents.

$ arp -an                        who shares our segment
$ traceroute 8.8.8.8             what the way out passes through
$ netstat -rn | head             who the gateway is

How to read it The address table says who is on the same segment; the route says where the exit is. Those two give a rough shape faster than hunting for documents, and more importantly they describe the situation right now.

What this does not prove This sees only what will talk to us. It does not see cables left plugged in, switches without an IP address, or the standby links STP has disabled — which are exactly what come back to life during an incident.

Case 2 — deciding where to run a spare cable

Situation There is budget for one more cable and you must choose where.

How to read it Ask which point, if it fails, affects the most — which in a star is always the middle. A spare cable between a user's machine and the switch buys almost nothing, because that failure affects one person anyway. A spare between switch and switch changes the outcome a great deal.

What this does not prove Spare cables help only when cables are what break. If the real weakness is power or the device itself, a second cable changes nothing. You have to know what actually fails most often in that place.

Case 3 — reading a diagram in a quotation

Situation A vendor sends a design to review.

How to read it Look for two things.

  1  is there a point whose failure takes everything down
  2  are the spare paths real, and what chooses between them

The first you find by covering each box with a finger and seeing what remains. The second matters because a spare path with nothing managing it is a loop waiting to happen.

What this does not prove A diagram says how things are meant to be connected, not how they will be configured. Equipment cabled correctly and configured wrongly behaves exactly like equipment cabled wrongly.

What the four shapes still teach

Even though nobody cables a bus or a ring any more, the four shapes remain valuable as two questions to ask of any design — what will the cabling cost, and how much damage will it survive?

The answers change with the era. When cable was expensive and equipment cheap, bus won. When cable got cheap and finding faults got expensive, star won. And in a data centre where both are cheap, a structured mesh came back.

The shapes did not change. The prices did.

References

Computed here

The cable-count table comes from n(n-1)/2 for a full mesh and n for a ring and a star
The resilience table comes from counting how many cables must be cut to partition the network