The hoover and the router

A network goes down. Redundant uplinks, documented failover, the whole nine yards. The cleaner needed a socket.


The pager went off at half seven in the morning. Three times in a row, which meant something upstream had stopped responding entirely. I was on the Tube and I spent the next hour convinced it would be something interesting — a BGP hijack, a fibre cut, one of those moments where you get to use the expensive monitoring software you spent weeks configuring.

It wasn't. The router was unplugged.

Not failed. Not blown a fuse. Physically unplugged from the wall socket, with the power cable coiled neatly on top, as though someone had tidied it away.

I'd been in the role about six months. The office was in central London, early 2000s, back when "sysadmin" meant you were personally responsible for everything that had electricity running through it. I'd spent that time building what I thought was a proper system. Redundant uplinks, documented failover paths, cable management that bordered on obsessive. I'd even designed a monitoring system that would catch problems before they became outages. If the router died, we'd fail over to the backup. If a cable got cut, the system would reroute. Everything was supposed to be resilient by design.

Except. The comms room had a single twin socket. The router was plugged into it. The other plug powered an extension lead that had been tucked behind the rack sometime around 1997 — no one really remembered. That extension lead snaked under the raised floor and surfaced in a cupboard on the other side of a partition wall, where the cleaning staff kept the hoover.

The cleaner, a woman named Rosa who had been in that building longer than most of the staff, had arrived before anyone else. The hoover wasn't working. She traced the cable, found it went to the other side of a wall, spotted the router's power cable, and made what she would later explain to me was entirely reasonable: the hoover needed to work. The router was just sitting there. She'd plug it back in when she finished hoovering.

She didn't. She went home.

What I learned, standing in that comms room processing the absurdity, is that systems don't fail. System boundaries fail. Every threat model you write has a boundary: the edge where the thing you designed stops and the world you didn't design starts. I'd modelled hardware failure, network degradation, load spikes — all real problems that existed inside my responsibility line. But the boundary between "network infrastructure" and "cleaning supplies" was never in any document I'd written. Rosa had her own system, her own goals, her own thread of causality. It intersected with mine at a power socket.

That's the thing about fuzzing or security testing that still holds true for everything else: the system fails where you forgot to draw a line. Not where you drew it wrong — where you didn't draw it at all. The simplest input, the one you never thought to test, the person who needs a socket because their tool isn't working. That's how things break.

I plugged the router back in. It came up in about forty-five seconds. The whole incident cost two hours, and half of that was me on the Tube. I found a second socket in that comms room and installed a proper power distribution unit, with labeled cables and lockable sockets. I added a line to the emergency procedures: Comms room power — check extension leads. I even put up a label in Italian and English: DO NOT UNPLUG — NETWORK.

I never found out if Rosa ever saw that label. But I did learn something that has stuck with me through every system I've designed since: the most important thing you can do is not design harder. It's define your boundary more carefully. Where does your control end? Where does someone else's reasonable behaviour begin? That's the gap you need to think about.