It Worked for Years — Until It Didn't

Situation

A growing company. Infrastructure built incrementally over time through quick fixes, individual decisions, and urgent responses to immediate needs. Each change made sense at the time. Everything "worked" — until random outages and instability began to appear.

The failures were difficult to diagnose because they appeared random. No single change had caused them. No obvious fault existed. The infrastructure was simply reaching the point where accumulated decisions were interfering with each other in unpredictable ways.

The Reality

Nothing broke suddenly. Complexity had accumulated silently over years, reaching a threshold where small perturbations — traffic variations, device reboots, software updates — began triggering cascading failures that a clean infrastructure would have handled trivially.

• Configuration drift across devices and segments — devices that were supposed to be identically configured had diverged over years of individual changes
• Overlapping address ranges and segments from different growth phases — network addressing that had been designed for a 50-person company, then extended without architectural review as the organisation grew
• Legacy rules interfering with newer configurations — firewall and routing rules from previous network designs that had never been removed, creating interactions with current configuration
• No consistent architectural baseline underpinning the whole — each phase of growth had added to the network without revisiting the foundation

The problem was diagnostic as much as technical. Because no single device was obviously broken, and no single change had triggered the failures, standard fault isolation approaches were ineffective. The issue only became visible when the environment was mapped holistically — when every device, rule, and segment was reviewed in relation to every other.

The Challenge of Incremental Growth

Infrastructure built incrementally is not inherently problematic. The problem arises when growth happens without periodic architectural review. Each individual change may be entirely correct. But without review of how changes interact, divergence accumulates — and at some point, the accumulated divergence exceeds the infrastructure's ability to behave predictably.

This is not a failure of the people who made the individual decisions. It is a structural consequence of growth without governance — and it is extremely common in organisations that have expanded faster than their IT management practices.

What FM-NetSec Nordic Did

• Cleaned up the entire structure systematically — device by device, segment by segment, building a complete picture of the actual configuration state before making any changes
• Standardised configurations across the environment — eliminating the drift between devices that were supposed to be identically configured
• Removed legacy rules and conflicts that had accumulated undetected — identifying and eliminating rules that were interfering with current infrastructure without serving any current purpose
• Rebuilt critical paths on a clean, documented architectural foundation — ensuring that the core infrastructure had a defined, intentional design that could be understood and maintained

"Infrastructure doesn't fail overnight. It degrades over time."

The outages stopped. The infrastructure became predictable again. Internal teams could now understand what they were managing — because for the first time, the infrastructure had a coherent, documented foundation rather than years of accumulated decisions with no common thread.