Queensland Outage Postmortem: 24 April 2026

Summary

On Friday 24 April 2026 at 15:40 AEST, customer traffic in Queensland dropped completely.

The interruption lasted approximately 37 minutes, with service fully restored at 16:17 AEST.

We know outages like this are disruptive, and we take them seriously. This post explains what happened, why it happened, and what we've done to prevent it happening again.

What happened

Our monitoring systems detected the traffic drop immediately, and customer reports started coming in shortly after.

We began investigating from our side while coordinating with our aggregation partner's engineers. Together, we reviewed alarms, traffic graphs, and switch logs to pinpoint the issue.

This log entry on our core switch turned out to be the key:

%STP-2-BLOCK_PVID_LOCAL: Blocking port-channel46 on VLAN0129. Inconsistent local vlan.

In simple terms, one of our core links (port-channel46) was automatically blocked by Spanning Tree Protocol (STP).

Why it happened

STP is designed to protect the network from loops—situations where traffic can circulate endlessly and cause widespread disruption.

In this case:

• The link was receiving STP (BPDU) messages from the far end
• The VLAN information in those messages didn't match our local configuration
• STP treated this as a risk and blocked the link to keep the network stable

Because that link carried NBN aggregation traffic of customers from Queensland, blocking it caused the outage.

The most likely trigger was a downstream device sending unexpected STP traffic upstream—either a misconfigured switch or a temporary layer 2 loop.

What is STP ?

Spanning Tree Protocol is a safety mechanism built into Ethernet networks.

It ensures there's only one active path between devices by blocking redundant links. This prevents loops, which can otherwise bring down an entire network.

The downside is that when STP detects something inconsistent, it can act quickly and block a link—even if that link is carrying production traffic.

How we fixed it

We applied BPDU filtering to port-channel46.

This prevents external STP messages from influencing our core network, which is the correct behaviour for this type of interface.

Traffic was restored immediately after the change at 16:17 AEST.

What we're doing next

After restoring service, we reviewed all similar interfaces across our network.

We are:

• Auditing BPDU filtering across all NNIs and upstream links
• Standardising configuration to prevent external STP influence
• Reviewing additional safeguards to reduce the blast radius of similar events

Closing

This was a preventable issue, and we've addressed the gap that allowed it to happen.

If you were affected, we appreciate your patience while we worked through it.