Packet loss on IoT Network Handler #2
Loss peaked around 6% for 54 minutes on Handler #2 only. Control plane, Hub, and other IoT handlers remained operational.
- Affected
- IoT Network Handler #2
- Started
- Oct 11, 2026, 12:42 AM UTC
- Resolved
- Oct 11, 2026, 1:36 AM UTC
- Duration
- 54 min
Response time
Peak 88 ms
{"labels":["0m","1m","2m","3m","5m","6m","7m","8m","9m","10m","11m","13m","14m","15m","16m","17m","18m","20m","21m","22m","23m","24m","25m","26m","28m","29m","30m","31m","32m","33m","34m","36m","37m","38m","39m","40m","41m","43m","44m","45m","46m","47m","48m","49m","51m","52m","53m","54m"],"series":[{"name":"Response time","color":"#ff4ecd","values":[29.1,30.3,29.2,28.3,28.2,27.7,28.6,28.1,29.7,29.5,29.1,29.2,29,39.2,49.2,58.5,66.9,74.2,80.1,84.4,87.1,88,87.1,84.4,80.1,74.2,66.9,58.5,49.2,39.2,29,29.7,29.8,28.4,28.3,28.6,30.3,29.4,27.8,28.5,29.6,27.7,28.7,28.3,30,28.6,27.7,28.5],"unit":"ms"}]}
Error rate
Peak 6.4 %
{"labels":["0m","1m","2m","3m","5m","6m","7m","8m","9m","10m","11m","13m","14m","15m","16m","17m","18m","20m","21m","22m","23m","24m","25m","26m","28m","29m","30m","31m","32m","33m","34m","36m","37m","38m","39m","40m","41m","43m","44m","45m","46m","47m","48m","49m","51m","52m","53m","54m"],"series":[{"name":"Error rate","color":"#3dd6f5","values":[0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,1.5,2.5,3.5,4.3,5,5.6,6,6.3,6.4,6.3,6,5.6,5,4.3,3.5,2.5,1.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5],"unit":"%"}]}
Forwarded capacity
Peak 25 Gbps
{"labels":["0m","1m","2m","3m","5m","6m","7m","8m","9m","10m","11m","13m","14m","15m","16m","17m","18m","20m","21m","22m","23m","24m","25m","26m","28m","29m","30m","31m","32m","33m","34m","36m","37m","38m","39m","40m","41m","43m","44m","45m","46m","47m","48m","49m","51m","52m","53m","54m"],"series":[{"name":"Forwarded capacity","color":"#8b6cff","values":[23.2,25,24.4,23.8,24.5,24.2,24.1,23.9,23.8,23.1,23.8,24.4,24,21.7,19.6,17.5,15.6,14,12.7,11.8,11.2,11,11.2,11.8,12.7,14,15.6,17.5,19.6,21.7,24,24.7,23.2,24.6,24.6,24.1,23.7,22.8,24.7,24.2,23.1,23.5,25.1,25.2,24.1,24.7,24.5,24.5],"unit":"Gbps"}]}
Timeline
-
Investigating
Mesh health checks reported rising drop counters on Handler #2.
-
Identified
One coordinator came back with a stale peer table after the rolling restart.
-
Mitigating
Coordinator rebuilt its peer list. Traffic slowly returned to the node.
-
Resolved
Loss back under 0.5%. Restart playbook now waits for peer-table warmup.