Major incident

balena.io: Elevated API Errors

Affected: API

Current status of balena.io: Operational — live balena.io status

Timeline

1. Jul 6, 2026, 13:47 UTC
   identified
   We're experiencing an elevated level of API errors and are currently looking into the issue.
2. Jul 6, 2026, 13:53 UTC
   monitoring
   A fix has been implemented and we are monitoring the results.
3. Jul 6, 2026, 14:37 UTC
   resolved
   This incident has been resolved.
4. Jul 9, 2026, 21:08 UTC
   postmortem
   A routine version upgrade to a backend cache changed how the API handled its request queues. Under unusually heavy request load, those queues filled up and the API started dropping traffic, returning timeouts and errors. Over the course of the incident, affected users saw elevated API error rates that intermittently disrupted dashboard logins, API access, and device connectivity. We mitigated it by rate-limiting the abnormal traffic at our edge and rolling the cache back to the previous version, which restored normal queue behaviour and recovered the platform. To avoid a repeat, we're improving rate limiting so a single heavy consumer can't degrade service for others, and exploring autoscaling and load testing so infrastructure changes are validated under realistic load before they reach production.

Updates as published by balena.io (original).