Major incident

Sauce Labs: 2026-September-28 Service Incident

Affected: US-West

Current status of Sauce Labs: Operational — live Sauce Labs status

Timeline

1. Sep 28, 2026, 16:04 UTC
   investigating
   We are currently seeing elevated error rates for automated real device tests in US-West-1 data center. We are currently investigating.
2. Sep 28, 2026, 16:56 UTC
   monitoring
   The root cause of the elevated error rates for Real Device tests in the US-West-1 data center has been identified. Remediation has been applied, and we are actively monitoring the situation.
3. Sep 28, 2026, 17:19 UTC
   resolved
   After taking remedial action, Real Device test error rates have returned to normal in the US-West-1 data center. All services are fully operational.
4. Oct 2, 2026, 19:15 UTC
   postmortem
   ### **Dates:** Monday, September 28th 2026, 14:19 UTC - 16:52 UTC. ### **What happened:** Real device tests in US West hung and failed to output test artifacts due to slow uploads. ### **Why it happened:** A degraded interconnect link suffered high packet loss from optical errors, but BGP stayed up, preventing automatic failover to healthy paths. ### **How we fixed it:** Manually disabled the degraded port on the edge router, forcing traffic onto healthy alternate interconnects. ### **What we are doing to prevent it from happening again:** Automating link draining based on CRC error thresholds, improving protocols for isolating flapping vendor links, and hardening upload resilience.

Updates as published by Sauce Labs (original).