Major incident

Mews: Downgraded Performance

Sep 9, 2026, 19:30 UTC
started
20:54 UTC
resolved
1 h 24 min
duration

Current status of Mews: Operational — live Mews status

Timeline

  1. Sep 9, 2026, 19:30 UTC
    investigating
    We have noticed a downgraded performance of the system. We are currently looking into it. Thank you for your patience.
  2. Sep 9, 2026, 19:38 UTC
    monitoring
    We're investigating the issue.
  3. Sep 9, 2026, 19:42 UTC
    monitoring
    We've been able to resolve the cause of the incident. The app had around 10-second response time or was unresponsive for the end users. The system is functional, we are working on containing the root cause. More updates soon.
  4. Sep 9, 2026, 20:19 UTC
    monitoring
    We have narrowed the problem to be related to a maintenance event performed by our cloud provider Microsoft Azure on our production database. We are cooperating with them to address the root cause.
  5. Sep 9, 2026, 20:54 UTC
    resolved
    We've been able to identify the cause the performance issues out to be an unplanned migration work on the side of our cloud provider, Microsoft Azure.
  6. Oct 7, 2026, 14:50 UTC
    postmortem
    **Problem** On 9 September between 21:07 and 21:25 CEST, the Mews platform experienced a performance degradation lasting about 18 minutes. About half of requests to the application and API timed out, including sign-in. Customers saw slow or unresponsive pages and intermittent errors. Some operations kept working, and no data was lost. **Action** Our automated monitoring detected the issue within seven minutes, and our support team was engaged within minutes to help affected customers. We kept customers informed through our status page and monitored until the service fully recovered at 21:25 CEST. We opened a case with our cloud provider the same evening. **Causes** A routine access change to our production database caused the incident. Because of how our tooling re-applied existing permissions, one small change became a large batch of permission updates. That batch caused the database replica that serves read traffic to stall. Our cloud provider has confirmed that the replica should have recovered far sooner than it did, and is working on improvements to how it recovers from this situation. The replica recovered on its own after about 18 minutes, and operations that write data were largely unaffected. **Solutions** * We are changing our database tooling to apply only the specific change being made, rather than re-applying every existing permission. * We are adding timing, announcement and approval controls for production database changes such as this one. * We are making infrastructure changes visible to our incident tooling, so similar changes are identified immediately. * We have added alerting on this type of database change, and are improving alerting on database connection errors. * We are working with our cloud provider on faster recovery from this replica behaviour, and adding a fallback so read traffic can continue if the replica is unavailable.

Updates as published by Mews (original).

Other services with incidents at the same time

This was part of a wider event: 12 services reported incidents within the hour.

More Mews incidents