Major incident
unico skill: Instabilidade na Integração das capacidades que utilizam IDScore, ID Unico (Verificação de Identidade), ID Trust (Alerta de Comportamento) e ID Live (Prova de Vida)
Jul 7, 2026, 21:22 UTC
started
22:14 UTC
resolved
53 min
duration
Current status of unico skill: Operational — live unico skill status
Timeline
- Jul 7, 2026, 21:22 UTCidentifiedPrezado Cliente, Identificamos uma latência seguida de indisponibilidade na criação de processos de algumas capacidades da Unico. Devido a isso, uma parcela de nossos clientes pode perceber lentidão ou aumento de erros em operações que envolvem o ID Score, ID Unico (Verificação de Identidade), ID Trust (Alerta de Comportamento) e ID Live (Prova de Vida). Nossa equipe de engenharia já foi acionada e está trabalhando no diagnóstico da causa raiz para normalizar o serviço o quanto antes. No momento, o time técnico realiza análises detalhadas em nossa camada de dados para mitigar os impactos. Comprometemo-nos a enviar novas atualizações em breve assim que tivermos novidades. Equipe Unico.
- Jul 7, 2026, 21:32 UTCmonitoringPrezado Cliente, Informamos que nossa equipe de tecnologia identificou e solucionou a instabilidade que afetou o tempo de resposta das requisições para retorno das capacidades de ID Score, ID Unico (Verificação de Identidade), ID Trust (Alerta de Comportamento) e ID Live (Prova de Vida). Dentro de alguns dias compartilharemos maiores detalhes através de um Postmortem. Pedimos desculpas pelo transtorno e nos colocamos à disposição para sanar dúvidas através dos nossos canais de atendimento. Atenciosamente, Equipe Unico!
- Jul 7, 2026, 22:14 UTCresolvedPrezado Cliente, Incidente resolvido. Nossa equipe identificou as causas do problema e realizou as ações para que este incidente fosse solucionado. Dentro de alguns dias compartilharemos maiores detalhes através de um Postmortem. Pedimos desculpas pelo transtorno e nos colocamos à disposição para sanar dúvidas através dos nossos canais de atendimento. Atenciosamente, Equipe Unico!
- Jul 13, 2026, 21:18 UTCpostmortem**Post-Mortem: Availability Instability in Identity Processing** **Summary** On two occasions on consecutive days in July 2026, the identity processing service experienced brief drops in availability. These were caused by saturation in a shared database that supports part of the verification flow. Both events were quickly identified by the monitoring teams, mitigated within approximately 30 minutes, and resolved with zero data loss. **Impact** Incident 1 — July 6, 2026, from 13:00 to 13:34 \(BRT\): A sudden and legitimate spike in request volume overloaded the database replica serving the service. Clients using services dependent on verification score generation experienced slowness, as requests were automatically retried instead of returning errors. Verification processes continued to complete successfully, albeit with delays. Incident 2 — July 7, 2026, from 17:52 to 18:05 \(BRT\): A database query lacking a row-return limit was triggered by an edge-case data pattern, saturating the CPU of the same database replica. This increased latency and, for a short window \(around 13 minutes\), caused errors in certain steps of the verification flow for a subset of clients. In both cases, the impact was limited to the windows described above, without total service downtime and with zero data loss. **Root Cause** Both incidents share the same underlying cause: CPU saturation on a relational database replica that serves multiple capabilities of the identity verification service. In the first incident, a specific request volume surged roughly 10x within a few minutes, exceeding the provisioned capacity for the replica. The database infrastructure was not scaled to sustain the peak, leading to processing exhaustion \(CPU scheduling contention\) and a subsequent cascade of timeouts in dependent services. In the second incident, a specific query lacked a limit on returned rows. When executed for a record with an unusual volume of associated images, the query triggered a full table scan and sort, consuming all available processing power on the replica. This effect was amplified by two internal mechanisms that, combined, tripled the number of queries generated per request. **Resolution** In both incidents, the operations teams: Vertically scaled a secondary database replica and redirected traffic to it via a DNS update; Temporarily disabled mechanisms that were generating additional database load; Monitored the stabilization of availability metrics before declaring the incident resolved; Following the second incident, applied a definitive fix to the problematic query \(enforcing a limit on returned records\) and initiated a data-cleansing process for non-standard records that could replicate the scenario. **Lessons Learned** These incidents reinforced the need for defensive and systematic infrastructure design, applying explicit limits and pagination on database queries to eliminate latent risks under atypical traffic. It became evident that component migrations can inadvertently strip away implicit protections against overload. This makes the implementation of declarative resilience mechanisms—such as proper timeouts, circuit breakers, and connection limits—indispensable. Additionally, the scenario highlighted the importance of investigating the root cause of recurring saturation patterns rather than relying solely on point-in-time mitigations, as well as the urgency of monitoring and refactoring architectures that multiply database calls per request to mitigate load amplification risks before they become critical. **Closing** We acknowledge the impact these instabilities caused and apologize for any inconvenience experienced by our clients and partners during these windows. Transparency regarding what happened is part of our commitment to those who trust our platform. The actions taken in response to these incidents prioritize reinforcing our environment's stability and reducing the likelihood of recurrence. We co
Updates as published by unico skill (original).
Other services with incidents at the same time
- External Clover 22:14 UTC · Telstra Outage in Australia
More unico skill incidents
- August 10, 2026 Instability affecting ID CLOUD capabilities
- July 21, 2026 Instability affecting ID CLOUD capabilities
- July 20, 2026 Instability affecting ID CLOUD capabilities
- July 16, 2026 Instability in Authentication Services for the UnicoPeople Portals and IDCloud messaging service
- July 13, 2026 Instability affecting ID CLOUD capabilities
- July 6, 2026 Instabilidade na Integração das capacidades que utilizam IDScore