Major incident

unico skill: Instability in Authentication Services for the UnicoPeople Portals and IDCloud messaging service

Affected: Notificações, Módulo Candidato

Current status of unico skill: Operational — live unico skill status

Timeline

1. Jul 16, 2026, 14:02 UTC
   investigating
   Dear Customer, We are investigating an instability in our authentication system for the Unico People portals and are also assessing issues in our messaging service affecting IDCloud customers. Our engineering team is mobilized to resolve the issue as soon as possible. We will provide updates shortly.
2. Jul 16, 2026, 14:08 UTC
   monitoring
   Dear Customer, Our team identified the issue and took action to resolve the incident. We experienced an instability in our authentication system for the Unico People portals, as well as issues in our messaging service affecting customers in the IDCloud flow, for SMS and email delivery. Our engineering team remains mobilized, monitoring the environment to ensure it continues functioning properly.
3. Jul 16, 2026, 15:28 UTC
   resolved
   Executive Summary and Impact At 10:51 AM on 07/16/2026, an issue was identified in the authentication service on the Unico People admission portal and in messaging, which affected the delivery of SMS and email links to Unico People and IDCloud clients, as well as B2C client authentication on the Unico People admission portal. The impact was limited to these two flows and did not affect the product's core critical journey. The incident lasted approximately 30 minutes until full resolution. Root Cause and Resolution The root cause was identified as storage pressure on one node of the messaging cluster, which reached a critical disk threshold, degrading the processing of queues responsible for message delivery and authentication. The engineering team increased the cluster's storage capacity at approximately 10:59 AM, which allowed queues and monitoring indicators to normalize within minutes. The environment was monitored until full recovery was confirmed, and the incident was closed at 11:20 AM. Commitment and Next Steps The engineering team will review alert escalation and prioritization criteria for this type of degradation, since the alerts correctly identified the issue, but the automatic escalation flow to the responsible team can be improved. A detailed postmortem is being prepared and will be shared in the coming days.
4. Aug 3, 2026, 12:14 UTC
   postmortem
   **Summary** On July 16, 2026, between 10:12 AM and 11:10 AM \(Brasília time\), the messaging and authentication service was degraded for approximately 58 minutes. The cause was disk saturation on one of the nodes in the message queue cluster, which, upon reaching the configured limit, automatically blocked all publishers, causing a backlog of up to 33 thousand messages in the queue. The impact affected the delivery of authentication links via SMS and email, as well as the end-user authentication flow on the Unico People admission portal. **Impact** Users waiting for authentication links via SMS or email did not receive them during the period. End users of a people management product attempting to authenticate on the admission portal were also unable to access the service. The impact was contained after the cluster's disk was expanded. **Root Cause** The root cause was identified in the message processing flow: excessively large payloads — generated by an auditing component that included unnecessary data — exceeded the size limit accepted by the message queue cluster. Since they were deterministically rejected on every attempt, these messages were routed to dead-letter queues after exhausting all retries. These queues had no expiration or automatic cleanup policy, causing messages to accumulate indefinitely and progressively consume the cluster's disk space. **Resolution** The cluster's disk was expanded to 500 GB per node, which immediately normalized the queues and cluster metrics. The service returned to normal at 11:10 AM. The structural root cause — the indefinite accumulation in the dead-letter queues — remains as a follow-up item to be addressed. **Lessons Learned** The incident reinforces that addressing only the symptom \(disk saturation\) without investigating the root cause of the accumulation guarantees recurrence of the problem. The absence of a lifecycle policy \(TTL/expiration\) for dead-letter queues and the lack of payload size validation prior to publishing are the two central structural gaps to be corrected. Additionally, the alert that flagged the issue was classified as low priority, which caused a delay of approximately 38 minutes between the alert being triggered and the formal opening of the incident — an unacceptable interval for a critical service degradation.

Updates as published by unico skill (original).