Minor incident

RIPE NCC: RIS: RRC12 MRT update and bview files are not produced

Affected: RIS (Routing Information Service)

Current status of RIPE NCC: Minor issues — live RIPE NCC status

Timeline

1. Oct 2, 2026, 07:42 UTC
   investigating
   The latest available MRT update file for RRC12 has a timestamp of 01:00 UTC, October 2nd, 2026. The latest available BVIEW file for RRC12 has a timestamp of 00:00 UTC , October 2nd, 2026. We are investigating.
2. Oct 2, 2026, 11:26 UTC
   identified
   At 1.30UTC, on October 2nd, 2026, a DE-CIX peer started sending an unusually high number of BGP messages to RRC12. Our processing pipeline consuming from kafka did not keep up with this volume of updates, causing a process to time out. When this process restarts, it times out again before making enough progress. We are now observing an impact on the production of other RRCs updates files, that are being delayed. We are troubleshooting this issue.
3. Oct 5, 2026, 12:46 UTC
   identified
   We continue to work on this issue. We have identified the root cause of the pipeline stall and are developing a workaround.
4. Oct 6, 2026, 06:44 UTC
   resolved
   This incident has been resolved. After the initial process recovered, the bview and update files since 2 October were produced successfully. The incident trigger was a high volume of updates from one peer. This caused load in a single kafka partition. When that process restarted, it hit a latent bug where it tried to re-scan a large volume of data on startup. This scan repeatedly timed out, which prevented the process from making progress. Additionally, due to a misconfiguration in the code, this scan took an order of magnitude longer than it should. After fixing this configuration issue the process recovered. We will work on a structural fix for the latent bug in the next sprint.

Updates as published by RIPE NCC (original).