Critical incident

iPaper: Flipbooks viewer is down

Affected: Viewer, Admin, Backend API, Pop-ups

Current status of iPaper: Operational — live iPaper status

Timeline

1. Aug 27, 2026, 06:01 UTC
   investigating
   We are currently unable to serve flipbooks and are investigating
2. Aug 27, 2026, 06:09 UTC
   monitoring
   We have identified the issue and applied a fix to our caching - flipbooks are now serving as normal again
3. Aug 27, 2026, 06:45 UTC
   identified
   We are working on another fix as we are still seeing slow page loads and occasional outages
4. Aug 27, 2026, 07:32 UTC
   monitoring
   Our caching servers were in a bad state due to networking issues causing them to be unable to replicate correctly. we have rolled completely new caching servers and this has resolved all issues we were seeing.
5. Aug 27, 2026, 08:16 UTC
   resolved
   The issue has been resolved and we have seen no more errors with the new caching.
6. Sep 4, 2026, 15:27 UTC
   postmortem
   ## Summary One of our nodes had a small milliseconds networking outage, which caused the IPsec connection used for our VXLAN between the node and another node to get out of sync. The node which lost connection was running one of our clustered caching instances, so the other instances took over with no issues. Once we manually fixed the node they were able to talk together again, which caused them to now try and sync up, but because they had been running for an extended time apart they failed to properly sync, and the application servers started to fail when using the caches. ## Impact Caching was unable to serve request fully and partially depending on which instance was used, which caused us to not load flipbooks viewer and admin. ## Cause The caching nodes were out of sync due to a networking issue on the internal VXLAN, which when fixed caused the instances to try and sync data and failed. ## Resolution We rolled entirely new caching instances, and updated all applications to use the new instances across all of our applications. We are working with our providers to implement a resilience fix to the core IPsec feature so this should not happen in the future. We have added extra monitoring as well to better catch this exact scenario. We will also be working on trying to understand why the sync failed, and we will be changing our clustering strategy to avoid the issue entirely.

Updates as published by iPaper (original).