Core, Viewer, Plan & Reserve outage

Incident Report for TimeEdit Status Page

Postmortem

Description

Our system experienced high volume of traffic, which exceeded our current rate-limiting capabilities. This led to intermittent service disruptions for a subset of our users, primarily impacting those in Europe.

What happened

Between 17:05 CET and 17:46 CET on Sunday, 19th January, our monitoring systems detected a significant increase in incoming traffic to our services. While our infrastructure is designed to handle high levels of activity, this spike exceeded the thresholds configured in our systems, resulting in delays and temporary unavailability for some users.

What we’re doing to prevent this in the future

To reduce the risk of similar disruptions, we are implementing several measures, including:

  • Strengthening our collaboration processes with customer integrations to better anticipate and plan for traffic changes.
  • Improving traffic management systems to dynamically adjust to unexpected load patterns.
  • Conducting a full review of our monitoring and alerting systems to ensure faster detection and response in the future.
Posted Jan 20, 2025 - 15:05 CET

Resolved

This incident has been resolved and all products can be accessed again. We will continue to monitor the products a bit extra to ensure stability and will provide a detailed postmortem as soon as possible.
Posted Jan 19, 2025 - 17:46 CET

Investigating

We are currently investigating an issue affecting following products: Core, Viewer, Plan & Reserve. Users may experience difficulty accessing or using these products at the moment.
Posted Jan 19, 2025 - 17:05 CET
This incident affected: Europe (Curriculum), Australia & New Zealand (Curriculum, Scheduling), and Singapore (Workload).