We have seen a significant improvement in the reliability of calls to Microsoft 365. This has now been close to, or within, the bounds of usual behavior for around an hour.
This aligns with Microsoft's communication against incident MO941162. Microsoft are not saying it is fully resolved but that they have mitigations in place, and are continuing work to fully resolve the problem.
As the impact of this has been minimal for a significant period, we are resolving this incident. We will keep a closer eye on this area until Microsoft call their incident resolved as well.
Posted Nov 25, 2024 - 22:27 UTC
Update
We have seen some improvement in the reliability of calls to Microsoft 365, but are far from normal service. We remain past the busiest point of the day for Cronofy's services which means we're in a more stable position overall.
We are continuing to monitor our systems and Microsoft incident MO941162. Microsoft are continuing to work on the underlying issue and sync tasks continue to retry automatically.
We will provide a further update by 22:30 UTC.
Posted Nov 25, 2024 - 21:27 UTC
Update
We have seen little improvement of reliability of calls to Microsoft 365. However, we are now well past the busiest point of the day for Cronofy's services which means we're in a more stable position overall.
We are continuing to monitor our systems and Microsoft incident MO941162. Microsoft are continuing to work on the underlying issue and sync tasks continue to retry automatically.
We will provide a further update by 21:30 UTC.
Posted Nov 25, 2024 - 20:26 UTC
Update
We have seen little further improvement of reliability of calls to Microsoft 365.
We are continuing to monitor our systems and Microsoft incident MO941162. Sync tasks continue to retry automatically.
We will provide a further update by 20:30 UTC.
Posted Nov 25, 2024 - 19:29 UTC
Monitoring
Many calls to Microsoft 365 are still failing. We have seen some improvement over the past hour but are still far from a full recovery.
Synchronization of affected calendars will be degraded with failures retried automatically as required.
Microsoft are reporting on this as incident MO941162, with fixes in progress but currently no timeline to resolution.
We are continuing to monitor and mitigate where we can, and will provide another update by 19:30 UTC.
Posted Nov 25, 2024 - 18:27 UTC
Update
Many Graph API calls to Microsoft 365 are still timing out and we have yet to see any improvement.
We are continuing to monitor the situation and mitigate any impact on operations.
We will update again at 18:30 UTC.
Posted Nov 25, 2024 - 17:31 UTC
Identified
We have been tracking a degradation in MS Graph connections since approximately 07:30 UTC today.
While most operations have been succeeding in short order, this has worsened to the point where sync is starting to be noticeably degraded for some accounts.
Microsoft are reporting on this as incident "MO941162".
We are continuing to monitor and mitigate where we can, and will provide another update at 17:30 UTC.
Posted Nov 25, 2024 - 16:27 UTC
This incident affected: Major Calendar Providers (Microsoft 365).