Nov 14, 14:45 GMT
The underlying AWS incident has been marked resolved and we have not seen any problems since then.
Nov 12, 14:30 GMT
Service seems stable but AWS are still reporting issues (see https://status.aws.amazon.com/
Until AWS close their incidents we will monitor in case of regression.
Nov 12, 11:51 GMT
There seems to have been a general issue with provisioning additonal servers within AWS Frankfurt (eu-central-1) as several mechanisms we tried were failing. This could be related to the network errors reported by AWS but is not confirmed by their status updates.
These appear to be working now, at least partially, and sufficient capacity has been added to resume normal service and also catch up with the backlog of work.
We are continuing to monitor and have changed configuration to ensure we are overprovisioned whilst we understand what is happening.
Nov 12, 09:59 GMT
Additional servers failed to be added as load increased, this may be related to some network issues reported by Amazon in the eu-central-1 region but we are still investigating.
We're taking steps to restore normal service whilst continuing investigation.
Nov 12, 09:25 GMT
We are currently investigating this issue.
Nov 12, 08:32 GMT