Resolved
The issue has been resolved and the API performance is back to normal.
Monitoring
The API error rate is returning to normal. We will continue to monitor the system.
Investigating
The latency issue with our cloud provider is still ongoing and we are waiting for an update from them on the root cause and time to remediate. The impact to gigs is a small amount of global traffic gets 5xx responses; it's less than 0.08% of requests failing due to timeouts and >50% of requests have no latency impact
Investigating
A small number of requests are currently impacted by high latency, and 500 errors at times. This appears to be caused by a networking issue on our cloud provider. We are in contact with them and attempting to resolve the issue as soon as possible.
Monitoring
The error rate seems to have been caused by a temporary issue with network latency in our cloud provider which is now resolved.
Investigating
We have observed an increase in API 5xx HTTP errors. We are investigating.