Read Outage - prod-us-central-0
Oct 14, 18:12 UTCResolved - This incident has been resolved.Oct 14, 15:14 UTCMonitoring - A fix has been implemented and we are monitoring the results.Oct 14, 14:46 UTCInvestigating - We are...
View ArticleInvestigating issue with some instances becoming unreachable with "Grafana...
Oct 16, 09:59 UTCResolved - This incident has been resolved.Oct 16, 09:46 UTCMonitoring - A fix has been implemented and we are monitoring the results. This issue only impacts users on the "fast"...
View ArticleLoki - Issues Writing Logs in AWS Singapore
Oct 16, 21:11 UTCResolved - We have continued to observe normal Loki write actions and behavior to the AWS Singapore cluster. As such, we are considering this incident resolved.No further updates.Oct...
View ArticleScheduled Maintenance
Oct 17, 19:00 UTCCompleted - The scheduled maintenance has been completed.Oct 17, 18:00 UTCIn progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Oct 16,...
View ArticleElevated Latency - prod-eu-west-0 and prod-us-central-0
Oct 18, 14:08 UTCResolved - We have observed a sustained period of normal behavior with no reoccurrences of latency. As such, we are considering this incident resolved, at this time. No further...
View ArticleScheduled Maintenance
Oct 18, 21:01 UTCCompleted - The scheduled maintenance has been completed.Oct 18, 20:01 UTCIn progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Oct 17,...
View ArticleScheduled Maintenance
Oct 19, 21:00 UTCCompleted - The scheduled maintenance has been completed.Oct 19, 20:00 UTCIn progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.Oct 18,...
View Articleexperiencing degraded write path performance http://logs-prod-013.grafana.net/
Oct 20, 18:02 UTCResolved - This incident has been resolved.Oct 20, 01:13 UTCUpdate - We are experiencing degraded performance for a small number of customers in our Azure Netherlands - prod-eu-west-3...
View ArticleDegraded write performance for Loki on region prod-ap-southeast-0 (GCP...
Oct 22, 08:29 UTCResolved - Our engineering team has discovered the cause of the degradation and applied measurements to restore the service. This incident has been resolved.Oct 22, 07:59...
View ArticleLoki query range error in the parallel read path on prod-eu-west-2 (AWS Germany)
Oct 23, 08:00 UTCResolved - There was an issue with the parallel read path on the Loki module on the cluster prod-eu-west-2.The issue was observed from 10:00 to 10:30 UTC time. This incident has been...
View ArticleAggregated Metrics not being generated on prod-us-central-0
Oct 23, 14:02 UTCResolved - Engineering has released a fix on the production cluster as of 13:40 UTC and aggregated metrics are being successfully generated again.This issue has been resolved.Oct 23,...
View ArticleMetrics Disruption - cortex-prod-10 cluster
Oct 25, 07:51 UTCResolved - Engineering has released a fix and as of 07:50 UTC, customers should no longer experience metric disruption in the cortex-prod-10 cluster. At this time, we are considering...
View ArticleMimir Request Errors
Oct 25, 14:05 UTCResolved - This incident has been resolved.Oct 25, 13:37 UTCInvestigating - We are currently seeing higher latency and errors on read paths. Our team is currently investigating this...
View ArticleLogs disruption - us-central2
Oct 26, 03:05 UTCResolved - Reads and writes in the us-central2.loki-prod region were affected for a short duration between 2:33 to 2:46 UTC. Engineering has released a fix and customers should no...
View ArticleLoki-prod Write and Read Issues in US-Central2 Region
Oct 26, 11:21 UTCResolved - This incident has been resolved.Oct 26, 10:26 UTCInvestigating - We are currently encountering write path challenges with loki-prod in the us-central2 region, accompanied by...
View ArticleLoki Read Outage
Oct 28, 01:34 UTCResolved - This incident has been resolved.Oct 27, 21:48 UTCMonitoring - From approximately 21:17 to 21:37 UTC we observed degraded performance. We are continuing to monitor the...
View ArticleGrafana Notifications - Email
Oct 30, 12:54 UTCResolved - This incident has been resolved.Oct 27, 23:23 UTCUpdate - We are continuing to monitor for any issues.NOTE: For alertmanager email contact points, specifying a smtp server...
View ArticleInvestigating issue writing Metrics Generator Metrics to Grafana Cloud Metrics
Nov 1, 15:02 UTCResolved - This incident has been resolved.Nov 1, 13:31 UTCInvestigating - We are currently investigating this issue.
View ArticleCertain Advanced Features Missing
Nov 6, 22:07 UTCResolved - We have successfully implemented updates to address the previously missing Cloud features, which are now fully operational. Our team is actively monitoring the service for...
View ArticleCertain Advanced Features Missing
Nov 7, 03:54 UTCResolved - This incident has been resolved.Nov 7, 01:56 UTCUpdate - We have successfully deployed the latest updates to 90% of the affected instances. Our team is closely monitoring the...
View Article
