Get email notifications whenever Netdata creates, updates or resolves an incident.
Get text message notifications whenever Netdata creates or resolves an incident.
Get incident updates and maintenance status messages in Slack.
Subscribe via Slack
By subscribing you acknowledge our Privacy Policy. In addition, you agree to the Atlassian Cloud Terms of Service and acknowledge Atlassian's Privacy Policy.
Get webhook notifications whenever Netdata creates an incident, updates an incident, resolves an incident or changes a component status.
Visit our support site.
All Systems Operational
Uptime over the past 90 days. View historical uptime.
Cloud Web UI
Operational
90 days ago
100.0
% uptime
Today
Agent-Cloud Link (ACLK)
Operational
90 days ago
99.23
% uptime
Today
Agent Services
Operational
90 days ago
100.0
% uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Related
No incidents or maintenance related to this downtime.
Past Incidents
Jun 30, 2022
Resolved -
Reverting the default away from MQTT5 removed the immediate issue, and most Agents on the nightlies are now on the latest (v1.35.0-104-nightly).
In the mean time we've also found the true cause: the Agent was not properly processing incoming commands in the MQTT5 implementation, due to a bug in how the parser interacted with the buffer of incoming data. This has been resolved in the upcoming nightly build of the Agent. As we want to do some more testing, for now the Agent will keep using the older MQTT library by default.
Jun 30, 17:35 UTC
In the mean time we've also found the true cause: the Agent was not properly processing incoming commands in the MQTT5 implementation, due to a bug in how the parser interacted with the buffer of incoming data. This has been resolved in the upcoming nightly build of the Agent. As we want to do some more testing, for now the Agent will keep using the older MQTT library by default.
Jun 30, 17:35 UTC
Update -
For completeness, the affected versions are v1.35.0-84-nightly and v1.35.0-96-nightly. Latest, corrected version is v1.35.0-104-nightly.
Jun 30, 08:03 UTC
Jun 30, 08:03 UTC
Monitoring -
The new nightly version of the Netdata Agent has been published and installed by a large portion of the agents that auto-update. We are monitoring the results.
Jun 30, 07:20 UTC
Jun 30, 07:20 UTC
Identified -
We have identified part of the cause of failing responses for alarm values. In yesterday's nightly build of the Agent, we enabled the use of the newer MQTT5 library by default. We will create another build to revert that. In the meanwhile, you can explicitly disable this library using the mqtt5 setting in your configuration as described here: https://github.com/netdata/cloud-backend/issues/178.
Additionally the other latencies appear to be another instance of a known issue that causes responses with a small payload to be delayed. We are working on resolving this issue.
Jun 29, 14:11 UTC
Additionally the other latencies appear to be another instance of a known issue that causes responses with a small payload to be delayed. We are working on resolving this issue.
Jun 29, 14:11 UTC
Investigating -
Users with nightly versions of the Netdata Agent are experiencing slow responses between Cloud and Agent, resulting in failing or slow charts in their Cloud dashboards. We are investigating the issue.
Jun 29, 08:54 UTC
Jun 29, 08:54 UTC
Jun 29, 2022
Jun 28, 2022
No incidents reported.
Jun 27, 2022
No incidents reported.
Jun 26, 2022
No incidents reported.
Jun 25, 2022
Resolved -
We have implemented a change that restores all charts. Unfortunately there remains a bug that causes the top gauges to be missing from the single node tabs, in certain situations. We will fix this in the coming week. Updates on this, including a workaround, by using the overview tab with node filtering, can be found here: https://github.com/netdata/netdata-cloud/issues/484#issuecomment-1166306503
Jun 25, 15:45 UTC
Jun 25, 15:45 UTC
Identified -
The issue has been identified and a fix is being implemented.
Jun 25, 12:43 UTC
Jun 25, 12:43 UTC
Investigating -
We are investigating an issue that causes some charts to missing from the single node view and overview tabs.
Jun 25, 11:11 UTC
Jun 25, 11:11 UTC
Jun 24, 2022
No incidents reported.
Jun 23, 2022
No incidents reported.
Jun 22, 2022
Completed -
The scheduled maintenance has been completed.
Jun 22, 08:00 UTC
Jun 22, 08:00 UTC
Update -
Backlogs have been consumed, and the service is performing as expected. We will continue to monitor Cloud over (our) night and leave the reachability notification disabled until the end of the scheduled maintenance window.
Jun 21, 21:20 UTC
Jun 21, 21:20 UTC
Update -
We have identified one source of processing delays and are implementing a fix.
Additionally, if you cannot yet see the Cloud UI in your browser, please restart your browser. We found that browsers based on Chrome may be caching DNS responses beyond the configured TTLs.
Jun 21, 19:04 UTC
Additionally, if you cannot yet see the Cloud UI in your browser, please restart your browser. We found that browsers based on Chrome may be caching DNS responses beyond the configured TTLs.
Jun 21, 19:04 UTC
Verifying -
We have switched DNS and are seeing claimed Agents reconnecting. While Cloud is taking in those Agents, there are some processing delays, and information shown in Cloud dashboards might not be accurate. We will continue to update you on progress while we are verifying the proper working of Cloud.
Jun 21, 18:16 UTC
Jun 21, 18:16 UTC
Update -
We have migrated all of Netdata Cloud to its new location as planned. We are running tests to verify that the services are behaving as expected, and scale the services to take in the load of all claimed nodes once we switch over DNS. We expect to switch DNS around 17:30 UTC and will provide an update when that happens.
Jun 21, 16:32 UTC
Jun 21, 16:32 UTC
In progress -
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jun 21, 08:00 UTC
Jun 21, 08:00 UTC
Scheduled -
This maintenance activity involves the migration of the Netdata Cloud services to another cloud provider. We have scheduled a maintenance window of one day (24 hours). Although we anticipate that Netdata Cloud will be available within 8 hours, we take that time to ensure services are working as expected, all agents have successfully reconnected, and the data backlog has been processed.
During this time, Netdata Cloud will initially be completely unavailable. After the migration and the initial 8 hour window, you may experience degradation of the service. We will keep you up-to-date throughout the maintenance work on this status page. You can subscribe to notifications about this maintenance by clicking the “Subscribe” button, or by following our Twitter account (@linuxnetdata).
While Netdata Cloud is unavailable, you won’t able to:
· Access the UI
· Get alert or health notifications from Netdata Cloud (Agent notifications aren’t impacted)
· Get your Nodes connected to Netdata Cloud
We advise you during this maintenance window to rely on your Agent local dashboards and alerting capabilities to be able to continue your monitoring and troubleshooting activities.
For those who need to change their firewall rules, these are public IPs we are going to use from now on:
- 54.198.178.11
- 44.207.131.212
- 44.196.50.41
Jun 9, 16:38 UTC
During this time, Netdata Cloud will initially be completely unavailable. After the migration and the initial 8 hour window, you may experience degradation of the service. We will keep you up-to-date throughout the maintenance work on this status page. You can subscribe to notifications about this maintenance by clicking the “Subscribe” button, or by following our Twitter account (@linuxnetdata).
While Netdata Cloud is unavailable, you won’t able to:
· Access the UI
· Get alert or health notifications from Netdata Cloud (Agent notifications aren’t impacted)
· Get your Nodes connected to Netdata Cloud
We advise you during this maintenance window to rely on your Agent local dashboards and alerting capabilities to be able to continue your monitoring and troubleshooting activities.
For those who need to change their firewall rules, these are public IPs we are going to use from now on:
- 54.198.178.11
- 44.207.131.212
- 44.196.50.41
Jun 9, 16:38 UTC
Jun 21, 2022
Jun 20, 2022
No incidents reported.
Jun 19, 2022
No incidents reported.
Jun 18, 2022
No incidents reported.
Jun 17, 2022
No incidents reported.
Jun 16, 2022
Resolved -
This incident has been resolved.
Jun 16, 14:42 UTC
Jun 16, 14:42 UTC
Monitoring -
A fix has been implemented and we are monitoring the results.
Jun 16, 13:28 UTC
Jun 16, 13:28 UTC
Update -
We are continuing to work on a fix for this issue.
Jun 16, 12:34 UTC
Jun 16, 12:34 UTC
Identified -
The issue has been identified and a fix is being implemented.
Jun 16, 12:32 UTC
Jun 16, 12:32 UTC
Investigating -
Charts metadata is not refreshed instantly, users might see old or not updated charts in their UI. Datapoints in displayed charts are up-to-date since it is streamed directly from the node.
Jun 16, 10:00 UTC
Jun 16, 10:00 UTC