Observability is critical piece of the operations puzzle - you have to be able to tell the state of your system based on data. Ideally, this data will be available from a single location. Having multiple applications, each handling separate pieces of data, is a direct way to serious troubles. When the issues start, you have to be able to tell what is going on quickly rather than trying to analyze and merge reports from multiple sources.
ClusterControl, among other features, provides users with a one single point from which to track the health of your databases. In this blog post, we will show some of the observability features in ClusterControl.
Overview Tab
Overview section is a one place to track the state of one cluster, including all the cluster nodes as well as any load balancers.
It provides easy access to multiple pre-defined dashboards which show the most important information for the given type of cluster. ClusterControl supports different open source datastores, and different graphs are displayed based on the vendor. ClusterControl also gives an option to create your own, custom dashboards:
One key feature is that graphs are aggregated across all cluster nodes. This makes it easier to track the state of the whole cluster. If you want to check graphs from each of the nodes, you can easily do that:
By ticking “Show Servers”, all nodes in the cluster will be shown separately allowing you to drill down into each one of them.
Nodes Tab
If you would like to check a particular node in more details, you can do so from the “Nodes” tab.
Here you can find metrics related to given host - CPU, disk, network, memory. All the important bits of data which define how a given server behaves and how loaded it is.
Nodes tab also gives you an option to check the database metrics for a given node:
All of those graphs are customizable, you can easily add more of them:
Nodes tab also contains metrics related to nodes other than databases. For example, for ProxySQL, ClusterControl provides extensive list of graphs to track the state of the most important metrics.
Advisors
Trending data is not enough on its own. Sure, it is great for post mortem analysis. Or when working on capacity planning, historical data stored in a form of graphs will be of great use. But to have a full view of the cluster, you have to have some sort of alerting too. If something is happening right now, the user has to be alerted.
ClusterControl provides a list of pre-defined advisors that are intended to track the state of different metrics and state of your databases. When needed, an alert is created. As you can see on the screenshot above, it is not only about metrics. ClusterControl runs sanity checks for important settings, it also does some predictions. For example, regarding disk space utilization, it attempts to alert the user in case disk utilization increases too fast. Of course, alerts are sent not only through advisors. Events like node down or failed backup will also result in a notification.
Worth to note that, advisors are written in a JavaScript-like language and can be edited using the Developer Studio within ClusterControl:
User can also create new advisors and schedule them to be executed by ClusterControl.
With this, users have the option to develop their own scripts which check for important bits specific to the environment. Such scripts can also leverage other ClusterControl functionality, for example, if you’d like to implement automated scaling based on the growth of some metric.