Tips for Upgrading ClusterControl

ClusterControl is an automation tool for managing open source databases. It supports various databases such as  MySQL, MariaDB, PostgreSQL, Timescale, and MongoDB. It is used to handle the full “deploy, monitor, and scale” cycle of your database, and supports a number of high availability clustered setups - from replication setups to synchronous multi-master clusters.

It plays an important role in managing database operations, for instance managing backups and verifying they are restorable, or managing a failover and orchestrating recovery. Therefore, it is important that the software is kept up to date. Patches are usually released on a weekly basis.

In this blog, we will discuss some tips when upgrading ClusterControl to the latest release.

ClusterControl Pre-Upgrade Tips

Before upgrading ClusterControl, you might want to check your current ClusterControl configuration and services, whether they are up and running. Also, check if there are any errors, misconfigurations, or under-optimized parameters for the CMON database, cmon cloud services, and cmon event services. You can dig into the ClusterControl logs. For the controller log, you can tail the logs on the path /var/log/cmon.log, cmon-event log can be found on /var/log/cmon-events.log, and the log related accessibility to the cloud can be checked at /var/log/cmon-cloud.log

Upgrading ClusterControl

There are two strategies for upgrading ClusterControl, you can upgrade the package directly on the ClusterControl host or you can install the latest version of ClusterControl on a new host, and after that migrate your managed databases to the new ClusterControl instance. 

Upgrade Process for an Existing ClusterControl Instance

Upgrading ClusterControl on the current host is really straightforward, the upgrade instructions will differ based on operating system distribution (CentOS or Debian/Ubuntu). Please take note, upgrading your ClusterControl instance won’t halt or interrupt your existing databases, but you might lose some functionality of ClusterControl eg: autorecovery cluster & node, during the time the upgrade is performed.

• For Redhat/CentOS distribution, clean the repository cache, and then install the clustercontrol packages as below: 

  $ yum clean all
  
  $ yum install clustercontrol clustercontrol-controller clustercontrol-ssh clustercontrol-notifications clustercontrol-cloud clustercontrol-clud s9s-tools

  It will automatically download the latest package and upgrade the existing package with the latest one. After that, restart the service to take effect.

  $ service cmon restart
  
  $ service cmon-ssh restart
  
  $ service cmon-events restart
  
  $ service cmon-cloud restart

• For Debian/Ubuntu distribution, you need to update the package repository, and then install the package through APT

  $ sudo apt-get update
  
  $ sudo apt-get install clustercontrol clustercontrol-controller clustercontrol-ssh clustercontrol-notifications clustercontrol-cloud clustercontrol-clud s9s-tools

  And then restart the ClusterControl services as below:

  $ service cmon restart
  
  $ service cmon-ssh restart
  
  $ service cmon-events restart
  
  $ service cmon-cloud restart

That’s it, you have now upgraded your existing ClusterControl instance. But if you would prefer to not touch your existing ClusterControl instance, for instance to make sure the new ClusterControl version works well in your environment, then you can set it up on a new host. 

Setup a New Host with the Latest ClusterControl Version

You can set up the latest ClusterControl on the new host, the easiest way to install ClusterControl is using the install-cc script as below:

$ wget http://www.severalnines.com/downloads/cmon/install-cc

$ chmod +x install-cc

$ sudo ./install-cc   # omit sudo if you run as root

Give execute permission to the file, and run as root user or other user that has sudo access. It will automatically install the latest version, you just need to follow the instructions. After the setup is finished, you can delete the managed databases and clusters from the old ClusterControl through Cluster Menu -> Delete Cluster.

It will prompt you with a dialogue box to confirm the removal. The database nodes won't be stopped or uninstalled, it just remove the nodes metadata from ClusterControl.

After that, you can import your database nodes into the new ClusterControl through Import Cluster.

And then choose the version, fill the ip address as below:

The last step is to press the Import button, it will automatically import your existing database into the new ClusterControl.

It would be great if you can test the upgrade on the staging/dev environment, before going to production. Also, if you are running ClusterControl HA on your environment, the best way to upgrade is through rolling upgrades. You can upgrade the ClusterControl on your standby node first before the primary upgrade.

Post-Upgrade

After upgrading finish, the easiest to verify if the version of ClusterControl is the latest version by running command below in controller node:

$ cmon -v

You sould also check the logs to see if there were any errors during the upgrade.