Hybrid cloud can be a great way of adding flexibility to your existing on-prem deployments. As we discussed in several blogs, public cloud can be a great addition to your own datacenter, ensuring you can easily scale out to handle the load, reduce your capex and be used to implement disaster recovery procedures. Security is another aspect you have to think through when you plan to build such systems. In this blog post, we will talk about some of the security considerations for hybrid cloud MariaDB deployments.
Connectivity
VPN
The major part of every hybrid infrastructure is the network. After all, we are talking about two environments, local, on-premises and a public cloud, that have to be connected and form a single entity. The connection has to be encrypted. How to approach it, there are numerous ways to do so.
One of them would be to use a solution that is made available by the cloud provider - most of them have some sort of a connectivity option available. It can be AWS Direct Connect if you happen to integrate with Amazon Web Services. If you plan to use Google Cloud, solutions are discussed on the following web site: https://cloud.google.com/hybrid-connectivity. In short, there are a significant number of different options that vary from hardware VPN integration to set up BGP peering.
On the other side of the spectrum, we have software VPN solutions. OpenVPN or similar kind of software can be used to set up a secure, encrypted network connection between your own datacenter and the public cloud. In such a case you would require a separate instance running in the public cloud that will be used for the VPN server. Utilizing software VPNs allows you to pick the solution that best suits your requirements and fits the best in your environment.
Firewall
Databases should never be accessible from external networks. It is paramount to build your environment in a way that the database tier will be reachable only from the limited set of hosts. Exactly what is required and how to do that, it is up to you to decide. A typical setup would consist of a secured database tier that can be accessed only from proxy tier and, if needed, some sort of a jump host should be implemented if required for automation and administration tasks.
Application servers should not have direct access to the database - they do not need to. All the application should be required to do is to connect to the load balancer. Load balancers should be able to connect to the database. A load balancer like ProxySQL is perfectly capable of performing the read/write split and sending the reads and writes to the correct database nodes. The application should be able to connect to the ProxySQL and the rest will be handled by the proxy - database authentication, traffic shaping, distributing the traffic across numerous replicas that you may have. All unnecessary access should be restricted. Security groups, firewalls - these are the tools you want to use to secure your environment.
In short, access to the database hosts should be allowed only on the required ports. For MariaDB it will be, obviously, a port used for the database but also other ports if needed - you may have some sort of exporters or agents installed. For Galera, you would need to open ports for intra-cluster communication. You may also want to have an open port for SSH connections. Ideally, limit the access on a per-host basis; only a limited set of hosts can access a given port. For example, the database port might be accessible from the other database nodes, localhost and proxy layer. There is no need to keep it open for other nodes. Your database nodes may even be located on a separate subnet, ensuring that the security is even tighter.
As for ports, best practices would be to change them from the default settings to something else. Ideally, something random. Changing SSH port from 22 to 2222 or MariaDB port from 3306 to 33306 may help to avoid some of the automated attacks but it can still be figured out if someone is actively looking to get into your network. If you want better security, you may just go ahead with some random values. Set SSH to 5762 and MariaDB to 24359. It is quite likely that no one will be able to guess those. Set your TCP timeouts so that the port scans would be very lengthy and expensive and this will surely increase your chances.
SSL
In addition to VPN and firewall, you should ensure that your database traffic is encrypted using SSL.
Ideally, you will protect both frontend connections (from the load balancers) and the communication between your database nodes (be it replication or the intra-cluster transfer in Galera clusters). ClusterControl can help you to enable those options with just a few clicks.
All you need to do is to let ClusterControl create new certificates or use one of the existing ones - you can import your own certificate if you want. Having SSL enabled ensures that the database traffic won’t be readable even by someone who gained access to your network.
Database Security
Of course, the network is not the only important aspect of security. Yes, it is critical, especially in the hybrid cloud environment, but there are also other very important aspects. One of them is the access control embedded into MariaDB.
Role-Based Access Control for MariaDB
MariaDB comes with a set of instruments to ensure that the database access is properly managed and restricted wherever it’s required. The first line of authentication is users. Everyone and everything that is allowed to access MariaDB should use an assigned user to connect to the database. Such users will have a proper password - you can have the password validation enabled in MariaDB to ensure that the passwords are strong enough. Ideally, you would limit the user’s access host only to hostnames or IP’s of load balancers - this should always be the way users connect to the database. For some administrative users, you may want to keep the localhost access if required. On top of enforcing the proper password strength you can configure the password to expire within some period of time or enforce password rotation on the users. As you can imagine, a proper password rotation policy is something you will want to have implemented.
Every user in MariaDB can have multiple privileges assigned. Privileges can be assigned on several levels - global level, database level, table level or even column level. The best practice is to grant a limited set of privileges to the users as possible. If the user requires only access to a particular table, just grant him that. There is no need for that user to access other tables not to mention other schemas. You can define quite detailed access rights using a large set of privileges that you can grant to users. It ranges from rights to read, update or delete data through database management privileges up to the “super” privilege that allows user to perform actions like managing the replication threads and bypass read_only setting.
On top of that, MariaDB comes with roles - to make user management easier it is possible to define roles with a given set of granted privileges and then assign those roles to the users. Such users will inherit grants related to the role it has been assigned to, making it way easier to manage grants on the large scale: instead of changing the grants for multiple users you can assign them to one specific role and then manage all their privileges by altering the privileges granted to the role they have been assigned to.
You should also ensure that you do not have any pre-existing users without an assigned password or with a too large set of privileges. Such security audit should be performed from time to time, ensuring that you are aware of potential security risks and you can plan to act on them.
Audit log
If your database comes with audit log, just like MariaDB does, you should consider using it to track the actions that are happening in the database. The audit log will help you to accomplish that. With it enabled you will be able to track even the details like which user executed what query. If you happen to use ClusterControl you can enable the audit log just with a couple of clicks:
To sum up this blog, there are a couple of things you should consider when designing a MariaDB deployment in the hybrid cloud environment. Some of them are strictly related to the way the environment is designed, some are pretty much related to the database type that you use and the fact that you use hybrid cloud doesn’t really change much. What is really important is to ensure that your database is properly protected - that’s the ultimate goal, no matter what the environment is.