There’s a way to have user friendly backups in an OpenStack environment – interested? Then this post for you!
For quite some time now I’ve been thinking about backups in OpenStack – how can I have a simple and clear solution to make a backup of Linux virtual machines. At the moment I can think of 1 or 2 backup solu
tions based on snapshots. These solutions are not exactly what I’d like or I could have confidence in the data safety. I don’t really like snapshot based backups so I’ve tried an alternative path of making backups in the old fashioned “Physical server” backup way.
I have selected a product from Veeam – Veeam Agent for Linux FREE version. It’s a free edition of their backup software – and I like free 🙂
Here’s the idea:
In this diagram you can see that one server in the infrastructure is going to serve as a backup storage to store Veeam created backup files. In this particular case the backup storage server will use Samba and CIFS. I think it will overall help me to improve the security side from Wannacry, Petya A or some other variants of crypto viruses. One could challenge a common CIFS shared folder? I say yes because it will be restricted to Veeam as a target repository to store backup archives. Another idea is to store backups in some external location if you’re really cautions 🙂
So, let’s start with the implementation.
In the Telia Cloud Platform we have inexpensive Tier 3 storage, so it is what what I need. I’ve launched the backup server and mounted the Tier-3 storage volume to /home/Archives.
root@backup-server:/# df -h Filesystem Size Used Avail Use% Mounted on udev 487M 0 487M 0% /dev tmpfs 100M 9.7M 90M 10% /run /dev/sda1 9.7G 2.3G 7.4G 24% / tmpfs 497M 0 497M 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 497M 0 497M 0% /sys/fs/cgroup /dev/sdb 296G 119G 163G 43% /home/Archives tmpfs 100M 0 100M 0% /run/user/1001
Install Samba and create a backup repository for first server. In this tutorial I will use my Zimbra server as the backup source, and back it up to /home/Archives/BackupServers/zimbra.
Next, we need to create a local user that will be allowed access to the CIFS share.
root@backup-server:/# ls -al /home/Archives/BackupServers/ total 12 drwxr-xr-x 3 root root 4096 May 17 15:50 . drwxr-xr-x 5 root root 4096 May 17 15:36 .. drwxrwx--- 3 zimbra_srv root 4096 May 28 09:51 zimbra
As you can see, only the zimbra_srv user and the root group have access. All others – do not.
In Samba configuration we have defined a share that allows to connect to the shared directory /Archives/BackupServers/zimbra/ for the user zimbra_srv only.
[ZIMBRA_BACKUP] path = /home/Archives/BackupServers/zimbra/ valid users = zimbra_srv write list = zimbra_srv public = no writable = yes read only = no printable = no create mask = 0700 directory mask = 0700
Now the setup of the backup Sever is complete and we can move on to the example Zimbra server and install Veeam software there.
The setup files for Veeam Linux backup is available from the Veeam site. To download it, though, you’ll to create a user account.
Download URL: https://www.veeam.com/linux-backup-free.html
The user guide: https://www.veeam.com/veeam_agent_linux_2_0_user_guide_pg.pdf
Bear in mind that the free version can only have one backup job. If that’s good enough for you – great, however, if not – you’ll need to buy a paid version of Veeam Agent that’ll remove a single job limitation and also add some other features.
In this case I’ll use the free version. Once the Veeam agent is installed, we need to create a backup job. In this example I will backup the whole OS and mounted volumes.
Run the Veeam agent configuration executable and create the backup job and set up the backup repository.
As you can see, there are three options for the backup to choose from:
Select “Shared folder” to connect to the backup server using CIFS:
Specify the network and share connection information, and define the number of restore points:
There are some nice features around the backup encryption but that’s available in a paid version only.
Set the backup schedule and you’re mostly done with the backup job definition:
In the end of the wizard you’ll have a summary, to confirm – select “Finish” and you’re done.
Here’s what I’ve got after some time in the form of daily backups:
All backups were successful, and let’s see how these backups are stored in the backup server with CIFS:
Right, so the backups for 7 days take up only 1.7G, which is nice.
root@backup-server:/# du -sh /home/Archives/BackupServers/zimbra/zimbra-server\ BackupJob1/ 1.7G /home/Archives/BackupServers/zimbra/zimbra-server BackupJob1/
If you to restore something for some reason, the process is not complicated. Launch the Veeam agent executable and select the backup job you need, push the “R” button on keyboard, then you’ll be prompted to select the day and time of the backup you’re interested in and press Enter. Veeam will then connect to the backup server CIFS share and mount the restore point to /mnt/backup:
Here’s an example of how the mount point with backup data looks like:
And as you see, Veeam agent has mounted the sdd volume as disk and we can do whatever we need to do with the data in that restore point.
Restoring data from individual volumes can be simple, so can you restore the whole operating system in the root disk? In this case Veeam helps us with their Recovery Media solution. It’s an ISO image which you can download from the Veeam site.
I’m going put it to /VeeamBoodISO/veeam-recovery-media-2.0.0.400_x86_64.iso and add new record to grub -> /etc/grub.d/40_custom
menuentry "Veeam Recovery Media" {
insmod ext2
set isofile="/VeeamBoodISO/veeam-recovery-media-2.0.0.400_x86_64.iso"
loopback loop (hd0,1)/$isofile
linux (loop)/isolinux/velinux iso-scan/filename=$isofile quiet noeject nopromt splash --
initrd (loop)/isolinux/initrd.img
}
…and modify…
root@zimbra-server:/# more /etc/default/grub
…to have ability to have choice select boot image:
GRUB_DEFAULT=0 GRUB_HIDDEN_TIMEOUT=10 GRUB_HIDDEN_TIMEOUT_QUIET=false GRUB_TIMEOUT=10
And then update grub records by running update-grub2.
Once you reboot your instance, check the VM instance KVM console in OpenStack Horizon – you’ll have the updated boot menu and from now on can have the ability to restore the OS, specific volumes, or all objects at the same time.
Select Veeam Recovery to boot.
After booting to “Veeam Recovery Media” you will have the following menu:
Next, you’ll be notified about network state and root credentials.
Go to Restore volumes, let’s assume I want restore the OS to the yesterday’s copy, and then select “Add shared folder…” to connect to the backup server CIFS share.
The connection setup is pretty straightforward:
…and here we are in the backup server CIFS share. Go to the Backup Job:
Select the backup points that you want to restore from:
Select the boot partition to restore and press “Enter”:
Select the destination to restore to:
In this example, I’ve selected the OS disk sde with a size of 20 GB.
Press “Enter” to confirm and start the recovery process:
Recovery in progress…
Once done:
Now, select “Reboot” and proceed with the bootloader to load the OS.
There are questions!
There may be situations when, for instance, the OS partition is corrupted and then the boot loader will fail over to the Veeam Recovery Media ISO. What to do then? It this case there are 2 options:
- Once your Linux deployment and setup are complete, along with the Veeam backup agent configuration, you’ll need to create the VM instance snapshot from the OpenStack Horizon dashboard or CLI/API. This will allow to rebuild the VM instance with the boot volume (created from snapshot) and perform a full restore from the Veeam Recovery media.
- You can install a new instance, mount the Veeam Recovery Media ISO, boot from it and then do the same recovery. In this case your VM instance IP address will be different from the original one. To overcome this limitation, you can leverage a OpenStack Neutron port to retain a specific IP address. This can’t be done from the OpenStack Horizon dashboard but can be using CLI/API.
I think the same solution can be used with a Windows VM instance. In that case the backup could be even more flexible as the Windows Veeam Agent version is much more user friendly with a nice GUI. You could then backup OS + disks + selected files to the same backup server with CIFS (as configured above) and use the Veeam Recovery Media for any restore operations.
Creating a port in OpenStack using CLI is quite simple once you know how to do it 🙂
First, you need to know the name for the network you want to create that virtual port in. Second, decide whether or not you want a specific IP address. If the former, then it’s really simple as:
openstack port create --network Internal-network my_port_nameIn this example a port named “my_port_name” with an IP address from the allocation pool will be created in the network called “Internal-network”.
If the latter, i.e. you want a specific IP address from a network subnet, you’ll need to know the subnet name. If you want all accessible subnet names, execute this:
openstack subnet listThis will give you a list of all subnets you can see. However, if you want to see the subnets for a specific network only, you’ll need this:
openstack subnet list --network Internal-networkThe “Internal-network” here is just a name, valid for my case, this needs to be tailored to each individual scenario.
Provided you have access to the subnet you want, you can then execute the following to allocate a specific IP address:
openstack port create --network Internal-network --fixed-ip subnet=opnsense-lan_subnet-5sz2kuf43q3p,ip-address=172.27.200.100 port_172.27.200.100In this example my environment specific values are “Internal-network”, “opnsense-lan_subnet-5sz2kuf43q3p”, IP address value (172.27.200.100) and the port name “port_172.27.200.100”. Again, change these values to your specific environment.
Thank you for your detailed explanation of OpenStack. It was helpful. Keep on posting!