KVM and Back Ups

DustinB3403

So in my lab, now that things are operational (Beginner's Guide to KVM Administration) we need to next look at how we're going to be backing up our VM's.

Using snapshots, while might work in a pinch but they aren't backups.

And while we could likely copy the files that make up the VM this requires us to shutdown our VM's to get those files.

Downtime is yet again a negative.

So what options do we have? We could use the mainstream tools for our VMs like Veeam or maybe something open source like UrBackup with an agent on every VM.

Of course installing an agent onto every VM is a pain in the rear. . . . but maybe.

So of course our next approach would be something run at the hypervisor, but using the CLI on a regular basis to create backups would be tedious. Maybe crontab or some such approach could work.

While digging around for an answer to this question, I found an answer on another forum, from several years ago. Anyhow that answer is at least 5 years old. Here is the most current answer, a very in-depth script to backing up your VMs.

So in-depth that so far I haven't finished reading through the entire process.

So at this point in the topic, I'm going to simply put a pause here. @scottalanmiller @JaredBusch or anyone else who may be using this script in their labs, let me know what if anything to watch out for.

Obsolesce

I'm not aware of any agentless KVM backup software.

I've been using ReaR, but that does the entire server and/or disks of the host (effectively backing up the VMs), but they aren't individually restorable.

However, ReaR is easy enough to set up automagically in each VM with Salt. I feel ya though, definitely a pain to have to install agents or software in each VM... But then again, Salt would take care of that.

I'll definitely be watching this thread for ideas or agentless KVM VM backup software.

DustinB3403

@tim_g I'm trying to figure out the system requirements to get this perl script to run, and then how to run it.

And maybe an easy way to update it for any systems I'd want to backup.

Obsolesce

@dustinb3403 said in KVM and Back Ups:

@tim_g I'm trying to figure out the system requirements to get this perl script to run, and then how to run it.

And maybe an easy way to update it for any systems I'd want to backup.

I only took a few 2-3 second glimpses along that script... and I seen a spot that turns the VM off. Does it actually do as you think? You looked at it more thann I did, so perhaps I seen that part out of context.

DustinB3403

@tim_g said in KVM and Back Ups:

@dustinb3403 said in KVM and Back Ups:

@tim_g I'm trying to figure out the system requirements to get this perl script to run, and then how to run it.

And maybe an easy way to update it for any systems I'd want to backup.

I only took a few 2-3 second glimpses along that script... and I seen a spot that turns the VM off. Does it actually do as you think? You looked at it more thann I did, so perhaps I seen that part out of context.

It has several different options, it can snap, turn off the vm and then power it back on etc.

Obsolesce

@dustinb3403 said in KVM and Back Ups:

@tim_g said in KVM and Back Ups:

@dustinb3403 said in KVM and Back Ups:

@tim_g I'm trying to figure out the system requirements to get this perl script to run, and then how to run it.

And maybe an easy way to update it for any systems I'd want to backup.

I only took a few 2-3 second glimpses along that script... and I seen a spot that turns the VM off. Does it actually do as you think? You looked at it more thann I did, so perhaps I seen that part out of context.

It has several different options, it can snap, turn off the vm and then power it back on etc.

Ah, good. That must have been the part I seen then. I'll look at it more another time when I have the time...

DustinB3403

For the requirements, I found out you need to install perl-XML-Simple

Obviously this needs to be installed to the server.

So. . .

ssh root@SERVER-IP-Address
sudo dnf install perl-XML-Simple

I now have some output to play with.

stacksofplates

https://mangolassi.it/topic/12537/kvm-snapshot-backup-script

Obsolesce

@stacksofplates said in KVM and Back Ups:

https://mangolassi.it/topic/12537/kvm-snapshot-backup-script

The name is a little misleading, I had to read into it and pick it apart to get it.

So it looks like this creates a full backup of the VM. This seems to accomplish what is needed here @DustinB3403

The function of the script is to create a full backup of the VM and VM's disk, without having to shut down the VM. The way it's done is by creating a snapshot, to get the VM off of it's main virtual disk, allowing you to get a solid replica of it, then the script deletes the snapshot, merging the it back into the main VD.

Nice @stacksofplates !

matteo nunziati

@dustinb3403 I've been using this script in production for somethin glike 1 year when I was using KVM as hypervisor.
this is the facto the only good solution for KVM. It also supports gluster volumes!
you do nopt need to bring the VM down IF you build your VM disk on some snapshottable stuff like LVM or qcow (not sure about the latter).

I've written a small note for my usage at the time... need a bit to translate it in english. this is how I used it in brief

wget http://repo.firewall-services.com/misc/virt/virt-backup.pl
chmod 755 virt-backup.pl
yum install perl perl-XML-Simple perl-Sys-Virt
mkdir -p /var/lib/libvirt/backup
#(maybe you want this to be a share mounted from a remote NAS)
/$PATH/$TO/virt-backup.pl --action=dump --vm=VM_NAME --compress=pbzip2

note that I installed bbzip from EPEL to speedup compression and make full use of all server cores: my backups were scheduled during not working hours with the server idling for the most.

matteo nunziati

@DustinB3403 almost forgotten... I've this stub on github

Emad R

@dustinb3403

In my previous experiments which worked but have not used it in production. I would just freeze the VM, then rsync it, then thaw it. If your backing up to LAN down time will be very minimal and with rsync you can backup incrementally of you wish.

It is recommended to also stop services first then freeze then rsync then thaw then start services again. I liked it and its KISS principle.

Obsolesce

I found this. I may try it on my laptop later.

Live backup of QEMU/KVM/libVirt virtual machines via Duplicity
https://itefix.net/content/live-backup-qemu-kvm-libvirt-virtual-machines-duplicity

Obsolesce

Here's something as well... KVM live backup software, that's actually current:

https://www.handybackup.net/kvm-backup.shtml

JaredBusch

I have found that there are many easy ways to get a backup of your VM.

What I have not found is something that can look at those snapshots that are made and backup incremental differences.

I cannot offsite 1+ TB per day in image files.

Obsolesce

@tim_g said in KVM and Back Ups:

Here's something as well... KVM live backup software, that's actually current:

https://www.handybackup.net/kvm-backup.shtml

Wtf... the download is an EXE. I don't get it...

Obsolesce

@jaredbusch said in KVM and Back Ups:

I have found that there are many easy ways to get a backup of your VM.

What I have not found is something that can look at those snapshots that are made and backup incremental differences.

I cannot offsite 1+ TB per day in image files.

Yeah that would be great to find something like that. I'm looking. I remember someone finding something that's paid. But I forget what it was. I forgot to bookmark it.

JaredBusch

@tim_g said in KVM and Back Ups:

@jaredbusch said in KVM and Back Ups:

I have found that there are many easy ways to get a backup of your VM.

What I have not found is something that can look at those snapshots that are made and backup incremental differences.

I cannot offsite 1+ TB per day in image files.

Yeah that would be great to find something like that. I'm looking. I remember someone finding something that's paid. But I forget what it was. I forgot to bookmark it.

I pay for that functionality now with Hyper-V and Veeam. So I do not mind paying for it for KVM.

Obsolesce

Found it:

https://storware.eu/en/storware-vprotect/

It shows incrementals in the screenshots anyways.

stacksofplates

You could theoretically just use one of the scripts to do incremental. It’s all AOW/ROW so you just get a new disk each snapshot. Then at the end of the week or whatever block commit everything back into the main image and start over.