Best backup strategy for NextCloud?

guyinpv

I'm running NC on a VULTR box. I'll have their built-in backup system doing dailies I suppose, but they only keep 2 copies if I remember right.
There are also snapshots I can take at any time.

My question is, for something like NC, there are really two things going on. There is the server and software itself, the NC "image" so to speak. And then the huge dump of all the files we store there.

I feel like doing a daily backup of the entire file system to be a bit aggressive. Overkill if you will.

Let's say we have 30GB of files in there. In a given day, we might edit 50 files to the tune of 10MB of changes.

Part of me says, do some kind of incremental backup just to get the changed files and store those by date.

If I only store changed files, I could backup multiple times a day, or even do something more akin to revisions. Of course, NC itself stores revisions, but I just mean a differential backup of only changed files.

Anyway, my point is, it seems wasteful and time consuming to do a full system backup daily when I only truly need about 10MB of changed files from the last image. It's not like the OS and NC software is changing. Only the files and DB.

I don't know how to set something like that up, it's probably not even possible since the files are so closely linked to the NC database. Copying in/out files directly would mess up the database right? And if I want to restore something like a deleted folder from 2 weeks ago, it would probably not be possible by just copying the files back, without also the DB parts. No?

Am I stuck with only doing a 30-40GB daily full system image, or can I do something more interesting with my backup strategy?

Note that on VULTR, you can only do a full system restore, so I wouldn't easily be able to jump in some backup system somewhere and look for a missing file from 2 weeks ago that was accidentally deleted.

That sort of thing would be nice though, to do a "restore" at the file level, assuming the missing file is not in the NC trash bin.

I guess I just want a couple levels to the backup plan. A full system image in the case of catastrophe, but also a kind of backup to the NC trash bin. Store my own 2 or 3 weeks of files so I can go back in time and find anything that's gone missing. I certainly don't want to do full system restore just to look for a few files, as this would wipe out all the other files added since that backup. I could never really do a full restore in this sense. I don't need the entire server restore, I just need the latest few changes.

I'm trying to picture a situation where this would be needed. I suppose it's simply that someone deleted a file from days ago accidentally. I assume when you delete a file in NC you also lose all the revisions. But there is no way I'm restoring a multi-day old full system image to look for one or two files. I need a way to browse in a more "partial" way for specific files, assuming the trash bin doesn't apply.

Francesco Provino

@guyinpv just put in your cron a daily script that put nextcloud in maintanance mode, run borg in the nextcloud folder (that contains the DB, also) and exit from maintanance mode.

DustinB3403

...

UrBackup could do only the Delta's and it's designed to be really flexible in what you opt to backup.

And it's free, and it can backup to anything you can mount to your UrBackup server.

Obsolesce

What's your retention policy for data on NextCloud? I think that would help determine your backup method and software (along with RTO and RPO policies) and several other factors of course.

At what point in time should you be allowed to say "Sorry, our backups don't go back that far, so we can't recover that file from that period of time..."? As well as "The oldest copy I have is from XXX, and the most recent is from YYY.". It will take this long XXX to restore it, and cost this much XXX to restore it.".

Obsolesce

Also, how much data is it? How much room do you have for backup data? How fast can you transfer backup data to your backup repository? Does it matter?

black3dynamite

Take a look at this script.
https://help.nextcloud.com/t/scripts-for-backup-restore/21433

JaredBusch

@black3dynamite said in Best backup strategy for NextCloud?:

Take a look at this script.
https://help.nextcloud.com/t/scripts-for-backup-restore/21433

That is a full system backup everytime which is not what the OP is wanting.

JaredBusch

@guyinpv said in Best backup strategy for NextCloud?:

I'm running NC on a VULTR box. I'll have their built-in backup system doing dailies I suppose, but they only keep 2 copies if I remember right.
There are also snapshots I can take at any time.

My question is, for something like NC, there are really two things going on. There is the server and software itself, the NC "image" so to speak. And then the huge dump of all the files we store there.

I feel like doing a daily backup of the entire file system to be a bit aggressive. Overkill if you will.

Let's say we have 30GB of files in there. In a given day, we might edit 50 files to the tune of 10MB of changes.

Part of me says, do some kind of incremental backup just to get the changed files and store those by date.

If I only store changed files, I could backup multiple times a day, or even do something more akin to revisions. Of course, NC itself stores revisions, but I just mean a differential backup of only changed files.

Anyway, my point is, it seems wasteful and time consuming to do a full system backup daily when I only truly need about 10MB of changed files from the last image. It's not like the OS and NC software is changing. Only the files and DB.

I don't know how to set something like that up, it's probably not even possible since the files are so closely linked to the NC database. Copying in/out files directly would mess up the database right? And if I want to restore something like a deleted folder from 2 weeks ago, it would probably not be possible by just copying the files back, without also the DB parts. No?

Am I stuck with only doing a 30-40GB daily full system image, or can I do something more interesting with my backup strategy?

Note that on VULTR, you can only do a full system restore, so I wouldn't easily be able to jump in some backup system somewhere and look for a missing file from 2 weeks ago that was accidentally deleted.

That sort of thing would be nice though, to do a "restore" at the file level, assuming the missing file is not in the NC trash bin.

I guess I just want a couple levels to the backup plan. A full system image in the case of catastrophe, but also a kind of backup to the NC trash bin. Store my own 2 or 3 weeks of files so I can go back in time and find anything that's gone missing. I certainly don't want to do full system restore just to look for a few files, as this would wipe out all the other files added since that backup. I could never really do a full restore in this sense. I don't need the entire server restore, I just need the latest few changes.

I'm trying to picture a situation where this would be needed. I suppose it's simply that someone deleted a file from days ago accidentally. I assume when you delete a file in NC you also lose all the revisions. But there is no way I'm restoring a multi-day old full system image to look for one or two files. I need a way to browse in a more "partial" way for specific files, assuming the trash bin doesn't apply.

BackBlaze B2 would be good. Or any other provider that you can script an rsync to. It sounds to me like you do not want backups. you just want a copy offsite.

If you want actual backups that is different. Then things like the script @black3dynamite mentioned or URBackup as @DustinB3403 mentioned become the best choice.

Emad R

@guyinpv

The thing is and most people forgot with NC is depends on the user to put the files in the NC folder that gets created, if the user does not put the files into the sync folder nothing gets backed up.

I am saying this right ? or there is way to force backup ? like mapping the sync folder to C:\Users. But that is not what NC is made for ...

My theory is if you create NC folder for users in the desktop, and they dont use it = no backups.

guyinpv

@obsolesce said in Best backup strategy for NextCloud?:

What's your retention policy for data on NextCloud? I think that would help determine your backup method and software (along with RTO and RPO policies) and several other factors of course.
At what point in time should you be allowed to say "Sorry, our backups don't go back that far, so we can't recover that file from that period of time..."? As well as "The oldest copy I have is from XXX, and the most recent is from YYY.". It will take this long XXX to restore it, and cost this much XXX to restore it.".

I don't have any of that

Essentially, VULTR's automatic backup only retains the latest 2 images. To me that's a little shallow. I'd like to push it to a week at least. For example if a virus or something screws up all our files late on Friday, I don't want to come in on Monday and find that our oldest backup is still screwed up. I haven't been able to find whether VULTR lets me download or transfer a backup on my own, then at least I could push that to another storage service.

Assuming I can't download an image, that leaves me having to do my own "standard" NextCloud backup procedure, grabbing my files and DB manually.

I'm ok setting this manual backup up, but I still am curious about doing it as an incremental backup just to save space. I'm just not sure how incremental will work with the rest of the NC files and database. Do I dump the DB and include it with each incremental backup? And if so, how does that help if I needed to restore a bunch of files from an incremental? I don't think that would work. Or maybe this situation would never happen?

@jaredbusch said in Best backup strategy for NextCloud?:

BackBlaze B2 would be good. Or any other provider that you can script an rsync to. It sounds to me like you do not want backups. you just want a copy offsite.
If you want actual backups that is different. Then things like the script @black3dynamite mentioned or URBackup as @DustinB3403 mentioned become the best choice.

We have a B2 account. That is really for archival storage but I guess that doesn't matter, I can still delete older backups without any issues in my account?

@emad-r said in Best backup strategy for NextCloud?:

@guyinpv

The thing is and most people forgot with NC is depends on the user to put the files in the NC folder that gets created, if the user does not put the files into the sync folder nothing gets backed up.

I am saying this right ? or there is way to force backup ? like mapping the sync folder to C:\Users. But that is not what NC is made for ...

My theory is if you create NC folder for users in the desktop, and they dont use it = no backups.

For us, NC is not for backups, it's for file sharing. We have different backup for workstations.

For that matter, I could cheat this whole thing and just backup the NC files daily as part of the normal workstation backup, but even those are one week incremental. At least it would be something, I'd have a week's worth of NC changes on a local workstation to pull files out of if ever needed.

Or likewise I could create a second backup on my workstation to do incrementals on NC only to our NAS.

black3dynamite

@emad-r said in Best backup strategy for NextCloud?:

@guyinpv

The thing is and most people forgot with NC is depends on the user to put the files in the NC folder that gets created, if the user does not put the files into the sync folder nothing gets backed up.

I am saying this right ? or there is way to force backup ? like mapping the sync folder to C:\Users. But that is not what NC is made for ...

My theory is if you create NC folder for users in the desktop, and they dont use it = no backups.

To make it transparent, you can create a junction point instead. That way it makes backing up easier.

guyinpv

I'm left with these options then:

VULTR daily backup, only keeps 2. It can only be restored by recreating the entire node, no partial recovery.
A server script to send incremental backups of just the files off to B2 or wherever. These couldn't do full restores of the node, or NC itself.
A server script to do the "official" backup procedure of NC and send off to B2 or wherever. This might be the most flexible and normal solution, but I don't like having to keep so many copies of the entire file store if I can help it. It could be used to restore NC itself in the event the software is hacked or an update screws it up thoroughly (although this would not happen, as I take snapshots before those kinds of updates, I would restore the snapshot instead).
Use my local workstation to pull daily or 2x daily incremental file backups onto our local NAS. This could only be used to find accidentally-deleted files that are not in the NC trash bin but it couldn't restore the server, or NC software directly.

At the end of the day, it's all about the files. If I have to rebuild a VPS and install NC from scratch, so be it, as long as I have the most recent copy of files possible to upload back to it. The second concern is fastest turnaround for people to keep working and use the files. In fact, having the files backed up via a workstation to the NAS is probably the fastest way people can get back to work, because I can just shared the NAS folder internally as I fix everything.

black3dynamite

Here's another backup option for file backups. It's called Restic.
https://restic.net/

You can also backup to B2.

Here's the documentation too.
https://restic.readthedocs.io/en/stable/

brandon220

This can be forked into a new thread if need be - For larger amounts of data in NC how are you all handling the backups? I have a script that backs up NC (database, nc folder, and data drive) to a dedicated NAS. At some point the offsite replication to a remote NAS will take too long and never catch up on a slower WAN link. Is there a recommended way to do incrementals of the NC data? Using rsync? From all my research, it seems like everyone does it different and there is no "best practice". I don't see the need to copy the entire data set for each backup - especially when sending the backup job offsite.

black3dynamite

@brandon220 said in Best backup strategy for NextCloud?:

This can be forked into a new thread if need be - For larger amounts of data in NC how are you all handling the backups? I have a script that backs up NC (database, nc folder, and data drive) to a dedicated NAS. At some point the offsite replication to a remote NAS will take too long and never catch up on a slower WAN link. Is there a recommended way to do incrementals of the NC data? Using rsync? From all my research, it seems like everyone does it different and there is no "best practice". I don't see the need to copy the entire data set for each backup - especially when sending the backup job offsite.

Before backing up the database, nc folder and data drive, do you turn on maintenance mode first, backup and then turn it off after backup is complete?

brandon220

@black3dynamite Yes. I also stop the httpd service even though it isn't necessary.

Obsolesce

@black3dynamite said in Best backup strategy for NextCloud?:

Here's another backup option for file backups. It's called Restic.
https://restic.net/

You can also backup to B2.

Here's the documentation too.
https://restic.readthedocs.io/en/stable/

I did a blog post on using Restic to back up to OneDrive

brandon220

@Obsolesce said in Best backup strategy for NextCloud?:

@black3dynamite said in Best backup strategy for NextCloud?:

Here's another backup option for file backups. It's called Restic.
https://restic.net/

You can also backup to B2.

Here's the documentation too.
https://restic.readthedocs.io/en/stable/

I did a blog post on using Restic to back up to OneDrive

Am I correct in saying rclone would not be needed if you had an nfs mount on the local server where restic is installed?

brandon220

@Obsolesce I have restic working without rclone and it is fast and simple. Have to try and script it now.

Obsolesce

@brandon220 said in Best backup strategy for NextCloud?:

@Obsolesce I have restic working without rclone and it is fast and simple. Have to try and script it now.

I used rclone in order to use OneDrive as a backup repository. It seemed like the easiest supported way to do it. How did you mount OneDrive without rclone? I'm sure there are more ways, just curious.