Backups - how much does backup performance matter to you?
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@BRRABill said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@BRRABill said in Backups - how much does backup performance matter to you?:
@Tim_G said
It can help a lot to plan backups by starting with recovery.
Right, and this has been discussed here many times in that if you have important data, it probably shouldn't be taking 8 hours to restore it all.
Perhaps a full restore is not the optimal way to restore in the case of a major issue.
Importance of the data doesn't determine time to restore,normally, but the importance of downtime. VERY different things.
My point being that if there are a few files or an application that NEEDS restoring ASAP, perhaps that should be part of the plan versus pegging 8 hours for a full backup.
Sure I suppose I can see what you're saying,
So let's say you know what files you need first, you loose the whole server, so you restore those few files first before kicking off the full restore, during which you often have to wait until it's completed before accessing most if not all of that data.
Definitely not an option when restoring an application system either.
Application system doesn't even need to be restored from backup, it can be built fresh.
The application data needs to be restored and that might be the bulk of what's actually being restored and takes a ton of time
Application "data" isn't in the application, it's in the database. That's part of the database restore, not the application restore.
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@BRRABill said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@BRRABill said in Backups - how much does backup performance matter to you?:
@Tim_G said
It can help a lot to plan backups by starting with recovery.
Right, and this has been discussed here many times in that if you have important data, it probably shouldn't be taking 8 hours to restore it all.
Perhaps a full restore is not the optimal way to restore in the case of a major issue.
Importance of the data doesn't determine time to restore,normally, but the importance of downtime. VERY different things.
My point being that if there are a few files or an application that NEEDS restoring ASAP, perhaps that should be part of the plan versus pegging 8 hours for a full backup.
Sure I suppose I can see what you're saying,
So let's say you know what files you need first, you loose the whole server, so you restore those few files first before kicking off the full restore, during which you often have to wait until it's completed before accessing most if not all of that data.
Definitely not an option when restoring an application system either.
Application system doesn't even need to be restored from backup, it can be built fresh.
The application data needs to be restored and that might be the bulk of what's actually being restored and takes a ton of time
This is why I use VMs, and separate the os, data, and application on different virtual disks if possible. In SQL cases, you will by default to follow best practices... Database, logs, etc on different .vhdx's, maybe on different tiers of storage, ssd vs hdd.
OS takes minutes, applications take minutes, then comes the data however you like it.
Typically those kinds of restores are relatively quick. Its usually file server data in SMBs that take forever to restore. Most SMB databases are less than 100 GB. But if the need comes to restore a 10 TB file store because the whole thing blew up, sure that will take a long time unless you have it replicated onsite or can spin it up in the cloud.
-
@Tim_G
RIght, that is kind of my point.
It's not feasible for me as a SOHO/SMB, but for many it might be.
A lot of times teeny companies think they need to be up 24/7 without downtime without realizing the cost.
If there are mission critical applications, perhaps they need live replication, or something such as that.
-
@BRRABill said in Backups - how much does backup performance matter to you?:
@Tim_G
RIght, that is kind of my point.
It's not feasible for me as a SOHO/SMB, but for many it might be.
A lot of times teeny companies think they need to be up 24/7 without downtime without realizing the cost.
If there are mission critical applications, perhaps they need live replication, or something such as that.
Still can work well for an SMB, a lot of those things, like separating applications from databases, can be free. Not necessarily a scale thing, can just be a better approach.
-
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
-
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
That's insane. Did you ever have to hand it back to them after a disaster? "Hey, this is what you said was acceptable, folks."
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
wat
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
I'm just going to assume you can hit a target of less than 4 days by now.
-
@wirestyle22 said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
Other than a board inappropriately getting involved with details they should know nothing about, the final decision is very sensible. What SMB needs all systems restored in less than four days? Almost none. SOme, certainly, but not the average.
-
@NetworkNerd said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
That's insane. Did you ever have to hand it back to them after a disaster? "Hey, this is what you said was acceptable, folks."
yep - they were fortunate, they never had to suffer a real 4 day outage - but after 2-3 four hour outages, I revisited the 4 day thing, they changed their tune. We still didn't drop to a solution that was less than 4 hours for catastrophic, but things did get better.
-
All a function of time and need and money.
-
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@wirestyle22 said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
Other than a board inappropriately getting involved with details they should know nothing about, the final decision is very sensible. What SMB needs all systems restored in less than four days? Almost none. SOme, certainly, but not the average.
We have these numbers of what it costs us to do something - though those numbers mostly don't exist for things like internal paperwork. i.e. EHR is down, fall back to paper - now imput that paper into the EHR (and not just scans of the paper - actual data entry). That data entry is the actual cost to us of downtime for the most part. We can still see patients in most cases in the clinic - surgeries of course stop.
So yeah, having this system down for four days would have been bad.
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@wirestyle22 said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
Other than a board inappropriately getting involved with details they should know nothing about, the final decision is very sensible. What SMB needs all systems restored in less than four days? Almost none. SOme, certainly, but not the average.
We have these numbers of what it costs us to do something - though those numbers mostly don't exist for things like internal paperwork. i.e. EHR is down, fall back to paper - now imput that paper into the EHR (and not just scans of the paper - actual data entry). That data entry is the actual cost to us of downtime for the most part. We can still see patients in most cases in the clinic - surgeries of course stop.
So yeah, having this system down for four days would have been bad.
But, if the math is sound that arrived at the 4 day limit, it may not be as critical as it seems connotatively.
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
That's insane. Did you ever have to hand it back to them after a disaster? "Hey, this is what you said was acceptable, folks."
yep - they were fortunate, they never had to suffer a real 4 day outage - but after 2-3 four hour outages, I revisited the 4 day thing, they changed their tune. We still didn't drop to a solution that was less than 4 hours for catastrophic, but things did get better.
So they didn't calculate correctly? Did they figure out the initial mistakes in their financial numbers? How did they get their financial numbers so far off?
-
@BRRABill said in Backups - how much does backup performance matter to you?:
All a function of time and need and money.
Only time and money, need in a business is always a function of money.
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@wirestyle22 said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
Other than a board inappropriately getting involved with details they should know nothing about, the final decision is very sensible. What SMB needs all systems restored in less than four days? Almost none. SOme, certainly, but not the average.
We have these numbers of what it costs us to do something - though those numbers mostly don't exist for things like internal paperwork. i.e. EHR is down, fall back to paper - now imput that paper into the EHR (and not just scans of the paper - actual data entry). That data entry is the actual cost to us of downtime for the most part. We can still see patients in most cases in the clinic - surgeries of course stop.
So yeah, having this system down for four days would have been bad.
And the board didn't know or care enough about the business to know that?
-
@art_of_shred said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@wirestyle22 said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
Other than a board inappropriately getting involved with details they should know nothing about, the final decision is very sensible. What SMB needs all systems restored in less than four days? Almost none. SOme, certainly, but not the average.
We have these numbers of what it costs us to do something - though those numbers mostly don't exist for things like internal paperwork. i.e. EHR is down, fall back to paper - now imput that paper into the EHR (and not just scans of the paper - actual data entry). That data entry is the actual cost to us of downtime for the most part. We can still see patients in most cases in the clinic - surgeries of course stop.
So yeah, having this system down for four days would have been bad.
But, if the math is sound that arrived at the 4 day limit, it may not be as critical as it seems connotatively.
Damn, someone downvoted this? WTH? Art's absolutely right!
In my situation the cost of shorting it to a day was significantly less expensive than the financial and reputation costs involved. Something they didn't consider when making the original choice.
-
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
That's insane. Did you ever have to hand it back to them after a disaster? "Hey, this is what you said was acceptable, folks."
yep - they were fortunate, they never had to suffer a real 4 day outage - but after 2-3 four hour outages, I revisited the 4 day thing, they changed their tune. We still didn't drop to a solution that was less than 4 hours for catastrophic, but things did get better.
So they didn't calculate correctly? Did they figure out the initial mistakes in their financial numbers? How did they get their financial numbers so far off?
If you have to ask, I think you know the answer.
-
@Dashrender said in Backups - how much does backup performance matter to you?:
@art_of_shred said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@scottalanmiller said in Backups - how much does backup performance matter to you?:
@wirestyle22 said in Backups - how much does backup performance matter to you?:
@Dashrender said in Backups - how much does backup performance matter to you?:
@NetworkNerd said in Backups - how much does backup performance matter to you?:
It all depends on the RTO you need to hit. But to the points being made here, the RTO may be different depending on the applications in your environment. Maybe the ERP system and its SQL database are all that need to come back within 1 hour of going down but the file server's RTO could be closer to 2 or 3 hours.
We talk about RTO and RPO, but I bet you in most places those are not clearly defined down to the VM / application level. If they are and you've planned accordingly, bravo.
I had an RTO of 4 days approved by the board for our EHR 10 years ago - I couldn't believe it when they said that was fine.
Other than a board inappropriately getting involved with details they should know nothing about, the final decision is very sensible. What SMB needs all systems restored in less than four days? Almost none. SOme, certainly, but not the average.
We have these numbers of what it costs us to do something - though those numbers mostly don't exist for things like internal paperwork. i.e. EHR is down, fall back to paper - now imput that paper into the EHR (and not just scans of the paper - actual data entry). That data entry is the actual cost to us of downtime for the most part. We can still see patients in most cases in the clinic - surgeries of course stop.
So yeah, having this system down for four days would have been bad.
But, if the math is sound that arrived at the 4 day limit, it may not be as critical as it seems connotatively.
Damn, someone downvoted this? WTH? Art's absolutely right!
In my situation the cost of shorting it to a day was significantly less expensive than the financial and reputation costs involved. Something they didn't consider when making the original choice.
I meant to upvote it, and clicked to quick without looking
