What Makes Parity RAID Safe on SSDs
-
@scottalanmiller said in What Makes Parity RAID Safe on SSDs:
@marcinozga said in What Makes Parity RAID Safe on SSDs:
Now another thing is most misunderstand what URE is and assume it's a set number and you always hit URE after reading 12.5 TB. It's not, it's just a probability or running into sector on disk that cannot be read from. Similar to playing Russian Roulette, if you're lucky, you can keep pulling the trigger indefinitely without blowing your brains out, you can have RAID 5 rebuilding PBs of data without ever hitting URE.
That's why it's a percentage risk number. Otherwise an array of 12.5TB would have a 100% failure rate, but that never quite happens. There is never 100% success nor 100% failure no matter how big or small the array. But it gets really high, really quickly.
I can honestly say in the dozen or so RAID-5 rebuilds that I've seen in the last 6 years, literally half of them were successful, and the other half were not. This was in arrays of 6 x 250GB drives to 8 x 1TB drives (all Winchester / spinning rust).
-
@dafyre How many of the failed ones were actually caused by URE? I'd be interested to see the %.
-
@marcinozga said in What Makes Parity RAID Safe on SSDs:
@dafyre How many of the failed ones were actually caused by URE? I'd be interested to see the %.
That I can't tell you. Drive "help me" lights came on, and we replaced drives.
At one point, with one of our LeftHand SANs, I was sent 8 brand new drives, as the techs saw severe errors on all 8 of them, lol. I had to re-install the LeftHand OS from scratch and let that thing sync with our other unit across the way. Only took a week to sync 7TB... The upshot is that nothing went down or slowed to a crawl.
-
If I'm thinking about this correctly, a help me light is a failed/failing drive, not usually a URE. Single UREs happen all the time but don't take the drive down/offline.
This kinda tells me that the lights indicate drive failures.
-
@dafyre said in What Makes Parity RAID Safe on SSDs:
@scottalanmiller said in What Makes Parity RAID Safe on SSDs:
@marcinozga said in What Makes Parity RAID Safe on SSDs:
Now another thing is most misunderstand what URE is and assume it's a set number and you always hit URE after reading 12.5 TB. It's not, it's just a probability or running into sector on disk that cannot be read from. Similar to playing Russian Roulette, if you're lucky, you can keep pulling the trigger indefinitely without blowing your brains out, you can have RAID 5 rebuilding PBs of data without ever hitting URE.
That's why it's a percentage risk number. Otherwise an array of 12.5TB would have a 100% failure rate, but that never quite happens. There is never 100% success nor 100% failure no matter how big or small the array. But it gets really high, really quickly.
I can honestly say in the dozen or so RAID-5 rebuilds that I've seen in the last 6 years, literally half of them were successful, and the other half were not. This was in arrays of 6 x 250GB drives to 8 x 1TB drives (all Winchester / spinning rust).
That's a rough fail rate.
-
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
-
@marcinozga said in What Makes Parity RAID Safe on SSDs:
@dafyre How many of the failed ones were actually caused by URE? I'd be interested to see the %.
Yeah. RAID fails from so many factors. Often no one records what affected what. We were hit with URE losses on an array that lost zero disks! We never used RAID 5 again
-
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Ha, I've been lucky to never use RAID 5
-
@Dashrender said in What Makes Parity RAID Safe on SSDs:
If I'm thinking about this correctly, a help me light is a failed/failing drive, not usually a URE. Single UREs happen all the time but don't take the drive down/offline.
This kinda tells me that the lights indicate drive failures.
URE would never cause a drive failed light as the drive is fine. Light indicators are always failed or failing drives. But a single drive loss would not be what killed the machine, it's a drive loss PLUS a URE that would do it.
-
@scottalanmiller said in What Makes Parity RAID Safe on SSDs:
@dafyre said in What Makes Parity RAID Safe on SSDs:
@scottalanmiller said in What Makes Parity RAID Safe on SSDs:
@marcinozga said in What Makes Parity RAID Safe on SSDs:
Now another thing is most misunderstand what URE is and assume it's a set number and you always hit URE after reading 12.5 TB. It's not, it's just a probability or running into sector on disk that cannot be read from. Similar to playing Russian Roulette, if you're lucky, you can keep pulling the trigger indefinitely without blowing your brains out, you can have RAID 5 rebuilding PBs of data without ever hitting URE.
That's why it's a percentage risk number. Otherwise an array of 12.5TB would have a 100% failure rate, but that never quite happens. There is never 100% success nor 100% failure no matter how big or small the array. But it gets really high, really quickly.
I can honestly say in the dozen or so RAID-5 rebuilds that I've seen in the last 6 years, literally half of them were successful, and the other half were not. This was in arrays of 6 x 250GB drives to 8 x 1TB drives (all Winchester / spinning rust).
That's a rough fail rate.
There for a few years, we had a crazy time with stuff crapping out. Even a new drive array went crazy. After the LeftHand, the RAID 5 rebuilds got better, but still lost far more of them than we should have.
-
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
I'm not actually 100% sure I've ever seen one recover. Seems like I must have, but I can't actually remember it.
-
@Dashrender Some RAID controllers have certain limit on number of URE they encounter before marking drive as failed. I had that happen on some LSI controllers, but the numbers were really high, perhaps in thousands.
-
@marcinozga said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Ha, I've been lucky to never use RAID 5
How long have you been in the game?
-
@Dashrender said in What Makes Parity RAID Safe on SSDs:
@marcinozga said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Ha, I've been lucky to never use RAID 5
How long have you been in the game?
Long enough. 17 years if my math is correct. I always managed to push RAID 10 over 5.
-
@marcinozga said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
@marcinozga said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Ha, I've been lucky to never use RAID 5
How long have you been in the game?
Long enough. 17 years if my math is correct. I always managed to push RAID 10 over 5.
Wow - so back in the late 90's you were able to always justify the price of RAID 10, great. I wasn't - and really it wasn't needed either.
-
@Dashrender I came from heavy gaming background, so performance was always my priority, sometimes the only priority, lol. So RAID 5 was off the table, period. And I was lucky enough to always work for people who had no clue what I was actually doing.
-
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Me either, technically.
Though I have lost a good percentage just from plugging the new drive in.
-
@BRRABill said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Me either, technically.
Though I have lost a good percentage just from plugging the new drive in.
This is what triggers the resilver
-
@dafyre said in What Makes Parity RAID Safe on SSDs:
@BRRABill said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Me either, technically.
Though I have lost a good percentage just from plugging the new drive in.
This is what triggers the resilver
I always thought it was the whining of my users as the server's performance dropped off a cliff lol
-
@MattSpeller said in What Makes Parity RAID Safe on SSDs:
@dafyre said in What Makes Parity RAID Safe on SSDs:
@BRRABill said in What Makes Parity RAID Safe on SSDs:
@Dashrender said in What Makes Parity RAID Safe on SSDs:
I've been lucky I guess - I've never lost an array when resilvering a RAID 5 array.
Me either, technically.
Though I have lost a good percentage just from plugging the new drive in.
This is what triggers the resilver
I always thought it was the whining of my users as the server's performance dropped off a cliff lol
You mean you can't hear your users when things are working well? Dang... I need to come work with you.
