ZFS is Perfectly Safe on Hardware RAID
-
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
In that scenario, where you've already chosen to use the software RAID and aren't in a position of having hardware that makes it problematic, yes.
-
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
How else would you connect 24 hard drives, or connedlct a DAS box, in that manner, without an HBA?
Why would you pay good money for a raid card just to not use it?
Ideally you would purchase a server vendor supported card.
Some servers comes with integrated RAID. At least on newer machines most of them are able to passthrough disk directly to the OS, just as I mentioned above on our LSI cards. So the RAID card works exactly like a dumb HBA - if you want.
Sure, but I'm guessing that's not @Obsolesce point. If you don't have an integrated solution - HBA would/should be less expensive than buying a RAID card, plus it's one less piece of complication to get in the way.
additionally - most systems don't have the ability to take 24 drives with an integrated solution.
-
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
The problem is, many RAID cards won't turn off the RAID logic and encapsulation.
Hence the need for RAID 0, which could shield the actual drive from ZFS, I'm guessing.
-
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
HBA would/should be less expensive than buying a RAID card, plus it's one less piece of complication to get in the way.
HBAs aren't always less expensive. And the complexity is identical no matter what.... you shift the complexity from hardware to OS or vice versa, but that's it. Same pieces, you are just choosing which place they go in.
But managing hardware RAID is standard and every bench tech knows how to do it, and every Windows admin, and everyone that uses FreeNAS.
ZFS is highly complicated and if you aren't a UNIX admin with ZFS experience, which essentially includes anyone that would consider something like FreeNAS, then ZFS is quite complicated from a human usability perspective. It exposes way more risk to the admin.
THis is why we see humans failing at ZFS RAID functions causing data loss all the time.
-
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
additionally - most systems don't have the ability to take 24 drives with an integrated solution.
I'm not aware of ANY that don't, 24 is a tiny number. Most integrated solutions are in the hundreds or thousands.
-
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
The problem is, many RAID cards won't turn off the RAID logic and encapsulation.
Hence the need for RAID 0, which could shield the actual drive from ZFS, I'm guessing.
No, needing RAID 0 is the thing I'm talking about. The RAID is still there, hence "can't turn it off."
-
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
additionally - most systems don't have the ability to take 24 drives with an integrated solution.
I'm not aware of ANY that don't, 24 is a tiny number. Most integrated solutions are in the hundreds or thousands.
People get confused by the number of SAS ports on a card and think that equals the number of drives the card can support.
It does not.
There is something called a SAS expander that allows many more drives on the same port. Some servers can be configured with integrated SAS expanders on the SAS backplane. Then you can run say 24 SAS drives on a RAID/HBA card with 4 SAS ports.
SAS expanders are also used in JBOD enclosures. So you don't need a 48 port RAID controller and the corresponding amount of cables to be able to run 48 bays.
-
P420i which is an old, integrated card, does 60 for example. On the low end, but higher than most people can provide to it since no chassis holds that much, and no standard two chassis do, either.
-
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
The problem is, many RAID cards won't turn off the RAID logic and encapsulation.
Hence the need for RAID 0, which could shield the actual drive from ZFS, I'm guessing.
No, needing RAID 0 is the thing I'm talking about. The RAID is still there, hence "can't turn it off."
That is what I was saying - the RAID is still there.. it's a RAID 0.. where HBA wouldn't have RAID at all. The disk is just presented as a disk.
-
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
additionally - most systems don't have the ability to take 24 drives with an integrated solution.
I'm not aware of ANY that don't, 24 is a tiny number. Most integrated solutions are in the hundreds or thousands.
People get confused by the number of SAS ports on a card and think that equals the number of drives the card can support.
It does not.
There is something called a SAS expander that allows many more drives on the same port. Some servers can be configured with integrated SAS expanders on the SAS backplane. Then you can run say 24 SAS drives on a RAID/HBA card with 4 SAS ports.
SAS expanders are also used in JBOD enclosures. So you don't need a 48 port RAID controller and the corresponding amount of cables to be able to run 48 bays.
Awww.. thanks for that.
-
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Dashrender said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Pete-S said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
@scottalanmiller said in ZFS is Perfectly Safe on Hardware RAID:
@Obsolesce said in ZFS is Perfectly Safe on Hardware RAID:
Hardware RAID controllers should not be used with ZFS
This part is obviously just BS. They even have this details list of reasons that if you study it, none of them would result in this conclusion. So this is where the "someone writing the docs through in a little unwarranted opinion" without anything substantiating it. In older docs, this was wrapped with a qualifying statement that includes "if you want ZFS to handle some of these functions." It looks like someone who didn't understand the source document copied what everyone things is the implication, but isn't at all, and missed their own technical details.
I'm not following the context here... I didn't say what you quoted me saying.
In a scenario where one wants to use ZFS as a LVM and software RAID, that is why I said to use HBA on hardware, rather than a RAID card: You would NOT want to use a hardware RAID card and use either passthrough disks or set each disk as a RAID 0. You should use an HBA instead, for ZFS' LVM and software RAID use.
What's the difference here? A RAID card is just like a pure HBA but with RAID logic.
The problem is, many RAID cards won't turn off the RAID logic and encapsulation.
Hence the need for RAID 0, which could shield the actual drive from ZFS, I'm guessing.
No, needing RAID 0 is the thing I'm talking about. The RAID is still there, hence "can't turn it off."
That is what I was saying - the RAID is still there.. it's a RAID 0.. where HBA wouldn't have RAID at all. The disk is just presented as a disk.
Many RAID cards do that, too. It is really just one popular one that doesn't let you, AFAIK.
-
If you use hardware RAID and ZFS, does this configuration support ZFS checksums; therefore, would be the ideal combination? I am getting ready to move my Windows Server over to Linux and would like to get the ideal filesystem right the first time. I was looking at ZFS precisely for checksums to identify bit rot. If it only does this in software RAID mode, is there any other filesystem that would accomplish using hardware RAID AND checksums?
-
@tjatwood109 said in ZFS is Perfectly Safe on Hardware RAID:
If you use hardware RAID and ZFS, does this configuration support ZFS checksums; therefore, would be the ideal combination? I am getting ready to move my Windows Server over to Linux and would like to get the ideal filesystem right the first time. I was looking at ZFS precisely for checksums to identify bit rot. If it only does this in software RAID mode, is there any other filesystem that would accomplish using hardware RAID AND checksums?
@scottalanmiller wrote about this back in June of 2014 (has it really been 8 years already?). http://www.smbitjournal.com/2014/05/the-cult-of-zfs/
If you want "The best filesystem", then ZFS is generally not what you are looking for. Is it bad? By no means, it's very good. The problem is that it is known to be slow (which is why it has such complex caching mechanisms), and many people that decide to use it make poor decisions thinking it does something magical that other storage does not (it doesn't). Scott tells it a lot better than I can here tho.
-
@tjatwood109 said in ZFS is Perfectly Safe on Hardware RAID:
If you use hardware RAID and ZFS, does this configuration support ZFS checksums; therefore, would be the ideal combination?
No, it uses the checksums from the hardware RAID. ZFS is just a normal filesystem when used this way. Totally safe, but can't leverage ZFS's RAID benefits, either.
-
@tjatwood109 said in ZFS is Perfectly Safe on Hardware RAID:
I was looking at ZFS precisely for checksums to identify bit rot.
This is essentially pointless. Bit rot, while real, is a trivial concern that is primarily promoted by the ZFS community to produce feature that need not exist to attempt to "sell" ZFS as if it magically handles bit rot dramatically better than other options.
In the real world, ZFS is great, but first bit rot isn't a major concern and second, almost every platform handles bit rot.
-
@tjatwood109 said in ZFS is Perfectly Safe on Hardware RAID:
If it only does this in software RAID mode, is there any other filesystem that would accomplish using hardware RAID AND checksums?
No, the problem here is that a filesystem is fundamentally the wrong place for this to be handled, that's why you aren't finding the answer you are looking for, because you are attempting to define the answer in the question.
What you WANT to ask is "how do I safely store data" and for most people the answer is "hardware RAID and either XFS on Linux or NTFS on Windows." It's that simple. Exotic filesystems aren't relevant. Normal storage systems handle things like bit rot and have done so for a very long time. ZFS uses the "in the filesystem" trickery to make it sound special by simply redefining what they call the filesystem. In reality, ZFS doesn't answer "yes" to your question either, because ZFS doesn't do it in the filesystem, it does it in the RAID and disks, so does everything else. It's all semantic trickery to confuse.
Bit rot is handled in the hardware and ZFS will never see it. It's just overhead that you don't need.
https://alastairs-place.net/blog/2014/01/16/bit-rot-and-raid/
-
-
@tjatwood109 said in ZFS is Perfectly Safe on Hardware RAID:
Thanks - I will proceed without using ZFS - I prefer hardware RAID.
Tim
ZFS is perfectly fine with hardware RAID, if you like ZFS' features otherwise (like zsend is nice) then there's no reason to avoid it. If you don't plan to use any unique features, then XFS is my "go to" choice by default. Very fast, very stable.
