Agreed, ZFS now more than ever. Especially after reading the above "why" comments.

It doesn't take long infront of OpenSolaris to see how sexy ZFS. FUSE just doesn't work at the moment as an alternative, but its an amazing effort.

Indeed, I am quite excited about Btrfs as well. I just tried your ebuilds and it works quite well. This will be the solution for todays large hard disks and often shoddy hardware RAID implementations.

I wish more of kernel gurus would help speed this awesome FS into mainline.

One of the reasons why PATA was replaced by SATA !

What about encrypted filesystems ? Would decryption process provide data integrity checks as a side bonus ?

FWIW: it seems there are at least two ZFS's. The other one (strictly, zFS, "z/OS Distributed File Service zSeries File System") is a file system for IBM System z9 and z10 mainframes' z/OS UNIX System Services, and is an alternative for the earlier HFS.

@Erik: It's important because error detection on lower levels just doesn't do the job. Check out page 9 of Eric Kustarz' ZFS presentation ("ZFS - The Last Word in File Systems") about what else can cause data corruption in addition the simple media errors that HW protection can cover.

Disks are a lot less reliable than RAM. However, don't think that you're right in thinking RAM is never protected by software means: just search for "Software detection mechanisms providing full coverage against single bit-flip faults" for an example.

Apart from filesystembitrot, how do you actually verify that your common-off-the-shelf platform actually DOES something with that expensive ecc-memory if it supports it all? I'm not talking serverhardware in here, but that stuff you'll get at consumerlevel. Desktops and Laptops and the like.

Why should data integrity be handled on the file system level? Does the operating system's memory manager handle data integrity of RAM? I thought that that was handled entirely in hardware and see no reason why permanent storage should be different.

I just found this blog entry:
http://blogs.sun.com/darren/entry/zfs_under_gplv2_already_exists
As far as I understand, the core ZFS is already GPL2 complaint. Or is this a misunderstanding?

The problem with btrfs is that it will take at least 2-3 years before it gets anywhere near production ... thats a bit too late for most people depending on computers storing the data safe. Actually, to me as a physicist, the failure probability given by the manufacturers seems to almost 'defy' the laws of nature:-) While not being an expert in this field, from what I read a few years ago, it seems to me that some of the technology just works somehow(TM) which doesnt give me a great sense of security ...

As mentioned, the discs arent the only issue. Actually, in the past few years, we had more problems of esp. silent data corruption caused by faulty cables and scsi cards (not only hw wise but also due to bad firmware).

Currently we solve the issue with userspace scripts and utilities (naturally applies only for situations where performance is not needed). We dont even rely on hw/sw raid due to bad experiences but distribute copies of files over dfferent discs from different vendors via rsync and perform regular and on write hashing against a database.

Souds like you're looking for http://en.wikipedia.org/wiki/Btrfs once it's mature.

Yep, silent data corruption scares me a LOT too. I have a fairly large (2TB) setup, running software raid5.

There is some sort of checking similar to ZFS's scrub built in. You do echo "check" > /sys/block/md0/md/sync_action to trigger it for md0. What it does is go through and verify the parity against the actual bits on disk. If it finds a problem, it tries to write the correct data back to the disk that had an error.

So, there is a somewhat similar thing on Linux, but I'm waiting for some really reliable filesystems.

Unfortunately, one of the disks in my array has a problem, and silently corrupts a few megabyte of data per week. The scrub corrects it (for now) but it's left me uneasy for the past few weeks.

chithanh,

Gentoo/FreeBSD is at only at 6.2 currently. ZFS is in 7.0... So yes, maybe soon, but not yet.

-Joe

Due to FreeBSD including ZFS, you can have it on Gentoo today! Just install Gentoo/FreeBSD.

They're not just rumours ... OS X 10.5 includes basic support for ZFS, and Apple is committed to improving that support in future releases.

From Kernel Planet, it might interest you: http://valhenson.livejournal.com/9540.html

Shweet. :)

FreeBSD rocks, and the free bsd ports system rocks harder !
Thanks for helping the freebsd community man !
Z.

>>cursor movement is kind of wacked at the >>very end of a command line

I've seen that! Sometimes when I have commands that are more than one line long, weird things happen when you are editing or adding to it. Since I do a lot of things with scp/ssh, I see this relatively often. I haven't seen that it is caused by any specific thing (and I've seen it in linux, not fbsd), but if you do figure this out, I will be very happy to hear about it.

Also, your comment engine doesn't seem to like my gmail address. Odd.