Blog

Frequently asked questions on RAID-Z

We take a look at the ZFS and RAID-Z, the file system and RAID tool for Solaris and its derivatives.

What is RAID-Z?
RAID-Z is a non-standard form of RAID (Redundant Array of Independent Disks) which was developed by Sun Microsystems for the company's Zettabyte File System (ZFS), which ships as the default filesystem in the Solaris operating system.

What does it do, in simple terms?
It's a redundant file system that uses ZFS, so multiple disks are used and data is written to more than one disk. If a disk fails, the data should still be safe, as it's been written to another drive too. It's most similar to RAID 5, but actually has a number of advantages. In the RAID world, it's considered one of the most sophisticated solutions available. And, because of the way ZFS works, if you pull the plug on your machine, the data is safe from corruption.

What operating systems can use RAID-Z?
ZFS, and therefore RAID-Z, are included in Solaris by default, as well as in the open source OpenSolaris. They are also available as an add-on in FreeBSD, and can be installed, with some effort, on Linux via Fuse. The ZFS licence clashes with the GPL licence used by Linux, so at present, ZFS can't be distributed with Linux, but it is possible for experienced users to install it themselves.

On Solaris and its derivitatives, you can set up RAID-Z by creating a special storage pool called a raidz pool.

What is a storage pool?
Storage pools are the basis of ZFS. They're effectively "virtualised volumes" which aggregate (or pool) all of the disk drives into a single volume. With traditional file systems, you're restricted to the space on a specific disk or partition. However, with ZFS, all of the storage is accessible, so if you run out of space, you can add another disk and make it available across the storage pool. Basically, a storage pool allows you to easily expand the size of your storage as your storage requirements change.

So ZFS is pretty scalable, then?
Seriously scalable. It's not called the Zettabyte Filesystem for nothing. According to Sun, it's a 128-bit filesystem, which means it can support up to 256 quadrillion zettabytes of storage.

A zettabyte is over a billion terabytes, so considering that the largest currently available single disk is 1.5TB, you're not going to exceed these storage capabilities in a hurry, even if you own a massive storage facility. No other filesystem comes close to the scalability of ZFS, as far as I'm aware. The fact that you can mirror stuff, and grow the size of the storage pool, make ZFS and RAID-Z an excellent choice for fileservers and NAS machines.

Can you create a mirrored storage pool without RAID-Z?
Yes, this is also really easy. Just install a couple of disks, one with the OS on and one as a spare, and then enter the following command (where c1t0d0 and c2t0d0 are the names of the disks):

$ zpool create mirroredpool mirror c1t0d0 c2t0d0

This creates a mirrored pool called "mirroredpool" (you can obviously call it whatever you like) which is mounted at /mirroredpool. You can then make a filesystem within the mirrored pool using the command:

$ zpool create mirroredpool/fs

Sun's website contains a really good guide to setting up a mirrored pool, which includes full details of all of the commands required.

How do I find out the labels for the disks on my machine?
As the root user, type format and hit return. This will list the available disks in the machine. Simply list these after the desired zpool command to add them to your storage pool.

Are there different types of RAID-Z?
Yes, there's standard RAID-Z (raidz), which creates a simple mirrored pool; there's RAID-Z1 (raidz1), which creates a striped pool with greater performance but no parity, and there's RAID-Z2 (raidz2), which combines mirroring and striping for combined performance boost and redundancy.

Creating a RAID-Z mirrored pool, or a RAID-Z1 striped pool is as easy as creating a normal ZFS storage pool. Again, it's just a simple one-liner command. Check out the Sun guide to creating RAID-Z pools for the full details.

How do you set it up?
A simple one line command (entered as the superuser) is all that's needed to create the disk pool, set it to raidz and add the desired disks to it. If you have three disks, labeled c1t0d0, c2t0d0 and c3t0d0, you can set up a simple RAID-Z with the following command:

$ raidz c1t0d0 c2t0d0 c3t0d0

Is any special hardware required?
No. You don't use hardware RAID cards with RAID-Z, nor do you need to use BIOS RAID or SATA RAID controllers. All that is needed is a collection of drives installed in the machine. There's no need for an expensive hardware RAID card, or a BIOS-based software- or FakeRAID controller, as everything is handled by the RAID-Z software that comes with ZFS.

Can the drives be different sizes and makes?
Yes, you can do this using the -f flag. You can add as many disks as you have connectors for.

Can RAID-Z do striping and mirroring?
Yes, RAID-Z can be set up very easily to mirror data (adding redundancy and protection against disk failure), striping (in which you get a performance boost but no protection against disk failure), or both striping and redundancy. The latter requires more disks and a slightly different configuration.

What else can it do?
One particularly cool feature is "self-healing". This feature is only available with RAID-Z in mirror configuration. Basically, when a bad block is detected on the drive, ZFS gets the correct data from another redundant copy on the RAID, then repairs the bad block by replacing it with another copy.

What is performance like?
Generally, very good indeed. According to this comparison of Solaris RAID-Z1 on ZFS to RAID-5 on Ubuntu, RAID-Z is no worse than RAID-5 using the Linux software RAID solution mdadm. In some cases it's quicker, and I believe that Sun has claimed that it's quicker than hardware RAID in some cases.

What happens if I need to replace a disk?
If a disk fails, you can replace it and rebuild the storage pool. If you're using a mirrored pool, the lost data gets backed up to the new disk when the RAID is rebuilt. Again, it's a single command.

Can I increase the size of the RAID if I need more storage?
Yes. You can add new disks to the storage pool to increase the size. The space available expands. [Update: Nope, I got this wrong. You can update a normal ZFS pool and expand it, but you can't expand a RAIDZ pool. Apparently this is a limitation of ZFS. The workaround, apparently, is to add a group of devices to your pool.]

What else does ZFS do?
Well, for a start, snapshots are built-in. A snapshot is a read-only copy of a filesystem or volume. OpenSolaris includes a great graphical tool called Time Slider which takes advantage of these snapshots. This can be set to take snapshots at chosen time intervals and it allows you to flick back through time to see previously changed files on your system.

Time Slider

I'm unaware of anything similar for Linux or Windows, but it's really quite similar in overall functionality to the Time Machine feature of Apple's Mac OS X Leopard. However, whereas Time Machine snapshots take up considerable amounts of storage space, ZFS snapshots do not. At least, not initially.

If the files do not change, then the snapshot does not add to the size of the storage pool. However, when files are changed, the changed files are stored and do take up some space.

How about quotas?
Yes, it's also really simple to set up disk quotas for your users, making ZFS a great choice for file and web servers, where you'd want to ensure that users didn't use up the lion's share of the disk space.

How do I check the health status of my zpool?
This is as easy as opening a terminal and entering:

$ zpool status

You'll then get some results which show the pool name, its current state, and the names of the disks it includes, as well as whether it currently contains any errors.