Linux Intro Wiki | Main / CreatingAFileServerUsingRAIDAndZFS

Setting up a network attached storage with zfs and RAID5

introduction

this article explains how I have set up a network attached storage box in a virtual machine as a part of the first batch of the advanced Linux course. My initial idea was to use a distribution such as FreeNas however the lack of accessibility of the FreeNas web interface precluded this option as a viable means of storage. I had already experimented with using RAID in Ubuntu thanks to a previous assignment so there was no point in me doing the same thing again for the final assignment. I shall give a few tips about FreeNas at the end of this article so that if you wish to try it, you don't make the same mistakes that I did. I had also looked at the possibility of using unraid but it requires a license and the documentation did not appear to be too friendly. I have to say that FreeNas's documentation is quite good and in spite of the screenshots, there is enough for you to be able to figure out how things work.

Considerations

I have not really considered performance as a significant factor in assembling my nas box. My goal has been to maximize data integrity and safety. My NAS box is not something I am going to use for video encoding or any other real-time work. I am going to use it for backing up data which means that it is going to be used in the wee hours of the morning. I have used the virtual machine supplied during the Linux course as a start point since I did not have any means of getting the requisite number of hard drives physically. I have modified the machine by adding a number of drives. There are now a total of 11 hard drives in the virtual machine. The first drive has been left alone since it contains the operating system but the remaining drives are used for holding the data and a single drive has been used as a spare and one for caching. Therefore, we have a RAID five array of eight drives. As of this writing, 5 TB are available to the user for storing data. Each disc is 950 GB in size. I wanted to create a file server with FreeNas's level of data integrity. I have not duplicated all the features of that operating system by any means but have taken the RAID and the zfs file system and incorporated them into my server. Be warned, according to what I have read, zfs on 32bit operating systems is unstable so if you are storing critical data, you should use a 64bit operating system.

Hang on, what does a NAS box look like?

Before we continue, it is important to understand what an NAS box looks like. I cannot comment on dedicated hardware NAS box is since I have never seen one. However, most of them allow you to store data in folders. You have your credentials as a user so when you access the NAS box, you are prompted for these credentials. Once you have entered them, you are given access to your folder. In this case, I have built a file server for myself however, this file server has NAS box like properties since it will be used for storing files. What this means is that I have not tried to set up a mechanism that caters for multiple users though doing so is easy enough.

Volumes and datasets

Unlike traditional RAID, zfs does not use the concept of volumes. You aggregate disks into a pool and you then create DataSets that are like folders in the pool. You can apply permissions and quotas on DataSets too. One configuration I have seen has DataSets mapped to individual users. If you do things this way, you may want to configure quotas and ensure that users cannot get out of their home folders.

Adding the drives

decide how many drives you want and add them using the VMware player interface. This is probably the most tedious part of the entire exercise. Once you've done that, boot the virtual machine and ensure that all the drives are visible. Your command to do that may look something like this.

 ls /dev/sd* -1 | less

Installing the required packages

Ubuntu does not have native support for zfs due to licensing issues. You need to install a few packages. More importantly, you first need to add the relevant repository to dpkg so that it is able to fetch the relevant packages. Please follow all instructions on screen while executing the steps listed below.

 add-apt-repository ppa:zfs-native/stable
 apt-get update
 apt-get install Ubuntu-zfs
	!!!Creating the disk pool

You now need to aggregate the disks into a pool. You will want to leave your system drive alone since it cannot be used for storage. You may also want to keep hot swappable drives and a caching drive handy. My command to create a pool with a spare drive and a caching drive is as follows.

 zpool create -f PrimaryPool raidz3 sdb sdc sdd sde sdf sdg sdh sdi sdj spare sdk cache sdl

Note

The -f option is a way to brute force the creation of the pool. I had to do this since zpool was referring to a drive that was not being used for anything and I wanted it allocated to the pool. Please see the zpool man page for more information.

Creating the DataSet

In this case, we are creating one big DataSet. If you want to create a multi-user configuration, then you can create one DataSet per user.

 zfs create PrimaryPool/datadump

The above command creates a DataSet called "datadump" in the PrimaryPool that we have created earlier. We need to activate duplication.

 zfs set dedup=on PrimaryPool/datadump

Mounting the DataSet

We first create the directory for mounting.

 mkdir /home/noddy/dump

We finally mount the DataSet in the user's home directory. We can mount this wherever we choose.

 zfs set mountpoint=/home/noddy/dump PrimaryPool/datadump

Appendix: my FreeNas adventure

I started by installing FreNas in a virtual machine. I did need sighted assistance for this since I was unable to find a way to install from the console and there is a point in the installer where I had to select the drive on which I wanted FreeNas installed. I was able to get until that point by using the Jaws OCR feature. Be warned, that the drive you want could be at the bottom of the screen. It may be possible to hit a series of down arrow keystrokes to get to it. I do not know if the selection wraps. The FreeNas documentation is good and explains what you will see during the installation. Once FreeNas is installed, you can access it via its ip address. This address is displayed once installation has completed or, you can see your router's logs to see which is the most recent ip address assignment. This assumes you are using DHCP. Alternatively, you can access the console that comes on boot up by using the jaws OCR feature. You need to enter a number corresponding to the option you want. The current IP address is given when you choose option 1. There is another way to use this console and monitor boot up messages. Use com0com. You can only do this once you have enabled the serial console under the advanced tab of the settings page. The navigation challenge in the FreeNas web interface is that there are menus. For example, the "accounts" link is actually a menu. You can navigate the interface using Jaws for Windows and Internet Explorer. For best results, you should disable the jaws virtual cursor when working with the menus. NVDA and Firefox or jaws and Firefox also work nicely. No special configuration changes are needed when using NVDA. Finally, the results of each action are displayed what, for a screen reader user is the top of the screen. For instance, if you enable the ssh service, then the results of that action are displayed right at the top of the page so you should hit ctrl+home and then scroll down to see what happened. It is tempting to get into the shell and to try doing things from there. Sorry, you cannot do so. You have to go through the web interface. See this thread that I started on the FreeNas forums for details.

References

ZFS on Linux Installing And Using ZFS In Linux Mint / Ubuntu – Part 5 What is ZFS? (Community Group zfs.whatis) ZFS on Linux