Software RAID on FC6

The document details the steps I had to take to build a software RAID array using Fedora Core 5 Linux. The process was very tedious and I could not find any one document that gave me enough information to complete the project. I will include all parts of the process from selecting hardware and cabling to building the array and setting it to mount automatically. I will assume the reader has a good knowledge of computer hardware and knows at least a little about SCSI and how to run Linux operating systems. This document is currently under development. Please e-mail me with specific questions.

THIS DOCUMENT IS NOT COMPLETE!! So, make sure you read the whole thing to see what's missing before you try this.

Linux commands starting with "$" can be run by any non-root user, those begining with "#" must be run as root.

Step 1 - Selecting hard drives, controllers, and cabling

You can use IDE drives and still follow these instructions perfectly. In fact, it would be much easier if you did use IDE, but SCSI is more reliable, faster, and just plain cooler. If you're interested in going IDE, I'll assume you know how to select, connect, and cable those.

Now, when selecting SCSI drives, go for whatever rotational speed and size spec you feel you need. If you're planning on going for RAID 5, you will need at least (3) drives. I'll discuss RAID types later. Once you select the size and speeds of your drives, it's time to select a connector type. Your three choices are 50-pin, 68-pin, or 80-pin SCA. I would recommend the 68 or 80-pin configuration for both speed and simplicity. I have (3) Seagate 181.8GB 7200RPM 80-pin devices.

To run your drives you will need a SCSI controller. Generally you can get a simple PCI controller, unless you have special requirements. I bought a very simple Startech.com PCI controller. It is obviously useless to get a controller with onboard RAID, as we are going to use software for the RAID array.

You will also need the proper cable to run your drives. I'd recommend you get a cable with a terminator to make controller configuration easier. Make sure there are enough or more than enough connectors for all your devices. For 50-pin devices, buy a 50-pin cable. For 68 and 80-pin devices, buy a 68-pin cable. 80-pin devices are meant to be put into a hot-swap tray, so you will also need to get an SCA adapter for each device. The SCA adapter provides all the information that a hot-swap rack would including the device ID. It also gives you the ability to hot swap drives after running a few commands.

A great place to get cabling and SCA adapters is www.stsi.com

Step 2 - Getting the Operating System to Recognize the New Drives

If you are installing FC with the drives in the system, the installation process will take care of the rest of the process for you. Simply follow the on-screen instructions and you'll be all set, stop reading now, bye bye, have fun.

If you already have FC installed, you're in for a treat. First thing to do is get your SCSI controller driver running. In my case, the driver is "initio". You'll have to look around and figure out what yours is. Once you have figured out your driver and installed it, you should start it by running:

#modprobe initio

replacing "initio" with your own driver name. Now, check to see if your drives were initialized by running:

#ls /dev/sc*

Hopefully, this will return an item for each of your new drives. If not, there was a problem, try again. Now it's time to add the driver to the kernel by running:

#new-kernel-pkg --mkinitrd --depmod --install `uname -r`

This command will update the currently running kernel with the new driver. If you use yum to upgrade your kernel, this change will apply to any newer kernel you install. If you don't believe me and want to see all the neat things that run when you install a new kernel run:

#rpm -q --scripts kernel

You might want to restart your system at this point to make sure that the proper driver has been installed in the kernel and that your devices are listed without calling modprobe.

Step 3 - Formatting the Drives Properly

Now this is where things get interesting, but not too complex. In order to make software RAID work, we need to make partitions on the devices before the array is built. So, use parted if you like command lines or gpared if you like GUIs to create a partition on each device. I believe you can use any filesystem you like, however I have only used ext3.

Step 4 - Building the Array

At this point, we are ready to build the array. This is fairly simple using mdadm, which is included in FC5. The command to build the array looks like:

#/sbin/mdadm --create --level=5 --raid-devices=3 --spare-devices=0 --name=store2 /dev/md0 /dev/sd[abc]1

You will need to chage the level item to match your desired RAID level. See the bottom of this page if you need a description of RAID levels. If you want spare devices to be automatically controlled by mdadm, note how many devices should be spares. name is the simple name of the array which can be set to anything you like; it is optional. /dev/md0 is the array device; this cannot be changed unless you have already used md0. /dev/sd[abc]1 tells mdadm to use sda1, sdb1, and sdc1 to create the array. Once you run this command, the array will be created. Check the array status by running:

#/sbin/mdadm --detail /dev/md0

The array may be listed as damaged or recreating. This is normal. To create the array for the first time mdadm marks some drives as "bad" to force creation of the array. Monitor the status of the array and wait until it is done building before moving to the next step.

Step 5 - Set Array to be Available On Boot

In order to make the array available on boot, the "FD" flag must be set on each partition. FC will detect the "FD" flags and initialize the array automatically. To set the "FD" flag simply run:

parted /dev/sda set 1 raid on

on each of your devices.

Step 6 - Create Filesystem on Array

Sorry, Haven't finished this document yet... I'll get the rest up soon!