Freitag, 19. September 2008

ZFS Root Mirror

This is a simple Tutorial how you can create a bootable ZFS Root Mirror with Opensolaris. I had some help from both Opensolaris-Forums (com/org) and this Blog from Malachi.

And here we go with the Tutorial:

Making a bootable ZFS Root Mirror

1. Install Opensolaris to Disk A (c3d0s0).

2. Format Disk B (c3d1s0) properly:
host:# format
(choose fdisk)
(create 100% Standard Solaris Partition over the full Disk)

3. Overwrite the Diskformat properly:
host:# prtvtoc /dev/rdsk/c3d0s2 | fmthard -s - /dev/rdsk/c3d1s2
(NOTE: s2! on BOTH Disks)

4. Attach Disk B to the ZFS Root Pool:
host:# zpool attach -f rpool c3d0s0 c3d1s0

5. Install the GRUB-Stuff to Disk B:
host:# installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c3d1s0

6. Test the worst Case: Disk A fails, you can still boot into Disk B
- Remove Disk A
- Replace it with an emtpy Disk C
- Start the System: Choose to boot from Disk B in the Bios
- Repeat Steps 2. till 5. for Disk C (swap c3d0s0 and c3d1s0)
- Start the System like before (from Disk C)

10 Kommentare:

Malachi de AElfweald hat gesagt…

Nice and concise!

In theory, you should be able to hotswap those drives rather than requiring reboot, as well.

Thanks for the link =)

Anonym hat gesagt…

This saved me installing OpenSolaris nv_101 rc1 to a t4540 server. Only the disk labels should read c4t0doso, the t0 is ommited in your post. Can you elaborate on why this works this way ? Confused by the prtvtoc part...

Guido Luetke Woestmann hat gesagt…

Please also read this Blog:

http://malsserver.blogspot.com/2008/08/mirroring-resolved-correct-way.html

It provides some more Information. I am not sure about the "t0"-thing. I tested this under VirtualBox with virtual Sata-Drives.
The prtvtoc-Part prepares the second Disks Partition-Table. If you just add the second Disk to the Mirror, it has the wrong Partition-Layout for a boot-Disk. There are some Bugs related to this (see 6668666).

HTH

Guido Luetke Woestmann hat gesagt…

Here is another Discussion related to this Topic:

http://www.opensolaris.org/jive/thread.jspa?threadID=83942&tstart=0

Anonym hat gesagt…

you should be able to hotswap the zfs volumes on many sata/scsi controllers.

you may need to reconfigure the disks.

if you hotplug your disks and cant see the new entry in /dev/dsk, just use cfgadm to see if any connected devices are in an unconfigured state. If they are, then do a cfgadm -c configure sata4/4 or whatever your devices is. zfs should see a re-inserted disk automatically but if you replaced a disk you may need to manually online the disk.

Anonym hat gesagt…

Tried to build a bootable zfs mirror with your howto, however I ran into some problems rebuilding it after a member loss.
Loosing disk1 rebuilt fine, but loosing disk0 hanged at boot, and I needed to import the root zpool with the livecd and fix it there.

Guido Luetke Woestmann hat gesagt…

If you want to boot from the second Disk, you have to choose/enable it in the BIOS first. Otherwise the Machine still boots from the (missing) first Disk.
NOTE: I tried this with 2008.05 in Virtualbox, maybe I need to Update this Tutorial ;-)

Anonym hat gesagt…

2008.11 on vmware:
I encountered an issue where fdisk when taking 100% of the disk for solaris would allocate one cylinder less.

c4t0d0:
1 Active Solaris2 1 4095 4095 100

(c4t1d0 is the exact same size as c4t0d0)

format c4t1d0
format> fdisk
No fdisk table exists. The default partition for the disk is:

a 100% "SOLARIS System" partition

Type "y" to accept the default partition, otherwise type "n" to edit the
partition table.
y
format> fdisk
Total disk size is 4096 cylinders
Cylinder size is 4096 (512 byte) blocks

Cylinders
Partition Status Type Start End Length %
========= ====== ============ ===== === ====== ===
1 Active Solaris2 1 4094 4094 100

you need to manually make sure that the partition size is 4095.

Unknown hat gesagt…

We just tested this on a "real", non-virtual server and had problems using "installgrub -m" to install the grub files into the MBR. After using it without "-m" on the zpool partition s0, everything worked as expected, but the mirror did not show any faults when pulling the original disk.

Pulling out the mirrored disk works as expected, showing the mirror in degraded state with one disk missing.

We'll try installing the latest recommended patchcluster and see how it works out...

Anonym hat gesagt…

Short and sweet, and still a nice how-to for OpenIndiana. Cheers!