Posts Tagged ‘LVM’

RAID and LVM tests

October 31, 2008

Goal is to create mirrored LVM partitions using fdisk and RAID. These are just my own notes… Apply with care..

This system is Ubuntu 8.04.

This RAID+LVM blog entry is adapted from

Other very useful links worth checking out:

Disks used in this test system. Note that RAID disks are not identical (different brand and size):

Primary master WD 40G sda (/ is here)
Primary slave Hitachi 164GB sdb
Secondary master Seagate 400G sdc

root@ubuntu:~# fdisk -l /dev/sda|head -n 2

Disk /dev/sda: 40.0 GB, 40020664320 bytes (primary master, root filesystem)

root@ubuntu:~# fdisk -l /dev/sdb|head -n 2 

Disk /dev/sdb: 164.6 GB, 164696555520 bytes (primary slave)
root@ubuntu:~# fdisk -l /dev/sdc|head -n 2 

Disk /dev/sdc: 400.0 GB, 400088457216 bytes (secondary master) 

root@ubuntu:~#

Install first needed lvm and mdadm packages:

# apt-get install lvm2 dmsetup mdadm

Let’s make 2x40G partitions on /dev/sdb and /dev/sdc

(fdisk commands not show here)

/dev/sdb1

/dev/sdb2

/dev/sdc1

/dev/sdc2

as linux raid autodetect partition (ID: fd):

Let’s view the partitions. Looking good…

root@ubuntu:~# fdisk -l /dev/sdb

Disk /dev/sdb: 164.6 GB, 164696555520 bytes
255 heads, 63 sectors/track, 20023 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0007634e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        4864    39070048+  fd  Linux raid autodetect
/dev/sdb2            4865        9728    39070080   fd  Linux raid autodetect
root@ubuntu:~# fdisk -l /dev/sdc

Disk /dev/sdc: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0008633f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        4864    39070048+  fd  Linux raid autodetect
/dev/sdc2            4865        9728    39070080   fd  Linux raid autodetect
root@ubuntu:~#

Create RAID arrays:

root@ubuntu:~# mdadm --create /dev/md0 --level=1 --raid-devices=2 \
/dev/sdb1 missing
mdadm: /dev/sdb1 appears to contain an ext2fs file system
    size=159420996K  mtime=Tue Oct 14 18:40:44 2008
Continue creating array? yes
mdadm: array /dev/md0 started.
root@ubuntu:~#

root@ubuntu:~# mdadm --create /dev/md1 --level=1 --raid-devices=2 \
/dev/sdb2 missing
mdadm: array /dev/md1 started.
root@ubuntu:~#

mdstat looks like this at this point:

root@ubuntu:~# cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]

md1 : active raid1 sdb2[0]

      39070016 blocks [2/1] [U_]
md0 : active raid1 sdb1[0]

      39069952 blocks [2/1] [U_]
unused devices: <none>

root@ubuntu:~#

Add partitions to array:

root@ubuntu:~# mdadm --manage --add /dev/md0 /dev/sdc1
mdadm: added /dev/sdc1
root@ubuntu:~# mdadm --manage --add /dev/md1 /dev/sdc2
mdadm: added /dev/sdc2
root@ubuntu:~#

mdstat looks like this at this point. Note the difference.

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdc1[2] sdb1[0]
      39069952 blocks [2/1] [U_]
      [>....................]  recovery =  1.2% (505856/39069952) finish=29.2min speed=21993K/sec

unused devices: <none>
root@ubuntu:~#

We get LVM to create a nice big partition out of the first two disks:

Create physical volume:

root@ubuntu:~# pvcreate /dev/md0 /dev/md1
  Physical volume "/dev/md0" successfully created
  Physical volume "/dev/md1" successfully created
root@ubuntu:~#

Create volume group:

root@ubuntu:~# vgcreate datavg /dev/md0 /dev/md1
  Volume group "datavg" successfully created
root@ubuntu:~#

Create logical volume. This will fail:

root@ubuntu:~#  lvcreate --name datalv --size 80G datavg
  Insufficient free extents (19076) in volume group datavg: 20480 required
root@ubuntu:~#

80G is 74.5GiB.

Link to convert GB to GiB.

So this will work:

root@ubuntu:~# lvcreate --name datalv --size 74.5G datavg
  Logical volume "datalv" created
root@ubuntu:~#

Mirroring still running in the backround:

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdc1[2] sdb1[0]
      39069952 blocks [2/1] [U_]
      [=====>...............]  recovery = 26.4% (10315264/39069952) finish=22.6min speed=21132K/sec

unused devices: <none>
root@ubuntu:~#

Useful info from the system:

root@ubuntu:~# pvscan
  PV /dev/md0   VG datavg   lvm2 [37.26 GB / 0    free]
  PV /dev/md1   VG datavg   lvm2 [37.26 GB / 16.00 MB free]
  Total: 2 [74.52 GB] / in use: 2 [74.52 GB] / in no VG: 0 [0   ]
root@ubuntu:~#

root@ubuntu:~# lvscan
  ACTIVE            '/dev/datavg/datalv' [74.50 GB] inherit
root@ubuntu:~#

root@ubuntu:~# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "datavg" using metadata type lvm2
root@ubuntu:~#

root@ubuntu:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/datavg/datalv
  VG Name                datavg
  LV UUID                ywtSGe-NYim-C8MM-D15Y-v9AZ-GyQR-6l3wUq
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                74.50 GB
  Current LE             19072
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:0

root@ubuntu:~#

root@ubuntu:~# vgdisplay
  --- Volume group ---
  VG Name               datavg
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               74.52 GB
  PE Size               4.00 MB
  Total PE              19076
  Alloc PE / Size       19072 / 74.50 GB
  Free  PE / Size       4 / 16.00 MB
  VG UUID               BPjMCp-BW7e-PgN6-6q0d-Y1QI-w4N9-4fkUKs

root@ubuntu:~#

root@ubuntu:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               datavg
  PV Size               37.26 GB / not usable 2.25 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              9538
  Free PE               0
  Allocated PE          9538
  PV UUID               6e9qx8-fMoT-3P3p-A8Bx-eE5z-RTft-EsYIvz

  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               datavg
  PV Size               37.26 GB / not usable 2.31 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              9538
  Free PE               4
  Allocated PE          9534
  PV UUID               xYyLz3-SI2z-Y0g6-K2gs-GlVv-0AcH-YcS06D

root@ubuntu:~#

LVM not yet mounted:

root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  2.8G   32G   9% /
varrun                252M   96K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M   76K  252M   1% /dev
devshm                252M     0  252M   0% /dev/shm
lrm                   252M   39M  213M  16% /lib/modules/2.6.24-21-generic/volatile
root@ubuntu:~#

Let’s create file system on /dev/datavg/datalv:

root@ubuntu:~# mkfs.ext3 -m 0 -T largefile /dev/datavg/datalv
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
76288 inodes, 19529728 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
596 block groups
32768 blocks per group, 32768 fragments per group
128 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@ubuntu:~#

And mount it:

root@ubuntu:~# mkdir /mnt/data
root@ubuntu:~# mount /dev/mapper/datavg-datalv /mnt/data/
root@ubuntu:~# chmod 777 /mnt/data/
root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  2.9G   31G   9% /
varrun                252M   96K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M   76K  252M   1% /dev
devshm                252M     0  252M   0% /dev/shm
lrm                   252M   39M  213M  16% /lib/modules/2.6.24-21-generic/volatile
/dev/mapper/datavg-datalv
                       75G  180M   75G   1% /mnt/data
root@ubuntu:~#

Add this line to /etc/fstab to ensure mounting it on startup:

root@ubuntu:~# tail -n 1 /etc/fstab
/dev/mapper/datavg-datalv /mnt/data ext3 rw,noatime 0 0
root@ubuntu:~#

Useful status:

root@ubuntu:/mnt/data# dmsetup status
datavg-datalv: 0 78135296 linear
datavg-datalv: 78135296 78102528 linear
root@ubuntu:/mnt/data#

Mirroring still running:

root@ubuntu:/mnt/data# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
      [========>............]  recovery = 40.5% (15860736/39070016) finish=17.6min speed=21867K/sec

md0 : active raid1 sdc1[1] sdb1[0]
      39069952 blocks [2/2] [UU]

unused devices: <none>
root@ubuntu:/mnt/data#

Mirroring is complete:

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[1] sdb2[0]
      39070016 blocks [2/2] [UU]

md0 : active raid1 sdc1[1] sdb1[0]
      39069952 blocks [2/2] [UU]

unused devices: <none>
root@ubuntu:~#
Advertisements

Day 1: Ubuntu RAID-1 and LVM enabled filer on a USB stick with backuppc

January 9, 2008

Ok for the time being I decided to abandon freenas and openfiler (since I could not get backuppc installed in them) and install Ubuntu Gutsy Gibbon on 4GB A-Data USB stick ($20 from Fry’s).

It is super easy to install Ubuntu on USB stick. Make sure CDROM is the first boot device. Pop-in Ubuntu cdrom and reboot.

I used Ubuntu 7.10 alternate cd image (so no Ubuntu server or Ubuntu desktop cd image). This alternate has no X11 and supports LVM/RAID configs. During install I just picked /dev/sda as drive to partition and install on. It might be good idea to disconnect IDE drives during install no prevent accidentally installing on /dev/hdX drive.

No special usb modules loading andor rebuilding initrd.img tricks were required. Even though initially I thought so that I need to follow instructions outlined at http://www.pendrivelinux.com/

After reboot change boot device to USB HDD in BIOS settings and remove cd from cdrom drive. It takes a while to boot up…

Then just update Ubuntu and install swat, backuppc, samba, nfs, sshd, apache, webmin etc to help admin etc work.

In my box (Shuttle SK41G AMD Athlon XP 1800+) I have:

  • 2x160GB Hitachi Deskstars in RAID-1 array (file system: ext3 – mounted as /nasse)
  • 1x400GB Seagate (filesystem: ReiserFS – mounted as /storage)

Mount and df from box:

root@nasse:~# mount /dev/sda1 on / type ext3 (rw,errors=remount-ro)

proc on /proc type proc (rw,noexec,nosuid,nodev)

/sys on /sys type sysfs (rw,noexec,nosuid,nodev)

varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)

varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)

udev on /dev type tmpfs (rw,mode=0755)

devshm on /dev/shm type tmpfs (rw)

devpts on /dev/pts type devpts (rw,gid=5,mode=620)

lrm on /lib/modules/2.6.22-14-generic/volatile type tmpfs (rw)

/dev/hda1 on /storage type reiserfs (rw)

securityfs on /sys/kernel/security type securityfs (rw)

nfsd on /proc/fs/nfsd type nfsd (rw)

/dev/mapper/nasse-nasse_store on /nasse type ext3 (rw,noatime)
root@nasse:~#

root@nasse:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 3.5G 762M 2.6G 23% / varrun 236M 184K 236M 1% /var/run varlock 236M 0 236M 0% /var/lock udev 236M 88K 236M 1% /dev devshm 236M 0 236M 0% /dev/shm lrm 236M 34M 203M 15% /lib/modules/2.6.22-14-generic/volatile /dev/hda1 373G 35M 373G 1% /storage /dev/mapper/nasse-nasse_store 154G 2.2G 152G 2% /nasse root@nasse:~#

This link https://help.ubuntu.com/community/Installation/RAID1+LVM was super useful for me.

More on backuppc set up and config later.