ACPI/APIC/IRQ/Transmit timed out blues with Ubuntu 8.04

January 16, 2009

My mythbackend kept rebooting randomly (about after 5-15 mins since starting to view recording) when viewing recordings while it was recording a new show.

Interestingly enough it did not reboot while not recording a show and when I was watching the show…

Another nasty symptom was classic NETDEV WATCHDOG: eth0: transmit timed out.
System does not crash, IP stack just goes to lala land.

I’m have Abit VT-7 motherboard (VIA PT880 / VT8237 chipset) , 478 socket P4 HT 3.0Ghz CPU and on-board eth (VIA).

root@hippo:~# lspci |grep Eth
00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 78)
root@hippo:~#

VIA eth module:

root@hippo:~# modinfo via_rhine
filename: /lib/modules/2.6.24-21-generic/kernel/drivers/net/via-rhine.ko
license: GPL
description: VIA Rhine PCI Fast Ethernet driver
author: Donald Becker
srcversion: 8158094F4B64FDC36073DA7
alias: pci:v00001106d00003053sv*sd*bc*sc*i*
alias: pci:v00001106d00003106sv*sd*bc*sc*i*
alias: pci:v00001106d00003065sv*sd*bc*sc*i*
alias: pci:v00001106d00003043sv*sd*bc*sc*i*
depends: mii
vermagic: 2.6.24-21-generic SMP mod_unload 586
parm: max_interrupt_work:VIA Rhine maximum events handled per interrupt (int)
parm: debug:VIA Rhine debug level (0-7) (int)
parm: rx_copybreak:VIA Rhine copy breakpoint for copy-only-tiny-frames (int)
parm: avoid_D3:Avoid power state D3 (work-around for broken BIOSes) (bool)

Kernel and ubuntu info:

root@hippo:~# uname -a
Linux hippo 2.6.24-21-generic #1 SMP Tue Oct 21 23:43:45 UTC 2008 i686 GNU/Linux
root@hippo:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=8.04
DISTRIB_CODENAME=hardy
DISTRIB_DESCRIPTION="Ubuntu 8.04.1"
root@hippo:~#

I tried several boot options from these links:

https://help.ubuntu.com/community/BootOptions#Common%20Boot%20Options

https://help.ubuntu.com/community/DebuggingIRQProblems

https://help.ubuntu.com/community/ReschedulingInterrupts

Finally it seems that noapic kernel option works (excerpt from /boot/grub/menu.lst):

title Ubuntu 8.04.1, kernel 2.6.24-21-generic (noapic)
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-21-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 noapic ro quiet splash
initrd /boot/initrd.img-2.6.24-21-generic

Uptime is now 3 days:

root@hippo:~# uptime
12:43:02 up 3 days, 14:26, 7 users, load average: 0.02, 0.02, 0.00
root@hippo:~#

Interestingly many IRQs are shared now (that was not the case before), but still backend is stable.

root@hippo:~# cat /proc/interrupts
CPU0 CPU1
0: 161 0 XT-PIC-XT timer
1: 1337 0 XT-PIC-XT i8042
2: 0 0 XT-PIC-XT cascade
8: 3 0 XT-PIC-XT rtc
9: 0 0 XT-PIC-XT acpi
10: 40262825 0 XT-PIC-XT ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3, eth1, ivtv1, VIA8237
11: 8216376 0 XT-PIC-XT sata_via, uhci_hcd:usb4, uhci_hcd:usb5, ivtv0
12: 112 0 XT-PIC-XT i8042
14: 1211069 0 XT-PIC-XT libata
15: 0 0 XT-PIC-XT libata
NMI: 0 0 Non-maskable interrupts
LOC: 40120197 43631526 Local timer interrupts
RES: 240304 951646 Rescheduling interrupts
CAL: 179 428 function call interrupts
TLB: 41068 48883 TLB shootdowns
TRM: 0 0 Thermal event interrupts
SPU: 0 0 Spurious interrupts
ERR: 0
MIS: 0

lspci:

root@hippo:~# lspci
00:00.0 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.1 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.2 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.3 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.4 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.7 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:01.0 PCI bridge: VIA Technologies, Inc. VT8237 PCI Bridge
00:09.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
00:0c.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
00:0f.0 RAID bus controller: VIA Technologies, Inc. VIA VT6420 SATA RAID Controller (rev 80)
00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)
00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.4 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 86)
00:11.0 ISA bridge: VIA Technologies, Inc. VT8237 ISA bridge [KT600/K8T800/K8T890 South]
00:11.5 Multimedia audio controller: VIA Technologies, Inc. VT8233/A/8235/8237 AC97 Audio Controller (rev 60)
00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 78)
01:00.0 VGA compatible controller: nVidia Corporation NV34 [GeForce FX 5200] (rev a1)
root@hippo:~#

I also blacklisted ipv6 (some blog suggested it):

root@hippo:/etc/modprobe.d# tail -4 blacklist

blacklist ipv6

blacklist pcspkr
root@hippo:/etc/modprobe.d#

I think there is not universal solution to this. You just need to try different command line options.
Google is your friend… Hope this post helps somebody.

Bug Shooting: Best screen capture utility for Windows

January 16, 2009

I think this is it. Best screen capture utility for Windows. I have tried many. ZScreen was my choice before until I discovered Bug Shooting.

Bug Shooting really stands out from the rest of the crowd.

How to fix a corrupted mythtv channel lineup

December 30, 2008

For some reason mythtv (mythweb) channel line up gets corrupted: same programs appear on multiple channels and channel ids are not correct.
Fix is pretty simple. As root run:

mythfilldatabase --do-channel-updates

See also mythfilldatabase in mythtv wiki.

Restoring mythtv database from backup

December 24, 2008

My mythtv database got corrupted and mysqlcheck repair and check commands

mysqlcheck --repair -u mythtv -pmythtv mythconverg
and
mysqlcheck --check -u mythtv -pmythtv mythconverg

were unable to fix it.

Ironically database got corrupted when my backend server crashed and rebooted while defragmenting xfs file system that stores my recordings..

Error message from log files:

root@hippo:/var/log# grep recorded syslog
syslog:Dec 23 20:12:42 hippo mysqld[3432]: 081223 20:12:42 [ERROR] /usr/sbin/mysqld: Incorrect key file for table './mythconverg/recorded'; try to repair it


root@hippo:/var/log/mythtv# grep Incorrect mythbackend.log
mythbackend.log:Incorrect key file for table './mythconverg/recorded'; try to repair it

Also myisamchk was unable to fix recorded.MYI file:

root@hippo: cd /var/lib/mysql/mythconverg
root@hippo:/var/lib/mysql/mythconverg# myisamchk -f -r recorded.MYI
myisamchk: Unknown error 126
myisamchk: error: 'recorded.MYI' doesn't have a correct index definition. You need to recreate it before you can do a repair

So mythtv backup file to rescue…

I’m using this perl script to backup my mythtv database daily.

My crontab:

root@hippo:/var/log/mythtv# crontab -l
# m h  dom mon dow   command
#
00 4 * * * nice -19 /usr/local/bin/mythconverg_backup.pl --rotate 3
#

Backup files are here:

root@hippo:~/mytti# ls -ls
total 165152
19284 -rw-r--r-- 1 root root 19721887 2008-12-21 04:00 mythconverg-20081221040003.sql.gz
18736 -rw-r--r-- 1 root root 19160852 2008-12-22 04:00 mythconverg-20081222040003.sql.gz
18180 -rw-r--r-- 1 root root 18589893 2008-12-23 04:00 mythconverg-20081223040002.sql.gz
root@hippo:~/mytti#

So restore full myth database per these instructions

Database itself must be created first:

root@hippo:~/mytti# cat /usr/share/mythtv/sql/mc.sql
CREATE DATABASE if not exists mythconverg;
GRANT ALL ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
GRANT CREATE TEMPORARY TABLES ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
ALTER DATABASE mythconverg DEFAULT CHARACTER SET latin1;

root@hippo:~/mytti# mysql -uroot -p < /usr/share/mythtv/sql/mc.sql
Enter password:

Next restore command:

root@hippo:~/mytti# mythconverg_restore.pl --filename mythconverg-20081223040002.sql.gz

ERROR:  Unable to do a full restore. The database contains data.

It fails since database contains data. So we drop database:

root@hippo:~/mytti# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 5.0.51a-3ubuntu5.4 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysq> DROP DATABASE IF EXISTS mythconverg;
Query OK, 96 rows affected (0.05 sec)

mysql> quit;
Bye

This fails since no database exists:

root@hippo:~/mytti# mythconverg_restore.pl --filename mythconverg-20081223040002.sql.gz

ERROR:  The database does not exist.

Need to recreate database:

root@hippo:~/mytti# mysql -uroot -p< /usr/share/mythtv/sql/mc.sql
Enter password:

Now restore works:

root@hippo:~/mytti# mythconverg_restore.pl --filename mythconverg-20081223040002.sql.gz

Successfully restored backup.
[ OK ]
root@hippo:~/mytti# /etc/init.d/mythtv-backend start
* Starting MythTV server: mythbackend                                                             [ OK ]
root@hippo:~/mytti#

And all of my recordings and recording schedule are backup again.

“Verifying DMI Pool Data” solution on Linux

December 19, 2008

“Verifying DMI Pool Data” message can appear for many reasons…

This may not be universal solution, but it worked for me.

Backround:
All of this trouble started when I inadvertently selected “Install the package maintainer’s version” option when updating Ubuntu 8.04. and my /boot/grub/menu.lst got messed up and upon next reboot got “Verifying DMI Pool Data” message and Linux system hanged.

In fact prompt about updating menu.lst is not very clear in Ubuntu…

Solution:

Rescue boot using Ubuntu 8.04 cdrom. Highlight “rescue broken system” and hit F6 and type “fb=false” and hit return.
This is a required step. Otherwise grub does not run from shell.

Keep hitting default values and select your root partition. Soon you get choice to start shell, reinstall grub or reboot system (dont remember all the choices).
Start shell and goto /boot/grub/.

Make sure menu.lst and device.map match your system config and edit if needed.

Dont panic. You may need multiple iterations to get things fixed. You can also edit boot parameters (essentially menu.lst) on the fly: upon rebooting Linux hit ESC to display grub boot menu, highlight desired kernel choice and hit ‘e’ to edit boot parameters. It is pretty handy and quick way to get (hdX,Y) line correct!

GRUB uses a special syntax for specifying disk drives which can be accessed by BIOS. This file is /boot/grub/device.map.

It is good idea to keep working backup copies of these files! Make also sure groot is correct in menu.lst file.

Finally reinstall grub and reboot.

These links helped me:
https://help.ubuntu.com/community/GrubHowto
http://ubuntuforums.org/archive/index.php/t-808339.html
http://ubuntuforums.org/showthread.php?t=151682&highlight=device.map

My device.map (sda is IDE, sdb is SATA and sdc is SATA disk):

root@hippo:/boot/grub# cat device.map
(hd0) /dev/sda
(hd1) /dev/sdb
(hd2) /dev/sdc

groot in menu.lst:

root@hippo:/boot/grub# grep groot menu.lst
## e.g. groot=(hd0,0)
## old groot=(hd2,0)
# groot=(hd0,0)

root lines in menu.lst:

root@hippo:/boot/grub# grep root menu.lst|grep -v \#
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-21-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro quiet splash
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-21-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro single
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-19-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro quiet splash
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-19-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro single
root (hd0,0)
root@hippo:/boot/grub#

Mythtv 0.20 to 0.21 upgrade problem solution

November 14, 2008

Mythtv 0.20 to 0.21 solution

While ago I was bit struggling with myth 0.20 on FC6 to 0.21 on Ubuntu 8.04 upgrade.

I posted this on mythtv users list

I finally figured this out.

This mythtv user post gave me a hint:

So…

mythv@hippo:~/mytti$ sudo apt-get remove --purge mythtv-database mysql-server-5.0
[sudo] password for mythtv:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libmpich1.0gf mytharchive-data libxml-perl python-mysqldb libgfortran2 fftw2
libcarp-clan-perl libxml-xql-perl libparse-yapp-perl libxml-dom-perl
libid3-3.8.3-dev python-xml libswscale1d ffmpeg zlib1g-dev libcdaudio1
libxml-simple-perl libimage-size-perl libxml-regexp-perl atomicparsley
libmyth-python perlmagick libid3-3.8.3c2a
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
mysql-server-5.0* mythtv-database*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
After this operation, 86.4MB disk space will be freed.
Do you want to continue [Y/n]?
(Reading database ... 168473 files and directories currently installed.)
Removing mysql-server-5.0 ...
* Stopping MySQL database server mysqld [ OK ]
Purging configuration files for mysql-server-5.0 ...
Removing mythtv-database ...
Purging configuration files for mythtv-database ...
mythtv@hippo:~/mytti$ sudo apt-get install mysql-server-5.0 mythtv-database mythtv-common mythtv-backend
Reading package lists... Done
Building dependency tree
Reading state information... Done
mythtv-common is already the newest version.
mythtv-common set to manually installed.
mythtv-backend is already the newest version.
The following packages were automatically installed and are no longer required:
libmpich1.0gf mytharchive-data libxml-perl python-mysqldb libgfortran2 fftw2
libcarp-clan-perl libxml-xql-perl libparse-yapp-perl libxml-dom-perl
libid3-3.8.3-dev python-xml libswscale1d ffmpeg zlib1g-dev libcdaudio1
libxml-simple-perl libimage-size-perl libxml-regexp-perl atomicparsley
libmyth-python perlmagick libid3-3.8.3c2a
Use 'apt-get autoremove' to remove them.
Suggested packages:
mysql-doc-5.0 tinyca
Recommended packages:
libhtml-template-perl mailx
The following NEW packages will be installed:
mysql-server-5.0 mythtv-database
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 0B/27.5MB of archives.
After this operation, 86.4MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously deselected package mysql-server-5.0.
(Reading database ... 166523 files and directories currently installed.)
Unpacking mysql-server-5.0 (from .../mysql-server-5.0_5.0.51a-3ubuntu5.1_i386.deb) ...
Selecting previously deselected package mythtv-database.
Unpacking mythtv-database (from .../mythtv-database_0.21.0+fixes16838-0ubuntu3.1_all.deb) ...
Setting up mysql-server-5.0 (5.0.51a-3ubuntu5.1) ...
* Stopping MySQL database server mysqld [ OK ]
Reloading AppArmor profiles : done.
* Starting MySQL database server mysqld [ OK ]
* Checking for corrupt, not cleanly closed and upgrade needing tables.

Setting up mythtv-database (0.21.0+fixes16838-0ubuntu3.1) ...
* Starting MySQL database server mysqld [ OK ]
Failed to connect to database: Access denied for user 'root'@'localhost' (using password: NO) at -e line 5, line 1.
Failed to create database (incorrect admin username/password?)
It's also possible that mysql-server wasn't running. After install
is completed, you will need to make sure mysql-server is running
and that you supplied correct information. Try:
sudo dpkg-reconfigure mythtv-database

mythtv@hippo:~/mytti$ sudo dpkg-reconfigure mythtv-common

mythtv@hippo:~/mytti$ sudo apt-get install mysql-server-5.0 mythtv-database mythtv-common mythtv-backend

mythtv@hippo:~/mytti$ sudo dpkg-reconfigure mythtv-database
* Starting MySQL database server mysqld [ OK ]

mythtv@hippo:~/mytti$ mysql -u mythtv -pmythtv mythconverg < restore.sql

mythtv@hippo:~/mytti$

mythtv@hippo:~/mytti$ ls -ls restore.sql mythtv_backup.sql
132896 -rw-r--r-- 1 mythtv mythtv 135943023 2008-10-12 13:04 mythtv_backup.sql
107752 -rw-r--r-- 1 mythtv mythtv 110220860 2008-10-12 15:43 restore.sql

MB to MiB and GB to GiB conversions

November 7, 2008

MiB and MB

1024 x 1024 = 1,048,576 bytes

1000 x 1000 = 1,000,000 bytes

To convert from MiB to MB, multiply by 1,048,576 / 1,000,000 = 1.048576

To convert from MB to MiB, divide by 1.048576.

Example:

500MB = 476.6 MiB

GiB and GB

1024 x 1024 x 1024 = 1,073,741,824 bytes

1000 x 1000 x 1000 = 1,000,000,000 bytes

To convert from GiB to GB, multiply by 1,073,741,824 / 1,000,000,000 = 1.073741824

To convert from GB to GiB, divide by 1.073741824.

Examples:

500 GB = 465.66 GiB

750 GB = 698.5 GiB

Simple conversion calculator:
http://www.cactus2000.de/uk/unit/massbyt.shtml

RAID and LVM tests

October 31, 2008

Goal is to create mirrored LVM partitions using fdisk and RAID. These are just my own notes… Apply with care..

This system is Ubuntu 8.04.

This RAID+LVM blog entry is adapted from

Other very useful links worth checking out:

Disks used in this test system. Note that RAID disks are not identical (different brand and size):

Primary master WD 40G sda (/ is here)
Primary slave Hitachi 164GB sdb
Secondary master Seagate 400G sdc

root@ubuntu:~# fdisk -l /dev/sda|head -n 2

Disk /dev/sda: 40.0 GB, 40020664320 bytes (primary master, root filesystem)

root@ubuntu:~# fdisk -l /dev/sdb|head -n 2 

Disk /dev/sdb: 164.6 GB, 164696555520 bytes (primary slave)
root@ubuntu:~# fdisk -l /dev/sdc|head -n 2 

Disk /dev/sdc: 400.0 GB, 400088457216 bytes (secondary master) 

root@ubuntu:~#

Install first needed lvm and mdadm packages:

# apt-get install lvm2 dmsetup mdadm

Let’s make 2x40G partitions on /dev/sdb and /dev/sdc

(fdisk commands not show here)

/dev/sdb1

/dev/sdb2

/dev/sdc1

/dev/sdc2

as linux raid autodetect partition (ID: fd):

Let’s view the partitions. Looking good…

root@ubuntu:~# fdisk -l /dev/sdb

Disk /dev/sdb: 164.6 GB, 164696555520 bytes
255 heads, 63 sectors/track, 20023 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0007634e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        4864    39070048+  fd  Linux raid autodetect
/dev/sdb2            4865        9728    39070080   fd  Linux raid autodetect
root@ubuntu:~# fdisk -l /dev/sdc

Disk /dev/sdc: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0008633f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        4864    39070048+  fd  Linux raid autodetect
/dev/sdc2            4865        9728    39070080   fd  Linux raid autodetect
root@ubuntu:~#

Create RAID arrays:

root@ubuntu:~# mdadm --create /dev/md0 --level=1 --raid-devices=2 \
/dev/sdb1 missing
mdadm: /dev/sdb1 appears to contain an ext2fs file system
    size=159420996K  mtime=Tue Oct 14 18:40:44 2008
Continue creating array? yes
mdadm: array /dev/md0 started.
root@ubuntu:~#

root@ubuntu:~# mdadm --create /dev/md1 --level=1 --raid-devices=2 \
/dev/sdb2 missing
mdadm: array /dev/md1 started.
root@ubuntu:~#

mdstat looks like this at this point:

root@ubuntu:~# cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]

md1 : active raid1 sdb2[0]

      39070016 blocks [2/1] [U_]
md0 : active raid1 sdb1[0]

      39069952 blocks [2/1] [U_]
unused devices: <none>

root@ubuntu:~#

Add partitions to array:

root@ubuntu:~# mdadm --manage --add /dev/md0 /dev/sdc1
mdadm: added /dev/sdc1
root@ubuntu:~# mdadm --manage --add /dev/md1 /dev/sdc2
mdadm: added /dev/sdc2
root@ubuntu:~#

mdstat looks like this at this point. Note the difference.

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdc1[2] sdb1[0]
      39069952 blocks [2/1] [U_]
      [>....................]  recovery =  1.2% (505856/39069952) finish=29.2min speed=21993K/sec

unused devices: <none>
root@ubuntu:~#

We get LVM to create a nice big partition out of the first two disks:

Create physical volume:

root@ubuntu:~# pvcreate /dev/md0 /dev/md1
  Physical volume "/dev/md0" successfully created
  Physical volume "/dev/md1" successfully created
root@ubuntu:~#

Create volume group:

root@ubuntu:~# vgcreate datavg /dev/md0 /dev/md1
  Volume group "datavg" successfully created
root@ubuntu:~#

Create logical volume. This will fail:

root@ubuntu:~#  lvcreate --name datalv --size 80G datavg
  Insufficient free extents (19076) in volume group datavg: 20480 required
root@ubuntu:~#

80G is 74.5GiB.

Link to convert GB to GiB.

So this will work:

root@ubuntu:~# lvcreate --name datalv --size 74.5G datavg
  Logical volume "datalv" created
root@ubuntu:~#

Mirroring still running in the backround:

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdc1[2] sdb1[0]
      39069952 blocks [2/1] [U_]
      [=====>...............]  recovery = 26.4% (10315264/39069952) finish=22.6min speed=21132K/sec

unused devices: <none>
root@ubuntu:~#

Useful info from the system:

root@ubuntu:~# pvscan
  PV /dev/md0   VG datavg   lvm2 [37.26 GB / 0    free]
  PV /dev/md1   VG datavg   lvm2 [37.26 GB / 16.00 MB free]
  Total: 2 [74.52 GB] / in use: 2 [74.52 GB] / in no VG: 0 [0   ]
root@ubuntu:~#

root@ubuntu:~# lvscan
  ACTIVE            '/dev/datavg/datalv' [74.50 GB] inherit
root@ubuntu:~#

root@ubuntu:~# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "datavg" using metadata type lvm2
root@ubuntu:~#

root@ubuntu:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/datavg/datalv
  VG Name                datavg
  LV UUID                ywtSGe-NYim-C8MM-D15Y-v9AZ-GyQR-6l3wUq
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                74.50 GB
  Current LE             19072
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:0

root@ubuntu:~#

root@ubuntu:~# vgdisplay
  --- Volume group ---
  VG Name               datavg
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               74.52 GB
  PE Size               4.00 MB
  Total PE              19076
  Alloc PE / Size       19072 / 74.50 GB
  Free  PE / Size       4 / 16.00 MB
  VG UUID               BPjMCp-BW7e-PgN6-6q0d-Y1QI-w4N9-4fkUKs

root@ubuntu:~#

root@ubuntu:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               datavg
  PV Size               37.26 GB / not usable 2.25 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              9538
  Free PE               0
  Allocated PE          9538
  PV UUID               6e9qx8-fMoT-3P3p-A8Bx-eE5z-RTft-EsYIvz

  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               datavg
  PV Size               37.26 GB / not usable 2.31 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              9538
  Free PE               4
  Allocated PE          9534
  PV UUID               xYyLz3-SI2z-Y0g6-K2gs-GlVv-0AcH-YcS06D

root@ubuntu:~#

LVM not yet mounted:

root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  2.8G   32G   9% /
varrun                252M   96K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M   76K  252M   1% /dev
devshm                252M     0  252M   0% /dev/shm
lrm                   252M   39M  213M  16% /lib/modules/2.6.24-21-generic/volatile
root@ubuntu:~#

Let’s create file system on /dev/datavg/datalv:

root@ubuntu:~# mkfs.ext3 -m 0 -T largefile /dev/datavg/datalv
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
76288 inodes, 19529728 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
596 block groups
32768 blocks per group, 32768 fragments per group
128 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@ubuntu:~#

And mount it:

root@ubuntu:~# mkdir /mnt/data
root@ubuntu:~# mount /dev/mapper/datavg-datalv /mnt/data/
root@ubuntu:~# chmod 777 /mnt/data/
root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  2.9G   31G   9% /
varrun                252M   96K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M   76K  252M   1% /dev
devshm                252M     0  252M   0% /dev/shm
lrm                   252M   39M  213M  16% /lib/modules/2.6.24-21-generic/volatile
/dev/mapper/datavg-datalv
                       75G  180M   75G   1% /mnt/data
root@ubuntu:~#

Add this line to /etc/fstab to ensure mounting it on startup:

root@ubuntu:~# tail -n 1 /etc/fstab
/dev/mapper/datavg-datalv /mnt/data ext3 rw,noatime 0 0
root@ubuntu:~#

Useful status:

root@ubuntu:/mnt/data# dmsetup status
datavg-datalv: 0 78135296 linear
datavg-datalv: 78135296 78102528 linear
root@ubuntu:/mnt/data#

Mirroring still running:

root@ubuntu:/mnt/data# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
      [========>............]  recovery = 40.5% (15860736/39070016) finish=17.6min speed=21867K/sec

md0 : active raid1 sdc1[1] sdb1[0]
      39069952 blocks [2/2] [UU]

unused devices: <none>
root@ubuntu:/mnt/data#

Mirroring is complete:

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[1] sdb2[0]
      39070016 blocks [2/2] [UU]

md0 : active raid1 sdc1[1] sdb1[0]
      39069952 blocks [2/2] [UU]

unused devices: <none>
root@ubuntu:~#

Dropbox, Jungledisk and Truecrypt

October 8, 2008

These are 3 super useful cross-platform solutions for everybody.

How to sync passwds with dropbox

10 ten uses for Dropbox

Secure online storage with Dropbox and Truecypt

Thoughts on Dropbox

Jungledisk ans Amazon S3 for home backups

I signed up for Amazon S3 (Jungledisk as frontend). I backed up:

Windows XP data 900MB
Linux data 2.1GB

First month Amazon S3 bill ONLY $0.85.

Highly recommended.

deleting files with rsync

October 1, 2008

To delete files that are on the destination system but no longer on the source system, use the –delete option:

[root@server:~/tmp]# ls -ls
total 0
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 1
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 2
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 3
[root@server:~/tmp]# ls -ls /tmp/b
total 0
[root@server:~/tmp]#  rsync -azv  /root/tmp/ /tmp/b
sending incremental file list
./
1
2
3

sent 163 bytes  received 72 bytes  470.00 bytes/sec
total size is 0  speedup is 0.00
[root@server:~/tmp]# ls -ls /tmp/b
total 0
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 1
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 2
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 3
[root@server:~/tmp]# rm 2
[root@server:~/tmp]# ls -ls
total 0
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 1
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 3
[root@server:~/tmp]# ls -ls /tmp/b
total 0
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 1
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 2
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 3
[root@server:~/tmp]#  rsync -azv  --delete /root/tmp/ /tmp/b
sending incremental file list
./
deleting 2

sent 53 bytes  received 15 bytes  136.00 bytes/sec
total size is 0  speedup is 0.00
[root@server:~/tmp]# ls -ls /tmp/b
total 0
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 1
0 -rw-r--r-- 1 root root 0 2008-10-01 11:18 3
[root@server:~/tmp]#