Some backround first.

I needed to get Linux FW distro that supports CUPS out of the box and more specifically CUPS with Samsung ML-2510 support.

I used to run IPcop as firewall and then CUPS on Ubuntu box. So some HW consolidation was needed…

After quite extensive research I found ClarkConnect Community Edition firewall that supports out of a box CUPS. I think it is the only one?

CUPS support on ClarkConnect is not well documented and Samsung PPD file from my working Ubuntu based CUPS print server did not work at all.

After spending couple of days on this here is how to get it to work (I may have missed some steps in this writeup):

For Samsung printers you need download and build Splix on ClarkConnect (apparently my Ubuntu had these files included.):

http://splix.sourceforge.net/

SpliX is a set of CUPS printer drivers for SPL (Samsung Printer Language) printers. If you have a such printer, you need to download and use SpliX.

From Splix INSTALL file:

If you're using a distribution with a real package manager you need to
install the cups developement package (libcups...-devel or something like that).
To finish the libjbig is needed if the JBIG algorithm compilation isn't
disabled.

Summary:
* CUPS
* libcups
* libjbig (If the JBIG algorithm compilation isn't disabled)


So first disable JBIGLIB in module.mk in splix src files (I could not get to compile otherwise):


# module.mk (C) 2007-2008, Aurélien Croc (AP²C)
#
# Compilation file for SpliX
#
# Options: DISABLE_JBIG
# DISABLE_THREADS
# DISABLE_BLACKOPTIM
# Compilation option:
# V=1 Verbose mode
# DESTDIR=xxx Change the destination directory prefix

# Default options
THREADS ?= 2
CACHESIZE ?= 30
DISABLE_JBIG ?= 1 (<======= set this to one (1) to disable
DISABLE_THREADS ?= 0
DISABLE_BLACKOPTIM ?= 0


My ClarkConnect details:


[root@firewall splix-2.0.0]# lsb_release -a
LSB Version: :core-3.1-ia32:core-3.1-noarch:graphics-3.1-ia32:graphics-3.1-noarch
Distributor ID: CentOS
Description: CentOS release 5.3 (Final)
Release: 5.3
Codename: Final
[root@firewall splix-2.0.0]#


I built splix on Ubuntu box since ClarkConnect does not come with full dev tools and just tarred up splix folder and scp’ed it to my ClarkConnect box.

Before “make install” works on ClarkConnect you need to install cups-devel (make install needs cups-config):

As root run on ClarkConnect:

yum install cups-devel

My CUPS rpms:


[root@firewall cups]# rpm -qa|grep -i cups
cups-1.3.7-8.3.v5
app-cups-5.0-10
cups-devel-1.3.7-8.3.v5
cups-libs-1.3.7-8.3.v5
[root@firewall cups]#


Then “make install” as root in splix subdir on firewall.

make -n install showed here for reference:

[root@firewall splix-2.0.0]# make -n install
mkdir -p `cups-config --serverbin`/filter
echo " INSTALL optimized/rastertoqpdl"
install -m 755 -s optimized/rastertoqpdl `cups-config --serverbin`/filter
echo " INSTALL optimized/pstoqpdl"
install -m 755 -s optimized/pstoqpdl `cups-config --serverbin`/filter
make --no-print-directory -C ppd install Q=@ \
DESTDIR= DISABLE_JBIG=1
echo " INSTALL Samsung PPD files"
install -d -m 755 `cups-config --datadir`/model/samsung
for filename in clp500 clp550 clp510 clp610 ml1510 ml1520 ml1610 ml1630 ml1640 ml1710 ml1740 ml1750 ml2010 ml2150 ml2250 ml2251 ml2510 ml2550 ml3050 ml3560 scx4200 scx4500; do \
install -m 644 $filename.ppd `cups-config --datadir`/model/samsung;\
for lang in fr pt; do \
install -m 644 $filename$lang.ppd `cups-config --datadir`/model/samsung;\
done; \
done;
echo " INSTALL Xerox PPD files"
install -d -m 755 `cups-config --datadir`/model/xerox
for filename in ph3115 ph3116 ph3117 ph3120 ph3121 ph3122 ph3130 ph3150 ph3160 ph3420 ph3425 ph5500 ph6100 wc3119; do \
install -m 644 $filename.ppd `cups-config --datadir`/model/xerox;\
for lang in fr pt; do \
install -m 644 $filename$lang.ppd `cups-config --datadir`/model/xerox;\
done; \
done;
echo " INSTALL Dell PPD files"
install -d -m 755 `cups-config --datadir`/model/dell
for filename in 1100 1110; do \
install -m 644 $filename.ppd `cups-config --datadir`/model/dell;\
for lang in fr pt; do \
install -m 644 $filename$lang.ppd `cups-config --datadir`/model/dell;\
done; \
done;
echo ""
echo "PLEASE INSTALL MANUALLY COLOR PROFILE FILES (CHECK INSTALL)"
echo " --- Everything is done! Have fun ---"
echo ""
[root@firewall splix-2.0.0]#


For Samsung ML2510 make sure you have these files after splix install:


[root@firewall splix-2.0.0]# locate 2510
/model/samsung/ml2510.ppd
/model/samsung/ml2510fr.ppd
/model/samsung/ml2510pt.ppd
/usr/share/cups/model/samsung/ml2510.ppd
/usr/share/cups/model/samsung/ml2510fr.ppd
/usr/share/cups/model/samsung/ml2510pt.ppd
[root@firewall splix-2.0.0]#


In /etc/cups/mime* files make sure octet-stream is ENABLED:


[root@firewall cups]# grep octet *
mime.convs:application/octet-stream application/vnd.cups-raw 0 -
mime.types:application/octet-stream
[root@firewall cups]#


Now with CUPS web interface configure ML2510 like this:


samsung (Default Printer)
Description: upstairs Samsung ML2510
Location: upstairs
Printer Driver: Samsung ML-2510, SpliX V. 2.0.0
Printer State: idle, accepting jobs, published.
Device URI: usb://Samsung/ML-2510%20Series


Enable Windows networking (Samba) in ClarkConnect with printers support. (Steps not shown here).

You also need dummy Samba username to be able to connect to printers shared over Samba to Windows users in your home network. (I could not get CUPS IPP based printer to work on Windows clients).

In ClarkConnect Windows Networking Global settings screen set

Printing RAW (other options Disable and Point and Click)

Here are my working smb.conf and cups.conf files. They may not be optimal but they are working.

Make sure CUPS is “driving” Samba printing in smb.conf:

# Printing
printcap name = cups
#printcap name = /etc/printcap
load printers = Yes


[root@firewall samba]# cat smb.conf
[global]
# General
netbios name = some_string
workgroup = WORKGROUP
server string = SOME_STRING
# Logging
syslog = 0
log level = 8
log file = /var/log/samba/%L-%m
max log size = 0
utmp = Yes

[profiles]
comment = Profile Share
path = /var/samba/profiles
read only = No
profile acls = Yes
browseable = No
available = No
[root@firewall samba]#


Here is cups.conf:


[root@firewall samba]# cat /etc/cups/cupsd.conf
MaxLogSize 2000000000
#LogLevel debug
LogLevel info
SystemGroup sys root
# Show shared printers on the local network.
Browsing On
BrowseOrder allow,deny
# (Change '@LOCAL' to 'ALL' if using directed broadcasts from another subnet.)
BrowseAllow @LOCAL
DefaultAuthType Basic
# Allow remote access
Port 631
Include /etc/cups/cupsd.listen.conf
Include /etc/cups/cupsd.location.conf
Include /etc/cups/cupsd.policy.conf
# Allow remote administration...
# Order allow,deny
# Allow @LOCAL
Order Deny,Allow
Deny From All
Allow From 127.0.0.1
Allow From 192.168.2.0/24
# Allow remote administration...
Order allow,deny
Allow @LOCAL

root@firewall samba]#


My mythbackend kept rebooting randomly (about after 5-15 mins since starting to view recording) when viewing recordings while it was recording a new show.

Interestingly enough it did not reboot while not recording a show and when I was watching the show…

Another nasty symptom was classic NETDEV WATCHDOG: eth0: transmit timed out.
System does not crash, IP stack just goes to lala land.

I’m have Abit VT-7 motherboard (VIA PT880 / VT8237 chipset) , 478 socket P4 HT 3.0Ghz CPU and on-board eth (VIA).

root@hippo:~# lspci |grep Eth
00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 78)
root@hippo:~#


VIA eth module:

root@hippo:~# modinfo via_rhine
filename: /lib/modules/2.6.24-21-generic/kernel/drivers/net/via-rhine.ko
license: GPL
description: VIA Rhine PCI Fast Ethernet driver
author: Donald Becker
srcversion: 8158094F4B64FDC36073DA7
alias: pci:v00001106d00003053sv*sd*bc*sc*i*
alias: pci:v00001106d00003106sv*sd*bc*sc*i*
alias: pci:v00001106d00003065sv*sd*bc*sc*i*
alias: pci:v00001106d00003043sv*sd*bc*sc*i*
depends: mii
vermagic: 2.6.24-21-generic SMP mod_unload 586
parm: max_interrupt_work:VIA Rhine maximum events handled per interrupt (int)
parm: debug:VIA Rhine debug level (0-7) (int)
parm: rx_copybreak:VIA Rhine copy breakpoint for copy-only-tiny-frames (int)
parm: avoid_D3:Avoid power state D3 (work-around for broken BIOSes) (bool)

Kernel and ubuntu info:

root@hippo:~# uname -a
Linux hippo 2.6.24-21-generic #1 SMP Tue Oct 21 23:43:45 UTC 2008 i686 GNU/Linux
root@hippo:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=8.04
DISTRIB_CODENAME=hardy
DISTRIB_DESCRIPTION="Ubuntu 8.04.1"
root@hippo:~#

I tried several boot options from these links:

https://help.ubuntu.com/community/BootOptions#Common%20Boot%20Options

https://help.ubuntu.com/community/DebuggingIRQProblems

https://help.ubuntu.com/community/ReschedulingInterrupts

Finally it seems that noapic kernel option works (excerpt from /boot/grub/menu.lst):

title Ubuntu 8.04.1, kernel 2.6.24-21-generic (noapic)
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-21-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 noapic ro quiet splash
initrd /boot/initrd.img-2.6.24-21-generic

Uptime is now 3 days:

root@hippo:~# uptime
12:43:02 up 3 days, 14:26, 7 users, load average: 0.02, 0.02, 0.00
root@hippo:~#

Interestingly many IRQs are shared now (that was not the case before), but still backend is stable.

root@hippo:~# cat /proc/interrupts
CPU0 CPU1
0: 161 0 XT-PIC-XT timer
1: 1337 0 XT-PIC-XT i8042
2: 0 0 XT-PIC-XT cascade
8: 3 0 XT-PIC-XT rtc
9: 0 0 XT-PIC-XT acpi
10: 40262825 0 XT-PIC-XT ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3, eth1, ivtv1, VIA8237
11: 8216376 0 XT-PIC-XT sata_via, uhci_hcd:usb4, uhci_hcd:usb5, ivtv0
12: 112 0 XT-PIC-XT i8042
14: 1211069 0 XT-PIC-XT libata
15: 0 0 XT-PIC-XT libata
NMI: 0 0 Non-maskable interrupts
LOC: 40120197 43631526 Local timer interrupts
RES: 240304 951646 Rescheduling interrupts
CAL: 179 428 function call interrupts
TLB: 41068 48883 TLB shootdowns
TRM: 0 0 Thermal event interrupts
SPU: 0 0 Spurious interrupts
ERR: 0
MIS: 0

lspci:

root@hippo:~# lspci
00:00.0 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.1 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.2 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.3 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.4 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:00.7 Host bridge: VIA Technologies, Inc. PT880 Host Bridge
00:01.0 PCI bridge: VIA Technologies, Inc. VT8237 PCI Bridge
00:09.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
00:0c.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
00:0f.0 RAID bus controller: VIA Technologies, Inc. VIA VT6420 SATA RAID Controller (rev 80)
00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)
00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.4 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 86)
00:11.0 ISA bridge: VIA Technologies, Inc. VT8237 ISA bridge [KT600/K8T800/K8T890 South]
00:11.5 Multimedia audio controller: VIA Technologies, Inc. VT8233/A/8235/8237 AC97 Audio Controller (rev 60)
00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 78)
01:00.0 VGA compatible controller: nVidia Corporation NV34 [GeForce FX 5200] (rev a1)
root@hippo:~#


I also blacklisted ipv6 (some blog suggested it):

root@hippo:/etc/modprobe.d# tail -4 blacklist

blacklist pcspkr
root@hippo:/etc/modprobe.d#

I think there is not universal solution to this. You just need to try different command line options.
Google is your friend… Hope this post helps somebody.

I think this is it. Best screen capture utility for Windows. I have tried many. ZScreen was my choice before until I discovered Bug Shooting.

Bug Shooting really stands out from the rest of the crowd.

For some reason mythtv (mythweb) channel line up gets corrupted: same programs appear on multiple channels and channel ids are not correct.
Fix is pretty simple. As root run:

mythfilldatabase --do-channel-updates

See also mythfilldatabase in mythtv wiki.

My mythtv database got corrupted and mysqlcheck repair and check commands

mysqlcheck --repair -u mythtv -pmythtv mythconverg
and
mysqlcheck --check -u mythtv -pmythtv mythconverg

were unable to fix it.

Ironically database got corrupted when my backend server crashed and rebooted while defragmenting xfs file system that stores my recordings..

Error message from log files:

root@hippo:/var/log# grep recorded syslog
syslog:Dec 23 20:12:42 hippo mysqld[3432]: 081223 20:12:42 [ERROR] /usr/sbin/mysqld: Incorrect key file for table './mythconverg/recorded'; try to repair it


root@hippo:/var/log/mythtv# grep Incorrect mythbackend.log
mythbackend.log:Incorrect key file for table './mythconverg/recorded'; try to repair it


Also myisamchk was unable to fix recorded.MYI file:

root@hippo: cd /var/lib/mysql/mythconverg
root@hippo:/var/lib/mysql/mythconverg# myisamchk -f -r recorded.MYI
myisamchk: Unknown error 126
myisamchk: error: 'recorded.MYI' doesn't have a correct index definition. You need to recreate it before you can do a repair

So mythtv backup file to rescue…

I’m using this perl script to backup my mythtv database daily.

My crontab:

root@hippo:/var/log/mythtv# crontab -l
# m h  dom mon dow   command
#
00 4 * * * nice -19 /usr/local/bin/mythconverg_backup.pl --rotate 3
#

Backup files are here:

root@hippo:~/mytti# ls -ls
total 165152
19284 -rw-r--r-- 1 root root 19721887 2008-12-21 04:00 mythconverg-20081221040003.sql.gz
18736 -rw-r--r-- 1 root root 19160852 2008-12-22 04:00 mythconverg-20081222040003.sql.gz
18180 -rw-r--r-- 1 root root 18589893 2008-12-23 04:00 mythconverg-20081223040002.sql.gz
root@hippo:~/mytti#

So restore full myth database per these instructions

Database itself must be created first:

root@hippo:~/mytti# cat /usr/share/mythtv/sql/mc.sql
CREATE DATABASE if not exists mythconverg;
GRANT ALL ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
GRANT CREATE TEMPORARY TABLES ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
ALTER DATABASE mythconverg DEFAULT CHARACTER SET latin1;

root@hippo:~/mytti# mysql -uroot -p < /usr/share/mythtv/sql/mc.sql
Enter password:


Next restore command:

root@hippo:~/mytti# mythconverg_restore.pl --filename mythconverg-20081223040002.sql.gz

ERROR:  Unable to do a full restore. The database contains data.


It fails since database contains data. So we drop database:

root@hippo:~/mytti# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 5.0.51a-3ubuntu5.4 (Ubuntu)

mysql> quit;
Bye


This fails since no database exists:

root@hippo:~/mytti# mythconverg_restore.pl --filename mythconverg-20081223040002.sql.gz

ERROR:  The database does not exist.

Need to recreate database:

root@hippo:~/mytti# mysql -uroot -p< /usr/share/mythtv/sql/mc.sql
Enter password:

Now restore works:

root@hippo:~/mytti# mythconverg_restore.pl --filename mythconverg-20081223040002.sql.gz

Successfully restored backup.
[ OK ]
root@hippo:~/mytti# /etc/init.d/mythtv-backend start
* Starting MythTV server: mythbackend                                                             [ OK ]
root@hippo:~/mytti#

And all of my recordings and recording schedule are backup again.

“Verifying DMI Pool Data” message can appear for many reasons…

This may not be universal solution, but it worked for me.

Backround:
All of this trouble started when I inadvertently selected “Install the package maintainer’s version” option when updating Ubuntu 8.04. and my /boot/grub/menu.lst got messed up and upon next reboot got “Verifying DMI Pool Data” message and Linux system hanged.

In fact prompt about updating menu.lst is not very clear in Ubuntu…

Solution:

Rescue boot using Ubuntu 8.04 cdrom. Highlight “rescue broken system” and hit F6 and type “fb=false” and hit return.
This is a required step. Otherwise grub does not run from shell.

Keep hitting default values and select your root partition. Soon you get choice to start shell, reinstall grub or reboot system (dont remember all the choices).
Start shell and goto /boot/grub/.

Make sure menu.lst and device.map match your system config and edit if needed.

Dont panic. You may need multiple iterations to get things fixed. You can also edit boot parameters (essentially menu.lst) on the fly: upon rebooting Linux hit ESC to display grub boot menu, highlight desired kernel choice and hit ‘e’ to edit boot parameters. It is pretty handy and quick way to get (hdX,Y) line correct!

GRUB uses a special syntax for specifying disk drives which can be accessed by BIOS. This file is /boot/grub/device.map.

It is good idea to keep working backup copies of these files! Make also sure groot is correct in menu.lst file.

Finally reinstall grub and reboot.

These links helped me:
https://help.ubuntu.com/community/GrubHowto
http://ubuntuforums.org/archive/index.php/t-808339.html
http://ubuntuforums.org/showthread.php?t=151682&highlight=device.map

My device.map (sda is IDE, sdb is SATA and sdc is SATA disk):

root@hippo:/boot/grub# cat device.map
(hd0) /dev/sda
(hd1) /dev/sdb
(hd2) /dev/sdc

groot in menu.lst:

root@hippo:/boot/grub# grep groot menu.lst
## e.g. groot=(hd0,0)
## old groot=(hd2,0)
# groot=(hd0,0)

root lines in menu.lst:

root@hippo:/boot/grub# grep root menu.lst|grep -v \#
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-21-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro quiet splash
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-21-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro single
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-19-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro quiet splash
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-19-generic root=UUID=2479d97c-58e1-4958-8340-3302beab8202 ro single
root (hd0,0)
root@hippo:/boot/grub#

Mythtv 0.20 to 0.21 solution

While ago I was bit struggling with myth 0.20 on FC6 to 0.21 on Ubuntu 8.04 upgrade.

I posted this on mythtv users list

I finally figured this out.

This mythtv user post gave me a hint:

So…

mythv@hippo:~/mytti$ sudo apt-get remove --purge mythtv-database mysql-server-5.0
[sudo] password for mythtv:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libmpich1.0gf mytharchive-data libxml-perl python-mysqldb libgfortran2 fftw2
libcarp-clan-perl libxml-xql-perl libparse-yapp-perl libxml-dom-perl
libid3-3.8.3-dev python-xml libswscale1d ffmpeg zlib1g-dev libcdaudio1
libxml-simple-perl libimage-size-perl libxml-regexp-perl atomicparsley
libmyth-python perlmagick libid3-3.8.3c2a
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
mysql-server-5.0* mythtv-database*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
After this operation, 86.4MB disk space will be freed.
Do you want to continue [Y/n]?
(Reading database ... 168473 files and directories currently installed.)
Removing mysql-server-5.0 ...
* Stopping MySQL database server mysqld [ OK ]
Purging configuration files for mysql-server-5.0 ...
Removing mythtv-database ...
Purging configuration files for mythtv-database ...
mythtv@hippo:~/mytti$ sudo apt-get install mysql-server-5.0 mythtv-database mythtv-common mythtv-backend
Reading package lists... Done
Building dependency tree
Reading state information... Done
mythtv-common is already the newest version.
mythtv-common set to manually installed.
mythtv-backend is already the newest version.
The following packages were automatically installed and are no longer required:
libmpich1.0gf mytharchive-data libxml-perl python-mysqldb libgfortran2 fftw2
libcarp-clan-perl libxml-xql-perl libparse-yapp-perl libxml-dom-perl
libid3-3.8.3-dev python-xml libswscale1d ffmpeg zlib1g-dev libcdaudio1
libxml-simple-perl libimage-size-perl libxml-regexp-perl atomicparsley
libmyth-python perlmagick libid3-3.8.3c2a
Use 'apt-get autoremove' to remove them.
Suggested packages:
mysql-doc-5.0 tinyca
Recommended packages:
libhtml-template-perl mailx
The following NEW packages will be installed:
mysql-server-5.0 mythtv-database
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 0B/27.5MB of archives.
After this operation, 86.4MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously deselected package mysql-server-5.0.
(Reading database ... 166523 files and directories currently installed.)
Unpacking mysql-server-5.0 (from .../mysql-server-5.0_5.0.51a-3ubuntu5.1_i386.deb) ...
Selecting previously deselected package mythtv-database.
Unpacking mythtv-database (from .../mythtv-database_0.21.0+fixes16838-0ubuntu3.1_all.deb) ...
Setting up mysql-server-5.0 (5.0.51a-3ubuntu5.1) ...
* Stopping MySQL database server mysqld [ OK ]
Reloading AppArmor profiles : done.
* Starting MySQL database server mysqld [ OK ]
* Checking for corrupt, not cleanly closed and upgrade needing tables.

mythtv@hippo:~/mytti$ ls -ls restore.sql mythtv_backup.sql
132896 -rw-r--r-- 1 mythtv mythtv 135943023 2008-10-12 13:04 mythtv_backup.sql
107752 -rw-r--r-- 1 mythtv mythtv 110220860 2008-10-12 15:43 restore.sql


MiB and MB

1024 x 1024 = 1,048,576 bytes

1000 x 1000 = 1,000,000 bytes

To convert from MiB to MB, multiply by 1,048,576 / 1,000,000 = 1.048576

To convert from MB to MiB, divide by 1.048576.

Example:

500MB = 476.6 MiB

GiB and GB

1024 x 1024 x 1024 = 1,073,741,824 bytes

1000 x 1000 x 1000 = 1,000,000,000 bytes

To convert from GiB to GB, multiply by 1,073,741,824 / 1,000,000,000 = 1.073741824

To convert from GB to GiB, divide by 1.073741824.

Examples:

500 GB = 465.66 GiB

750 GB = 698.5 GiB

Simple conversion calculator:
http://www.cactus2000.de/uk/unit/massbyt.shtml

Goal is to create mirrored LVM partitions using fdisk and RAID. These are just my own notes… Apply with care..

This system is Ubuntu 8.04.

This RAID+LVM blog entry is adapted from

• http://help.ubuntu.com/community/Installation/RAID1+LVM

Other very useful links worth checking out:

• http://www.linuxconfig.org/Linux_Software_Raid_1_Setup

• http://www.linuxdevcenter.com/pub/a/linux/2006/04/27/managing-disk-space-with-lvm.html

• http://www.oreillynet.com/sysadmin/blog/2008/04/lvm_and_software_raid_a_powerf.html

Disks used in this test system. Note that RAID disks are not identical (different brand and size):

Primary master WD 40G sda (/ is here)
Primary slave Hitachi 164GB sdb
Secondary master Seagate 400G sdc

root@ubuntu:~# fdisk -l /dev/sda|head -n 2

Disk /dev/sda: 40.0 GB, 40020664320 bytes (primary master, root filesystem)

root@ubuntu:~# fdisk -l /dev/sdb|head -n 2 

Disk /dev/sdb: 164.6 GB, 164696555520 bytes (primary slave)
root@ubuntu:~# fdisk -l /dev/sdc|head -n 2 

Disk /dev/sdc: 400.0 GB, 400088457216 bytes (secondary master) 

root@ubuntu:~#

Install first needed lvm and mdadm packages:

# apt-get install lvm2 dmsetup mdadm

Let’s make 2×40G partitions on /dev/sdb and /dev/sdc

(fdisk commands not show here)

/dev/sdb1

/dev/sdb2

/dev/sdc1

/dev/sdc2

as linux raid autodetect partition (ID: fd):

Let’s view the partitions. Looking good…

root@ubuntu:~# fdisk -l /dev/sdb

Disk /dev/sdb: 164.6 GB, 164696555520 bytes
255 heads, 63 sectors/track, 20023 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0007634e

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        4864    39070048+  fd  Linux raid autodetect
/dev/sdb2            4865        9728    39070080   fd  Linux raid autodetect
root@ubuntu:~# fdisk -l /dev/sdc

Disk /dev/sdc: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0008633f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        4864    39070048+  fd  Linux raid autodetect
/dev/sdc2            4865        9728    39070080   fd  Linux raid autodetect
root@ubuntu:~#

Create RAID arrays:

root@ubuntu:~# mdadm --create /dev/md0 --level=1 --raid-devices=2 \
/dev/sdb1 missing
mdadm: /dev/sdb1 appears to contain an ext2fs file system
    size=159420996K  mtime=Tue Oct 14 18:40:44 2008
Continue creating array? yes
mdadm: array /dev/md0 started.
root@ubuntu:~#

root@ubuntu:~# mdadm --create /dev/md1 --level=1 --raid-devices=2 \
/dev/sdb2 missing
mdadm: array /dev/md1 started.
root@ubuntu:~#

mdstat looks like this at this point:

root@ubuntu:~# cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]

md1 : active raid1 sdb2[0]

      39070016 blocks [2/1] [U_]

md0 : active raid1 sdb1[0]

      39069952 blocks [2/1] [U_]

unused devices: <none>

root@ubuntu:~#

Add partitions to array:

root@ubuntu:~# mdadm --manage --add /dev/md0 /dev/sdc1
mdadm: added /dev/sdc1
root@ubuntu:~# mdadm --manage --add /dev/md1 /dev/sdc2
mdadm: added /dev/sdc2
root@ubuntu:~#

mdstat looks like this at this point. Note the difference.

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdc1[2] sdb1[0]
      39069952 blocks [2/1] [U_]
      [>....................]  recovery =  1.2% (505856/39069952) finish=29.2min speed=21993K/sec

unused devices: <none>
root@ubuntu:~#

We get LVM to create a nice big partition out of the first two disks:

Create physical volume:

root@ubuntu:~# pvcreate /dev/md0 /dev/md1
  Physical volume "/dev/md0" successfully created
  Physical volume "/dev/md1" successfully created
root@ubuntu:~#

Create volume group:

root@ubuntu:~# vgcreate datavg /dev/md0 /dev/md1
  Volume group "datavg" successfully created
root@ubuntu:~#

Create logical volume. This will fail:

root@ubuntu:~#  lvcreate --name datalv --size 80G datavg
  Insufficient free extents (19076) in volume group datavg: 20480 required
root@ubuntu:~#

80G is 74.5GiB.

Link to convert GB to GiB.

So this will work:

root@ubuntu:~# lvcreate --name datalv --size 74.5G datavg
  Logical volume "datalv" created
root@ubuntu:~#

Mirroring still running in the backround:

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
        resync=DELAYED

md0 : active raid1 sdc1[2] sdb1[0]
      39069952 blocks [2/1] [U_]
      [=====>...............]  recovery = 26.4% (10315264/39069952) finish=22.6min speed=21132K/sec

unused devices: <none>
root@ubuntu:~#

Useful info from the system:

root@ubuntu:~# pvscan
  PV /dev/md0   VG datavg   lvm2 [37.26 GB / 0    free]
  PV /dev/md1   VG datavg   lvm2 [37.26 GB / 16.00 MB free]
  Total: 2 [74.52 GB] / in use: 2 [74.52 GB] / in no VG: 0 [0   ]
root@ubuntu:~#

root@ubuntu:~# lvscan
  ACTIVE            '/dev/datavg/datalv' [74.50 GB] inherit
root@ubuntu:~#

root@ubuntu:~# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "datavg" using metadata type lvm2
root@ubuntu:~#

root@ubuntu:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/datavg/datalv
  VG Name                datavg
  LV UUID                ywtSGe-NYim-C8MM-D15Y-v9AZ-GyQR-6l3wUq
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                74.50 GB
  Current LE             19072
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:0

root@ubuntu:~#

root@ubuntu:~# vgdisplay
  --- Volume group ---
  VG Name               datavg
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               74.52 GB
  PE Size               4.00 MB
  Total PE              19076
  Alloc PE / Size       19072 / 74.50 GB
  Free  PE / Size       4 / 16.00 MB
  VG UUID               BPjMCp-BW7e-PgN6-6q0d-Y1QI-w4N9-4fkUKs

root@ubuntu:~#

root@ubuntu:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               datavg
  PV Size               37.26 GB / not usable 2.25 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              9538
  Free PE               0
  Allocated PE          9538
  PV UUID               6e9qx8-fMoT-3P3p-A8Bx-eE5z-RTft-EsYIvz

  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               datavg
  PV Size               37.26 GB / not usable 2.31 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              9538
  Free PE               4
  Allocated PE          9534
  PV UUID               xYyLz3-SI2z-Y0g6-K2gs-GlVv-0AcH-YcS06D

root@ubuntu:~#

LVM not yet mounted:

root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  2.8G   32G   9% /
varrun                252M   96K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M   76K  252M   1% /dev
devshm                252M     0  252M   0% /dev/shm
lrm                   252M   39M  213M  16% /lib/modules/2.6.24-21-generic/volatile
root@ubuntu:~#

Let’s create file system on /dev/datavg/datalv:

root@ubuntu:~# mkfs.ext3 -m 0 -T largefile /dev/datavg/datalv
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
76288 inodes, 19529728 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
596 block groups
32768 blocks per group, 32768 fragments per group
128 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@ubuntu:~#

And mount it:

root@ubuntu:~# mkdir /mnt/data
root@ubuntu:~# mount /dev/mapper/datavg-datalv /mnt/data/
root@ubuntu:~# chmod 777 /mnt/data/
root@ubuntu:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              36G  2.9G   31G   9% /
varrun                252M   96K  252M   1% /var/run
varlock               252M     0  252M   0% /var/lock
udev                  252M   76K  252M   1% /dev
devshm                252M     0  252M   0% /dev/shm
lrm                   252M   39M  213M  16% /lib/modules/2.6.24-21-generic/volatile
/dev/mapper/datavg-datalv
                       75G  180M   75G   1% /mnt/data
root@ubuntu:~#

Add this line to /etc/fstab to ensure mounting it on startup:

root@ubuntu:~# tail -n 1 /etc/fstab
/dev/mapper/datavg-datalv /mnt/data ext3 rw,noatime 0 0
root@ubuntu:~#

Useful status:

root@ubuntu:/mnt/data# dmsetup status
datavg-datalv: 0 78135296 linear
datavg-datalv: 78135296 78102528 linear
root@ubuntu:/mnt/data#

Mirroring still running:

root@ubuntu:/mnt/data# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[2] sdb2[0]
      39070016 blocks [2/1] [U_]
      [========>............]  recovery = 40.5% (15860736/39070016) finish=17.6min speed=21867K/sec

md0 : active raid1 sdc1[1] sdb1[0]
      39069952 blocks [2/2] [UU]

unused devices: <none>
root@ubuntu:/mnt/data#

Mirroring is complete:

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdc2[1] sdb2[0]
      39070016 blocks [2/2] [UU]

md0 : active raid1 sdc1[1] sdb1[0]
      39069952 blocks [2/2] [UU]

unused devices: <none>
root@ubuntu:~#

These are 3 super useful cross-platform solutions for everybody.

How to sync passwds with dropbox

10 ten uses for Dropbox

Secure online storage with Dropbox and Truecypt

Thoughts on Dropbox

Jungledisk ans Amazon S3 for home backups

I signed up for Amazon S3 (Jungledisk as frontend). I backed up:

Windows XP data 900MB
Linux data 2.1GB

First month Amazon S3 bill ONLY $0.85.

Highly recommended.