HOW-TO setup block devices with O_DIRECT on RHEL5

Now Free shipping to

I was looking for the information on how setup and use Informix block device
s
on Red Hat Enterprise Linux 5, since that OS version discontinues raw device
s.

I have found no document that entirely fulfilled my needs, so I've investiga
ted
by myself and figured out the solution. Since it can be of interest for othe
rs
as well, I am posting my findings here - hope it can be useful.

Background - what are raw devices and block devices?

* RAW DEVICE (or CHARACTER DEVICE) = devices through which data is transmitt
ed
one character at a time, using unbuffered input and output routines; such
devices are managed by using direct access and bypassing the OS layer and
its cache; raw devices are not mounted and don't have any filesystem on them
.

* BLOCK DEVICE = devices through which data is transmitted in the form of
blocks, the most significant difference between block and character devices
is that block devices use buffered input and output routines; normally a blo
ck
device is a mounted partition with a filesystem.

With IDS 10 and any Linux kernel greater or equal to 2.6 (e.g. RHEL4 onwards
),
it is (automatically) possible to use block devices with O_DIRECT access met
hod,
which makes block devices to be used as if they were raw devices (unbuffered
access via KAIO - Kernel Asynchronous I/O). Source:
http://www-128.ibm.com/developerwor...e/dm-0503szabo/

Here is how I managed to correctly configure block devices to be used for
Informix on RHEL5:

1) If the Informix block devices are currently mounted, do 'df -k' and redir
ect
the output to a file, to create a permanent record of the correspondence
between the physical device (e.g. /dev/cciss/cXdXpX or /dev/mdX or whatever
else, depending if you are using HW or SW mirroring or nothing) and the
partition name. If they were not mounted, it is assumed you know which
physical devices you have allocated for Informix during the disks
partitioning.

2) If mounted, 'umount' the Informix block devices comment their correspondi
ng
entries in /etc/fstab to prevent them to be mounted when server is rebooted.

3) For easier usability of Informix block devices, create symbolic links und
er
/etc/udev/devices, e.g.

ln -s /dev/cciss/c1d1p8 /etc/udev/devices/online_logdbs
ln -s /dev/cciss/c1d0p2 /etc/udev/devices/online_rootdbs01
ln -s /dev/cciss/c1d1p5 /etc/udev/devices/online_rootdbs02
ln -s /dev/cciss/c1d1p9 /etc/udev/devices/online_tempdbs

the above setting makes the symbolic links (which by default will be created
under /dev once 'udev' service is [re]started) to be persistent across
reboots.

4) Set persistent ownership and permissions for the Informix physical device
s,
by adding the proper entries to /etc/udev/rules.d/50-udev.rules, e.g.:
----------------------------------------------------------
# Informix block devices configuration
KERNEL=="cciss/c1d1p8",         NAME="%k", OWNER="informix", GROUP="informix
",
MODE="0660", OPTIONS="last_rule"
KERNEL=="cciss/c1d0p2",         NAME="%k", OWNER="informix", GROUP="informix
",
MODE="0660", OPTIONS="last_rule"
KERNEL=="cciss/c1d1p5",         NAME="%k", OWNER="informix", GROUP="informix
",
MODE="0660", OPTIONS="last_rule"
KERNEL=="cciss/c1d1p9",         NAME="%k", OWNER="informix", GROUP="informix
",
MODE="0660", OPTIONS="last_rule"
# End of Informix block devices configuration
----------------------------------------------------------

5) [re]start udev (or reboot the machine)
----------------------------------------------------------
/sbin/start_udev
----------------------------------------------------------

6) check result:
----------------------------------------------------------
[root@zeus dev]# ls -la /dev/online_*
lrwxrwxrwx 1 root root 17 May  3 17:49 /dev/online_logdbs -> /dev/cciss/c1d1
p8
lrwxrwxrwx 1 root root 17 May  3 17:49 /dev/online_rootdbs01 -> /dev/cciss/c
1d0p2
lrwxrwxrwx 1 root root 17 May  3 17:49 /dev/online_rootdbs02 -> /dev/cciss/c
1d1p5
lrwxrwxrwx 1 root root 17 May  3 17:49 /dev/online_tempdbs -> /dev/cciss/c1d
1p9
[root@zeus dev]# ls -lL /dev/online_*
brw-rw---- 1 informix informix 105, 24 May  3 16:02 /dev/online_logdbs
brw-rw---- 1 informix informix 105,  2 May  3 16:02 /dev/online_rootdbs01
brw-rw---- 1 informix informix 105, 21 May  3 16:02 /dev/online_rootdbs02
brw-rw---- 1 informix informix 105, 25 May  3 16:02 /dev/online_tempdbs
----------------------------------------------------------

Note the "b" prefix for the block devices.

Should work, at least it does for me.

Cheers
Rupan3rd (from rainy Italy)

P.S. IDS 10 is not yet officially certified for RHEL5 but AFAICS it seems to
work flawlessly.