Informix - HOW-TO setup block devices with O

Author

HOW-TO setup block devices with O_DIRECT on RHEL5

Rupan3rd

2007-05-04, 7:16 am

I was looking for the information on how setup and use Informix block devices
on Red Hat Enterprise Linux 5, since that OS version discontinues raw devices.

I have found no document that entirely fulfilled my needs, so I've investigated
by myself and figured out the solution. Since it can be of interest for others
as well, I am posting my findings here - hope it can be useful.

Background - what are raw devices and block devices?

* RAW DEVICE (or CHARACTER DEVICE) = devices through which data is transmitted
one character at a time, using unbuffered input and output routines; such
devices are managed by using direct access and bypassing the OS layer and
its cache; raw devices are not mounted and don't have any filesystem on them.

* BLOCK DEVICE = devices through which data is transmitted in the form of
blocks, the most significant difference between block and character devices
is that block devices use buffered input and output routines; normally a block
device is a mounted partition with a filesystem.

With IDS 10 and any Linux kernel greater or equal to 2.6 (e.g. RHEL4 onwards),
it is (automatically) possible to use block devices with O_DIRECT access method,
which makes block devices to be used as if they were raw devices (unbuffered
access via KAIO - Kernel Asynchronous I/O). Source:
http://www-128.ibm.com/developerwor...e/dm-0503szabo/

Here is how I managed to correctly configure block devices to be used for
Informix on RHEL5:

1) If the Informix block devices are currently mounted, do 'df -k' and redirect
the output to a file, to create a permanent record of the correspondence
between the physical device (e.g. /dev/cciss/cXdXpX or /dev/mdX or whatever
else, depending if you are using HW or SW mirroring or nothing) and the
partition name. If they were not mounted, it is assumed you know which
physical devices you have allocated for Informix during the disks
partitioning.

2) If mounted, 'umount' the Informix block devices comment their corresponding
entries in /etc/fstab to prevent them to be mounted when server is rebooted.

3) For easier usability of Informix block devices, create symbolic links under
/etc/udev/devices, e.g.

ln -s /dev/cciss/c1d1p8 /etc/udev/devices/online_logdbs
ln -s /dev/cciss/c1d0p2 /etc/udev/devices/online_rootdbs01
ln -s /dev/cciss/c1d1p5 /etc/udev/devices/online_rootdbs02
ln -s /dev/cciss/c1d1p9 /etc/udev/devices/online_tempdbs

the above setting makes the symbolic links (which by default will be created
under /dev once 'udev' service is [re]started) to be persistent across
reboots.

4) Set persistent ownership and permissions for the Informix physical devices,
by adding the proper entries to /etc/udev/rules.d/50-udev.rules, e.g.:
----------------------------------------------------------
# Informix block devices configuration
KERNEL=="cciss/c1d1p8", NAME="%k", OWNER="informix", GROUP="informix",
MODE="0660", OPTIONS="last_rule"
KERNEL=="cciss/c1d0p2", NAME="%k", OWNER="informix", GROUP="informix",
MODE="0660", OPTIONS="last_rule"
KERNEL=="cciss/c1d1p5", NAME="%k", OWNER="informix", GROUP="informix",
MODE="0660", OPTIONS="last_rule"
KERNEL=="cciss/c1d1p9", NAME="%k", OWNER="informix", GROUP="informix",
MODE="0660", OPTIONS="last_rule"
# End of Informix block devices configuration
----------------------------------------------------------

5) [re]start udev (or reboot the machine)
----------------------------------------------------------
/sbin/start_udev
----------------------------------------------------------

6) check result:
----------------------------------------------------------
[root@zeus dev]# ls -la /dev/online_*
lrwxrwxrwx 1 root root 17 May 3 17:49 /dev/online_logdbs -> /dev/cciss/c1d1p8
lrwxrwxrwx 1 root root 17 May 3 17:49 /dev/online_rootdbs01 -> /dev/cciss/c1d0p2
lrwxrwxrwx 1 root root 17 May 3 17:49 /dev/online_rootdbs02 -> /dev/cciss/c1d1p5
lrwxrwxrwx 1 root root 17 May 3 17:49 /dev/online_tempdbs -> /dev/cciss/c1d1p9
[root@zeus dev]# ls -lL /dev/online_*
brw-rw---- 1 informix informix 105, 24 May 3 16:02 /dev/online_logdbs
brw-rw---- 1 informix informix 105, 2 May 3 16:02 /dev/online_rootdbs01
brw-rw---- 1 informix informix 105, 21 May 3 16:02 /dev/online_rootdbs02
brw-rw---- 1 informix informix 105, 25 May 3 16:02 /dev/online_tempdbs
----------------------------------------------------------

Note the "b" prefix for the block devices.

Should work, at least it does for me.

Cheers
Rupan3rd (from rainy Italy)

P.S. IDS 10 is not yet officially certified for RHEL5 but AFAICS it seems to
work flawlessly.

Mike Badar

2007-05-04, 1:15 pm

Great write-up. I always appreciate it when someone submits summaries
like yours for the benefit of the group. Thanks.

Someday I'm going to Italy; rain or shine!

Mike Badar
ESRI
ArcSDE for Informix UNIX administrator
1 International Ct.
Broomfield, CO 80021
mbadar@esri.com

> -----Original Message-----
> From: informix-list-bounces@iiug.org=20
> [mailto:informix-list-bounces@iiug.org] On Behalf Of Rupan3rd
> Sent: Friday, May 04, 2007 5:03 AM
> To: informix-list@iiug.org
> Subject: HOW-TO setup block devices with O_DIRECT on RHEL5
>=20
> I was looking for the information on how setup and use=20
> Informix block devices on Red Hat Enterprise Linux 5, since=20
> that OS version discontinues raw devices.
>=20
> I have found no document that entirely fulfilled my needs, so=20
> I've investigated by myself and figured out the solution.=20
> Since it can be of interest for others as well, I am posting=20
> my findings here - hope it can be useful.
>=20
> Background - what are raw devices and block devices?
>=20
> * RAW DEVICE (or CHARACTER DEVICE) =3D devices through which=20
> data is transmitted
> one character at a time, using unbuffered input and output=20
> routines; such
> devices are managed by using direct access and bypassing=20
> the OS layer and
> its cache; raw devices are not mounted and don't have any=20
> filesystem on them.
>=20
> * BLOCK DEVICE =3D devices through which data is transmitted in=20
> the form of
> blocks, the most significant difference between block and=20
> character devices
> is that block devices use buffered input and output=20
> routines; normally a block
> device is a mounted partition with a filesystem.
>=20
> With IDS 10 and any Linux kernel greater or equal to 2.6=20
> (e.g. RHEL4 onwards), it is (automatically) possible to use=20
> block devices with O_DIRECT access method, which makes block=20
> devices to be used as if they were raw devices (unbuffered=20
> access via KAIO - Kernel Asynchronous I/O). Source:=20
> http://www-128.ibm.com/developerwor...ry/techarticle/
> dm-0503szabo/
>=20
>=20
> Here is how I managed to correctly configure block devices to=20
> be used for Informix on RHEL5:
>=20
> 1) If the Informix block devices are currently mounted, do=20
> 'df -k' and redirect
> the output to a file, to create a permanent record of the=20
> correspondence
> between the physical device (e.g. /dev/cciss/cXdXpX or=20
> /dev/mdX or whatever
> else, depending if you are using HW or SW mirroring or=20
> nothing) and the
> partition name. If they were not mounted, it is assumed=20
> you know which
> physical devices you have allocated for Informix during the disks
> partitioning.
>=20
> 2) If mounted, 'umount' the Informix block devices comment=20
> their corresponding
> entries in /etc/fstab to prevent them to be mounted when=20
> server is rebooted.
>=20
> 3) For easier usability of Informix block devices, create=20
> symbolic links under
> /etc/udev/devices, e.g.
>=20
> ln -s /dev/cciss/c1d1p8 /etc/udev/devices/online_logdbs
> ln -s /dev/cciss/c1d0p2 /etc/udev/devices/online_rootdbs01
> ln -s /dev/cciss/c1d1p5 /etc/udev/devices/online_rootdbs02
> ln -s /dev/cciss/c1d1p9 /etc/udev/devices/online_tempdbs
>=20
> the above setting makes the symbolic links (which by=20
> default will be created
> under /dev once 'udev' service is [re]started) to be=20
> persistent across
> reboots.
>=20
>=20
> 4) Set persistent ownership and permissions for the Informix=20
> physical devices,
> by adding the proper entries to=20
> /etc/udev/rules.d/50-udev.rules, e.g.:
> ----------------------------------------------------------
> # Informix block devices configuration
> KERNEL=3D=3D"cciss/c1d1p8", NAME=3D"%k", OWNER=3D"informix",=20
> GROUP=3D"informix",=20
> MODE=3D"0660", OPTIONS=3D"last_rule"
> KERNEL=3D=3D"cciss/c1d0p2", NAME=3D"%k", OWNER=3D"informix",=20
> GROUP=3D"informix",=20
> MODE=3D"0660", OPTIONS=3D"last_rule"
> KERNEL=3D=3D"cciss/c1d1p5", NAME=3D"%k", OWNER=3D"informix",=20
> GROUP=3D"informix",=20
> MODE=3D"0660", OPTIONS=3D"last_rule"
> KERNEL=3D=3D"cciss/c1d1p9", NAME=3D"%k", OWNER=3D"informix",=20
> GROUP=3D"informix",=20
> MODE=3D"0660", OPTIONS=3D"last_rule"
> # End of Informix block devices configuration
> ----------------------------------------------------------
>=20
> 5) [re]start udev (or reboot the machine)
> ----------------------------------------------------------
> /sbin/start_udev
> ----------------------------------------------------------
>=20
> 6) check result:
> ----------------------------------------------------------
> [root@zeus dev]# ls -la /dev/online_*
> lrwxrwxrwx 1 root root 17 May 3 17:49 /dev/online_logdbs ->=20
> /dev/cciss/c1d1p8 lrwxrwxrwx 1 root root 17 May 3 17:49=20
> /dev/online_rootdbs01 -> /dev/cciss/c1d0p2 lrwxrwxrwx 1 root=20
> root 17 May 3 17:49 /dev/online_rootdbs02 ->=20
> /dev/cciss/c1d1p5 lrwxrwxrwx 1 root root 17 May 3 17:49=20
> /dev/online_tempdbs -> /dev/cciss/c1d1p9 [root@zeus dev]# ls=20
> -lL /dev/online_*
> brw-rw---- 1 informix informix 105, 24 May 3 16:02 /dev/online_logdbs
> brw-rw---- 1 informix informix 105, 2 May 3 16:02=20
> /dev/online_rootdbs01
> brw-rw---- 1 informix informix 105, 21 May 3 16:02=20
> /dev/online_rootdbs02
> brw-rw---- 1 informix informix 105, 25 May 3 16:02=20
> /dev/online_tempdbs
> ----------------------------------------------------------
>=20
> Note the "b" prefix for the block devices.
>=20
> Should work, at least it does for me.
>=20
> Cheers
> Rupan3rd (from rainy Italy)
>=20
>=20
> P.S. IDS 10 is not yet officially certified for RHEL5 but=20
> AFAICS it seems to work flawlessly.
> ________________________________________
_______
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>=20
>=20