Under the heading of improper disk setup there are
many sub topics. Some of these disk setup topics include:
·
Interface issues
·
Mount options
·
Filesystem choices
·
RAID setup
·
Disk size and speed choices
Let’s look at each of these in the Oracle environment.
Interface Issues
Generally interface issues resolve to bandwidth
issues. A case in point, a major bakery had upgraded their system,
putting in more, faster CPUs, higher speed disks and newer hardware
overall. They calculated on the average they only used 75% of the
bandwidth on the old system so they reduced the number of HBAs from 12
dual-channel to 8 dual-channel.
After the upgrade performance looked great, until the
end of month processing crunch, suddenly performance dropped to half
of what it was before. Investigation showed that while on the average
they only needed 75% of the bandwidth of the 12 HBAs during end of
month, end of quarter and end of year processing they actually
required more. Luckily for them the HBAs in the old system where
compatible and, they had the needed expansion slots to add the needed
HBAs to the new system. With the 4 additional HBAs in place they
quadrupled their performance.
The other major choice in interfaces is in interface
type, SCSI, Fibre, Fabric. Unfortunately there is no simple answer,
you need to examine your system and if IO timing is bad, find out
whether it is related to contention or bandwidth issues.
Note that when you monitor IO timing you need to look
at it from Oracle’s perspective, that is, from the time Oracle
requests the IO to the time the IO is received by the Oracle system,
not strictly at the operating system level. If you see a large
difference between what Oracle is saying IO timings are and what the
OS is saying you need to track down where the time is being consumed
between the disk packs and the Oracle database.
Mount Options
Essentially Oracle doesn’t like any mount option
involving logging on its datafile mount points. Anything you can do to
make the disk look RAW to Oracle is a good thing. This means nologging,
noatime, async, aio and many other mount options need to be considered
depending on your operating system. Dramatic improvements in
performance have been reported from just changing the mount options on
the drives otr filesystems that support your Oracle datafiles.
Generally it is not advised to switch to nologging type options on
filesystems where non-Oracle or binary type files (executables) are
stored.
In UNIX you can control whether a file system uses
buffered or unbuffered IO. With Oracle the use of a buffered
filesystem is redundant and dangerous. An example of the dangers of a
buffered filesystem with Oracle is when power is lost. The buffer in a
buffered filesystem depends on the cache battery to provide enough
power to allow the buffer to be written to disk before the disk spins
down. However, many shops fail to monitor the cache battery lifetime
limitations or fail to change the batteries at all. This can result in
loss of data in a buffered filesystem on loss of power.
You can turn off buffered writes in several ways
(buffered reads aren’t an issue, but you should always use
write-through caching). One is to mount the filesystems used with
Oracle files as non-buffered using such options as:
AIX: “dio”, “rbrw”, “nointegrity”
SUN: “delaylog”, “mincache=direct”,
“convosync=direct” ,”nodatainlog”
LINUX: “async”, “noatime”
HP: Use VxFS with: “delaylog”, “nodatainlog”,
“mincache=direct”, “convosync=direct”
Using Direct IO at the Oracle Level
For information about Oracle direct I/O, refer to this
URL by Steve Adams:
*
http://www.ixora.com.au/notes/filesystemio_options.htm
Checking Your Server
Methods for configuring the OS will vary depending on
the operating system and file system in use. Here are some examples of
quick checks that anyone can perform to ensure that you are using
direct I/O:
·
Solaris - Look for a "forcedirectio" option. Oracle
Remote DBAs find this option often makes a huge difference in I/O speed for
Sun servers. Here is the Sun documentation:
http://docs.sun.com/db/doc/816-0211/6m6nc6713?a=view
·
AIX - Look for a "dio" option. Here is a great link for
AIX direct I/O:
http://www-106.ibm.com/developerworks/eserver/articles/DirectIO.html
·
Veritas VxFS - (including HP-UX, Solaris and AIX), look
for "convosync=direct". It is also possible to enable direct I/O on
a per-file basis using Veritas QIO; refer to the "qiostat" command
and corresponding man page for hints. For HPUX, see
Oracle on HP-UX – Best Practices.
·
Linux - Linux systems support direct I/O on a per-filehandle
basis (which is much more flexible), and I believe Oracle enables this
feature automatically. Someone should verify at what release Oracle
started to support this feature (it is called O_DIRECT). See
Kernel Asynchronous I/O (AIO) Support for
Linux and this great OTN article:
Talking Linux: OCFS Update.
