BC remote Oracle DBA - Call (800) 766-1884  
Oracle Consulting Oracle Training Development

Remote DBA

Remote DBA Plans  

Remote DBA Service

Remote DBA RAC

Remote DBA Oracle Home
Remote DBA Oracle Training
Remote DBA SQL Tuning Consulting
Remote DBA Oracle Tuning Consulting
Remote DBA Data Warehouse Consulting
Remote DBA Oracle Project Management
Remote DBA Oracle Security Assessment
Remote DBA Unix Consulting
Burleson Books
Burleson Articles
Burleson Web Courses
Burleson Qualifications
Oracle Links
Remote DBA Oracle Monitoring
Remote DBA Support Benefits
Remote DBA Plans & Prices
Our Automation Strategy
What We Monitor
Oracle Apps Support
Print Our Brochure
Contact Us (e-mail)
Oracle Job Opportunities
Oracle Consulting Prices





Remote DBA services

Remote DBA Support

Remote DBA RAC

Remote DBA Reasons

Remote Oracle Tuning

Remote DBA Links

Oracle DBA Support

Oracle DBA Forum

Oracle Disaster

Oracle Training

Oracle Tuning

Oracle Training

 Remote DBA SQL Server

Remote MSSQL Consulting

Oracle DBA Hosting

Oracle License Negotiation








Extending STATSPACK for OPS Information

Oracle Tips by Burleson Consulting

Once the IDLM is running, you can run the lkdump command to determine the values of IDLM resources and locks, as well as overall IDLM processes. Checking these values is a very critical activity because these values cannot exceed the values defined in your IDLM configuration file. If any one of these values exceeds the value in the configuration file, the IDLM will hang, causing your entire OPS system to lock up.

It is very important to keep track of the high-water mark for these IDLM values. The following script will get the existing initial values for locks, resources, and programs from your conf file and compare these values to the maximum values from your Oracle IDLM statistics table. It is a good idea to run this script at least once per day to ensure that the Oracle database does not hang.

Here is the code to create a STATSPACK extension table to hold the IDLM information.


L 13-9

connect perfstat/perfstat;

drop table perfstat.stats$idlm_stats;

create table perfstat.stats$idlm_stats
   snap_time       date,
   pro             number,
   res             number,
   loc             number
tablespace perfstat
storage (initial 1m next 1m maxextents unlimited)

It is always a good idea to keep a running list of these critical IDLM values, and the script here will capture these values and place them into STATSPACK extension tables for later analysis.


L 13-10

# Capture IDLM statistics – 1998 by Donald Keith Burleson
DAY_OF_WEEK=`date +"%A"`
REPORT_FILE=/oracle/HOME/Remote DBA/dlm_monitor.${MACHINE_NAME}.${DAY_OF_WEEK}.log
# Set up the file to log the lock to:
TIMESTAMP=`date +"%C%y.%m.%d-%H:%M:%S"`
DLM_RESOURCES=`/oracle/HOME/bin/lkdump -a res | head -2 | awk 'getline'`
DLM_LOCKS=`/oracle/HOME/bin/lkdump -a lock | head -2 | awk 'getline' `
DLM_PROCESS=`/oracle/HOME/bin/lkdump -a proc | head -2 | awk 'getline'`

RES=`echo $DLM_RESOURCES|cut -f2 -d '='`
LOC=`echo $DLM_LOCKS|cut -f2 -d '='`
PRO=`echo $DLM_PROCESS|cut -f2 -d '='`

PATH=$PATH:/oracle/HOME/bin; export PATH;

/oracle/HOME/bin/sqlplus <<! >> /dev/null

connect perfstat/perfstat;

insert into perfstat.stats$idlm_stats
 values (
   $LOC );


This get_idlm.ksh script is normally scheduled to run every five minutes to gather a snapshot of the important IDLM behaviors. Most Remote DBAs place the execution for the script in their UNIX crontab file.

Now that we have defined a table for the IDLM information, we are ready to look at a report from our stats$idlm_stats table. The next script is run against our STATSPACK extension table to report on exceptional conditions within the IDLM.


L 13-11

#! /bin/ksh
# Display high-water mark of IDLM
# Donald K. Burleson
# get the max values . . . .

#grep ^lkmgr_args /oracle/HOME/ops/HOME.conf
MAX_LOC=`grep ^lkmgr_args /oracle/HOME/ops/HOME.conf|cut -f6 -d ' '`
MAX_RES=`grep ^lkmgr_args /oracle/HOME/ops/HOME.conf|cut -f8 -d ' '`
MAX_PRO=`grep ^lkmgr_args /oracle/HOME/ops/HOME.conf|cut -f10 -d ' '|sed -e 's/"//

PATH=$PATH:/oracle/HOME/bin; export PATH;

unalias rm
rm -f *.tmp

/oracle/HOME/bin/sqlplus <<! > /dev/null

connect perfstat/perfstat;

set newpage 0;
set space 0
set pages 0
set termout off
set feedback off
set echo off
set heading off

spool pro
select max(processes) from perfstat.stats$dlm_stats;
spool res
select max(resources) from perfstat.stats$dlm_stats;
spool loc
select max(locks) from perfstat.stats$dlm_stats;
spool off

PRO=`grep '^  ' pro.lst|awk '{print $1}'`

RES=`grep '^  ' res.lst|awk '{print $1}'`

LOC=`grep '^  ' loc.lst|awk '{print $1}'`

# Now the fun part . . . .

PCT_PRO=`expr $PRO \* 100 \/ $MAX_PRO`
echo "IDLM Process high-water mark is $PRO, or $PCT_PRO percent of max val of

PCT_RES=`expr $RES \* 100 \/ $MAX_RES`
echo "IDLM Resource high-water mark is $RES, or $PCT_RES percent of max val of

PCT_LOC=`expr $LOC \* 100 \/ $MAX_LOC`
echo "IDLM Locks high-water mark is $LOC, or $PCT_LOC percent of max val of $MAX_LOC"

Again, most Remote DBAs schedule this report to run hourly so they can track changes in the high-water mark for the IDLM. Next, let's look at the v$ views that can help us track down OPS locking issues with the IDLM.

Querying the v$ Views for Oracle Parallel Server

Oracle has several v$ views that were created especially for OPS. These views can be used to tell the Remote DBA the cumulative rate of OPS activity since the instance was started. Here are the most interesting v$ views for OPS:

  • The v$lock_activity view This view is a very good way to determine if you have reached the maximum lock convert rate for your IDLM. If the maximum lock convert rate has been reached, you will need to repartition the application to balance alike transactions into common instances.

  • The v$ping view This view tracks the number of IDLM pings that relate to this database instance.

The v$ping view is especially useful for showing lock conversions in OPS. Oracle Parallel Server provides a view called v$ping to show lock conversions. We start by querying the v$ping view to see if there are any data files experiencing a high degree on lock conversions, as follows.


L 13-12

GROUP BY 1, 2, 3
ORDER BY 1, 2, 3;

Here is the output from this script. The XNC column is the one that directly relates to lock conversions.

L 13-13

Name     File #   Class #   Max (XNC)
------   ------   -------   --------
Customer     13         1        556
Customer     13         4         32
Item          6         1          1
Item          3         4         32
Order        16         1      33456

Here, we see that the order table (File 16) may have a problem with excessive lock conversions because of the high value for Max XNC. To further investigate, return to v$ping, and get the sums for File 16, as follows:

L 13-14


Now, we can see additional detail about the contents of File 16, as shown here:

L 13-15

File #     Block #      Stat       XNC   Class #  Name     Kind
------     -------      ----     -----   -------  ----     -----
16            11        XCUR         5     1     ORDER     Table
16            12        XCUR     33456     1     ORDER     Table
16            13        XCUR        12     1     ORDER     Table

From this output, we can see that block 12 inside File 16 is the source of our contention. The following query against the order table will reveal the contents of the rows in the data block. Remember, data blocks are numbered in hex, so we convert the decimal number 12 to a hexadecimal “C”.

L 13-16

chattorowid(rowid) LIKE '0000000C%;

Here we see the results from this query, showing the details for all data rows on this block:

L 13-17

------------------  -----------   ---------------
0000000C.0000.0008         1212                73
0000000C.0000.0008         1213                73
0000000C.0000.0008         1214                73

These results indicate that lock conversion relates to orders placed by customer number 73. Other than a random coincidence, we can assume that there may be freelist contention in the order table as new orders are added to the database. Adding additional freelists will allow more concurrency during SQL insert operations, and the value for freelists should be reset to the maximum number of end users who are expected to be inserting an order row at any given time. In Oracle8i, you can issue the following command:

L 13-18

alter table
   ( freelists 7 )

Before Oracle8i, Oracle does not allow the dynamic modification of freelists, because they were physically stored in each data block. So, the only alternative is to drop and re-create the table with more freelists in each block header. Following is the SQL used to drop and recreate the order table:

L 13-19

   STORAGE (freelists 10)

Next, let’s compare OPS with RAC and understand how RAC is used for continuous availability. One of the most exciting new features of Oracle9i is their new and improved clustering product, Real Application Clusters.  This next section explores techniques to achieve success with RAC and see alternative mechanisms for automated failover and continuous availability.

This is an excerpt from "Oracle9i High Performance tuning with STATSPACK" by Oracle Press.

If you like Oracle tuning, you may enjoy the new book "Oracle Tuning: The Definitive Reference", over 900 pages of BC's favorite tuning tips & scripts. 

You can buy it direct from the publisher for 30%-off and get instant access to the code depot of Oracle tuning scripts.

Expert Remote DBA

BC is America's oldest and largest Remote DBA Oracle support provider.  Get real Remote DBA experts, call
BC Remote DBA today.




Remote DBA Service

Oracle Tuning Book


Advance SQL Tuning Book 

BC Oracle support

Oracle books by Rampant

Oracle monitoring software







BC Remote Oracle Support

Remote DBA

Remote DBA Services

Copyright © 1996 -  2013 by Burleson. All rights reserved.

Oracle® is the registered trademark of Oracle Corporation.

Hit Counter