Oninit Logo
The Down System Specialists
+1-913-674-0360
+44-2081-337529
Partnerships Contact
Onstat -a -b/-B -C -c -d/-D -F -f -G -h -i -j -k/-K -L -l -m -o -P -p -R -r -s -t/-T -u -X -x -z

onstat -g rqm ER Statistics for RQM Queues

The onstat -g rqm command displays the statistics for Reliable Queue Manager [RQM] queues. The RQM manages the insertion and removal of items to and from the various queues. The RQM also manages spooling of the in-memory portions of the queue to and from disk. The onstat -g rqm command displays the contents of the queue, size of the transactions in the queue, how much of the queue is in memory and on disk, the location of various handles to the queue, and the contents of the various progress tables. You can choose to print information for all queues or for just one queue by using one of the modifiers described below.

If a queue is empty, no information is printed for that queue.

The onstat -g rqm can take an optional modifier. The following table describes the values for the modifier.

ModifierAction
ACKQPrints the ack send queue
CNTRLQPrints the control send queue
RECVQPrints the receive queue
SBSPACESPrints detailed statistical information about the sbspaces configured for CDR_QDATA_SBSPACE.
SENDQPrints the send queue
SYNCQPrints the sync send queue
FULLPrints full information about every in-memory transaction for every queue
BRIEFPrints a brief summary of the number of transactions in each of the queues and the replication servers for which the data is queued Use this modifier to quickly identify sites where a problem exists. If large amounts of data are queued for a single server, then that server is probably down or off the network.
VERBOSEPrints all the buffer headers in memory


When you specify a modifier to select a specific queue, the command prints all the statistics for that queue and information about the first and last in-memory transactions for that queue. When you select the SBSPACES modifier, the command prints information about the sbspaces being used for replication, including how full those sbspaces are.

The other modifiers of the onstat -g rqm command are used primarily as a debugging tool

The output for the SENDQ modifier contains the following sections:

  • The current statistics section (Transaction spool name through Pending Txn Data): Contains information about the current contents of the queue, such as how many bytes are contained in the queue, how many transactions are in the queue, how many transactions are currently in memory, how many have been spooled to disk, how many exist only on disk, and so on. The Insert Stamp field value is used to maintain the order of the transactions within the queue. The Size of Data in queue field shows the size of the queue when combining the in-memory transactions with the spool-only transactions. The Pending Txn Buffers field contains information about transactions that are in the process of being queued into the send queue. The historical statistics section (Max Real memory data used through Total Txn Lookups): contains a summary of what has been placed in the queue in the past. The Max Real memory data used field contains the largest in memory size of the queue. The Total Txn Recovered field shows the transactions that existed only in the spool when the server was started. The Total Txns deleted field shows the number of transactions that have been removed from the queue. The Total Txns duplicated field contains the number of times attempted to queue a transaction that had already been processed. The Total Txn Lookups field is a counter of the number of times that an ER thread attempted to read a transaction.
  • The Progress Table section: contains information on what is currently queued, to which server it is queued for, and what has been acknowledged from each of the participants of the replicate. The first part of the progress table section is a summary . Below the summary section is a list of the servers and group entries that contain what is currently queued for each server, what has been sent to the remote server, and what has been acknowledged from the remote server. The contents of the ACKed and Sent columns contains the key of the last transaction that was acknowledged from the remote server or sent to that server. The key is a multi-part number consisting of source_node/unique_log_id/logpos/incremental number. The transaction section contains the first and last transaction in the queue that are currently in memory. The NeedAck field shows from which server the transaction is waiting for an acknowledgment. You can use this bitmap mask with the output from the onstat -g cat command to determine the name of the server which server Enterprise Replication is waiting on for an acknowledgment.
  • The Transverse handle section: contains the position within the queue that any thread is currently processing. Each thread that attempts to read a transaction from the queue, or to place a transaction into the queue must first allocate a handle. This handle is used to maintain the positioning within the queue.
onstat -g rqm sendq

CDR Reliable Queue Manager (RQM) Statistics:

RQM Statistics for Queue (0x3d914a028) trg_send
 Transaction Spool Name: trg_send_stxn
 Insert Stamp: 22293692
 Flags: SEND_Q, SPOOLED, PROGRESS_TABLE, NEED_ACK
 Txns in queue:             0
 Log Events in queue:       0
 Txns in memory:            0
 Txns in spool only:        0
 Txns spooled:              0
 Unspooled bytes:           0
 Size of Data in queue:     0 Bytes
 Real memory in use:        0 Bytes
 Pending Txn Buffers:       0
 Pending Txn Data:          0 Bytes
 Max Real memory data used: 4194248 (4194304) Bytes
 Max Real memory hdrs used  3005856 (3081680832) Bytes
 Total data queued:         14658380490 Bytes
 Total Txns queued:         22293692
 Total Txns spooled:        2656
 Total Txns restored:       682
 Total Txns recovered:      0
 Spool Rows read:           682
 Total Txns deleted:        22293692
 Total Txns duplicated:     0
 Total Txn Lookups:         66810608

 Progress Table:
        Progress Table is Stable
                On-disk table name............:         spttrg_send
                Flush interval (time).........:         30
                Time of last flush............:         1370363073
                Flush interval (serial number):         1000
                Serial number of last flush...:         109915
                Current serial number.........:         109917

Server    Group Bytes Queued      Acked                     Sent
------------------------------------------------------------------------------
     1  0x2003b                  0 2/3668/0x898a16c/0x0     -    2/3668/0x898a16c/0x0
     1  0x20019                  0 2/3610/0xccf0458/0x0     -    2/3610/0xccf0458/0x0
     1  0x200cd                  0 2/1394/0x136cb2f0/0x0    -          No Content Yet
     1  0x20035                  0 2/2964/0x25e32220/0x0    -          No Content Yet
     1  0x2008e                  0 2/887/0xedd11b8/0x0      -          No Content Yet
     1  0x2008d                  0 2/990/0x1c5683a4/0x0     -          No Content Yet
     1  0x20091                  0 2/1384/0x1ecab428/0x0    -          No Content Yet
     1  0x20092                  0 2/1384/0x1ec6d7b0/0x0    -          No Content Yet
     1  0x20011                  0 2/439/0xa10f3f4/0x0      -          No Content Yet
     1  0x20053                  0 2/3503/0x5f55638/0x0     -    2/3503/0x5f55638/0x0
     1  0x20052                  0 2/1929/0x137a2d8/0x0     -          No Content Yet
     1  0x2005b                  0 2/3665/0x300071b0/0x0    -   2/3665/0x300071b0/0x0
     1  0x2000c                  0 2/633/0x2e8b001c/0x0     -          No Content Yet
     1  0x2003c                  0 2/3674/0x125065c8/0x0    -   2/3674/0x125065c8/0x0
     1  0x2000f                  0 2/647/0x1d691318/0x0     -          No Content Yet
     1  0x20013                  0 2/647/0x1d832434/0x0     -          No Content Yet
     1  0x20016                  0 2/647/0x1d76d708/0x0     -          No Content Yet
     1  0x20055                  0 2/3674/0x1365e690/0x0    -   2/3674/0x1365e690/0x0
     1  0x20054                  0 2/3674/0x12e0b254/0x0    -   2/3674/0x12e0b254/0x0
     1  0x2006c                  0 2/3674/0x12e3a31c/0x0    -   2/3674/0x12e3a31c/0x0
     1  0x2000a                  0 2/347/0xbb67340/0x0      -          No Content Yet
     1  0x20057                  0 2/3674/0x14d4d7c8/0x0    -   2/3674/0x14d4d7c8/0x0
     1  0x20065                  0 2/3674/0x14d6137c/0x0    -   2/3674/0x14d6137c/0x0
     1  0x20038                  0 2/3668/0x2f7bf1b0/0x0    -   2/3668/0x2f7bf1b0/0x0
     1  0x2001b                  0 2/3674/0x11cb4364/0x0    -   2/3674/0x11cb4364/0x0
     1  0x20001                  0 2/3674/0x12f0618c/0x0    -   2/3674/0x12f0618c/0x0


 Traverse handle (0x3d9788028) for thread CDRNsT1 at Head_of_Q,  Flags: None
.......
 Traverse handle (0x3d97c8028) for thread CDRNr1 at Head_of_Q,  Flags: None


onstat -g rqm recv

CDR Reliable Queue Manager (RQM) Statistics:

RQM Statistics for Queue (0x3d919f028) trg_receive
 Transaction Spool Name: trg_receive_stxn
 Insert Stamp: 367458171
 Communal Stamp: 0
 Flags: RECV_Q, SPOOLED, PROGRESS_TABLE
 Txns in queue:             6
 Txns in memory:            6
 Txns in spool only:        0
 Txns spooled:              0
 Unspooled bytes:           0
 Size of Data in queue:     782281 Bytes
 Real memory in use:        782281 Bytes
 Pending Txn Buffers:       0
 Pending Txn Data:          0 Bytes
 Max Real memory data used: 5999987 (4194304) Bytes
 Max Real memory hdrs used  6503584 (3081680832) Bytes
 Total data queued:         23884344839 Bytes
 Total Txns queued:         37092106
 Total Txns spooled:        0
 Total Txns restored:       0
 Total Txns recovered:      0
 Spool Rows read:           0
 Total Txns deleted:        37092100
 Total Txns duplicated:     0
 Total Txn Lookups:         74184207

 Progress Table:
        Progress Table is Stable
                On-disk table name............:         spttrg_receive
                Not keeping dirty list.

Server    Group Bytes Queued      Acked                     Sent
------------------------------------------------------------------------------
     1  0x2001b                  0 1/268769/0xe2593cc/0x0   -          No Content Yet
     1  0x20054                  0 1/268196/0xd1bf2b8/0x0   -          No Content Yet
     1  0x2006c                  0 1/268196/0xe02d174/0x0   -          No Content Yet
     1  0x20055                  0 1/268196/0xdb02258/0x0   -          No Content Yet
     1  0x2003c                  0 1/271769/0x25716258/0x0  - 1/271769/0x25716258/0x0
     1  0x2000c                  0 1/271759/0x36241b0/0x0   -  1/271759/0x36241b0/0x0
     1  0x2003b                  0 1/271761/0x2fb351b0/0x0  - 1/271761/0x2fb351b0/0x0
     1  0x20019                  0 1/268757/0x18987220/0x0  -          No Content Yet
     1  0x20090                  0 1/271708/0x2d9b1318/0x0  - 1/271708/0x2d9b1318/0x1
     1  0x200cd             652211 1/271769/0x2710a138/0x0  -1/271769/0x275b4140/0x2a3
     1  0x20011                  0 1/267739/0x186a8314/0x0  -          No Content Yet
     1  0x20038                  0 1/268761/0x1b97b1b0/0x0  -          No Content Yet
     1  0x20016                  0 1/271769/0x245ca0f8/0x0  -1/271769/0x245ca0f8/0x137
     1  0x2008e                  0 1/271761/0x73c53cc/0x0   -  1/271761/0x73c53cc/0x0
     1  0x20013                  0 1/271769/0x24935580/0x0  -1/271769/0x24935580/0x137
     1  0x20065                  0 1/268769/0xff1b37c/0x0   -          No Content Yet
     1  0x20057                  0 1/268769/0xff147c8/0x0   -          No Content Yet
     1  0x2000f                  0 1/271769/0x248990e0/0x0  -1/271769/0x248990e0/0x130
     1  0x2000a                  0 1/269782/0x275512b0/0x0  -          No Content Yet
     1  0x20092                  0 1/271769/0x27091718/0x0  -1/271769/0x27091718/0xe8
     1  0x2008d                  0 1/271731/0x4239220/0x0   -  1/271731/0x4239220/0x0
     1  0x200aa                  0 1/267744/0x6446118/0x0   -          No Content Yet
     1  0x2005b                  0 1/268181/0x2010e1b0/0x0  -          No Content Yet
     1  0x20091                  0 1/271769/0x270d4068/0x0  -1/271769/0x270d4068/0x55
     1  0x20001                342 1/271769/0x2710b18c/0x0  - 1/271769/0x275b518c/0x0

 First Txn (0x3e43277b0) Key:  1/271769/0x2746b074/0x00000000
 Txn Stamp: 367458166, Reference Count: 1.
 Txn Flags: None
 Txn Commit Time: (1370363010) 2013/06/04 11:23:30
 Txn Size in Queue: 619337
 First Buf's (0x3d9a740d0) Queue Flags: Resident
 First Buf's Buffer Flags: TRG, Stream
 Traverse handle (0x3dd127218) for thread CDRD_8397 at 
      txn (0x3e43277b0): 1/271769/0x2746b074/0x00000000 Flags: In_Transaction

 Last Txn (0x3dd5b0a78) Key:  1/271769/0x275b518c/0x00000000
 Txn Stamp: 367458171, Reference Count: 0.
 Txn Flags: None
 Txn Commit Time: (1370363011) 2013/06/04 11:23:31
 Txn Size in Queue: 146
 First Buf's (0x3dd5b0c90) Queue Flags: Resident
 First Buf's Buffer Flags: TRG, Stream
 Traverse handle (0x3dd345028) for thread CDRD_8425 at txn (0x3dd5b0a78) End_of_Q,  Flags: None

 Traverse handle (0x3d97c6028) for thread CDRNr1 at Head_of_Q,  Flags: None
 Traverse handle (0x3dc80a028) for thread CDRD_8375 at Head_of_Q,  Flags: None
 Traverse handle (0x3dd2a1028) for thread CDRD_8390 at Head_of_Q,  Flags: None
......
 Traverse handle (0x3dd127218) for thread CDRD_8397 at 
      txn (0x3e43277b0): 1/271769/0x2746b074/0x00000000
 Flags: In_Transaction
 Traverse handle (0x3dd1282c0) for thread CDRD_8397 at Head_of_Q,  Flags: None
......
 Traverse handle (0x3dd345028) for thread CDRD_8425 at txn (0x3dd5b0a78) End_of_Q,  Flags: None
......

To discuss how Oninit ® can assist please call on +1-913-674-0360 or alternatively just send an email specifying your requirements.