RE: Incremental Archive performance issue......

Rajesh, As You mentioned, Your DLT speed is 35GB/hour. This is why it takes 10-11 hours to make the L0 backup of Your 400GB database. Your L0 backup speed is limited by the DLT device. For L1 archive, the backup speed is mostly limited by the disk array throughput: as I've already mentioned, Informix must read the entire database (like in L0 archive) You can also check whether any significant database writes occur during the backup. If this is the case, You might experience 'very old page' problem ------------------------------------------ Alexey Sonkin ' -----Original Message----- From: Rajasekaran, Rajesh [mailto:RRajasekaran@forestpharm.com] Sent: Friday, April 09, 2004 9:15 AM To: 'Alexey Sonkin' Cc: 'sapmix@iiug.org'; 'informix-list@iiug.org' Subject: RE: Incremental Archive performance issue...... Alexey, ''''' Our level-0 bkup on QAS is currently taking 10-11 hrs for the 400 GB database with the serial backup and single stream. The level-1 bkup used to take not more than 2 hrs to backup. Our PRD level-1 is still taking 1-1.5 hrs and its storedge array is SUN 3510 FC array.' Moreover on the QAS, I dont think its the T3 array throughput bcos if thats the case, it would have degraded the level-0 as well but its not. We dont take level-2 here. Iam not suspecting the DLT drive too. Its somewhere btw the ONBAR & the IDS. 'Let me check what else is changed on the box' such as O/S patches, T3 disk array related patches in the recent past. More suggestions are greatly appreciated. Thanks RAJ -----Original Message----- From: Alexey Sonkin [mailto:alexeis@grandvirtual.com] Sent: Thursday, April 08, 2004 5:19 PM To: 'Rajasekaran, Rajesh'; 'sapmix@iiug.org' Cc: 'informix-list@iiug.org' Subject: RE: Incremental Archive performance issue...... Rajesh, It's not clear to me from Your email how long it takes to make the level-0 backup of Your QAS. I suppose it is about the same as Your Level-1 backup. Is Your question why You see such a significant difference between level-1 and level-0 on PRD and no difference on QA? During Level-1 backup, Informix is reading from the database EXACTLY THE SAME amount of data as during level-0 backup. For each page IDS makes timestamp comparison to decide, whether to put that page into L1 backup or not. The only difference between L1 and L2 is how much data the engine is sending to tape. It means, that in Your case, on QAS backup performance is limited by the T3 array throughput rather then by DLT limitations. More powerful (I suppose) production disk array can ship data much faster to IDS for analysis, and backup performance is constrained by the DLT limitations... ------------------------------------------ Alexey Sonkin ' -----Original Message----- From: Rajasekaran, Rajesh [mailto:RRajasekaran@forestpharm.com] Sent: Thursday, April 08, 2004 4:56 PM To: 'sapmix@iiug.org' Cc: 'informix-list@iiug.org' Subject: Incremental Archive performance issue...... Hi, ''' Our QAS system is a Sun E450 box with a sun storedge T/3 array attached. Its a solaris o/s with IDS 7.31 database. The DB size is around 400 GB. Its backup is directed over the network to a tape library attached to a master server which is another host. Our Master server is also the' PRD box. Its a Sun Fire V880 with Sun L40 tape libary attached. We have the Veritas Netbackup Data center 4.5 for our backups & restores. The QAS DB backup is going over the network to the master server's DLT tape library that gives 35 GB/HR performance. We got that throughput for our QAS level-0 bkup even though its over the T3 network. But the Level-1 backup has severe performance degradation and it takes almost 5-7 hrs to complete. Our PRD level-1's are taking anywhere btw 1.5 - 2 hrs. We are only running the onbar whole system serial bkup. It should not put much I/O bottlenecks since only one dbspace bkup is running at one time and not parallel. 'I can see the iostat shows 10000-25000 Kb/sec for the T3 storedge array continously during the level-1 bkups. The informix "onstat -g stq" shows Full queue 0, Empty queue with Cnt 100 all the time. I dont see the I/O bottleneck here and i dont see the problem with the Veritas Netbackup side since our level-0 works as expected. I dont see any "arc_very_old_pages()" running during the level-1 bkups. Questions 1) What could be the reason for level-1 backup degradation in performance when level-0 is performing well ? 2) What better way to monitor btw the onbar and the engine ? Will the bar_dbug = 9 lvl trace shows' more detail towards my problem ? Any suggestions/comments is greatly appreciated. ' Regards Rajesh Rajasekaran Informix Database Administrator Forest Pharmaceuticals Inc. (314) 493-7073 ____________________________________________________________________' This e-mail and its attachments may contain Forest Laboratories, Inc. proprietary information that is privileged, confidential or subject to copyright belonging to Forest Laboratories, Inc. This e-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this e-mail, or the employee or agent responsible for delivering this e-mail to the intended recipient, you are hereby notified that any dissemination, distribution, copying or action taken in relation to the contents of and attachments to this e-mail is strictly prohibited and may be unlawful. If you have received this e-mail in error, please notify the sender immediately and permanently delete the original and any copy of this e-mail and any printout. sending to informix-list