ontape -s -L 0 performance on Win2k3_x64
Posted in 2008
A user running IDS 10.0 TC5 on Windows 2003 x64 with chunks on an EMC Clariion SAN reported that an 'ontape -s -L 0' archive of ~325GB usually took about 13 hours, starting at ~140MB/sec and dropping to 4-5MB/sec after 30 minutes, yet occasionally the identical job finished in about 2h20. Plain file copies to the same SAN target ran at 200-260MB/sec, so raw hardware throughput looked fine. Suggestions included checking the message log for timestamp wrapping (checked, not present), open transactions, SAN bandwidth monitoring, Windows write caching masking true I/O rates, confirming all compared backups were level 0, and comparing logical-log activity during fast versus slow runs. No resolution is recorded in the thread.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Backup & Restore, Performance & Tuning, Storage & Space Management, Versions, Editions & End-of-Life
Hi !
I am running IDS 10.0 TC5 on a MS-Win2k3_x64
The chunks are stored on a EMC-Clarion-SAN
My Problem is, that a backup-job run's about 13hours
and writes a filesize of about 325 GB.
That's pretty slow I think !
3 Weeks ago I have noticed that the same job, without
changing anything, run's 2hours and 20minutes !!!
That happens about 3 times (Monday, Wednesday, Thursday) !
I have missed to copy such a backup-file to run an archecker !!!
Do you have any ideas why the backup-task run's so fast ?
Or better, why it is'nt so fast every day ???
The backup-file is also written to the SAN!
I have been watching disk-io while running ontape.
The job starts with about 140MB/sec and slows down after 30min to 4-5MB/sec !!!
I have also tried to copy any files about 2GB to the backup-file destination
while backup-job running -> 200MB/sec !!! -> So it isn't the hardware or Win
?!?!
Please, give me any ideas !!!
thx
Martin
MARTIN GREINER wrote:
> Hi !
>
> I am running IDS 10.0 TC5 on a MS-Win2k3_x64
> The chunks are stored on a EMC-Clarion-SAN
>
> My Problem is, that a backup-job run's about 13hours
> and writes a filesize of about 325 GB.
> That's pretty slow I think !
>
> 3 Weeks ago I have noticed that the same job, without
> changing anything, run's 2hours and 20minutes !!!
> That happens about 3 times (Monday, Wednesday, Thursday) !
> I have missed to copy such a backup-file to run an archecker !!!
>
> Do you have any ideas why the backup-task run's so fast ?
> Or better, why it is'nt so fast every day ???
>
> The backup-file is also written to the SAN!
> I have been watching disk-io while running ontape.
> The job starts with about 140MB/sec and slows down after 30min to 4-5MB/sec
> !!!
> I have also tried to copy any files about 2GB to the backup-file destination
> while backup-job running -> 200MB/sec !!! -> So it isn't the hardware or Win
> ?!?!
>
> Please, give me any ideas !!!
>
Look to see if there are any entries in your online message log file
about the timestamp wrapping. If that happens then the archive has to
restamp older data pages so it won't mistake them for newly modified
ones based on the timestamp. If this is happening, there is a workaround.
Art S. Kagel
Oninit
> thx
> Martin
>
>
================================================================================
===========
Please access the attached hyperlink for an important electronic
communications disclaimer:
http://www.oninit.com/home/disclaimer.php
================================================================================
===========
No, I can't find any entrys with "wrap" ! There much others, but they appears bevor and after the backup: 18:06:45 Checkpoint loguniq 2254, logpos 0xedb2ec, timestamp: 0x920f310c 18:11:45 Checkpoint loguniq 2254, logpos 0xf09110, timestamp: 0x920f5d47 18:16:45 Checkpoint loguniq 2254, logpos 0x11641d0, timestamp: 0x921085b8 18:21:45 Checkpoint loguniq 2254, logpos 0x11921f0, timestamp: 0x9210b479 18:26:45 Checkpoint loguniq 2254, logpos 0x1293340, timestamp: 0x928e2c35 18:31:45 Checkpoint loguniq 2254, logpos 0x1354390, timestamp: 0x92b83b19 18:36:45 Checkpoint loguniq 2254, logpos 0x1480450, timestamp: 0x92b88262 18:42:14 Checkpoint loguniq 2254, logpos 0x14812a0, timestamp: 0x92b8a880 18:47:14 Checkpoint loguniq 2254, logpos 0x14861c0, timestamp: 0x92b8c380 18:52:14 Checkpoint loguniq 2254, logpos 0x148a070, timestamp: 0x92b8eed0 18:54:42 Logical Log 2254 Complete, timestamp: 0x92bafcc0. 18:57:14 Checkpoint loguniq 2255, logpos 0x1a8e070, timestamp: 0x92bff034 19:02:14 Checkpoint loguniq 2255, logpos 0x1ab5190, timestamp: 0x92c01906 19:03:26 Logical Log 2255 Complete, timestamp: 0x92c0db3e. 19:04:01 Logical Log 2256 Complete, timestamp: 0x92c6c315. 19:05:38 Logical Log 2257 Complete, timestamp: 0x92cc0469. 19:06:56 Logical Log 2258 Complete, timestamp: 0x92d1fc31. 19:07:14 Checkpoint loguniq 2259, logpos 0xf781c0, timestamp: 0x92d4852f 19:08:30 Logical Log 2259 Complete, timestamp: 0x92d74fb1. 19:10:27 Logical Log 2260 Complete, timestamp: 0x92dd270a. 19:12:14 Checkpoint loguniq 2261, logpos 0x1beb1c0, timestamp: 0x92e23382 19:17:14 Checkpoint loguniq 2261, logpos 0x1c14080, timestamp: 0x92e2587a 19:22:14 Checkpoint loguniq 2261, logpos 0x1c15050, timestamp: 0x92e269ac 19:27:14 Checkpoint loguniq 2261, logpos 0x1c1c050, timestamp: 0x92e26a4f 19:28:02 Logical Log 2261 Complete, timestamp: 0x92e27c30. 19:31:00 Checkpoint loguniq 2262, logpos 0xbf3a20, timestamp: 0x92e4ad1b 19:34:33 Logical Log 2262 Complete, timestamp: 0x92e690bd. 19:34:42 Checkpoint loguniq 2263, logpos 0x472940, timestamp: 0x92e6f895 19:35:20 Checkpoint loguniq 2263, logpos 0x19c4ca8, timestamp: 0x92e92cfd 19:37:55 Logical Log 2263 Complete, timestamp: 0x92ebcf90. 19:40:42 Checkpoint loguniq 2264, logpos 0x102774, timestamp: 0x92ed0193 thx
MARTIN GREINER wrote: > No, I can't find any entrys with "wrap" ! > Yes, looks normal. Ahh well, one idea gone. Art S. Kagel Oninit > There much others, but they appears bevor and after the backup: > > 18:06:45 Checkpoint loguniq 2254, logpos 0xedb2ec, timestamp: 0x920f310c > 18:11:45 Checkpoint loguniq 2254, logpos 0xf09110, timestamp: 0x920f5d47 > 18:16:45 Checkpoint loguniq 2254, logpos 0x11641d0, timestamp: 0x921085b8 > 18:21:45 Checkpoint loguniq 2254, logpos 0x11921f0, timestamp: 0x9210b479 > 18:26:45 Checkpoint loguniq 2254, logpos 0x1293340, timestamp: 0x928e2c35 > 18:31:45 Checkpoint loguniq 2254, logpos 0x1354390, timestamp: 0x92b83b19 > 18:36:45 Checkpoint loguniq 2254, logpos 0x1480450, timestamp: 0x92b88262 > 18:42:14 Checkpoint loguniq 2254, logpos 0x14812a0, timestamp: 0x92b8a880 > 18:47:14 Checkpoint loguniq 2254, logpos 0x14861c0, timestamp: 0x92b8c380 > 18:52:14 Checkpoint loguniq 2254, logpos 0x148a070, timestamp: 0x92b8eed0 > 18:54:42 Logical Log 2254 Complete, timestamp: 0x92bafcc0. > 18:57:14 Checkpoint loguniq 2255, logpos 0x1a8e070, timestamp: 0x92bff034 > 19:02:14 Checkpoint loguniq 2255, logpos 0x1ab5190, timestamp: 0x92c01906 > 19:03:26 Logical Log 2255 Complete, timestamp: 0x92c0db3e. > 19:04:01 Logical Log 2256 Complete, timestamp: 0x92c6c315. > 19:05:38 Logical Log 2257 Complete, timestamp: 0x92cc0469. > 19:06:56 Logical Log 2258 Complete, timestamp: 0x92d1fc31. > 19:07:14 Checkpoint loguniq 2259, logpos 0xf781c0, timestamp: 0x92d4852f > 19:08:30 Logical Log 2259 Complete, timestamp: 0x92d74fb1. > 19:10:27 Logical Log 2260 Complete, timestamp: 0x92dd270a. > 19:12:14 Checkpoint loguniq 2261, logpos 0x1beb1c0, timestamp: 0x92e23382 > 19:17:14 Checkpoint loguniq 2261, logpos 0x1c14080, timestamp: 0x92e2587a > 19:22:14 Checkpoint loguniq 2261, logpos 0x1c15050, timestamp: 0x92e269ac > 19:27:14 Checkpoint loguniq 2261, logpos 0x1c1c050, timestamp: 0x92e26a4f > 19:28:02 Logical Log 2261 Complete, timestamp: 0x92e27c30. > 19:31:00 Checkpoint loguniq 2262, logpos 0xbf3a20, timestamp: 0x92e4ad1b > 19:34:33 Logical Log 2262 Complete, timestamp: 0x92e690bd. > 19:34:42 Checkpoint loguniq 2263, logpos 0x472940, timestamp: 0x92e6f895 > 19:35:20 Checkpoint loguniq 2263, logpos 0x19c4ca8, timestamp: 0x92e92cfd > 19:37:55 Logical Log 2263 Complete, timestamp: 0x92ebcf90. > 19:40:42 Checkpoint loguniq 2264, logpos 0x102774, timestamp: 0x92ed0193 > > thx > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > See you at the IIUG Informix 2008 Conference > The Power Conference for Informix Professionals > April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas > http://www.iiug.org/conf > Registration Now Open!! > > ================================================================================ =========== > Please access the attached hyperlink for an important electronic communications disclaimer: > > http://www.oninit.com/home/disclaimer.php > > ================================================================================ =========== > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
The experience I had (with IDS 7.31 on Dynix) was that if the ontape backup
level 0 was terminated, then the very next backup took longer -- I don't have
an explanation for it.
For your case, do you think there may be series of opened transactions which
may cause the backup to wait/slow down?
----- Original Message ----
From: MARTIN GREINER <martin.greiner@softline.at>
To: ids@iiug.org
Sent: Thursday, February 21, 2008 8:21:59 AM
Subject: ontape -s -L 0 performance on Win2k3_x64 [11358]
Hi !
I am running IDS 10.0 TC5 on a MS-Win2k3_x64
The chunks are stored on a EMC-Clarion-SAN
My Problem is, that a backup-job run's about 13hours
and writes a filesize of about 325 GB.
That's pretty slow I think !
3 Weeks ago I have noticed that the same job, without
changing anything, run's 2hours and 20minutes !!!
That happens about 3 times (Monday, Wednesday, Thursday) !
I have missed to copy such a backup-file to run an archecker !!!
Do you have any ideas why the backup-task run's so fast ?
Or better, why it is'nt so fast every day ???
The backup-file is also written to the SAN!
I have been watching disk-io while running ontape.
The job starts with about 140MB/sec and slows down after 30min to 4-5MB/sec
!!!
I have also tried to copy any files about 2GB to the backup-file destination
while backup-job running -> 200MB/sec !!! -> So it isn't the hardware or Win
?!?!
Please, give me any ideas !!!
thx
Martin
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
See you at the IIUG Informix 2008 Conference
The Power Conference for Informix Professionals
April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas
http://www.iiug.org/conf
Registration Now Open!!
________________________________________________________________________________
____
Looking for last minute shopping deals?
Find them fast with Yahoo! Search.
http://tools.search.yahoo.com/newsearch/category.php?category=shopping
Hi Martin,
MARTIN GREINER schrieb:
> Hi !
>
> I am running IDS 10.0 TC5 on a MS-Win2k3_x64
> The chunks are stored on a EMC-Clarion-SAN
How is this SAN connected to your Win2k3 machine?
Is it possible to monitor this connection to find out
how much of the possible bandwith your database machine can
actually use?
>
> My Problem is, that a backup-job run's about 13hours
> and writes a filesize of about 325 GB.
> That's pretty slow I think !
what is the target of your backup? Another SAN or tape?
>
> 3 Weeks ago I have noticed that the same job, without
> changing anything, run's 2hours and 20minutes !!!
Are all backups you take LEVEL 0 backups?
(command line option '-L') or are the fasterones -L 1 or
even -L 2 (LEVEL 1 or LEVEL 2 incremental) backups?
> That happens about 3 times (Monday, Wednesday, Thursday) !
> I have missed to copy such a backup-file to run an archecker !!!
>
> Do you have any ideas why the backup-task run's so fast ?
> Or better, why it is'nt so fast every day ???
>
> The backup-file is also written to the SAN!
> I have been watching disk-io while running ontape.
> The job starts with about 140MB/sec and slows down after 30min to 4-5MB/sec
> !!!
> I have also tried to copy any files about 2GB to the backup-file destination
> while backup-job running -> 200MB/sec !!! -> So it isn't the hardware or Win
How much memory is in your Win2k3 machine?
It is very likely, that you just write to your main memory, which is
used as cache and that physical I/O of files smaller than the memory size
is written to the output device in write behind style.
Try to test I/O thruput using bonnie++ or something similar.
I am sorry, but I do not have enough Windows experience when it
comes to server setups.
> ?!?!
>
> Please, give me any ideas !!!
HTH as a starter
dic_k
>
> thx
> Martin
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
> See you at the IIUG Informix 2008 Conference
> The Power Conference for Informix Professionals
> April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas
> http://www.iiug.org/conf
> Registration Now Open!!
>
--
Richard Kofler
SOLID STATE EDV
Dienstleistungen GmbH
Vienna/Austria/Europe
Of course there are Transaktions running,
but over night there should not be so much !
I've got the same performance when trying a backup
in worktime !!!
The SAN is connected via 4GBit FC Adapter and I am able to reach a troughput
of about 260MB/sec when copying files with Windows.
I have contacts they told me I should spend time to take a look at "onstat -g
stk all", but I can't read that !?!?
I am really sure that I compare level 0 backups and not 0 with 1 !!!
The backup files have the same size (+- a few MB).
Maybe someone of you can read my stack-info ???
thx
Check the volume of Logical Log usage during the backups. Is there a significant difference during the fast backup compared with the first 2 hours of the slow backup?
Related threads
- Posting from the Informix-list
- Migrating from IDS 9.40.UC6 to 11.50.UC3
- Ip for a network session
- questions onstat -g