11.5 - Dbimport performance
A user migrating from IDS 9.40 to 11.50 on HP-UX found dbimport extremely slow (28, 36, even 60 hours, with checkpoints up to 238 seconds), while the same export loaded into 11.50 on Linux/Intel in 24 hours. Respondents suggested: set PDQPRIORITY at the top of the .sql schema file to speed index builds, use buffered logging (or load unlogged and add logging later), enlarge BUFFERPOOL/LRU settings and MGM memory for 11.50's non-blocking checkpoints, check KAIO and disk devices, reuse existing extents via TRUNCATE, and strip the redundant UPDATE STATISTICS from the 9.40-generated schema since 11.50 builds stats during index creation. No outcome was reported by the original poster, so no confirmed resolution is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Hi Guys,
3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was about
migration strategy in a HPUX environment.
When I export and import database in the same instance, with no parameter
changes, it takes 36 hours. In another test, I performed a dbexport and after
that, the instalation migration process (installing 11.5 over the 9.4
enginne). Well, for my surprise, it takes 60 hours. Once and again: no
parameters change and no tunning. The same dbexport data was "imported" in a
11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
First thing - anybody saw this situation before?
Second one - Does anybody here have any advice for dbimport performance in
11.5 enginne?
Thanks a lot
André
_________________________________________________________________
Você sabia que o Hotmail mudou? Clique e descubra as novidades.
http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
I saw the same thing going from HP-UX to Linux with dbimport. It seems to
process a lot slower, however, I suspect the reason is that I'm using a
Left-Hand Networks iSCSI SAN (SCSI over Ethernet) as apposed to my EMC
Clariion Fiber-Channel SCSI SAN as being the reason for the difference in
performance of processing a dbimport.
One of the things I did was to turn on buffered logging, as apposed to
unbuffered logging. That did help a lot.
Here is an example importing a database called "common" into dbspace "dbs0":
# dbimport common -c -d dbs0 -l buffered -I /tmp/export
The "-l buffered" made a big difference over just simply using "-l" switch
(unbuffered).
I hope that information is helpful.
Sincerely,
Jonathan B. Smaby
Pomona College
phone: (909) 621-8506
email: jonathan.smaby@pomona.edu
web: http://www.facebook.com/jonathan.smaby
I Support Our Volunteer Heroes:
[United States Coast Guard Auxiliary]
-------------------------------------------------------------
This message has been scanned by Postini anti-virus software.
Set a pdqpriority at the top of the ".sql" file in the ".exp" directory.
If don't do this already, you will see a huge improvement when the indexes
get created. That's the simplest way to get a quick improvement.
From:
"André Luiz Rufino" <andre_rufino@msn.com>
To:
ids@iiug.org
Date:
09/02/2009 09:18 AM
Subject:
11.5 - Dbimport performance [16837]
Sent by:
ids-bounces@iiug.org
Hi Guys,
3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was about
migration strategy in a HPUX environment.
When I export and import database in the same instance, with no parameter
changes, it takes 36 hours. In another test, I performed a dbexport and
after
that, the instalation migration process (installing 11.5 over the 9.4
enginne). Well, for my surprise, it takes 60 hours. Once and again: no
parameters change and no tunning. The same dbexport data was "imported" in
a
11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
First thing - anybody saw this situation before?
Second one - Does anybody here have any advice for dbimport performance in
11.5 enginne?
Thanks a lot
André
_________________________________________________________________
Você sabia que o Hotmail mudou? Clique e descubra as novidades.
http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
Hi André, Are you using the same disk device used for IDS 9.40 to import all data in the IDS 11.50 ? Is KAIO enabled? Read this article (portuguese) : http://www.imartins.com.br/informix/artigos/recomendacoes-carga-dados (english by google translate): http://translate.google.com/translate?u=http%3A%2F%2Fwww.imartins.com.br%2Finfor mix%2Fartigos%2Frecomendacoes-carga-dados&sl=pt&tl=en
Tuning IDS 11.50 for all of its new and highly vaunted "autonomic"
functionality, is still an esoteric and iterative art. Did you start with
the 9.40 ONCONFIG file or a fresh 11.50 ONCONFIG file? There will be
differences. Did you increase the MGM memory (11.xx needs more memory for
internal functions)? Did you increase your LRU_MIN/MAX_DIRTY settings -
with 11.50's non-blocking checkpoints it operates much better with mostly
chunk writes while earlier releases liked at least 60% LRU writes.
If you can post some specific things you think are slowing you down we'll
try to interpret.
Beyond that I would have to have hands on with your server. Contact Oninit
if you want/need billable help.
Art
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
2009/9/2 André Luiz Rufino <andre_rufino@msn.com>
> Hi Guys,
>
> 3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was about
> migration strategy in a HPUX environment.
> When I export and import database in the same instance, with no parameter
> changes, it takes 36 hours. In another test, I performed a dbexport and
> after
> that, the instalation migration process (installing 11.5 over the 9.4
> enginne). Well, for my surprise, it takes 60 hours. Once and again: no
> parameters change and no tunning. The same dbexport data was "imported" in
> a
> 11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
>
> First thing - anybody saw this situation before?
>
> Second one - Does anybody here have any advice for dbimport performance in
> 11.5 enginne?
>
> Thanks a lot
>
> André
>
> _________________________________________________________________
> Você sabia que o Hotmail mudou? Clique e descubra as novidades.
> http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--00151747bf2c4faa9a04729f0f69
Hello Art,
I performed the dbimport three times. The first one took 28 hours. Nothing
about anything was made. I simply exported and imported the database. The
second test, I exported database, initialized the instance, made the set of
the new onconfig file, and I neither increased the mgm memory nor even changed
LRU parameters too. Time= 36 hours
After all this step, I imagine that the new parameters set to default values
was makeing my engine slower. So, I migrated the IDS enginne without change a
thing in oncofnig file, once the 9.4 enginne had the best performance until
there.
That was the worst test. It took 60 hours and checkpoints durations went 'till
238 seconds.
does it make sense to you?
Thanks in advance,
André
> To: ids@iiug.org
> From: art.kagel@gmail.com
> Subject: Re: 11.5 - Dbimport performance [16844]
> Date: Wed, 2 Sep 2009 17:36:38 -0400
>
> Tuning IDS 11.50 for all of its new and highly vaunted "autonomic"
> functionality, is still an esoteric and iterative art. Did you start with
> the 9.40 ONCONFIG file or a fresh 11.50 ONCONFIG file? There will be
> differences. Did you increase the MGM memory (11.xx needs more memory for
> internal functions)? Did you increase your LRU_MIN/MAX_DIRTY settings -
> with 11.50's non-blocking checkpoints it operates much better with mostly
> chunk writes while earlier releases liked at least 60% LRU writes.
>
> If you can post some specific things you think are slowing you down we'll
> try to interpret.
>
> Beyond that I would have to have hands on with your server. Contact Oninit
> if you want/need billable help.
>
> Art
>
> Art S. Kagel
> Oninit (www.oninit.com)
> IIUG Board of Directors (art@iiug.org)
>
> Disclaimer: Please keep in mind that my own opinions are my own opinions and
> do not reflect on my employer, Oninit, the IIUG, nor any other organization
> with which I am associated either explicitly or implicitly. Neither do
> those opinions reflect those of other individuals affiliated with any entity
> with which I am affiliated nor those of the entities themselves.
>
> 2009/9/2 André Luiz Rufino <andre_rufino@msn.com>
>
> > Hi Guys,
> >
> > 3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was about
> > migration strategy in a HPUX environment.
> > When I export and import database in the same instance, with no parameter
> > changes, it takes 36 hours. In another test, I performed a dbexport and
> > after
> > that, the instalation migration process (installing 11.5 over the 9.4
> > enginne). Well, for my surprise, it takes 60 hours. Once and again: no
> > parameters change and no tunning. The same dbexport data was "imported" in
> > a
> > 11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
> >
> > First thing - anybody saw this situation before?
> >
> > Second one - Does anybody here have any advice for dbimport performance in
> > 11.5 enginne?
> >
> > Thanks a lot
> >
> > André
> >
> > _________________________________________________________________
> > Você sabia que o Hotmail mudou? Clique e descubra as novidades.
> > http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
> >
> >
> >
> >
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --00151747bf2c4faa9a04729f0f69
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
_________________________________________________________________
Você sabia que pode acessar o Messenger direto do seu Hotmail? Descubra como!
http://www.microsoft.com/brasil/windows/windowslive/products/tutoriais.aspx
When I ran 9.4 I increased LRU just as Art suggested and saw a imporvement. Checkpoint? Are you creating the database logged? If so, create the db unlogged. After the import you can turn on logging. Where is the import slow? If you're slow when updating stats, maybe turning on PDQ and getting the multitheading going could help. Maybe skip the update stats and use Art's dostats utility - it's fast.
Honestly, I/we just don't have enough to go on to help. From my own
experiences it is probably just a tuning problem, but I would have to be
hands on. One suggestion, part of the problem may be extent allocation. In
the original engine, if you were loading into the existing extents you would
have saved the extent allocation time or perhaps the pattern of existing
extents in the dbspaces allowed a more efficient allocation of new extents
for the tables than in the empty 11.50 instance. Try using the new TRUNCATE
statement to quickly remove all of the data from the tables and rerun the
load with the existing extents intact.
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
2009/9/3 André Luiz Rufino <andre_rufino@msn.com>
> Hello Art,
>
> I performed the dbimport three times. The first one took 28 hours. Nothing
> about anything was made. I simply exported and imported the database. The
> second test, I exported database, initialized the instance, made the set of
> the new onconfig file, and I neither increased the mgm memory nor even
> changed
> LRU parameters too. Time= 36 hours
>
> After all this step, I imagine that the new parameters set to default
> values
> was makeing my engine slower. So, I migrated the IDS enginne without change
> a
> thing in oncofnig file, once the 9.4 enginne had the best performance until
> there.
>
> That was the worst test. It took 60 hours and checkpoints durations went
> 'till
> 238 seconds.
>
> does it make sense to you?
>
> Thanks in advance,
>
> André
>
> > To: ids@iiug.org
> > From: art.kagel@gmail.com
> > Subject: Re: 11.5 - Dbimport performance [16844]
> > Date: Wed, 2 Sep 2009 17:36:38 -0400
> >
> > Tuning IDS 11.50 for all of its new and highly vaunted "autonomic"
> > functionality, is still an esoteric and iterative art. Did you start with
> > the 9.40 ONCONFIG file or a fresh 11.50 ONCONFIG file? There will be
> > differences. Did you increase the MGM memory (11.xx needs more memory for
> > internal functions)? Did you increase your LRU_MIN/MAX_DIRTY settings -
> > with 11.50's non-blocking checkpoints it operates much better with mostly
> > chunk writes while earlier releases liked at least 60% LRU writes.
> >
> > If you can post some specific things you think are slowing you down we'll
> > try to interpret.
> >
> > Beyond that I would have to have hands on with your server. Contact
> Oninit
> > if you want/need billable help.
> >
> > Art
> >
> > Art S. Kagel
> > Oninit (www.oninit.com)
> > IIUG Board of Directors (art@iiug.org)
> >
> > Disclaimer: Please keep in mind that my own opinions are my own opinions
> and
> > do not reflect on my employer, Oninit, the IIUG, nor any other
> organization
> > with which I am associated either explicitly or implicitly. Neither do
> > those opinions reflect those of other individuals affiliated with any
> entity
> > with which I am affiliated nor those of the entities themselves.
> >
> > 2009/9/2 André Luiz Rufino <andre_rufino@msn.com>
> >
> > > Hi Guys,
> > >
> > > 3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was
> about
> > > migration strategy in a HPUX environment.
> > > When I export and import database in the same instance, with no
> parameter
> > > changes, it takes 36 hours. In another test, I performed a dbexport and
> > > after
> > > that, the instalation migration process (installing 11.5 over the 9.4
> > > enginne). Well, for my surprise, it takes 60 hours. Once and again: no
> > > parameters change and no tunning. The same dbexport data was "imported"
> in
> > > a
> > > 11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
> > >
> > > First thing - anybody saw this situation before?
> > >
> > > Second one - Does anybody here have any advice for dbimport performance
> in
> > > 11.5 enginne?
> > >
> > > Thanks a lot
> > >
> > > André
> > >
> > > _________________________________________________________________
> > > Você sabia que o Hotmail mudou? Clique e descubra as novidades.
> > >
> http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
> > >
> > >
> > >
> > >
> >
>
>
*******************************************************************************
> > > Forum Note: Use "Reply" to post a response in the discussion forum.
> > >
> > >
> >
> > --00151747bf2c4faa9a04729f0f69
> >
> >
> >
>
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
>
> _________________________________________________________________
> Você sabia que pode acessar o Messenger direto do seu Hotmail? Descubra
> como!
> http://www.microsoft.com/brasil/windows/windowslive/products/tutoriais.aspx
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001517448ad2705a780472aeb822
Do not use the logging option:
dbimport db_hds -d hds_03 -l &
This takes longer, so turn on logging after loading.
Thanks,
*******************************************************************
Ernie Knox
IT Database Administrator Specialist
Sears Holdings
3333 Beverly Rd., B4-266A
Hoffman Estates, IL. 60179
Office: (847) 286-5735
Email: Ernest.Knox@searshc.com
Blackberry: 2244650553@messaging.sprintpcs.com
Page via Skytel: 2244650553@sprint.skytel.com
Informix or MySQL Primary: 9110210@skytel.com
Informix or MySQL Secondary: 7276872@skytel.com
" Yes we can make a Change! "
" It's always a great day to watch Sports - GO LIONS, TIGERS, and BEARS! "
" Lets not forget - GO Pistons and Red Wings! "
GSU
*******************************************************************
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Art Kagel
Sent: Thursday, September 03, 2009 11:18 AM
To: ids@iiug.org
Subject: Re: 11.5 - Dbimport performance [16854]
Honestly, I/we just don't have enough to go on to help. From my own
experiences it is probably just a tuning problem, but I would have to be
hands on. One suggestion, part of the problem may be extent allocation. In
the original engine, if you were loading into the existing extents you would
have saved the extent allocation time or perhaps the pattern of existing
extents in the dbspaces allowed a more efficient allocation of new extents
for the tables than in the empty 11.50 instance. Try using the new TRUNCATE
statement to quickly remove all of the data from the tables and rerun the
load with the existing extents intact.
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
2009/9/3 André Luiz Rufino <andre_rufino@msn.com>
> Hello Art,
>
> I performed the dbimport three times. The first one took 28 hours. Nothing
> about anything was made. I simply exported and imported the database. The
> second test, I exported database, initialized the instance, made the set of
> the new onconfig file, and I neither increased the mgm memory nor even
> changed
> LRU parameters too. Time= 36 hours
>
> After all this step, I imagine that the new parameters set to default
> values
> was makeing my engine slower. So, I migrated the IDS enginne without change
> a
> thing in oncofnig file, once the 9.4 enginne had the best performance until
> there.
>
> That was the worst test. It took 60 hours and checkpoints durations went
> 'till
> 238 seconds.
>
> does it make sense to you?
>
> Thanks in advance,
>
> André
>
> > To: ids@iiug.org
> > From: art.kagel@gmail.com
> > Subject: Re: 11.5 - Dbimport performance [16844]
> > Date: Wed, 2 Sep 2009 17:36:38 -0400
> >
> > Tuning IDS 11.50 for all of its new and highly vaunted "autonomic"
> > functionality, is still an esoteric and iterative art. Did you start with
> > the 9.40 ONCONFIG file or a fresh 11.50 ONCONFIG file? There will be
> > differences. Did you increase the MGM memory (11.xx needs more memory for
> > internal functions)? Did you increase your LRU_MIN/MAX_DIRTY settings -
> > with 11.50's non-blocking checkpoints it operates much better with mostly
> > chunk writes while earlier releases liked at least 60% LRU writes.
> >
> > If you can post some specific things you think are slowing you down we'll
> > try to interpret.
> >
> > Beyond that I would have to have hands on with your server. Contact
> Oninit
> > if you want/need billable help.
> >
> > Art
> >
> > Art S. Kagel
> > Oninit (www.oninit.com)
> > IIUG Board of Directors (art@iiug.org)
> >
> > Disclaimer: Please keep in mind that my own opinions are my own opinions
> and
> > do not reflect on my employer, Oninit, the IIUG, nor any other
> organization
> > with which I am associated either explicitly or implicitly. Neither do
> > those opinions reflect those of other individuals affiliated with any
> entity
> > with which I am affiliated nor those of the entities themselves.
> >
> > 2009/9/2 André Luiz Rufino <andre_rufino@msn.com>
> >
> > > Hi Guys,
> > >
> > > 3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was
> about
> > > migration strategy in a HPUX environment.
> > > When I export and import database in the same instance, with no
> parameter
> > > changes, it takes 36 hours. In another test, I performed a dbexport and
> > > after
> > > that, the instalation migration process (installing 11.5 over the 9.4
> > > enginne). Well, for my surprise, it takes 60 hours. Once and again: no
> > > parameters change and no tunning. The same dbexport data was "imported"
> in
> > > a
> > > 11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
> > >
> > > First thing - anybody saw this situation before?
> > >
> > > Second one - Does anybody here have any advice for dbimport performance
> in
> > > 11.5 enginne?
> > >
> > > Thanks a lot
> > >
> > > André
> > >
> > > _________________________________________________________________
> > > Você sabia que o Hotmail mudou? Clique e descubra as novidades.
> > >
> http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
> > >
> > >
> > >
> > >
> >
>
>
*******************************************************************************
> > > Forum Note: Use "Reply" to post a response in the discussion forum.
> > >
> > >
> >
> > --00151747bf2c4faa9a04729f0f69
> >
> >
> >
>
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
>
> _________________________________________________________________
> Você sabia que pode acessar o Messenger direto do seu Hotmail? Descubra
> como!
> http://www.microsoft.com/brasil/windows/windowslive/products/tutoriais.aspx
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001517448ad2705a780472aeb822
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
OOO OOO OOO, another idea!
You created the dbexport in 9.40! In 11.50 creating indexes includes
generating basic medium data distributions and low statistics information
about the index keys and the index that's being created. This is faster
than doing the stats separately from the create index but it slows down the
create indexes a bit. The problem may be that dbexport includes UPDATE
STATISTICS commands to duplicate the current set of stats much of which will
be redundant in 11.50 since some of them will already have been build during
the index builds. I don't know if the 11.50 dbexport does the stats
differently, I've never done a comparison of ostensibly identical exports
from 11.xx versus ones from earlier releases.
Anyway, that will be part of the difference. Try editing the schema file in
the dbexport directory and comment out all of the update statistics commands
and create index commands (and constraints) and see how fast the load is
just loading the data in both versions. That would be a fair comparison.
I still think that there's a tuning issue here, but this is definitely a
part of the longer dbimport time.
FYI, my dbschema replacement utility, myschema, can generate a dbimport
compatible schema file ( with the -l flag) and if you also pass a second
filename on the commandline it will automatically place all of the indexes
and constraints into the second schema file and only the table definitions
in the first file. Myschema is part of the package utils2_ak that you can
download free from either the Oninit web site (www.oninit.com/utils) or from
the IIUG Software Repository (www.iiug.org/software).
Art
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
On Thu, Sep 3, 2009 at 12:17 PM, Art Kagel <art.kagel@gmail.com> wrote:
> Honestly, I/we just don't have enough to go on to help. From my own
> experiences it is probably just a tuning problem, but I would have to be
> hands on. One suggestion, part of the problem may be extent allocation. In
> the original engine, if you were loading into the existing extents you would
> have saved the extent allocation time or perhaps the pattern of existing
> extents in the dbspaces allowed a more efficient allocation of new extents
> for the tables than in the empty 11.50 instance. Try using the new TRUNCATE
> statement to quickly remove all of the data from the tables and rerun the
> load with the existing extents intact.
>
> Art S. Kagel
> Oninit (www.oninit.com)
> IIUG Board of Directors (art@iiug.org)
>
> Disclaimer: Please keep in mind that my own opinions are my own opinions
> and do not reflect on my employer, Oninit, the IIUG, nor any other
> organization with which I am associated either explicitly or implicitly.
> Neither do those opinions reflect those of other individuals affiliated
> with any entity with which I am affiliated nor those of the entities
> themselves.
>
>
>
> 2009/9/3 André Luiz Rufino <andre_rufino@msn.com>
>
>> Hello Art,
>>
>> I performed the dbimport three times. The first one took 28 hours. Nothing
>> about anything was made. I simply exported and imported the database. The
>> second test, I exported database, initialized the instance, made the set
>> of
>> the new onconfig file, and I neither increased the mgm memory nor even
>> changed
>> LRU parameters too. Time= 36 hours
>>
>> After all this step, I imagine that the new parameters set to default
>> values
>> was makeing my engine slower. So, I migrated the IDS enginne without
>> change a
>> thing in oncofnig file, once the 9.4 enginne had the best performance
>> until
>> there.
>>
>> That was the worst test. It took 60 hours and checkpoints durations went
>> 'till
>> 238 seconds.
>>
>> does it make sense to you?
>>
>> Thanks in advance,
>>
>> André
>>
>> > To: ids@iiug.org
>> > From: art.kagel@gmail.com
>> > Subject: Re: 11.5 - Dbimport performance [16844]
>> > Date: Wed, 2 Sep 2009 17:36:38 -0400
>> >
>> > Tuning IDS 11.50 for all of its new and highly vaunted "autonomic"
>> > functionality, is still an esoteric and iterative art. Did you start
>> with
>> > the 9.40 ONCONFIG file or a fresh 11.50 ONCONFIG file? There will be
>> > differences. Did you increase the MGM memory (11.xx needs more memory
>> for
>> > internal functions)? Did you increase your LRU_MIN/MAX_DIRTY settings -
>> > with 11.50's non-blocking checkpoints it operates much better with
>> mostly
>> > chunk writes while earlier releases liked at least 60% LRU writes.
>> >
>> > If you can post some specific things you think are slowing you down
>> we'll
>> > try to interpret.
>> >
>> > Beyond that I would have to have hands on with your server. Contact
>> Oninit
>> > if you want/need billable help.
>> >
>> > Art
>> >
>> > Art S. Kagel
>> > Oninit (www.oninit.com)
>> > IIUG Board of Directors (art@iiug.org)
>> >
>> > Disclaimer: Please keep in mind that my own opinions are my own opinions
>> and
>> > do not reflect on my employer, Oninit, the IIUG, nor any other
>> organization
>> > with which I am associated either explicitly or implicitly. Neither do
>> > those opinions reflect those of other individuals affiliated with any
>> entity
>> > with which I am affiliated nor those of the entities themselves.
>> >
>> > 2009/9/2 André Luiz Rufino <andre_rufino@msn.com>
>> >
>> > > Hi Guys,
>> > >
>> > > 3 weeks ago I performed a few tests with IDS 9.4 and 11.5 . It was
>> about
>> > > migration strategy in a HPUX environment.
>> > > When I export and import database in the same instance, with no
>> parameter
>> > > changes, it takes 36 hours. In another test, I performed a dbexport
>> and
>> > > after
>> > > that, the instalation migration process (installing 11.5 over the 9.4
>> > > enginne). Well, for my surprise, it takes 60 hours. Once and again: no
>> > > parameters change and no tunning. The same dbexport data was
>> "imported" in
>> > > a
>> > > 11.5 enginne in a Linux / Intell environment. And it takes 24 hours.
>> > >
>> > > First thing - anybody saw this situation before?
>> > >
>> > > Second one - Does anybody here have any advice for dbimport
>> performance in
>> > > 11.5 enginne?
>> > >
>> > > Thanks a lot
>> > >
>> > > André
>> > >
>> > > _________________________________________________________________
>> > > Você sabia que o Hotmail mudou? Clique e descubra as novidades.
>> > >
>> http://www.microsoft.com/brasil/windows/windowslive/products/hotmail.aspx
>> > >
>> > >
>> > >
>> > >
>> >
>>
>>
*******************************************************************************
>> > > Forum Note: Use "Reply" to post a response in the discussion forum.
>> > >
All the test I had performed, was importing data without logging > To: ids@iiug.org > From: gentrym@staples.com > Subject: Re: RE: 11.5 - Dbimport performance [16852] > Date: Thu, 3 Sep 2009 10:57:13 -0400 > > When I ran 9.4 I increased LRU just as Art suggested and saw a imporvement. > Checkpoint? Are you creating the database logged? If so, create the db > unlogged. After the import you can turn on logging. > > Where is the import slow? If you're slow when updating stats, maybe turning on > PDQ and getting the multitheading going could help. Maybe skip the update > stats and use Art's dostats utility - it's fast. > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > _________________________________________________________________ Acesse seu Hotmail de onde quer que esteja através do celular. Clique aqui. http://www.windowslive.com.br/celular/home.asp?utm_source=MSN_Hotmail&utm_medium =Tagline&utm_campaign=MobileServices200908
My dbimport was taking 8+ hours for a database under 50Gb in size.
What I did to cut down the import time was 2 things:
#1: use buffered logging with the option "-l buffered"
#2: Significantly increase buffers in my BUFFERPOOL in my ONCONFIG file.
Here is my current BUFFERPOOL setting:
BUFFERPOOL
size=2K,buffers=4000000,lrus=32,lru_min_dirty=1.000000,lru_max_dirty=4.000000
HTH
Jonathan B. Smaby
Pomona College
phone: (909) 621-8506
email: jonathan.smaby@pomona.edu
web: http://www.facebook.com/jonathan.smaby
I Support Our Volunteer Heroes
[United States Coast Guard Auxiliary]
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of André
Luiz Rufino
Sent: Thursday, September 03, 2009 10:38 AM
To: ids@iiug.org
Subject: RE: 11.5 - Dbimport performance [16857]
All the test I had performed, was importing data without logging
> To: ids@iiug.org
> From: gentrym@staples.com
> Subject: Re: RE: 11.5 - Dbimport performance [16852]
> Date: Thu, 3 Sep 2009 10:57:13 -0400
>
> When I ran 9.4 I increased LRU just as Art suggested and saw a imporvement.
> Checkpoint? Are you creating the database logged? If so, create the db
> unlogged. After the import you can turn on logging.
>
> Where is the import slow? If you're slow when updating stats, maybe turning
on
> PDQ and getting the multitheading going could help. Maybe skip the update
> stats and use Art's dostats utility - it's fast.
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
_________________________________________________________________
Acesse seu Hotmail de onde quer que esteja através do celular. Clique aqui.
http://www.windowslive.com.br/celular/home.asp?utm_source=MSN_Hotmail&utm_medium
=Tagline&utm_campaign=MobileServices200908
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
-------------------------------------------------------------
This message has been scanned by Postini anti-virus software.