Re: LOADING DATA INTO WAREHOUSING
Hi,
I think this is a case of using the wrong tool for the job.
The loader is not designed nor optimized to load only 30
records, its optimized for loading millions. For small loads
use SQL or dbload.
Regards - Lester
T.Suresh Kumar wrote:
>
> --------------A6770B456A20800C597E4886
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
> Hello Lester
> At present I am facing a probelm with HPL. Long back I bech marked it
> with by loading 1.6 million records. It loaded in just 4 mins. This is nice
> But It is giving a bad performance when I am loading 1260 tables with each
> table contains rows around 30. I am running 3 ONPLOAD statements at command line
> simultaneously.
> But the time it is taking is almost a day to load all tables.
> my plconfig file looks like this
>
> # System Configuration
> CONVERTTHREADS 2 # Number of conversion threads per device
> CONVERTVPS 6 # Max number of vps for converters (total)
> # Buffer Configuration
>
> STRMBUFFSIZE 64 # Buffer size for server stream buffer (kbytes)
> STRMBUFFERS 4 # Number of server stream buffers per device
> AIOBUFSIZE 64 # Buffer size for tape/file I/O (kbytes)
> AIOBUFFERS 4 # Number of buffers for I/O per device
>
> How can I improve performance of HPL. In a day I may need to load 2 to 3 times
> all the tables for some testing purposes.
> Any help in improving would be greately appriciated.
>
> Please remember that I cannot use other loading utilities like DBLOAD, onload.
>
> I can use only ONPLOAD .
>
> Wishes
> Suresh
>
> Lester Knutsen wrote:
>
> > Hi,
> >
> > I really like the high performance loader, its very fast.
> >
> > We load about 11GB per hour using the high performance loader.
> > Its well worth the time to set up and learn. I have also used it
> > on a NT machine with SCO's Xvision X server. Saves buying an
> > X-workstation. Once you have built the load formats you can
> > run everything from the command line or as we do in cron jobs.
> >
> > Regards - Lester
> >
>
> --------------A6770B456A20800C597E4886
> Content-Type: text/html; charset=us-ascii
> Content-Transfer-Encoding: 7bit
>
> <HTML>
> Hello Lester
> <BR> At present I am facing a
> probelm with HPL. Long back I bech marked it with by loading 1.6 million
> records. It loaded in just 4 mins. This is nice
> <BR>But It is giving a bad performance when I am loading 1260 tables with
> each table contains rows around 30. I am running 3 ONPLOAD statements at
> command line simultaneously.
> <BR>But the time it is taking is almost <B>a day</B> to load all tables.
> <BR>my plconfig file looks like this
>
> <P># System Configuration
> <BR>CONVERTTHREADS 2
> # Number of conversion threads per device
> <BR>CONVERTVPS 6
> # Max number of vps for converters (total)
> <BR># Buffer Configuration
>
> <P>STRMBUFFSIZE 64
> # Buffer size for server stream buffer (kbytes)
> <BR>STRMBUFFERS 4
> # Number of server stream buffers per device
> <BR>AIOBUFSIZE 64
> # Buffer size for tape/file I/O (kbytes)
> <BR>AIOBUFFERS 4
> # Number of buffers for I/O per device
>
> <P>How can I improve performance of HPL. In a day I may need to load 2
> to 3 times all the tables for some testing purposes.
> <BR>Any help in improving would be greately appriciated.
>
> <P>Please remember that I cannot use other loading utilities like DBLOAD,
> onload.
>
> <P>I can use only ONPLOAD .
>
> <P>Wishes
> <BR>Suresh
>
> <P>Lester Knutsen wrote:
> <BLOCKQUOTE TYPE=CITE>Hi,
>
> <P>I really like the high performance loader, its very fast.
>
> <P>We load about 11GB per hour using the high performance loader.
> <BR>Its well worth the time to set up and learn. I have also used it
> <BR>on a NT machine with SCO's Xvision X server. Saves buying an
> <BR>X-workstation. Once you have built the load formats you can
> <BR>run everything from the command line or as we do in cron jobs.
>
> <P>Regards - Lester
> <BR> </BLOCKQUOTE>
> </HTML>
>
> --------------A6770B456A20800C597E4886--
--
______________________________________________________________________
Lester Knutsen lester@advancedatatools.com
Advanced DataTools Corporation Voice: 703-256-0267
Grant group privileges for Informix databases with DB Privileges
Visit our Web page: www.advancedatatools.com
Washington Area Informix User Group: www.iiug.org/~waiug/
______________________________________________________________________