Long transactions
Posted in 2003
Topics: Server Administration
If I'm using a dbaccess script to load data from a file is there a way I can
tell the load to commit the transaction after every 'x' rows processed. I'm
trying to load a file with about 180000 records. I know I can use dbload,
but with the automated tools I have dbaccess works better.
My script file is:
database db1;
load from loadfile.dat insert into table1;commit;
Thanks. M
"mpw" <em-2@cox.net> wrote
> If I'm using a dbaccess script to load data from a file is there a way I can
> tell the load to commit the transaction after every 'x' rows processed. I'm
> trying to load a file with about 180000 records. I know I can use dbload,
> but with the automated tools I have dbaccess works better.
>
> My script file is:
>
> database db1;
> load from loadfile.dat insert into table1;> commit;
use Jonathan Leffler's sqlreload.
sqlreload -d database -t table -i unload_file -n commit_after_n_rows.
The default for -n is 1024.
I find it to be the best as compared to dbaccess or dbload. Great for
automated tools.
Something like
#! /bin/ksh
split loadfile.data
for F in `ls x*`
do
dbaccess db1 <<-EOF
load from $F insert into eric; EOF
done
mpw wrote:
>
> If I'm using a dbaccess script to load data from a file is there a way I can
> tell the load to commit the transaction after every 'x' rows processed. I'm
> trying to load a file with about 180000 records. I know I can use dbload,
> but with the automated tools I have dbaccess works better.
>
> My script file is:
>
> database db1;
> load from loadfile.dat insert into table1;> commit;
>
> Thanks. M
--
Paul Watson #
Oninit Ltd # Growing old is mandatory
Tel: +44 1436 672201 # Growing up is optional
Fax: +44 1436 678693 #
Mob: +44 7818 003457 #
www.oninit.com #
You could drop the indexes, alter the table to "raw", do the load, and then
alter it back to "standard".
"mpw" <em-2@cox.net> wrote in message
news:lmXMa.564558$vU3.107079@news1.central.cox.net...
> If I'm using a dbaccess script to load data from a file is there a way I
can
> tell the load to commit the transaction after every 'x' rows processed.
I'm
> trying to load a file with about 180000 records. I know I can use dbload,
> but with the automated tools I have dbaccess works better.
>
> My script file is:
>
> database db1;
> load from loadfile.dat insert into table1;> commit;
>
> Thanks. M
>
>
On Thu, 03 Jul 2003 14:47:45 GMT, "mpw" <em-2@cox.net> wrote:
If you are running on Unix/Linux you can use split to divide your input file to
smaller pices. Here is from 'man split'
"The split command reads file and writes it in as many n-line pieces as
necessary (default 1000), onto a set of output files. The name of the first
output file is name with aa appended, and so on lexicographically. If no
output name is given, x is default."
So you can write script:
split loadfile.dat ABC
small_files=`ls ABC*`
# make SQL script file
echo " " > load.sql
for b in $small_files; do
echo "load from $b insert into table1;" >> load.sql
done
# run dbaccess
dbaccess db1 load.sql
#
rm ABC* load.sql
Nebojsa
>If I'm using a dbaccess script to load data from a file is there a way I can
>tell the load to commit the transaction after every 'x' rows processed. I'm
>trying to load a file with about 180000 records. I know I can use dbload,
>but with the automated tools I have dbaccess works better.
>
>My script file is:
>
>database db1;
>load from loadfile.dat insert into table1;>commit;
>
>Thanks. M
>
------------------------------------
Remove spam block (DELETE_) to reply