high performance loader
Posted in 2004
I had to download 13 million rows from a table and I decided to use
onpload for that. so I used myonpload to download.
I noticed that it spawned off many onpload process to download
the table:-
dba 16892 14234 0 13:15:40 pts/10 0:00 myonpload -d big_bro -t search_request2003 -f
search2003.unl -U D
dba 16904 16892 0 13:15:42 pts/10 0:00 sh -c onpload -p PR16892 -j JB16892 -fu
dba 16954 16905 0 13:15:47 pts/10 0:01 onpload -p PR16892 -j JB16892 -fu
dba 16952 16905 0 13:15:46 pts/10 0:00 onpload -p PR16892 -j JB16892 -fu
dba 16915 16905 0 13:15:43 pts/10 0:00 onpload -p PR16892 -j JB16892 -fu
dba 16955 16905 4 13:15:47 pts/10 0:17 onpload -p PR16892 -j JB16892 -fu
dba 16916 16905 1 13:15:43 pts/10 0:02 onpload -p PR16892 -j JB16892 -fu
dba 16905 16904 41 13:15:42 pts/10 5:08 onpload -p PR16892 -j JB16892 -fu
dba 16934 16905 0 13:15:44 pts/10 0:00 onpload -p PR16892 -j JB16892 -fu
dba 16953 16905 0 13:15:46 pts/10 0:00 onpload -p PR16892 -j JB16892 -fu
dba 16914 16905 0 13:15:43 pts/10 0:00 onpload -p PR16892 -j JB16892 -fu
Just out of curiosity, how does it work. If there are many processes parallely
downloading the table, how do they write to the same unload file simultaneously.
I was expecting each process to write to its own unload file and merge all files
into one at the end. It doesn't seem so.
BTW onpload is so DAMN fast. Just 3 seconds to download 100000 rows.