myunload question
A user running Art Kagel's myexport on CentOS 6.7 hit gawk errors ("unterminated string" at line 58) during the data unload phase. Kagel identified the downloaded script as corrupted — line 58 had been split across two lines — and emailed a clean copy, which fixed it. He also explained that -p launches one task per table (very resource-hungry, not throttleable), that -O should be used to preserve referential dependency order, and that the -E external-table option requires the output directory to be visible to both client and server on the same path.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Hi.
I'm trying to run myunload on a CentOS 6.7 box. I downloaded the newest
version (as well as utils_2k) from the archive. The schema builds fine.
[informix@restore export]$ time myexport joakolap -p -E -m -u
Building schema file
myschema -q -d joakolap -l -U=/backup/export/joakolap.exp/joakolap3.sql
/backup/export/joakolap.exp/joakolap.sql
/backup/export/joakolap.exp/joakolap2.sql
Beginning data export
Unloading data for table:
gawk: cmd. line:24: warning: escape sequence `\\\\.' treated as plain `.'
gawk: cmd. line:58: printf "echo unload
gawk: cmd. line:58: ^ unterminated string
/usr/local/bin/myexport: line 395: | dbaccess %s - ", DBASE;
}
}: not found
Done
Has anyone experienced this?
When using the -p switch (parallel) how many concurrent jobs are started?
Thanks in advance,
-Snorri
Snorri:
It looks like line 58 in myexport got broken into two lines. I will email
you a proper copy directly. Can't attach to postings on the forums.
On your second question, -p launches one task for every table in the
database, so you don't want to do that on a production system unless no one
is using it and you'd only want to do that on a fairly beefy system
anyway. The myimport script has an option to specify how many tasks to
launch in parallel, but I've never put that option into myexport for
several reasons the most compelling (to me) is that the main reason that
the -p option is there is to minimize the possibility that there will be
child records with no parent in referential relationships and partial
parallelism doesn't resolve that problem.
FWIW, if you have dependencies it is a good idea to include the -O option
to myexport so that the tables will be reloaded by myimport/dbimport in the
order of their dependencies, parents before children.
Art
Art S. Kagel, President and Principal Consultant
ASK Database Management
www.askdbmgt.com
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on the IIUG, nor any other organization with which I am
associated either explicitly, implicitly, or by inference. Neither do
those opinions reflect those of other individuals affiliated with any
entity with which I am affiliated nor those of the entities themselves.
On Tue, Nov 17, 2015 at 8:01 AM, SNORRI BERGMANN <snorri@init.is> wrote:
> Hi.
>
> I'm trying to run myunload on a CentOS 6.7 box. I downloaded the newest
> version (as well as utils_2k) from the archive. The schema builds fine.
>
> [informix@restore export]$ time myexport joakolap -p -E -m -u
> Building schema file
> myschema -q -d joakolap -l -U=/backup/export/joakolap.exp/joakolap3.sql
> /backup/export/joakolap.exp/joakolap.sql
> /backup/export/joakolap.exp/joakolap2.sql
> Beginning data export
> Unloading data for table:
> gawk: cmd. line:24: warning: escape sequence `\\\\.' treated as plain `.'
> gawk: cmd. line:58: printf "echo unload
> gawk: cmd. line:58: ^ unterminated string
> /usr/local/bin/myexport: line 395: | dbaccess %s - ", DBASE;
>
> }
> }: not found
> Done
>
> Has anyone experienced this?
> When using the -p switch (parallel) how many concurrent jobs are started?
>
> Thanks in advance,
> -Snorri
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001a113fe84ee5baee0524bcca35
Thanks Art. Now the script is working, thank you. If you use the -p option on a large database with > 800 tables, won't the system get really hogged, regardless of #CPUs and memory? Regards, -Snorri
-p is definitely a resource hog, yes. Note that unlike dbexport, you CAN
run myexport on a separate client host from the server to take advantage of
separate memory, cpu, and IO bandwidth. Even with the large resource
requirements, myexport with -p is often faster than without it, but
remember, speed isn't the main reason it exists in myexport (unlike
myimport which has "-p" and "-r -n" specifically for faster loading.
Myimport with -p or -r and -E uses few resources outside of the engine
itself since it uses external tables for the loading.
Art
Art S. Kagel, President and Principal Consultant
ASK Database Management
www.askdbmgt.com
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on the IIUG, nor any other organization with which I am
associated either explicitly, implicitly, or by inference. Neither do
those opinions reflect those of other individuals affiliated with any
entity with which I am affiliated nor those of the entities themselves.
On Tue, Nov 17, 2015 at 9:08 AM, SNORRI BERGMANN <snorri@init.is> wrote:
> Thanks Art.
>
> Now the script is working, thank you.
>
> If you use the -p option on a large database with > 800 tables, won't the
> system get really hogged, regardless of #CPUs and memory?
>
> Regards,
> -Snorri
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001a11349ceee2655b0524bd2cf6
Is it possible to use the -E option on a different server? Like:
$ myexport joakolap@restore -p -E -m -u -O
And the external tables get created locally?
No. The server writes to the files so they have to live on the server. You
can run myexport remotely but ONLY if the -o directory is visible by both
the client and server on the same path since both have to write to and read
from there.
Art
On Nov 17, 2015 08:32, "SNORRI BERGMANN" <snorri@init.is> wrote:
> Is it possible to use the -E option on a different server? Like:
>
> $ myexport joakolap@restore -p -E -m -u -O>
> And the external tables get created locally?
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--089e013c6fa8925f1a0524be0227
That applies to the -E option only. The default can run remotely without
the shared mount, but I don't think the high speed loader options can run
remotely at all.
On Nov 17, 2015 09:16, "Art Kagel" <art.kagel@gmail.com> wrote:
> No. The server writes to the files so they have to live on the server. You
> can run myexport remotely but ONLY if the -o directory is visible by both
> the client and server on the same path since both have to write to and read
> from there.
>
> Art
> On Nov 17, 2015 08:32, "SNORRI BERGMANN" <snorri@init.is> wrote:
>
> > Is it possible to use the -E option on a different server? Like:
> >
> > $ myexport joakolap@restore -p -E -m -u -O> >
> > And the external tables get created locally?
> >
> >
> >
> >
>
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --089e013c6fa8925f1a0524be0227
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--089e0111bea2e435580524be0a85