dbexport
A user on Informix 5.10UC5 / HP-UX 10.20 asked how to dbexport a table larger than 2 GB given the 2 GB file-size limit. Suggestions included exporting to a 'tape device' (possibly a named pipe feeding compress or a large file), enabling OS large-file support, and Art Kagel's myexport/myschema plus sqlcmd utilities from the IIUG repository. Caveats were raised: the 5.10 binaries aren't linked for HP-UX large files, and one poster reported the named-pipe trick failing with dbexport; one site wrote its own export program instead. A side discussion confirmed 5.x tables can exceed 2 GB. No single confirmed fix emerged.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome the 2 gig limitations and
dbexport a table that is over 3.0 gigs???
=====
David Kulkarni
New York, NY 10021
email: dkulkarni@cgsh.com
__________________________________________________
Do You Yahoo!?
Yahoo! Mail - Free email you can access from anywhere!
http://mail.yahoo.com/
David Kulkarni wrote:
> On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome
> the 2 gig limitations and dbexport a table that is over 3.0 gigs???
How did you get a table that is over 3 GB? Or is it that the export
format (ASCII data) is larger than 3 GB but the table in the database is
smaller than 2 GB?
I don't think there's a simple way to do it. One possibility would be
to use a 'tape device' -- export to a tape, specifying, for example, a
fifo as the output device and then having something read the fifo and
stash the data away in a large file. Make sure you keep the schema file
separate from the tape output. Otherwise, you're into jimmying the
system, and you need to know more about what it takes to make the system
work with large files than I do.
--
Yours,
Jonathan Leffler (Jonathan.Leffler@Informix.com) #include <disclaimer.h>
Guardian of DBD::Informix v1.00.PC1 -- http://www.perl.com/CPAN
"I don't suffer from insanity; I enjoy every minute of it!"
Get my dbexport/dbimport replacement utility, myexport, from the IIUG Software
Repository in the package of the same name. You will also need
two other packages that myexport depends on: utils2_ak (actually just the
myschema utility from there) and Jonathan Leffler's sqlcmd (v5.2 or higher)
package both also available from the repository. If your OS supports files
larger than 2GB this will work. Otherwise you have to dbexport to tape.
Some have worked out elaborate schemes using named pipes and gzip also.
Art S. Kagel
David Kulkarni wrote:
>
> On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome the 2 gig limitations and
> dbexport a table that is over 3.0 gigs???
>
> =====
> David Kulkarni
> New York, NY 10021
> email: dkulkarni@cgsh.com
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Mail - Free email you can access from anywhere!
> http://mail.yahoo.com/
Jonathan Leffler wrote:
>
> David Kulkarni wrote:
> > On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome
> > the 2 gig limitations and dbexport a table that is over 3.0 gigs???
>
> How did you get a table that is over 3 GB? Or is it that the export
Why so? OL5.xx supports tables up to 32GB in size.
Art S. Kagel
> format (ASCII data) is larger than 3 GB but the table in the database is
> smaller than 2 GB?
>
> I don't think there's a simple way to do it. One possibility would be
> to use a 'tape device' -- export to a tape, specifying, for example, a
> fifo as the output device and then having something read the fifo and
> stash the data away in a large file. Make sure you keep the schema file
> separate from the tape output. Otherwise, you're into jimmying the
> system, and you need to know more about what it takes to make the system
> work with large files than I do.
>
> --
> Yours,
> Jonathan Leffler (Jonathan.Leffler@Informix.com) #include <disclaimer.h>
> Guardian of DBD::Informix v1.00.PC1 -- http://www.perl.com/CPAN
> "I don't suffer from insanity; I enjoy every minute of it!"
On Tue, 29 Aug 2000, Art S. Kagel wrote:
>Jonathan Leffler wrote:
>> David Kulkarni wrote:
>> > On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome
>> > the 2 gig limitations and dbexport a table that is over 3.0 gigs???
>>
>> How did you get a table that is over 3 GB? [...snip...]
>
>Why so? OL5.xx supports tables up to 32GB in size.
Oh? OK; my brain is fried. Yeah, I guess you're right; individual
chunks in a dbspace can't be more than 2 GB each, but you can have a
tablespace tablespace that spans multiple chunks. I'll reserve judgement
on 32 GB versus any other upper limit -- my (erroneous) point was that
you can't have more than 2 GB in a table and that's wrong.
Ignore me and I'll crawl back into the hole where they keep me... :-)
--
Yours,
Jonathan Leffler (Jonathan.Leffler@Informix.com) #include <disclaimer.h>
Guardian of DBD::Informix v1.00.PC1 -- http://www.perl.com/CPAN
"I don't suffer from insanity; I enjoy every minute of it!"
Jonathan Leffler wrote:
>
> On Tue, 29 Aug 2000, Art S. Kagel wrote:
> >Jonathan Leffler wrote:
> >> David Kulkarni wrote:
> >> > On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome
> >> > the 2 gig limitations and dbexport a table that is over 3.0 gigs???
> >>
> >> How did you get a table that is over 3 GB? [...snip...]
> >
> >Why so? OL5.xx supports tables up to 32GB in size.
>
> Oh? OK; my brain is fried. Yeah, I guess you're right; individual
> chunks in a dbspace can't be more than 2 GB each, but you can have a
> tablespace tablespace that spans multiple chunks. I'll reserve judgement
> on 32 GB versus any other upper limit -- my (erroneous) point was that
OK 16,777,216 pages or ~32GB on a 2K page machine ~64GB on a 4K page machine
same as the current 7.xx/9.xx maximum fragment limit.
Art S. Kagel
> you can't have more than 2 GB in a table and that's wrong.
>
> Ignore me and I'll crawl back into the hole where they keep me... :-)
>
> --
> Yours,
> Jonathan Leffler (Jonathan.Leffler@Informix.com) #include <disclaimer.h>
> Guardian of DBD::Informix v1.00.PC1 -- http://www.perl.com/CPAN
> "I don't suffer from insanity; I enjoy every minute of it!"
David Kulkarni wrote:
> On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome the 2 gig limitations and
> dbexport a table that is over 3.0 gigs???
you could try dumping the database to a compressed pipe. i have not tried it with dbexport, but
i have used i with onunload and onload.
something like:
mknod export.pipe p
cat export.pipe | compress > export.file.Z &
dbexport database -t export.pipe ...
ty
--
Ty O'Kelly
tokelly@maxor.invalid.com
You could try the following :
1. Turn on "Large file" support at the OS level.
2. Try dbexporting to tape (not sure about this)
Rudy
David Kulkarni wrote:
> On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome the 2 gig limitations and
> dbexport a table that is over 3.0 gigs???
>
> =====
> David Kulkarni
> New York, NY 10021
> email: dkulkarni@cgsh.com
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Mail - Free email you can access from anywhere!
> http://mail.yahoo.com/
Rudy Fernandes <rferdy@americasm01.nt.com> writes:
> You could try the following :
>
> 1. Turn on "Large file" support at the OS level.
A good thing in general if you plan on creating bigger files than 2GB on
the filesystem.
Unfortunately enough with Informix 5.10 that is not enough. To be able
to use the HP-UX functions for large file support you should link the
proper libraries with your executable. But that has not been done with
the Informix 5.10 executables.
We had an Informix database with about 6GB of data in a blobspace and
the dbexport invariably managed to unload just 2GB of the table that
contained the BYTE values and then it died.
What we ended up doing was our own program for exporting that particular
table and then importing the same table to the other system.
> 2. Try dbexporting to tape (not sure about this)
Basically ought to be ok, provided that the dbexport tool can understand
that tapes need to be changed and that the tape capacity is not over 2GB.
> Rudy
>
>
> David Kulkarni wrote:
>
> > On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome the 2 gig limitations and
> > dbexport a table that is over 3.0 gigs???
> >
> > =====
> > David Kulkarni
> > New York, NY 10021
> > email: dkulkarni@cgsh.com
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Mail - Free email you can access from anywhere!
> > http://mail.yahoo.com/
Just my .2 cents worth...
HTK
--
Heikki.Karhunen@kone.com
Heikki.Karhunen@iki.fi
Hi,
> > On Informix 5.10UC5 running on HPUX10.20 is there a way to overcome the
2 gig limitations and
> > dbexport a table that is over 3.0 gigs???
>
> you could try dumping the database to a compressed pipe. i have not tried
it with dbexport, but
> i have used i with onunload and onload.
I tried a time ago to export more than 2GB to a fifo, in the same way as Ty
described it, but it didn't work with dbexport (Informix Online 7.2x).
>
> something like:
>
> mknod export.pipe p
> cat export.pipe | compress > export.file.Z &
> dbexport database -t export.pipe ...
>
Heinz Werner
=========================================
Heinz Werner hwerner@hdi.de
HDI - Haftpflichtverband der Deutschen Industrie
=========================================