Dbload and Load: Maximum File Size

I seem to remember there perhaps being a maximum file size or record count and want to do if I need to start figuring out ways to make my unload files smaller. I have blobspaces over 16 GB in size and some normal tables that I believe will be over 2GB in size when unloaded. As I do not have a test bed available to load these back in at this time, I thought I would ask the group. I also need a good way of determining how much disk space I will need for unloads. I know of sum(nrows*rowsize) for all non-system tables; is there a better way, especially with the tables using those blobspaces? I will be unloading from a v5 and loading into a v10, both having OS AIX 5.2. Thanks in advance. Take care. Clifton M. Bean Informix DBA / AIX System Admin Currency Technics & Metrics 1431 Greenway Drive #700 Irving, Texas 75038 Main (972) 812-1411 x244 Toll Free (800) 834-8807 x244 Fax (469) 417-0665

Hi, from v5 to v10 seems a huge step. First you could try if you can create synonyms in the v10-database for the v5-tables. If you can do that (there might be issues with LANG parameters) you could move your data with insert into <table> select * from <synonym> (you must run both database without locking and lock the target table in exclusive mode). Bye Andreas > ------------------------------------------- SPAR Oesterreichische Warenhandels-AG Hauptzentrale Europastrasse 3 A - 5015 Salzburg Tel: +43 662 4470 24223 Mobile: +43 664 6259575 E-Mail: Andreas.KUTSCHE@spar.at Internet: http://www.spar.at Wichtiger Hinweis: Der Inhalt dieser E-Mail kann vertrauliche und rechtlich geschützte Informationen, insbesondere Betriebs- oder Geschäftsgeheimnisse, enthalten, zu deren Geheimhaltung der Empfänger verpflichtet ist. Die Informationen in dieser E-Mail sind ausschließlich für den Adressaten bestimmt. Sollten Sie die E-Mail irrtümlich erhalten haben so ersuchen wir Sie, die Nachricht von Ihrem System zu löschen und sich mit uns in Verbindung zu setzen. Über das Internet versandte E-Mails können leicht manipuliert oder unter fremdem Namen erstellt werden. Daher schließen wir die rechtliche Verbindlichkeit der in dieser Nachricht enthaltenen Informationen aus. Der Inhalt der E-Mail ist nur rechtsverbindlich, wenn er von uns schriftlich bestätigt und gezeichnet wird. Sollte trotz der von uns verwendeten Virus-Schutzprogramme durch die Zusendung von E-Mails ein Virus in Ihre Systeme gelangen, haften wir nicht für evtl. hieraus entstehende Schäden. Wir danken für Ihr Verständnis. Important notice: The contents of this e-mail may contain confidential and legally protected information that is in particular related to operational and trade secrets, which the recipient is obliged to treat as confidential. The information in this e-mail is made available exclusively for use by the addressee. In the event that the e-mail may have been sent to you in error, we would ask you to kindly delete this communication from your system and to contact us. E-mails sent via the Internet can be easily manipulated or sent out under someone else's name. We therefore do not accept legal liability for the information contained in this communication. The contents of the e-mail are only legally binding if they have been confirmed and signed by us in writing. If, in spite of our using Antivirus protection software, a virus may have penetrated your system through the sending of this e-mail, we do not accept liability for any damage that may possibly arise as a result of this. We trust that you appreciate our position. ------------------------------------------- -----Ursprüngliche Nachricht----- > Von: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org]Im Auftrag von > Clifton Bean > Gesendet: Dienstag, 19. Dezember 2006 16:05 > An: ids@iiug.org > Betreff: Dbload and Load: Maximum File Size [8026] > > > > I seem to remember there perhaps being a maximum file size or > record count > and want to do if I need to start figuring out ways to make > my unload files > smaller. I have blobspaces over 16 GB in size and some normal > tables that I > believe will be over 2GB in size when unloaded. As I do not > have a test bed > available to load these back in at this time, I thought I > would ask the > group. > > I also need a good way of determining how much disk space I > will need for > unloads. I know of sum(nrows*rowsize) for all non-system > tables; is there a > better way, especially with the tables using those blobspaces? > > I will be unloading from a v5 and loading into a v10, both > having OS AIX > 5.2. > > Thanks in advance. > > Take care. > > Clifton M. Bean > Informix DBA / AIX System Admin > Currency Technics & Metrics > 1431 Greenway Drive #700 > Irving, Texas 75038 > > Main (972) 812-1411 x244 > > Toll Free (800) 834-8807 x244 > > Fax (469) 417-0665 > > > ************************************************************** > ***************** > Forum Note: Use "Reply" to post a response in the discussion forum. > >

On 19/12/06, Andreas.KUTSCHE@spar.at <andreas.kutsche@spar.at> wrote: > > Hi, > > from v5 to v10 seems a huge step. First you could try if you can > create synonyms in the v10-database for the v5-tables. If you can > do that (there might be issues with LANG parameters) you could move > your data with > insert into <table> select * from <synonym> > (you must run both database without locking and lock the target table > in exclusive mode). > > Bye > Andreas > > > > ------------------------------------------- > SPAR Oesterreichische Warenhandels-AG > Hauptzentrale > Europastrasse 3 > A - 5015 Salzburg > > Tel: +43 662 4470 24223 > Mobile: +43 664 6259575 > E-Mail: Andreas.KUTSCHE@spar.at > Internet: http://www.spar.at > > Wichtiger Hinweis: Der Inhalt dieser E-Mail kann vertrauliche und rechtlich > geschützte Informationen, insbesondere Betriebs- oder Geschäftsgeheimnisse, > enthalten, zu deren Geheimhaltung der Empfänger verpflichtet ist. Die > Informationen in dieser E-Mail sind ausschließlich für den Adressaten > bestimmt. Sollten Sie die E-Mail irrtümlich erhalten haben so ersuchen wir > Sie, die Nachricht von Ihrem System zu löschen und sich mit uns in Verbindung > zu setzen. > Über das Internet versandte E-Mails können leicht manipuliert oder unter > fremdem Namen erstellt werden. Daher schließen wir die rechtliche > Verbindlichkeit der in dieser Nachricht enthaltenen Informationen aus. Der > Inhalt der E-Mail ist nur rechtsverbindlich, wenn er von uns schriftlich > bestätigt und gezeichnet wird. > Sollte trotz der von uns verwendeten Virus-Schutzprogramme durch die Zusendung > von E-Mails ein Virus in Ihre Systeme gelangen, haften wir nicht für evtl. > hieraus entstehende Schäden. > Wir danken für Ihr Verständnis. > > Important notice: The contents of this e-mail may contain confidential and > legally protected information that is in particular related to operational and > trade secrets, which the recipient is obliged to treat as confidential. The > information in this e-mail is made available exclusively for use by the > addressee. In the event that the e-mail may have been sent to you in error, we > would ask you to kindly delete this communication from your system and to > contact us. > E-mails sent via the Internet can be easily manipulated or sent out under > someone else's name. We therefore do not accept legal liability for the > information contained in this communication. The contents of the e-mail are > only legally binding if they have been confirmed and signed by us in writing. > If, in spite of our using Antivirus protection software, a virus may have > penetrated your system through the sending of this e-mail, we do not accept > liability for any damage that may possibly arise as a result of this. > We trust that you appreciate our position. > > ------------------------------------------- > -----Ursprüngliche Nachricht----- > > > Von: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org]Im Auftrag von > > Clifton Bean > > Gesendet: Dienstag, 19. Dezember 2006 16:05 > > An: ids@iiug.org > > Betreff: Dbload and Load: Maximum File Size [8026] > > > > > > > > I seem to remember there perhaps being a maximum file size or > > record count > > and want to do if I need to start figuring out ways to make > > my unload files > > smaller. I have blobspaces over 16 GB in size and some normal > > tables that I > > believe will be over 2GB in size when unloaded. As I do not > > have a test bed > > available to load these back in at this time, I thought I > > would ask the > > group. > > > > I also need a good way of determining how much disk space I > > will need for > > unloads. I know of sum(nrows*rowsize) for all non-system > > tables; is there a > > better way, especially with the tables using those blobspaces? > > > > I will be unloading from a v5 and loading into a v10, both > > having OS AIX > > 5.2. > > > > Thanks in advance. > > > > Take care. > > > > Clifton M. Bean > > Informix DBA / AIX System Admin > > Currency Technics & Metrics > > 1431 Greenway Drive #700 > > Irving, Texas 75038 > > > > Main (972) 812-1411 x244 > > > > Toll Free (800) 834-8807 x244 > > > > Fax (469) 417-0665 > > Clifton If you can set it up then an sql unload to a named pipe in one session with a dbload from that pipe is quick, efficient and doesn't need masses of disk space to hold intermediate data (it is consummed as soon as it is produced). I shifted 300 Gb of data between two machines (one pretty slow) in concurrent streams last year. Keith

Clifton, I saw an item a while back where someone measured the database size by unloading to a pipe to dd, and then piping to /dev/null. When it was done, dd printed out the number of bytes that had passed through. Voilá! You have your measurement, and it is pretty quick. Scott MacKenzie - Dine' College ITD Phone/Voice Mail: 928-724-6639 Email: scottm at dinecollege dot edu -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Clifton Bean Sent: Tuesday, December 19, 2006 8:05 AM To: ids@iiug.org Subject: Dbload and Load: Maximum File Size [8026] I seem to remember there perhaps being a maximum file size or record count and want to do if I need to start figuring out ways to make my unload files smaller. I have blobspaces over 16 GB in size and some normal tables that I believe will be over 2GB in size when unloaded. As I do not have a test bed available to load these back in at this time, I thought I would ask the group. I also need a good way of determining how much disk space I will need for unloads. I know of sum(nrows*rowsize) for all non-system tables; is there a better way, especially with the tables using those blobspaces? I will be unloading from a v5 and loading into a v10, both having OS AIX 5.2. Thanks in advance. Take care. Clifton M. Bean Informix DBA / AIX System Admin Currency Technics & Metrics 1431 Greenway Drive #700 Irving, Texas 75038 Main (972) 812-1411 x244 Toll Free (800) 834-8807 x244 Fax (469) 417-0665 ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

If you want the size of the raw data, then this would be useful. However, it doesn't account for index size, or the actual size of the data on disk (including overhead). You might want to just read sum(npused) from sysptnhdr - with appropriate filters for the tables/databases you are interested in. j. ----- Original Message ----- From: "Scott Mackenzie" <scottm@dinecollege.edu> To: <ids@iiug.org> Sent: Tuesday, December 19, 2006 10:54 AM Subject: RE: Dbload and Load: Maximum File Size [8035] > > Clifton, > > I saw an item a while back where someone measured the database size by > unloading to a pipe to dd, and then piping to /dev/null. When it was > done, dd printed out the number of bytes that had passed through. > > Voilá! You have your measurement, and it is pretty quick. > > Scott MacKenzie - Dine' College ITD > Phone/Voice Mail: 928-724-6639 > Email: scottm at dinecollege dot edu > > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Clifton > Bean > Sent: Tuesday, December 19, 2006 8:05 AM > To: ids@iiug.org > Subject: Dbload and Load: Maximum File Size [8026] > > I seem to remember there perhaps being a maximum file size or record count > and want to do if I need to start figuring out ways to make my unload files > smaller. I have blobspaces over 16 GB in size and some normal tables that I > believe will be over 2GB in size when unloaded. As I do not have a test bed > available to load these back in at this time, I thought I would ask the > group. > > I also need a good way of determining how much disk space I will need for > unloads. I know of sum(nrows*rowsize) for all non-system tables; is there a > better way, especially with the tables using those blobspaces? > > I will be unloading from a v5 and loading into a v10, both having OS AIX > 5.2. > > Thanks in advance. > > Take care. > > Clifton M. Bean > Informix DBA / AIX System Admin > Currency Technics & Metrics > 1431 Greenway Drive #700 > Irving, Texas 75038 > > Main (972) 812-1411 x244 > > Toll Free (800) 834-8807 x244 > > Fax (469) 417-0665 > > > **************************************************************************** *** > Forum Note: Use "Reply" to post a response in the discussion forum. > > > **************************************************************************** *** > Forum Note: Use "Reply" to post a response in the discussion forum. >