Size limitation on dbload?
Posted in 2003
Topics: General Discussion
We are using dbload for our ETL process. We ran into an error today
for the first time on an existing process.
The error is
Cannot open raw data file -> (dirname)/(filename).
error occurred: 1
The file is 3 GB, and 3 million records. This is much larger than our
normal load.
Could we be running into size limitations?
Many older operating systems have 2 GB file size limit. It is possible that you ran into it. Since you haven't stated the exact
operating system name and version and IDS version, I can't tell if this is your problem.
Gorazd
<generic@arigatoo.net> wrote in message news:1194685a.0312301424.4d2b553d@posting.google.com...
> We are using dbload for our ETL process. We ran into an error today
> for the first time on an existing process.
>
> The error is
>
> Cannot open raw data file -> (dirname)/(filename).
> error occurred: 1
>
> The file is 3 GB, and 3 million records. This is much larger than our
> normal load.
>
> Could we be running into size limitations?
generic@arigatoo.net wrote:
> We are using dbload for our ETL process. We ran into an error today
> for the first time on an existing process.
>
> The error is
>
> Cannot open raw data file -> (dirname)/(filename).
> error occurred: 1
>
> The file is 3 GB, and 3 million records. This is much larger than our
> normal load.
>
> Could we be running into size limitations?
Yes.
Starting with 9.40, the utilities support big files on all systems
that support big files (and since you have a big file to load, you
must have such support). Prior to that, the utilities used the 32-bit
limited I/O calls, which made 2**31-1 bytes the maximum size of file
that the utilities could access, even on a system with support for
bigger files.
Of course, as usual, version and platform information would have helped!
--
Jonathan Leffler #include <disclaimer.h>
Email: jleffler@earthlink.net, jleffler@us.ibm.com
Guardian of DBD::Informix v2003.04 -- http://dbi.perl.org/