Re: dbaccess unload to a pipe
DL Redden wrote:
> Here is my code. I should clarify that I don't close
> the pipe. I close the file that I'm writing to. Thank
> everyone for your input.
>
> #!/opt/perl5/bin/perl
Add -w to that line - it warns you about some possible problems not
covered by 'use strict;'.
> use strict;
> use MyModule;
What's in MyModule?
> use Getopt::Long;
Where do you use Getopt::Long?
> use vars qw($fifo $file @table $filesize $filenum $key
> $line);
>
> # Set my FIFO device to whatever name is given me
> $fifo = shift;
> $key = shift;
Chances are you're OK, but it is as well to make sure. For example,
were you passed at least 2 arguments?
> # Check to make sure that I received a name
> unless(-p "$fifo"){
> die "I cannot start unless you give me a fifo device
> to listen to.\\n";
> }
>
> # Open my FIFO device
> open(FIFO, "<$fifo");
And there's no possible reason you could fail to open the fifo?
open(FIFO, "<$fifo") || die "Aargh!";
> # Until the world ends do this stuff
> while (1){
> # If this is my code to die then die
> if ($line =~ /REST_NOW_$key/o){
> close(FIFO);
> exit;
> }
On the first cycle, $line is unset - nothing has been read into it...
So, the preceding test is redundant, isn't it?
> # If I recieve a line from my FIFO device
'i' before 'e' except after 'c' :-)
> if ($line = <FIFO>){
Now initialize $line...
> # If this is my code to die then die
> if ($line =~ /REST_NOW_$key/o){
> close(FIFO);
> exit;
> } else {
> # Until the current line tells me to close the
> unload file do this stuff
> until ($line =~ /END_TABLE_$key/o || $line =~
> /REST_NOW_$key/o){
> # If the current line is the start of a table
> unload then set up my little world
> if($line =~ /START_TABLE_$key/o){
OK - so, we need to understand what the unloader is doing. All else
apart, it is not DB-Access, nor any of the other regular Informix
programs.
It generates a key - I'll assume the key is ABCD - and then generates
output like:
START_TABLE_ABCD=table_name
data-line.
data-line..
END_TABLE_ABCD
...possibly repeat for more tables...
REST_NOW_ABCD
There's no actual requirement that the tag words start at the
beginning of the line (you didn't include ^ at the start of the
expressions). However, the key is long enough and weird enough that
we don't worry about one of the tags appearing in the middle of the data.
> # Trim and split the current line
> chomp($line);
> @table = split(/=/o,$line);
> # Set the file name = to the current table name
> $file = $table[1];
There's no possibility of malformed data?
> # Set the filesize to 0 and the file number to 1
> $filesize = 0;
> $filenum = 1;
> # Open the output file handle with the current
> table name
> open(FILE,">$file.unl");
There's no chance of failing to create the file?
> # Write this filename to the .ctrl file
> open(CTRL,">$file.ctrl");
> print CTRL "$file.unl\\n";
> close(CTRL);
So, you want to clobber the control file on the first pass for each
table? Later, you append to $file.ctrl...And there's no chance of
failing to create $file.ctrl?
> # select the output file handle and change it to
> unbuffered
> select((select(FILE), $| = 1)[0]);
Hmmmm; this smacks of a desparation measure that didn't seem to
work...I'd also have to seriously manual bash to know what that does.
> } else {
> # Print the current line to the unload file
> print FILE "$line";
> # Add the current line length to the filesize
> $filesize += length($line);
> # If the file size is larger than 1900000000
> bytes
> if ($filesize > 1900000000){
> # Close my unload file
> close(FILE);
> # Open a new unload file with the file number
> appended to the table name
> open(FILE,">${file}_${filenum}.unl");
> # Write this new file name to the .ctrl file
> open(CTRL,">>$file.ctrl");
> print CTRL "${file}_${filenum}.unl\\n";
> close(CTRL);
> # select the output file handle and change it to
> unbuffered
> select((select(FILE), $| = 1)[0]);
> # Add 1 to the file number in case we need
> another file later
> $filenum++;
> # Reset the file size to 0
> $filesize = 0;
> }
> }
> # Read the next line
> $line = <FIFO>;
No possibility of EOF before REST_NOW_ABCD appears? And the
following close brace is matched with
> }
> }
> # Close my unload file
> close(FILE);
> }
> }
One thing you could try is just printing out line numbers as you read
lines. And diagnosing when you switch files. I note, too, that if
your unload data is exactly the wrong size, then you get an empty file
created. That's the least of your problems while data is vanishing.
OK - so I modified your code to include -w and
use constant MAXFILESIZE => 19;
and changed the references to 1900000000 to MAXFILESIZE (it's easier
to generate test cases like this).
I created a test file:
START_TABLE_ABCD=data
row 1..................................finished
row 2..................................finished
row 3..................................finished
END_TABLE_ABCD
START_TABLE_ABCD=text
row 1==================================finished
row 2==================================finished
row 3==================================finished
END_TABLE_ABCD
REST_NOW_ABCD
And a fifo, and ran:
$ mkfifo fifo
$ (cat datafile > fifo) & perl script.pl fifo ABCDValue of <HANDLE> construct can be "0"; test with defined() at
script.pl line 29.
Use of uninitialized value in pattern match (m//) at script.pl line 24.
$
The warning at line 24 arises from the uninitialized $line just inside
the 'while (1)' loop; remove that and the warning goes away.
I altered the value of MAXFILESIZE, and increased the data in each
table to 9 lines (of 48 characters each). No obvious flaws in the
logic. But I got nowhere near filling up the fifo...
So, replace the data file with a perl script, gen.pl, to generate the
data...
#!/bin/perl -w
@list = ("data", "text");
foreach $name (@list)
{
print "START_TABLE_ABCD=$name\\n";
for ($i = 0; $i < 10000; $i++)
{
printf("row %4d %s finished\\n", $i, ("." x 100));
}
print "END_TABLE_ABCD\\n";
}
print "REST_NOW_ABCD\\n";
Ran that:
(perl gen.pl > fifo) & perl script.pl fifo ABCD
Got 10000 lines of output in the data*.unl files, and another 10000
lines of output in the text*.unl files. No obvious skipping...
So, you script appea