DBD versus dblist

>> Jacob Davenport wrote: >> We use a heavily modified version of dblist, originally written >> by David Snyder, to extract data from our Informix databases. It >> is a compiled c program using esql libraries. Jonathan Leffler replied: >The real dblist program is a simple, compact ESQL/C program. >DBD::Informix has much the same core code, but also has to deal with >some idiosyncracies of the DBI spec, and then gets a layer of Perl >wrapped around the top. No, I'm not very surprised that a custom >program like dblist can outperform DBD::Informix. The code looks slim >enough that dblist probably outperforms SQLCMD on raw speed - but >SQLCMD does a whole heap of things that DBLIST cannot do. Agreed. We have a number of small utilities for other aspects of database manipulation. dblist is just for our select statements. But I'm still surprised because, compared to the time it takes to get data from the database, Perl is very fast. I have found many programs written by my peers that have complicated queries, and I speed them up significantly by making the queries very simple and then manipulating the data in Perl. Time and time again, benchmarks show that Perl runs in milliseconds and dblist runs in seconds. The way our dblist variant works is to create a pipe delimited string with one record per row, and then our Perl code must split it on carriage returns and then split it again on pipes. If any of our data has pipes or carriage returns in it, we've got strange and unnatural work-arounds. In my benchmarking, I compare DBD::Informix to running dblist, doing the splits, and putting the data into an array of hashes. All that extra Perl manipulation adds practically no time, even for thousands of rows. So, I'll believe you that DBI imposes extra layers of Perl code that significantly impact all database queries, but I'll still be surprised and, frankly, wonder why anyone would use DBD::Informix. This is all very frustrating for me, because I think DBI is the right design choice, and I can sell any design choice to my managers if it runs as fast or faster. -Jacob