Programming language on SUN? / Load Time
Posted in 1997
Topics: SQL Development & Query Writing, Connectivity: ESQL/C, 4GL & Embedded SQL
Please excuse this lengthy entry, but I have two questions, and I would like to be as clear a possible. I work with a data warehouse for an insurance company, and I am relatively new at this. We have a SUN Enterprise 4000 with Informix 7.23 ODS. We have an unusual amount of dimensions - 20, and one years worth of data is approx 160 million rows in our fact table. We hope to store 10 years worth of data in the warehouse. My first question concerns programming languages available on the SUN, and my second question concerns the length of time it is taking us to load our second years worth of data. #1 We currently have no programming languages on the SUN. We are open to any language that will allow us to update/process tables in our warehouse and that will allow normal flat ASCII file manipulation. We have heard of 4GL from our Informix contacts and from this news group. Does 4GL allow processing with flat ASCII files? Any other suggestions? #2 We start our load process with approx 35 mainframe tapes worth of data. We do as much processing on the mainframe as we can. Once the fact table and dimension records are built on the mainframe, we load the data, from tape using PLOADER, onto the SUN. Three of our dimensions have serial keys that force us to join our fact table records from a temporary load table to the final fact table. This is the process that is taking too long. To perform this join on our second years worth of data took us 4 days! Is this what others have experienced? Or is this insane? Would 4GL or other programming language (in question #1) help us here? Thanks for your help, Brenda Wilson