Re[2]: Top Ten Errors In Data Wareousing

> > It seems to me that a significant amount of analysis work would be needed > on the TPS side to ensure that data that is summarized on the warehouse > side, is not updated on the TPS side at a later date. Is this correct? In > a previous job, we looked at this "functionality". However we ran into the > problem of "gee, we really need to update the data from two years ago" We > now need to re-run all the monthly reports for the previous two-years, and > store the new numbers in the warehouse. Since this was a common > occurrence, (on a newly deployed system), and the reprocessing of reports > would significantly affect the production times, we decided to wait until > the data was more "stable". > A great deal of analysis work does indeed go into the warehouse as well as in coding the 'strip' programs or 'fetch' programs (depending on whether you are pushing or pulling). The guesstimate is that 80% of the effort goes here. One of our criteria for inclusion of data into the warehouse is that it be somewhat stable. Even so you have to plan to handle some corrections. Our plan is to allow for these corrections and to monitor them. As the number of corrections increases a corresponding 'confidence in the data' value decreases for that data. That 'confidence' number is kept in plain view where the user can easily see it. In our warehouse itself is kept a 'current' view of the data along with other historical views as necessary. This 'current' view can be corrected, the historical views cannot. At any time a view can be requested that is effective as_of a given date - this is rebuilt from the archives. It does not overwrite the 'current' date, but occupies its own table space. Better ideas? j. _____________________________________________________________________________ Jack Parker - Hewlett Packard, BSMC Boise, Idaho, USA jparker@hpbs3645.boi.hp.com _____________________________________________________________________________ Subtlety is the art of saying what you think and getting out of the way before it is understood. _____________________________________________________________________________ Any opinions expressed herein are my own and not those of my employers. _____________________________________________________________________________