Re: data warehousing discussion

nnaas@mailhost.ecn.uoknor.edu (Nadia Naas) wrote: >"The technology supporting backup and recovery, transaction and data integrity, >and the detection and remedy of deadlock is quite complex and unnecessary for >for data warehouse processing.". >Backup and recovery seem as an important issue in data warehousing for me. All of the above issues are still relevant to data warehousing (except may be deadlocks) but may be in a different context. For example, in a data warehouse you might not need to log transactions but you might want to log records as you are loading them into the database to allow to restart your load jobs if required. Following are some reasons why you might back/recovery and other mechanism to preserve data integrity: - For example in a large data warehouse you still want to have ability to recover from database dumps otherwise you will have to reload all the data from scratch. Does not look like a good option if you have several years of historical data in your warehouse. - Because of the data volumes you also want ability to perform incremental backups and restores. - You will require recovery mechanisms which will allow you to restart database load jobs which failed halfway through. - If your warehouse database does not have decent backup and recovery mechanism you might end up spending a lot of time trying to rebuild your warehouse. in some situation from scratch. - If you cannot maintain the integrity of your data in the warehouse your data quality will suffer!!! What happens if you load data in your fact table with data that does not reference existing dimensions values? Because we are dealing with large data volumes and different requirements to a conventional databases there several new issues that need to be addressed by database vendors venturing into data warehousing. ---------------------------------------------------------------------------------------------------------------------- Rafal Czerniawski Dataspace Consulting Pty Ltd rafal@dataspace.com.au