HUGE DB
Posted in 1995
Topics: Storage & Space Management
I am working on some initial architecture and proof of concept tasks and am hoping to get some related experiences from this group. I have a DB that I consider huge. Each record will be 222 bytes and their will be about 1.5 million records per day. That works out to be about 333mb/day of storage needed. (without indexes etc ) Thats 10.3 gig of space for 31 day months. I have several questions, Disk space is cheap now days but not that cheep. 1) I need to weight the pros and cons about indexes, ie processing time vs storage. How can one calculate the size of indexes chossen ? (ie tempory sort storage vs static index storage ) 2) Additional storage overhead needed ? I am guessing this will be such a small fraction of the overall storage I don't need to worry but I want to be sure. 3) Data aggregation may be possible but I would still like to keep the raw data on tape somewhere. I need to come up with a plan of how to deal with archiving the data. Their will be daily weekly and monthly reports run from cron jobs. It's the special "get data for me by tomorrow" kind of reports that scare me, the VP's like 100 and 200 day moving averages kind of reports. That means the archiving architecture needs to be sound. Anyone have any experance with DB's this size ? Thanks, Bert