Re: blobspaces
On Thu, 14 Aug 1997, Chao Y. Din wrote: > Art, > > Your response is well written and I will keep a permanent record of your > post. However, I must disagree with some of your statements. > > Art S. Kagel wrote: > > > > Sujit Pal wrote: > > > > > > Maria > > > > > Also BLOBspaces are locked during archives since there are no log > > pages to insure Archive consistency. > > I am not sure I understand you correctly. We do database backup during > office hour. It usually takes a whole day to backup one server. Sorry it has been so long since we used BLOBSPACE blobs that I have forgotten that 7.xx uses a different strategy from 5.0x. From the 5.0x Administrator's Guide (2-141): "During an archive, the tbtape process blocks allocation of blobspace blobpages in a chunk until it has read the chunk and archived all used blobpages therein." Rel 7.0x removed this problem by logging the overhead pages and not freeing the logfile until it has been backed up at which time the pre-modification versions of changed blobpages are copied directly from blobspace and then freed for reuse. > > > Also when BLOBs are updated ALL of the BLOB pages are > > actually copied to another location with the changes then the original > > I totally agree with you. > > > So, in summary: > > > 1) If the BLOBs are to be repeatedly FETCHED over a relatively short > > timeframe (like the whole department looking at this morning's status > > report) favor tablespace BLOBS. > > As stated above, I do not agree on this part. Well, two years ago the Loraine Bobbit story brought our system to a complete halt when 40,000 users all asked to see the same uncached story blob within a few seconds of each other. There were other system problems that contributed to the halt but the blob buffering was a big part as the next story reveals. About a year later, after fixing the problems with our proprietary ipc mechanisms which contributed to the "Bobbit" problem were fixed the leader of the "Grateful Dead" passed away. Then 55,000 users asked to read that story over a 60 second period. The stories were still in Blobspace but the other problems were eliminated so everyone got to see their story it just too up to 10 minutes! Today with stories in tablespace we could respond to all 78,000 users asking to see the same few stories in Bloomberg's famous 10 second maximum response time. > > > 2) If a large percentage of BLOBs will be updated at least once during > > their lifetime, favor tablespace BLOBS. > > Again, I do not agree on this part. Considering our image size 80K in a > single blobpage, both read and write require a single disk access > (actually, two disk read/writes, I believe). If it is allocated on > tablespace, chances are the image may not occupy contigeous disk space An 80K image takes up 40 buffers. Our news story sizes have 2 peaks 40% of all stories are 2-4K while 30% of the remaining stories are 40K, so our databases are not so different. We run with 200,000 buffers and so can cache 5,000 stories with 100,000 buffers free for index and lookup table pages. Realistically only the most recent 100 stories are frequently looked at so we are essentially caching all active stories for several hours each before something more interesting finally forces the pages to be reused when interest wanes on the oldest stories. Index and lookup pages are just too active to ever get swapped out. If you have the memory to give, I still maintain that tablespace blobs and a large buffer cache is the way to go. If you tracked how many BLOBs are actively fetched during the day and the total BLOB pages written between checkpoints you will probably find that there is a cache size that can dramatically improve performance. > > > 3) If you cannot perform Archive offline, or at least without > > inserting/updating BLOBS, favor tablespace BLOBS. > > As stated above, I do not agree. You are right for ODS 7.xx but I am correct for Online 5.0x. This was my fault. > > > 5) Otherwise BLOBspace BLOBS can be faster so favor BLOBspace BLOBS. > > > > Weight any INSERT performance gain against these other factors. Is > > As I stated above, it is not feasible to buffer large volume of large > size images. Only the number of BLOB pages written from one checkpoint to the next, or from one LRU flush to the next actually, will be dirty and active pages like index pages, lookup table pages, and the most active BLOB pages will not be flushed out. Mostly the pages which are reused will be for Blobs less recently written and never fetched. > > Art S. Kagel > > Chao Y. Din > Art S. Kagel, kagel@bloomberg.com