TPC-C Real Story

Hi, I only sign on to Delphi on weekends, and I received several complaints about my brevity in my comments on the TPC-C. An Informix marketeer was wondering why Oracle needed to use 7.3 of their software to run the tests, ie. performance in today's release wasn't up to snuff. I figured someone from Oracle would respond so I wouldn't have to: I'm a little lazy sometimes. Here's my opinion, and the facts as I understand them: TPC's have never been the best indicator of real world performance. Unfortunately, many people are somewhat influenced by the results of these tests. In TPC-A, Oracle came out on top, and greatly exaggerated the significance of the test results in some really obnoxious advertisements. Other database vendors on the TPC council started exerting their influence on the make-up of future test requirements. They successfully added a requirement to TPC-C for a certain type of transaction known as SERIALIZEABLE, which they knew was not one of the transaction isolation levels implemented by Oracle. Please ask for re-posting from the people much smarter than me about why this isolation level is important: Rich Finkelstein made a big deal about it recently. Here's what is interesting: Even though official TPC-C results required that the vendor demonstrate a serializeable transaction, the actual performance part of the test was NEVER run in that mode by ANYONE. The cost in loss of concurrency and performance was too great, so no one ever used this "feature". So, the payback began. Ads from Sybase and Informix that touted TPC-C as real world, and Oracle doesn't run them for performance reasons. In my opinion, although TPC-C is closer to real world than TPC-A, it still is not an indicator of real world performance. This was just "turnabout is fair play" behavior in the ad campaigns. I really wish the childish behavior would end, but now that Oracle has good results they quickly reverted to the old ad campaigns. Caveat Emptor: Your mileage may vary. Dan