Re: ** FOR IMMEDIATE RELEASE - RECORD-BREAKING TPC-C RECORD!! **

In article <47o5v9$5an@fido.asd.sgi.com>, gainer@gainer.engr.sgi.com (Patrick Gainer) writes: > In article <47ntu8$of0@pong.lasc.lockheed.com>, > Terry Greenlaw <z50816@mip.lasc.lockheed.com> wrote: > > > >I just can't stomach anyone trying to claim that these numbers are even > >remotely related to anything that 99 percent of RDBMS users are trying > >to achieve with their systems. > > I was not claiming this. I was explaining the tpc-c benchmark. I have never > claimed 99% of users real workloads are modelled by the tpc-c benchmark. > The majority of the corporate world is buying systems based on these numbers. I thought that the intent of the TPC was to give a level playing field in which to compare dissimilar systems. Are there benchmarking groups out there doing real-world performance testing? How about RDBMS vendor vs. vendor for a given platform, or platform vs. platform for a given RDBMS with reasonable parameters. > >All the TPC-C numbers I've seen to date are for unrealistic business > >environments. My car goes 130 MPH and uses no gas. Did I mention I > >dropped it out of a B-52 bomber? > > The environments may be unrealistic but they are definately extreme. > That is to say that they probably represent the worst case. They > produce the extremely large demands on the system under test. Just because > your particular environment is much smaller and simpler doesn't mean > the benchmark isn't measuring what it was intended to measure. I would think that the large number of users skews the numbers higher as opposed to being worse case. The vendors are using big dollar hardware to parallelize the transactions and artifically crank up their tpc-c results. If it really was a worse case scenario, than a single user going directly against the DEC Alpha box they tested would get MORE than 9,000 tpc-c. That's why I was dividing the tpc-c by the number of users to show what each user would expect from the system. > Your analogy is flawed. A more similar analogy (if you insist upon > using cars) is the most recent Car and Driver issue. In it they compare > a camaro against a mustang in several tests: drag racing and an open track > being two of these. Nobody would argue that this is how one should decide > which car to buy (because these tests don't realistically simulate normal > driving conditions) but it does accurately test the both cars at their > limits. To continue the analogy, you would then reply that these tests > are bogus because 99% of users don't drive on open tracks. Of course > that is true but you have missed the point. The point is to illustrate > the system behavior under extreme conditions. One would hope an > intelligent user would use more than a tpc-c benchmark number to select > an RDBMS. > > Pat The intelligent users usually don't have a lot of input in a large company when it comes to multimillion dollar investments. VP's and CIO's 12 levels removed from technology are making purchases based on heavily biased sales reps, potentially biased magazine and journal articles, and input from only slightly less technologically removed direct reports. Most of these guys are basing their decision on: 1) How much does it cost? 2) How fast does it go? (currently being equated to tpc-c) WRT the car analogy, a lot of people DO buy cars because they go fast (all the car manufacturers are pitching horsepower in their ads nowadays), or look pretty, or are cheap. By the same token, alot of IS people use tpc-c just like horsepower. I agree with you wholeheartedly that this is a bad thing, but it will continue, and it would be nice to have a benchmark that was a little more grounded in reality. Terry