Re: ** FOR IMMEDIATE RELEASE - RECORD-BREAKING TPC-C RECORD!! **

In article <47dn67$ni4@pong.lasc.lockheed.com>, Terry Greenlaw <z50816@mip.lasc.lockheed.com> wrote: > >If you want to see something a little closer to reality: >1) take the huge TPC-C numbers being thrown around by everyone, > (results are available at http://www.tpc.org). >2) look at the number of users they are using to get these kind of numbers. >3) wait 'til the laughter dies down. C'mon folks, 8000 simultaneous users? >4) divide the TPC-C number from step 1 by the number of users in step 2 to > get the average TPC-C/user. > >The three platforms we looked at running Oracle7 were: >HP 9000 K410 @ > 3000 tpmC with 3300 users >HP 9000 T500 @ > 5500 tpmC with 6600 users >Dec Alpha xxxx @ 9000 tpmC with 8000 users > >When you do the division you get 1.12, 1.15, and 1.17 TPC-C/user. Not much >of a difference, really, except the price. > >For a true comparison, why not try benchmarking with the same number >(how about 100) of simultaneous users connected directly over FDDI >instead of through 16 front-end user servers, terminal concentrators, >and whatever other kind of smoke and mirror tricks you've been doing ? I am afraid you've slightly misunderstood the benchmark if you think the RDBMS vendors want to use as many users as they do. The benchmark *requires* the database size be scaled to the throughput. As the throughput increases, the database size is required to grow. The benchmark also *requires* the number of users to scale with database size. The number of connected users is not directly up to the RDBMS vendor to decide. Usually the benchmarking process goes a little like this: 1, Decide what tpmC number you wish to try to obtain. Let's use 5000 tpmC as an example. 2, Build a database with the number of warehouse table rows equal to 1/11 of the desired tpmC. In this case, we'd use a 450 warehouse row database (AKA a "scale 450" db). The 1/11 figure is a rule-of-thumb, not an absolute value. 3, This database is about 45 GB in size (using a back-of-the-envelope 1 GB per 10 warehouse rows.). 4, Start benchmarking. Using a scale 450 db, you are *required* to use 4500 users. The only way to alter this values is to change the size (scale) of your database. 5, If you have sufficient hardware and a fast RDBMS engine, you may obtain your goal. If you don't, you scale back the database size and try it again. Most unix RDBMS can not handle 5000-10,000 users gracefully. Instead, most end up using a transaction monitor to multiplex users down to somewhere in the area of 50-350 database connections. So despite the large number of connected users seen in the benchmark, usually there are only really a couple hundred actual db connections. So your point about needing 8000 users for good numbers is misleading at best and conceptually wrong. So really, the TPC-C benchmark isn't so much smoke and mirrors as you have insinuated. The terminal concentrators etc are there because the benchmark spec basically requires them to be there. An interesting point: the benchmark also specifies think times and keying times. These are used to simulate users spending time thinking and typing in queries. By using the values required by the benchmark, one can calculate the absolute maximum throughput an RDBMS can obtain. This value is somewhere in the neighborhood of 1.28 * number of users. So to see values of 1.15tpmC/user is not unreasonable since the absolute speed of light for tpmC is 1.28 tpmC/user. >Until TPC enforces the benchmark more rigidly, it's pointless to try and >compare. It is enforced extremely rigidly. You have obviously never been remotely involved in a TPC-C benchmark or you would not be saying such things. The TPC-C numbers may be compared absolutely. The only variables a customer should assume are the hardware differences and the RDBMS engine differences. Apart from this, the work done in the benchmark and the rules governing the response times, throughput, SQL executed, users, etc are identical. Pat