Re: Sybase 11 TPCC on DEC (your comments please)

Dear Right Reverend Sir, How much do you actually know about TPC-C (or TPC-A or TPC-B) benchmarks? Not enough, I think. To reach an auditable figure of 7580 tpm, the database had to be scaled big enough; it was probably set up for either 7600 or 8000 tpm. That means, if the TPC-A and TPC-B figures are anything to go by, that the database had some 7600 * 100,000 * 100 bytes of data in the Account table, which amounts to some 76 GB of data; that won't fit into a mere 6 GB of memory. Suppose that the machine had been given 80 GB of memory; then the performance would have improved dramatically, maybe running at 5 times the reported figure; call it 40,000 tpm. But then, to get auditable results, they would have had to resize the database to suit 40,000 tpm, so the data space needed would shoot up to 400 GB. And the machine would have needed enough terminal connections for 40,000 or so users (well, it would for TPC-A; I'm not certain of the rules for TPC-C, but even 4000 users is a non-negligible load). That is a highly non-trivial number of connections. And setting up the test would be ghastly; you'd need multiple auxilliary machines simulating those hordes of users. Ugh! The whole point about the TPC benchmarks is that they deliberately force the database to be so big that it is not feasible to have the whole thing in memory. Those figures are impressive. Whether they can be beaten or not is a matter, mainly, of time and energy. And I don't know whether there are comparable figures for Informix or not. I'd suspect that OnLine has not been tested on that configuration. And your simple tests simply cannot be scaled to produce what might be a representative answer. Yours, Jonathan Leffler (johnl@informix.com) #include <disclaimer.h> >From: cjames@melchizedek.cec-services.com (The Right Reverend Colin James III) >Date: Wed, 03 Jan 1996 22:53:44 GMT >X-Informix-List-Id: <news.20023> > >cjames@melchizedek.cec-services.com (The Right Reverend Colin James >III) posted with deletions: > >| If a 300 MHz Alpha is linearly equivalent to a 133 MHz Pentium (which >| it may not be), then 300/133 or about 2.25 * 3360 tpm = 7580 tpm which >| is about 45% less than the Sybase mark. (It should be an order of >| magnitude less (10-times less or 1000% less), because my test is in a >| high-order language BASIC, not an intermediate-order language C.) >| >| The Sybase results do not impress this writer. > >Correction: I did not bother to read far enough into the puff-piece >press release to see that the DEC 8400 is not a mere $40K box. It has >10-processors and 6GB of RAM, which means the DEC probably jammed the >entire datbase into memory, ie, no physical disk I/O. > >By contrast, my linear extrapolation of 7580 tpm should be scaled up >by 10 to 75,800 tpm which would be about 7-times _faster_, at least, >than Sybase. > >Even less impressed with Sybase now.