Re: TPC-A and -B Benchmarks

Sandor (Mr. Oracle7)" writes: |> In-Reply-To: NLUNIX:rmy.emory.edu!ilist@sun4nl.nl.oracle.com's message of 07-07-93 16:08 |> |> |> As well explained by the Standish report, Oracle has effectively eliminated |> |> TPC-As as an effective comparison through their implementation of "discrete |> |> transactions" in Version 7, a feature that is clearly there for the sole |> |> purpose of improving TPC-A results (the nature of TPC-C eliminates their |> |> use). Irrespective of this, the industry has long known that TPC-As are |> |> woefully inadequate as a true measure of database performance. Their |> |> primary strength was in the number of results published. |> The quote above was made by me, so the attribute line is a bit confusing. Anyway... |> I do not agree, if you (and the people at Standish) had a good look at what |> discrete transactions are, you would certainly see some purpose for it, |> especially in OLTP kind of systems with a large number of simple transactions. |> The same is true for other kinds of functionality present in a lot of |> databases, like hashing. Hashing is of little use for most situations, but |> there are situations where this is very useful (TPC's is accidentally one of |> them). The rumour stated in a previous message that discrete transactions are |> only available in a special version of Oracle7 is completely untrue. It is a |> standard feature, completely documented. Well, of course given your employer I would not have expected you to agree. Allow me to quote from an article on BUSINESS WIRE: "Last August, Oracle announced the ORACLE7 DBMS. ORACLE7 implements a poorly documented, special transaction model option known as the discrete transaction. Discrete transactions bypass many of the integrity features and general functionality of ORACLE7, and significantly cuts the processing "path length" of the transaction. Curiously, the integrity features remaining in the discrete transaction option are those just sufficient to execute the TPC-A benchmark test - a nototiously undemanding test. The Standish Group's research indicates that this limited functionality option could be used by very few, if any, users developing production applications. This being the case, The Standish Group beleives that the discrete transaction was implemented in ORACLE7 soley for the purpose of running the TPC-A benchmark as efficiently as possible. The Standish Group also believes that Oracle developed the discrete transaction option because ORACLE7 would not perform any better than Oracle version 6 - thus discrete transactions were implemented to provide a system bypass and "demonstrate" dramatically improved TPC-A results." And from Open OLTP Report, Vol. 4 No 3, "Foul Flag Flies for ORACLE7, TPC" "The following list exposes deficiencies in the discrete transaction option as compared to more standard ORACLE7 transaction models: 1. The discrete transaction option does not allow the application to read the records it is updating; 2. The discrete transaction option prevents changes to any database block more than once per transaction; 3. The discrete transaction option prevents changes to more than a few database blocks within a single transaction. Coincidentally, the TPC-A transaction only requires updates to four blocks; 4. The discrete transaction option requires the programmer to understand how records are physically arranged in the database; 5. The discrete transaction option may require that the physical storage of the data remain unchanged, or, if changed, the application programmer will be required to re-write the application; 6. The discrete transaction option prevents verification of updates within the transaction itself; 7. The discrete transaction option prevents queries of modified data; 8. The discrete transaction option prevents the read consistency mechanism from working; 9. The discrete transaction option does not support distributed transactions; and 10. The discrete transaction option requires special handling by the system administrator." "Even a blind squirrel will find an occasional acorn, but we couldn't find one user who would implement the restrictive functions of the discrete transaction option. Oracle claims that the discrete transaction option is built for users and not just for the TPC benchmarks, but they cannot provide us with any user references. In addition, we queried our DARTS database of 3,598 applications to match the features of the discrete transaction option with the commensurate functions in the DARTS survey and found no matches. Therefore, we have concluded that the discrete transaction option has no commercial usage merit." I'd venture to say that Standish *has* taken a good look at what discrete transaction are, and found them wanting. Dave Disclaimer: These opinions are not those of Informix Software, Inc. ************************************************************************** "I look back with some satisfaction on what an idiot I was when I was 25, but when I do that, I'm assuming I'm no longer an idiot." - Andy Rooney