Re: Informix beats Oracle
Not a support question but a debate sparked by a link to informixcity.com claiming Informix outperforms Oracle. An Oracle advocate dismissed it, citing Oracle's licence ban on unapproved benchmark publication, wrong kernel settings and no published test source; others challenged that the cited PDFs were Informix-only and that Oracle won't install with bad kernel params. Discussion moved to whether audited TPC benchmarks are meaningful (argued to test hardware and use 'benchmark specials' like hashing, which IDS lacks, so it can't enter TPC-A/B/C) versus Informix's real strength in low admin cost and reliability. No conclusion or resolution is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Data Cruncher wrote: > http://www.informixcity.com/news6.aspx > > I am bit anxious to know the legality of this > result. Doesn't oracle have a EUA about publishing > benchmark results without their written approval. Correct. A read of the PDFs shows two things. 1. The kernel parameters were not set per Oracle's docs. 2. No source code for the test and as everyone knows a decent developer can manufacture any result they want by using the most or least efficient method of accomplishing a task. It is so lame I expect Oracle will just ignore it. I know I will. -- Daniel A. Morgan University of Washington damorgan@x.washington.edu (replace x with u to respond)
DA Morgan said: > Data Cruncher wrote: >> http://www.informixcity.com/news6.aspx >> >> I am bit anxious to know the legality of this >> result. Doesn't oracle have a EUA about publishing >> benchmark results without their written approval. > > Correct. > > A read of the PDFs shows two things. > > 1. The kernel parameters were not set per Oracle's docs. Which PDF was that, Daniel? The only PDFs I could see referred to Informix-only "benchmarks". They appear to be entirely unrelated to the main article. As far as I can tell, Oracle won't even install if you don't set the kernel up as per the release notes, so this is a spurious bit of misdirection on your part. > 2. No source code for the test and as everyone knows a > decent developer can manufacture any result they want > by using the most or least efficient method of > accomplishing a task. Just for the sake of the argument, Daniel, given that document talks about TPS, how would you feel if the benchmark used was a TPC-A or TPC-B style, bog-standard, clearly defined application? One where there was no real variation possible, or where they might have used a reference application? IF THAT WERE THE CASE, would you feel that this comment might also be null and void? But having said all that, like Daniel, I fear that this "benchmark" does nothing to substantiate the claims made and it would be lovely if the world was allowed to see the raw facts. I really wish IBM would stump up for a proper fully-audited benchmark. Just out of interest, Daniel, what's your personal best time for installing Oracle, given a raw, basic Unix box with nothing else loaded? The whole enchilada? Or don't you do installs? If not, what's the best time you've ever seen? -- Bye now, Obnoxio "I'm astonished anyone pays real money for this crap." -- Cosmo -- This message has been scanned for viruses and dangerous content by OpenProtect(http://www.openprotect.com), and is believed to be clean.
On 17 Jun, 19:52, DA Morgan <damor...@psoug.org> wrote: > Data Cruncher wrote: > >http://www.informixcity.com/news6.aspx > > > I am bit anxious to know the legality of this > > result. Doesn't oracle have a EUA about publishing > > benchmark results without their written approval. > > Correct. > > A read of the PDFs shows two things. > > 1. The kernel parameters were not set per Oracle's docs. > > 2. No source code for the test and as everyone knows a > decent developer can manufacture any result they want > by using the most or least efficient method of > accomplishing a task. > > It is so lame I expect Oracle will just ignore it. > I know I will. > -- > Daniel A. Morgan > University of Washington > damor...@x.washington.edu (replace x with u to respond) Your post is lame, which kernel parameter did not match which Oracle doc?
On Jun 17, 12:32 pm, "Obnoxio The Clown" <obno...@serendipita.com> wrote: > DA Morgan said: > > > Data Cruncher wrote: > >>http://www.informixcity.com/news6.aspx > > >> I am bit anxious to know the legality of this > >> result. Doesn't oracle have a EUA about publishing > >> benchmark results without their written approval. > > > Correct. > > > A read of the PDFs shows two things. > > > 1. The kernel parameters were not set per Oracle's docs. > > Which PDF was that, Daniel? The only PDFs I could see referred to > Informix-only "benchmarks". They appear to be entirely unrelated to the > main article. > > As far as I can tell, Oracle won't even install if you don't set the > kernel up as per the release notes, so this is a spurious bit of > misdirection on your part. > > > 2. No source code for the test and as everyone knows a > > decent developer can manufacture any result they want > > by using the most or least efficient method of > > accomplishing a task. > > Just for the sake of the argument, Daniel, given that document talks about > TPS, how would you feel if the benchmark used was a TPC-A or TPC-B style, > bog-standard, clearly defined application? One where there was no real > variation possible, or where they might have used a reference application? > IF THAT WERE THE CASE, would you feel that this comment might also be null > and void? > > But having said all that, like Daniel, I fear that this "benchmark" does > nothing to substantiate the claims made and it would be lovely if the > world was allowed to see the raw facts. > > I really wish IBM would stump up for a proper fully-audited benchmark. > > Just out of interest, Daniel, what's your personal best time for > installing Oracle, given a raw, basic Unix box with nothing else loaded? > The whole enchilada? > > Or don't you do installs? If not, what's the best time you've ever seen? > > -- > Bye now, > Obnoxio > > "I'm astonished anyone pays real money for this crap." > -- Cosmo > > -- > This message has been scanned for viruses and > dangerous content by OpenProtect(http://www.openprotect.com), and is > believed to be clean. The problem with an audited benchmark is that they often don't tell the real story. Benchmarks like TPC (A,B,C,D,H,etc...) are highly specialized and aren't really tests of the DBMS, they are tests of the hardware. The DBMS code path for these benchmarks is highly optimized often using technology and features which cost millions to develop and offer very little customer benefit because most applications won't/ can't use the technology that's helping achieve the winning number. I would venture to guess that all the major DBMS players (Oracle, DB2, Informix, Sybase, MySQL, etc) are within 10%-20% in terms of raw OLTP performance on basic tables. Add in constraints, triggers and a few other tid bits, you might get some more differientation. (BI workloads are a different story!) Most of the differientiation can be overcome with a bit more hardware (extra cpu or 2, a bit more memory). Another aspect to consider is workload flexibility. If the workload is fairly steady and changes slightly over time, why does one need scalability? You only need scalability if your workload is going to change substantially over a short period of time. Just about all the major players also do pretty well at scalability... there are differientiators here, some scale better than others... so if this is the area of most concern, do diligence to understand the differences here. Lastly, and this is where Informix shines above the others, reliability, predictability and ease of use. There are clear differientiators here where Informix outshines its peers with its low administration costs, its ability to adjust to small changes in workload, its predicitable and reliable behavior. DBAs love Informix IDS because it makes their jobs easy. Ever wonder why there's such passionate and loyal support for IDS? DBAs who know Informix want to remain lazy, especially those who work in mixed shops. They spend more effort defending their support for and promoting IDS than they spend on maintaining their databases (not really true... but a grain of truth there!). Bottomline... benchmarks are very expensive to do. Do you want IBM to pour $$ into benchmarks, or pour $$ into making IDS even better?
pokeyman said: > The problem with an audited benchmark is that they often don't tell > the real story. Benchmarks like TPC (A,B,C,D,H,etc...) are highly > specialized and aren't really tests of the DBMS, they are tests of the > hardware. The DBMS code path for these benchmarks is highly optimized > often using technology and features which cost millions to develop and > offer very little customer benefit because most applications won't/ > can't use the technology that's helping achieve the winning number. Not strictly true. Yes, Oracle did once violate the spirit of TPC-B by introducing discrete transactions, but I'm not sure that they would still win this race on a like for like box. > I > would venture to guess that all the major DBMS players (Oracle, DB2, > Informix, Sybase, MySQL, etc) are within 10%-20% in terms of raw OLTP > performance on basic tables. I would venture to guess that you might be surprised. -- Bye now, Obnoxio "I'm astonished anyone pays real money for this crap." -- Cosmo -- This message has been scanned for viruses and dangerous content by OpenProtect(http://www.openprotect.com), and is believed to be clean.
On Jun 18, 9:36 am, "Obnoxio The Clown" <obno...@serendipita.com> wrote: > pokeyman said: > > > The problem with an audited benchmark is that they often don't tell > > the real story. Benchmarks like TPC (A,B,C,D,H,etc...) are highly > > specialized and aren't really tests of the DBMS, they are tests of the > > hardware. The DBMS code path for these benchmarks is highly optimized > > often using technology and features which cost millions to develop and > > offer very little customer benefit because most applications won't/ > > can't use the technology that's helping achieve the winning number. > > Not strictly true. Yes, Oracle did once violate the spirit of TPC-B by > introducing discrete transactions, but I'm not sure that they would still > win this race on a like for like box. > > > I > > would venture to guess that all the major DBMS players (Oracle, DB2, > > Informix, Sybase, MySQL, etc) are within 10%-20% in terms of raw OLTP > > performance on basic tables. > > I would venture to guess that you might be surprised. > > -- > Bye now, > Obnoxio > > "I'm astonished anyone pays real money for this crap." > -- Cosmo > > -- > This message has been scanned for viruses and > dangerous content by OpenProtect(http://www.openprotect.com), and is > believed to be clean. since i do this for a living, i guess i have some experience in that area. i've certainly come across examples where 1 DBMS truly out performs another for a particular query or set of queries, but, in my experiece of running hundreds of comparisons, i rarely see a significant difference that would lead me to believe 1 DBMS is significantly better than another... at least in terms of raw OLTP power. your reference to discrete transactions wasn't really what I was thinking, although that's a good one. probably one of the biggest benchmark specials that all the players use is hashing (DB2 has Range cluster, etc...). IDS does not support a hashed access method thus making it impossible to compete in the TPC (a,b,c) arena.