Lies, Damn Lies and Statistics...

I'm trying to create a new thread so that we can try and stay focused. I do want to thank Mark for the numbers. Its nice that a competitive wonk from Oracle to pipe up and show us the 'bigger picture'. I should add that the numbers from IDC and other research groups are estimates. They have a certain degree of accuracy, however, they hold value in terms of demonstrating a market place's trends. The actual numbers are not important. What is important is that IBM is trending lower when it comes to the adoption of its Information Management (Relation Databases) Since IBM doesn't break out the individual product numbers to the public, we have to rely on 'rumors' and 'whisper numbers'. (Don't worry, its not a violation of Rule FD, so knowing these numbers isn't really verboten.) These numbers indicate two things. Specifically in the LUW environment... 1) IDS has continued to grow year over year and quarter over quarter. 2) IDS's rumored growth rate(s) are in the 'double digits'. Meaning some number > 10%. (Some rumors peg this number much higher.) And the verboten rumor? IDS is outselling DB2 in the LUW space. (This one is something that can not be confirmed.) What we don't know is what these rumored numbers reflect. Is the license growth due to IDS's installed base of customers buying more licenses? Is there a growth in net new customers? Is it a combination of both? The significance is that if a portion of the numbers reflect existing customers buying net new licenses, it means that they are running current apps on IDS and its not a 'legacy' platform. If the numbers include net new customers, it means that there is a longer term growth that has a larger impact on IBM SWG as a whole. If IDS is outselling DB2 in the LUW space then it makes sense to 'lead with IDS', unless there is a specific reason to lead with DB2. The reason I have to point this out is that most client teams rely on an SSR and they do not have an assigned Software Sales Specialist. The SSR is responsible for having a basic understanding of all of the pillars software products and you have to keep the message simple. In 2004, I was at club and got to hear one of the Senior VPs talk. He said that based on IBMs size, a single percentage point in overall customer satisfaction equated to a billion dollars towards the bottom line. That is, if customer sat goes up 1%, then you can expect to see an additional billion dollars towards the bottom line. If customer sat goes down a %1, well... you end up with a RIF. ;-) Thats an additional important number. The customer sat score. IDS continually ranks high. Heck, the user community is doing more that the company's sales force in selling and promoting IDS. Says a lot, doesn't it? So why doesn't IBM SWG and specifically IM get on board and actually do the little things that will increase the sales of IDS. Oh and one last note... While the guys in the trenches don't care, cross sell / up sell is very important to Ambush and Steve Mills. It means more revenue for IBM as a whole. Using IDS to gain *traction* in to the SMB space should be a critical component of IBM's strategy going forward. But hey! What do I know? I just cc'd Ambuj and Steve Mills knowing that this will hit their admin's inbox and will be filtered before either of them gets a chance to read it. ;-) -G

On Jan 23, 5:04 pm, Ian Michael Gumby <im_gu...@hotmail.com> wrote: > I'm trying to create a new thread so that we can try and stay focused. > > I do want to thank Mark for the numbers. Its nice that a competitive > wonk from Oracle to pipe up and show us the 'bigger picture'. > > I should add that the numbers from IDC and other research groups are > estimates. They have a certain degree of accuracy, however, they hold > value in terms of demonstrating a market place's trends. > > The actual numbers are not important. > What is important is that IBM is trending lower when it comes to the > adoption of its Information Management (Relation Databases) > > Since IBM doesn't break out the individual product numbers to the > public, we have to rely on 'rumors' and 'whisper numbers'. (Don't > worry, its not a violation of Rule FD, so knowing these numbers isn't > really verboten.) > > These numbers indicate two things. > > Specifically in the LUW environment... > > 1) IDS has continued to grow year over year and quarter over quarter. > 2) IDS's rumored growth rate(s) are in the 'double digits'. Meaning > some number > 10%. > (Some rumors peg this number much higher.) > > And the verboten rumor? IDS is outselling DB2 in the LUW space. > (This one is something that can not be confirmed.) > > What we don't know is what these rumored numbers reflect. > Is the license growth due to IDS's installed base of customers buying > more licenses? > Is there a growth in net new customers? > Is it a combination of both? > > The significance is that if a portion of the numbers reflect existing > customers buying net new licenses, it means that they are running > current apps on IDS and its not a 'legacy' platform. If the numbers > include net new customers, it means that there is a longer term growth > that has a larger impact on IBM SWG as a whole. > > If IDS is outselling DB2 in the LUW space then it makes sense to 'lead > with IDS', unless there is a specific reason to lead with DB2. As we two already pointed out several times: You can't sell a product that you don't know, so you can't probably lead with a product that you don't know. The missing IDS education/certification of the bulk of IBM IT specialists really hurts here and IBM constantly misses (potential) money and (potential) customers because of this. However from my point of view IBM upper management already discovered the problem with DB2 LUW. It is not without reason that they try to establish a dual database strategy. They wouldn't do that if DB2 would be a top selling product. They are creating some kind of emergency plan. If OS2 - pardon DB2 - continues to fail, they have sth. up one's sleeve. It seems that they are not betting their future solely on DB2 LUW. > The reason I have to point this out is that most client teams rely on an > SSR and they do not have an assigned Software Sales Specialist. The > SSR is responsible for having a basic understanding of all of the > pillars software products and you have to keep the message simple. > > In 2004, I was at club and got to hear one of the Senior VPs talk. He > said that based on IBMs size, a single percentage point in overall > customer satisfaction equated to a billion dollars towards the bottom > line. That is, if customer sat goes up 1%, then you can expect to see > an additional billion dollars towards the bottom line. > If customer sat goes down a %1, well... you end up with a RIF. ;-) > > Thats an additional important number. The customer sat score. IDS > continually ranks high. Heck, the user community is doing more that > the company's sales force in selling and promoting IDS. Says a lot, > doesn't it? Good point. The customers gave Informix an excellent vote (http:// informix-zone.com/vendorrate-informix) in a recent survey. Did you see any press release from IBM mentioning this excellent result ? No, nothing. Even on the main IBM/Informix website: nothing ! It's just unbelievable. With this customer satisfaction behind the product, it should be a no-brainer for IBM to sell IDS to new customers (if IBM sales would have a clue about the product). What is needed is a marketing machine that transports such messages to the rest of the world and not relying on the IIUG to do the work. As much as I respect the work of the IIUG, they can only transport those messages to the existing Informix customers. Unfortunately it isn't a new message for the existing customers as they were it that gave IDS those excellent notes in the survey. Every day I receive about a dozen news messages around Oracle thru my google alert. Not all of them but a major part is related to the DBMS. The same is true for MySQL. Sometimes I receive an Informix related message, most of the time it is an Ex-Informix employee that has a new job now, so the string "Informix" is somewhere in the message text. The unique press releases - I'm not counting the duplication of such a unique message on several news sites - that come from IBM about IDS could be counted by a single hand in a whole year. I'm really asking me what are the press people at IBM doing ? > So why doesn't IBM SWG and specifically IM get on board and actually > do the little things that will increase the sales of IDS. > > Oh and one last note... > While the guys in the trenches don't care, cross sell / up sell is > very important to Ambush and Steve Mills. It means more revenue for > IBM as a whole. Using IDS to gain *traction* in to the SMB space > should be a critical component of IBM's strategy going forward. > > But hey! What do I know? > I just cc'd Ambuj and Steve Mills knowing that this will hit their > admin's inbox and will be filtered before either of them gets a chance > to read it. ;-) > > -G

JAJAJA! really nice... (not to say clever... ;) ) J. 2009/1/23 <eric@herber-consulting.de> > ... > product. They are creating some kind of emergency plan. If __OS2__ - > pardon __DB2__ - > continues to fail, they have sth. up one's sleeve. It seems that they > ...

On Jan 23, 11:31 am, e...@herber-consulting.de wrote: > However from my point of view IBM upper management already discovered > the problem with DB2 LUW. It is not without reason that they try to > establish > a dual database strategy. They wouldn't do that if DB2 would be a top > selling > product. They are creating some kind of emergency plan. If OS2 - > pardon DB2 - > continues to fail, they have sth. up one's sleeve. It seems that they > are not > betting their future solely on DB2 LUW. > > Eric, This has been an issue since before Janet left the building. (IDS outselling DB2 LUW and having to live down the DB2 V8 rollout fiasco?) I hate to say it, I doubt that there is anyone in S&D or in the pillar sales team that could actually white board the entire IM product offering and how they can be used together to implement a solution. Its not an easy task. To make things worse, under Janet, there were product silos. DB2 z/OS, DB2 LUW, CM, Informix, Ass-n-tail ... each silo focused on their own thing and each had their own chain of command up to Janet P. ( Note: with the inclusion of Informix, some of the functions like services and support went through Alan Grady and then to Janet. But that didn't mean that there weren't silos and issues.) There isn't even a cheat sheet that a rep could use. (This would be a function of Product Managers / Product Marketing but again. You had silos.) The problem with a dual database strategy, it required that the S&D client team understood the strategy. Under SSM (Solution Selling Methodology) you have to know your product so that you can either raise issues to the customer about problems with your competitor if he's in column A and you're not, or how to make your product look superior to your competitors. You can't do that if you don't know your product.) IBM had a 'stop gap' solution. Ensuring that there was at least someone in the pillar S&D team that knew the product. This means that if a client team was facing a challenge, there was a 'go to' that could help close the deal. Only there's a catch. This only works if the customer is asking for a specific product that you sell, or their preferred solution embeds the product. If the client team doesn't know the product, they can't offer it as a solution, nor know to call the resource until its too late. The mantra lead with DB2 and if all else fails go with Informix as a last resort doesn't work. SSM doesn't allow you to pursue this strategy. If you're already column fodder with DB2, how can you change your position with your second guess at a solution? Your credibility is gone. The other problem is that the client team doesn't like to work with people outside of their team. If the client team is assigned an IT Specialist named Joe, then if Fred is pulled in on a pre-sales issue, there is some hesitation. The fear is that the IT Specialist may say something that would trigger an adverse response or objection. I really feel sorry for some of the Software sales specialists that have to cover several (7-10) named accounts. During those account's planning sessions, you have to be there to make sure your products are represented. These planning sessions are usually a day long event. Unfortunately because you track to only 7-10 accounts, you're not going to be able to make it to all of your planning sessions, so you may end up not being able to sell anything to that client. In Jerry Keesee's defense, I do know that the product team has tried to educate the sales team. Unfortunately their efforts were ineffective. I hate to say it, Charlie Ills was able to use fear as a way to get his message across. Perhaps after this latest RIF action, the S&D team (s) will be more attentive? But hey! What do I know? ;-)

In article <9592c236-c268-4ced-a89f-6d2a4c4c8ce6@r41g2000prr.googlegroups.com>, Ian Michael Gumby says... >Eric, >This has been an issue since before Janet left the building. >(IDS outselling DB2 LUW and having to live down the DB2 V8 rollout >fiasco?) DB2 LUW is a very good product by itself. But then I am talking about ver 9.5, the multi-threaded server. It is a question of time before it overtakes informix in performance.

> From: dcruncher4@aim.not.com > Subject: Re: Lies, Damn Lies and Statistics... > Date: Fri, 23 Jan 2009 16:36:33 -0800 > To: informix-list@iiug.org > > In article <9592c236-c268-4ced-a89f-6d2a4c4c8ce6@r41g2000prr.googlegroups.com>, > Ian Michael Gumby says... > > >Eric, > >This has been an issue since before Janet left the building. > >(IDS outselling DB2 LUW and having to live down the DB2 V8 rollout > >fiasco?) > > DB2 LUW is a very good product by itself. But then I am talking > about ver 9.5, the multi-threaded server. > > It is a question of time before it overtakes informix in performance. > I seriously doubt that it will ever 'overtake' IDS. Where do you think DB2 got its multi-threaded server from? Hint: Informix had to write their own thread libraries because not all of the platforms at the time supported multi-threading. Std Unix(s) at the time had fork(). _________________________________________________________________ Windows Live™: E-mail. Chat. Share. Get more ways to connect. http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t2_allup_explore_012009

Ian Michael Gumby wrote: > > > > From: dcruncher4@aim.not.com > > Subject: Re: Lies, Damn Lies and Statistics... > > Date: Fri, 23 Jan 2009 16:36:33 -0800 > > To: informix-list@iiug.org > > > > In article > <9592c236-c268-4ced-a89f-6d2a4c4c8ce6@r41g2000prr.googlegroups.com>, > > Ian Michael Gumby says... > > > > >Eric, > > >This has been an issue since before Janet left the building. > > >(IDS outselling DB2 LUW and having to live down the DB2 V8 rollout > > >fiasco?) > > > > DB2 LUW is a very good product by itself. But then I am talking > > about ver 9.5, the multi-threaded server. > > > > It is a question of time before it overtakes informix in performance. > > > > I seriously doubt that it will ever 'overtake' IDS. > Where do you think DB2 got its multi-threaded server from? DB2 for LUW was multi-threaded on Windows since inception (DB2 V5.1 I think). All that was done in DB2 9.5 was to extend the concept to Unix/Linux. You shouldn't talk about things you know nothing about. -- Serge Rielau DB2 Solutions Development IBM Toronto Lab

In article <mailman.181.1232806520.1831.informix-list@iiug.org>, Ian Michael Gumby says... >> DB2 LUW is a very good product by itself. But then I am talking >> about ver 9.5=2C the multi-threaded server. >> It is a question of time before it overtakes informix in performance. >> > >I seriously doubt that it will ever 'overtake' IDS. >Where do you think DB2 got its multi-threaded server from? why can't it overtake? Why do you think DB2LUW is so inferior. I work with IDS (at least for few more months before it retires at my company) and I play with Db2 at my home. So I don't intend to pass all-knowing statement without having a clue.

On Jan 24, 7:38 pm, dcrunch...@aim.not.com wrote: > In article <mailman.181.1232806520.1831.informix-l...@iiug.org>, Ian Michael > Gumby says... > > >> DB2 LUW is a very good product by itself. But then I am talking > >> about ver 9.5=2C the multi-threaded server. > >> It is a question of time before it overtakes informix in performance. > > >I seriously doubt that it will ever 'overtake' IDS. > >Where do you think DB2 got its multi-threaded server from? > > why can't it overtake? Why do you think DB2LUW is so > inferior. > > I work with IDS (at least for few more months before > it retires at my company) and I play with Db2 at my > home. So I don't intend to pass all-knowing statement > without having a clue. So you are playing with DB2 at your home ? Probably on a real big machine with 50 CPU's and more :-) Claiming that DB2 will overtake IDS in terms of performance based on such a comparision is just nonsense. I can even achieve an extremely good performance with MySQL on my dualcore notebook. Even better than IDS or DB2. The real difference will show up when you run on a bigger SMP machine, with thousands of connections and a versatile load produced by the clients, probably replicating the data to several other sites in realtime. I'm not saying that DB2 cannot do that. I'm just saying that your statement isn't based on real life (production) experience.

On Jan 24, 8:55 am, Serge Rielau <srie...@ca.ibm.com> wrote: > Ian Michael Gumby wrote: > > >  > From: dcrunch...@aim.not.com > >  > Subject: Re: Lies, Damn Lies and Statistics... > >  > Date: Fri, 23 Jan 2009 16:36:33 -0800 > >  > To: informix-l...@iiug.org > > >  > In article > > <9592c236-c268-4ced-a89f-6d2a4c4c8...@r41g2000prr.googlegroups.com>, > >  > Ian Michael Gumby says... > > >  > >Eric, > >  > >This has been an issue since before Janet left the building. > >  > >(IDS outselling DB2 LUW and having to live down the DB2 V8 rollout > >  > >fiasco?) > > >  > DB2 LUW is a very good product by itself. But then I am talking > >  > about ver 9.5, the multi-threaded server. > > >  > It is a question of time before it overtakes informix in performance. > > > I seriously doubt that it will ever 'overtake' IDS. > > Where do you think DB2 got its multi-threaded server from? > > DB2 for LUW was multi-threaded on Windows since inception > (DB2 V5.1 I think). > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > You shouldn't talk about things you know nothing about. > Why, That doesn't stop you. :-P IIRC, when IBM did the acquisition of Informix, we had an indoctrination session from a DB2 'wonk'. Or rather a 'wonk lite'. Two things seemed to stick out. 1) The git kept insisting that IBM's type 2 JDBC driver support was superior to the native type 4 drivers that Informix has had for years. Claimed that it was 'better performing.' 2) 90% of the DB2 code was the same across all platforms. Meaning both distributed and mainframe. If what you've just said was true, it would mean that he was yet again blowing smoke out his arse and was repeating marketing BS where Janet was insisting that DB2 on the mainframe was the same as DB2 distributed. Again to the OP's post, DB2 has a lot to work on before they catch up to Informix, in-spite of Janet's efforts to slow development down on IDS. (Meaning DB2 had 5 years to play catch up. (And I know I'm going to burn in hell for that comment. :-P )

In article <e0df5930-bd98-43be-a84f-6d48d5bce3ad@w39g2000prb.googlegroups.com>, eric@herber-consulting.de says... > >Claiming that DB2 will overtake IDS in terms of performance based on >such a >comparision is just nonsense. I can even achieve an extremely good >performance with MySQL on my dualcore notebook. Even better than >IDS or DB2. The real difference will show up when you run on a >bigger SMP machine, with thousands of connections and a versatile >load produced by the clients, probably replicating the data to >several >other sites in realtime. > >I'm not saying that DB2 cannot do that. I'm just saying that your >statement >isn't based on real life (production) experience. fair enuf. but on the same laptop I also have IDS and I can make IDS vs DB2 comparison.

>> Where do you think DB2 got its multi-threaded server from? > DB2 for LUW was multi-threaded on Windows since inception > (DB2 V5.1 I think). > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > You shouldn't talk about things you know nothing about. > And I wouldn't really call it multi-threaded. Rather just a bunch of processes now run as threads in a single process, with a single address space. The diagram at http://www.ibm.com/developerworks/db2/library/techarticle/dm-0807kharche/ shows this quite well Biggest advantage is that shared memory management becomes easier (for the developers, mainly). Not at all like what Informix did when they rewrote the engine from 5 to 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but introduced a completely different set.

Mark Townsend wrote: > >>> Where do you think DB2 got its multi-threaded server from? >> DB2 for LUW was multi-threaded on Windows since inception >> (DB2 V5.1 I think). >> All that was done in DB2 9.5 was to extend the concept to Unix/Linux. >> >> You shouldn't talk about things you know nothing about. >> > > And I wouldn't really call it multi-threaded. Rather just a bunch of > processes now run as threads in a single process, with a single address > space. The diagram at > http://www.ibm.com/developerworks/db2/library/techarticle/dm-0807kharche/ > shows this quite well > > Biggest advantage is that shared memory management becomes easier (for > the developers, mainly). The self tuning memory manager is a wonderful thing. Customers and ISV's love it. I know it won't be called multi-threaded until Oracle has it. Cheers Serge -- Serge Rielau DB2 Solutions Development IBM Toronto Lab

eric@herber-consulting.de wrote: > On Jan 24, 7:38 pm, dcrunch...@aim.not.com wrote: >> In article <mailman.181.1232806520.1831.informix-l...@iiug.org>, Ian Michael >> Gumby says... >> >>>> DB2 LUW is a very good product by itself. But then I am talking >>>> about ver 9.5=2C the multi-threaded server. >>>> It is a question of time before it overtakes informix in performance. >>> I seriously doubt that it will ever 'overtake' IDS. >>> Where do you think DB2 got its multi-threaded server from? >> why can't it overtake? Why do you think DB2LUW is so >> inferior. >> >> I work with IDS (at least for few more months before >> it retires at my company) and I play with Db2 at my >> home. So I don't intend to pass all-knowing statement >> without having a clue. > > So you are playing with DB2 at your home ? Probably on a real big > machine > with 50 CPU's and more :-) Whoops.. 50CPUs for those SMBs? Cheers Serge -- Serge Rielau DB2 Solutions Development IBM Toronto Lab

On Jan 24, 5:24 pm, Serge Rielau <srie...@ca.ibm.com> wrote: > Mark Townsend wrote: > > >>> Where do you think DB2 got its multi-threaded server from? > >> DB2 for LUW was multi-threaded on Windows since inception > >> (DB2 V5.1 I think). > >> All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > >> You shouldn't talk about things you know nothing about. > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > processes now run as threads in a single process, with a single address > > space. The diagram at > >http://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > shows this quite well > > > Biggest advantage is that shared memory management becomes easier (for > > the developers, mainly). > > The self tuning memory manager is a wonderful thing. > Customers and ISV's love it. > > I know it won't be called multi-threaded until Oracle has it. > Serge, If you're trying to be an IBM DB2 'wonk' you're going to have to learn to be realistic and give credit where credit is due. At least Mark, the Oracle 'wonk' is admitting that Informix did something that was truly advanced for the time. Yes he's right that by writing their own thread libraries they were able to write a truly threaded engine. Yes there are distinct advantages to what was done. At the same time, there were also technical issues. And much to the credit of the Informix 'wonks' they solved a lot of those issues. The area where performance took a hit was when Informix acquired Illustra. Stonebraker had the right concept, but it wasn't as worked out as it could have been. It took 5 years to integrate Illustra in to the code stream and to get similar performance of the 7.x engine. I think its important to understand that not all databases are created equal and that there are two factors that are critical to the evolution of the databases. First is the vision of the architects and whether they can predict what the market is going to want in advanced features. The second is the legacy of the code stream and how willing the company is in evolving / refactoring the existing code stream. If you want to be considered a 'wonk' then you have to understand not just your technology, but you have to be able to separate marketing BS from reality along with understanding the competition and appreciate what they do right.

On Jan 24, 3:28 pm, dcrunch...@aim.not.com wrote: > fair enuf. but on the same laptop I also have IDS and > I can make IDS vs DB2 comparison. And I'm sure you could also add derby/javadb to the mix. Yet when you start to get beyond a certain size, derby doesn't scale like IDS or DB2. Design decisions and legacy code will impact performance and scalability. Apples to Apples comparisons on small data sets isn't going to show the true potential of any product, nor their limitations.

On 24 Jan, 22:17, Mark Townsend <markbtowns...@sbcglobal.net> wrote: > >> Where do you think DB2 got its multi-threaded server from? > > DB2 for LUW was multi-threaded on Windows since inception > > (DB2 V5.1 I think). > > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > > You shouldn't talk about things you know nothing about. > > And I wouldn't really call it multi-threaded. Rather just a bunch of > processes now run as threads in a single process, with a single address > space. The diagram athttp://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > shows this quite well I have some questions 1 Can network communicating threads run in a seperate process from the rest of the engine? A process that can block inside an OS call. Informix can with NETTYPE NET and under certain workloads this gives better performance. 2. Can network communicating threads run in the db2sync process? This means the threads have to use network calls that do not block. Informix can with NETTYPE CPU and under certain workloads this gives better performance due to lower context switching. 3. Can you have multiple db2sync processes? Otherwise how does db2 use >1 cpu? If so are session threads (EDUs) able to migrate between db2sync processes? I believe Sybase statically binds sessons to engines hence if the engine is busy running the thread for one engine then other threads on that engine cannot migrate to idle engines! Being able to migrate threads between engines was one of Informixs advantages over other multi-thread engines. > > Biggest advantage is that shared memory management becomes easier (for > the developers, mainly). Nice to see DB2 catching up with Informix. Pity DB2 cannot dynamically add shared memory segments on Linux. > > Not at all like what Informix did when they rewrote the engine from 5 to > 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but > introduced a completely different set.

On 24 Jan, 23:24, Serge Rielau <srie...@ca.ibm.com> wrote: > Mark Townsend wrote: > > >>> Where do you think DB2 got its multi-threaded server from? > >> DB2 for LUW was multi-threaded on Windows since inception > >> (DB2 V5.1 I think). > >> All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > >> You shouldn't talk about things you know nothing about. > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > processes now run as threads in a single process, with a single address > > space. The diagram at > >http://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > shows this quite well > > > Biggest advantage is that shared memory management becomes easier (for > > the developers, mainly). > > The self tuning memory manager is a wonderful thing. > Customers and ISV's love it. > > I know it won't be called multi-threaded until Oracle has it. > Is Oracle truely multi-threaded now ? Last I heard (Oracle 8) it was just a pool of work processes rather than truely multi-threaded. > Cheers > Serge > -- > Serge Rielau > DB2 Solutions Development > IBM Toronto Lab- Hide quoted text - > > - Show quoted text -

david@smooth1.co.uk wrote: >> I know it won't be called multi-threaded until Oracle has it. >> > > Is Oracle truely multi-threaded now ? Last I heard (Oracle 8) it > was just a pool of work processes rather than truely multi-threaded. > >> Cheers >> Serge You have your OPs backwards. On Linux/Unix, Oracle is process based. On Windows it does processes as threads, because it has to for shared memory. The whole multi-threaded IDS vs process based Oracle argument was largely a furphy. Basically, IDS blocks more than Oracle, so context switches became way expensive, hence it made sense for IDS to rewrite the engine. With Oracle, the same was not true.

On Jan 25, 7:24 pm, "da...@smooth1.co.uk" <da...@smooth1.co.uk> wrote: > On 24 Jan, 22:17, Mark Townsend <markbtowns...@sbcglobal.net> wrote: > > > >> Where do you think DB2 got its multi-threaded server from? > > > DB2 for LUW was multi-threaded on Windows since inception > > > (DB2 V5.1 I think). > > > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > > > You shouldn't talk about things you know nothing about. > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > processes now run as threads in a single process, with a single address > > space. The diagram athttp://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > shows this quite well > > I have some questions > > 1 Can network communicating threads run in a seperate process from > the rest of the engine? A process that can block inside > an OS call. Informix can with NETTYPE NET and under certain > workloads this gives better performance. > > 2. Can network communicating threads run in the db2sync process? This > means the threads have to use network calls that > do not block. Informix can with NETTYPE CPU and under certain > workloads this gives better performance due to lower > context switching. > > 3. Can you have multiple db2sync processes? Otherwise how does db2 use>1 cpu? If so are session threads (EDUs) able to > > migrate between db2sync processes? I believe Sybase statically > binds sessons to engines hence if the engine is busy > running the thread for one engine then other threads on that engine > cannot migrate to idle engines! Being able to migrate > threads between engines was one of Informixs advantages over other > multi-thread engines. > > > > > Biggest advantage is that shared memory management becomes easier (for > > the developers, mainly). > > Nice to see DB2 catching up with Informix. Pity DB2 cannot > dynamically add shared memory segments on Linux. > > > > > Not at all like what Informix did when they rewrote the engine from 5 to > > 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but > > introduced a completely different set. Hi David, DB2's threading model is different than IDS. With DB2, each thread maps directly to a kernel thread, so each can be individually scheduled by the OS. So, if any one DB2 thread is blocked, the OS is free to schedule any other DB2 thread on that same processor, or any other processor. That also means that as one CPU gets too overloaded, the OS is free to re-schedule any DB2 threads on other, less loaded, CPUs, as it sees fit. One of the main benefits with going threaded on UNIX/Linux is the simplification to our memory model, both internally (for development), but more importantly for customers, in terms of ease of administration. The following sections in the docs highlight these changes: https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.wn.doc/doc/c0051445.html The following link gives a little more detail on how the memory parameters and the self-tuning memory manager interact with each other: https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.dbobj.doc/doc/c0053287.html For your last question, DB2 does in fact allocate new shared memory segments during runtime. If you want to see this in practice, just activate your database, then dynamically create a new bufferpool (or dynamically increase your locklist, shared sort heap, etc). In 9.5, this operation should always succeed, and if you issue 'db2pd -db dbname -memsets', you should see that there are multiple shared memory segments now associated with your database. If you tried this same exercise in DB2 9 or earlier, this same operation would have likely failed, since we could not dynamically allocate new shared memory segments during runtime. Cheers, Liam.

Not to bust Liam's bubble, But what happens when you have two different threads from the same parent trying to run at the same time on different CPUs? Me thinks that there's more spin and double talk that meets the eye? I wonder when there will be a processor affinity switch added to the OSs if there isn't one already. Also whenever you create a thread, any thread, the OS has some level of schedule over it. ;-) But Hey What do I know? I've never looked at the source code of either IDS or DB2. ;-) -G On Jan 26, 9:05 am, lfin...@ca.ibm.com wrote: > On Jan 25, 7:24 pm, "da...@smooth1.co.uk" <da...@smooth1.co.uk> wrote: > > > > > On 24 Jan, 22:17, Mark Townsend <markbtowns...@sbcglobal.net> wrote: > > > > >> Where do you think DB2 got its multi-threaded server from? > > > > DB2 for LUW was multi-threaded on Windows since inception > > > > (DB2 V5.1 I think). > > > > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > > > > You shouldn't talk about things you know nothing about. > > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > > processes now run as threads in a single process, with a single address > > > space. The diagram athttp://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > > shows this quite well > > >   I have some questions > > >  1 Can network communicating threads run in a seperate process from > > the rest of the engine? A process that can block inside > >   an OS call. Informix can with NETTYPE NET and under certain > > workloads this gives better performance. > > > 2. Can network communicating threads run in the db2sync process? This > > means the threads have to use network calls that > >    do not block. Informix can with NETTYPE CPU and under certain > > workloads this gives better performance due to lower > >    context switching. > > > 3. Can you have multiple db2sync processes? Otherwise how does db2 use>1 cpu? If so are session threads (EDUs) able to > > >    migrate between db2sync processes? I believe Sybase statically > > binds sessons to engines hence if the engine is busy > >    running the thread for one engine then other threads on that engine > > cannot migrate to idle engines! Being able to migrate > >    threads between engines was one of Informixs advantages over other > > multi-thread engines. > > > > Biggest advantage is that shared memory management becomes easier (for > > > the developers, mainly). > > >  Nice to see DB2 catching up with Informix. Pity DB2 cannot > > dynamically add shared memory segments on Linux. > > > > Not at all like what Informix did when they rewrote the engine from 5 to > > > 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but > > > introduced a completely different set. > > Hi David, > > DB2's threading model is different than IDS.  With DB2, each thread > maps directly to a kernel thread, so each can be individually > scheduled by the OS.  So, if any one DB2 thread is blocked, the OS is > free to schedule any other DB2 thread on that same processor, or any > other processor.  That also means that as one CPU gets too overloaded, > the OS is free to re-schedule any DB2 threads on other, less loaded, > CPUs, as it sees fit. > > One of the main benefits with going threaded on UNIX/Linux is the > simplification to our memory model, both internally (for development), > but more importantly for customers, in terms of ease of > administration.  The following sections in the docs highlight these > changes:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > The following link gives a little more detail on how the memory > parameters and the self-tuning memory manager interact with each > other:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > For your last question, DB2 does in fact allocate new shared memory > segments during runtime.  If you want to see this in practice, just > activate your database, then dynamically create a new bufferpool (or > dynamically increase your locklist, shared sort heap, etc).  In 9.5, > this operation should always succeed, and if you issue 'db2pd -db > dbname -memsets', you should see that there are multiple shared memory > segments now associated with your database.  If you tried this same > exercise in DB2 9 or earlier, this same operation would have likely > failed, since we could not dynamically allocate new shared memory > segments during runtime. > > Cheers, > Liam.

On Jan 27, 9:37 am, Ian Michael Gumby <im_gu...@hotmail.com> wrote: > Not to bust Liam's bubble, > > But what happens when you have two different threads from the same > parent trying to run at the same time on different CPUs? > > Me thinks that there's more spin and double talk that meets the eye? > > I wonder when there will be a processor affinity switch added to the > OSs if there isn't one already. > > Also whenever you create a thread, any thread, the OS has some level > of schedule over it. ;-) > > But Hey What do I know? > I've never looked at the source code of either IDS or DB2. ;-) > > -G > > On Jan 26, 9:05 am, lfin...@ca.ibm.com wrote: > > > On Jan 25, 7:24 pm, "da...@smooth1.co.uk" <da...@smooth1.co.uk> wrote: > > > > On 24 Jan, 22:17, Mark Townsend <markbtowns...@sbcglobal.net> wrote: > > > > > >> Where do you think DB2 got its multi-threaded server from? > > > > > DB2 for LUW was multi-threaded on Windows since inception > > > > > (DB2 V5.1 I think). > > > > > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > > > > > You shouldn't talk about things you know nothing about. > > > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > > > processes now run as threads in a single process, with a single address > > > > space. The diagram athttp://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > > > shows this quite well > > > > I have some questions > > > > 1 Can network communicating threads run in a seperate process from > > > the rest of the engine? A process that can block inside > > > an OS call. Informix can with NETTYPE NET and under certain > > > workloads this gives better performance. > > > > 2. Can network communicating threads run in the db2sync process? This > > > means the threads have to use network calls that > > > do not block. Informix can with NETTYPE CPU and under certain > > > workloads this gives better performance due to lower > > > context switching. > > > > 3. Can you have multiple db2sync processes? Otherwise how does db2 use>1 cpu? If so are session threads (EDUs) able to > > > > migrate between db2sync processes? I believe Sybase statically > > > binds sessons to engines hence if the engine is busy > > > running the thread for one engine then other threads on that engine > > > cannot migrate to idle engines! Being able to migrate > > > threads between engines was one of Informixs advantages over other > > > multi-thread engines. > > > > > Biggest advantage is that shared memory management becomes easier (for > > > > the developers, mainly). > > > > Nice to see DB2 catching up with Informix. Pity DB2 cannot > > > dynamically add shared memory segments on Linux. > > > > > Not at all like what Informix did when they rewrote the engine from 5 to > > > > 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but > > > > introduced a completely different set. > > > Hi David, > > > DB2's threading model is different than IDS. With DB2, each thread > > maps directly to a kernel thread, so each can be individually > > scheduled by the OS. So, if any one DB2 thread is blocked, the OS is > > free to schedule any other DB2 thread on that same processor, or any > > other processor. That also means that as one CPU gets too overloaded, > > the OS is free to re-schedule any DB2 threads on other, less loaded, > > CPUs, as it sees fit. > > > One of the main benefits with going threaded on UNIX/Linux is the > > simplification to our memory model, both internally (for development), > > but more importantly for customers, in terms of ease of > > administration. The following sections in the docs highlight these > > changes:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > The following link gives a little more detail on how the memory > > parameters and the self-tuning memory manager interact with each > > other:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > > For your last question, DB2 does in fact allocate new shared memory > > segments during runtime. If you want to see this in practice, just > > activate your database, then dynamically create a new bufferpool (or > > dynamically increase your locklist, shared sort heap, etc). In 9.5, > > this operation should always succeed, and if you issue 'db2pd -db > > dbname -memsets', you should see that there are multiple shared memory > > segments now associated with your database. If you tried this same > > exercise in DB2 9 or earlier, this same operation would have likely > > failed, since we could not dynamically allocate new shared memory > > segments during runtime. > > > Cheers, > > Liam. Why would it matter if the OS tries to schedule two different threads at the same time on different CPUs? Why would this be any different than if the OS tried to schedule two different processes from the same parent on different CPUs? When DB2 creates threads, we specify the 1:1 model, where each thread maps to it's own kernel thread. So, when scheduling, each kernel thread is seen as a distinct schedulable entity, just as if it was a separate process. The only difference is that depending on the platform, the OS can usually context switch faster between two kernel threads within the same process. Going to a threaded model does mean though that there will now be OS kernel contention for some OS APIs (i.e. any API that needs a lock on the process-wide address space, for instance, allocating a new shared memory segment), so as part of our normal performance validation work, the development team worked through these new synchronization bottlenecks and either reduced them to acceptable levels (i.e. performance similar to previous non- threaded architecture), or redesigned DB2 code to avoid the bottleneck. For your other comment (processor affinity), pretty much all platforms support this. If you're not careful though, just pinning a particular process or kernel thread to a processor can hurt performance - the OS cannot do any load balancing (and I doubt any application would be able to do a better job at load balancing than the OS, especially when there are other applications running on the box). An area that is much more interesting is memory access costs relative to particular CPUs - particularly in large AIX machines, which are becoming more and more NUMA like (Non-Uniform Memory Architecture). Even smaller AMD64 boxes have NUMA characteristics (can't recall offhand if each core has it's own memory controller, or if each chip has it's own, but it's cheaper to access memory controlled by the current CPU's memory controller than some other CPU's memory controller). Trying to solve this in large software systems that make extensive use of shared memory is very difficult (i.e. where do you try to locate a memory page, for example, a page in the bufferpool, when you don't know which CPUs may need access to that page?). Ever since DB2 v8 (before DB2 was threaded), we've tried to optimize for this type of system - if you look up the documentation on the DB2_RESOURCE_POLICY registry variable, you'll see what's available in this

On Jan 27, 12:35 pm, lfin...@ca.ibm.com wrote: > Why would it matter if the OS tries to schedule two different threads > at the same time on different CPUs?  Why would this be any different > than if the OS tried to schedule two different processes from the same > parent on different CPUs? > Gee, I don't know. It depends on the OS and hardware configuration. One issue is how does the stack get handled by multiple cpus. One common stack or a common stack that feeds in to a stack for each core? Does it differentiate on the core level or on the cpu level. (meaning different physical CPU chip) ? And if a process migrates from one core stack to another, what happens then two cpus try to attach to the same shared memory segment at the same time? There's a difference between thread and process. A process has its own copy of memory so that you can have multiple forked children on different cpus and not get in to a blocking situation. But this is getting off in to an area of discussion that is way out of scope. The point of this thread is that the numbers being tossed out by IDS and Gartner are to be taken with a large grain of salt and that the whisper numbers that we hear about from the rumor mills may not have any real meaning outside of their trending value. > > Cheers, > Liam.

Ian Michael Gumby wrote: > On Jan 27, 12:35 pm, lfin...@ca.ibm.com wrote: > But this is getting off in to an area of discussion that is way out of > scope. For once we agree. Cheers Serge -- Serge Rielau DB2 Solutions Development IBM Toronto Lab

On Jan 27, 2:04 pm, Ian Michael Gumby <im_gu...@hotmail.com> wrote: > On Jan 27, 12:35 pm, lfin...@ca.ibm.com wrote: > > > Why would it matter if the OS tries to schedule two different threads > > at the same time on different CPUs? Why would this be any different > > than if the OS tried to schedule two different processes from the same > > parent on different CPUs? > > Gee, I don't know. > It depends on the OS and hardware configuration. > > One issue is how does the stack get handled by multiple cpus. > One common stack or a common stack that feeds in to a stack for each > core? > > Does it differentiate on the core level or on the cpu level. > (meaning different physical CPU chip) ? > > And if a process migrates from one core stack to another, what happens > then two cpus try to attach to the same shared memory segment at the > same time? > > There's a difference between thread and process. A process has its own > copy of memory so that you can have multiple forked children on > different cpus and not get in to a blocking situation. > > But this is getting off in to an area of discussion that is way out of > scope. > > The point of this thread is that the numbers being tossed out by IDS > and Gartner are to be taken with a large grain of salt and that the > whisper numbers that we hear about from the rumor mills may not have > any real meaning outside of their trending value. > > > > > Cheers, > > Liam. I agree this discussion is getting way out of scope. To answer your technical issues though: - DB2 manages it's own stack memory. Each DB2 thread is given it's own stack, so it's own dedicated region of process-wide-address- space. If a thread migrates from one CPU to another, it doesn't matter, no one else is allowed to use that thread's stack. - as for two CPUs attaching to the same shared memory segment.... DB2 avoids this, since we're just threads in the same address space, we ensure that only one thread is responsible for attaching to a shared memory segment, and all other threads within the same process get the attachment for free. Even without this support though, the OS implements address space locks, so will ensure that only one segment can be mapped in at a time. Rest assured that there are lots of competent DB2 kernel developers that think of these types of issues during development :-) I'll sign off on this thread for now (unless you have more technical threading issues you'd like answered). Cheers, Liam.

On Jan 27, 9:08 pm, lfin...@ca.ibm.com wrote: > On Jan 27, 2:04 pm, Ian Michael Gumby <im_gu...@hotmail.com> wrote: > > > > > On Jan 27, 12:35 pm, lfin...@ca.ibm.com wrote: > > > > Why would it matter if the OS tries to schedule two different threads > > > at the same time on different CPUs?  Why would this be any different > > > than if the OS tried to schedule two different processes from the same > > > parent on different CPUs? > > > Gee, I don't know. > > It depends on the OS and hardware configuration. > > > One issue is how does the stack get handled by multiple cpus. > > One common stack or a common stack that feeds in to a stack for each > > core? > > > Does it differentiate on the core level or on the cpu level. > > (meaning different physical CPU chip) ? > > > And if a process migrates from one core stack to another, what happens > > then two cpus try to attach to the same shared memory segment at the > > same time? > > > There's a difference between thread and process. A process has its own > > copy of memory so that you can  have multiple forked children on > > different cpus and not get in to a blocking situation. > > > But this is getting off in to an area of discussion that is way out of > > scope. > > > The point of this thread is that the numbers being tossed out by IDS > > and Gartner are to be taken with a large grain of salt and that the > > whisper numbers that we hear about from the rumor mills may not have > > any real meaning outside of their trending value. > > > > Cheers, > > > Liam. > > I agree this discussion is getting way out of scope. > > To answer your technical issues though: > - DB2 manages it's own stack memory.  Each DB2 thread is given it's > own stack, so it's own dedicated region of process-wide-address- > space.  If a thread migrates from one CPU to another, it doesn't > matter, no one else is allowed to use that thread's stack. > - as for two CPUs attaching to the same shared memory segment.... DB2 > avoids this, since we're just threads in the same address space, we > ensure that only one thread is responsible for attaching to a shared > memory segment, and all other threads within the same process get the > attachment for free.  Even without this support though, the OS > implements address space locks, so will ensure that only one segment > can be mapped in at a time. > > Rest assured that there are lots of competent DB2 kernel developers > that think of these types of issues during development :-) > > I'll sign off on this thread for now (unless you have more technical > threading issues you'd like answered). > > Cheers, > Liam. Just one question: How does this desgin change affect the 90% code compatibility between DB2 z/OS and LUW ? It would just scare me if DB2 isn't DB2 anymore. Maybe I'm too worried about that. I should stick with my DB3...., pardon, IDS :-)

On 27 Jan, 18:35, lfin...@ca.ibm.com wrote: > On Jan 27, 9:37 am, Ian Michael Gumby <im_gu...@hotmail.com> wrote: > > > > > > > Not to bust Liam's bubble, > > > But what happens when you have two different threads from the same > > parent trying to run at the same time on different CPUs? > > > Me thinks that there's more spin and double talk that meets the eye? > > > I wonder when there will be a processor affinity switch added to the > > OSs if there isn't one already. > > > Also whenever you create a thread, any thread, the OS has some level > > of schedule over it. ;-) > > > But Hey What do I know? > > I've never looked at the source code of either IDS or DB2. ;-) > > > -G > > > On Jan 26, 9:05 am, lfin...@ca.ibm.com wrote: > > > > On Jan 25, 7:24 pm, "da...@smooth1.co.uk" <da...@smooth1.co.uk> wrote: > > > > > On 24 Jan, 22:17, Mark Townsend <markbtowns...@sbcglobal.net> wrote: > > > > > > >> Where do you think DB2 got its multi-threaded server from? > > > > > > DB2 for LUW was multi-threaded on Windows since inception > > > > > > (DB2 V5.1 I think). > > > > > > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > > > > > > You shouldn't talk about things you know nothing about. > > > > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > > > > processes now run as threads in a single process, with a single address > > > > > space. The diagram athttp://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > > > > shows this quite well > > > > >   I have some questions > > > > >  1 Can network communicating threads run in a seperate process from > > > > the rest of the engine? A process that can block inside > > > >   an OS call. Informix can with NETTYPE NET and under certain > > > > workloads this gives better performance. > > > > > 2. Can network communicating threads run in the db2sync process? This > > > > means the threads have to use network calls that > > > >    do not block. Informix can with NETTYPE CPU and under certain > > > > workloads this gives better performance due to lower > > > >    context switching. > > > > > 3. Can you have multiple db2sync processes? Otherwise how does db2 use>1 cpu? If so are session threads (EDUs) able to > > > > >    migrate between db2sync processes? I believe Sybase statically > > > > binds sessons to engines hence if the engine is busy > > > >    running the thread for one engine then other threads on that engine > > > > cannot migrate to idle engines! Being able to migrate > > > >    threads between engines was one of Informixs advantages over other > > > > multi-thread engines. > > > > > > Biggest advantage is that shared memory management becomes easier (for > > > > > the developers, mainly). > > > > >  Nice to see DB2 catching up with Informix. Pity DB2 cannot > > > > dynamically add shared memory segments on Linux. > > > > > > Not at all like what Informix did when they rewrote the engine from 5 to > > > > > 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but > > > > > introduced a completely different set. > > > > Hi David, > > > > DB2's threading model is different than IDS.  With DB2, each thread > > > maps directly to a kernel thread, so each can be individually > > > scheduled by the OS.  So, if any one DB2 thread is blocked, the OS is > > > free to schedule any other DB2 thread on that same processor, or any > > > other processor.  That also means that as one CPU gets too overloaded, > > > the OS is free to re-schedule any DB2 threads on other, less loaded, > > > CPUs, as it sees fit. > > > > One of the main benefits with going threaded on UNIX/Linux is the > > > simplification to our memory model, both internally (for development), > > > but more importantly for customers, in terms of ease of > > > administration.  The following sections in the docs highlight these > > > changes:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > > The following link gives a little more detail on how the memory > > > parameters and the self-tuning memory manager interact with each > > > other:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > > > For your last question, DB2 does in fact allocate new shared memory > > > segments during runtime.  If you want to see this in practice, just > > > activate your database, then dynamically create a new bufferpool (or > > > dynamically increase your locklist, shared sort heap, etc).  In 9.5, > > > this operation should always succeed, and if you issue 'db2pd -db > > > dbname -memsets', you should see that there are multiple shared memory > > > segments now associated with your database.  If you tried this same > > > exercise in DB2 9 or earlier, this same operation would have likely > > > failed, since we could not dynamically allocate new shared memory > > > segments during runtime. > > > > Cheers, > > > Liam. > > Why would it matter if the OS tries to schedule two different threads > at the same time on different CPUs?  Why would this be any different > than if the OS tried to schedule two different processes from the same > parent on different CPUs? > > When DB2 creates threads, we specify the 1:1 model, where each thread > maps to it's own kernel thread.  So, when scheduling, each kernel > thread is seen as a distinct schedulable entity, just as if it was a > separate process.  The only difference is that depending on the > platform, the OS can usually context switch faster between two kernel > threads within the same process.  Going to a threaded model does mean > though that there will now be OS kernel contention for some OS APIs > (i.e. any API that needs a lock on the process-wide address space, for > instance, allocating a new shared memory segment), so as part of our > normal performance validation work, the development team worked > through these new synchronization bottlenecks and either reduced them > to acceptable levels (i.e. performance similar to previous non- > threaded architecture), or redesigned DB2 code to avoid the > bottleneck. > > For your other comment (processor affinity), pretty much all platforms > support this.  If you're not careful though, just pinning a particular > process or kernel thread to a processor can hurt performance - the OS > cannot do any load balancing (and I doubt any application would be > able to do a better job at load balancing than the OS, especially when > there are other applications running on the box).  An area that is > much more interesting is memory access costs relative to particular > CPUs - particularly in large AIX machines, which are becoming more and > more NUMA like (Non-Uniform Memory Architecture).  Even smaller AMD64 > boxes have NUMA characteristics (can't recall offhand if each core has > it's own memory controller, or if each chip has it's own, but it's > cheaper to access memory controlled by the current CPU's memory > controller than some other CPU's memory controller).  Trying to solve > this in large software systems that make extensive use of shared > memory is very difficult (i.e. where do you try to locate a memory > page, for example, a page in the

eric@herber-consulting.de wrote: > Just one question: > How does this design change affect the 90% code compatibility between > DB2 z/OS and LUW ? A design change does not affect DB2 LUW to DB2 zOS LANGUAGE compatibility. What is "code compatible" anyways? If any IBMer told you that you that DB2 for LUW and Db2 for zOS uses shared code on the server uses the same code, please send me teh name and I'll educate them. DB2 for LUW is >90% SHARED code across all supported platforms. DB2 is DB2 is DB2 wherever Oracle is Oracle is Oracle. And unlike Oracle is ships on all those platforms concurrently, ships APARs on all those platforms and doesn't need hordes of developers for each of these platforms. I have no problem with you sticking with IDS. Jonathan and I get paid from the same pot of gold. I do have a problem with DB2 being dissed based on inaccurate information. Cheers Serge -- Serge Rielau DB2 Solutions Development IBM Toronto Lab

On Jan 28, 5:21 am, Serge Rielau <srie...@ca.ibm.com> wrote: > e...@herber-consulting.de wrote: > > Just one question: > > How does this design change affect the 90% code compatibility between > > DB2 z/OS and LUW ? > > A design change does not affect DB2 LUW to DB2 zOS LANGUAGE compatibility. > What is "code compatible" anyways? > > If any IBMer told you that you that DB2 for LUW and Db2 for zOS uses > shared code on the server uses the same code, please send me teh name > and I'll educate them. > DB2 for LUW is >90% SHARED code across all supported platforms. > > DB2 is DB2 is DB2 wherever Oracle is Oracle is Oracle. > And unlike Oracle is ships on all those platforms concurrently, ships > APARs on all those platforms and doesn't need hordes of developers for > each of these platforms. > > I have no problem with you sticking with IDS. Jonathan and I get paid > from the same pot of gold. > I do have a problem with DB2 being dissed based on inaccurate information. > > Cheers > Serge > -- > Serge Rielau > DB2 Solutions Development > IBM Toronto Lab Serge, you lost your sense of humor. Let's agree that DB2is DB2 is DB2 like IDS is: Informix Dynamic Server. To say it in german: "Nur wo Nutella draufsteht, ist auch Nutella drin..." :-) Take care.

Hmmm Nuella :-) Es gibt gewisse "Elemente" hier die echte Nervensaegen sind. Und nach acht Jahren finde ich diesen Schwachsinn echt nicht mehr witzig. Serge -- Serge Rielau DB2 Solutions Development IBM Toronto Lab

Serge Rielau wrote: > Hmmm Nuella :-) > > Es gibt gewisse "Elemente" hier die echte Nervensaegen sind. > Und nach acht Jahren finde ich diesen Schwachsinn echt nicht mehr witzig. Und er ist auch ein grosser arschloch. :o) -- Cheers, Obnoxio The Clown http://obotheclown.blogspot.com -- This message has been scanned for viruses and dangerous content by OpenProtect(http://www.openprotect.com), and is believed to be clean.

In article <6uav1aFeh1jpU1@mid.individual.net>, Serge Rielau says... > >Hmmm Nuella :-) > >Es gibt gewisse "Elemente" hier die echte Nervensaegen sind. >Und nach acht Jahren finde ich diesen Schwachsinn echt nicht mehr witzig. > >Serge Translated for those who are proficient in C/Java but not German. Hmmm Nuella:-) There are certain "elements" here the real pains in the neck are. And after eight years I find this nonsense really no more witty.

On Jan 27, 6:27 pm, "da...@smooth1.co.uk" <da...@smooth1.co.uk> wrote: > On 27 Jan, 18:35, lfin...@ca.ibm.com wrote: > > > > > On Jan 27, 9:37 am, Ian Michael Gumby <im_gu...@hotmail.com> wrote: > > > > Not to bust Liam's bubble, > > > > But what happens when you have two different threads from the same > > > parent trying to run at the same time on different CPUs? > > > > Me thinks that there's more spin and double talk that meets the eye? > > > > I wonder when there will be a processor affinity switch added to the > > > OSs if there isn't one already. > > > > Also whenever you create a thread, any thread, the OS has some level > > > of schedule over it. ;-) > > > > But Hey What do I know? > > > I've never looked at the source code of either IDS or DB2. ;-) > > > > -G > > > > On Jan 26, 9:05 am, lfin...@ca.ibm.com wrote: > > > > > On Jan 25, 7:24 pm, "da...@smooth1.co.uk" <da...@smooth1.co.uk> wrote: > > > > > > On 24 Jan, 22:17, Mark Townsend <markbtowns...@sbcglobal.net> wrote: > > > > > > > >> Where do you think DB2 got its multi-threaded server from? > > > > > > > DB2 for LUW was multi-threaded on Windows since inception > > > > > > > (DB2 V5.1 I think). > > > > > > > All that was done in DB2 9.5 was to extend the concept to Unix/Linux. > > > > > > > > You shouldn't talk about things you know nothing about. > > > > > > > And I wouldn't really call it multi-threaded. Rather just a bunch of > > > > > > processes now run as threads in a single process, with a single address > > > > > > space. The diagram athttp://www.ibm.com/developerworks/db2/library/techarticle/dm-0807khar... > > > > > > shows this quite well > > > > > > I have some questions > > > > > > 1 Can network communicating threads run in a seperate process from > > > > > the rest of the engine? A process that can block inside > > > > > an OS call. Informix can with NETTYPE NET and under certain > > > > > workloads this gives better performance. > > > > > > 2. Can network communicating threads run in the db2sync process? This > > > > > means the threads have to use network calls that > > > > > do not block. Informix can with NETTYPE CPU and under certain > > > > > workloads this gives better performance due to lower > > > > > context switching. > > > > > > 3. Can you have multiple db2sync processes? Otherwise how does db2 use>1 cpu? If so are session threads (EDUs) able to > > > > > > migrate between db2sync processes? I believe Sybase statically > > > > > binds sessons to engines hence if the engine is busy > > > > > running the thread for one engine then other threads on that engine > > > > > cannot migrate to idle engines! Being able to migrate > > > > > threads between engines was one of Informixs advantages over other > > > > > multi-thread engines. > > > > > > > Biggest advantage is that shared memory management becomes easier (for > > > > > > the developers, mainly). > > > > > > Nice to see DB2 catching up with Informix. Pity DB2 cannot > > > > > dynamically add shared memory segments on Linux. > > > > > > > Not at all like what Informix did when they rewrote the engine from 5 to > > > > > > 6 (7) to become multi-threaded. Which, BTW, solved on set of issues, but > > > > > > introduced a completely different set. > > > > > Hi David, > > > > > DB2's threading model is different than IDS. With DB2, each thread > > > > maps directly to a kernel thread, so each can be individually > > > > scheduled by the OS. So, if any one DB2 thread is blocked, the OS is > > > > free to schedule any other DB2 thread on that same processor, or any > > > > other processor. That also means that as one CPU gets too overloaded, > > > > the OS is free to re-schedule any DB2 threads on other, less loaded, > > > > CPUs, as it sees fit. > > > > > One of the main benefits with going threaded on UNIX/Linux is the > > > > simplification to our memory model, both internally (for development), > > > > but more importantly for customers, in terms of ease of > > > > administration. The following sections in the docs highlight these > > > > changes:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > > > The following link gives a little more detail on how the memory > > > > parameters and the self-tuning memory manager interact with each > > > > other:https://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic... > > > > > For your last question, DB2 does in fact allocate new shared memory > > > > segments during runtime. If you want to see this in practice, just > > > > activate your database, then dynamically create a new bufferpool (or > > > > dynamically increase your locklist, shared sort heap, etc). In 9.5, > > > > this operation should always succeed, and if you issue 'db2pd -db > > > > dbname -memsets', you should see that there are multiple shared memory > > > > segments now associated with your database. If you tried this same > > > > exercise in DB2 9 or earlier, this same operation would have likely > > > > failed, since we could not dynamically allocate new shared memory > > > > segments during runtime. > > > > > Cheers, > > > > Liam. > > > Why would it matter if the OS tries to schedule two different threads > > at the same time on different CPUs? Why would this be any different > > than if the OS tried to schedule two different processes from the same > > parent on different CPUs? > > > When DB2 creates threads, we specify the 1:1 model, where each thread > > maps to it's own kernel thread. So, when scheduling, each kernel > > thread is seen as a distinct schedulable entity, just as if it was a > > separate process. The only difference is that depending on the > > platform, the OS can usually context switch faster between two kernel > > threads within the same process. Going to a threaded model does mean > > though that there will now be OS kernel contention for some OS APIs > > (i.e. any API that needs a lock on the process-wide address space, for > > instance, allocating a new shared memory segment), so as part of our > > normal performance validation work, the development team worked > > through these new synchronization bottlenecks and either reduced them > > to acceptable levels (i.e. performance similar to previous non- > > threaded architecture), or redesigned DB2 code to avoid the > > bottleneck. > > > For your other comment (processor affinity), pretty much all platforms > > support this. If you're not careful though, just pinning a particular > > process or kernel thread to a processor can hurt performance - the OS > > cannot do any load balancing (and I doubt any application would be > > able to do a better job at load balancing than the OS, especially when > > there are other applications running on the box). An area that is > > much more interesting is memory access costs relative to particular > > CPUs - particularly in large AIX machines, which are becoming more and > > more NUMA like (Non-Uniform Memory Architecture). Even smaller AMD64 > > boxes have NUMA characteristics (can't recall offhand if each core has > > it's own memory controller, or if each chip has it's own, but it's > > cheaper to access memory controll