Wanna try SolidDB for IDS?
Not a troubleshooting thread: someone posted a link to IBM's solidDB (in-memory database/cache for IDS and DB2) download page, and Neil Truby asked what it is actually good for and what the trade-offs are, noting the steep ~US$28,500-30,500 per-core list price (and that the UK is quoted higher). Replies suggested it suits microsecond-latency lookups against streaming data (e.g. CDR processing) and very high availability, while warning of cost, SQL/functionality differences versus IDS, loss of extensibility, and its niche rather than general-purpose nature; there was disagreement over durability and whether home-grown in-memory approaches (e.g. Kevin Brown's Real Time Loader/time series work) are better. No firm conclusion or recommendation is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
http://www.ibm.com/developerworks/downloads/im/soliddb/learn.html -- Bye now, Obnoxio http://obotheclown.blogspot.com/
>> "Obnoxio The Clown" <obnoxio@serendipita.com> wrote in message >> news:mailman.1469.1217974618.20610.informix-list@iiug.org... http://www.ibm.com/developerworks/downloads/im/soliddb/learn.html I've thought of downloading this several times, but would really like to understand it better, and can't really get the info I'd like from the various factsheets and wepages. What particualr circumstances would it be really good for? The Highlights are described as: Accelerates Access to Data in DB2 and IDS Achieves Extreme Speed with In-Memory Database Technology Keeps Data Persistent and Recoverable Combines High Availability and Extreme Speed Balances Data Safety, Application Throughput and Recovery Time Lowers costs But there must be some disadvantages too, as all these technologies involve trade-offs; ie there must be some circumstances where its use would have negligible effect or perhaps even be negative. Can anyone describe what these are? Basically, it looks quite sexy and I'd like to try it, but at '30,500 per core to licence, I really am going to have a compelling reason even to trial it. Thanks Neil
> Basically, it looks quite sexy and I'd like to try it, but at '30,500 per > core to licence, I really am going to have a compelling reason even to > trial it. In fact the UK price, as quoted on the IBM website, is US$30,500 per core, not '30,500 (not sure why the UK price is quoted in US dollars). Interestingly, the US price is US$28,500! I feel a letter to the Daily Mail about "Rip-off Britain" coming on ...
Neil Truby said: > --===============1548689465== > >>> "Obnoxio The Clown" <obnoxio@serendipita.com> wrote in message >>> news:mailman.1469.1217974618.20610.informix-list@iiug.org... > http://www.ibm.com/developerworks/downloads/im/soliddb/learn.html > > I've thought of downloading this several times, but would really like to > understand it better, and can't really get the info I'd like from the > various factsheets and wepages. > > What particualr circumstances would it be really good for? Because it was designed solely as an in-memory database, the access mechanisms and so on are roughly an order of magnitude quicker than IDS. So if you want to access rows in microseconds, rather than milliseconds, it's your man. > The Highlights are described as: > > Accelerates Access to Data in DB2 and IDS > Achieves Extreme Speed with In-Memory Database Technology > Keeps Data Persistent and Recoverable > Combines High Availability and Extreme Speed > Balances Data Safety, Application Throughput and Recovery Time > Lowers costs > > But there must be some disadvantages too, as all these technologies > involve > trade-offs; ie there must be some circumstances where its use would have > negligible effect or perhaps even be negative. Can anyone describe what > these are? My best guess would be the price. > Basically, it looks quite sexy and I'd like to try it, but at £30,500 per > core to licence, I really am going to have a compelling reason even to > trial > it. Ah! My best guess was a good one! You will also find some other things, such as possibly a slight difference in SQL and limits in the functionality it offers. I'm not sure, but I think you'd probably lose the option of extensibility, too. So you might have to re-code specific applications. I think it's a niche database, a bit like RedBrick was: outstanding at what it was designed to do, pretty much useless at being a general purpose database. In general, I think if you were out of its target market, using it to address performance issues would be much the same as chucking tin at a performance problem: it might get you out of it, but it wouldn't address the root cause, which would probably just be a missing index somewhere. :o) -- Bye now, Obnoxio http://obotheclown.blogspot.com/
Neil Truby said: > --===============1252718721== > >> Basically, it looks quite sexy and I'd like to try it, but at £30,500 >> per >> core to licence, I really am going to have a compelling reason even to >> trial it. > > In fact the UK price, as quoted on the IBM website, is US$30,500 per core, > not £30,500 (not sure why the UK price is quoted in US dollars). > > Interestingly, the US price is US$28,500! I feel a letter to the Daily > Mail > about "Rip-off Britain" coming on ... Yeah, but they've got to get it all the way over here, Neil.... :o> -- Bye now, Obnoxio http://obotheclown.blogspot.com/
Neil Truby wrote: >>> "Obnoxio The Clown" <obnoxio@serendipita.com> wrote in message >>> news:mailman.1469.1217974618.20610.informix-list@iiug.org... > http://www.ibm.com/developerworks/downloads/im/soliddb/learn.html > > I've thought of downloading this several times, but would really like to > understand it better, and can't really get the info I'd like from the > various factsheets and wepages. > > What particualr circumstances would it be really good for? > > The Highlights are described as: > > Accelerates Access to Data in DB2 and IDS > Achieves Extreme Speed with In-Memory Database Technology > Keeps Data Persistent and Recoverable > Combines High Availability and Extreme Speed > Balances Data Safety, Application Throughput and Recovery Time > Lowers costs > > But there must be some disadvantages too, as all these technologies involve > trade-offs; ie there must be some circumstances where its use would have > negligible effect or perhaps even be negative. Can anyone describe what > these are? > > Basically, it looks quite sexy and I'd like to try it, but at '30,500 per > core to licence, I really am going to have a compelling reason even to trial > it. > > Thanks > Neil > > Basically, I'd say three: - Performance - Possible to "link" it in an application more or less in a shared library... (solidDB, not sure about Solid cache) - Extremely high availability (maybe more than IDS itself) But take in consideration that "performance" is not the same concept here as we're used too... It should be another order of magnitude. One scenario where it can be used is a constant "lookup" you have to do to perform some operation on a stream of data. I had a situation some years ago where this would have helped. A guy (very techie person) came up to me and asked me to optimize a query he had to do constantly to process a stream of CDRs. But, as he was a very good programmer he already did everything (indexes, prepared statements etc.). But it simply didn't run faster (TCP to exchange data with the database, fetching etc.). He ended up by loading it into memory and created a very simple expression parser to allow him some flexibility in the "queries". It got a lot faster. The result? Well, he met his objectives and the company got a piece of code that no other person could maintain :) Today I would have suggested SolidDB Cache for IDS probably... And possibly they would refuse it because of the price :P A usual misconception: Being in memory, you risk loss of data: simply NOT TRUE. Besides, if you take a look at the functionality specs you'll find a lot of similarities with IDS: checkpoints (non blocking) and HDR (with rolling upgrades) come to mind. So, very good, although I'd like to see more features when acting as cache for DB2/IDS. Regards. -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently...
On Aug 5, 6:20 pm, "Obnoxio The Clown" <obno...@serendipita.com> wrote: > Neil Truby said: > > > What particualr circumstances would it be really good for? > > Because it was designed solely as an in-memory database, the access > mechanisms and so on are roughly an order of magnitude quicker than IDS. > So if you want to access rows in microseconds, rather than milliseconds, > it's your man. > Actually no. Did you ever look at Kevin Brown's stuff? (Real TIme Loader) It loaded the inserts into a shared memory segment that to the database looked like a table, and could be read as a table until it was buffered in to the actual database table. This is how you could load 25K real time ticks per "node". (Of course its not just the high speed of loading but that you could actually query said data too.) This was 1/3 of the Financial datablade bundle that got caught in the chaos when IBM stepped in. > > The Highlights are described as: > > > Accelerates Access to Data in DB2 and IDS > > Achieves Extreme Speed with In-Memory Database Technology > > Keeps Data Persistent and Recoverable > > Combines High Availability and Extreme Speed > > Balances Data Safety, Application Throughput and Recovery Time > > Lowers costs > > > But there must be some disadvantages too, as all these technologies > > involve > > trade-offs; ie there must be some circumstances where its use would have > > negligible effect or perhaps even be negative. Can anyone describe what > > these are? > > My best guess would be the price. No, go beyond that. Kick the plug out of the wall. Poof! Your database is gone. Also if you're bitching about the price, how much does the kit cost that has enough CPU horsepower, along with buying the memory? 32GB? 64GB? 128GB of memory? (And then thing about what happens if , heaven forbid, you have to *gasp* swap? ) Of course if you're spending $500,000 (USD) on hardware, 30K for an in memory DB is cheap.But that's list. Wait to you see the pricing if you're getting J level discounts to start with and then negotiate a deal off of that. ;-) (Ooops! Yes it does happen. ;-) > > > Basically, it looks quite sexy and I'd like to try it, but at £30,500 per > > core to licence, I really am going to have a compelling reason even to > > trial > > it. > > Ah! My best guess was a good one! > > You will also find some other things, such as possibly a slight difference > in SQL and limits in the functionality it offers. I'm not sure, but I > think you'd probably lose the option of extensibility, too. So you might > have to re-code specific applications. > > I think it's a niche database, a bit like RedBrick was: outstanding at > what it was designed to do, pretty much useless at being a general purpose > database. > > In general, I think if you were out of its target market, using it to > address performance issues would be much the same as chucking tin at a > performance problem: it might get you out of it, but it wouldn't address > the root cause, which would probably just be a missing index somewhere. > :o) No there's more to it than that. You never defined the "target" market. I can think of several. Especially when you get in to cloud and grid computing. Maybe that's why you're not seeing an IDS driven TPC-E benchmark. Maybe they decided to skip IDS and then do one with "Solid". Then you'll see Oracle retaliate with their times ten. Of course I do wonder since TPC-E is a stock simulation, I wonder why they don't do the IDS benchmark using the two components of the financial foundation? Time Series and Real Time Loader? Sorry, there are some applications, where this makes sense. However, if TimesTen's lessons teach us anything, its not a long term viable platform on its own.
On Aug 5, 7:43 pm, Fernando Nunes <domusonl...@gmail.com> wrote: > Neil Truby wrote: > >>> "Obnoxio The Clown" <obno...@serendipita.com> wrote in message > >>>news:mailman.1469.1217974618.20610.informix-list@iiug.org... > >http://www.ibm.com/developerworks/downloads/im/soliddb/learn.html > > > I've thought of downloading this several times, but would really like to > > understand it better, and can't really get the info I'd like from the > > various factsheets and wepages. > > > What particualr circumstances would it be really good for? > > > The Highlights are described as: > > > Accelerates Access to Data in DB2 and IDS > > Achieves Extreme Speed with In-Memory Database Technology > > Keeps Data Persistent and Recoverable > > Combines High Availability and Extreme Speed > > Balances Data Safety, Application Throughput and Recovery Time > > Lowers costs > > > But there must be some disadvantages too, as all these technologies involve > > trade-offs; ie there must be some circumstances where its use would have > > negligible effect or perhaps even be negative. Can anyone describe what > > these are? > > > Basically, it looks quite sexy and I'd like to try it, but at £30,500 per > > core to licence, I really am going to have a compelling reason even to trial > > it. > > > Thanks > > Neil > > Basically, I'd say three: > - Performance > - Possible to "link" it in an application more or less in a shared library... > (solidDB, not sure about Solid cache) > - Extremely high availability (maybe more than IDS itself) > Depends. Kevin Brown did an interesting thing with IDS in his real time loader and his time series stuff. Had he taken it a step farther, you'd have the ability to create a "in memory" table or rather table space. (Yes there are still some desirable design considerations so I won't discuss them here. ;-) > But take in consideration that "performance" is not the same concept here as > we're used too... It should be another order of magnitude. > Again, not necessarily. It depends on a couple of factors, one of which is hardware. > One scenario where it can be used is a constant "lookup" you have to do to > perform some operation on a stream of data. > I had a situation some years ago where this would have helped. A guy (very > techie person) came up to me and asked me to optimize a query he had to do > constantly to process a stream of CDRs. But, as he was a very good programmer > he already did everything (indexes, prepared statements etc.). But it simply > didn't run faster (TCP to exchange data with the database, fetching etc.). > This gets in to cloud computing and in to grid computing. Unfortunately I am not legally allowed to talk about one such a scenario. > He ended up by loading it into memory and created a very simple expression > parser to allow him some flexibility in the "queries". It got a lot faster. > The result? Well, he met his objectives and the company got a piece of code > that no other person could maintain :) > Sounds like you have two issues. One is that he didn't mentor co- workers and the other is that he didn't document his work. There are a lot of ways you can do things in memory that do not require a general purpose database. Again, when looking at cloud computing and grid computing, you may be better off creating your own in house solution. (Its called protecting your IP along with getting a specialized system tuned to your needs). > A usual misconception: Being in memory, you risk loss of data: simply NOT TRUE. > Uhm yes, you do. Even with "flash", you run the risk of data loss. Even with hard drives you run the risk. I would guess that this has one advantage over taking what Kevin did and extending it. Kevin's solution was IDS specific. Solid is database agnostic.