Re: Wanna try SolidDB for IDS?

Ian Michael Gumby wrote: > On Aug 5, 7:43 pm, Fernando Nunes <domusonl...@gmail.com> wrote: >> A usual misconception: Being in memory, you risk loss of data: simply NOT TRUE. >> > Uhm yes, you do. > Even with "flash", you run the risk of data loss. Even with hard > drives you run the risk. > > I would guess that this has one advantage over taking what Kevin did > and extending it. Kevin's solution was IDS specific. Solid is database > agnostic. You're more and more like another character that likes to post here... Please, check things first, at least when you pretend to be so sure. As you surely know (and I really believe this), it's not the disk that prevents data loss in relational databases. It's the logging. And you can (and in many situation you should) do logging with in memory databases. So, to be clear: You DON'T LOOSE data if you have a power outage. UNLESS of course you CHOOSE to loose it by not doing logging which may be an acceptable risk in some situations, if you really need the extra performance you may get by not doing logging. if you're thinking about the fact that you may loose your logging, then, yes, but there is disk redundancy etc... The rest are bugs, and any software may have them. Regards. -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently...

Sigh. Lets try this again.... You said: " A usual misconception: Being in memory, you risk loss of data: simply NOT TRUE." And I will repeat what I've said... You will always have the "risk" of data loss. First, I guess you need to understand what "risk" means. Second I think you need to understand that all electronic forms of data have a risk of data loss. You have something stored in RAM. You lose power to your RAM, you lost what was in memory. Flash? You can only read/write to flash so many times. You therefore run the risk of losing something written to a flash drive as that spot in flash memory becomes unusable. Hard Drive? Geez have you ever heard of a head crash? No? Then you must be new to IT. If you have data on a hard drive, and it crashes, you have just lost your data. (Wonder why you do tape backups or put your data base on a RAID system?) Ok, so now you have your data stored on a RAID system. But your server just got fried from your building being hit by lighting. So you've lost your entire raid subsystem. Aren't you glad that you have installed some form of HA/ER with a redundant server at your alternative data site? (And if you want a real life example. Think back to 9/11. ) Companies found that you need to replicate data to a secondary site in order to either remain up, or reduce the time it takes to come back online. (Of course if you're HXXX, you need to make sure that the "dark fiber" connecting your two sites is marked on the maps so that constuction companies don't accidently cut the fiber, or that you actually test your failover system and make sure that it works. But that's another story...) The point is that you always have risk of data loss. How large said risk is will depend on the medium and redundancy of your system. The fact is that an "in memory" system will face the issue of some data loss. Since you pretend to know all about IDS, talk to Kevin Brown or Alan Caldera (last time I checked, he's the product manager for datablades) and talk to them about the "real time loader". (You may also want to e-mail Don Payne if he's still around. Or go bother Pickford if you'd like since he's in the UK and closer to you...) The point is that with the RTL, the feed(s) are written to shared memory for immediate use and then it gets buffered to disk. If you have a power loss prior to the data being buffered to disk, you run the risk of losing said data. The work around was to tee (a unix term) the data feed and write to a flat file. Looking at any design of an "in memory" database, you're still going to write the data to disk at some point. Since there is a delta between what's in memory and what's being buffered to disk, any time you have a power loss, said delta will be lost. I really wish you would think about how things work, before you post the dribble that you get in marketing pitches. > From: domusonline@gmail.com > Subject: Re: Wanna try SolidDB for IDS? > Date: Thu, 7 Aug 2008 19:21:40 +0100 > To: informix-list@iiug.org > > Ian Michael Gumby wrote: > > On Aug 5, 7:43 pm, Fernando Nunes <domusonl...@gmail.com> wrote: > > >> A usual misconception: Being in memory, you risk loss of data: simply NOT TRUE. > >> > > Uhm yes, you do. > > Even with "flash", you run the risk of data loss. Even with hard > > drives you run the risk. > > > > I would guess that this has one advantage over taking what Kevin did > > and extending it. Kevin's solution was IDS specific. Solid is database > > agnostic. > > You're more and more like another character that likes to post here... Please, > check things first, at least when you pretend to be so sure. > > As you surely know (and I really believe this), it's not the disk that prevents > data loss in relational databases. It's the logging. And you can (and in many > situation you should) do logging with in memory databases. > > So, to be clear: You DON'T LOOSE data if you have a power outage. UNLESS of > course you CHOOSE to loose it by not doing logging which may be an acceptable > risk in some situations, if you really need the extra performance you may get > by not doing logging. > > if you're thinking about the fact that you may loose your logging, then, yes, > but there is disk redundancy etc... > The rest are bugs, and any software may have them. > > Regards. > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list _________________________________________________________________ Get more from your digital life. Find out how. http://www.windowslive.com/default.html?ocid=TXT_TAGLM_WL_Home2_082008

Ian Michael Gumby wrote: > Sigh. > > Lets try this again.... > > You said: > " A usual misconception: Being in memory, you risk loss of data: simply > NOT TRUE." > > And I will repeat what I've said... > > You will always have the "risk" of data loss. Sure. But that's because what you say: We're dealing with bits and bytes. Not because we're using a in-memory database. So, hopefully, you'll understand why I said "a usual misconception". If you don't have that misconception fine. > You have something stored in RAM. You lose power to your RAM, you lost > what was in memory. Losing what is in memory is not losing data. > The fact is that an "in memory" system will face the issue of some data > loss. Exactly the same as in IDS or any other RDBMS. If the redundancy and the logging fails, you loose data. > The point is that with the RTL, the feed(s) are written to shared memory > for immediate use and then it gets buffered to disk. If you have a power > loss prior to the data being buffered to disk, you run the risk of > losing said data. The work around was to tee (a unix term) the data feed > and write to a flat file. > There are several points here: 1- I didn't talk about RTL, neither did the OP. So that part of the conversation although very interesting is pointless to the fact that SolidDB looses data as any other RDBMS, and not a bit more because it's an in-memory database 2- You need to make a point that proves you're right. I really don't mind about that (not even care), but it's a bit tiring... > Looking at any design of an "in memory" database, you're still going to > write the data to disk at some point. Since there is a delta between > what's in memory and what's being buffered to disk, any time you have a > power loss, said delta will be lost. As other DBMS, you write committed transactions to a log file that will permit you to recover all your data (consistently). The commit return is sent after logging. > I really wish you would think about how things work, before you post the > dribble that you get in marketing pitches. I do... And I try to stick to the point. There is no marketing here. Only pure technological concepts, that as I said, I'm sure you're aware and know about them. It's a shame that you take this as a championship to see who is right, so you start wondering about "there is always space for failure", and "secondary sites" etc. That is all true BUT it's true for ALL RDBMS, including SolidDB, and not specially true for SolidDB, and not specially true because it's an in-memory RDBMS. The point here should be not to confuse other people. And the original point still stands: There is a misconception that in-memory databases loose data because they are in memory. This is simply not true. In SolidDB case (and I believe in TimesTen is the same, but Mark can answer for that), you CAN configure it to don't log... but it's your choice... As usual, performance vs security. Regards, -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently...

Backpeddling are we? Lets face it you made an incorrect generalization and then tried to explain to me why I was wrong when I wasn't. You need to understand risk and how to assess the risk and the alternatives. Then decide whether or not its worth the price to be risk adverse. I have a production database sitting on a single SATA drive. It was a proof of concept that ended up going live in production as a SAAS app. I now have to decide what I want to do to reduce the risk of a disk failure and data loss. I could mirror the drive. I could add a raid card and SAS drives then create a new copy of the database. Maybe consider porting it to Informix. But I'm not making enough money from the SAS to pay for the IDS license. The reality is that each option reduces the risk(s) and increases the probability of uptime. But with each option there is a cost and I have to decide whether the cost is enough to offset the risk. When you talk about an in-memory database, you are creating a db that has fairly fast response time. But the risk is that if there is a power failure, there is going to be data loss. There are ways to mitigate this risk such that you can live with an in memory database. Had IBM not purchased Informix from ASS-N-TAIL, the concept of in memory tables would have been a logical extension to the work Kevin Brown did with the real time loader. (So give credit where credit is due.) What really sucks is that neither IBM nor Informix really got a chance to market Financial Foundation. Terry was on the way and had some really brilliant ideas in the UK. But alas, PG sold us out to IBM... (Don't believe me, as Pickford) On Aug 7, 4:58 pm, Fernando Nunes <domusonl...@gmail.com> wrote: > Ian Michael Gumby wrote: > > Sigh. > > > Lets try this again.... > > > You said: > > " A usual misconception: Being in memory, you risk loss of data: simply > > NOT TRUE." > > > And I will repeat what I've said... > > > You will always have the "risk" of data loss. > > Sure. But that's because what you say: We're dealing with bits and bytes. Not > because we're using a in-memory database. So, hopefully, you'll understand why > I said "a usual misconception". If you don't have that misconception fine. > > > You have something stored in RAM. You lose power to your RAM, you lost > > what was in memory. > > Losing what is in memory is not losing data. > > > The fact is that an "in memory" system will face the issue of some data > > loss. > > Exactly the same as in IDS or any other RDBMS. If the redundancy and the > logging fails, you loose data. > > > The point is that with the RTL, the feed(s) are written to shared memory > > for immediate use and then it gets buffered to disk. If you have a power > > loss prior to the data being buffered to disk, you run the risk of > > losing said data. The work around was to tee (a unix term) the data feed > > and write to a flat file. > > There are several points here: > 1- I didn't talk about RTL, neither did the OP. So that part of the > conversation although very interesting is pointless to the fact that SolidDB > looses data as any other RDBMS, and not a bit more because it's an in-memory > database > 2- You need to make a point that proves you're right. I really don't mind about > that (not even care), but it's a bit tiring... > > > Looking at any design of an "in memory" database, you're still going to > > write the data to disk at some point. Since there is a delta between > > what's in memory and what's being buffered to disk, any time you have a > > power loss, said delta will be lost. > > As other DBMS, you write committed transactions to a log file that will permit > you to recover all your data (consistently). The commit return is sent after > logging. > > > I really wish you would think about how things work, before you post the > > dribble that you get in marketing pitches. > > I do... And I try to stick to the point. There is no marketing here. Only pure > technological concepts, that as I said, I'm sure you're aware and know about > them. It's a shame that you take this as a championship to see who is right, so > you start wondering about "there is always space for failure", and "secondary > sites" etc. That is all true BUT it's true for ALL RDBMS, including SolidDB, > and not specially true for SolidDB, and not specially true because it's an > in-memory RDBMS. > The point here should be not to confuse other people. > And the original point still stands: There is a misconception that in-memory > databases loose data because they are in memory. This is simply not true. > In SolidDB case (and I believe in TimesTen is the same, but Mark can answer for > that), you CAN configure it to don't log... but it's your choice... As usual, > performance vs security. > > Regards, > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently...