RE: Webcast - Rapid Data Warehouse Development - Build a Pilot in 8 weeks!
No Lester, you missed the point. First the data warehouse I talked about that would be 2.7 PB was First Union's marketing database that Informix did back in the 90's. If you were to scale that and put it in to perspective with today's DW practices, you'd see just how large the system was. XPS was capable of handling that large of volume and scale in terms of near linear growth. If you were to look at Twitter, Facebook, Yahoo! and of course Google, they all capture really massive amounts of data and they need to be able to process it in relatively short periods of time. I can't go in to details on some of these because I'm not allowed to go in to details. But if you look at the open presentations made by these companies at Hadoop World (Not google since they have their own big table ...) you will see some interesting very large DW. HBase isn't perfect, and truthfully I've only started to scratch the surface of what it can do. But yeah, for an open source project, companies are moving in that direction because of the cost of doing business with the likes of IBM and Oracle. I'm seeing more companies move towards postgress because they can get a database for free that will handle their needs. Companies have stuck their little toe in to the pool of Open Source. There isn't the perceived risk. So you're finding companies that will switch to lower cost alternatives than those provided by IBM or Oracle. Like I said, for the cost of IDS EE on a single 8core Xeon box, you can buy 5-10 servers to be put in a cloud. The range is because I don't know your discount, or how much you spend on a box. Oh and one last pearl of wisdom. 'One man's junk is another man's treasure.' ;-) Once you understand that, they you'll start to realize just how dangerous Google really is. ;-) -G > Date: Mon, 18 Jan 2010 22:08:06 -0500 > From: lester@iiug.org > To: im_gumby@hotmail.com > CC: obnoxio@serendipita.com; informix-list@iiug.org > Subject: Re: Webcast - Rapid Data Warehouse Development - Build a Pilot in 8 weeks! > > Gumby, > > You are still missing the point, having a couple of petabytes of data on fast hardware and software does not make a data warehouse. Most of the time it ends up being a data junk yard. > > The point is how do you quickly design and build something so users can gain insight into their data and make meaningful decisions. > > Regards - Lester > > Ian Michael Gumby wrote: > > Well... > > Glad you asked. :-) > > > > You can use all of the above, or none of above. (Ok, really below, but > > you know what I meant.) > > > > You can write a map/reduce, or series of map/reduce(s) and output to a > > flat file, or even to a relational database. > > > > Then there's pig, but I haven't had time to play with it yet. And > > there's now a JDBC conectivity, which I haven't have time to play with > > yet, either. > > > > The point is that you can build data warehouses, data marts, etc to > > handle large amounts of data fairly quickly using non traditional means. > > > > Now I do see issues with Hadoop, but I think that over time, it will get > > better. > > > > Don't take my word for it, Accenture has already started their own cloud > > computing practice. Sun's released their vision and well, then there's > > IBM.... > > > > ;-) > > > > -G > > > > PS. Here's some food for thought... Back in the 90's Informix's XPS had > > customers who created 27TB data warehouses. If we rely on Moore's law, > > that database would probably be comparable to one of ~2.7 petabytes. ;-) > > Can DB2 do that? > > > > > > > Date: Tue, 19 Jan 2010 00:29:25 +0000 > > > From: obnoxio@serendipita.com > > > CC: informix-list@iiug.org > > > Subject: Re: Webcast - Rapid Data Warehouse Development - Build a > > Pilot in 8 weeks! > > > > > > Ian Michael Gumby wrote: > > > > But I can build a cluster of Hadoop/HBase machines and set up a DW in > > > > less than 8 days! (Ok, assuming that the machines are already built > > and > > > > are already networked...) > > > > > > And I can do that in half a day. So fucking what? > > > > > > And once you have those servers up and running, what are you going to > > > use to query them? Cognos? Brio? Business Objects? Crystal Reports? > > > > > > -- > > > Cheers, > > > Obnoxio The Clown > > > > > > http://obotheclown.blogspot.com > > > I will now proceed to pleasure myself with this fish. > > > > > > -- > > > This message has been scanned for viruses and > > > dangerous content by OpenProtect(http://www.openprotect.com), and is > > > believed to be clean. > > > > > > _______________________________________________ > > > Informix-list mailing list > > > Informix-list@iiug.org > > > http://www.iiug.org/mailman/listinfo/informix-list > > > > ------------------------------------------------------------------------ > > Hotmail: Trusted email with powerful SPAM protection. Sign up now. > > <http://clk.atdmt.com/GBL/go/196390707/direct/01/>; > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Informix-list mailing list > > Informix-list@iiug.org > > http://www.iiug.org/mailman/listinfo/informix-list > > -- > ______________________________________________________________________ > Lester Knutsen lester@advancedatatools.com > Advanced DataTools Corporation Voice: 703-256-0267 x102 > Visit our Web page: http://www.advancedatatools.com > ______________________________________________________________________ _________________________________________________________________ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. http://clk.atdmt.com/GBL/go/196390709/direct/01/