Re: Data Warehousing Tools - To Bundle Or Not to Bundle?

>Start with the database. Oracle is required for large companies. Smaller ones can >run SQL just fine. I think you just said it all right there. I hope this does not also speak to your credentials for discussing this topic. >This relates to >the previous threads about transaction performance. High volume >requires a robust database. Low volume will run on SQL just fine. So >if you are Low volume, you can get the microsoft bundle. SQL, MS >Access, fits in well with the windows appliction structure most >companies have. If you need OLAP and data mining you are still open >to outside products. You might want to check out www.tpc.org and see just how low volume SQL-server is running at. I admit to being an Informix bigot - I was surprised at how large 'low-volume' has become. That still begs the 'high-availability' question which is a given with Unix based RDBMs - errrr - depending on the sysadmin. > >>2. If they are what category of tools should make up a bundle? What, if >>anything, should not be in a bundle? >Database, Data Extraction tool, report/query tool (client), OLAP tool, >data mining tool. >Plus ODBC/SQL drivers if required, client OS, and network >infrastructure (Duh!). I don't like the concept of bundling here. Your Mileage WILL vary and along with it your tool needs. On my first DW I wrote 75% of the stuff that was missing - you may or may not depending on the size of the data, programming capabilities, schedule blah-blah. But I'll take a stab at what every DW should have: Communication/file transfer SW between source and DW machines. ftp may not be enough. Transformation tools Name standardization, Merge/Purge, Householding (if you're doing marketing) High performance loader Robust Scheduler Notification/monitoring tools (could be part of scheduler - but you really want to keep an eye on the DB AND all the processes) DSS capable database - I'll leave it at that. Summarization tools - esql/c - someway to perform data 'augmentation' etc. Metadata Tools - including modelling SW Archive tools - problematic for 1TB+ DWs - although of course Informix has an excellent solution. High performance extraction/unloader tool, communication to data mart and loader to DM (repeat) OLAP/ROLAP tools... of the things mentioned above: Database - given Extraction - sure if you want to call it that, Informix has it bundled already of course report/query tool (client) - ok OLAP tool - ok data mining - sure why not, YMMV ODBC/SQL drivers/OS/network - pretty much a given and not really part of a toolset you'd want to sell Oh, I forgot revision control. I'm sure there are a few things I've left out. What I see from what was originally posted is a tendency to view the DW from the client side - and in fact really a question of accessing a DM. It's not likely I'm going to let anybody run client SW against a real DW. >> >>3. If money was not an issue and you were able to put together a >>cross-vendor bundle, what currently shipping products would you put in it? >No limit? I might just go with Straight Oracle. Not if I wanted to go into the land of 1TB+. >> >>4. Since money actually is an issue in the real world, what would be the >>optimum price that a customer should pay for a bundle? This figure has >>nothing to do with consultancy or hardware. >Real world we all have budgets, and you can get along fine with >existing products. As I said earlier, we have an existing Oracle DB, >and a site license for MS Access. This has served us well and didn't >cost us an extra dime. We did look at other products but they all >cost too much for what improvements we were getting. And once you get >in bed with those guys they will milk you for more money. >Real world budgets depend on the size of your company. If you had to >start from ground zero expect to pay $1M! Database, user licenses, >development costs, etc. If you have existing infrastructure, your >costs will be less. My advice is to leverage off of existing >infrastructure. Fine thing to do if you have all the time in the world and not a lot of data to deal with, been there, done that, took 18 months, not much money (aside from time) the end result was under 10GB. If you're going to the land of 1TB+, the hardware/SW alone is going to be at least into the 6 figures, and more likely over $1M. The analysis that goes into the data before you even start extracting it is going to be in the 6/7 figures range - it goes on from there. A single tool at this level can range in price from $30K to $300K. and some of them are well worth it. Even so, a great deal of the value from some of these tools is the consultant who comes along with it. Of course I gather that is not normally the case with Oracle consultants - mind you I'm not trying to be nasty - they just have a different view of reality and their role in the real world. >> >>5. What effect will the pending Microsoft BackOffice that includes NT5.0/SQL >>Server 7.0 do to the efforts of the vendors currently offering similar one >>stop data warehousing bundles. MS? Go warehouse your .ini files. Miscrosoft has a long way to go to convince the real world that they have a) an OS than can handle size (of machine), volume (of data), reliability (staying up except for scheduled downtime) b) a database that can do the same thing c) the necessary skills to offer consultants at anything other configuration. Microsoft is like TV, good to watch their shows, but don't try to get your own programming on it. >It will drive the price down. Take a serious look at MS. They are >geared toward the masses. The small and medium size companies that >don't have a fortune and have already invested in MS products. The masses are not geared towards DW. If you're trying to sell DW to mom and pop, I have to really question the gullibility of mom and pop. The kind of places that need true DW capabilities tend to be dealing with a large number of legacy/source systems and a commesurate quantity of data. Now if you want to populate your spreadsheet with sensible data - then that's another story. Would we have used it on my first DW? (the small one) Perhaps if we would have picked up a complete bundle at under 10K. > >But if you have an existing database then you will want to go down >that path. Could be > >OBTW: >We have multiple source databases (IMS, Oracle, DB2) >We have Oracle 8.0 DB for DW tables >Query tool=MS Access (custom interface). >OLAP=Cognos (just getting this going). >We use ETIs extract tool to move the data but we have reservations >recommending this tool. It's not for wimps. > >My .02 I work in an Oracle shop. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Steve Wulz stevewulz-@-usa.net By US Code Title 47, Sec.227(a)(2)(B), a computer/modem/printer meets the definition of a telephone fax machine. By Sec.227(b) (1)(C), it is unlawful to send any unsolicited advertisement to such equipment, punishable by action to recover actual monetary loss,