Red Brick and Datastage
Posted in 2000
Neil Truby asked for real-world experience of Red Brick (RBW) and DataStage, which his client planned to buy to replace scripted MIS on IDS 7.31, questioning Informix sales claims of 7x faster loads and of a "black box" needing no training or support effort. Respondents confirmed the load speed (1M rows loaded and indexed in ~1m20s vs ~4m in IDS 9.20) but rejected the black-box claim: RBW is arcane, rule-based optimiser, slow updates/deletes, awkward schema changes. Advice: design DataStage jobs to run in parallel and feed flat files to RB_TMU, get RBW and DataStage consultants in early, get the schema right, and read Kimball's Data Warehouse Toolkit. No formal resolution beyond this advice.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Versions, Editions & End-of-Life
My client has seems set upon buying Red Brick and Datastage to replace the existing scripted MIS offering running on IDS 7.31. The decision to do this seems to be based upon information from Informix sales, and high-level references quoting "load times are up to 7 times faster with Red Brick". I've no problem with this at all. I'm happy to broaden my skill base. But does anyone have any real-world experience of Red Brick and Datastage, perhaps with some hard facts and figures to bear out their efficacy? thanks Neil
I've already had a good, private, reply. To clarify: I suppose in the short term the thing I'm trying to establish is: how true is it when Informix say it's all automatic and black box, and no-one will need any training to maintain it once it's set up. The client will go aheasd with Red Brick in part on the grounds that it will reduce the support effort required. I find it difficult to believe that, in the short-term, the support effort can be reduced by substituting a dbms about which we know nothing for one we know inside out, although I'm sure there may well be long-term gains. thanks Neil Neil Truby <ntruby@netcomuk.co.uk> wrote in message news:926151$etk$1@lyonesse.netcom.net.uk... > My client has seems set upon buying Red Brick and Datastage to replace the > existing scripted MIS offering running on IDS 7.31. The decision to do this > seems to be based upon information from Informix sales, and high-level > references quoting "load times are up to 7 times faster with Red Brick". > > I've no problem with this at all. I'm happy to broaden my skill base. But > does anyone have any real-world experience of Red Brick and Datastage, > perhaps with some hard facts and figures to bear out their efficacy? > > thanks > Neil > >
In the year of Our Lord Sun, 24 Dec 2000 23:32:21 -0000, "Neil Truby" <ntruby@netcomuk.co.uk> broke a vow of silence to utter: >My client has seems set upon buying Red Brick and Datastage to replace the >existing scripted MIS offering running on IDS 7.31. The decision to do this >seems to be based upon information from Informix sales, and high-level >references quoting "load times are up to 7 times faster with Red Brick". > >I've no problem with this at all. I'm happy to broaden my skill base. But >does anyone have any real-world experience of Red Brick and Datastage, >perhaps with some hard facts and figures to bear out their efficacy? Sure. RedBrick probably loads at least 7 times faster than IDS, my eyebrows were at the back of my head the first time I benchmarked a load in RBW. The performance problem is with DataStage -- it can't really match that. So, here's what you do: 1. Design *ALL* your DS jobs so that they can be run in parallel. This is possibly the most important consideration. Also, try to get your jobs to generate flat files that can be easily handed off to RB_TMU. 2. GET AN RBW CONSULTANT IN. I'm not kidding. Although I'm fairly competent as an Informix DBA, I was completely lost with RBW. Things are quite topsy-turvy. Plan to have a consultant in during the planning, design and a couple of times during the implementation and acceptance. 3. Get your database design right. RBW is rather disappointing at things like adding columns and things like that. Prepopulate your database with a couple of spare columns of each datatype, just in case. 4. If you get the design right for the queries, queries are astonishingly quick. Stuff like updates and deletes are agonisingly slow. The optimiser is rule based and can be rather dim sometimes. I believe 6.1 is better, but we haven't tried it yet. 5. Load times _are_ impressive. I did a 1M row load into IDS9.20 -- just over 4 minutes. RBW loaded *and* *indexed* 1M rows in 1 minute 20 seconds. 6. DS is basically a BASIC interpreted environment, so although it's reasonably quick, it's no match for C. Also, like RBW, there's a right way and a wrong way to use DS. Plan to get a competent DS consultant in. The earlier the better. (My recommendations to get consultants in are not just for show. On both products we had *major* problems solved by getting them in, but some of the fixes required major rewrites. If we'd known that up front, it would have been just as easy to do it right. Do not count on the info in the basic training material to get you through the project without pain.)
In the year of Our Lord Mon, 25 Dec 2000 09:38:45 -0000, "Neil Truby" <ntruby@netcomuk.co.uk> broke a vow of silence to utter: >I've already had a good, private, reply. > > To clarify: I suppose in the short term the thing I'm trying to establish >is: how true is it when Informix say it's all automatic and black box, and >no-one will need any training to maintain it once it's set up. The client >will go aheasd with Red Brick in part on the grounds that it will reduce the >support effort required. I find it difficult to believe that, in the >short-term, the support effort can be reduced by substituting a dbms about >which we know nothing for one we know inside out, although I'm sure there >may well be long-term gains. RBW is black box? Bollocks. It's frighteningly arcane. Once the database is stable and running well, it may be more of a black box, but after six months I can't see where that point will be. DS, OTOH, is pretty black box and astoundingly stable for a development tool. (It's not perfect, definitely, but it's not at all bad.)
"Neil Truby" <ntruby@netcomuk.co.uk> a écrit : >I'm happy to broaden my skill base. (Apologies if this is egg-sucking) Get hold of Ralph Kimball's book "The Data Warehouse Toolkit" Cheap, thin, and one of the best IT books I've ever seen. All OLAP is weird and topsy-turvy, and this book is the best explanation around. It even changed how I designed "traditional" RDBMS -- Smert' Spamionam
In the year of Our Lord Wed, 27 Dec 2000 22:32:34 +0000, Andy Dingley <dingbat@codesmiths.com> broke a vow of silence to utter: >"Neil Truby" <ntruby@netcomuk.co.uk> a 'crit : > >>I'm happy to broaden my skill base. > >(Apologies if this is egg-sucking) > >Get hold of Ralph Kimball's book "The Data Warehouse Toolkit" > >Cheap, thin, and one of the best IT books I've ever seen. All OLAP is >weird and topsy-turvy, and this book is the best explanation around. >It even changed how I designed "traditional" RDBMS God help you then. I'm more of an Inmon kind of guy. :-) But certainly, if you're going to use RedDick, then the Kimball books are a must.
Yes, I already have it. He used to be the Chief Executive or something of Red Brick, didn't he? Any views on my original question, Andy - ie is it all as simple to administer as the marketing puff says? thanks Neil Andy Dingley <dingbat@codesmiths.com> wrote in message news:arok4tgn46rk31d6n0l402ge788l80j0qa@4ax.com... > "Neil Truby" <ntruby@netcomuk.co.uk> a 'crit : > > >I'm happy to broaden my skill base. > > (Apologies if this is egg-sucking) > > Get hold of Ralph Kimball's book "The Data Warehouse Toolkit" > > Cheap, thin, and one of the best IT books I've ever seen. All OLAP is > weird and topsy-turvy, and this book is the best explanation around. > It even changed how I designed "traditional" RDBMS > > -- > Smert' Spamionam
Related threads
- onint - cannot open chunk error 2
- Last page of first extent in a partition
- Review on Report Generators
- HDR in IDS 10.0xc5: slow Secondary restart