Re: Data Quality / Profile / Cleansing Tools

Data Goob <datagoob@hotmail.com> wrote in message news:<9KaXc.140$bL4.79@fe20.usenetserver.com>... > [snip..] > Informatica is > way too complicated and just can't get it done compared to Ab Initio, > and only runs on Windows. ( I know you didn't mention Informatica or > Ab Initio ) As a test of any of these products ask yourself how much > training is required to make them work. AI works on Linux, AIX, etc. What is complicated about Informatica compared to Ab Initio ? Do you have examples where Ab Initio can do something Informatica can't ? Nick

Nick wrote: > Data Goob <datagoob@hotmail.com> wrote in message news:<9KaXc.140$bL4.79@fe20.usenetserver.com>... > > >>[snip..] >> Informatica is >>way too complicated and just can't get it done compared to Ab Initio, >>and only runs on Windows. ( I know you didn't mention Informatica or >>Ab Initio ) As a test of any of these products ask yourself how much >>training is required to make them work. AI works on Linux, AIX, etc. > > > What is complicated about Informatica compared to Ab Initio ? I don't have the time to chat about it. > Do you have examples where Ab Initio can do something Informatica can't ? > > Nick Clustering and/or partitioning across multiple servers?

Data Goob <datagoob@hotmail.com> wrote in message news:<oPlZc.2133$_65.1473@fe46.usenetserver.com>... > Nick wrote: > > Data Goob <datagoob@hotmail.com> wrote in message news:<9KaXc.140$bL4.79@fe20.usenetserver.com>... > > > > > >>[snip..] > >> Informatica is > >>way too complicated and just can't get it done compared to Ab Initio, > >>and only runs on Windows. ( I know you didn't mention Informatica or > >>Ab Initio ) As a test of any of these products ask yourself how much > >>training is required to make them work. AI works on Linux, AIX, etc. > > > > > > What is complicated about Informatica compared to Ab Initio ? > > I don't have the time to chat about it. Right. Personally I'd consider Informatica easier to use than Ab Initio (certainly training requirements I've seen seem to indicate this). I have a limited knowledge of Ab Initio so I'd hope that you had something more substantial to say. Things I'm aware about: It's easier to handle errors in AI (not that it's overly tricky in Informatica) Informatica isn't relationally complete (not that SQL is either?). I'm thinking of transitive closure here. Ab Initio has a proper type system ? (People have mentioned vectors. Is this a type - if so, does it violate first normal form?) Ab Initio has iteration capabilities ? Surely this breaks partitioning potential (i.e. it is not set based) Does AI have change management facilities? I've heard memory management is poor in Ab Initio and it crashes a lot without careful set up ? > > Do you have examples where Ab Initio can do something Informatica can't ? > > > > Nick > > Clustering and/or partitioning across multiple servers? Informatica runs on Unix and you can set up server grids. Nick

Nick wrote: > Data Goob <datagoob@hotmail.com> wrote in message news:<oPlZc.2133$_65.1473@fe46.usenetserver.com>... > >>Nick wrote: >> >>>Data Goob <datagoob@hotmail.com> wrote in message news:<9KaXc.140$bL4.79@fe20.usenetserver.com>... >>> >>> >>> >>>>[snip..] >>>>Informatica is >>>>way too complicated and just can't get it done compared to Ab Initio, >>>>and only runs on Windows. ( I know you didn't mention Informatica or >>>>Ab Initio ) As a test of any of these products ask yourself how much >>>>training is required to make them work. AI works on Linux, AIX, etc. >>> >>> >>>What is complicated about Informatica compared to Ab Initio ? >> >>I don't have the time to chat about it. > > > Right. Personally I'd consider Informatica easier to use than Ab > Initio > (certainly training requirements I've seen seem to indicate this). I > have a > limited knowledge of Ab Initio so I'd hope that you had something more > substantial to say. > Well, ok, FWIW AI does require a bit of training as does anything you buy. AI comes from a UNIX perspective, not Windows, so in some regards Windows peoples will have some adjusting. But not a lot. Where the real training needs to be applied is in the terminology of ETL, the religion of ETL, and how best to do things. AI is powerful in that it requires you to really rethink your processes. What can we do and what's the best way? If we can white-board it first the way it should be done, can the ETL tool do what we draw on the board? What if no matter what we draw on the board were possible? I doubt seriously that Informatica can live up to that criteria. White-board or Visio your process the way you want to do it then ask yourself if there could be a way to do it with an ETL tool. This is what AI is, Visio that works. > Things I'm aware about: > > It's easier to handle errors in AI (not that it's overly tricky in > Informatica) > I don't know where to say this to you but I thought I'd throw it in: As the administrator of the system I do not build graphs every day like the rest of the team, but I can tell you the flexibility of AI is amazing. I think it has made better ETL people and smarter in solving problems, and actually deliver some innovation. I think one of the strengths of AI is its ability to build processes that can have checkpoints, multiple parts executing at the same time, good monitoring tool in the EME ( Enterprise Meta Environment ), clustering, partitioning, and most importantly, high-end thinking behind the software from a UNIX perspective, not just a bunch of Windows Weenies pushing a Windows-centric product. AI is for serious ETL, so it's difficult to compare to others. > Informatica isn't relationally complete (not that SQL is either?). I'm > thinking > of transitive closure here. > I don't know what transitive closure is. Is that like dating two women at the same time, and breaking up with one of them? > Ab Initio has a proper type system ? (People have mentioned vectors. > Is this > a type - if so, does it violate first normal form?) > I dunno. In my understanding of what you're asking, I am not totally sure of what you're asking. But if I can respond to what I **think** you're asking, AI permits typing of every field/column, to also indicate that fields/columns must be typed from input thru output. > Ab Initio has iteration capabilities ? Surely this breaks partitioning > potential > (i.e. it is not set based) > I dunno. But I do know that AI will leverage parallelism whereever it can be used, and allow some pretty crazy feeding and re-feeding of the data. > Does AI have change management facilities? > Yes, and is quite comprehensive. We have not implemented a lot of the change management features simply due to resource constraints. It is available but also keep in mind it requires your team to make it happen as a culture as well as the tools being available. A lot of the management of AI is on the command-line in UNIX/Linux, so don't expect a lot of GUI tools to do things. It is for serious UNIX shops and these shops will benefit the most from AI. The GUI to build graphs is quite comprehensive, straight-forward, but the server-side requires someone with a UNIX background as much of the management is command-line. > I've heard memory management is poor in Ab Initio and it crashes a lot > without careful set up ? > Depends. On Windows we crashed the system, but it's a Windows problem that is exposed, and not necessarily AI. We found Windows performance so miserable that when we went to Linux we instantly saw a 3x to 10x improvement in speed. I would never recommend AI's server piece on Windows, this is something they should not even offer as it breaks systems--and AI is really a UNIX/Linux product that is offered for Windows more as an entry point into the Windows market. The client piece runs on Windows no matter what, but I would definitely recommend you run the server piece on UNIX or Linux. 32-bit Linux will have the memory limitations that 64-bit won't, same for UNIX. CPU utilization on both Windows and Linux is difficult to manage more than memory management. AI will consume the box, and we use 4-CPU DellPowerEdge 6650's. We optimized everything on these boxes and are now looking at AIX 64-bit as the next step. If there is a product that maxxes-out your box AI will push those CPUs to the max. Never seen anything like it. Fire up xosview and watch it light up like fireworks when graphs kick off. > >>>Do you have examples where Ab Initio can do something Informatica can't ? >>> >>>Nick >> >>Clustering and/or partitioning across multiple servers? > > > Informatica runs on Unix and you can set up server grids. > > Nick I don't know what server grids are, but imagine they are similar. AI has adopted URI constructions in their configurations that point to a more open-systems type of thinking. ( connect:// server:// etc ) I don't know much about Informatica other than it looks like it caters more to the Windows world. AI also has partitioning components that map directly to DB2 and Oracle partitioning, which on the surface can be confusing to the newcomer. AI has its own partitioning, fan-in, fan-out theology but it maps into DB2 partitioning. So you can have a complete AI partitioning scheme that has nothing to do with DB2 partitioning, but push data into a DB2 partitioned database. It begs the question of which came first the DB2 partitioning or AI partitioning. There does not seem to be much product attention to Informix fragmentation, but you should ask Ab Initio what they offer. Both DB2 amd Ab Initio seem to understand clustering of data in a manner consistent with the optimal use of MPP systems. I do not get that UNIX-feeling from Informatica, it just smacks of a Windows world trying to do things that require UNIX, same for Ascential. We found instead why not use a UNIX-based system that understands that big ETL will require something other than Windows. Ab Initio pushed into Windows rather than pushing into UNIX, profoundly different perspective on understanding parallelism, etc.@@NL