String Parsing?
A developer new to Informix-4GL asked how to parse messy CSV/text files for import, having found only the substring [] operator and external C functions in the docs. Respondents confirmed 4GL's string handling is minimal and suggested alternatives: ready-made string-splitting routines from Marco Greco's site or the IIUG repository (including Richard Harnden's unl_utils csv/unl converters), the dbload utility for delimited or fixed-position files, doing the parsing outside 4GL in Perl/PHP or an ETL tool such as BIE, or the regexp DataBlade. No single fix was settled on; the poster accepted he'd likely write C.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
I've recently been given an assignment to create a set of routines that will be used to import csv and other text files (some nicely formated, others a hideous mess). I'm experienced with UniVerse (Pick) and C#. This project is running within an Informix 4gl environment though so I'm a stranger in a strange land. I've looked all through several online documentation sources for Informix 4gl and I can't find anything other than references to external c functions and the substring [] operator. Are these the only choices for string manipulation within Informix? Is the SOP within the Informix community to write a C function to perform chores like this? Thanks for your help.
donkibbey@gmail.com wrote: > I've recently been given an assignment to create a set of routines that > will be used to import csv and other text files (some nicely formated, > others a hideous mess). I'm experienced with UniVerse (Pick) and C#. > This project is running within an Informix 4gl environment though so > I'm a stranger in a strange land. > > I've looked all through several online documentation sources for > Informix 4gl and I can't find anything other than references to > external c functions and the substring [] operator. Are these the only > choices for string manipulation within Informix? Is the SOP within the > Informix community to write a C function to perform chores like this? > > Thanks for your help. I did write a few routines some time ago to split strings into fields, etc, and I'm sure others have too.Have a look a my web site (see sig), or wander to the IIUG software repository (www.iiug.org) and navigate your way from there... -- Ciao, Marco ______________________________________________________________________________ Marco Greco /UK /IBM Standard disclaimers apply! Structured Query Scripting Language http://www.4glworks.com/sqsl.htm 4glworks http://www.4glworks.com Informix on Linux http://www.4glworks.com/ifmxlinux.htm
On Thu, 2006-05-18 at 07:05, donkibbey@gmail.com wrote: > I've recently been given an assignment to create a set of routines that > will be used to import csv and other text files (some nicely formated, > others a hideous mess). I'm experienced with UniVerse (Pick) and C#. > This project is running within an Informix 4gl environment though so > I'm a stranger in a strange land. > > I've looked all through several online documentation sources for > Informix 4gl and I can't find anything other than references to > external c functions and the substring [] operator. Are these the only > choices for string manipulation within Informix? As far as string parsing is concerned, that's probably true. > Is the SOP within the > Informix community to write a C function to perform chores like this? The fact that the data is stored in an Informix engine and the product you're integrating with is written in Informix-4GL does not per se imply that you have to do everything in Informix-4GL. I don't presume to speak for the Informix community, but my gut reaction would be to write an external parser in Perl/Python/whatnot to read the files and digest them into something that the database can handle more easily, such as | (pipe) delimited unload files. Giving more specific advice is difficult since you're not saying what platform (Operating system, Informix engine version, Informix 4GL version) you're on. Best regards, -- Carsten Haese | Phone: (419) 861-3331 Software Engineer | Direct Line: (419) 794-2531 Unique Systems, Inc. | FAX: (419) 893-2840 1687 Woodlands Drive | Cell: (419) 343-7045 Maumee, OH 43537 | Email: carsten@uniqsys.com
Could you use the dbload? You can specify delimited (comma, pipe,
semicolon or whatever) input file format or character-position format
(e.g. positions 1-2 for state, 4-24 for county and so on) so some logic
to determine which type of file is expected should be all the other
legwork you need to do......
Of course that's the 'dbload' utility and if you're wondering where to
find the syntax for it it's in the migration guide, chapter 9 at last
perusal.
It's a bastard setup of vb6 front end, calling into informix 4gl backend compiled into exe's running off a windows based middle tier working from a Micro$oft sql server db. No, I didn't make all that up. They call their company and product "Elite", but I have my doubts.
Thank you to all who posted. My fears are confirmed. Now, where did I put that C manual...
donkibbey@gmail.com wrote: > I've recently been given an assignment to create a set of routines that > will be used to import csv and other text files (some nicely formated, > others a hideous mess). I'm experienced with UniVerse (Pick) and C#. > This project is running within an Informix 4gl environment though so > I'm a stranger in a strange land. > > I've looked all through several online documentation sources for > Informix 4gl and I can't find anything other than references to > external c functions and the substring [] operator. Are these the only > choices for string manipulation within Informix? Is the SOP within the > Informix community to write a C function to perform chores like this? > > Thanks for your help. > I'm in the same camp of people who do the string parsing outside of 4GL, but I imagine Marco's stuff will be exceptional. Don't give up. Perl has a CSV parser, as well as PHP. Lots of solutions out there. C should be your absolute last choice as you will have to write a lot of routines to do what Perl or PHP can do for you. Or break down and buy an ETL product, which is probably what you should really do, but writing your own? No way dude. Head over to www.cpan.org for Perl, www.phpclasses.org for PHP or as was already suggested, www.iiug.org or Marco's site and see if something there doesn't help. PHP and Perl are true geek sites, they will require a bit of stumbling around to find what you need. Perl is a bit easier to install modules right on the command line but I'll spare you the details unless you are interested.
I came across a set of file utilities on www.iiug.org today written by Richard Harnden. They are, "Utilities for converting csv-files to Informix unl-files and vice-versa.". They're called unl_utils and are written C. HTH
> > I've looked all through several online documentation sources for > > Informix 4gl and I can't find anything other than references to > > external c functions and the substring [] operator. Are these the only > > choices for string manipulation within Informix? Is the SOP within the > > Informix community to write a C function to perform chores like this? > I'm in the same camp of people who do the string parsing outside of 4GL, > but I imagine Marco's stuff will be exceptional. Don't give up. Perl has > a CSV parser, as well as PHP. Lots of solutions out there. C should be > your absolute last choice as you will have to write a lot of routines to > do what Perl or PHP can do for you. Or break down and buy an ETL product, > which is probably what you should really do, but writing your own? No way > dude. H We use BIE (http://sourceforge.net/projects/bie) to transport/transform data between databases, text files, and an LDAP server. It will automatically generate schemas and provides a GUI map builder.
If you are on v9/10 then install the regexp datablade Cheers Paul Paul Watson Tel: +44 1414161772 Mob: +44 7818003457 GO FURTHER with DB2 GET THERE FASTER with Informix. Attend the IDUG 2006 European Conference. Vienna, Austria. 2-6 October 2006 Visit http://www.iiug.org/conf for more information. > -----Original Message----- > From: donkibbey@gmail.com [mailto:donkibbey@gmail.com] > Posted At: 18 May 2006 06:05 > Posted To: comp.databases.informix > Conversation: String Parsing? > Subject: String Parsing? > > > I've recently been given an assignment to create a set of > routines that > will be used to import csv and other text files (some nicely formated, > others a hideous mess). I'm experienced with UniVerse (Pick) and C#. > This project is running within an Informix 4gl environment though so > I'm a stranger in a strange land. > > I've looked all through several online documentation sources for > Informix 4gl and I can't find anything other than references to > external c functions and the substring [] operator. Are > these the only > choices for string manipulation within Informix? Is the SOP > within the > Informix community to write a C function to perform chores like this? > > Thanks for your help. >