RE: Monitoring the change log

solaris 10 IDS 11.70.FC2 Is there a good way to read the log files or some other way to monitor all the data changes ocurring on a database between a certain date/time range? We only need some of the data fields from up to 65 tables, but we would like to copy the delta to another database on a nightly basis for those particular fields. If the data were readable in some way as data string that could be parsed, that might be an option in parsing the logs; but if not, does anyone else have a suggestion? Any suggestions would be appreciated. The target database is a Postgres database (FYI) Thanks in advance. Larry

The best thing to do is to use the Change Data Capture API to write an app that can subscribe to the logical log record stream. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> wrote: > solaris 10 > IDS 11.70.FC2 > > Is there a good way to read the log files or some other way to monitor all > the > data changes ocurring on a database between a certain date/time range? We > only > need some of the data fields from up to 65 tables, but we would like to > copy > the delta to another database on a nightly basis for those particular > fields. > If the data were readable in some way as data string that could be parsed, > that might be an option in parsing the logs; but if not, does anyone else > have > a suggestion? > > Any suggestions would be appreciated. The target database is a Postgres > database (FYI) > > Thanks in advance. > > Larry > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --20cf307cfbca5944ce04a32dcbf2

Hi Larry Please see www.lintel.co.uk/infotrace. It would appear that this does exactly what you want with no programming required and no additional Informix licence costs. The data changes get written to a separate Informix database and so could easily then be filtered to include just the columns and date ranges that you want and then transferred each night to your Postgres database. If you are at the IIUG conference in Kansas this week then we could meet up and I can demo. it to you. Otherwise please send me an email if you have any questions. Regards David Linthwaite Lintel Software Consultancy Ltd IBM Business Partner (www.lintel.co.uk) Tel.: 01244 357250 Fax.: 01244 357248 mailto:dlinthwaite@lintel.co.uk -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of LARRY SORENSEN Sent: 13 May 2011 20:55 To: ids@iiug.org Subject: RE: Monitoring the change log [23699] solaris 10 IDS 11.70.FC2 Is there a good way to read the log files or some other way to monitor all the data changes ocurring on a database between a certain date/time range? We only need some of the data fields from up to 65 tables, but we would like to copy the delta to another database on a nightly basis for those particular fields. If the data were readable in some way as data string that could be parsed, that might be an option in parsing the logs; but if not, does anyone else have a suggestion? Any suggestions would be appreciated. The target database is a Postgres database (FYI) Thanks in advance. Larry **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum.

Art Do you have anymore information on this or links that explain how this can be done? Larry > To: ids@iiug.org > From: art.kagel@gmail.com > Subject: Re: Monitoring the change log [23700] > Date: Fri, 13 May 2011 16:03:10 -0400 > > The best thing to do is to use the Change Data Capture API to write an app > that can subscribe to the logical log record stream. > > Art > > Art S. Kagel > Advanced DataTools (www.advancedatatools.com) > Blog: http://informix-myview.blogspot.com/ > > Disclaimer: Please keep in mind that my own opinions are my own opinions and > do not reflect on my employer, Advanced DataTools, the IIUG, nor any other > organization with which I am associated either explicitly, implicitly, or by > inference. Neither do those opinions reflect those of other individuals > affiliated with any entity with which I am affiliated nor those of the > entities themselves. > > On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> wrote: > > > solaris 10 > > IDS 11.70.FC2 > > > > Is there a good way to read the log files or some other way to monitor all > > the > > data changes ocurring on a database between a certain date/time range? We > > only > > need some of the data fields from up to 65 tables, but we would like to > > copy > > the delta to another database on a nightly basis for those particular > > fields. > > If the data were readable in some way as data string that could be parsed, > > that might be an option in parsing the logs; but if not, does anyone else > > have > > a suggestion? > > > > Any suggestions would be appreciated. The target database is a Postgres > > database (FYI) > > > > Thanks in advance. > > > > Larry > > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > --20cf307cfbca5944ce04a32dcbf2 > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >

The change data capture manual is a good source. Art On May 16, 2011 10:12 AM, "LARRY SORENSEN" <lsorensen25@msn.com> wrote: > Art > > Do you have anymore information on this or links that explain how this can be > done? > > Larry > >> To: ids@iiug.org >> From: art.kagel@gmail.com >> Subject: Re: Monitoring the change log [23700] >> Date: Fri, 13 May 2011 16:03:10 -0400 >> >> The best thing to do is to use the Change Data Capture API to write an app >> that can subscribe to the logical log record stream. >> >> Art >> >> Art S. Kagel >> Advanced DataTools (www.advancedatatools.com) >> Blog: http://informix-myview.blogspot.com/ >> >> Disclaimer: Please keep in mind that my own opinions are my own opinions and >> do not reflect on my employer, Advanced DataTools, the IIUG, nor any other >> organization with which I am associated either explicitly, implicitly, or by >> inference. Neither do those opinions reflect those of other individuals >> affiliated with any entity with which I am affiliated nor those of the >> entities themselves. >> >> On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> wrote: >> >> > solaris 10 >> > IDS 11.70.FC2 >> > >> > Is there a good way to read the log files or some other way to monitor all >> > the >> > data changes ocurring on a database between a certain date/time range? We >> > only >> > need some of the data fields from up to 65 tables, but we would like to >> > copy >> > the delta to another database on a nightly basis for those particular >> > fields. >> > If the data were readable in some way as data string that could be parsed, >> > that might be an option in parsing the logs; but if not, does anyone else >> > have >> > a suggestion? >> > >> > Any suggestions would be appreciated. The target database is a Postgres >> > database (FYI) >> > >> > Thanks in advance. >> > >> > Larry >> > >> > >> > >> > >> > ******************************************************************************* >> > Forum Note: Use "Reply" to post a response in the discussion forum. >> > >> > >> >> --20cf307cfbca5944ce04a32dcbf2 >> >> >> > ******************************************************************************* >> Forum Note: Use "Reply" to post a response in the discussion forum. >> > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > --20cf3071cd14e8be3304a3669963

Art, Do I need to install this API, or is natively available? Larry > To: ids@iiug.org > From: art.kagel@gmail.com > Subject: Re: Monitoring the change log [23700] > Date: Fri, 13 May 2011 16:03:10 -0400 > > The best thing to do is to use the Change Data Capture API to write an app > that can subscribe to the logical log record stream. > > Art > > Art S. Kagel > Advanced DataTools (www.advancedatatools.com) > Blog: http://informix-myview.blogspot.com/ > > Disclaimer: Please keep in mind that my own opinions are my own opinions and > do not reflect on my employer, Advanced DataTools, the IIUG, nor any other > organization with which I am associated either explicitly, implicitly, or by > inference. Neither do those opinions reflect those of other individuals > affiliated with any entity with which I am affiliated nor those of the > entities themselves. > > On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> wrote: > > > solaris 10 > > IDS 11.70.FC2 > > > > Is there a good way to read the log files or some other way to monitor all > > the > > data changes ocurring on a database between a certain date/time range? We > > only > > need some of the data fields from up to 65 tables, but we would like to > > copy > > the delta to another database on a nightly basis for those particular > > fields. > > If the data were readable in some way as data string that could be parsed, > > that might be an option in parsing the logs; but if not, does anyone else > > have > > a suggestion? > > > > Any suggestions would be appreciated. The target database is a Postgres > > database (FYI) > > > > Thanks in advance. > > > > Larry > > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > --20cf307cfbca5944ce04a32dcbf2 > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >

It is native. There was ay least one session on using CDC here at the IIUG conference. The presentations will be on the IIUG web site later today or tomorrow. Look for it. Art On May 18, 2011 10:21 AM, "LARRY SORENSEN" <lsorensen25@msn.com> wrote: > Art, > > Do I need to install this API, or is natively available? > > Larry > >> To: ids@iiug.org >> From: art.kagel@gmail.com >> Subject: Re: Monitoring the change log [23700] >> Date: Fri, 13 May 2011 16:03:10 -0400 >> >> The best thing to do is to use the Change Data Capture API to write an app >> that can subscribe to the logical log record stream. >> >> Art >> >> Art S. Kagel >> Advanced DataTools (www.advancedatatools.com) >> Blog: http://informix-myview.blogspot.com/ >> >> Disclaimer: Please keep in mind that my own opinions are my own opinions and >> do not reflect on my employer, Advanced DataTools, the IIUG, nor any other >> organization with which I am associated either explicitly, implicitly, or by >> inference. Neither do those opinions reflect those of other individuals >> affiliated with any entity with which I am affiliated nor those of the >> entities themselves. >> >> On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> wrote: >> >> > solaris 10 >> > IDS 11.70.FC2 >> > >> > Is there a good way to read the log files or some other way to monitor all >> > the >> > data changes ocurring on a database between a certain date/time range? We >> > only >> > need some of the data fields from up to 65 tables, but we would like to >> > copy >> > the delta to another database on a nightly basis for those particular >> > fields. >> > If the data were readable in some way as data string that could be parsed, >> > that might be an option in parsing the logs; but if not, does anyone else >> > have >> > a suggestion? >> > >> > Any suggestions would be appreciated. The target database is a Postgres >> > database (FYI) >> > >> > Thanks in advance. >> > >> > Larry >> > >> > >> > >> > >> > ******************************************************************************* >> > Forum Note: Use "Reply" to post a response in the discussion forum. >> > >> > >> >> --20cf307cfbca5944ce04a32dcbf2 >> >> >> > ******************************************************************************* >> Forum Note: Use "Reply" to post a response in the discussion forum. >> > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > --bcaec5014c47eecda804a3902b68

Sorry Art, but I have another question: We are running HDR. Do you know if we can just set this up on one server, possibly the secondary, or does it need to be set up on both... Thank you again. Larry > To: ids@iiug.org > From: art.kagel@gmail.com > Subject: RE: Monitoring the change log [23754] > Date: Wed, 18 May 2011 13:25:17 -0400 > > It is native. There was ay least one session on using CDC here at the IIUG > conference. The presentations will be on the IIUG web site later today or > tomorrow. Look for it. > > Art > On May 18, 2011 10:21 AM, "LARRY SORENSEN" <lsorensen25@msn.com> wrote: > > Art, > > > > Do I need to install this API, or is natively available? > > > > Larry > > > >> To: ids@iiug.org > >> From: art.kagel@gmail.com > >> Subject: Re: Monitoring the change log [23700] > >> Date: Fri, 13 May 2011 16:03:10 -0400 > >> > >> The best thing to do is to use the Change Data Capture API to write an > app > >> that can subscribe to the logical log record stream. > >> > >> Art > >> > >> Art S. Kagel > >> Advanced DataTools (www.advancedatatools.com) > >> Blog: http://informix-myview.blogspot.com/ > >> > >> Disclaimer: Please keep in mind that my own opinions are my own opinions > and > >> do not reflect on my employer, Advanced DataTools, the IIUG, nor any > other > >> organization with which I am associated either explicitly, implicitly, or > by > >> inference. Neither do those opinions reflect those of other individuals > >> affiliated with any entity with which I am affiliated nor those of the > >> entities themselves. > >> > >> On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> > wrote: > >> > >> > solaris 10 > >> > IDS 11.70.FC2 > >> > > >> > Is there a good way to read the log files or some other way to monitor > all > >> > the > >> > data changes ocurring on a database between a certain date/time range? > We > >> > only > >> > need some of the data fields from up to 65 tables, but we would like to > > >> > copy > >> > the delta to another database on a nightly basis for those particular > >> > fields. > >> > If the data were readable in some way as data string that could be > parsed, > >> > that might be an option in parsing the logs; but if not, does anyone > else > >> > have > >> > a suggestion? > >> > > >> > Any suggestions would be appreciated. The target database is a Postgres > > >> > database (FYI) > >> > > >> > Thanks in advance. > >> > > >> > Larry > >> > > >> > > >> > > >> > > >> > > > > ******************************************************************************* > > >> > Forum Note: Use "Reply" to post a response in the discussion forum. > >> > > >> > > >> > >> --20cf307cfbca5944ce04a32dcbf2 > >> > >> > >> > > > > ******************************************************************************* > > >> Forum Note: Use "Reply" to post a response in the discussion forum. > >> > > > > > > > > ******************************************************************************* > > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > --bcaec5014c47eecda804a3902b68 > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >

Good question. I don't know. Art On May 18, 2011 3:19 PM, "LARRY SORENSEN" <lsorensen25@msn.com> wrote: > Sorry Art, but I have another question: > > We are running HDR. Do you know if we can just set this up on one server, > possibly the secondary, or does it need to be set up on both... > > Thank you again. > > Larry > >> To: ids@iiug.org >> From: art.kagel@gmail.com >> Subject: RE: Monitoring the change log [23754] >> Date: Wed, 18 May 2011 13:25:17 -0400 >> >> It is native. There was ay least one session on using CDC here at the IIUG >> conference. The presentations will be on the IIUG web site later today or >> tomorrow. Look for it. >> >> Art >> On May 18, 2011 10:21 AM, "LARRY SORENSEN" <lsorensen25@msn.com> wrote: >> > Art, >> > >> > Do I need to install this API, or is natively available? >> > >> > Larry >> > >> >> To: ids@iiug.org >> >> From: art.kagel@gmail.com >> >> Subject: Re: Monitoring the change log [23700] >> >> Date: Fri, 13 May 2011 16:03:10 -0400 >> >> >> >> The best thing to do is to use the Change Data Capture API to write an >> app >> >> that can subscribe to the logical log record stream. >> >> >> >> Art >> >> >> >> Art S. Kagel >> >> Advanced DataTools (www.advancedatatools.com) >> >> Blog: http://informix-myview.blogspot.com/ >> >> >> >> Disclaimer: Please keep in mind that my own opinions are my own opinions >> and >> >> do not reflect on my employer, Advanced DataTools, the IIUG, nor any >> other >> >> organization with which I am associated either explicitly, implicitly, or >> by >> >> inference. Neither do those opinions reflect those of other individuals >> >> affiliated with any entity with which I am affiliated nor those of the >> >> entities themselves. >> >> >> >> On Fri, May 13, 2011 at 3:55 PM, LARRY SORENSEN <lsorensen25@msn.com> >> wrote: >> >> >> >> > solaris 10 >> >> > IDS 11.70.FC2 >> >> > >> >> > Is there a good way to read the log files or some other way to monitor >> all >> >> > the >> >> > data changes ocurring on a database between a certain date/time range? >> We >> >> > only >> >> > need some of the data fields from up to 65 tables, but we would like to >> >> >> > copy >> >> > the delta to another database on a nightly basis for those particular >> >> > fields. >> >> > If the data were readable in some way as data string that could be >> parsed, >> >> > that might be an option in parsing the logs; but if not, does anyone >> else >> >> > have >> >> > a suggestion? >> >> > >> >> > Any suggestions would be appreciated. The target database is a Postgres >> >> >> > database (FYI) >> >> > >> >> > Thanks in advance. >> >> > >> >> > Larry >> >> > >> >> > >> >> > >> >> > >> >> >> > >> >> > ******************************************************************************* >> >> >> > Forum Note: Use "Reply" to post a response in the discussion forum. >> >> > >> >> > >> >> >> >> --20cf307cfbca5944ce04a32dcbf2 >> >> >> >> >> >> >> > >> >> > ******************************************************************************* >> >> >> Forum Note: Use "Reply" to post a response in the discussion forum. >> >> >> > >> > >> > >> >> > ******************************************************************************* >> >> > Forum Note: Use "Reply" to post a response in the discussion forum. >> > >> >> --bcaec5014c47eecda804a3902b68 >> >> >> > ******************************************************************************* >> Forum Note: Use "Reply" to post a response in the discussion forum. >> > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > --bcaec548a6db55238e04a3954da3