incremental data extraction

Hi, I would like to know which approach would give better performance for incremental approach. Approach 1: 1.I have the last extracted data in a old table 2.I am reading the new data from another table in an ESQL/C program. 3.I declare cursor for both the tables in ESQL/C and compare row by row in a for loop. This is at the database end. I feel in this case the complexity would be n square, where n is the no of rows in both the tables. Approach 2: 1.I keep the data last extracted in a text file in sorted order based on a key. 2.I take the newly extracted data into another text file in sorted order and compare both files for any modifications. This is at the OS level. In this case the complexity would be n+constant. If any body can throw some insight from the OS block size and DataBase RA_PAGES point of view. Which approach would be faster. Bye.

PARAMESHWAR.... said: > Hi, > I would like to know which approach would give better performance for > incremental approach. Timestamp the rows? Then it's trivial in the database. > Approach 1: > 1.I have the last extracted data in a old table > 2.I am reading the new data from another table in an ESQL/C program. > 3.I declare cursor for both the tables in ESQL/C and compare row by row in > a for loop. > > This is at the database end. > I feel in this case the complexity would be n square, where n is the no of > rows > in both the tables. > > Approach 2: > 1.I keep the data last extracted in a text file in sorted order based on a > key. > 2.I take the newly extracted data into another text file in sorted order > and compare both files for any modifications. > > This is at the OS level. > In this case the complexity would be n+constant. > > If any body can throw some insight from the OS block size and DataBase > RA_PAGES> point of view. > > > Which approach would be faster. > > > > > Bye. > > > -- Bye now, Obnoxio "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" - Coluche "I'm trying to see things your way, but I can't get my head up my ass" - JCH "Ogni uomo mi guarda come se fossi una testa di cazzo" - Marco Travel broadens a person. You look as if you have been all over the world. I went to the airport to check in and they asked what I did because I looked like a terrorist. I said I was a comedian. They said, "Say something funny then." I told them I had just graduated from flying school. -- Ahmed Ahmed http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg

Hi, How do you timestamp the rows. Bye. Obnoxio The Chav <obnoxio@serendipita.com> wrote: PARAMESHWAR.... said: > Hi, > I would like to know which approach would give better performance for > incremental approach. Timestamp the rows? Then it's trivial in the database. > Approach 1: > 1.I have the last extracted data in a old table > 2.I am reading the new data from another table in an ESQL/C program. > 3.I declare cursor for both the tables in ESQL/C and compare row by row in > a for loop. > > This is at the database end. > I feel in this case the complexity would be n square, where n is the no of > rows > in both the tables. > > Approach 2: > 1.I keep the data last extracted in a text file in sorted order based on a > key. > 2.I take the newly extracted data into another text file in sorted order > and compare both files for any modifications. > > This is at the OS level. > In this case the complexity would be n+constant. > > If any body can throw some insight from the OS block size and DataBase > RA_PAGES> point of view. > > > Which approach would be faster. > > > > > Bye. > > > -- Bye now, Obnoxio "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" - Coluche "I'm trying to see things your way, but I can't get my head up my ass" - JCH "Ogni uomo mi guarda come se fossi una testa di cazzo" - Marco Travel broadens a person. You look as if you have been all over the world. I went to the airport to check in and they asked what I did because I looked like a terrorist. I said I was a comedian. They said, "Say something funny then." I told them I had just graduated from flying school. -- Ahmed Ahmed http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg --------------------------------- Do you Yahoo!? Yahoo! Small Business - Try our new resources site!

Approach 3: 1. Same extracted data in old table. 2. Include an ORDER BY clause in both cursors on the two tables on the same sort key so that you can do a parallel loop comparison which is O(n) where n is the average of the two row counts. Art S. Kagel ----- Original Message ----- From: Parameshwar.... <pcdudyala@yahoo.com> At: 3/22 0:42 > Hi, > I would like to know which approach would give better performance for > incremental approach. > > Approach 1: > 1.I have the last extracted data in a old table > 2.I am reading the new data from another table in an ESQL/C program. > 3.I declare cursor for both the tables in ESQL/C and compare row by row in a for > loop. > > This is at the database end. > I feel in this case the complexity would be n square, where n is the no of rows > in both the tables. > > Approach 2: > 1.I keep the data last extracted in a text file in sorted order based on a key. > 2.I take the newly extracted data into another text file in sorted order and > compare both files for any modifications. > > This is at the OS level. > In this case the complexity would be n+constant. > > If any body can throw some insight from the OS block size and DataBase RA_PAGES> point of view. > > > Which approach would be faster. > > > > > Bye.

Chakravarth.... said: > Hi, > How do you timestamp the rows. Add a timestamp column with a default of CURRENT? > Obnoxio The Chav <obnoxio@serendipita.com> wrote: > > PARAMESHWAR.... said: >> Hi, >> I would like to know which approach would give better performance for >> incremental approach. > > Timestamp the rows? Then it's trivial in the database. > >> Approach 1: >> 1.I have the last extracted data in a old table >> 2.I am reading the new data from another table in an ESQL/C program. >> 3.I declare cursor for both the tables in ESQL/C and compare row by row >> in >> a for loop. >> >> This is at the database end. >> I feel in this case the complexity would be n square, where n is the no >> of >> rows >> in both the tables. >> >> Approach 2: >> 1.I keep the data last extracted in a text file in sorted order based on >> a >> key. >> 2.I take the newly extracted data into another text file in sorted order >> and compare both files for any modifications. >> >> This is at the OS level. >> In this case the complexity would be n+constant. >> >> If any body can throw some insight from the OS block size and DataBase >> RA_PAGES>> point of view. >> >> >> Which approach would be faster. >> >> >> >> >> Bye. >> >> >> > > > -- > > Bye now, > Obnoxio > > "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" > - Coluche > > "I'm trying to see things your way, but I can't get my head up my ass" > - JCH > > "Ogni uomo mi guarda come se fossi una testa di cazzo" > - Marco > > Travel broadens a person. You look as if you have been all over the world. > > I went to the airport to check in and they asked what I did because I > looked like a terrorist. I said I was a comedian. They said, "Say > something funny then." I told them I had just graduated from flying > school. > > -- Ahmed Ahmed > > http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg > > > --------------------------------- > Do you Yahoo!? > Yahoo! Small Business - Try our new resources site! > -- Bye now, Obnoxio "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" - Coluche "I'm trying to see things your way, but I can't get my head up my ass" - JCH "Ogni uomo mi guarda come se fossi una testa di cazzo" - Marco Travel broadens a person. You look as if you have been all over the world. I went to the airport to check in and they asked what I did because I looked like a terrorist. I said I was a comedian. They said, "Say something funny then." I told them I had just graduated from flying school. -- Ahmed Ahmed http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg

David Williams said: > > current can still have duokicates. > add a serial column with a unique index. That is surely irrelevant for the purpose at hand? > ----- Original Message ----- > From: "Obnoxio The...." <obnoxio@serendipita.com> > To: <ids@iiug.org> > Sent: Tuesday, March 22, 2005 5:02 PM > Subject: Re: incremental data extraction [4587] > > >> >> Chakravarth.... said: >>> Hi, >>> How do you timestamp the rows. >> >> Add a timestamp column with a default of CURRENT? >> >>> Obnoxio The Chav <obnoxio@serendipita.com> wrote: >>> >>> PARAMESHWAR.... said: >>>> Hi, >>>> I would like to know which approach would give better performance for >>>> incremental approach. >>> >>> Timestamp the rows? Then it's trivial in the database. >>> >>>> Approach 1: >>>> 1.I have the last extracted data in a old table >>>> 2.I am reading the new data from another table in an ESQL/C program. >>>> 3.I declare cursor for both the tables in ESQL/C and compare row by >>>> row >>>> in >>>> a for loop. >>>> >>>> This is at the database end. >>>> I feel in this case the complexity would be n square, where n is the >>>> no >>>> of >>>> rows >>>> in both the tables. >>>> >>>> Approach 2: >>>> 1.I keep the data last extracted in a text file in sorted order based >>>> on >>>> a >>>> key. >>>> 2.I take the newly extracted data into another text file in sorted >>>> order >>>> and compare both files for any modifications. >>>> >>>> This is at the OS level. >>>> In this case the complexity would be n+constant. >>>> >>>> If any body can throw some insight from the OS block size and DataBase >>>> RA_PAGES>>>> point of view. >>>> >>>> >>>> Which approach would be faster. >>>> >>>> >>>> >>>> >>>> Bye. >>>> >>>> >>>> >>> >>> >>> -- >>> >>> Bye now, >>> Obnoxio >>> >>> "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" >>> - Coluche >>> >>> "I'm trying to see things your way, but I can't get my head up my ass" >>> - JCH >>> >>> "Ogni uomo mi guarda come se fossi una testa di cazzo" >>> - Marco >>> >>> Travel broadens a person. You look as if you have been all over the >>> world. >>> >>> I went to the airport to check in and they asked what I did because I >>> looked like a terrorist. I said I was a comedian. They said, "Say >>> something funny then." I told them I had just graduated from flying >>> school. >>> >>> -- Ahmed Ahmed >>> >>> http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg >>> >>> >>> --------------------------------- >>> Do you Yahoo!? >>> Yahoo! Small Business - Try our new resources site! >>> >> >> >> -- >> >> Bye now, >> Obnoxio >> >> "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" >> - Coluche >> >> "I'm trying to see things your way, but I can't get my head up my ass" >> - JCH >> >> "Ogni uomo mi guarda come se fossi una testa di cazzo" >> - Marco >> >> Travel broadens a person. You look as if you have been all over the >> world. >> >> I went to the airport to check in and they asked what I did because I >> looked like a terrorist. I said I was a comedian. They said, "Say >> something funny then." I told them I had just graduated from flying >> school. >> >> -- Ahmed Ahmed >> >> http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg >> >> --- >> [This E-mail has been scanned for viruses but it is your responsibility >> to maintain up to date anti virus software on the device that you are >> currently using to read this email. ] >> >> > > --- > [This E-mail has been scanned for viruses but it is your responsibility > to maintain up to date anti virus software on the device that you are > currently using to read this email. ] > -- Bye now, Obnoxio "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" - Coluche "I'm trying to see things your way, but I can't get my head up my ass" - JCH "Ogni uomo mi guarda come se fossi una testa di cazzo" - Marco Travel broadens a person. You look as if you have been all over the world. I went to the airport to check in and they asked what I did because I looked like a terrorist. I said I was a comedian. They said, "Say something funny then." I told them I had just graduated from flying school. -- Ahmed Ahmed http://i2.photobucket.com/albums/y41/Obnoxio/thinkIfoundtheproblem.jpg