RE: dbcopy over secure/encrypted line
Posted in 2008
Topics: Performance & Tuning
> From: Ian Michael Gumby [mailto:im_gumby@hotmail.com] > Sent: Thursday, September 18, 2008 10:03 AM > To: Mosser, Paul A.; curtis@crowson1.com; informix-list@iiug.org > Subject: RE: dbcopy over secure/encrypted line > > Why not write a simple python script that has two database connections. > One to the old database and one to the new database? > > In short, you select data from one table, insert row in to new table on > second box. > > If you want better performance, then create multiple threads and do a > table per thread. > > This will move your data over, never having to stage it on disk. > That is exactly what Art's dbcopy utility does, and it does it very well. The problem is that we need to do the data transfer over secure/encrypted connection. The recent suggestions of using HPL with named pipes and ssh look very promising... Thanks, Paul M.
mosserp@wellsfargo.com wrote:
>> From: Ian Michael Gumby [mailto:im_gumby@hotmail.com]
>> Sent: Thursday, September 18, 2008 10:03 AM
>> To: Mosser, Paul A.; curtis@crowson1.com; informix-list@iiug.org
>> Subject: RE: dbcopy over secure/encrypted line
>>
>> Why not write a simple python script that has two database
> connections.
>> One to the old database and one to the new database?
>>
>> In short, you select data from one table, insert row in to new table
> on
>> second box.
>>
>> If you want better performance, then create multiple threads and do a
>> table per thread.
>>
>> This will move your data over, never having to stage it on disk.
>>
>
> That is exactly what Art's dbcopy utility does, and it does it very
> well. The problem is that we need to do the data transfer over
> secure/encrypted connection. The recent suggestions of using HPL with
> named pipes and ssh look very promising...
>
> Thanks,
> Paul M.
>
HPL will work faster, and you'll have to work more :)
Well... with onpladm your work may be greatly reduced.
If performance is not the most important, than a direct insert (insert into
select from) can also work. You can do this over an SSH tunnel.
But given that Andrew already published an article for HPL/SSH/pipes, it's
probably worth the try.
Using Python or any other language to do this when you can do it with just
dbaccess doesn't appeal to me...
Regards.
--
Fernando Nunes
Portugal
http://informix-technology.blogspot.com
My email works... but I don't check it frequently...
Fernando Nunes wrote:
> mosserp@wellsfargo.com wrote:
>>> From: Ian Michael Gumby [mailto:im_gumby@hotmail.com]
>>> Sent: Thursday, September 18, 2008 10:03 AM
>>> To: Mosser, Paul A.; curtis@crowson1.com; informix-list@iiug.org
>>> Subject: RE: dbcopy over secure/encrypted line
>>>
>>> Why not write a simple python script that has two database
>> connections.
>>> One to the old database and one to the new database?
>>>
>>> In short, you select data from one table, insert row in to new table
>> on
>>> second box.
>>>
>>> If you want better performance, then create multiple threads and do a
>>> table per thread.
>>>
>>> This will move your data over, never having to stage it on disk.
>>>
>>
>> That is exactly what Art's dbcopy utility does, and it does it very
>> well. The problem is that we need to do the data transfer over
>> secure/encrypted connection. The recent suggestions of using HPL with
>> named pipes and ssh look very promising...
>>
>> Thanks,
>> Paul M.
>>
>
> HPL will work faster, and you'll have to work more :)
> Well... with onpladm your work may be greatly reduced.
>
> If performance is not the most important, than a direct insert (insert
> into select from) can also work. You can do this over an SSH tunnel.
>
> But given that Andrew already published an article for HPL/SSH/pipes,
> it's probably worth the try.
>
> Using Python or any other language to do this when you can do it with
> just dbaccess doesn't appeal to me...
>
> Regards.
>
Are they? Try to create an SSH tunnel between two hosts, like host1:port1 and
host2:port2 where host1 is the client machine, host2 is the server machine,
port1 is any port and port2 is the engine listener port... Then configure
SQLHOSTS on host1 to point to port1...
To have a bit more fun, establish the tunnel with keys... Make sure you don't
establish the trusts between the machines...
Then try to connect and enjoy... then come back and complain about something ;)
Regards.
--
Fernando Nunes
Portugal
http://informix-technology.blogspot.com
My email works... but I don't check it frequently...
Fernando Nunes wrote:
> Fernando Nunes wrote:
>> mosserp@wellsfargo.com wrote:
>>>> From: Ian Michael Gumby [mailto:im_gumby@hotmail.com]
>>>> Sent: Thursday, September 18, 2008 10:03 AM
>>>> To: Mosser, Paul A.; curtis@crowson1.com; informix-list@iiug.org
>>>> Subject: RE: dbcopy over secure/encrypted line
>>>>
>>>> Why not write a simple python script that has two database
>>> connections.
>>>> One to the old database and one to the new database?
>>>>
>>>> In short, you select data from one table, insert row in to new table
>>> on
>>>> second box.
>>>>
>>>> If you want better performance, then create multiple threads and do a
>>>> table per thread.
>>>>
>>>> This will move your data over, never having to stage it on disk.
>>>>
>>>
>>> That is exactly what Art's dbcopy utility does, and it does it very
>>> well. The problem is that we need to do the data transfer over
>>> secure/encrypted connection. The recent suggestions of using HPL with
>>> named pipes and ssh look very promising...
>>>
>>> Thanks,
>>> Paul M.
>>>
>>
>> HPL will work faster, and you'll have to work more :)
>> Well... with onpladm your work may be greatly reduced.
>>
>> If performance is not the most important, than a direct insert (insert
>> into select from) can also work. You can do this over an SSH tunnel.
>>
>> But given that Andrew already published an article for HPL/SSH/pipes,
>> it's probably worth the try.
>>
>> Using Python or any other language to do this when you can do it with
>> just dbaccess doesn't appeal to me...
>>
>> Regards.
>>
>
> Are they? Try to create an SSH tunnel between two hosts, like
> host1:port1 and host2:port2 where host1 is the client machine, host2 is
> the server machine, port1 is any port and port2 is the engine listener
> port... Then configure SQLHOSTS on host1 to point to port1...
>
> To have a bit more fun, establish the tunnel with keys... Make sure you
> don't establish the trusts between the machines...
>
> Then try to connect and enjoy... then come back and complain about
> something ;)
>
> Regards.
>
If you're wondering what is this all about, Ian sent me an email (not to the
list) and this is my reply...
Sorry for the confusion...
So, I'm not (yet) talking to myself... ;)
Regards.
--
Fernando Nunes
Portugal
http://informix-technology.blogspot.com
My email works... but I don't check it frequently...