Mix in encoding...

Hi, I am running an engine which we want to be en-us.utf-8 (we want it that way to have the maximum flexibility over ANY character set, if we are wrong, please advice). Nevertheless, we have a couple of apps that uses it fine, setting DB_LOCALE and SERVER_LOCALE. Unfortunately, one app (to rename nameless ;-) but not one of ours), has modified "some" of the data and now we get plenty of "java.sql.SQLException: Encoding or code set not supported." Of course, things like DBexport because our tables are too big. Can I convert all my fields to UTF8? Is there some kind of magical tool out there? jgp

Hi Jean George, Currently the only method to "Convert" A non-utf8 db to a utf8 db is the old fashioned way (i.e. dbexport/dbimport) I know that there is a feature request for a means to convert a database from non-utf8 to utf8, I entered it, but nothing like that exists today. If you don't think it is corruption, or are willing to work with the corruption after the fact, and have the space, you can move it using the old method of insert into utf8db:tabname select * from olddb:tabname but you need to have a certain Undocumented environment variable set in order to do this, since a non-UTF8 db is not supposed to be able to talk to an UTF8 db. I suppose I should also take this moment to plug a forthcoming presentation at IIUG this year. It's on best practices and tips and tricks to migrate to Unicode/utf8. ----- Original Message ---- From: Jean Georges Perrin <jgp@jgp.net> To: ids@iiug.org Sent: Tuesday, December 18, 2007 3:16:10 AM Subject: Mix in encoding... [10781] Hi, I am running an engine which we want to be en-us.utf-8 (we want it that way to have the maximum flexibility over ANY character set, if we are wrong, please advice). Nevertheless, we have a couple of apps that uses it fine, setting DB_LOCALE and SERVER_LOCALE. Unfortunately, one app (to rename nameless ;-) but not one of ours), has modified "some" of the data and now we get plenty of "java.sql.SQLException: Encoding or code set not supported." Of course, things like DBexport because our tables are too big. Can I convert all my fields to UTF8? Is there some kind of magical tool out there? jgp ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

All --- Please don't use undocumented stuff. It's not documented for a= reason. If an undocumented EV were safe for general use, then it would= be documented. ------------------------------------- Madison Pruet, STSM IDS Replication Architect = "Mark Jamison" = <majp51@yahoo.com = > = To Sent by: ids@iiug.org = ids-bounces@iiug. = cc org = Subj= ect Re: Mix in encoding... [10782] = 12/18/2007 09:22 = AM = = = Please respond to = ids@iiug.org = = = Hi Jean George, Currently the only method to "Convert" A non-utf8 db to a utf8 db is th= e old fashioned way (i.e. dbexport/dbimport) I know that there is a feature request for a means to convert a databas= e from non-utf8 to utf8, I entered it, but nothing like that exists today. If you don't think it is corruption, or are willing to work with the corruption after the fact, and have the space, you can move it using th= e old method of insert into utf8db:tabname select * from olddb:tabname but you need to have a certain Undocumented environment variable set in= order to do this, since a non-UTF8 db is not supposed to be able to talk to a= n UTF8 db. I suppose I should also take this moment to plug a forthcoming presenta= tion at IIUG this year. It's on best practices and tips and tricks to migrate t= o Unicode/utf8. ----- Original Message ---- From: Jean Georges Perrin <jgp@jgp.net> To: ids@iiug.org Sent: Tuesday, December 18, 2007 3:16:10 AM Subject: Mix in encoding... [10781] Hi, I am running an engine which we want to be en-us.utf-8 (we want it that= way to have the maximum flexibility over ANY character set, if we are wrong, please advice). Nevertheless, we have a couple of apps that uses it fine, setting DB_LOCALE and SERVER_LOCALE. Unfortunately, one app (to rename nameless ;-) but not one of ours), has modified "some" of the data and now we get plenty of "java.sql.SQLException: Encoding or code set not supported." Of course, things like DBexport because our tables are too big. Can I convert all my fields to UTF8? Is there some kind of magical tool= out there? jgp ***********************************************************************= ******** Forum Note: Use "Reply" to post a response in the discussion forum. ***********************************************************************= ******** Forum Note: Use "Reply" to post a response in the discussion forum. =

Thanks Mark, that was NOT helpful, but thanks for the plug to the conference, I can add mine... Finally a session that will explain you what is SOA in a pragmatic approach based on real life implementation of GreenIvory's product. Don't worry: there is no marketing, but a real description of a SOA project's lifecycle. We will introduce what SOA is, how we used it and finally show a small Java-based, stores-based application featuring SOA. Some knowledge of web architecture, Java development and Eclipse will help the attendee to focus on the project's "core". ;-) Still stuck with my data "corruption"... -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Mark Jamison Sent: Tuesday, December 18, 2007 16:22 To: ids@iiug.org Subject: Re: Mix in encoding... [10782] Hi Jean George, Currently the only method to "Convert" A non-utf8 db to a utf8 db is the old fashioned way (i.e. dbexport/dbimport) I know that there is a feature request for a means to convert a database from non-utf8 to utf8, I entered it, but nothing like that exists today. If you don't think it is corruption, or are willing to work with the corruption after the fact, and have the space, you can move it using the old method of insert into utf8db:tabname select * from olddb:tabname but you need to have a certain Undocumented environment variable set in order to do this, since a non-UTF8 db is not supposed to be able to talk to an UTF8 db. I suppose I should also take this moment to plug a forthcoming presentation at IIUG this year. It's on best practices and tips and tricks to migrate to Unicode/utf8. ----- Original Message ---- From: Jean Georges Perrin <jgp@jgp.net> To: ids@iiug.org Sent: Tuesday, December 18, 2007 3:16:10 AM Subject: Mix in encoding... [10781] Hi, I am running an engine which we want to be en-us.utf-8 (we want it that way to have the maximum flexibility over ANY character set, if we are wrong, please advice). Nevertheless, we have a couple of apps that uses it fine, setting DB_LOCALE and SERVER_LOCALE. Unfortunately, one app (to rename nameless ;-) but not one of ours), has modified "some" of the data and now we get plenty of "java.sql.SQLException: Encoding or code set not supported." Of course, things like DBexport because our tables are too big. Can I convert all my fields to UTF8? Is there some kind of magical tool out there? jgp **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum. **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum.

Hi Jean George, Might I suggest contacting technical support. At the very least having another customer add their name to a feature request allowing DB's to be moved in-place from a non-unicode DB to a Unicode db increases the likelyhood of the problem being addressed sooner. Thanks, Mark ----- Original Message ---- From: Jean Georges Perrin <jgp@jgp.net> To: ids@iiug.org Sent: Thursday, December 20, 2007 12:14:33 AM Subject: RE: Mix in encoding... [10792] Thanks Mark, that was NOT helpful, but thanks for the plug to the conference, I can add mine... Finally a session that will explain you what is SOA in a pragmatic approach based on real life implementation of GreenIvory's product. Don't worry: there is no marketing, but a real description of a SOA project's lifecycle. We will introduce what SOA is, how we used it and finally show a small Java-based, stores-based application featuring SOA. Some knowledge of web architecture, Java development and Eclipse will help the attendee to focus on the project's "core". ;-) Still stuck with my data "corruption"... -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Mark Jamison Sent: Tuesday, December 18, 2007 16:22 To: ids@iiug.org Subject: Re: Mix in encoding... [10782] Hi Jean George, Currently the only method to "Convert" A non-utf8 db to a utf8 db is the old fashioned way (i.e. dbexport/dbimport) I know that there is a feature request for a means to convert a database from non-utf8 to utf8, I entered it, but nothing like that exists today. If you don't think it is corruption, or are willing to work with the corruption after the fact, and have the space, you can move it using the old method of insert into utf8db:tabname select * from olddb:tabname but you need to have a certain Undocumented environment variable set in order to do this, since a non-UTF8 db is not supposed to be able to talk to an UTF8 db. I suppose I should also take this moment to plug a forthcoming presentation at IIUG this year. It's on best practices and tips and tricks to migrate to Unicode/utf8. ----- Original Message ---- From: Jean Georges Perrin <jgp@jgp.net> To: ids@iiug.org Sent: Tuesday, December 18, 2007 3:16:10 AM Subject: Mix in encoding... [10781] Hi, I am running an engine which we want to be en-us.utf-8 (we want it that way to have the maximum flexibility over ANY character set, if we are wrong, please advice). Nevertheless, we have a couple of apps that uses it fine, setting DB_LOCALE and SERVER_LOCALE. Unfortunately, one app (to rename nameless ;-) but not one of ours), has modified "some" of the data and now we get plenty of "java.sql.SQLException: Encoding or code set not supported." Of course, things like DBexport because our tables are too big. Can I convert all my fields to UTF8? Is there some kind of magical tool out there? jgp **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum. **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.