Compressed data in logical logs
Posted in 2008
The poster found that 'onlog -l' shows only the before/after images of changed columns for an UPDATE, not the whole row, and asked how to disable this "audit compression" (as with Oracle's supplemental logging) so full row images get written to the logical logs. Answer: it can't be done — IDS always logs only the changed portion. Respondents suggested using the Secure Audit Facility for auditing, and for his actual goal (replication) pointed him to built-in HDR, ER, RSS, SDS, Enterprise Gateway Manager/WebSphere or third-party tools rather than rolling his own. No way to force full-row logging was offered.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Security, Permissions & Auditing, Logging & Checkpoints
Hi,
I have an issue when I try to look at the data in the logical logs using
"onlog -l".
I created a simple table:
create table t1 (a char(10), b char(10), c char(10));
insert into t1 values ('aaaaaaaaaa','bbbbbbbbbb','cccccccccc');
and I updated column c:
update t1 set c='gggggggggg';
Now I am looking at the output of "onlog -l" to see the UPDATE representative
and this is what I get:
addr len type xid id link
d5044 84 HUPDAT 5 0 d5018 20003d 101 0 30 30 1
54000000 00004900 10000000 00002074 T.....I. ...... t
05000000 18500d00 e98c0500 3d002000 .....P.. ....=. .
3d002000 01010000 00000000 1e001e00 =. ..... ........
01000000 00000000 00000000 0014000a ........ ........
63636363 63636363 63636767 67676767 cccccccc ccgggggg
67676767 gggg
As you can see, the binary data contains only the before/after image of the
column that was updated (ccccccccccgggggggggg).
My question is whether there is a way to force the database to write all the
columns values to the logical logs the same way it does in case I do an
insert??
In other words, Is there a way to turn off the audit compression?
I know how to do it in other datatabases (In Oracle it's ALTER TABLE ADD
SUPPLEMENTAL LOG, In SQL/MP it's ALTER TABLE NO AUDITCOMPRESS). What's the
equivalent in informix??
Thanks,
Sami.
SAMI ZEITOUN wrote:
> Hi,
> I have an issue when I try to look at the data in the logical logs using
> "onlog -l".
>
> I created a simple table:
> create table t1 (a char(10), b char(10), c char(10));
> insert into t1 values ('aaaaaaaaaa','bbbbbbbbbb','cccccccccc');>
> and I updated column c:
>
> update t1 set c='gggggggggg';>
> Now I am looking at the output of "onlog -l" to see the UPDATE representative
> and this is what I get:
>
> addr len type xid id link
>
> d5044 84 HUPDAT 5 0 d5018 20003d 101 0 30 30 1
>
> 54000000 00004900 10000000 00002074 T.....I. ...... t
> 05000000 18500d00 e98c0500 3d002000 .....P.. ....=. .
> 3d002000 01010000 00000000 1e001e00 =. ..... ........
> 01000000 00000000 00000000 0014000a ........ ........
> 63636363 63636363 63636767 67676767 cccccccc ccgggggg
> 67676767 gggg
>
> As you can see, the binary data contains only the before/after image of the
> column that was updated (ccccccccccgggggggggg).
>
> My question is whether there is a way to force the database to write all the
> columns values to the logical logs the same way it does in case I do an
> insert??
> In other words, Is there a way to turn off the audit compression?
> I know how to do it in other datatabases (In Oracle it's ALTER TABLE ADD
> SUPPLEMENTAL LOG, In SQL/MP it's ALTER TABLE NO AUDITCOMPRESS). What's the
> equivalent in informix??
>
What you are looking to do cannot be done. You are trying to use IDS's
recovery logs for audit purposes. What you want is the Secure Audit
Facility which is an add-on feature you can purchase. It has fully
configurable audit and reporting features and can capture many items of
detail that are needed for auditing that are not required for data recovery.
Art S. Kagel
Oninit
> Thanks,
> Sami.
>
>
>
================================================================================
===========
Please access the attached hyperlink for an important electronic
communications disclaimer:
http://www.oninit.com/home/disclaimer.php
================================================================================
===========
Thanks for the reply. Actually, I need this for replication purpose. Are you saying that the data buffer in the logical logs is allways in a compressed mode?? There is no way for me to make the database write to the logical logs the entire data buffer of the updated record?? and the only thing that is written to the logical logs in case of UPDATE is the before/after values of the changed columns?? Thanks, Sami.
Could you elaborate? What are the things missing from the current prod= uct with respect to replication that raised the need for you to develop you= r own replication solution? Thanks ------------------------------------- Madison Pruet, STSM IDS Replication Architect = "SAMI ZEITOUN" = <sami.zeitoun@att = unity.com> = To Sent by: ids@iiug.org = ids-bounces@iiug. = cc org = Subj= ect Re: Compressed data in logical l= ogs 02/13/2008 09:12 [11297] = AM = = = Please respond to = ids@iiug.org = = = Thanks for the reply. Actually, I need this for replication purpose. Are you saying that the data buffer in the logical logs is allways in a= compressed mode?? There is no way for me to make the database write to the logical logs t= he entire data buffer of the updated record?? and the only thing that is written to the logical logs in case of UPDATE is the before/after values of the= changed columns?? Thanks, Sami. ***********************************************************************= ******** Forum Note: Use "Reply" to post a response in the discussion forum. See you at the IIUG Informix 2008 Conference The Power Conference for Informix Professionals April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas http://www.iiug.org/conf Registration Now Open!! =
SAMI ZEITOUN wrote: > Thanks for the reply. > Actually, I need this for replication purpose. > Are you saying that the data buffer in the logical logs is allways in a > compressed mode?? > There is no way for me to make the database write to the logical logs the > entire data buffer of the updated record?? and the only thing that is written > to the logical logs in case of UPDATE is the before/after values of the > changed columns?? > Correct. But, I'm with Madison, IDS has several flavors of replication built in. Why are you reinventing the wheel when you have HDR (High Availability Data Replication), ER (Enterprise Replication), RSS (Remote Server Secondaries), and SDS (Shared Disk Secondary Servers) to do the job for you? And when they can all be combined into myriad combinations I can't think of any replication, recover, or high availability requirement that couldn't be met. If you have such a requirement then tell Madison and likely he'll add it in! But let's explore the real need first! :-) Art S. Kagel Oninit > Thanks, > Sami. > > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
On 13/02/2008, Art S. Kagel (Oninit LLC) <art@oninit.com> wrote: > SAMI ZEITOUN wrote: > > Thanks for the reply. > > Actually, I need this for replication purpose. > > Are you saying that the data buffer in the logical logs is allways in a > > compressed mode?? > > There is no way for me to make the database write to the logical logs the > > entire data buffer of the updated record?? and the only thing that is > written > > to the logical logs in case of UPDATE is the before/after values of the > > changed columns?? > > > > Correct. But, I'm with Madison, IDS has several flavors of replication > built in. Why are you reinventing the wheel when you have HDR (High > Availability Data Replication), ER (Enterprise Replication), RSS (Remote > Server Secondaries), and SDS (Shared Disk Secondary Servers) to do the > job for you? And when they can all be combined into myriad combinations > I can't think of any replication, recover, or high availability > requirement that couldn't be met. If you have such a requirement then > tell Madison and likely he'll add it in! But let's explore the real > need first! :-) > > Art S. Kagel > Oninit > > > Thanks, > > Sami. > > > > > > > > > ================================================================================ =========== > Please access the attached hyperlink for an important electronic > communications disclaimer: > > http://www.oninit.com/home/disclaimer.php > > > ================================================================================ =========== > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > See you at the IIUG Informix 2008 Conference > The Power Conference for Informix Professionals > April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas > http://www.iiug.org/conf > Registration Now Open!! > And if you need to replicate to other providers there is Enterprise Gateway Manager or the real sledge-hammer of WebSphere. Keith
Keith Simmons wrote: > On 13/02/2008, Art S. Kagel (Oninit LLC) <art@oninit.com> wrote: > >> SAMI ZEITOUN wrote: >> >>> Thanks for the reply. >>> Actually, I need this for replication purpose. >>> Are you saying that the data buffer in the logical logs is allways in a >>> compressed mode?? >>> There is no way for me to make the database write to the logical logs the >>> entire data buffer of the updated record?? and the only thing that is >>> >> written >> >>> to the logical logs in case of UPDATE is the before/after values of the >>> changed columns?? >>> >>> >> Correct. But, I'm with Madison, IDS has several flavors of replication >> built in. Why are you reinventing the wheel when you have HDR (High >> Availability Data Replication), ER (Enterprise Replication), RSS (Remote >> Server Secondaries), and SDS (Shared Disk Secondary Servers) to do the >> job for you? And when they can all be combined into myriad combinations >> I can't think of any replication, recover, or high availability >> requirement that couldn't be met. If you have such a requirement then >> tell Madison and likely he'll add it in! But let's explore the real >> need first! :-) >> >> Art S. Kagel >> Oninit >> >> >>> Thanks, >>> Sami. >>> >>> >>> >>> >> >> > ================================================================================ =========== > >> Please access the attached hyperlink for an important electronic >> communications disclaimer: >> >> http://www.oninit.com/home/disclaimer.php >> >> >> >> > ================================================================================ =========== > >> >> > ******************************************************************************* > >> Forum Note: Use "Reply" to post a response in the discussion forum. >> >> See you at the IIUG Informix 2008 Conference >> The Power Conference for Informix Professionals >> April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas >> http://www.iiug.org/conf >> Registration Now Open!! >> >> > And if you need to replicate to other providers there is Enterprise > Gateway Manager or the real sledge-hammer of WebSphere. > > Keith > Not to mention a few existing third party cross server replication tools that support IDS already. Art S. Kagel Oninit ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========