Compression

Hi, IDS 11.50.FC4 on solaris 10 I read: New Format for Backup Filters Backups have a new format for version 11.50.xC2 if they are performed with the ontape command using a filter specified by the BACKUP_FILTER configuration parameter. The new backup format is not backwards compatible. Any backups that were made prior to version 11.50.xC2 must be restored with the earlier versions of the filter. since upgrade i did few level 0 backups and all of almost same size ~3.5 Gb. yesterday after compressing many table from one of my database i ran update stats and did a level 0 backup and this time the size of backup is ~5.7 Gb, Is that possible ? any thing to do with new format for backup_filter along with compression? I checked after compression repack and shrink that almost 2Gb of free space added to free list of chunk. I have not changed ^BACKUP_FILTER /bin/gzip It is still the same for last years. I've certainly not added any data to this instance and its my test instance so no one uses it except for me. Regards Vikas

Howe Another database (most of the big tables) compressed repack and shrink and another level 0 with increase backup size before any compression = 3509846016 After 1 db compression = 5731647488 After 2 db compression = 6277529600 Any Idea ? Regards Vikas ******************************************************************************* Hi, IDS 11.50.FC4 on solaris 10 I read: New Format for Backup Filters Backups have a new format for version 11.50.xC2 if they are performed with the ontape command using a filter specified by the BACKUP_FILTER configuration parameter. The new backup format is not backwards compatible. Any backups that were made prior to version 11.50.xC2 must be restored with the earlier versions of the filter. since upgrade i did few level 0 backups and all of almost same size ~3.5 Gb. yesterday after compressing many table from one of my database i ran update stats and did a level 0 backup and this time the size of backup is ~5.7 Gb, Is that possible ? any thing to do with new format for backup_filter along with compression? I checked after compression repack and shrink that almost 2Gb of free space added to free list of chunk. I have not changed ^BACKUP_FILTER /bin/gzip It is still the same for last years. I've certainly not added any data to this instance and its my test instance so no one uses it except for me. Regards Vikas

Compressed data doesn't gzip well especially when, as is the case with the IDS compression, the compression algorithm is substantially the same as the one that gzip uses. So, now that your rows are already compressed, you archives are not compressing. You might try bz2 it can sometimes get further compression out of gzipped files. Art Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Jul 10, 2009 at 8:29 AM, VIKAS HIVARKAR <vikas.hivarkar@gmail.com>wrote: > Hi, > > IDS 11.50.FC4 on solaris 10 > > I read: > New Format for Backup Filters > Backups have a new format for version 11.50.xC2 if they are performed with > the > ontape command using a filter specified by the BACKUP_FILTER configuration > parameter. The new backup format is not backwards compatible. Any backups > that > were made prior to version 11.50.xC2 must be restored with the earlier > versions of the filter. > > since upgrade i did few level 0 backups and all of almost same size ~3.5 > Gb. > > yesterday after compressing many table from one of my database i ran update > stats and did a level 0 backup and this time the size of backup is ~5.7 Gb, > Is that possible ? any thing to do with new format for backup_filter along > with compression? > > I checked after compression repack and shrink that almost 2Gb of free space > added to free list of chunk. > > I have not changed ^BACKUP_FILTER /bin/gzip It is still the same for last > years. I've certainly not added any data to this instance and its my test > instance so no one uses it except for me. > > Regards > Vikas > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --0016e68fc5bbff79ec046e5bf510

Thanks Mr Kagel ! Your explanation holds very true in my case. I dont know why did is not mentioned in any documentation or may be i missed it. "Compression and Storage Optimization can save disk space and thus money" Now how do i convience this to my businees? on one hand using compression,repack and shrink i freed up lots of sapce and will be getting the benefits of compression but on other hand my backup size using the backup filter /bin/gzip is Increasing. Just as an example of my test database i use to get a backup file of size ~3.5 Gb using gzip filter ( ~ 50 Gb with out gzip ) but after compressing just one database (only big tables) the new backup size is ~ 5.7 Gb and is increasing as i compress the next databases. We store atleast 3 backup files and the logical logs for these. Now assuming the logical logs size will decrease due to compressed data I will gain some space resulting less storage cost and ofcourse the performance boost which is already good with 11.10 BUT my backup file storage cost is going to increase and increase considerably. This is not an argument I'm trying to put forward my concern :( I will try bzip2 and see how much compression i get. Regards, Vikas ******************************************************************************* Compressed data doesn't gzip well especially when, as is the case with the IDS compression, the compression algorithm is substantially the same as the one that gzip uses. So, now that your rows are already compressed, you archives are not compressing. You might try bz2 it can sometimes get further compression out of gzipped files. Art Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Jul 10, 2009 at 8:29 AM, VIKAS HIVARKAR <vikas.hivarkar@gmail.com>wrote: > Hi, > > IDS 11.50.FC4 on solaris 10 > > I read: > New Format for Backup Filters > Backups have a new format for version 11.50.xC2 if they are performed with > the > ontape command using a filter specified by the BACKUP_FILTER configuration > parameter. The new backup format is not backwards compatible. Any backups > that > were made prior to version 11.50.xC2 must be restored with the earlier > versions of the filter. > > since upgrade i did few level 0 backups and all of almost same size ~3.5 > Gb. > > yesterday after compressing many table from one of my database i ran update > stats and did a level 0 backup and this time the size of backup is ~5.7 Gb, > Is that possible ? any thing to do with new format for backup_filter along > with compression? > > I checked after compression repack and shrink that almost 2Gb of free space > added to free list of chunk. > > I have not changed ^BACKUP_FILTER /bin/gzip It is still the same for last > years. I've certainly not added any data to this instance and its my test > instance so no one uses it except for me. > > Regards > Vikas > > > > *******************************************************************************

Hello Mr Kagel ! I got almost 2 Gb of compression in the backup file size using the BACKUP_FILTER /bin/bzip3 :) Regards Vikas ******************************************************************************* Thanks Mr Kagel ! Your explanation holds very true in my case. I dont know why did is not mentioned in any documentation or may be i missed it. "Compression and Storage Optimization can save disk space and thus money" Now how do i convience this to my businees? on one hand using compression,repack and shrink i freed up lots of sapce and will be getting the benefits of compression but on other hand my backup size using the backup filter /bin/gzip is Increasing. Just as an example of my test database i use to get a backup file of size ~3.5 Gb using gzip filter ( ~ 50 Gb with out gzip ) but after compressing just one database (only big tables) the new backup size is ~ 5.7 Gb and is increasing as i compress the next databases. We store atleast 3 backup files and the logical logs for these. Now assuming the logical logs size will decrease due to compressed data I will gain some space resulting less storage cost and ofcourse the performance boost which is already good with 11.10 BUT my backup file storage cost is going to increase and increase considerably. This is not an argument I'm trying to put forward my concern :( I will try bzip2 and see how much compression i get. Regards, Vikas ******************************************************************************* Compressed data doesn't gzip well especially when, as is the case with the IDS compression, the compression algorithm is substantially the same as the one that gzip uses. So, now that your rows are already compressed, you archives are not compressing. You might try bz2 it can sometimes get further compression out of gzipped files. Art Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Jul 10, 2009 at 8:29 AM, VIKAS HIVARKAR <vikas.hivarkar@gmail.com>wrote: > Hi, > > IDS 11.50.FC4 on solaris 10 > > I read: > New Format for Backup Filters > Backups have a new format for version 11.50.xC2 if they are performed with > the > ontape command using a filter specified by the BACKUP_FILTER configuration > parameter. The new backup format is not backwards compatible. Any backups > that > were made prior to version 11.50.xC2 must be restored with the earlier > versions of the filter. > > since upgrade i did few level 0 backups and all of almost same size ~3.5 > Gb. > > yesterday after compressing many table from one of my database i ran update > stats and did a level 0 backup and this time the size of backup is ~5.7 Gb, > Is that possible ? any thing to do with new format for backup_filter along > with compression? > > I checked after compression repack and shrink that almost 2Gb of free space > added to free list of chunk. > > I have not changed ^BACKUP_FILTER /bin/gzip It is still the same for last > years. I've certainly not added any data to this instance and its my test > instance so no one uses it except for me. > > Regards > Vikas > > > > *******************************************************************************