backup_filter gzip ontape duration llog backupsize

Dear Group, I've got informix 11.7 in heritage that did not have any logical log backups whatsoever. Furthermore it was running nightly full ontape backup, which lasted 1H and 10min and makes a file around 600GB. Since there is a disc capacity constraint the next night the backup is overwritten. So in my opinion if something goes wrong with the engine within this 70minutes, we are completely out? What I did is added another ontape L0 backup before it, using a pigz compression which leads me to a ~70GB compressed file in 90min. , which is perfect. Idea behind having 1 large and 1 small file is in case of restore need, not to have to wait on decompression. Now I added llog backups to disk. I put compression in the onconfig file gzip and gunzip, so the llog backups are now much much smaller. However once i run ontape, (without pigz) it takes 4hours (since i guess there is a congesstion on the gzip process). Have set LTAPESIZE to 2097152, just following the documentation (othervise the alarm won't fire wiht this setting set to 0) http://www.informix-dba.com/2010/07/informix-backup-and-restore-bare.html Now my questions are: 1. is this parameter actually taken into consideration if i am doing disk backups? 2. is there any tweak to continue using pigz for nightly backups, but use gzip in the filter settings of onconfig. My final point is to have gzip-ed llogs and fast nightly backup on disk. 3. is there any way I can do a script which will delete all these small llog backups once they are not needed any more (I cannot find a way once the new ontape file is there that logs lower than xx id can be deleted)? Thank you Aleksandar

> On 11 Mar 2017, at 13:32, ALEKSANDAR IVANOVSKI = <aleksandar.ivanovski@gmail.com> wrote: >=20 > Dear Group,=20 > I've got informix 11.7 in heritage that did not have any logical log = backups=20 > whatsoever. Furthermore it was running nightly full ontape backup, = which=20 > lasted 1H and 10min and makes a file around 600GB. Since there is a = disc=20 > capacity constraint the next night the backup is overwritten. So in my = opinion=20 > if something goes wrong with the engine within this 70minutes, we are=20= > completely out?=20 Yes. And even if you could restore, you would potentially lose an entire = day=E2=80=99s work. > What I did is added another ontape L0 backup before it, using a pigz=20= > compression which leads me to a ~70GB compressed file in 90min. , = which is=20 > perfect.=20 > Idea behind having 1 large and 1 small file is in case of restore = need, not to=20 > have to wait on decompression.=20 Replace gzip/gunzip with pigz or lbzip2 http://lbzip2.org = <http://lbzip2.org/>; >=20 > Now I added llog backups to disk. I put compression in the onconfig = file gzip=20 > and gunzip, so the llog backups are now much much smaller.=20 > However once i run ontape, (without pigz) it takes 4hours (since i = guess there=20 > is a congesstion on the gzip process). Have set LTAPESIZE to 2097152, = just=20 > following the documentation (othervise the alarm won't fire wiht this = setting=20 > set to 0)=20 > = http://www.informix-dba.com/2010/07/informix-backup-and-restore-bare.html=20= >=20 > Now my questions are:=20 > 1. is this parameter actually taken into consideration if i am doing = disk=20 > backups?=20 > 2. is there any tweak to continue using pigz for nightly backups, but = use gzip=20 > in the filter settings of onconfig. My final point is to have gzip-ed = llogs=20 > and fast nightly backup on disk.=20 Use lpzip2. Or use pigz for both? > 3. is there any way I can do a script which will delete all these = small llog=20 > backups once they are not needed any more (I cannot find a way once = the new=20 > ontape file is there that logs lower than xx id can be deleted)?=20 Yes, there is. :-)

Thank you for the response i'll check other zip utilities. Can you please point me where to continue reading about removing "the old" llog backups? "Yes, there is. :-) " :) Thank you Aleksandar

Look at the man page for the "find" utility. Use it to "find" log files older than the oldest archive you are keeping. Art Art S. Kagel, President and Principal Consultant ASK Database Management www.askdbmgt.com Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Mon, Mar 13, 2017 at 3:07 AM, ALEKSANDAR IVANOVSKI < aleksandar.ivanovski@gmail.com> wrote: > Thank you for the response i'll check other zip utilities. > Can you please point me where to continue reading about removing "the old" > llog backups? "Yes, there is. :-) " :) > Thank you > Aleksandar > > > ************************************************************ > ******************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --f403045d57cc8faf1c054a9a333a

Spokey, you probably think Pbzip2 ?? @Art, since i am not able to do uncompressed ontape backups i am left with pretty much space in backup directory,(which means more than 1 compressed ontape any time) so I did find /LOG -mtime +2 -exec rm {} \\\\; Thank you Aleksandar

Sorry, I forgot to tell... It is always a good practice to keep more than 1 L0 log retention (I would rather prefer at least 2 L0 transactional logs on disk), or 3 days. This is mainly explained in case you need to restore an old transaction, during logical recovery (started before the beginning of current L0), or even if you need to recover from one specific transaction (using point in time restore). Best regards. Atenciosamente, Alexandre Marini [http://mcsoftware.com.br/mc_conteudo/assinaturas/logoassinatura.png] [http://mcsoftware.com.br/mc_conteudo/assinaturas/arquiteturalogo.png] ________________________________ De: Alexandre Marini Enviado: segunda-feira, 13 de março de 2017 08:41:43 Para: ids@iiug.org Assunto: Re: backup_filter gzip ontape duration llog backup [38736] Hello. You could easily find a script to keep a logging retention policy. As Art said, look for find + mtime in google, there are hundreds of good sources for using in your own script. Hope it helps. Best regards. Atenciosamente, Alexandre Marini [http://mcsoftware.com.br/mc_conteudo/assinaturas/logoassinatura.png] [http://mcsoftware.com.br/mc_conteudo/assinaturas/arquiteturalogo.png] ________________________________ De: ids-bounces@iiug.org <ids-bounces@iiug.org> em nome de Art Kagel <art.kagel@gmail.com> Enviado: segunda-feira, 13 de março de 2017 07:30:39 Para: ids@iiug.org Assunto: Re: backup_filter gzip ontape duration llog backup [38736] Look at the man page for the "find" utility. Use it to "find" log files older than the oldest archive you are keeping. Art Art S. Kagel, President and Principal Consultant ASK Database Management www.askdbmgt.com<http://www.askdbmgt.com>; Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Mon, Mar 13, 2017 at 3:07 AM, ALEKSANDAR IVANOVSKI < aleksandar.ivanovski@gmail.com> wrote: > Thank you for the response i'll check other zip utilities. > Can you please point me where to continue reading about removing "the old" > llog backups? "Yes, there is. :-) " :) > Thank you > Aleksandar > > > ************************************************************ > ******************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --f403045d57cc8faf1c054a9a333a ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

Aleksandar: 1. If you don't have room to keep the previous archive until you are ready to take the 3rd one, then how about copying it to another system for safe keeping before deleting it? Or go to Staples (or the equivalent) and by a 1TB external USB drive for less than $100 to hold the older archives? 2. Yes -mtime will work. I was also thinking of " ! -newer <path to archive file> -exec ... >. It gets around the problem of losing the logical log backups you'll need to restore if you happen to not be able to make a new archive for more than two days for some reason! As long as the archive file exists, logical log backups created after the archive was taken won't be deleted accidentally. Art Art S. Kagel, President and Principal Consultant ASK Database Management www.askdbmgt.com Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Mon, Mar 13, 2017 at 7:56 AM, ALEKSANDAR IVANOVSKI < aleksandar.ivanovski@gmail.com> wrote: > Spokey, you probably think Pbzip2 ?? > > @Art, since i am not able to do uncompressed ontape backups i am left with > pretty much space in backup directory,(which means more than 1 compressed > ontape any time) so I did find /LOG -mtime +2 -exec rm {} \\\\; > > Thank you > Aleksandar > > > ************************************************************ > ******************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --001a114cc32cafb9e1054a9b9030

Hello. You could easily find a script to keep a logging retention policy. As Art said, look for find + mtime in google, there are hundreds of good sources for using in your own script. Hope it helps. Best regards. Atenciosamente, Alexandre Marini [http://mcsoftware.com.br/mc_conteudo/assinaturas/logoassinatura.png] [http://mcsoftware.com.br/mc_conteudo/assinaturas/arquiteturalogo.png] ________________________________ De: ids-bounces@iiug.org <ids-bounces@iiug.org> em nome de Art Kagel <art.kagel@gmail.com> Enviado: segunda-feira, 13 de março de 2017 07:30:39 Para: ids@iiug.org Assunto: Re: backup_filter gzip ontape duration llog backup [38736] Look at the man page for the "find" utility. Use it to "find" log files older than the oldest archive you are keeping. Art Art S. Kagel, President and Principal Consultant ASK Database Management www.askdbmgt.com<http://www.askdbmgt.com>; Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Mon, Mar 13, 2017 at 3:07 AM, ALEKSANDAR IVANOVSKI < aleksandar.ivanovski@gmail.com> wrote: > Thank you for the response i'll check other zip utilities. > Can you please point me where to continue reading about removing "the old" > llog backups? "Yes, there is. :-) " :) > Thank you > Aleksandar > > > ************************************************************ > ******************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --f403045d57cc8faf1c054a9a333a ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

> On 11 Mar 2017, at 13:32, ALEKSANDAR IVANOVSKI = <aleksandar.ivanovski@gmail.com> wrote: >=20 > Dear Group,=20 > I've got informix 11.7 in heritage that did not have any logical log = backups=20 > whatsoever. Furthermore it was running nightly full ontape backup, = which=20 > lasted 1H and 10min and makes a file around 600GB. Since there is a = disc=20 > capacity constraint the next night the backup is overwritten. So in my = opinion=20 > if something goes wrong with the engine within this 70minutes, we are=20= > completely out?=20 Sounds like it > What I did is added another ontape L0 backup before it, using a pigz=20= > compression which leads me to a ~70GB compressed file in 90min. , = which is=20 > perfect.=20 > Idea behind having 1 large and 1 small file is in case of restore = need, not to=20 > have to wait on decompression.=20 >=20 You don=E2=80=99t have to wait gzip -d ontape.gz | ontape -t STDIO -r=20 I have been investigating // bzip on the advice of Spokey Wheeler If you have spare cores this is the way to go - it gives better = compression than gzip and is much faster http://compression.ca/pbzip2/ <http://compression.ca/pbzip2/>; =E2=80=94 Clive