Backup Help

Hi, I have a test Informix instance setup. I am testing doing a level 0 backup. The database is very small, yet it seems to be running for over an hour now. When I run onstat -m, I see it says Level0 Archive started on rootdbs, logdbs, datadbs I thought it must have completed so then I ran a command to try a level 1 backup, and it is telling me Archive failed - Another archive is already in progress on DBspace rootdbs. So this is a small database with literally nothing hardly in it. The size of a backup I took back when I first made this in April was 20 MB I believe. What could be holding that up? I was expecting the backup to take a couple of seconds at the most... Is there any troubleshooting I can do to find out what is going on or what actually is running? I hit cntrol-c to get out of it, but it is still running the archive in the background.

Are you using "onbar"? If so it's likely a problem connecting to the storage manager or a problem on the storage manager side - maybe it's busy doing other backups. Check the onbar activity file (BAR_ACT_LOG) to see if it says anything interesting, or the BAR_DEBUG_LOG, if you have set up debugging. -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of BENJI LONG Sent: Wednesday, September 13, 2017 9:13 AM To: ids@iiug.org Subject: Backup Help [39823] Hi, I have a test Informix instance setup. I am testing doing a level 0 backup. The database is very small, yet it seems to be running for over an hour now. When I run onstat -m, I see it says Level0 Archive started on rootdbs, logdbs, datadbs I thought it must have completed so then I ran a command to try a level 1 backup, and it is telling me Archive failed - Another archive is already in progress on DBspace rootdbs. So this is a small database with literally nothing hardly in it. The size of a backup I took back when I first made this in April was 20 MB I believe. What could be holding that up? I was expecting the backup to take a couple of seconds at the most... Is there any troubleshooting I can do to find out what is going on or what actually is running? I hit cntrol-c to get out of it, but it is still running the archive in the background. **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum.

Assuming that you're on Unix/Linux environment, running relatively not-too-old version of Informix, and using ontape utility. Type: onstat -c | grep TAPELook at your current settings and compare with what in the Informix manual, pay attention to TAPEDEV, make sure the directory exists. You can always kill the backup process with: kill -9 <pid that you get from ps command> Let's go GreenThis email contains 100% recycled electrons. From: BENJI LONG <ruggedmouse@hotmail.com> To: ids@iiug.org Sent: Wednesday, September 13, 2017 11:13 AM Subject: Backup Help [39823] Hi, I have a test Informix instance setup. I am testing doing a level 0 backup. The database is very small, yet it seems to be running for over an hour now. When I run onstat -m, I see it says Level0 Archive started on rootdbs, logdbs, datadbs I thought it must have completed so then I ran a command to try a level 1 backup, and it is telling me Archive failed - Another archive is already in progress on DBspace rootdbs. So this is a small database with literally nothing hardly in it. The size of a backup I took back when I first made this in April was 20 MB I believe. What could be holding that up? I was expecting the backup to take a couple of seconds at the most... Is there any troubleshooting I can do to find out what is going on or what actually is running? I hit cntrol-c to get out of it, but it is still running the archive in the background. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

Hi Mike, no i am using ontape. I ran: ontape -s -L 0 -t STDIO >/opt/informix/full_backup/mybackup_archive That is still running I believe after 2 hours! How can I kill it? I see when I run ls -lh that the file has been sitting at 32K for a long time, and the one I took in April was 20M, so I assume this somehow got hung up

Thanks Kern. I will try to kill the process

This might be the issue right here. My TAPEDEV is set to /opt/informix/full_backup/old_archive and today I tried to rename the backup with my intake command, but I thought STDIO overrode this TAPEDEV command???

can you post an 'onstat -g ath', please? From: "BENJI LONG" <ruggedmouse@hotmail.com> To: ids@iiug.org Date: 09/13/2017 05:37 PM Subject: Re: RE: Backup Help [39826] Sent by: ids-bounces@iiug.org Hi Mike, no i am using ontape. I ran: ontape -s -L 0 -t STDIO >/opt/informix/full_backup/mybackup_archive That is still running I believe after 2 hours! How can I kill it? I see when I run ls -lh that the file has been sitting at 32K for a long time, and the one I took in April was 20M, so I assume this somehow got hung up ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

Hi, I need to figure out how to copy and paste from the shell in my VM, but in the meantime, I see the last two lines might mean something cond wait block 1 cpu* dbWorker3 cold wait block 1 cpu ontape

When I run ps, it only has two things running bash ps I was expecting to see ontape??? arghhhhhh...

blocked checkpoint? -> onstat - Maybe your logical logs are full? -> onstat -l | grep C- From: "BENJI LONG" <ruggedmouse@hotmail.com> To: ids@iiug.org Date: 09/13/2017 05:52 PM Subject: Re: RE: Backup Help [39830] Sent by: ids-bounces@iiug.org Hi, I need to figure out how to copy and paste from the shell in my VM, but in the meantime, I see the last two lines might mean something cond wait block 1 cpu* dbWorker3 cold wait block 1 cpu ontape ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

since this is your test environment, bounce your instance! retry Let's go GreenThis email contains 100% recycled electrons. From: BENJI LONG <ruggedmouse@hotmail.com> To: ids@iiug.org Sent: Wednesday, September 13, 2017 11:55 AM Subject: Re: Backup Help [39831] When I run ps, it only has two things running bash ps I was expecting to see ontape??? arghhhhhh... ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

ps -e From: "BENJI LONG" <ruggedmouse@hotmail.com> To: ids@iiug.org Date: 09/13/2017 05:55 PM Subject: Re: Backup Help [39831] Sent by: ids-bounces@iiug.org When I run ps, it only has two things running bash ps I was expecting to see ontape??? arghhhhhh... ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

When I ran onstat -, it says Blocked: LAST_LOG_RESERVED4BACKUP When I ran onstat -l grep C-, it gives me U---C-L

When I ran onstat -, it says Blocked: LAST_LOG_RESERVED4BACKUP When I ran onstat -l grep C-, it gives me U---C-L and the "C" is RED coloured.

ps -e shows a bunch of stuff there now. What should I be looking for so that I can kill it? I think the logs are full. I will go back and read up on how to get those to backup automatically after they become full. I thought I had that set when I installed the instance.

Never thought of that. I will do that if I can't work my way out of this one. Learning a lot here, today on this issue.

When you run "onstat -l" do you see most of the logs as "U-B----" or just "U-----"? How about the log AFTER the current log, shown by the "C"? Have you checked that you have free space in the filesystem where you are trying to write out the backup, /opt/informix/full_backup/mybackup_archive? Also, if INFORMIXDIR=/opt/informix, then I would suggest putting backups on another filesystem - just good practice. To find the ontape unix process, run: ps -ef | grep ontape -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of BENJI LONG Sent: Wednesday, September 13, 2017 10:11 AM To: ids@iiug.org Subject: Re: Backup Help [39838] ps -e shows a bunch of stuff there now. What should I be looking for so that I can kill it? I think the logs are full. I will go back and read up on how to get those to backup automatically after they become full. I thought I had that set when I installed the instance. **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum.

Hi, so onstat -l gives six files. One is U---C-L and the other five are U------ ps -ef | grep ontape gives informix 1897 1677 0 12:24 tty1 I will see if there is any space left. I'm new at Linux so need to look up how to do that as well...ugh

I presume you want to perform logical log backup in a directory. Set the LTAPEDEV parameter to the destination dir and check if the informix user is able to read and write to this dir. (as root): chmod 770 dir chown informix:informix dir perform ontape -a -dmanually and your logs should be written in this dir one by one, with a numeric increasing extension. onstat -l should now mark the backed up logs with a "B" flag. The engine needs backup to be able to restore to a consistent state any time after the last full backup. If that is not necessary for your setup, you could skip the log backup process and set the LTAPEDEV to /dev/null, but that does not solve your situation now, because the engine does only throw away the logs automatically when this parameter is detected at startup time. Thus, a ontape or onbar backup of logs will be needed in any case to clear your situation. For setting up an automatic backup, check the alarm program (in onconfig, normally $INFORMIXDIR/etc/alarmprogram.sh) edit the alarmprogram to react on a "log full" event by calling "ontape -a" (or a shell script which will do that and monitor the output, to send you a mail if it fails). ADMINEMAIL=admin@yourdomain.xx BACKUPLOGS=Y (Standard is N) BACKUPCMD="/opt/informix/bin/ontape -a -d" (or a shell script which will give your more control). the logs should be backed up automatically now. Pls keep in mind the management of the files is up to you (onbar has a backup manager which keeps track on which files are needed). So you have to apply a logic for your full backups, and find out yourself which logs are not needed any more. Best regards, Marcus Haarmann Von: "BENJI LONG" <ruggedmouse@hotmail.com> An: "ids" <ids@iiug.org> Gesendet: Mittwoch, 13. September 2017 19:25:56 Betreff: Re: RE: Backup Help [39844] Hi, so onstat -l gives six files. One is U---C-L and the other five are U------ ps -ef | grep ontape gives informix 1897 1677 0 12:24 tty1 I will see if there is any space left. I'm new at Linux so need to look up how to do that as well...ugh ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

Hi, the LTAPEDEV parameter was set to the proper place, and when I installed, I must have given informix and group informix full permissions on that directory. In alarmprogram.sh, I see I had an error. I had BACKUP_CMD = 'ontape -a -d" (single quote instead of double) So I'm hoping that is why the logs weren't working. They went right in when I ran the manual command you gave me. I will now rerun my backup and hope this is all working now. Question for you: On our production server, in alarm program.sh, they have BACKLOGS set to N. I couldn't figure this out when we took it over. So from what I get from your response, they must have a script that is doing the log backups, but I cannot find it anywhere. I found a directory called scripts and I don't see anything in there that looks like it has the word log in the filename. I found other .sh files but nothing looks like it has the word log in it which I would think the developer would have named it log something or other. Is there any usual place something like this would be that I could look, or any way I can figure out what calls it so I can find it somehow?

On your production server, confirm which script is called when a log backup event is called - perhaps it's using something other than alarmprogram.sh. Confirm this by checking the value of ALARMPROGRAM in the onconfig. Also confirm that LTAPEDEV on production is NOT set to /dev/null. if it is, then the logs are not actually being backed up anywhere. -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of BENJI LONG Sent: Wednesday, September 13, 2017 1:01 PM To: ids@iiug.org Subject: Re: Backup Help [39847] Hi, the LTAPEDEV parameter was set to the proper place, and when I installed, I must have given informix and group informix full permissions on that directory. In alarmprogram.sh, I see I had an error. I had BACKUP_CMD = 'ontape -a -d" (single quote instead of double) So I'm hoping that is why the logs weren't working. They went right in when I ran the manual command you gave me. I will now rerun my backup and hope this is all working now. Question for you: On our production server, in alarm program.sh, they have BACKLOGS set to N. I couldn't figure this out when we took it over. So from what I get from your response, they must have a script that is doing the log backups, but I cannot find it anywhere. I found a directory called scripts and I don't see anything in there that looks like it has the word log in the filename. I found other .sh files but nothing looks like it has the word log in it which I would think the developer would have named it log something or other. Is there any usual place something like this would be that I could look, or any way I can figure out what calls it so I can find it somehow? **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum.

I appreciate so much you guys sticking with this... ALARMPROGRAM on our PROD server is set to /opt/informix/11.50/etc/log_full.sh That file has a line down about 5 lines which I believe does the magic. BACKUP_CMD="ontape -a" I know the logs are put into a folder, and there is a cronjob that runs to clean old ones out.

I appreciate so much you guys sticking with this... ALARMPROGRAM on our PROD server is set to /opt/informix/11.50/etc/log_full.sh That file has a line down about 5 lines which I believe does the magic. BACKUP_CMD="ontape -a" I know the logs are put into a folder, and there is a cronjob that runs to clean old ones out.

onmode -wf LTAPEDEV=/dev/null From: "BENJI LONG" <ruggedmouse@hotmail.com> To: ids@iiug.org Date: 09/13/2017 06:17 PM Subject: Re: Backup Help [39838] Sent by: ids-bounces@iiug.org ps -e shows a bunch of stuff there now. What should I be looking for so that I can kill it? I think the logs are full. I will go back and read up on how to get those to backup automatically after they become full. I thought I had that set when I installed the instance. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.