IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener
Posted in 2021
Hrvoje Zokovic's 14.10.FC5 on Ubuntu 20.04 panicked at startup when the shm listener failed (assert in shm_be.c, errno 13, could not open shm file). He and Jon Ritson found the cause was kernel setting fs.protected_regular=2; setting it to 0 fixed it, though a reboot reset it. Andreas Seifert advised re-applying it with sysctl -p late in boot (rc.local or a systemd unit), which worked.
Auto-generated by Claude from the posts below — may be imperfect; read the full thread.
Topics: Installation, Setup & Upgrades, Error Codes & Troubleshooting, Networking & sqlhosts Configuration, Versions, Editions & End-of-Life
Before I open PMR, does anyone have this problem:
Normal installation, DBSERVERALIAS for shm connection:
DBSERVERNAME ifxpriDBSERVERALIASES lo_ifxpri,ifxpri_shm
sqlhosts with shm record:
ifxpri onsoctcp 192.168.193.209 ifxpri
lo_ifxpri onsoctcp 127.0.0.1 lo_ifxpri
ifxpri_shm onipcshm 127.0.0.1 ifxpri_shmStarting informix (oninit -v) ends with
Initializing DR structures...succeeded
Forking 1 'soctcp' listener threads...succeeded
Forking 1 'soctcp' listener threads...succeeded
Forking 1 'ipcshm' listener threads...informix@mate2004:~/i1410fc5/etc$
And in online log I have this
14:50:39 SQL_FEAT_CTRL value set to 0x8008
14:50:39 SQL_FEAT_CTRL2 value set to 0x1010
14:50:39 SQL_DEF_CTRL value set to 0x44004b0
14:50:39 Assert Failed: shm_be.c, line 3415, thread 12, errno=13, could not open shm file.
14:50:39 IBM Informix Dynamic Server Version 14.10.FC5
14:50:39 Who: Session(6, informix@, 0, (nil))
Thread(12, sm_listen, 0, 1)
File: neterrb.c Line: 530
14:50:39 stack trace for pid 888592 written to /home/informix/i1410fc5/tmp/af.3f44e2e
14:50:39 See Also: /home/informix/i1410fc5/tmp/af.3f44e2e
14:50:47 Thread ID 12 will NOT be suspended because
it is deemed too critical to the server.
14:50:47 See Also: /home/informix/i1410fc5/tmp/af.3f44e2e, shmem.3f44e2e.0
14:50:48 Starting crash time check of:
14:50:48 1. memory block headers
14:50:48 2. stacks
14:50:48 Crash time checking found no problems
14:50:48 neterrb.c, line 530, thread 12, proc id 888592, shm_be.c, line 3415, thread 12, errno=13, could not open shm file..
14:50:48 PANIC: Attempting to bring system down
Without shm listener instance is starting as usual.
Anyone?
Hrvoje
------------------------------
Hrvoje Zokovic
------------------------------
#Informix
Ask Art K Eric Vercelletto Data Management Architect and Owner / Begooden IT Consulting KandooERP Founder and CTO IBM Champion 2013,2014,2015,2016,2017,2018,2019,2020 Tel: +33(0) 298 51 3210 Mob : +33(0)626 52 50 68 skype: begooden-it Google Hangout: eric.vercelletto@begooden-it.com Email: eric.vercelletto@begooden-it.com www : http://www.vercelletto.com www https://kandooerp.org
I haven't played with ubuntu in a while but I found this older document, 2017. https://www.techrepublic.com/article/how-to-enable-secure-shared-memory-on-ubuntu-server/ Is it possible that the newer version of Ubuntu has it enabled by default? Peer speculation on my part here. David
Error 13 is a permission error....so, worth verifying ownership on $INFORMIXDIR, and that the oninit has the correct permissions and owner.
I assume that the install was run as "root"?
Maybe run onclean - make sure no leftover processes or memory segments?
------------------------------
Mike Walker
------------------------------
Thanks David but I believe it is not reason: root@mate2004:~# grep shh /etc/fstab root@mate2004:~# i.e. there's no shm in fstab. Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
Thanks Mike, yes, normal root installation.
ipcs shows nothing:
informix@ifx-tecaj:~/i1410/bin$ ipcs
------ Message Queues --------
key msqid owner perms used-bytes messages
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
------ Semaphore Arrays --------
key semid owner perms nsems
informix@ifx-tecaj:~/i1410/bin$
And permissions (on bin directory at least) seems OK to me:
drwxr-xr-x 2 informix informix 4096 Feb 27 12:00 about_files
-rw-r--r-- 1 informix informix 577 Nov 30 23:00 about.html
-rwxr-xr-x 1 informix informix 3346896 Nov 30 23:22 archecker
-rwxr-xr-x 1 informix informix 2400712 Nov 30 23:19 blademgr
-rwxr-xr-x 1 informix informix 3495960 Nov 30 23:19 cdr
-rwxr-xr-x 1 informix informix 897096 Nov 30 23:00 chkenv
drwxr-xr-x 2 informix informix 4096 Feb 27 12:00 configuration
-rwxr-xr-x 1 informix informix 897440 Nov 30 23:00 crtcmap
-rwxr-xr-x 1 informix informix 61346 Nov 30 23:19 db2dbgm.jar
-rwxr-xr-x 1 informix informix 2366624 Nov 30 23:19 dbaccess
-rwxr-xr-x 2 informix informix 7847 Nov 30 23:19 dbaccessdemo
-rwxr-xr-x 2 informix informix 7847 Nov 30 23:19 dbaccessdemo7
-rwxr-xr-x 2 informix informix 6443 Nov 30 23:19 dbaccessdemo9
-rwxr-xr-x 2 informix informix 6443 Nov 30 23:19 dbaccessdemo_ud
-rwxr-xr-x 1 informix informix 2236536 Nov 30 23:19 dbexport
-rwxr-xr-x 1 informix informix 1888592 Nov 30 23:19 dbimport
-rwxr-xr-x 1 informix informix 1789008 Nov 30 23:19 dbload
-rwxr-xr-x 1 informix informix 2235944 Nov 30 23:19 dbschema
-rwxr-xr-x 1 informix informix 1985432 Nov 30 23:19 drdaprint
-rwxr-xr-x 1 informix informix 56395 Nov 30 23:19 filtersym.sh
-rwxr-xr-x 1 informix informix 5586 Nov 30 23:00 finderr
-rwxr-xr-x 1 informix informix 1041512 Nov 30 23:00 GenMacKey
-rwxr-xr-x 1 informix informix 1020480 Nov 30 23:00 genoncfg
-rwxr-xr-x 1 informix informix 9721 Nov 30 23:00 glfiles
-rwxr--r-- 1 informix informix 6357 Nov 30 23:19 hdrmkpri.sh
-rwxr--r-- 1 informix informix 5178 Nov 30 23:19 hdrmksec.sh
-rwxr-xr-- 1 informix informix 1373696 Nov 30 23:19 idsd
-rwsr-s--- 1 root informix 8272 Nov 30 23:19 ifmxgcore
-r-xr-xr-x 2 informix informix 21183 Nov 30 23:19 ifmx_security.sh
-rwxr-xr-x 1 informix informix 6276365 Nov 30 23:00 ifxbkpcloud.jar
-rwxr-xr-x 1 informix informix 2643808 Nov 30 23:19 ifxclone
-rwxr-xr-x 1 informix informix 1084960 Nov 30 23:19 ifxcollect
-rwxr-xr-x 1 informix informix 1660736 Nov 30 23:00 ifxdeploy
-rwxr-xr-x 1 informix informix 71162 Nov 30 23:00 ifxdeployassist
-rwxr-xr-x 1 informix informix 1750944 Nov 30 23:19 ifxguard
-rwxr-xr-x 1 informix informix 24392 Nov 30 23:21 ifxpipecat
-rwxr-xr-x 1 informix informix 126904 Nov 30 23:21 infoshp
-rwxr-xr-x 1 informix informix 892928 Nov 30 23:00 infxmsg
-rwxr-xr-x 1 informix informix 4500024 Nov 30 23:19 ipload
-rwxr-xr-x 1 informix informix 9999507 Nov 30 23:00 jsonListener.jar
-rwxr-xr-x 1 informix informix 335360 Nov 30 23:00 libcairo-swt.so
-rwxr-xr-x 1 informix informix 142464 Nov 30 23:21 loadshp
-rwxr-xr-x 1 informix informix 892880 Nov 30 23:00 msgfile
-rwsr-sr-x 1 root informix 1810032 Nov 30 23:19 onaudit
-rwxr-xr-x 1 informix informix 3998 Nov 30 23:22 onbar
-rwsr-sr-x 1 root informix 4774392 Nov 30 23:22 onbar_d
-rwxr-sr-x 1 informix informix 3736488 Nov 30 23:19 oncheck
-rwsr-sr-x 1 root informix 1779288 Nov 30 23:19 onclean
-rwxr-sr-x 1 informix informix 1335280 Nov 30 23:19 onconfig_diff
-rwsr-sr-x 1 root informix 3912680 Nov 30 23:22 ondblog
-rwxr-xr-x 1 informix informix 11498 Nov 30 23:21 ondwachk
-rwsr-sr-x 1 root informix 2841664 Nov 30 23:19 onedcu
-rwxr-sr-x 1 informix informix 1010904 Nov 30 23:19 onedpu
-rwsr-sr-- 1 root informix 27470872 Nov 30 23:19 oninit
-rwxr-xr-x 1 informix informix 988048 Nov 30 23:00 onkstash
-rwsr-sr-- 1 informix informix 1934536 Nov 30 23:19 onkstore
-rwxr-sr-x 1 informix informix 2267576 Nov 30 23:19 onload
-rwxr-sr-x 1 informix informix 4150648 Nov 30 23:19 onlog
-rwsr-sr-x 1 root informix 2762536 Nov 30 23:19 onmode
-rwxr-sr-x 1 informix informix 3054096 Nov 30 23:19 onparams
-rwxr-xr-x 1 informix informix 1012080 Nov 30 23:19 onpassword
-rwxr-xr-x 1 informix informix 2594776 Nov 30 23:19 onperf
-rwxr-xr-x 1 informix informix 2488528 Nov 30 23:19 onpladm
-rwxr-sr-x 1 informix informix 4016128 Nov 30 23:19 onpload
-rwxr-xr-x 1 informix informix 3043312 Nov 30 23:22 onpsm
-rwxr-xr-x 1 informix informix 3342768 Nov 30 23:22 onrestorept
-rwxr-xr-x 1 informix informix 1044664 Nov 30 23:19 onsecurity
-rwsr-sr-x 1 root informix 1092648 Nov 30 23:19 onshowaudit
-rwxr-xr-- 1 informix informix 4096 Nov 30 23:19 onshutdown.sh
-rwsr-sr-x 1 root informix 4817496 Nov 30 23:22 onsmsync
-rwsr-s--- 1 root informix 4149896 Nov 30 23:21 onsnmp
-rwxr-sr-x 1 informix informix 3053760 Nov 30 23:19 onspaces
-rwsr-s--- 1 root informix 1068280 Nov 30 23:21 onsrvapd
-rwxr-sr-x 1 informix informix 8171568 Nov 30 23:19 onstat
-rwsr-sr-x 1 root informix 4214416 Nov 30 23:19 ontape
-rwxr-sr-x 1 informix informix 2267584 Nov 30 23:19 onunload
drwxr-xr-x 3 informix informix 4096 Feb 27 12:00 plugins
-rwxr-xr-x 1 informix informix 4989 Nov 30 23:00 rofferr
-rwxr-x--- 1 informix informix 614384 Nov 30 23:21 snmpdm
-r-xr-xr-x 1 informix informix 939288 Nov 30 23:22 txbsapswd
-rwxr-xr-x 1 informix informix 137112 Nov 30 23:21 unloadshp
-rwxr-xr-x 1 informix informix 2694952 Nov 30 23:19 xtrace
-rwxr-sr-x 1 informix informix 2844192 Nov 30 23:19 xtree
Hrvoje
------------------------------
Hrvoje Zokovic
------------------------------
David, Mike, thank you - you pointed me to right google direction. It seems that you must sudo sysctl fs.protected_regular=0 Here's explanation (from https://github.com/sysrepo/sysrepo/issues/2080): This is not a Ubuntu-specific issue. It's caused by recent enough kernel (4.19+) and systemd (>= 241). The /dev/shm directory has the sticky bit set, this kernel patch added support for the fs.protected_regular sysctl knob, and this knob in turn gets enabled via systemd . As a result, one cannot use O_CREAT on "somebody else's files" on those world-writable sticky directories. That is NOT mentioned in documentation :) Regards, Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
Hrvoje, Your grep was for shh not shm? Was that what you meant?
Yes, typo, but for shm it is the same: informix@ifx-tecaj:~/i1410$ grep shm /etc/fstab informix@ifx-tecaj:~/i1410$ Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
Yeah saw your other email after I sent this one. Glad you got it resolved.
I raised this internally end of last year ... (i.e. there is an internal defect) "Root cause is that the default kernel setting fs.protected_regular is 2 On Ubuntu 20.04 the default setting is 2; setting this to 0 (not 1 or 2) and the instance starts successfully." Pending release notes / machine notes update. JJ ------------------------------ Jon Ritson ------------------------------
13 is perms – something weird under SE Linux ? I have 14.10.FC5 running on Ubuntu.latest Cheers Paul
Paul, SHM listener is problem. Do you have SHM listener started? Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
Hi JJ, yes I found that solution. The same problem is in on 14.10.FC4W1 as well (also for all v14 back to 14.10.fc1 but those were released before Apr 2020). V12 shm listener works fine (on ubuntu 20.04) What is best solution to have permanent fs.protected_regular=0 ? Regards Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
"Best" is subjective :) It works when fs.protected_regular=0 For further "discussion" ... https://askubuntu.com/questions/1250974/user-root-cant-write-to-file-in-tmp-owned-by-someone-else-in-20-04-but-can-in ------------------------------ Jon Ritson ------------------------------
Does the service name you are using for the SHM connection ( ifxpri_shm) exist in /etc/services? If so, try something completely dummy that doesn't like 'kagel' or 'dummy'. I have seen issues when the service name exists. ------------------------------ Art S. Kagel, President and Principal Consultant ASK Database Management Corp. www.askdbmgt.com ------------------------------
I'm afraid it is not working on 20.04: informix@mate2004:~/i1410fc5$ sudo sysctl fs.protected_regular [sudo] password for informix: fs.protected_regular = 2 informix@mate2004:~/i1410fc5$ sudo sysctl fs.protected_regular=0 fs.protected_regular = 0 informix@mate2004:~/i1410fc5$ sudo sysctl fs.protected_regular fs.protected_regular = 0 informix@mate2004:~/i1410fc5$ sudo reboot ... informix@mate2004:~$ sudo sysctl fs.protected_regular [sudo] password for informix: fs.protected_regular = 2 informix@mate2004:~$ Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
Art, there are only records added by installer in /etc/services: informix@mate2004:~$ grep ifxpri /etc/services ifxpri 9088/tcp lo_ifxpri 3572/tcp ifxpri_json 31495/tcp #JSON listener for ifxpri informix@mate2004:~$ Obvious it is kernel hardening repercussion. Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
This was sent a few days ago, no idea why IBM held it
This is a problem of Ubuntu 20.04. After the kernel variables are set, kernel modules are still loaded which change settings again. The problem can be solved by calling a sysctl -p again in a later rc file, e.g. in rc.local if rc.local is active. You can read more about this here: https://bugs.launchpad.net/ubuntu/+source/procps/+bug/50093 ------------------------------ Andreas Seifert CURSOR Software AG http://www.admin-scout.com ------------------------------
Hi Andreas, thank you for info and solution - I did that with systemd service Regards Hrvoje ------------------------------ Hrvoje Zokovic ------------------------------
The problem is a systemd problem. You can help yourself with rc.local or your own small systemd unit. ------------------------------ Andreas Seifert CURSOR Software AG http://www.admin-scout.com ------------------------------
Replied 2021-03-02 14:47
Related threads
- unable to run restore
- onstat shared memory error
- 951: Incorrect password error after Big Fix.
- Assert Failed: Physical log recovery error
- Shared memory segments are not removed with the onmode -ky command