Informix Error -12

Operating-System Meaning

errno 12 is ENOMEM — Cannot allocate memory. The official text's "Not enough core" is the historic wording; "core" meant physical memory in the era the catalogue was written, and the official note that it "probably refers to data space in memory" is a fair modern gloss.

An allocation was refused. That can be malloc(), a shared-memory attach, an mmap(), or a fork() that could not get an address space.

The first question is always the same:

Was it the operating system that refused, or Informix's own configured ceiling?

Those look similar from a distance and have entirely different remedies. An engine that has reached SHMTOTAL will refuse to grow while the host has ample free memory; a host under genuine pressure will refuse Informix regardless of what SHMTOTAL says.

A third possibility is worth ruling out early because it is not this error at all: if the Linux OOM killer terminated a process, that arrives as a SIGKILL, not as ENOMEM. A process that vanished without an error is an OOM kill; a process still running and reporting -12 was refused an allocation. dmesg distinguishes them immediately.

What This Means in Informix

Shared memory

The engine's shared memory is the usual context. Informix allocates a resident segment, a virtual segment that grows, and message segments; a refusal at any of those points surfaces as -12.

Both the OS and the engine impose limits:

Layer	Controls
Kernel	kernel.shmmax (largest single segment), shmall (total pages), shmmni (number of segments)
Engine	SHMTOTAL (total Informix may use; 0 means unlimited), SHMVIRTSIZE (initial virtual segment), SHMADD (growth increment), EXTSHMADD
Process	ulimit -v / RLIMIT_AS, ulimit -d, RLIMIT_MEMLOCK when RESIDENT is set
Container	cgroup memory.max

A segment larger than shmmax fails however much memory is free. Equally, SHMADD asking for an increment that cannot be satisfied fails even when the total is nowhere near SHMTOTAL.

The RESIDENT trap

If RESIDENT is set, the engine asks the kernel to lock shared memory into physical RAM. That allocation is bounded by RLIMIT_MEMLOCK for the installation owner, which on many systems defaults to a very small value. The result is a server that starts fine with RESIDENT off and fails with it on, while free shows plenty of memory — because the constraint is lockable memory, not memory.

Other contexts

• fork() or exec() refused — a backup wrapper, ALARMPROGRAM, or a SYSTEM call unable to get an address space
• Client-side malloc() failures in ESQL/C or a 4GL runner, particularly 32-bit binaries against their address-space ceiling
• onbar and storage-manager processes unable to allocate
• Large sorts, hash joins or index builds demanding more than the memory manager can provide
• PDQ driving concurrent memory demand well above the steady state
• Hugepages configured but not available, so an allocation that expects them fails

Common Causes

1. SHMTOTAL reached — the engine's own ceiling, with the host still healthy.
2. kernel.shmmax too small for the segment being requested.
3. RLIMIT_MEMLOCK too low with RESIDENT set.
4. The host is genuinely short of memory, often with swap exhausted too.
5. A cgroup memory limit in a container, far below the host's capacity.
6. ulimit -v set for the installation owner, capping address space.
7. Overcommit disabled or restricted (vm.overcommit_memory=2), so allocations are refused that would previously have succeeded.
8. 32-bit client binaries reaching their address-space limit regardless of host memory.
9. Hugepages reserved but insufficient, or requested and not configured.

Diagnostic Checks

Separate engine ceiling from host pressure first. Compare what Informix is using against what it is allowed, then against what the host has:

onstat -g seg                 # segments, sizes, and what is in use
onstat -g mem                 # memory pools
onstat -g mgm                 # memory grant manager — PDQ demand
grep -nE 'SHMTOTAL|SHMVIRTSIZE|SHMADD|EXTSHMADD|RESIDENT|SHMBASE' \
     "$INFORMIXDIR/etc/$ONCONFIG"

If the total in onstat -g seg is at or near SHMTOTAL, that is your answer and the host's free memory is irrelevant.

Then the host:

free -m
cat /proc/meminfo | grep -iE 'memtotal|memfree|memavailable|swap|commit|huge'
vmstat 1 5

Kernel shared-memory limits, and what is currently allocated:

ipcs -l                       # limits
ipcs -m                       # segments in use
sysctl kernel.shmmax kernel.shmall kernel.shmmni

shmmax smaller than the segment Informix is trying to create is a common and easily-missed cause — compare it against SHMVIRTSIZE and SHMADD, not against total memory.

Rule out an OOM kill, which is a different failure:

dmesg -T | grep -iE 'out of memory|oom-killer|killed process'
journalctl -k --since '1 day ago' | grep -i oom
grep -i 'killed process' /var/log/messages | tail

If Informix processes appear there, the engine was killed rather than refused, and the investigation is about host memory pressure, not about -12.

If an assert failure was generated, read the signal number — it is often the fastest route to the answer. Look in DUMPDIR (defaulting to $INFORMIXDIR/tmp) for af.* files and check the message log around the same timestamp:

grep -n DUMPDIR "$INFORMIXDIR/etc/$ONCONFIG"
ls -lt "${DUMPDIR:-$INFORMIXDIR/tmp}"/af.* 2>/dev/null | head
grep -iE 'signal|assert|af\.' "$INFORMIXDIR/tmp/online.log" | tail -20
head -40 "${DUMPDIR:-$INFORMIXDIR/tmp}"/af.<id>

A recorded signal 9 points to an OOM kill far more often than to anything else. SIGKILL cannot be caught, blocked or handled, so the engine can never have raised it against itself — something outside the process did. On a Linux host under memory pressure that is almost always the OOM killer; the only common alternative is an administrator or a script issuing kill -9.

That distinction is worth making early, because an assert failure naturally reads as an internal engine fault and sends the investigation toward the database when the cause is on the host. Signals that the engine did generate for itself — a SIGSEGV or SIGBUS recorded in an AF — are the genuinely internal cases.

# did anyone kill it deliberately?
grep -rn 'kill -9\|kill -KILL' /informix/scripts/ /etc/cron* 2>/dev/null
ausearch -m all -sc kill 2>/dev/null | tail -20

Check the limits applying to the installation owner, including the locked-memory one:

ulimit -v ; ulimit -d ; ulimit -l
grep -rnE 'memlock|as\b|data' /etc/security/limits.conf /etc/security/limits.d/ 2>/dev/null
systemctl show <informix-unit> -p LimitMEMLOCK -p LimitAS -p MemoryMax -p MemoryCurrent

In a container:

cat /sys/fs/cgroup/memory.max /sys/fs/cgroup/memory.current   # cgroup v2
cat /sys/fs/cgroup/memory/memory.limit_in_bytes               # cgroup v1

Overcommit policy, which changes whether large allocations are permitted at all:

sysctl vm.overcommit_memory vm.overcommit_ratio
cat /proc/meminfo | grep -i commit

Hugepages, if they are in use:

grep -i huge /proc/meminfo
sysctl vm.nr_hugepages

Solutions / Resolution

1. Establish which ceiling was hit before changing anything. onstat -g seg against SHMTOTAL against free -m answers it in three commands, and the remedies do not overlap.
2. If SHMTOTAL is the limit, raise it or set it to 0 for unlimited — but only after confirming the host can actually back the memory. SHMTOTAL exists to stop the engine exhausting the machine, and removing it because the engine hit it can turn a contained failure into an OOM kill.
3. If shmmax is the limit, raise it to comfortably exceed the largest segment Informix will request:

   sysctl -w kernel.shmmax=<bytes>
   # persist in /etc/sysctl.d/
   

4. If RESIDENT is set and memlock is the constraint, raise RLIMIT_MEMLOCK for the installation owner — and on systemd, LimitMEMLOCK on the unit as well, which overrides limits.conf:

   ifxprod  soft  memlock  unlimited
   ifxprod  hard  memlock  unlimited
   

5. If the host is genuinely short, reduce demand rather than raising limits: buffer pool sizing, DS_TOTAL_MEMORY and PDQ, session counts, and any sort-heavy workload are the levers. Adding a limit increase to a host that is already swapping makes matters worse.
6. If a cgroup is the limit, change it in the pod or container spec. Changing it inside a running container will not survive a restart.
7. For 32-bit clients, the fix is a 64-bit build. No host-side change raises a 32-bit address-space ceiling.
8. Review overcommit settings before assuming a shortage — vm.overcommit_memory=2 refuses allocations that a default-configured host would have granted.
9. Re-test the specific operation, and watch onstat -g seg while it runs rather than checking afterwards.

Examples

The engine's ceiling, not the host's

$ free -m
               total        used        free      available
Mem:          128000       41200       78000          84000

$ onstat -g seg | tail -3
Total:      -         -        8388608   8384512      -

$ grep SHMTOTAL "$INFORMIXDIR/etc/$ONCONFIG"
SHMTOTAL 8388608

84 GB available on the host and the engine will not grow by another page, because it is exactly at its configured total. Nothing on the host needs changing; SHMTOTAL does — after deciding what the right ceiling is, rather than simply removing it.

shmmax smaller than the segment being requested

$ sysctl kernel.shmmax
kernel.shmmax = 4294967296

$ grep -E 'SHMVIRTSIZE|SHMADD' "$INFORMIXDIR/etc/$ONCONFIG"
SHMVIRTSIZE 8388608
SHMADD      1048576

SHMVIRTSIZE is expressed in KB, so the requested virtual segment is 8 GB against a 4 GB per-segment kernel limit. The allocation fails at startup regardless of the 128 GB in the machine. This one is easy to miss because every "how much memory does this box have" check comes back healthy.

RESIDENT against a small memlock limit

$ grep RESIDENT "$INFORMIXDIR/etc/$ONCONFIG"
RESIDENT 1

$ ulimit -l
64

$ systemctl show informix -p LimitMEMLOCK
LimitMEMLOCK=65536

64 KB of lockable memory. The engine starts cleanly with RESIDENT 0 and fails with it set, while free reports tens of gigabytes available — because the constraint is on locking memory, not on having it.

Not this error at all — an AF recording signal 9

$ ls -lt "$INFORMIXDIR/tmp"/af.* | head -2
-rw-rw---- 1 ifxprod ifxprod 284112 Sep  9 04:11 /home/ifxprod/informix/tmp/af.5a3f1c

$ grep -iE 'signal|caught' /home/ifxprod/informix/tmp/af.5a3f1c | head -3
Signal:      9

$ dmesg -T | grep -i 'killed process'
[Tue Sep  9 04:11:52 2026] Out of memory: Killed process 3312 (oninit) \
    total-vm:74203392kB, anon-rss:68112384kB

The assert failure looks like an engine fault and is not one. Signal 9 cannot be raised by the process against itself, so it came from outside — and the dmesg line confirms which outside. The engine did not receive an error at all; it was terminated.

This is worth recognising quickly, because an AF naturally draws the investigation into the database. Here it belongs with host memory pressure and whatever else on the machine is competing for it. Had the AF recorded a SIGSEGV or SIGBUS instead, the opposite would be true.

Platform Note

errno 12 is ENOMEM on Linux, AIX, Solaris, HP-UX and the BSD-derived systems — stable, so the number is reliable.

Almost everything else here is platform-specific, and the shared-memory tunables in particular:

Task	Linux	Solaris	AIX
Shared memory limits	kernel.shmmax, shmall, shmmni	project.max-shm-memory (projmod)	largely dynamic; vmo
View limits	ipcs -l	prctl -n project.max-shm-memory -i project <id>	ipcs -m, lsattr -El sys0
Memory tuning	sysctl vm.*	/etc/system (legacy), projects	vmo -a
Lockable memory	RLIMIT_MEMLOCK, LimitMEMLOCK	project.max-locked-memory	v_pinshm, maxpin%
Large pages	vm.nr_hugepages	ISM / DISM (automatic)	vmo -o lgpg_regions

On Solaris the System V /etc/system tunables are long obsolete in favour of the project framework, and a process running in an unexpected project is a recurring cause of surprise limits. On AIX, pinned-memory limits (maxpin%) are the analogue of the RESIDENT/memlock interaction described above and are worth checking in the same circumstances.

Related Errors / Related Topics

• -11 — Resource temporarily unavailable. The other exhaustion error; on a host under real pressure the two typically arrive together, one for threads and one for memory.
• -28 — No space left on device. Worth distinguishing when the failing operation involves temporary space, since a large sort can exhaust either.
• -24 — Too many open files, the third member of the resource-exhaustion set.
• -7 — Argument list too long, which also manifests during process creation.

An -12 on a production instance is a capacity finding, not a one-off. Shared-memory use against SHMTOTAL, host memory against swap activity, and PDQ memory grants are the three worth watching continuously — all give warning well before an allocation is refused.