Informix Error -31

Operating-System Meaning

errno 31 is EMLINK — Too many links. It is about hard links, and it is one of the most frequently misread errors in this range because three of its neighbours sound like the same thing and are not:

Error	Symbol	What is actually exhausted
-31	EMLINK	Hard links to one inode, or subdirectories in one directory
-24	EMFILE	File descriptors in one process
-23	ENFILE	File descriptors system-wide
(40 on Linux)	ELOOP	Levels of symbolic link traversal

If someone reports "too many links" and the conversation is about symlinks, open files, or a chain of -> in ls -l, this is not the error they have.

EMLINK is returned by exactly two operations:

• link() — creating a hard link to a file that has already reached LINK_MAX.
• mkdir() and rename() — creating or moving a directory into a parent whose link count is at its limit.

The second case is the one that occurs in practice. The arithmetic behind it tells you how close you are before anything fails.

The subdirectory arithmetic

Every directory's link count is:

2 + (number of subdirectories)

One link for its own name in the parent, one for its own ., and one for the .. entry inside each child. So ls -ld on a directory tells you how many subdirectories it holds without counting anything:

$ ls -ld /informix/backups
drwxr-x--- 4021 ifxprod ifxprod 131072 Sep 11 02:00 /informix/backups
             ^^^^ 4019 subdirectories

On ext3, and on ext4 without the dir_nlink feature, that count is stored in a field that runs out — historically at 32000 subdirectories, later 65000. The 32001st mkdir in that directory returns EMLINK while the filesystem still has plenty of free space and free inodes.

Modern ext4 with dir_nlink removes the limit by freezing the count at 1 once it is exceeded, and XFS never had a practical one. So this is increasingly a legacy-filesystem error — which is precisely why it tends to appear on the oldest and least-touched server in the estate.

Hard links to a regular file have a separate ceiling (65000 on ext4), and reaching it requires deliberate effort. Ask the filesystem rather than assuming either number:

getconf LINK_MAX /informix/backups

What This Means in Informix

The engine's own operation does not create hard links, so a -31 reaching the database server is nearly always produced by something around the instance rather than inside it. Look away from the engine and towards whatever else writes into the same filesystem.

Plausible sources:

• A backup target with one subdirectory per run. ontape, onbar, or a wrapper writing to /backups/<instance>/<date>/ accumulates a subdirectory per backup. At one per hour that is 8,760 a year — comfortable, until nothing ever prunes it and the filesystem is ext3.
• Logical-log backups organised by subdirectory rather than by filename. A busy instance turns over logs quickly and the count climbs far faster than a daily scheme suggests.
• A rotation scheme built on hard links. rsync --link-dest, cp -al, and the backup tools built on them deduplicate by hard-linking unchanged files between snapshots. Every retained snapshot adds a link to the same inode, and a file that rarely changes — an onconfig, a static chunk, a script — accrues one link per snapshot held. This is the realistic route to exhausting the file limit rather than the directory one.
• External-table or load/unload staging that creates a directory per job.
• $INFORMIXDIR/tmp or a DUMPDIR that has never been cleared, where an assert-failure or diagnostic scheme creates subdirectories.

Note what is not on that list: the number of chunks, dbspaces, logical logs, or open files an instance has. None of those approach any link limit, and none of them are what -31 is reporting.

Because the engine surfaces the OS error without necessarily knowing which path produced it, the message log entry may name the operation and not the directory. The filesystem itself is usually the faster source — the offending directory is visible in a single find, as below.

Common Causes

1. A directory holding tens of thousands of subdirectories on ext3 or on ext4 without dir_nlink — a backup, log, or staging tree that grows and is never pruned.
2. A snapshot-style backup using hard links, where one inode has accumulated a link per retained snapshot.
3. A retention policy that was never implemented, or a cleanup job that has been failing silently for long enough that nobody noticed.
4. A filesystem carried forward from an old build, still ext3, with the growth pattern of a modern workload on it.
5. A restore or migration that recreated a tree whose original structure was flatter than the destination filesystem can hold.
6. rename() moving a directory into a parent that is already at its limit.

Diagnostic Checks

Find the directory. The link count is the whole diagnosis, and one command finds any directory anywhere near a limit:

find /informix -xdev -type d -links +10000 -printf '%n\t%p\n' 2>/dev/null | sort -rn
find / -xdev -type d -links +30000 -printf '%n\t%p\n' 2>/dev/null | sort -rn

If nothing comes back, the problem is a hard-linked file rather than a directory:

find /informix -xdev -type f -links +1000 -printf '%n\t%p\n' 2>/dev/null | sort -rn | head

Check the limit that applies to that path, rather than working from a remembered number:

getconf LINK_MAX /informix/backups
stat -c '%h links  %n' /informix/backups

Establish whether the filesystem still has the old behaviour. For ext2/3/4, the dir_nlink feature is the deciding factor:

findmnt -T /informix/backups -o TARGET,SOURCE,FSTYPE,OPTIONS
tune2fs -l /dev/sdX1 | grep -iE 'filesystem features|inode count|free inodes'
dumpe2fs -h /dev/sdX1 2>/dev/null | grep -i dir_nlink

dir_nlink present means the subdirectory limit does not apply and you are looking at a hard-linked file instead. Absent means it does.

Rule out the errors it is confused with, given how similar the wording is:

ulimit -n                                  # -24 EMFILE territory
cat /proc/sys/fs/file-nr                   # -23 ENFILE territory
df -i /informix/backups                    # inode exhaustion, not this error
namei -l /path/that/failed                 # symlink chains, ELOOP territory

Confirm free space and inodes are not the actual constraint. Resolve a filesystem that is also full before drawing conclusions from EMLINK:

df -h  /informix/backups
df -i  /informix/backups

Then look at growth, because the fix depends on whether this is a one-off or a scheme that will hit the wall again next month:

ls -1 /informix/backups | wc -l
ls -ld /informix/backups
find /informix/backups -maxdepth 1 -type d -printf '%T@ %p\n' | sort -n | head -3
find /informix/backups -maxdepth 1 -type d -printf '%T@ %p\n' | sort -n | tail -3

The oldest and newest entries together give you the accumulation rate, and therefore how long the directory has been running without a retention policy.

Solutions / Resolution

1. Find the directory before changing anything. The find -links command above names it directly, and the count tells you whether you are near a limit or nowhere near it.
2. Prune, if there is a retention policy that simply is not running. This is the common case and the fastest resolution — but establish why it stopped rather than just running it by hand, or the same directory is back at the limit on the same schedule.
3. Do not delete backups to clear the error without checking what is still needed. A -31 on a backup tree is not urgent enough to justify losing a recovery point; the engine is not down because of it.
4. Restructure into tiers. A directory per day inside a directory per month inside a directory per year holds effectively unlimited history and no single directory ever approaches a link limit. This is the durable fix for a scheme that has outgrown a flat layout.
5. Consider whether subdirectories were ever the right structure. Files in a directory are not limited by link count at all. A scheme that names objects <instance>_<date>_<seq>.bak in one directory sidesteps this error entirely — though a directory with hundreds of thousands of entries has performance characteristics of its own.
6. If it is a hard-linked snapshot scheme, reduce retention or break the chain by taking a fresh full copy periodically, rather than extending the link chain indefinitely.
7. If the filesystem is ext3, plan to move it. The limit is a property of the on-disk format. A migration to XFS, or to ext4 with dir_nlink, removes the class of problem rather than deferring it — and an ext3 filesystem under a production instance in this decade is worth reviewing for reasons beyond this error.
8. Do not simply retry. Unlike a transient resource error, nothing about EMLINK resolves on its own; a retry loop will fail identically every time and will bury the original message.

Examples

A backup directory at the ext3 subdirectory limit

$ ls -ld /backups/prod_inst
drwxr-x--- 32000 ifxprod ifxprod 2621440 Sep 11 02:00 /backups/prod_inst

$ df -h /backups; df -i /backups
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1       4.0T  1.1T  2.8T  28% /backups
Filesystem      Inodes  IUsed   IFree IUse% Mounted on
/dev/sdb1     268435456 891204 267544252    1% /backups

$ findmnt -T /backups -o FSTYPE
FSTYPE
ext3

Twenty-eight percent full, one percent of inodes used, and the next mkdir fails. The constraint is the link count — 32000 is 2 plus 31998 subdirectories — and no amount of free space changes it. This is the signature of the error: a filesystem that looks entirely healthy by every metric anyone normally checks.

The same tree, restructured

$ ls -ld /backups/prod_inst /backups/prod_inst/2026 /backups/prod_inst/2026/09
drwxr-x--- 6 ifxprod ifxprod  4096 Jan  1  2022 /backups/prod_inst
drwxr-x--- 11 ifxprod ifxprod 4096 Sep  1 02:00 /backups/prod_inst/2026
drwxr-x--- 13 ifxprod ifxprod 4096 Sep 11 02:00 /backups/prod_inst/2026/09

Four years of hourly backups, and no directory in the tree holds more than a few dozen children. The deepest level would need a century of continuous accumulation to approach the old limit.

A hard-linked snapshot chain

$ find /backups -xdev -type f -links +1000 -printf '%n\t%p\n' | sort -rn | head -3
2847    /backups/snapshots/2019-03-02/informix/etc/onconfig.prod
2847    /backups/snapshots/2019-03-02/informix/etc/sqlhosts
2847    /backups/snapshots/2019-03-02/informix/bin/oninit

An rsync --link-dest scheme holding 2,847 snapshots. Files that never change carry a link per snapshot, so the static parts of $INFORMIXDIR are the first to approach the ceiling — the configuration files and binaries, not the data. The count itself is the retention count, which makes it a useful audit even when nothing is failing.

Platform Note

errno 31 is EMLINK on Linux, AIX, Solaris, HP-UX and the BSD-derived systems, so the number is reliable.

The limit behind it is not portable at all, and neither is the tooling:

Task	Linux	Solaris	AIX
Query the limit	getconf LINK_MAX <path>	getconf LINK_MAX <path>	getconf LINK_MAX <path>
Filesystem features	tune2fs -l, xfs_info	fstyp -v	lsfs, lsjfs2
Find high link counts	find -links +N -printf	find -links +N -exec ls -ld {} \;	as Solaris

getconf LINK_MAX <path> is the one portable answer and should be preferred to any remembered figure — it reports what the filesystem under that specific path will actually allow, which on a host with mixed filesystem types is not a single value.

On Solaris, UFS has a subdirectory limit of the same historical character as ext3's; ZFS does not. On AIX, JFS2 does not impose a practical one. As on Linux, encountering this error is a reasonable indicator that the filesystem predates the workload running on it.

Related Errors / Related Topics

• -24 — Too many open files. The error -31 is most often mistaken for, and an entirely different resource: descriptors held by one process rather than links to one inode.
• -28 — No space left on device. The other error a growing backup or staging tree eventually produces. Worth checking together, since a directory scheme that has exhausted its link count is usually also close to something else.
• -1 — Operation not permitted. A filesystem that does not support hard links at all — some network and gateway filesystems — refuses link() with EPERM rather than EMLINK, so a failing hard link on an unusual filesystem may arrive as -1.

Pruning the directory takes a minute. The findings behind it — a tree growing without a retention policy, on a filesystem old enough to still carry the limit — are the ones to fix while the evidence is in front of you.