Oninit® Snooper — Operations

CLI

oni_snoop --listen <bind:port> --upstream <host:port>
          [--out FILE] [--only=TOK,TOK,...] [--udt-defs FILE] [--hex N]
          [--tls-listen --tls-listen-cert FILE --tls-listen-key FILE]
          [--tls-upstream [--tls-upstream-ca FILE | --insecure-upstream]]
          [--conssl-cfg FILE] [--tls-min-version 1.0|1.1|1.2|1.3]

Flag	Effect
--listen <bind:port>	Address and TCP port the snoop binds for incoming client connections. Use 0.0.0.0:<port> to accept on every interface or <ip>:<port> to bind to a specific NIC. Required.
--upstream <host:port>	Real IDS server endpoint. The snoop opens a fresh upstream TCP connection for every client that connects in — no pooling, no multiplexing, one inbound connection equals one upstream connection. Pass an IP address (a hostname works but pulls a glibc shared lib at runtime, which defeats the static-binary deployment story). Required.
--out FILE	Write the TSV log and the #-prefixed banner / connection notices to FILE in append mode instead of standard error. The file is opened once at startup and flushed after each line, so a tail-follower sees events as they happen. If fopen(3) fails the snoop falls back to standard error with a one-line warning. Optional; default is standard error.
--only=TOK,TOK,...	Comma-separated whitelist of token names; up to 32 entries, case-sensitive against the names sqli_token_name(3) emits (ONI_TUPLE, ONI_DONE, ONI_PREPARE, ...). When set, only PFPDUs whose token matches one of these names are written to the log. Per-connection timing state (round_us, stmt_us, gap_us) still tracks the full stream, so the surviving rows reflect timing across the actual conversation, not the filtered subset. Optional; default is capture-all.
-h, --help	Print the usage line and exit.

TLS proxy comms

From v4.0 the snoop can terminate TLS on the listen side, re-encrypt to the upstream IDS, or both — decoding the plaintext PFPDU stream in between exactly as it does for an unencrypted connection. The two sides are independent: run TLS only towards the client, only towards the server, or end to end. When TLS is off on a side that side stays raw TCP byte-replay, unchanged from earlier releases.

Flag	Effect
--tls-listen	Terminate TLS on the listen side. The snoop presents a server certificate to the connecting Informix client and decrypts inbound traffic. Requires --tls-listen-cert + --tls-listen-key (or a --conssl-cfg keystore).
--tls-listen-cert FILE	PEM certificate chain presented to clients on the listen side.
--tls-listen-key FILE	Unencrypted PEM private key matching the listen certificate. An encrypted key inside an Informix keystore is reached via --conssl-cfg instead.
--tls-upstream	Initiate TLS to the real IDS server. By default the upstream certificate is verified against the system trust store and the hostname is matched against the certificate SAN; --tls-upstream-ca overrides the trust root, --insecure-upstream disables verification entirely.
--tls-upstream-ca FILE	PEM CA chain used to verify the upstream IDS certificate. Mutually exclusive with --insecure-upstream.
--insecure-upstream	Skip upstream certificate and hostname verification. Development and diagnostic use only — the upstream certificate is still logged in the connection banner but its trust is not checked.
--conssl-cfg FILE	Reuse an existing Informix conssl.cfg — reads SSL_KEYSTORE_FILE (the GSKit keystore) and SSL_KEYSTORE_STH (its password stash). The keystore supplies the listen certificate and key plus the upstream CA chain; the password is recovered from the .sth stash. The keystore must be exported to PKCS #12 first (OpenSSL cannot read a native GSKit .kdb — see Install). Direct --tls-* flags win when both name the same item.
--tls-min-version 1.0\|1.1\|1.2\|1.3	Lowest TLS version negotiated on both sides. Default 1.2. Lower below 1.2 only for legacy Informix servers or clients that cannot do TLS 1.2 — doing so widens the cipher allowlist to the CBC suites those versions require and drops the OpenSSL security level to 0, which weakens the connection. The startup banner flags a lowered floor; leave it at 1.2 unless a handshake fails with unsupported protocol.

A typical end-to-end invocation — terminate a modern client, verify and re-encrypt to IDS:

oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:9088 \
    --tls-listen --tls-listen-cert snoop.pem --tls-listen-key snoop.key \
    --tls-upstream --tls-upstream-ca ids-ca.pem \
    2>/var/log/oni_snoop.log

For an older IDS that only speaks TLS 1.0, add the protocol floor; the client side still negotiates the highest version it can while the upstream side drops to the version the server supports:

oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:9088 \
    --tls-listen --tls-listen-cert snoop.pem --tls-listen-key snoop.key \
    --tls-upstream --tls-upstream-ca ids-ca.pem \
    --tls-min-version 1.0

Each TLS handshake adds a connection-banner line documenting the negotiated version and cipher per side (tls.app= for the client side, tls.srv= for the upstream side) and a peer-certificate line (tls.app.peer / tls.up.peer) carrying the subject, SAN list, and verify result — grep tls.app for client-side issues, tls.up for upstream-side issues. A full TLS session on the log looks like this (end-to-end terminate + re-encrypt, upstream cert verified against --tls-upstream-ca):

# oni_snoop  listen=0.0.0.0:9089  upstream=198.51.100.42:9088
# tls=OpenSSL 3.2.1 30 Jan 2024
# tls.listen=on  tls.upstream=on
# tls.origin=cert=/etc/oni_snoop/snoop.pem  cert=loaded  key=loaded  ca=loaded
# tls.min-version=1.2
# wall_ts  mono_us  conn  dir  token  len  round_us  stmt_us  gap_us  summary
# conn 1 open  app→198.51.100.42:9088  tls.app=TLSv1.3/TLS_AES_256_GCM_SHA384  tls.srv=TLSv1.3/TLS_AES_256_GCM_SHA384
# tls.app.peer peer_cert=<none>  verify=0
# tls.up.peer peer_cert=/O=Example Corp/CN=ids.example.com  san=[DNS:ids.example.com,IP:198.51.100.42]  verify=0 (ok)
2026-06-03T10:14:07.512Z  1500  1  >  ONI_PREPARE  64  -  -  -  prepare: select * from orders where id=?
2026-06-03T10:14:07.514Z  3220  1  <  ONI_DESCRIBE  48  1804  -  900  describe: 3 cols
2026-06-03T10:14:07.515Z  4010  1  >  ONI_TUPLE     22  790   2503 -  v0=42 v1='shipped' v2=DATE(20243)
# conn 1 close

Two reads of the banner: the conn 1 open line confirms TLS 1.3 was negotiated on both legs and the plaintext PFPDUs (ONI_PREPARE, ONI_TUPLE, ...) are decoded in between; tls.up.peer ... verify=0 (ok) confirms the upstream IDS certificate passed verification. The tls.app.peer peer_cert=<none> line is normal — the client presented no certificate (Snooper does not require client-auth).

When --tls-min-version is lowered for a legacy IDS, the banner flags the weakened floor and the upstream leg shows the older version / CBC cipher while the client leg still negotiates the highest it can:

# tls.min-version=1.0  (LEGACY: CBC ciphers + SECLEVEL=0 — connection weakened)
# conn 1 open  app→198.51.100.42:9088  tls.app=TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384  tls.srv=TLSv1/AES128-SHA
# tls.up.peer peer_cert=/CN=legacy-ids  san=[IP:198.51.100.42]  verify=0 (ok)

A failed upstream verification (wrong CA, expired cert, or SAN mismatch) aborts the connection and logs a tls.up.handshake error instead of a conn open line — e.g. tls.up.handshake: error:0A000086:SSL routines::certificate verify failed. Use --insecure-upstream (development only) to bypass it; the connection then logs (insecure) on the conn open line.

Per-connection logger

Each accepted connection owns a 256 KiB byte ring and a dedicated logger pthread. Both forwarder threads (app→server, server→app) push pre-formatted lines into the ring with a single mutex-guarded memcpy(3) and return; the logger thread drains the ring and writes to --out (or stderr) on its own. The forward path therefore never blocks on output I/O, which keeps the snoop's contribution to round-trip latency below the noise floor.

If the logger falls behind — a slow disk, a paused tail process — the producer drops the new entry and increments a per-connection counter. When the connection closes, the snoop emits a # conn N dropped=K (logger fell behind) footer so the operator knows how many lines were lost. With a typical TSV line of ~150 bytes the ring holds ~1700 events before overflow; in practice this is reached only when output is redirected to a remote filesystem or paused on a stuck terminal.

Output destination

By default the snoop writes the TSV event stream to standard error and prints a short header on startup. Nothing is written to standard output. Operators have three options for capturing the log:

# 1. Redirect stderr — the original v1 pattern, still works:
oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    2>/var/log/oni_snoop.log

# 2. --out FILE — open the log file directly, leave stderr free for
#    actual diagnostics:
oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    --out /var/log/oni_snoop.log

# 3. Quick interactive look — tee for split-to-screen-and-file:
oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    2>&1 | tee snoop.log

--out FILE opens in append mode, so re-runs accumulate on top of earlier captures rather than truncating. Pair with --only=TOK,TOK,... to drop the noise floor for steady-state captures — e.g. --only=ONI_PREPARE,ONI_COMMAND,ONI_ERR,ONI_DONE keeps statement starts, completions, and errors and discards the TUPLE / FETCH / EOT chatter.

Lifecycle

The snoop is a foreground process by design. There is no daemon mode, no PID file, no systemd unit shipped. Run it under whatever lifecycle manager fits the deployment:

nohup oni_snoop ... & for ad-hoc background.
systemd-run --unit oni_snoop -- oni_snoop ... for one-shot service.
A short systemd unit file the operator drops at /etc/systemd/system/oni_snoop.service for persistent service. Type=simple, ExecStart=, StandardError=append:/var/log/oni_snoop.log.
tmux / screen session for interactive observation.

On SIGINT or SIGTERM the snoop stops accepting new connections and the process exits. In-flight client connections close with their counterparties when the kernel tears the socket down.

Tagged log lines

Tag	Meaning
# oni_snoop ...	Header banner printed once at startup; lists the listen and upstream endpoints.
# wall_ts ... summary	TSV column header line; printed once after the banner. Documents the schema for downstream parsers.
# conn N open ...	One connection accepted; N is the connection id used in the conn column of every PFPDU line that follows. With TLS on, this line carries tls.app=<ver>/<cipher> (client→snoop leg) and/or tls.srv=<ver>/<cipher> (snoop→IDS leg), plus (insecure) when --insecure-upstream is set.
# conn N close	That connection has closed in both directions. Both forwarder threads have joined.
# conn N dropped=K (logger fell behind)	Optional footer printed at connection close when the per-connection ring buffer overflowed during the session and dropped K events. Indicates the output sink (file or terminal) couldn't keep up; the snoop continued forwarding bytes correctly but some log lines never made it to the file. Increase the ring size or move the log to faster storage if this appears.
# only=A,B,...	Optional banner line printed once at startup when --only is in effect. Documents the active token whitelist so log readers know events outside the list were filtered, not absent.
# tls=..., # tls.listen=... tls.upstream=..., # tls.origin=...	TLS startup banner lines (v4.0+). Report the OpenSSL build, which sides have TLS enabled, and the certificate / key / CA origin (and whether each loaded). Printed once after the header.
# tls.min-version=V	The negotiated-protocol floor (default 1.2). When lowered below 1.2 via --tls-min-version it appends (LEGACY: CBC ciphers + SECLEVEL=0 — connection weakened) so a downgraded connection is visible in the log.
# tls.app.peer ... / # tls.up.peer ...	Per-connection peer-certificate line, one per TLS leg (app = client side, up = upstream side). Carries the cert subject, DNS/IPv4 SAN list, and verify=<code> (<reason>). peer_cert=<none> means the peer presented no certificate (normal on the client side — no client-auth required).
tls.app.handshake: ... / tls.up.handshake: ...	A TLS handshake failure (e.g. certificate verify failed, tlsv1 alert protocol version). The connection aborts without a conn N open line. Grep tls.app for client-side, tls.up for upstream-side faults.
(no #) the TSV row	Every other line is a PFPDU event. See Output Schema for the column layout.

Capture window

For most diagnostics the operator only needs to capture the few seconds around the slow event. The simplest pattern is to start the snoop with its log going to a fresh file, reproduce the issue, then stop the snoop and grep through the log:

oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    2>/tmp/snoop-$(date +%Y%m%d-%H%M%S).log &
SPID=$!

# ...reproduce the slow query / hung session / mysterious disconnect...

kill -INT $SPID
wait $SPID
ls -la /tmp/snoop-*.log

For long-running observation, redirect to a file and let logrotate handle rotation. The snoop writes line-by-line and flushes after each PFPDU, so a rotated file truncated mid-line is recoverable on the next read.