Oninit Logo
The Down System Specialists
+1-913-732-8892
+44-2081-337529
Partnerships Contact

Oninit® Snooper — Operations

CLI

oni_snoop --listen <bind:port> --upstream <host:port>
          [--out FILE] [--only=TOK,TOK,...]
FlagEffect
--listen <bind:port> Address and TCP port the snoop binds for incoming client connections. Use 0.0.0.0:<port> to accept on every interface or <ip>:<port> to bind to a specific NIC. Required.
--upstream <host:port> Real IDS server endpoint. The snoop opens a fresh upstream TCP connection for every client that connects in — no pooling, no multiplexing, one inbound connection equals one upstream connection. Pass an IP address (a hostname works but pulls a glibc shared lib at runtime, which defeats the static-binary deployment story). Required.
--out FILE Write the TSV log and the #-prefixed banner / connection notices to FILE in append mode instead of standard error. The file is opened once at startup and flushed after each line, so a tail-follower sees events as they happen. If fopen(3) fails the snoop falls back to standard error with a one-line warning. Optional; default is standard error.
--only=TOK,TOK,... Comma-separated whitelist of token names; up to 32 entries, case-sensitive against the names sqli_token_name(3) emits (ONI_TUPLE, ONI_DONE, ONI_PREPARE, ...). When set, only PFPDUs whose token matches one of these names are written to the log. Per-connection timing state (round_us, stmt_us, gap_us) still tracks the full stream, so the surviving rows reflect timing across the actual conversation, not the filtered subset. Optional; default is capture-all.
-h, --help Print the usage line and exit.

Per-connection logger

Each accepted connection owns a 256 KiB byte ring and a dedicated logger pthread. Both forwarder threads (app→server, server→app) push pre-formatted lines into the ring with a single mutex-guarded memcpy(3) and return; the logger thread drains the ring and writes to --out (or stderr) on its own. The forward path therefore never blocks on output I/O, which keeps the snoop's contribution to round-trip latency below the noise floor.

If the logger falls behind — a slow disk, a paused tail process — the producer drops the new entry and increments a per-connection counter. When the connection closes, the snoop emits a # conn N dropped=K (logger fell behind) footer so the operator knows how many lines were lost. With a typical TSV line of ~150 bytes the ring holds ~1700 events before overflow; in practice this is reached only when output is redirected to a remote filesystem or paused on a stuck terminal.

Output destination

By default the snoop writes the TSV event stream to standard error and prints a short header on startup. Nothing is written to standard output. Operators have three options for capturing the log:

# 1. Redirect stderr — the original v1 pattern, still works:
oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    2>/var/log/oni_snoop.log

# 2. --out FILE — open the log file directly, leave stderr free for
#    actual diagnostics:
oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    --out /var/log/oni_snoop.log

# 3. Quick interactive look — tee for split-to-screen-and-file:
oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    2>&1 | tee snoop.log

--out FILE opens in append mode, so re-runs accumulate on top of earlier captures rather than truncating. Pair with --only=TOK,TOK,... to drop the noise floor for steady-state captures — e.g. --only=ONI_PREPARE,ONI_COMMAND,ONI_ERR,ONI_DONE keeps statement starts, completions, and errors and discards the TUPLE / FETCH / EOT chatter.

Lifecycle

The snoop is a foreground process by design. There is no daemon mode, no PID file, no systemd unit shipped. Run it under whatever lifecycle manager fits the deployment:

  • nohup oni_snoop ... & for ad-hoc background.
  • systemd-run --unit oni_snoop -- oni_snoop ... for one-shot service.
  • A short systemd unit file the operator drops at /etc/systemd/system/oni_snoop.service for persistent service. Type=simple, ExecStart=, StandardError=append:/var/log/oni_snoop.log.
  • tmux / screen session for interactive observation.

On SIGINT or SIGTERM the snoop stops accepting new connections and the process exits. In-flight client connections close with their counterparties when the kernel tears the socket down.

Tagged log lines

TagMeaning
# oni_snoop ... Header banner printed once at startup; lists the listen and upstream endpoints.
# wall_ts ... summary TSV column header line; printed once after the banner. Documents the schema for downstream parsers.
# conn N open ... One connection accepted; N is the connection id used in the conn column of every PFPDU line that follows.
# conn N close That connection has closed in both directions. Both forwarder threads have joined.
# conn N dropped=K (logger fell behind) Optional footer printed at connection close when the per-connection ring buffer overflowed during the session and dropped K events. Indicates the output sink (file or terminal) couldn't keep up; the snoop continued forwarding bytes correctly but some log lines never made it to the file. Increase the ring size or move the log to faster storage if this appears.
# only=A,B,... Optional banner line printed once at startup when --only is in effect. Documents the active token whitelist so log readers know events outside the list were filtered, not absent.
(no #) the TSV row Every other line is a PFPDU event. See Output Schema for the column layout.

Capture window

For most diagnostics the operator only needs to capture the few seconds around the slow event. The simplest pattern is to start the snoop with its log going to a fresh file, reproduce the issue, then stop the snoop and grep through the log:

oni_snoop --listen 0.0.0.0:9089 --upstream 198.51.100.42:19089 \
    2>/tmp/snoop-$(date +%Y%m%d-%H%M%S).log &
SPID=$!

# ...reproduce the slow query / hung session / mysterious disconnect...

kill -INT $SPID
wait $SPID
ls -la /tmp/snoop-*.log

For long-running observation, redirect to a file and let logrotate handle rotation. The snoop writes line-by-line and flushes after each PFPDU, so a rotated file truncated mid-line is recoverable on the next read.

To discuss how Oninit ® can assist please call on +1-913-732-8892 or alternatively just send an email specifying your requirements.


You get all this for free.. think about what you get if you pay us