Oninit® Log Ripper — Operations

CLI

oni_ripper -c <config.yml> [-l <lsn>] [-w <threads>]
              [-i <secs>] [-d] [-v] [-T] [-S] [-C] [-D] [-X]
              [-R] [-V] [-h]

Daemon mode

oni_ripper -c config.yml -d
cat /tmp/oni_ripper.pid

# Reload config without restart
kill -HUP $(cat /tmp/oni_ripper.pid)

# Stop
kill $(cat /tmp/oni_ripper.pid)

Or use the shipped oni_ripper.sh wrapper:

./oni_ripper.sh start -c config.yml
./oni_ripper.sh status
./oni_ripper.sh stop
./oni_ripper.sh restart -c config.yml

Graceful shutdown via SIGTERM with a 10-second timeout before force-kill. In-flight transactions get drained as _NO_COMMIT incomplete files so nothing is lost.

Tagged log lines

Watching the lag

The [LSN] line shows source head, min/max processed across workers, and the byte lag — the early-warning signal before the source ever recycles a log and a CDC_REC_DISCARD fires:

[STATUS 2026-04-27 14:49:35] uptime=24s workers=2/2 active_tx=0 ...
[LSN    2026-04-27 14:49:35] source=0x43006d0000 min_processed=0x43006c220c
                              max_processed=0x43006c220c lag_max=56820 lag_min=56820
[STATUS 2026-04-27 14:49:35]   worker 0: sid=... lsn=0x43006c220c
[STATUS 2026-04-27 14:49:35]   worker 1: sid=... lsn=0x43006c220c

If lag is growing across reports, either the source log size needs to go up (more headroom) or the ripper needs more workers / faster output. If lag still wins and a DISCARD lands, the configured discard_command alert fires.

Target-offline buffering

When the configured database / streaming target returns a connection-class failure, the dispatcher routes incoming transactions into a memory ring instead of dropping them. When memory fills, the oldest batch spills to a memory-mapped file under buffering.disk_overflow_path. A probe thread retries the target every buffering.target_check_seconds; on success the buffer drains chronologically (spill files first, then memory) and the ripper resumes pass-through.

The state machine has five states — OFF (memory_max_bytes=0), ONLINE (target up, pass-through), BUFFERING (target down, accumulating), DRAINING (target back, replaying backlog), SUSPENDED (memory + disk both full). The periodic [BUFFER] line in the daemon log reports current state plus memory and disk fill so operators see the backlog grow and shrink in real time:

[BUFFER] target offline (rc=-2) — switching to memory buffer (cap=268435456)
[BUFFER] memory at 23% (61865984 / 268435456)
[BUFFER] memory full — spilling oldest batch to /var/lib/oni_ripper/buffer
[BUFFER] spilled 18 tx (1247432 bytes) to /var/lib/oni_ripper/buffer/1714398234_0.buf
[DRAIN]  target back — draining backlog (probe rc=0)
[DRAIN]  processed /var/lib/oni_ripper/buffer/1714398234_0.buf (18 tx), removed
[DRAIN]  complete (143 tx) — target online
[BUFFER 2026-04-29 06:30:00] buffer=online mem=0/268435456 disk=0/10737418240 files=0 buffered=143 drained=143

On SIGTERM the buffering.on_shutdown policy decides: flush (default) spills the in-memory tail to disk_overflow_path so the next startup picks it up via the crash-recovery scan that runs before the CDC session opens; drain attempts a final drain and falls through to flush if the target is still down; abandon drops the in-memory tail with a [CRITICAL] audit line.

Prometheus metrics & Grafana dashboards

The ripper ships an embedded HTTP server exposing /metrics in Prometheus text-exposition format v0.0.4 and /healthz for liveness probes. Default-disabled; enable with:

monitoring:
  prometheus:
    port: 9091
    bind: "0.0.0.0"   # 127.0.0.1 to keep loopback-only

Curated Grafana provisioning files + dashboard JSON ship under share/grafana/: a Prometheus datasource path and an Oninit Grafana plugin path, each with a Capture Health dashboard and a per-worker drilldown. See the dedicated Grafana & Prometheus page for the metric set, the install steps for both datasource flavors, panel-by-panel breakdown, the operator playbook, and ad-hoc PromQL recipes.

Live release / re-acquire

To ALTER one table without stopping the ripper, drop sentinel files in control_dir:

oni_logripper_control -d /var/lib/oni_ripper/control release t_customer
                                          # ripper drops cdc_endcapture +
                                          # cdc_set_fullrowlogging(0); ALTER OK
oni_logripper_control -d /var/lib/oni_ripper/control acquire t_customer
                                          # ripper re-DESCRIBEs +
                                          # cdc_startcapture under new schema

Released tables drop their row events until acquired. Every other table in the capture set keeps reading uninterrupted — no ripper restart required.