Oninit® Log Ripper — Features

Capture

• Real-time CDC capture using the Informix CDC API (cdc_opensess / cdc_activatesess / ifx_lo_read).
• Big-endian CDC binary record parsing with internal ld4() byte-order helpers.
• Multi-threaded: tables distributed across configurable worker threads; one CDC session per worker.
• Glob pattern table selection (*, ?, [abc]) with !exclude support.
• Per-table column filtering with include / exclude patterns.
• Configurable max_recs_per_read to amortize ifx_lo_read overhead.

Output

• File mode — one SQL file per committed transaction, named with microsecond timestamp + worker id. Incomplete transactions flagged as _NO_BEGIN / _NO_COMMIT / _FRAGMENT.
• Informix mode — direct ESQL/C execution against a target database (same server or cross-server db@server).
• ODBC mode — replay to PostgreSQL, MySQL, or any ODBC-connected database.
• Per-record LSN written before each statement.
• Tx file header carries User / Started / Committed / Begin LSN / Commit LSN / record count.
• Single-line [TX] summary in the main log per committed transaction.

Column types

• SERIAL, INTEGER, SMALLINT, BIGINT, INT8, SERIAL8.
• FLOAT, SMALLFLOAT.
• DECIMAL, MONEY — full max-precision DECIMAL(32, p) round-trip preserved; rendered with lddecimal / dectoasc.
• BOOLEAN.
• CHAR, NCHAR — quoted string with byte-count trim; embedded NULs survive.
• VARCHAR, NVARCHAR — quoted string from vardatalen array.
• LVARCHAR — 3-byte length prefix, same path as VARCHAR afterwards.
• DATE — 'YYYY-MM-DD' via rfmtdate; the Informix wire NULL pattern (0x80000000) is recognized by ifxcdc_valisnull and emitted as SQL NULL.
• DATETIME — emitted with the column's actual qualifier (round-trip safe).
• INTERVAL — INTERVAL ( value ) start TO end syntax.
• Embedded NULs in CHAR / VARCHAR / LVARCHAR survive via ' || CHR(0) || ' splicing.

Schema safety

• Three-layer drift detection on every CDC TABSCHEMA record: structural triple (cols / var / fix bytes), per-column NAMES (catches RENAME COLUMN), per-column TYPE keywords (catches same-byte-width type swaps like INT↔DATE, BIGINT↔FLOAT).
• Schema archive recovery: optional schema_archive_dir lets the ripper swap to an archived <db>_<tab>.sql layout when drift can be reconciled.
• Per-column archive-vs-live comparison at startup, an extra safety net when the archive is present.
• Skip-on-unreconcilable-drift: per-table skip_capture keeps the rest of the workload running; the ripper exits only when every monitored table is skipped.
• Unsupported-column policy: tables with a column type Informix CDC rejects (TEXT / BYTE simple LOs, SET / MULTISET / LIST / ROW collections, SQLUDTVAR opaque UDTs such as JSON / BSON) are detected at DESCRIBE time and aborted with a [CRITICAL] log line; per-table skip_unsupported_columns: true auto-excludes those columns instead. BLOB and CLOB are supported — the ripper drains the ifx_lo_t locator and emits the bytes as a per-target hex literal.

Operations

• Live release / re-acquire of tables via release_<tab> / acquire_<tab> sentinel files in control_dir — one table can cycle through release → ALTER → acquire while every other table keeps capturing uninterrupted.
• LSN checkpoint persistence: optional state_file holds min(last_committed_lsn) across restarts; -R on the CLI forces a clean restart for one run.
• Error recovery: ifx_lo_read failure closes the failed session, drains active transactions as _NO_COMMIT, and re-opens via cdc_activatesess at the last committed LSN.
• Orphan CDC session cleanup at startup (cleanup_orphans).
• Init mutex serializes cdc_opensess across workers (works around a known multi-worker race).
• Daemon mode with PID file, size-capped log file with numeric rotation (log_keep rotated copies retained), SIGHUP runtime config reload, SIGTERM grace shutdown.
• Periodic [STATUS] + [LSN] lines: per-worker statistics (records/sec, tx/sec, bytes read, recovery count, LSN), source head LSN polled from sysmaster:syslogs, and the byte lag delta.
• Optional DISCARD alerting: operator-configured shell command runs once per CDC_REC_DISCARD event, rate-limited so a discard storm does not fork-bomb the alerter.
• Target-offline buffering: when the target returns a connection-class failure (e.g. SQLSTATE 08*, libpq CONNECTION_BAD, mysql_errno 2002 / 2006 / 2013, librdkafka __TRANSPORT / __ALL_BROKERS_DOWN), transactions accumulate in a memory ring; spilled batches memory-map to disk under buffering.disk_overflow_path; a probe thread drains chronologically when the target returns. Survives restart via the startup spill-recovery scan.
• Optional Prometheus metrics endpoint (/metrics) + liveness probe (/healthz) via an embedded HTTP server. Per-worker counters (records by op, recovery attempts) and gauges (capture lag in seconds, worker state) plus a build-info gauge. Drop-in Grafana provisioning YAML + curated dashboards (Capture Health + Worker Drilldown) ship under share/grafana/ with two datasource paths: Prometheus or the Oninit Grafana plugin.
• Multiple debug levels: -v verbose, -T trace, -S SQL, -C CDC, -D column detail, -X raw hex dump.

Tooling

• YAML-based configuration with documented example.
• RPM packaging with systemd unit; sysusers / tmpfiles pre-wired; runs as the oni_ripper system user by default, override-friendly to informix:informix.
• Connection-test tool for validating Informix and CDC API connectivity end-to-end.

Embeddable

• Engine ships as a shared library (liboni_ripper.so) with a public C API. The standalone oni_ripper binary is a thin CLI wrapper around the same library.
• Embedders register a per-transaction callback and receive captured changes synchronously in-process — no file, socket, or Kafka detour. Stable-layout oni_tx_t / oni_record_t structs with txid, LSNs, op, owner / table, and rendered SQL.
• Pluggable log callback — every [TAG] line routes into the host's logging framework instead of stderr.
• Library does not install signal handlers; the embedder owns its own SIGTERM / SIGHUP handling and calls oni_engine_stop() / oni_engine_reload() as appropriate.
• pkg-config manifest, static and shared variants, ABI versioned via SONAME.