Oninit® Log Ripper — Column Types

Supported types

Column sizes are determined by DESCRIBE / SQLDA with rtypsize(). Null detection is via Informix ifxcdc_valisnull(). Decimal conversion uses Informix lddecimal() + dectoasc(). Embedded NULs in CHAR / VARCHAR / LVARCHAR are preserved by splicing ' || CHR(0) || ' into the emitted literal so captured SQL replays cleanly.

Smart large objects (BLOB / CLOB): contrary to the "smart-LO not supported" wording in IBM's CDC programmer's guide, cdc_startcapture empirically accepts column lists containing BLOB and CLOB on Informix 14.10. The wire format is a 4-byte prefix followed by an ifx_lo_t smart-LO locator; the Ripper opens the locator read-only, drains the bytes via ifx_lo_read, and emits them as an X'NN…' hex literal. The dialect rewriter (sql_dialect) then converts the hex literal to the target's native blob form — PostgreSQL '\xNN…'::bytea, MySQL 0xNN…, Db2 BLOB(X'NN…'), Oracle HEXTORAW('NN…'). The hex form is capped at roughly 4 KiB per column with the current per-column buffer; a larger column truncates and is annotated /* truncated */ so the captured SQL stays parseable. Truly large blobs (megabyte range) need the per-column buffer raised.

Types cdc_startcapture rejects (with a fallback)

The following column types are rejected by cdc_startcapture itself — IBM's documentation ("Data for capture") lists them as not supported, so the column list must exclude them before the CDC session opens:

The CDC pipe never delivers these column values, but the row data still lands in the logical log when the source table commits. recover_unsupported_via_log: true opts in to a raw-log fallback that opens the source server's logical-log files at each row's LSN, reads the full record image via pread(O_RDONLY), and recovers the rejected column's bytes alongside the CDC-supplied columns. For BLOB/CLOB and simple LOBs (TEXT/BYTE) the descriptor bytes ship as an X'NN…' hex literal in the captured SQL; downstream dialect rewriting (PostgreSQL '\xNN…'::bytea, Oracle HEXTORAW('NN…'), etc.) is unchanged from the BLOB/CLOB main path. Encrypted dbspaces are supported when security.decrypt_keystore is set — see safety.html.

The fallback requires the Ripper to run locally on the source server as user informix (or another OS account in the informix group with read access to the log dbspace files): raw log chunks live with informix:informix 0660 permissions and no remote/network protocol exposes them. Tables on encrypted chunks add the decrypt_keystore configuration above.

The reject codes come from running each shape against cdc_startcapture on Informix 14.10 (test/cdc_pmr.sh). Smart large objects (BLOB, CLOB) are not in this list any more — cdc_startcapture accepts them and the Ripper handles them as documented in the supported-types section above. SQLUDTFIXED as a category is also no longer auto-rejected: the Ripper allows it through and the value-decode path opens the locator and drains the bytes; operators with non-BLOB/CLOB SQLUDTFIXED columns (TimeSeries, spatial, BTS) should verify cdc_startcapture accepts their specific column list before relying on capture — if it does, the existing decoder handles the wire format unchanged.

The Ripper detects these columns at startup (during DESCRIBE) and applies a configurable policy so the doomed cdc_startcapture call is never made:

JSON / BSON note: these are SQLUDTVAR-typed columns — variable-length opaque UDTs. Their content is server-internal so the CDC API has no way to serialize it; cdc_startcapture rejects column lists that include them with rc=-83718. A table holding a JSON or BSON column has three operator-actionable options: (a) skip_unsupported_columns: true drops just the JSON column from capture and keeps the rest of the row; (b) recover_unsupported_via_log: true reconstructs the JSON bytes from the logical log so the captured row carries the JSON content verbatim (subject to the local-deployment + keystore requirements above); (c) an explicit columns: ["!doc"] filter does the same as (a) with column-level precision.

Examples

source:
  tables:
    # default: any unsupported column aborts the whole table
    - t_plain

    # auto-exclude unsupported columns; capture the rest
    - name: t_with_row
      skip_unsupported_columns: true

    # JSON column on a table the operator still wants captured;
    # filter the JSON column explicitly so the rest of the row flows.
    - name: t_with_json
      columns:
        - "!doc"

    # Recover TEXT/BYTE/BLOB/CLOB or SQLUDTVAR via raw-log fallback;
    # the column ships as a hex literal alongside the supported cols.
    # Requires the Ripper to run locally on the source as user
    # informix; on encrypted dbspaces also set security.decrypt_keystore.
    - name: t_with_text
      recover_unsupported_via_log: true

A column-filter list of all-excludes (e.g. ["!badcol"]) defaults to "include everything else"; a list with any include entry switches to "include only the listed columns". One IBM caveat for collections: any column that appears AFTER a SET / MULTISET / LIST in the table layout is also not replicated, so for tables with these types the safest patterns are (a) put the collection at the tail of the table layout, or (b) filter out every column to the right of it.