Oninit® ifxTAP — Runner and TAP Output

Status: pre-release. Described here as built and tested on Informix 14.10 and 15.0.

ifxtap is a single Java jar. It uses only java.sql interfaces, loading the Informix classes by reflection or unwrap(), so it compiles and starts without the driver present.

What it does

• Discovers test files in a directory.
• Opens one transaction per file, with an optional savepoint per test.
• Reads the results table, rolls back, and prints TAP 13 with a plan, numbered results and YAML diagnostics.
• Writes tests that pin the current schema (--generate).
• Logs the server and driver versions as TAP comments.
• Aborts the run if an error has ended the transaction.

Generating tests from a schema

ifxtap --generate <dir> reads the catalog of the database it is connected to and writes test files that pin down what exists today, using the assertions in the library. The library must be installed first.

ifxtap --generate tests/schema                          # every user table, view and routine
ifxtap --generate tests/schema-cust --match 'cust%'     # only objects whose names match a LIKE pattern
ifxtap --generate tests/schema-app --owner appuser      # only one owner's tables
ifxtap tests/schema                                     # run what was generated

Use a directory of its own for each filter. The two summary files, schema-views.sql and schema-routines.sql, hold only the views and routines that the run matched. A filtered run into a directory that already has the summaries of a full run replaces them, because they are unchanged generated files, and the views and routines the filter left out are no longer pinned. The per-table files are not touched, so the directory ends up with a mixture. Different directories keep each run's tests whole.

The two summary files have a hyphen in their names, which no table name can have, so a table called views or routines keeps its own file. The generated files are the same every time for the same schema, so regenerating changes nothing and a difference shows up in version control. Each file begins with a checksum of its contents. Regenerating replaces a file only while it is unchanged: a file you edited, or a file of that name that ifxTAP did not write, is left alone and reported (add --force to replace it); keep hand-written tests in files of their own. A generated file whose table has since been dropped is reported, not deleted.

The tests pin what exists: a dropped column, a changed type, a missing index or a changed default fails them; a new column, table or index does not. They do not test behaviour. Objects whose names are not plain lower-case identifiers (quoted mixed-case or names with special characters) are skipped and listed. In a routine, only its existence, parameter types and return types are pinned. Tables that exist under two owners (possible in an ANSI database) are written with owner-qualified names.

Command reference

ifxtap [options] <test.sql | directory> ...     run test files
ifxtap --check                                  connect and report versions
ifxtap --install <file>                         install the library
ifxtap --upgrade <sql-dir>                      upgrade the library
ifxtap --generate <dir> [--owner N] [--match P] [--force]   write tests that pin the schema

Which files are run

Each argument is a file or a directory. A file is run as given. A directory contributes the files directly in it whose names end in .sql, sorted by name; it is not searched recursively, and other files are ignored. Arguments are run in the order given. A name that does not exist, or no test file at all, is a usage error (exit 2) and nothing is run. A file that cannot be read is a failing test of its own, with its own number in the stream. Name files 01_...sql, 02_...sql if order matters. Each file is a separate transaction and must not depend on another.

Exit status

A runner killed by a signal exits with the shell's code for it (137 for kill -9).

A run that does not complete

If the runner is killed, or its connection to the server is lost, in the middle of a file, the server rolls the transaction back, so the file leaves nothing behind. A server-side routine that is still running delays that rollback until it returns. The runner exits non-zero (137 when it is killed with kill -9, 3 when the connection is lost), and the output stops without the closing summary comment (# N file(s), ...).

Running more than one runner

The results tables are local to each session, so runners never see each other’s results. Runners against different databases are independent. Runners against the same database work only while their test files do not create or drop objects: DDL holds locks on the system catalog until the file’s transaction ends, and a second runner then fails with SQLCODE -244 or -312. Run one DDL-heavy runner per database at a time.

Locating the driver

Supported platforms

Databases without logging are not supported. The runner refuses to run against one and exits with status 3, because isolation depends on BEGIN WORK and ROLLBACK WORK.

CI

The pass/fail signal for CI is the exit status: 0 when every assertion passed, non-zero otherwise. The output is TAP 13, and a run of several files is one TAP stream: one plan, printed last, with the numbers continuing from file to file (the # file: comments show where each file starts). A file that did not keep to its plan, or has no plan and never reached tap_finish(), or was abandoned after an error, appears as a failing test named after the file. A consumer reads it as one run: prove from Perl's Test::Harness 3.52 accepts it and reports the failing tests, and so do tap4j (the parser inside the Jenkins TAP plugin), the node tap-parser package and tap.py, which all report the same counts as the runner. Jenkins with its TAP plugin (version 2.4.4) was run on the runner's output too: a run with no failures gave a successful build, a run with failures gave an unstable one, and the TAP Test Results page showed the same test and failure counts as the runner. Running one file per process works too (prove -e ifxtap tests/*.sql), and then each file has its own plan first.

--per-file-plans gives the older layout for a run of several files: a plan at the start of every file and numbering that restarts at 1. That is not one valid TAP stream, and a consumer such as prove rejects it (“More than one plan found”), so use it only to read the output yourself or to split it by file.