Informix DataBlade API Programmer's Manual Extending Data Types

To create an opaque data type, follow these steps:

1. Design the structure of the opaque data type.
2. Write the opaque-type support functions.
3. Take special measures if the opaque type is for multirepresentational data.

Tip: The Informix BladeSmith development tool, which is part of the DataBlade Developers Kit, automatically generates C source code for the support routines of an opaque type as well as the SQL statements to register the opaque type. For more information, see the "DataBlade Developers Kit User's Guide."

Register an opaque type with the CREATE OPAQUE TYPE statement, which stores information about an opaque type in the sysxtdtypes system catalog table.

The external structure of an opaque data type is a character string. This string is the format that end users can use to specify a value for the opaque type. It is important that the external format be reasonably intuitive and easy to enter.

Tip: The external format of an opaque data type is its text representation.

Suppose you need to create an opaque type that holds information about a circle. You could create the external format that Figure 14-1 shows for this circle.

Figure 14-1 External Format of the circle Opaque Data Type

With the external format in Figure 14-1, an INSERT statement can specify a value for a column of type circle with the following format:

INSERT INTO tab1 (circle_col) VALUES ("(2, 3, 9)")

The internal format of an opaque data type is a C data structure that holds the information that the opaque type needs. Inside this internal C structure, use the platform-independent DataBlade API data types (such as mi_integer and mi_real) to improve the portability of the opaque data type. For example, the circle_t data structure holds the values for the circle opaque data type, as Figure 14-2 shows.

Figure 14-2 Internal Structure of a Sample Opaque Type

Tip: The internal format of an opaque data type is its binary representation.

The internal structure uniquely names the opaque data type. Informix recommends that you develop a unique prefix for the opaque data type. You can use this prefix with each member of the internal structure and with the structure itself. If your DataBlade module uses a prefix that is registered with Informix, begin the names of opaque types with this prefix. For example, if you are creating a Geodetic DataBlade module, you might use the prefix geo on all database objects that your DataBlade module creates. Your preceding circle_t opaque type could be named geo_circle_t to ensure that it does not conflict with opaque types that other DataBlade modules might create.

When you create the internal structure for the opaque type, consider the following affects of the size of this structure:

• The final structure size of the new opaque data type
• How the opaque data type should be aligned in memory
• How the opaque data type should be passed in an MI_DATUM structure

You provide this information when you create the opaque data type with the CREATE OPAQUE TYPE statement.

To save space in the database, you should lay out internal structures as compactly as possible. The database server stores values in their internal representation, so any internal structure with padding between entries consumes unnecessary space. You must also decide whether your opaque data type is be fixed length or varying-length. The following sections briefly describe each of these kinds of opaque types.

If the C structure that holds your opaque type is always the same size, regardless of the data it holds, you can declare the opaque type as a fixed-length opaque type. You tell the database server that an opaque type is fixed length when you register the opaque type. In the CREATE OPAQUE TYPE statement, you must include the INTERNALLENGTH modifier to specify the actual size of the C structure. The database server stores the value of the INTERNALLENGTH modifier in the length column of the sysxtdtypes system catalog table. The size of a fixed-length opaque data type must match the value that the C-language sizeof directive returns for the internal structure.

The internal C structure, circle_t, (which Figure 14-2 on page 14-5 defines) is a fixed-length structure because all of its member fields have a constant size. The following CREATE OPAQUE TYPE statement registers a fixed-length opaque type called circle for the circle_t structure:

CREATE OPAQUE TYPE circle (INTERNALLENGTH = 24)

On most compilers, the sizeof() directive performs cast promotion to the nearest 4-byte size to ensure that the pointer match on arrays of structures works correctly. However, you do not need to round up for the size of a fixed-length opaque data type. Instead you can specify alignment for the opaque data type with the ALIGNMENT modifier. For more information, see Memory Alignment.

Important: The routine manager does perform cast promotion on argument values smaller than the size of an MI_DATUM when it pushes routine arguments onto the stack. On some platforms, small values can create problems with pointer matching. For more information, see Pushing Arguments Onto the Stack.

The size of the fixed-length opaque type determines the passing mechanism for the opaque type. For more information, see Passing Mechanism.

If the C structure that holds your opaque type can vary in size depending on the data it holds, you can declare the opaque type as a varying-length opaque type. A varying-length opaque type might contain the following kinds of data:

• Varying-length character strings
• Multirepresentational data

A multirepresentational data type has data that changes based on some predetermined threshold. If the size of the data is less than or equal to this threshold, the structure holds the actual data. However, if the data is large than this threshold, the data is stored elsewhere, usually in a smart large object. Only the last member of the internal structure can be of varying size.

Figure 14-3 shows the internal C structure for a multirepresentational data type.

Figure 14-3 Internal Structure for the image Opaque Data Type

You tell the database server that an opaque type is varying length when you register the opaque type. In the CREATE OPAQUE TYPE statement, you must include the INTERNALLENGTH modifier and include the keyword VARIABLE. The database server stores the value of the INTERNALLENGTH modifier in the length column of the sysxtdtypes system catalog table. For varying-length opaque types, this column holds a value of zero (0).

The internal C structure, image_t, is a varying-length structure because its fourth field is either an LO handle (to hold the data if it exceeds the value in the img_thresh field) or a character string (to hold data that can fit into the 4-character array). The following CREATE OPAQUE TYPE statement registers a varying-length opaque type called image for the internal C structure that Figure 14-3 on page 14-7 defines:

CREATE OPAQUE TYPE circle (INTERNALLENGTH = VARIABLE)

The database server stores a varying-length data type in a varying-length structure. Therefore the size limitations of a varying-length type apply to a varying-length opaque type as follows:

• By default, the maximum size for a varying-length opaque type is 2 kilobytes.
• You specify a different maximum size for a varying-length opaque type when you register the opaque type.

In the CREATE OPAQUE TYPE statement, use the MAXLEN modifier. You can specify a maximum length of up to 32 kilobytes. The database server stores the value of the MAXLEN modifier in the maxlen column of the sysxtdtypes system catalog table.

For example, the following CREATE OPAQUE TYPE statement defines a varying-length opaque type called var_type whose maximum size is 4 kilobytes:

CREATE OPAQUE TYPE var_type 
	(INTERNALLENGTH=VARIABLE, MAXLEN=4096);

The passing mechanism for a varying-length opaque type is always by reference. For more information, see Passing Mechanism.

When the database server passes the data type to a user-defined routine, it aligns the opaque-type data on a certain byte boundary. By default, the database server assumes a 4-byte alignment for an opaque-type structure. Four bytes is the standard alignment required for 32-bit platforms.

On 64-bit platforms, alignment should usually be 8 bytes.

You can specify a different memory-alignment requirement for your opaque type with the ALIGNMENT modifier of the CREATE OPAQUE TYPE statement. The database server stores the value of the ALIGNMENT modifier in the align column of the sysxtdtypes system catalog table.

Actual alignment requirements depend on the C definition of the opaque type and on the system (hardware and compiler) on which the opaque data type is compiled. The following table summarizes valid alignment values for C opaque types.

Arrays of a data type must follow the same alignment restrictions as the data type itself. However, structures that begin with single-byte characters (such as mi_boolean or mi_char) can be aligned anywhere.

When you obtain aligned data for an opaque data type from a varying-length structure, use the mi_get_vardata_align() function. Make sure that the align argument of mi_get_vardata_align() matches the value of the align column in the sysxtdtypes table. For example, the mi_double_precision data type is aligned on an 8-byte boundary. If an opaque type contains an array of mi_double_precision values, use mi_get_vardata_align() with an align value of 8 to access the data portion of the mi_double_precision array.

The following call to mi_get_vardata_align() obtains data that is aligned on 8-byte boundaries from the var_struc varying-length structure:

char *buff;
mi_lvarchar *var_struc;


buff = mi_get_vardata_align(var_struc, 8);

By default, when the DataBlade API passes the internal format of an opaque type in an MI_DATUM, it always passes the opaque type by reference. That is, an MI_DATUM contains a pointer to the internal C structure of the opaque type. Both varying-length opaque types and most fixed-length opaque types must be passed by reference. However, if your fixed-length opaque type is always smaller than the size of an MI_DATUM, you can tell the database server that it can be passed by value when you register the opaque type. In the CREATE OPAQUE TYPE statement, you must include the PASSEDBYVALUE modifier. The database server stores the value of the ALIGNMENT modifier in the byvalue column of the sysxtdtypes system catalog table.

Important: Only a fixed-length opaque type that can always fit into an MI_DATUM can be passed by value.

Suppose you have the declaration in Figure 14-4 for a fixed-length opaque type called two_bytes.

Figure 14-4 Internal Structure for the two_bytes Opaque Data Type

The following CREATE OPAQUE TYPE statement specifies that the two_bytes opaque type can be passed by value:

CREATE OPAQUE TYPE two_bytes (INTERNALLENGTH=2, 
	ALIGNMENT=2, PASSEDBYVALUE)

The support functions of an opaque data type tells the database server how to interact with the opaque type. The support functions for opaque data types use the following varying-length data types to hold data passed to and from these functions.

Tip: The Informix BladeSmith development tool, which is part of the DataBlade Developers Kit, automatically generates C source code for the support routines of an opaque type. For more information, see the "DataBlade Developers Kit User's Guide."

For example, a send opaque-type support function must return an SENDRECV value, as the following declaration shows:

mi_sendrecv *circle_send(internal_struc intr_struc_t)

The database server uses the SENDRECV data type to locate the correct support function when it needs to cast from the internal to external format of an opaque type during a transfer of data to a client application.

All of these varying-length data types have the same internal structure. However, you might need to cast between these types to avoid compilation warnings. In addition, varying-length opaque data types use this structure. Therefore, any DataBlade API function that is declared to handle the mi_lvarchar data type can also handle these other varying-length data types. If you are using a varying-length data type other than mi_lvarchar, you can cast between the varying-length type you are using and mi_lvarchar.

For example, the mi_string_to_lvarchar() function converts a null-terminated string to an mi_lvarchar varying-length data type. However, you can use casting to have this function convert a null-terminated string to an mi_sendrecv varying-length data type, as follows:

mi_sendrecv *new_sendrecv;
...
new_sendrecv = (mi_sendrecv *)mi_string_to_lvarchar(strng);

This casting is not strictly required, but many compilers recommend it and it does improve clarity of purpose.

Any size of data can fit into a varying-length structure. However, only 2 kilobytes can fit into an mi_lvarchar data type when it holds a value for an LVARCHAR column. When a varying-length data type holds a value for an opaque-type column, this 2-kilobyte size restriction does not apply. You can write the appropriate support functions of the opaque data type to handle more than 2 kilobytes. For more information on how to manage these varying-length structures, see Varying-Length Data Structures.

The input and output support functions are the cast functions for an opaque type between its external (text) and internal (binary) formats. The database server stores the external format of an opaque type in an mi_lvarchar structure. The mi_lvarchar structure is always passed by reference. Therefore, the input and output support routines have the following basic signatures.

There is no limitation on the size of an mi_lvarchar structure. The DataBlade API can transport mi_lvarchar data to and from the database server. However, to conform to the storage limit of a database table (32 kilobytes for a table, 2 kilobytes for an LVARCHAR column), the input support function might need to handle extra data and the output support function might need to generate the extra data.

The 2-kilobyte restriction does not apply to an mi_lvarchar structure that holds the external format of an opaque-type column. If the input and output support functions of the opaque data type can handle more than 2 kilobytes, the mi_lvarchar structure can hold more than 2 kilobytes. See also mi_lvarchar Data Type.

The input support function takes the text representation of the opaque type, which is encapsulated in an mi_lvarchar structure, and returns the binary representation of that type. The following function signature is for an input support function of an opaque data type whose internal format is circle_t (which Figure 14-2 on page 14-5 defines):

circle_t *circle_input(extrnl_format)
	mi_lvarchar *extrnl_format;

The circle_input() function is a cast function from the mi_lvarchar data type to the circle_t internal structure (on the server computer). It returns a pointer to the circle_t data type because this C structure must be passed by reference.

If an opaque type was registered with the PASSEDBYVALUE modifier, the input function can return the actual internal value. The following function signature is for the input support function of an opaque data type whose internal format is two_bytes_t (which Figure 14-4 on page 14-10 defines):

two_bytes_t two_bytes_input(extrnl_format)
	mi_lvarchar *extrnl_format;

An mi_lvarchar is always passed by reference. Therefore, the argument to the input support function must always be a pointer to the mi_lvarchar data type. For information on how to obtain information from this varying-length structure, see Obtaining Varying-Length Data.

The DataBlade API provides the mi_string_to_lvarchar() function to convert character data from its text representation (null-terminated string) to its binary representation (an mi_lvarchar structure). The input function can call this function on character fields in the internal format of the opaque data type.

The database server calls the input support routine for an opaque-type value when a DataBlade API module passes it in its text representation to an INSERT or UPDATE statement. The text representation can be either the literal value in the statement or an input parameter in a prepared statement.

The output support function takes the binary representation of the opaque data type and returns text representation of that type, which is encapsulated in an mi_lvarchar structure. The following function signature is for an output support function of an opaque data type whose internal format is circle_t (which Figure 14-2 on page 14-5 defines):

mi_lvarchar *circle_output(intrnl_format)
	circle_t *intrnl_format;

The circle_output() function is a cast function from the circle_t internal structure (on the server computer) to the mi_lvarchar data type. It defines its argument as a pointer to the circle_t data type because this C structure must be passed by reference.

If an opaque type was registered with the PASSEDBYVALUE modifier, the output function can accept the actual internal value as an argument. The following function signature is for the output support function of an opaque data type whose internal format is two_bytes_t (which Figure 14-4 on page 14-10 defines):

mi_lvarchar *two_bytes_output(intrnl_format)
	two_bytes_t intrnl_format;

The DataBlade API provides the mi_lvarchar_to_string() function to convert character data from its binary representation (an mi_lvarchar structure) to its text representation (null-terminated string). The output function can call this function on character fields in the internal format of the opaque data type.

The database server calls the output support routine for an opaque-type value when the database server passes it in its text representation back to a DataBlade API module (SELECT or EXECUTE FUNCTION).

The input and output support functions can call the following DataBlade API functions to convert the atomic C data types within the internal structure of the opaque data type between their text and binary representations.

Most functions that convert between text and binary representations use the end-user formats for any data in a locale-specific format. For more information about how to internationalize a C UDR, see the chapter on database server features in the Informix Guide to GLS Functionality.

Because the request for the opaque type might originate from a client application, the database server must handle the possibility that the client computer uses a different byte ordering than the server computer. The receive and send support functions are the cast functions for an opaque type between its internal (binary) format on a client computer and its internal (binary) format on the server computer.

The database server stores the internal format of an opaque type from the client computer in an mi_sendrecv structure. The mi_sendrecv structure is a varying-length structure, which encapsulates the client internal format to handle any possible changes in the size of the value when it is converted from its server to client internal format. For example, the client and server computers might have different packing rules for structures.

A varying-length structure is always passed by reference. Therefore, the receive and send support routines have the following basic signatures.

The database server receives a description of the client computer when the client application establishes a connection. The DataBlade API provides several functions that access this information for use in send and receive functions. For more information, see Converting Opaque-Type Data With Machine-Specific Data Types.

The receive support function takes the binary representation of the opaque type on the client computer, which is encapsulated in an mi_sendrecv structure, and returns the binary representation of the opaque data type on the server computer. The following function signature is for a receive support function of an opaque data type whose internal format is circle_t (which Figure 14-2 on page 14-5 defines):

circle_t *circle_receive(client_intrnl_format)
	mi_sendrecv *client_intrnl_format;

The circle_receive() function is a cast function from the mi_sendrecv data type to the circle_t internal structure (on the server computer). It returns a pointer to the circle_t data type because this C structure must be passed by reference.

If an opaque type was registered with the PASSEDBYVALUE modifier, the receive function can return the actual internal value. The following function signature is for the receive support function of an opaque data type whose internal format is two_bytes_t (which Figure 14-4 on page 14-10 defines):

two_bytes_t two_bytes_receive(client_intrnl_format)
	mi_sendrecv *client_intrnl_format;

An mi_sendrecv is always passed by reference. Therefore, the argument to the receive support function must always be a pointer to the mi_sendrecv data type. For information on how to obtain information from this structure, see Obtaining Varying-Length Data.

The DataBlade API provides functions to convert simple C data types from client to server binary representations. The receive function can call these functions on members of the internal format for the opaque data type. For example, to convert the double-precision values in the circle_t structure to their binary representation on the server computer, the circle_receive() function can call the mi_get_double_precision() function. For a list of these DataBlade API functions, see Converting Opaque-Type Data With Machine-Specific Data Types.

The database server calls the receive support routine for an opaque-type value when a client application passes the value in its binary representation to an INSERT or UPDATE statement.

The send support function takes the binary representation of the opaque data type on the server computer and returns the binary representation of the opaque type on the client computer, encapsulated in an mi_sendrecv structure. The following function signature is for a send support function of an opaque data type whose internal format is circle_t (which Figure 14-2 on page 14-5 defines):

mi_sendrecv *circle_send(srvr_intrnl_format)
	circle_t *srvr_intrnl_format;

The circle_output() function is a cast function from the circle_t internal structure (on the server computer) to the mi_sendrecv data type. It must define its argument as a pointer to the circle_t data type because this C structure must be passed by reference.

If an opaque type was registered with the PASSEDBYVALUE modifier, the send function can accept the actual internal value as its argument. The following function signature is for the send support function of an opaque data type whose internal format is two_bytes_t (which Figure 14-4 on page 14-10 defines):

mi_sendrecv *two_bytes_send(srvr_intrnl_format)
	two_bytes_t srvr_intrnl_format;

The DataBlade API provides functions to convert simple C data types from server to client binary representations. The send function can call these functions on members of the internal format for the opaque data type. For example, to convert the double-precision values in the circle_t structure to their binary representation on the client computer, the circle_send() function can call the mi_put_double_precision() function. For a list of these DataBlade API functions, see Converting Opaque-Type Data With Machine-Specific Data Types.

The database server calls the send support function for an opaque-type value that the database server passes back in its binary representation to a client application (SELECT or EXECUTE FUNCTION).

The send and receive support functions can call DataBlade API functions to convert the atomic C data types within the internal structure of an opaque data type. Figure 14-5 shows the DataBlade API functions that can convert a difference in alignment or byte order between the client computer and the server computer.

Characters have the same binary representation on all architectures. Therefore, they do not need to be converted. However, if the code sets of the server-processing locale (in which the UDR executes) and the client locale differ, the mi_get_string() and mi_put_string() functions automatically perform the appropriate code-set conversion (assuming that the two code sets are compatible). For more information about how to internationalize a C UDR, see the chapter on database server features in the Informix Guide to GLS Functionality.

Multirepresentational data can be stored in the table itself or as a smart large object, depending on the actual size of this data. This limit in size is called the threshold for the multirepresentational data. You implement multirepresentational data as an opaque type. Benefits to this kind of opaque data type are as follows:

• Optimize storage and access for individual values that are small enough to store in-row.
• Remove size limits for individual values that are stored smart large objects.
• Make the details of storage implementation transparent to the end user.

The DataBlade API provides the MI_MULTIREP_DATA data type to hold multirepresentational data. Unlike most DataBlade API structures, the MI_MULTIREP_DATA structure is not an opaque C data structure. To access the multirepresentational data, you must allocate an MI_MULTIREP_DATA structure and directly access its fields. Figure 14-6 shows the fields in the MI_MULTIREP_DATA structure.

Important: The MI_MULTIREP_DATA structure does not contain a field to track whether a value is stored in-row or in a smart large object. You must add this field as part of the internal structure of the opaque data type. For more information, see Defining the Opaque Type.

Implementing the Multirepresentational Opaque Type

To implement a multirepresentational opaque type, take the following steps:

1. Determine the threshold for your multirepresentational data.
2. Define an opaque type for the multirepresentational data.
3. Write the support functions for the multirepresentational opaque type.

To choose a threshold value, you must balance the following factors:

• Keep the threshold small enough to create fast in-row access.
• Keep the threshold large enough to maintain the usability of the opaque type.

For example, specifying a threshold of 32 kilobytes would limit end users to creating a table that contained only one multirepresentational column.

The threshold cannot be greater than the value you set with the MAXLEN modifier of the CREATE OPAQUE TYPE statement. However, the threshold can be smaller than this value. This MAXLEN value applies only to data stored in-row. It does not limit the amount of data that you can store in the smart large object.

To hold multirepresentational data, an opaque type must have the following characteristics:

• Its internal structure must have two fields:
• A field whose data type is MI_MULTIREP_DATA
• A threshold-tracking field to indicate whether the data is stored in-row or in a smart large object
• It must be registered as a varying-length opaque type; that is, the CREATE OPAQUE TYPE statement must specify the following modifier:
INTERNALLENGTH = VARIABLE

The DataBlade API provides the following functions to assist in the management of multirepresentational data types.

The support functions of the multirepresentational opaque type must manage the threshold for the data type. The following table lists the basic tasks that the support functions of the multirepresentational opaque type must perform.