The LBM Self-Describing Message (SDM) API provides a framework for applications to create and use messages containing self-describing data (name and type). An SDM message contains one or more fields. Each field consists of:

A name, limited to 255 characters in length. Field names are not case-sensitive. So, "price" is the same as "Price" is the same as "PRICE".
A type (discussed below).
A value (particular to the field type). Each named field may only appear once in a message. If multiple fields of the same name and type are needed, create an array field. A field in a nested message may have the same name as a field in the outer message, though.

Field types

The following field types (and arrays thereof) are supported by SDM:

Description	SDM Type	C Type
Boolean	LBMSDM_TYPE_BOOLEAN	`uint8_t`
8-bit signed integer	LBMSDM_TYPE_INT8	`int8_t`
8-bit unsigned integer	LBMSDM_TYPE_UINT8	`uint8_t`
16-bit signed integer	LBMSDM_TYPE_INT16	`int16_t`
16-bit unsigned integer	LBMSDM_TYPE_UINT16	`uint16_t`
32-bit signed integer	LBMSDM_TYPE_INT32	`int32_t`
32-bit unsigned integer	LBMSDM_TYPE_UINT32	`uint32_t`
64-bit signed integer	LBMSDM_TYPE_INT64	`int64_t`
64-bit unsigned integer	LBMSDM_TYPE_UINT64	`uint64_t`
Single-precision floating point	LBMSDM_TYPE_FLOAT	`float`
Double-precision floating point	LBMSDM_TYPE_DOUBLE	`double`
String	LBMSDM_TYPE_STRING	`char *`
Scaled decimal	LBMSDM_TYPE_DECIMAL	lbmsdm_decimal_t
Timestamp	LBMSDM_TYPE_TIMESTAMP	`struct timeval`
Nested message	LBMSDM_TYPE_MESSAGE	lbmsdm_msg_t `*`
Binary large object (BLOB)	LBMSDM_TYPE_BLOB	`void *`
Unicode string	LBMSDM_TYPE_UNICODE	`wchar_t *`

: Note that arrays are homogeneous. All elements of an array must be of the same type. An error is reported if an attempt is made to add an element of one type to an array containing elements of a different type.

Building a message: A message must be created (via lbmsdm_msg_create() or lbmsdm_msg_parse()) before fields can be added.

Once a field exists within a message, it can be referenced in one of three ways:

By the name associated with the field.
By index. This refers to the sequential position of the field within a message. The first field has index 0, the second has index 1, and so forth.
By iterator. See below for more information on iterators.

Adding fields to a message: Scalar (non-array) fields are added to a message via the lbmsdm_msg_add_xxx() API functions, where xxx is the type of the field being added. See the module Add a field to a message for information on these functions.

: When adding a field, data of the appropriate type must be supplied. As an example, to add a 32-bit signed integer field named "quantity" to a message:
int32_t quant = 50;
int rc;
rc = lbmsdm_msg_add_int32(msg, "quantity", quant);

: Alternatively, literals may be used to specify the value. The above example could also be coded as:
rc = lbmsdm_msg_add_int32(msg, "quantity", 50);

Adding array fields to a message: Array fields are added to a message in two steps. First, the field itself is added via the lbmsdm_msg_add_xxx_array() API functions, where xxx is the type of the field being added. This does not provide a value for the field. See the module Add an array field to a message for information on these functions.

: Second, individual elements are added to the array field. This is done via the lbmsdm_msg_add_xxx_elem_idx(), lbmsdm_msg_add_xxx_elem_name(), and lbmsdm_iter_add_xxx_elem() API functions. See the modules Add an element to an array field by field index, Add an element to an array field by field name, and Add an element to an array field referenced by an iterator for detailed information on these functions.

: As an example, the following code illustrates how to create a string array field, and add 3 elements to it.
rc = lbmsdm_msg_add_string_array(msg, "string_array");
rc = lbmsdm_msg_add_string_elem_name(msg, "string_array", "String1");
rc = lbmsdm_msg_add_string_elem_name(msg, "string_array", "String2");
rc = lbmsdm_msg_add_string_elem_name(msg, "string_array", "String3");

Serializing the message: Once the SDM message is constructed, it must be serialized for transmission. The API function lbmsdm_msg_get_data() returns a static pointer to a buffer containing the serialized form of the message, suitable for transmission. The length of the serialized data may be obtained via the API function lbmsdm_msg_get_datalen(). For example, a constructed message may be sent by:
rc = lbm_src_send(src, lbmsdm_msg_get_data(msg), lbmsdm_msg_get_length(msg), 0);

: The pointer returned by lbmsdm_msg_get_data() is owned by the SDM API, and will automatically be freed when the message is destroyed.

Deserializing a message: When a message is received, it must be deserialized so that individual fields can be accessed. This is done via the lbmsdm_msg_parse() API function:
lbmsdm_msg_t * sdmmsg;
rc = lbmsdm_msg_parse(&sdmmsg, lbmmsg->data, lbmmsg->len);

Disposing of a message: Once an SDM message (created by either the lbmsdm_msg_create() or lbmsdm_msg_parse() API calls) is no longer needed, it must be disposed of to avoid a resource leak. This is done via the lbmsdm_msg_destroy() API call.

Retrieving field information

A number of API functions are available to retrieve information about individual fields.

lbmsdm_msg_get_fldcnt() returns the number of fields in the message.
lbmsdm_msg_get_name_idx() and lbmsdm_iter_get_name() return the field name associated with the referenced field.
lbmsdm_msg_get_type_name(), lbmsdm_msg_get_type_idx(), and lbmsdm_iter_get_type() return the type of the referenced field.
lbmsdm_msg_get_elemcnt_name(), lbmsdm_msg_get_elemcnt_idx(), and lbmsdm_iter_get_elemcnt() return the number of elements in an array field.
lbmsdm_msg_get_len_name(), lbmsdm_msg_get_len_idx(), and lbmsdm_iter_get_len() return the length (in bytes) required for a field.
lbmsdm_msg_get_elemlen_name(), lbmsdm_msg_get_elemlen_idx(), and lbmsdm_iter_get_elemlen() return the length (in bytes) for a specific element in an array field.

Fetching fields from a message: When fetching a field from a message, the field may be referenced by name, by index, or via an iterator.

: Scalar (non-array) fields may be retrieved via the lbmsdm_msg_get_xxx_name(), lbmsdm_msg_get_xxx_idx(), or lbmsdm_iter_get_xxx() functions, where xxx is the type the field value should be retrieved as. The sections Get scalar field values by field name, Get scalar field values by field index, and Get a scalar field via an iterator contain detailed information on these functions.

: Array field elements may be retrieved via the lbmsdm_msg_get_xxx_elem_name(), lbmsdm_msg_get_xxx_elem_idx(), or lbmsdm_iter_get_xxx_elem() functions, where xxx is the type the field element value should be retrieved as. The sections Get an element from an array field by field name, Get an element from an array field by field index, and Get an element from an array field referenced by an iterator contain detailed information on these functions.

Type conversion

A limited form of automatic type conversion is provided. Numeric-based fields can be converted to other numeric-based fields. For example, a field defined as LBMSDM_TYPE_BOOLEAN may be retrieved as an LBMSDM_TYPE_FLOAT. The numeric types are:

boolean
int8
uint8
int16
uint16
int32
uint32
int64
uint64
float
double
decimal

Note that if the value being retrieved cannot be represented in the type it is retrieved as, the result is not defined, but no error is indicated. For example if a uint32 field with the value 300 is retrieved as an int8, the result is not defined.

Non-numeric types may not be converted to other types. The non-numeric types are:

string
unicode
timestamp
BLOB
message

: The above conversion rules apply also when retrieving array elements.

Fetching string, unicode, and BLOB values: When fetching a field or array element value as a string, unicode, or BLOB, the data is copied into a buffer provided by the application. In addition to the buffer, the size of the buffer must be given. The size is specified in bytes for string and BLOB fields, and in wchar_ts for unicode fields. If the size specified is too small for the data, the error code LBMSDM_INSUFFICIENT_BUFFER_LENGTH is returned.

Fetching message fields: When fetching the value of a message field, a copy of the message is created (via lbmsdm_msg_clone()) and returned. It is the application's responsibility to destroy the message (via lbmsdm_msg_destroy()) when it is no longer needed.

Modifying fields in a message: Existing fields in a message may be modified, both in terms of the field type and field value. For scalar (non-array) fields, the lbmsdm_msg_set_xxx_idx(), lbmsdm_msg_set_xxx_name(), and lbmsdm_iter_set_xxx() API functions may be used, where xxx is the type to be assigned to the field. See the sections Set a field value in a message by field index, Set a field value in a message by field name, and Set a field value in a message referenced by an iterator for information on these functions.

: For array fields, the lbmsdm_msg_set_xxx_array_idx(), lbmsdm_msg_set_xxx_array_name(), and lbmsdm_iter_set_xxx_array() API functions may be used, where xxx is the type to be assigned to the field. See the sections Set a field value in a message by field index to an array field, Set a field value in a message by field name to an array field, and Set a field value in a message, referenced by an iterator, to an array field. for information on these functions. As when adding an array field to a message, once the field type has been set to an array type, individual elements must be added to the array field.

: Individual elements of an array field may be modified via the lbmsdm_msg_set_xxx_elem_idx(), lbmsdm_msg_set_xxx_elem_name(), and lbmsdm_iter_set_xxx_elem() API functions. See the sections Set an array field element value by field index, Set an array field element value by field name, and Set an array field element value for a field referenced by an iterator for information on these functions. Note that arrays must contain homogeneous elements, so the type of an array element may not be changed, and is considered an error.

Deleting fields from a message: A field may be deleted from a message via the lbmsdm_msg_del_idx(), lbmsdm_msg_del_name(), and lbmsdm_iter_del() API calls. Deleting a field will cause any fields following it to be moved down one position, changing the index of those fields and potentially invalidating any iterators for that message.

Deleting elements from an array field: Individual elements may be deleted from an array field via the lbmsdm_msg_del_elem_idx(), lbmsdm_msg_del_elem_name(), and lbmsdm_iter_del_elem() API functions.

Null fields: SDM supports the concept of a null field. A null field is present in the message, but has no value associated with it. Once added to a message, a field may be set to null via the lbmsdm_msg_set_null_idx(), lbmsdm_msg_set_null_name(), or lbmsdm_iter_set_null() API functions. Setting the field (which may be either a scalar or array field) to null removes any values currently associated with the field.

: The lbmsdm_msg_is_null_idx(), lbmsdm_msg_is_null_name(), and lbmsdm_iter_is_null() API functions allow an application to determine if a given field is null.

: Attempting to retrieve a value or element value from a null field is not allowed, and will return an error.

Iterators: A field iterator allows sequential operation on the fields of a message without requiring the field name or index. An iterator is created via lbmsdm_iter_create(), the first field in a message is located via lbmsdm_iter_first(), and the next field is located via lbmsdm_iter_next(). An iterator should be destroyed when no longer needed, using the lbmsdm_iter_destroy() API call.

: Message fields may be queried, fetched, modified, and deleted via an iterator. In each case, the operation applies to the field currently referenced by the iterator.

Error information: All functions return a value to indicate the success of failure of the operation. Most return LBMSDM_SUCCESS to indicate success, or LBMSDM_FAILURE otherwise. Consult the individual function documentation for exceptions.

: The function lbmsdm_errnum() can be used to retrieve a detailed error code for the last error encountered, while lbmsdm_errmsg() will return a descriptive error message.

Message Options: The performance of SDM can be tuned through the use of message options. Options are contained within an attributes object (lbmsdm_msg_attr_t), which is created via lbmsdm_msg_attr_create(). When no longer needed, an attributes object can be discarded by calling lbmsdm_msg_attr_delete(). Individual options within an attributes object can be set via lbmsdm_msg_attr_setopt() and lbmsdm_msg_attr_str_setopt(), and can be queried via lbmsdm_msg_attr_getopt() and lbmsdm_msg_attr_str_getopt(). A set of options can be specified at message creation time using lbmsdm_msg_create_ex() and lbmsdm_msg_parse_ex().

The following table lists the supported message options.

Option Data type Allowed values Default Description

field_array_allocation

int

Any integer >= 0

32

Internally, SDM maintains an array of field entries within a message. This option controls both the number of field entries initially allocated when the message is created, and the increment used when the array must be expanded when a field is added but the array is full.

If it is known that a large number of fields will be added to a message, setting this option to a larger value will result in a slight performance boost, since reallocation of the field array will occur less frequently.

Similarly, if the number of fields to be added is small, setting this option to a smaller value will effect a small savings in memory usage (as well as reducing memory fragmentation).

name_tree

int

1 (enable) or 0 (disable)

1

In order to speed up access to fields by name, a tree of field names is maintained for each message. Maintenance of this tree (such as when fields are added or deleted) does incur some overhead.

In situations where fields are only added to a message (and not retrieved), this overhead can be eliminated by setting this option to 0. Be aware, however, that retrieving a field by name from a message for which the name tree is disabled, will degenerate to using a linear search of the field array to locate the named field. This can incur much more overhead than was saved by not using the name tree.

In addition, the use of the name tree allows SDM to detect (and prevent) the attempt to add two fields with the same name. Disabling the name tree also disables this safeguard. Further, any attempts to parse a message with duplicate field names using lbmsdm_msg_parse() or lbmsdm_msg_parse_ex() will fail and return an error.

integer_fieldname

int

1 (enable) or 0 (disable)

0

Some optimizations can be done within SDM if it is known that each field name will be the string representation of a non-negative integer. Rather than looking up by name, each field name is used as the index of the field, and name lookups can be faster.

This option requires each field name to be a string representation of a non-negative integer. Any attempts to add a field or parse a message containing a field name which is not the string representation of a non-negative integer, when this option is set, will fail and return an error.

: As an example, the following code fragment creates an attributes object, sets options, get the options, creates a message using the attributes object, then destroys the attributes object. For the sake of brevity, error checking has been omitted, as has the code to add fields to the message.
lbmsdm_msg_attr_t * attr;
lbmsdm_msg_t * msg;
int name_tree;
int alloc_size;
char val_buf[256];
size_t val_len;
lbmsdm_msg_attr_create(&attr);
name_tree = 0;
lbmsdm_msg_attr_setopt(attr, "name_tree", (void *)&name_tree, sizeof(name_tree));
lbmsdm_msg_attr_str_setopt(attr, "field_array_allocation", "128");
val_len = sizeof(val_buf);
lbmsdm_msg_attr_str_getopt(attr, "name_tree", val_buf, &val_len);
printf("name_tree=%s\n", val_buf);
val_len = sizeof(alloc_size);
lbmsdm_msg_attr_getopt(attr, "field_array_allocation", (void *)&alloc_size, &val_len);
printf("field_array_allocation=%d\n", alloc_size);
lbmsdm_msg_create_ex(&msg, attr);
lbmsdm_msg_attr_delete(attr);