51. Release LBM 3.4 / UME 2.1 - February 2009

New Monitoring Statistics: Added Context statistics and Event Queue statistics. Context statistics help you monitor topic resolution activity, along with the number of unknown messages received and the number of sends and responses that were blocked or returned EWOULDBLOCK. Context statistics also contain transport statistics for Multicast Immediate Messaging (MIM) activity and the existing transport statistics for all of the sources or receivers in a context. Event Queue statistics help you monitor the number of events currently on the queue, how long it takes to service them (maximum, minimum and mean service times) and the total number of events for the monitoring period.

Implementing the new context and Event Queue statistics in an existing lbmmon statistics receiver application requires the addition of a callback for each statistic type. All new statistics are available in the C API, the Java API and the .NET API. See also LBM API Functions and Data Structures in LBM Design Concepts. The new statistics have also been added to the Ultra Messaging® MIB and the InterMapper probe files.

Application Change Required: LBM Version 3.4 has broken source and binary compatibility with monitoring applications that use a monitoring receiver controller. In order for your existing monitoring application to work with LBM Version 3.4, you must change the application code and re-compile. This change, although troublesome now, allows the addition of more statistics in the future without any further application change required.

In the C API, the method for specifying all statistics callbacks has been changed. Note that this change applies only to applications that act as a receiver of statistics. No changes are required for most applications that use the lbmmon API to publish statistics. See the lbmmon API documentation for information about the new parameters passed to lbmmon_rctl_create(), as well as the new API calls (lbmmon_rctl_attr_*()) along with the structures (lbmmon_rctl_attr_t and lbmmon_*_statistics_func_t. See also Receiving Monitoring Data in LBM Design Concepts for code examples that use the new lbmmon API calls to create a statistics receiver.

The Java API and .NET API also require a new way to specify statistics callbacks for all statistics. However, callbacks for existing statistics are now deprecated and will continue to work until an undetermined future release.

Added API functions to reset statistics for all functions that retrieve statistics. For more information, see the C API, the Java API and the .NET API.

Added a new monitoring function, lbmmon_sctl_sample_ex(). This is an extension to the existing lbmmon_sctl_sample() call, but allows an AppID string to be specified. This string overrides the AppID string specified in any lbmmon_*_monitor() calls for this call only, and can be used to allow dynamic identification of the statistics message (via the AppID) to the receiving monitoring application.

Added the capability to monitor an LBM context automatically by setting LBM Automatic Monitoring Options in a configuration file or by setting environment variables, which allow you to automatically monitor all LBM contexts. See also Automatic Monitoring in LBM Design Concepts.

Added the number of retransmission bytes sent (rx_bytes_sent) to the LBT-RM and LBT-RU source statistics. This allows the calculation of a source's retransmission rate as well as the data rate. This new statistic has also been added to the Ultra Messaging MIB and the InterMapper probe files. The Ultra Messaging MIB has also been restructured to the following order: Index fields (OIDs .1 through .10), auxiliary fields (OIDs .11 through .20) and the normal statistics fields beginning with OID .21.

Added new API attribute management functions (lbm_XXX_attr_create(), lbm_XXX_attr_delete(), lbm_XXX_attr_dup(), and lbm_XXX_attr_default()). Now attribute structures can only be allocated, manipulated, copied or destroyed via the new API functions. The existing functions, lbm_XXX_attr_init() and lbm_XXX_attr_cleanup(), still work but have been deprecated. All new development should be done with the new functions. When the deprecated functions are removed in a future release, all lbm_XXX_attr_t structures will become completely opaque, and the individual fields will be removed from lbm.h.

Added a new API function lbm_src_flush(), which allows you to flush all batched messages from both the implicit and explicit batch buffer. For more information, see the LBM C API, the LBM Java API and the LBM .NET API.

Added context source event callbacks. LBM now delivers source events for any message types that can be sent directly from a context rather than requiring a separate lbm_src_t (or LBMSource) object. These message types include multicast immediate messages, unicast immediate messages, requests and responses. Currently, the only supported context source event is the wakeup source event type. Also, wakeup source event types for all sources now include a flag field in the event data to indicate what type of source has woken up.

Added new API functions to retrieve current attribute settings for various LBM objects.

Added C API functions (lbm_*_dump()) and Java and .NET methods (dumpAttributeList) that retrieve an LBM object's (context, receiver, Event Queue, etc.) current LBM configuration options.

A new C API function, lbm_context_delete_ex(), may be used with an Event Queue cancel callback to provide the application notification that all context-level source events have been processed or purged from any Event Queues. It is similar to the existing *_delete_ex() C API functions.

Added get_sec() and get_usec() to Java and .NET Self Describing Messaging (SDM). Now you can easily get the values for seconds and microseconds for a timestamp (LBMSDMRawTimestamp.java and LBMSDMRawTimestamp.cs).

Added the ability to Java and .NET Self Describing Messaging (SDM) to set the size of the preallocated field array when you create messages with LBMSDMFieldsAttribute.set_field_prealloc(). In the C API, this corresponds to the option field_array_allocation used with lbmsdm_msg_attr_str_setopt().

Corrected a problem in the .NET LBMSDMRawDecimal class that was misinterpreting the exponent as an unsigned number instead of a signed number when deserializing a message.

Corrected a problem with Java and .NET Self Describing Messaging (SDM) that prevented the use of strings greater than 32K in length. SDM strings can now be greater than 32K.

Corrected a problem with Self Describing Messaging (SDM) that occurred when building the serialized form of the message. If the initial estimate of the serialized message length was too small, the output buffer was reallocated. In the cases where the reallocation caused the memory block to be moved, rather than simply extended, this could lead to a write to an invalid memory location.

Updated JNI to catch exceptions in objects created from JNI. If an exception occurs, the exception is logged and the current operation is terminated.

Corrected a problem with Java timers discovered when using the Java example application, lbmreq with Event Queues. Now when an LBMTimer object expires and goes out of scope, it will be finalized and collected immediately.

Fixed a memory corruption problem in the Java API that caused crashes during cleanup of transport sessions for receivers using LBM Event Queues. This problem only occurred on 32-bit systems.

Fixed race conditions with Java LBMTimers that are constructed without appropriate callbacks being set.

Added additional null checks in the Java and .NET APIs to prevent null pointer exceptions when handling messages containing empty arrays.

Fixed an issue with .NET and Java that caused Hot Failover Receiver statistics to be empty. Also fixed .NET and Java lbmrcv example programs that printed incorrect values for messages received (msgs_rcved) when using the TCP transport and periodic statistics collection.

Added LBMContextAttributes.setImmediateMessageCallback() function in Java and .NET to allow setting a context's topic-less immediate message callback before context creation. This method is now preferred over the older LBMContext.enableImmediateMessageReceiver() and LBMContext.addImmediateMessageReceiver() methods.

Updated Java and .NET finalizers to catch unexpected exceptions and dump the stacktrace of the exception to the standard Java API printStackTrace (stderr).

Updated LBMReceiver constructors in Java and .NET to throw an LBM.EINVAL exception if they receive an LBMTopic object that has already been used in the creation of an LBMReceiver. Re-using LBMTopic objects in this manner has always been unsupported by LBM, but no error was logged previously.

Process-wide LBM configuration option defaults may now be automatically loaded at startup without code changes by setting the environment variable LBM_DEFAULT_CONFIG_FILE to the path (or URL) of an LBM configuration file.

Added the source_event_function context configuration option as an alternative way to set a context's topic-less immediate messaging callback. The older method, using the lbm_context_rcv_immediate_msgs API call, is still supported, but not recommended for new code.

Added warning message to rcv_create functions if the transport_lbtrm_activity_timeout is less than transport_lbtrm_nak_generation_interval. The warning states that this condition results in silent data loss if loss occurs within the activity timeout interval prior to the end of the transport session.

Added two new configuration options, transport_lbtrm_nak_initial_backoff_interval and mim_nak_initial_backoff_interval, which control latency due to loss. The default for both options is 50 milliseconds. A value of zero for either option turns on immediate NAKs.

Added a new Wildcard Receiver option receiver_create_callback that allows a callback when a Wildcard Receiver matches a topic and is about to create an actual receiver for the topic. This allows, among other things, an application to modify the receiver attributes to be used in creating the receiver.

Removed the explicit setting of request_tcp_port from the C example application, lbmreq. lbmreq now uses the request_tcp_port_low and request_tcp_port_high range.

lbm_config, LBM.setConfiguration() and lbmaux functions can now load configuration files from a URL using HTTP or FTP. (e.g. lbm_config http://www.example.com/lbm.cfg).

Corrected a problem with an LBT-RU source with source-side filtering enabled that would cause a segfault if the source option transport_lbtru_client_activity_timeout was set to less than 10 seconds, and a timeout was detected after a receiver connect.

Fixed an error where held responses could trigger a memory read error. "Held responses" refers to an LBM response or Unicast Immediate Message for which the response object has been retained after the response has been sent, and subsequently, the requestor disconnects. The memory read error occurs upon the next response sent with the response object.

Upon expiration of the response_tcp_deletion_timeout, logic was added to ensure that the response message had been sent. If not, LBM restarts the timer to give additional time for the send to complete.

Corrected a problem on Mac OS X that resulted in a bind failure of the request_tcp_interface or the response_tcp_interface. The bind failure prevented the initialization of the LBM context and produced the message, could not find open TCP server port in range. This problem was most prevalent in Java applications, but could also occur in C applications.

Corrected a problem with sending responses that returned an LBM_EOP error, response channel disconnected unexpectedly due to the selection of a disconnected channel that could also result in the access of freed memory. Now LBM ignores disconnected response channel connection objects to avoid inadvertently referencing freed memory. This correction forces responses sent to a disconnected channel (requester) to re-establish the connection, instead of failing with the LBM_EOP error, response channel disconnected unexpectedly.

Corrected the condition that prevented the flushing of the message batching buffer with the LBM_MSG_FLUSH flag after starting a message batch with the LBM_MSG_START_BATCH flag. The LBM_MSG_FLUSH flag now flushes the buffer, but LBM continues to batch messages until it receives the LBM_MSG_END_BATCH flag. Additionally, if LBM receives a LBM_MSG_COMPLETE_BATCH flag without previously receiving a LBM_MSG_START_BATCH flag, it does not report an error.

LBM now supports setting the SO_RCVBUF and SO_SNDBUF on a Stratus VOS system. If running on a version of VOS that does not support this functionality, LBM logs a warning. You may set these parameters via LBM configuration options ending in socket_buffer.

LBM now conforms to the POSIX standard when sending a message on a Stratus VOS system. Previously, VOS 17.x did not function properly with LBM.

LBM now allows the mim_src_deletion_timeout option to increase as needed to ensure the last message has completed. Previously, when sending large immediate messages, the message could take longer to send than the mim_src_deletion_timeout default, which resulted in a failure.

LBM rate limiter now supports values more than 4 gigabits/sec. Previously, rate limits of more than 4 gigabits per second were not supported on 32-bit or 64-bit Microsoft® Windows®, and were not supported on other 32-bit platforms. Also, configuration file parsing for rate limiter settings has been changed to allow larger (64-bit) values.

LBM now reports an error if you configure Wildcard Receiver and Event Queue options with non-numeric values. Previously LBM would assign a value of zero without any notification.

Reactivated the Linger option (linger for xx seconds before closing context) for the Java source code example, lbmimsg.java.

Corrected how LBM Debug enables certain debug levels that resulted in segmentation faults on Solaris.

In a few error cases LBM aborts an operation and returns without unlocking its mutex locks. LBM has been updated to unlock mutex locks before returning from an error.

Fixed an issue where a partial message could be delivered after a burst loss notification. As a result of this change, if any portion of a message is lost, the entire message is considered lost.

Changed the error message produced when a receiver goes down prior to an application sending a Unicast Immediate Message. Error message now reads, LBM send error: no connection to target while sending unicast immediate message.

Changed TCP-LB to not send partial messages. When using Bounded Latency TCP and sending large fragmented messages, if LBM needs to remove data from the buffer to make room for new data, LBM removes all fragments of a message so the receiver does not receive any partial messages.

Fixed a problem when using LBT-RU on Microsoft Windows in conjunction with the wincompport fd_management_type. Previously, a single receiver disconnecting from a topic caused all other receivers for that topic to stop receiving data.

Corrected a problem in the LBM Gateway that prevented messages from being forwarded across the Gateway if you used a non-default value for the Unicast topic resolution port in tunnel mode.

Corrected a problem with TCP transport that occurred when the transport_tcp_multiple_receiver_behavior was set to bounded_latency or source_paced. The "total buffered" counter increased when data was added to the buffer but was not properly decreased when data was removed. Over time, this resulted in the failed assertion [src->totalbuffered>=len]. This counter supplies the bytes_buffered source transport TCP statistic and does not impact the performance of LBM.

Corrected a problem with lbm_log that resulted in a system crash when lbm_log was called multiple times on machines with multiple interfaces.

Fixed a condition on Microsoft Windows that caused a blocking MIM send to return immediately without sending any data if it followed a non-blocking send which had returned LBM_EWOULDBLOCK.

Corrected a problem with LBT-RU and source-side filtering that caused a deleted and then re-created receiver to connect to the source but not receive any messages.

Fixed a .NET marshalling issue affecting LBM.version(), LBM.errorMessage() and LBMMonitor.errorMessage() on some 64-bit systems which resulted in crashes when calling these functions or if the .NET API attempted to throw an exception.