Release Notes

UM Version 6.12

The most significant update to UM version 6.12 is the introduction of a TCP-based Stateful Topic Resolution Service (SRS) which optimizes the Topic Resolution Process.

Attention
  There are some special upgrade instructions for UM version 6.12 that will affect a small percentage of users. See Special Upgrade Instructions for 6.12. For general upgrade instructions, see Upgrade Procedure. You are also advised to inspect the full list of Deprecations.

# Enhancements for 6.12 <-

## Streaming Enhancements for 6.12 <-

The following new features and enhancements apply to UMS, UMP, and UMQ products.

- TCP-Based Topic Resolution. 6.12 introduces a new component, the SRS (Stateful Resolution Service). This is the first phase of a plan to significantly improve UM's topic resolution characteristics. The SRS service can be run on 64-bit Windows and 64-bit Linux. (An SRS running on either one of those platforms can service clients across the full range of supported platforms.)

  For more information, see Topic Resolution Description.

- Streaming Receiver Optimization. Message receive latency has been improved with the following features:

  - Receive Buffer Recycling
  - Single Receiving Thread
  - Context Lock Reduction
  - Gettimeofday Reduction

- Smart Source Message Sizing Flexibility. The Smart Sources feature now supports greater flexibility in application message sizes, including:

  - UM Message Fragmentation.
  - User Supplied Buffers.

  For more information, see Smart Source Message Buffers.

  Also, a problem in Smart Source was fixed (see Fixed Limitation 10567) which users of Smart Source need to be aware of. Some users will need to change their configuration as a result. See Smart Source Header Size Change for details.

- Windows Service Enhancements. Support for running the 3 main pre-6.12 UM Daemons as Windows Services has been improved. This includes the UM Message Router (DRO), the persistent Store, and the unicast topic resolution daemon (lbmrd). The new service, SRS, is introduced with the same support for Windows Service. (Note that the Ultra Messaging Manager daemon `ummd` is not available as a windows service at this time.) For more information, see UM Daemons as Windows Services.

- XSP Enhancements. The Transport Services Provider (XSP) feature has been enhanced to support the following new capabilities:

  - Registration of File Descriptors. The Transport Services Provider (XSP) feature has been enhanced to support registration of application file descriptors. For more information, see Other XSP Operations.
  - Persistence. Persistent receivers can now be assigned to a Transport Services Provider (XSP).

- Leading Loss Events Suppressed. UM no longer delivers unrecoverable loss events to receiver callbacks prior to the first message delivery. See bug 5199.

- AIX. Informatica no longer ships the AIX platform with every new version of UM. Instead, AIX is now considered an "on demand" platform. Note that AIX is still a supported platform for UM. Customers of the AIX platform may specifically request that an AIX build be generated for the most-recent general UM version.

- Daemon Platform Deprecations. The original 6.11.1 release notes announced the deprecation of certain platforms for UM daemons, claiming that they would be discontinued in UM version 6.12. However, some users requested more time for the transition. Starting with UM version 6.12, these deprecated daemon platforms have not yet been removed, and are therefore still fully supported in 6.12. Starting with UM version 6.13, the UM daemons will no longer be available on certain platforms. See Platform Deprecations for Daemons.

## Persistence Enhancements for 6.12 <-

The following new features and enhancements apply to UMP and UMQ products.

- XSP. Persistent receivers can now be assigned to a Transport Services Provider (XSP).

- Receiver re-registration. Persistent receivers can now re-register with a store immediately following certain events. See bug 5890.

## Queuing Enhancements for 6.12 <-

The following new features and enhancements apply to the UMQ product.

- Ultra Load Balancing Throughput. The Ultra Load Balancing (ULB) feature has been enhanced to significantly improve throughput when used with unicast transport protocols TCP and LBT-RU. For more information see ULB Performance.

## Dynamic Router Enhancements for 6.12 <-

The following new features and enhancements apply to the Dynamic Routing Option (DRO).

- None.

# Fixed Problems and Limitations for 6.12 <-

## Streaming Fixed Problems and Limitations for 6.12 <-

The following bug fixes apply to UMS, UMP, and UMQ products.

Change Request | Description
--- | ---
10567 | FIXED: Smart Sources reserved a differing amount of space for worst-case headers than traditional sources. For users of kernel bypass network drivers like Onload, this can lead to having to set the transport_lbtrm_datagram_max_size (context) option to different values for apps using Smart Sources vs. apps that do not. For apps that use both, this creates a sub-optimal solution for one or the other. UM now uses the same reserve size for Smart Sources as it always has for traditional sources. Attention Users of Smart Source may need to update their configuration because of this. See Smart Source Header Size Change.
10542 | FIXED: When an XSP user's transport_mapping_function callback is invoked, the second parameter is: `"lbm_new_transport_info_t *info"` which has the field `"char source[LBM_MSG_MAX_SOURCE_LEN];"`. However, that field does not contain the source string. Note To pre-6.12 users who worked around this problem by defining `"LBM_INTERNAL_USE_ONLY"`: your application will continue to work properly with 6.12 without any need to recompile. However, we do recommend that you remove the LBM_INTERNAL_USE_ONLY definition when it is convenient.
5199, 9415 | ENHANCEMENT: There are a variety of circumstances where a newly-created streaming receiver will be delivered unrecoverable loss events before the first message is received. This is normally not useful to the application since it does not represent a gap in the received message stream, but rather just difficulty in getting successfully joined to the stream. UM now will suppress unrecoverable loss events prior to the first message delivery. Informatica believes that this new behavior is the correct behavior for UM, and that there should be no cause for concern by users. This behavior change does not apply to Persisted data streams where delivery of all messages is important. See Leading Loss for more information.
3398 | FIXED: Windows-based UM closes TCP connections abruptly, causing TCP "RST" packets on the network, and "connection reset" warnings in UM log files. Windows-based UM now closes TCP connections gracefully.
10518 | FIXED: When installing UM on a Windows machine using the UM package installer, the system PATH can be corrupted or cleared if the updated length is greater than 1023 bytes. This is related to a limitation in the Windows installer that UM uses; it fails on long PATH content. UM had a bug in which UM did not detect the failure and instead wrote the bad PATH. UM has been updated to detect the failure, issue an error, and not update the PATH.
10380, 10381 | FIXED: lbm_rcv_delete_ex and lbm_wildcard_rcv_delete_ex can crash when unsubscribing from spectrum channels.
10469, 10486, 10466 | FIXED: When sending a message using a Smart Source send API, UM can crash with a failed assertion of the form: `"failed assertion [...-\>sminfo.timer_id == -1]"`. This happens when a TSNI is in the process of being sent by the context thread at the same time that the application sends a message.
10492 | FIXED: When the lbmrd is configured for incorrect interface specification, either in its XML configuration file, or on the command line, the lbmrd did not detect any error and appears to be running. However, it does not function. Also, the lbmrd does not accept interface specifications other than a simple IP address. The lbmrd interface can now be specified as a simple IP address, a CIDR network specification (e.g. 10.1.2.0/24), a quoted device name (e.g. "eth0"), or a DNS host name. The lbmrd will report an error if an invalid interface specification is configured, including 0.0.0.0.
10288 | FIXED: Sending Unicast Immediate (UIM) Requests on Sparc machines can result in high CPU burn with no messages being sent.
10602 | FIXED: The `tnwgdmonmsgs.h` header file was omitted from the distribution package. This made it impossible to write daemon stats monitoring applications.
10519 | FIXED: The Unix-based UMS and UMP products were built such that liblbm.so had a dependency on libqpid-proton.so.2, which was not included in the package. This did not cause improper behavior because the UMS and UMP products never attempt to call any of the missing entry points. However, having the dependency is unnecessary. The UMS and UMP dependencies to libqpid-proton.so.2 have been removed.
10490, 10491 | FIXED: The log message: `Core-5688-447: LBMC datagram malformed.` does not give any information about the sender of the malformed datagram. The IP and port of the sender is now included in the log message.
10456 | FIXED: The configuration options zero_transports_function (xsp) and operational_mode (xsp) cannot be specified in a "flat" UM configuration file. The error: `CoreApi-5688-61: object xsp not supported` is logged.
10398 | FIXED: On many Linux systems, the `"Makefile.unix"` shipped with the example programs can produce a build error of the form: /usr/bin/ld: ....o: undefined reference to symbol 'sqrt...' The solution was to add "-lm" to the builds of the examples.
10383 | FIXED: Due to an improper installation of Solarflare's `"Onload"` library, certain socket calls can return an error. UM reported this error with the unhelpful log: `Core-5688-1881: handle events returned error 0` UM has been changed to properly decode and print the proper error. In addition, if the socket reports that the `"recvmmsg()"` function is not implemented, UM disables the multiple_receive_maximum_datagrams (context) feature.
9972 | FIXED: If an application attempts to unsubscribe from a Spectrum channel that it had not previously been subscribed to, the application crashes with an illegal memory access.
9798 | FIXED: If a receiver of a TCP or IPC transport gets messages out of order due to the data coming through a DRO and the originating source is UDP-based, the receiving application declares immediate unrecoverable loss, even though the missing messages arrive shortly thereafter. The TCP transport receiver can now be configured to wait for a period of time before declaring unrecoverable loss. This is controlled by two new configuration options: transport_tcp_dro_loss_recovery_timeout (receiver) and transport_lbtipc_dro_loss_recovery_timeout (receiver). See DRO Reliable Loss for more information.
10435 | FIXED: UM does not properly time out the total Late Join recovery process. This can lead to premature termination of Late Join recovery. UM now correctly calculates the total Late Join recovery timeout as late_join_info_request_interval (receiver) multiplied by late_join_info_request_maximum (receiver).
10640 | FIXED: A timing window exists that could cause Late Join to not complete. This can occur if the max number of late join requests has been completed before the total Late Join recovery process times out.

## Persistence Fixed Problems and Limitations for 6.12 <-

The following bug fixes apply to UMP and UMQ products.

Change Request | Description
--- | ---
5890, 9054 | ENHANCEMENT: There are certain circumstances where a receiver has to unnecessarily wait for the ume_activity_timeout (receiver) to expire before it can register with a Store. An enhancement was made to persistent receivers that allows for immediate re-registration in the following circumstances: A persistent source application is started and takes over from a proxy source. A persistent source application using a session ID re-registers with the Store. A receiver process is restarted and it has the same session ID, request_tcp_interface (context), and request_tcp_port (context) values.
10446, 10450 | FIXED: When applications and/or Stores are restarted in certain orders, the receiver can log the warning, `"Core-9743-08: WARNING: proactive keepalive is not supported at the store"`. This can happen even though the Store does support the Proactive Keepalive feature.
10379 | FIXED: When a persistent source deregisters from the Store by API call (lbm_src_ume_deregister()), the Store will occasionally crash.
10617 | FIXED: The Store code sometimes reads an uninitialized memory location. No improper behavior has been seen when this happens, but there is a very small chance that a receiver's outstanding messages will not be properly acknowledged, leading to potential unnecessary message recovery if the receiver restarts. This has never been observed in the field.
10564 | FIXED: When a Smart Source is configured for persistence, incorrect internal handling can lead to incorrect Forced Reclaims.
10462, 10432, 10482 | FIXED: After a set of multiple store restarts without clearing the state and cache files, there can be an extended period of topic deafness where persistent receivers will not see messages sent by the source.
10454 | FIXED: When an RPP store is writing to disk due to a non-blocking receiver exiting, and the disk fills and needs to wrap, the store can stop accepting new messages.
10447, 10451, 10458 | FIXED: If a source sends messages and then immediately deregisters, there is a chance that the Store will crash with an illegal memory access.
10343 | FIXED: When a Smart Source is publishing a persisted message stream, and there is a large amount of network packet loss, the publisher can crash with the log: `FATAL: failed assertion [info->sqn == ctlr->trail_sqn]`
10341 | FIXED: Minor misspelling in Store log message: `Store-8079-10: Staring store...` (Should be "Starting"; the second "t" is missing.)

## Queuing Fixed Problems and Limitations for 6.12 <-

The following bug fixes apply to the UMQ product.

Change Request | Description
--- | ---
9974 | FIXED: There was a scenario where a late arriving ACK from a ULB receiver could cause the ULB source to crash with an illegal memory access inside `lbm_ulb_src_ctlr_handle_ack()`.
10548 | FIXED: Starting in UM 6.11, a non-fatal assertion is generated when shutting down a brokered queue receiver: `WARNING: failed assertion [broker->rcv_list->sz==0]`
8899 | FIXED: A ULB source will get stuck in an infinite loop if umq_ulb_application_set_message_reassignment_timeout (source) is set to zero and one of the receivers exits.
3632, 8596, 8525 | FIXED: If you attempt to disable ULB message reassignment by setting option umq_ulb_application_set_message_reassignment_timeout (source) to '`x:0`', and option umq_ulb_application_set_message_lifetime (source) is already set to the default value of 0, in-flight messages are not discarding. As a workaround, to inhibit message reassignment use option umq_ulb_application_set_message_max_reassignments (source) set to 1, and leave umq_ulb_application_set_message_reassignment_timeout (source) at its default value. This has now been fixed. The workaround is no longer needed.

## Dynamic Router Fixed Problems and Limitations for 6.12 <-

The following bug fixes apply to the Dynamic Routing Option (DRO).

Change Request | Description
--- | ---
10443 | FIXED: When a DRO portal is queuing outgoing messages (perhaps due to a low rate limiter or a low-bandwidth link), and a topic has more than delivery_control_maximum_burst_loss (receiver) (default 1024) messages queued to be sent, there is a possibility that receivers will deliver a burst loss event to the application and not deliver the queued messages, even though they are eventually sent. This is because the originating source sent a TSNI, and the DRO allowed it to bypass the data queue and be sent before the queued data messages. Thus the receiver sees a gap greater than the burst loss configuration and does not wait for the missing data to be sent. The DRO now queues TSNIs behind data messages.
10499 | FIXED: The DRO configuration element <route-info> has the attribute `"max-hop-count"` which is documented as defaulting to 100. However, the DRO did not properly initialize its internal default, which had the effect of defaulting to 255. The DRO now properly defaults `"max-hop-count"` to 100.
3548 | FIXED: The DRO cannot run as a standalone Windows service. Now fixed. See UM Daemons as Windows Services.

# Special Upgrade Instructions for 6.12 <-

## Smart Source Header Size Change <-

A problem in Smart Source was fixed (bug 10567) which users of Smart Source need to be aware of.

When a kernel bypass network driver is used, users often change their datagram max size larger than a network MTU to ensure that UM data packets are efficiently filled. See Datagram Max Size and Network MTU for details.

However, because of bug 10567, users had to set the datagram max size to different values depending on whether their application used Smart Sources or traditional sources. This was particularly a problem for applications that make use of both Smart Sources and traditional sources; no value was optimal for both.

The fix for 10567 was to have Smart Sources reserve the same amount of space for headers as traditional sources.

Warning
  For users who have set the datagram max size larger than an MTU, you may need to reduce the configured value to avoid IP fragmentation in UM version 6.12 and beyond.

As described in Setting Datagram Max Sizes High, you will need to experimentally determine the optimal value for datagram max size for your use case.

## Previous Special Instructions <-

If you are upgrading from a UM version prior to 6.10, you must also examine the Special Upgrade Instructions for 6.10.

Also see Deprecations.