Configuration Guide
Transport Acceleration Options

Transport acceleration options enable kernel-bypass acceleration in conjunction with the following vendor solutions:

  • Myricom® Datagram Bypass Layer (DBL™)
  • Solarflare® Onload
  • Mellanox® 10-Gigabit Ethernet or InfiniBand hardware


Myricom® Datagram Bypass Layer (DBL™)  <-

DBL is a kernel-bypass technology that accelerates sending and receiving UDP traffic and operates with DBL-enabled Myricom 10-Gigabit Ethernet adapter cards for Linux and Microsoft® Windows.

DBL does not support fragmentation and reassembly, so do not send messages larger than the MTU size configured on the DBL interface.

DBL acceleration is compatible with the following Ultra Messaging transport types:

  • LBT-RM (UDP-based reliable multicast)
  • LBT-RU (UDP-based reliable unicast)
  • Multicast Immediate Messaging
  • Multicast Topic Resolution

To use DBL Transport Acceleration, perform the following steps:

  1. Install the Myricom 10-Gigabit Ethernet NIC.
  2. Install the DBL shared library.
  3. Update your search path to include the location of the DBL shared library.
  4. Set option transport_*_datagram_max_size and option resolver_datagram_max_size (context) to a value of no more than 28 bytes smaller than the Myricom interface's configured MTU size.


Reference  <-


dbl_lbtrm_acceleration (context)  <-

Flag indicating if DBL acceleration is enabled for LBT-RM transports.
See Myricom® Datagram Bypass Layer (DBL™).
Scope: context
Type: int
When to Set: Can only be set during object initialization.
Version: This option was implemented in LBM 4.0.

Value Description
1

DBL acceleration is enabled for LBT-RM.

0 DBL acceleration is not enabled for LBT-RM. Default for all.


dbl_lbtru_acceleration (context)  <-

Flag indicating if DBL acceleration is enabled for LBT-RU transports.
See Myricom® Datagram Bypass Layer (DBL™).
Scope: context
Type: int
When to Set: Can only be set during object initialization.
Version: This option was implemented in LBM 4.0.

Value Description
1

DBL acceleration is enabled for LBT-RU.

0 DBL acceleration is not enabled for LBT-RU. Default for all.


dbl_mim_acceleration (context)  <-

Flag indicating if DBL acceleration is enabled for multicast immediate messaging (MIM).
See Myricom® Datagram Bypass Layer (DBL™).
Scope: context
Type: int
When to Set: Can only be set during object initialization.
Version: This option was implemented in LBM 4.0.

Value Description
1

DBL acceleration is enabled for MIM.

0 DBL acceleration is not enabled for MIM. Default for all.


dbl_resolver_acceleration (context)  <-

Flag indicating if DBL acceleration is enabled for topic resolution.
See Myricom® Datagram Bypass Layer (DBL™).
Scope: context
Type: int
When to Set: Can only be set during object initialization.
Version: This option was implemented in LBM 4.0.

Value Description
1

DBL acceleration is enabled for topic resolution.

0 DBL acceleration is not enabled for topic resolution. Default for all.


Solarflare® Onload  <-

Solarflare Onload is a kernel-bypass technology that accelerates message traffic and operates with Solarflare 10GbE Ethernet NICs.

Note
Onload does not support fragmentation and reassembly, so do not send messages larger than the MTU size configured on the Solarflare interface.
Warning
Onload does not support both accelerated and non-accelerated processes subscribing to the same multicast group on the same host. An attempt to do so will result in the non-accelerated process becoming "deaf" to the shared multicast group. See "Onload User Guide", chapter 9.11 "Multicast Operation and Stack Sharing", sub-section "Multicast Receive ‐ Onload Stack and Kernel Stack".

Ultra Messaging loads the Onload library dynamically during Ultra Messaging initialization on the following Ultra Messaging platforms:

  • Linux-glibc-2.3-i686
  • Linux-glibc-2.3-x86_64 Linux-glibc-2.5-x86_64

Onload default behavior accelerates all sockets. You can access the Onload onload_set_stackname API extension to select the sockets you want to accelerate by using UM configuration options. Selecting sockets with a stackname lets you accelerate data transmission sockets and not sockets for control messages, topic resolution, or responses.

You can select a stackname with the configuration options onload_acceleration_stack_name (receiver) and onload_acceleration_stack_name (source) for the following Ultra Messaging transport types:

  • LBT-RM (UDP-based reliable multicast)
  • LBT-RU (UDP-based reliable unicast)
  • TCP
Note
If you set the LBM_SUPPRESS_ONLOAD environment variable to any value, Ultra Messaging does not dynamically load the Onload library at runtime. In this case, you cannot use the onload_acceleration_stack_name options.

If you use the onload_set_stackname API directly for any other accelerated sockets, note that after Ultra Messaging accelerates a transport socket, Ultra Messaging resets the stackname to the default for all threads by calling:

onload_set_stackname(ONLOAD_ALL_THREADS, ONLOAD_SCOPE_NOCHANGE, "");

Ultra Messaging resets the stackname during source creation and when a receiver matched topic opens a transport session.

To enable Onload socket acceleration for selected transports, perform the following steps:

  1. Install Onload.
  2. Set the Onload environment variable EF_DONT_ACCELERATE = 1 to disable Onload default behavior.
  3. To enable acceleration for all applications in an environment, export the following environment variable:

    export LD_PRELOAD=libonload.so

  4. To enable acceleration on a per-application basis, start the application as in the following example:

    onload <app_name> [app_options]

  5. Set UM configuration option onload_acceleration_stack_name (source) according to the thread the source uses.

    Note: Disable batching to ensure that it is the application thread that sends the data out.

  6. Set UM configuration option onload_acceleration_stack_name (receiver) according to the thread the receiver uses.

    Note: Receiver transports might not share the same thread if MTT is enabled.

  7. Set option transport_*_datagram_max_size and option resolver_datagram_max_size (context) to a value of no more than 28 bytes smaller than the Solarflare interface's configured MTU size.

For detailed information about onload stack names, refer to the Solarflare® Onload User Guide.


Reference  <-


onload_acceleration_stack_name (receiver)  <-

The stackname to use when creating an OpenOnload transport data socket.
The stackname must be eight characters or less. Because this is a transport setting, the first receiver applies its configuration option setting, and other receivers that join the transport inherit the setting of the first source. To disable the stackname, set this option to NULL, which must be all uppercase.
Note: Use of this option requires Solarflare OpenOnload and applies to LBT-RM, LBT-RU, and TCP transports.
Scope: receiver
Type: string
Default value: NULL
When to Set: Can only be set during object initialization.
Version: This option was implemented in UM 6.5.


onload_acceleration_stack_name (source)  <-

The stackname to use when creating an OpenOnload transport data socket.
The stackname must be eight characters or less. Because this is a transport setting, the first source applies its configuration option setting, and other sources that join the transport inherit the setting of the first source. To disable the stackname, set this option to NULL, which must be all uppercase.
Note: Use of this option requires Solarflare OpenOnload and applies to LBT-RM, LBT-RU, and TCP transports.
Scope: source
Type: string
Default value: NULL
When to Set: Can only be set during object initialization.
Version: This option was implemented in UM 6.5.


UD Acceleration for Mellanox® Hardware Interfaces  <-

UD (Unreliable Datagram) acceleration is a kernel-bypass technology that accelerates sending and receiving UDP traffic and operates with Mellanox 10-Gigabit Ethernet or InfiniBand adapter cards for 64-bit Linux on X86 platforms.

UD acceleration does not support fragmentation and reassembly, so do not send messages larger than the MTU size configured on the Mellanox interface.

UD acceleration is available for the following Ultra Messaging transport types:

  • LBT-RM (UDP-based reliable multicast)
  • LBT-RU (UDP-based reliable unicast)
  • Multicast Immediate Messaging
  • Multicast Topic Resolution

To use UD acceleration, perform the following steps:

  1. Install the Mellanox NIC.
  2. Install the VMA package, which is part of the UD acceleration option .
  3. Include the appropriate transport acceleration options in your Ultra Messaging Configuration File.
  4. Set option transport_*_datagram_max_size and option resolver_datagram_max_size (context) to a value of no more than 28 bytes smaller than the Mellanox interface's configured MTU size.


Reference  <-


resolver_ud_acceleration (context)  <-

Flag indicating if Accelerated Multicast is enabled for Topic Resolution. Accelerated Multicast requires Mellanox InfiniBand or 10 Gigabit Ethernet hardware.
UD Acceleration of topic resolution relies on hardware-supported loopback, which InfiniBand provides, but which the 10 Gigabit Ethernet ConnectX hardware does not.
Note: If 10 Gigabit Ethernet ConnectX hardware is used and multiple UM contexts are desired on the host, this option must be disabled.
Scope: context
Type: int
When to Set: Can only be set during object initialization.
Version: This option was implemented in LBM 5.2.

Value Description
1

Accelerated Topic Resolution is enabled.

0 Accelerated Topic Resolution is not enabled. Default for all.


ud_acceleration (context)  <-

Flag indicating if Accelerated Multicast is enabled for LBT-RM.
Accelerated Multicast requires InfiniBand or 10 Gigabit Ethernet hardware and the purchase and installation of the Ultra Messaging Accelerated Multicast Module. See your Ultra Messaging representative for licensing specifics.
Scope: context
Type: int
When to Set: Can only be set during object initialization.
Version: This option was implemented in LBM 4.1.

Value Description
1

Accelerated Multicast is enabled.

0 Accelerated Multicast is not enabled. Default for all.