An Ultra Messaging queuing system uses a broker to manage the transfer of messages from a source to a receiver. When you build an Ultra Messaging queuing system, identify the broker to use and write a client application that defines the behavior of the source and the receiver.

You can configure the broker to use the Ultra Messaging persistence adapter. When you use the Ultra Messaging persistence adapter, the broker uses Ultra Messaging Persistence (UMP) functionality to manage the event stream and save messages to long-term storage.

To implement high availability, you can install and configure brokers on multiple nodes. A high availability queuing system includes an active broker and multiple standby brokers. Each broker maintains a copy of the messages and event stream.

Ultra Messaging generates events that you can listen for to determine the state of the source, receiver, and messages. You can write a callback function to check for and handle events according to the requirements of the queuing system.

You can use an index to group messages and send them to a specific receiver for processing. Use indexed queuing to ensure that all related messages are processed in the correct order.

You can configure a composite destination in the broker to distribute messages from one source to different receivers based on specific selection criteria. You can also use a composite destination to send messages to one receiver and send copies of the messages to another receiver.

Brokered Context <-

The brokered context establishes the environment for an Ultra Messaging queuing system.

When you build an Ultra Messaging queuing system, you must create a context and define the broker for the context. You can have multiple brokered contexts in a queuing system.

You can define the broker for the context in the client configuration file or in the application code. When you define the broker for a context, you must specify the IP address of the machine that hosts the broker and the Advanced Message Queuing Protocol (AMQP) listening port.

To define the broker for a brokered context in a configuration file, the option broker (context) as follows:

context broker broker_ip_address:port

For more information about the Ultra Messaging context, see Context Object.

Brokered Source <-

In an Ultra Messaging queuing system, the source is the component that sends messages to a broker.

In a client application, you must create a source object that connects and sends messages to the broker. You can include metadata with a message through message properties. The client application can listen for events to determine the state of the source, message, or broker.

When you create a source, you must specify the name of a message destination. The broker creates a queue destination with the specified name. If you create multiple sources, specify a different message destination for each source. A source sends messages to the message destination associated with it.

You can configure the source in the configuration file or in the application code. When you configure the source for a brokered context, you must set the transport option to broker.

To set up the broker transport for a source in the client configuration file, include the following parameter:

source transport broker

Source Event Concepts <-

Ultra Messaging generates source events that you can listen for and respond to based on your requirements.

You can use source events to determine the state of the source and the messages sent by the source. For example, when you write a callback function to respond to source events, you can check for a registration complete event and start sending messages only after you receive the event. You can also check events to determine whether a message is stable or whether the memory allocated to the message is reclaimed.

Ultra Messaging provides other events that you can check to manage the message queuing process, including events such as a registration error and flight size change.

For more information about source events, see Source Events.

Brokered Receiver <-

A receiver is the component in an Ultra Messaging queuing system that consumes the messages that are sent by a source. The source sends messages to a broker and the broker assigns messages to the receiver. The receiver does not directly access messages from the source.

In a client application, you must create a receiver object that connects to and gets messages from a broker. The receiver connects to the broker defined in the context. When you create a receiver, you must specify the message destination in the broker from which to receive messages.

A receiver must be registered with a broker before the receiver can receive messages from the broker. When you create a receiver, you trigger a receiver registration. A receiver registration complete event is a receiver event that indicates that the receiver is registered and can receive messages from the broker.

When a receiver consumes a message, it sends an acknowledgment to the broker. If the broker does not get an acknowledgment from the receiver after it assigns a message, the broker could resend the message to the receiver. When the broker resends the messages, it sets the LBM_MSG_FLAG_RETRANSMIT flag to indicate that the message has been sent previously and could be a duplicate. In the client application, you must check for the LBM_MSG_FLAG_RETRANSMIT flag to determine if a message is a duplicate and then take action based on your requirements.

The Broker <-

The message broker manages the transfer of messages from the source to a receiver. The broker receives messages from the source and assigns them to a receiver based on the message queue configuration.

The Ultra Messaging message broker provides the following services:

Connection services. The broker manages the connections between the source and the broker and between the broker and the receiver.
Message delivery services. The broker assigns and delivers messages and generates events to provide information about the state of the broker.
Persistence services. The broker writes messages and events to storage.
High availability services. The broker defines a high availability process that uses Ultra Messaging functionality to save messages and replicate them to multiple brokers.

Note: The Ultra Messaging message broker is built on top of the Apache ActiveMQ message broker. The Ultra Messaging Queuing API connects to the Advanced Message Queuing Protocol (AMQP) transport connector in ActiveMQ. The Ultra Messaging broker uses the same default listening port number 5672 that ActiveMQ uses. The Ultra Messaging broker also uses the same default file name activemq.xml for the broker configuration file. You can change the port number and configuration file name for the Ultra Messaging broker.

Message Destination <-

The message destination is the location on the broker where a message is stored before the broker assigns the message to the receiver.

When you create a source, you must specify the name of the message destination to which the source sends messages. When you create a receiver, you must specify the message destination from which the receiver consumes messages.

Note: Ultra Messaging uses the term topic to refer to the message destination. However, the message destination created in an Ultra Messaging queuing system is a queue, not a topic.

Message Store <-

You can configure the Ultra Messaging broker to use the Ultra Messaging persistence adapter that replicates events generated by the active broker to the standby brokers. When you use the Ultra Messaging persistence adapter, each broker maintains an independent message store.

The Ultra Messaging broker uses Ultra Messaging Persistence (UMP) functionality to manage events generated within the Ultra Messaging queuing system. The active broker uses a UMP source to replicate events to the standby brokers. A standby broker uses a UMP receiver to receive events from the active broker. If a broker fails, it uses a UMP store daemon to recover events that it missed when it was down.

After a standby broker receives events from the active broker, the standby broker stores the messages locally in the default long-term storage.

Flight Size Concepts <-

When a source sends a message, the message is considered to be in-flight until the broker generates a message stable event. Flight size is the number of messages that are in-flight at any point in time.

You can set a maximum flight size for a source to control the message load on the broker. Set the flight size based on the requirements of the queuing system and the capacity of the network. In an Ultra Messaging queuing system, setting the flight size flight size is the only method available to limit the volume of messages from the source.

High Availability Concepts <-

You can install and configure brokers on multiple nodes to implement high availability in an Ultra Messaging queuing system.

When you configure the context for a multi-node queuing system, you must specify the number of brokers and the IP address and port number of each broker. A queuing system set up for high availability has one active broker and multiple standby brokers.

The source sends messages to the active broker and the active broker assigns messages to the receiver. If the active broker fails, Ultra Messaging elects one of the standby brokers as an active broker. When the failed broker restarts, it runs as a standby broker and synchronizes with other brokers to recover events that it missed while it was down.

To have a successful failover, each broker must have the same copy of the messages and event stream. If a standby broker is elected as the active broker, the standby broker must have the same data as the previous active broker to be in the correct state to start as an active broker.

The broker in each node stores a copy of the messages and event stream. The active broker replicates the event stream to the UMP store daemons and the standby brokers. The active broker does not send a message stable event back to the client or save a message to long term storage until the replicated event stream is stable in more than half of the UMP store daemons.

Indexed Queuing <-

Indexed queuing is a way to group messages and ensure that one receiver processes all messages in the group in the order that they are sent. The source uses an index to specify the messages in a group. The Ultra Messaging broker uses the ActiveMQ message group functionality to implement indexed queuing.

If a set of messages must be processed in order, it can be more efficient for one receiver to process all the messages. For example, if the messages represent bids in an auction, you might want to assign all bids for an item to one receiver to ensure that the bids are processed in the right order. Include the same index in all messages that relate to an item to ensure that all bids for the item go to one receiver.

To set up indexed queuing, define an index with a string value and add the index to the message as metadata. By default, the broker checks the index of a message before it assigns the message to a receiver. The broker designates a receiver to process messages with a specific index. When the source sends a message with the index, the broker assigns the message to the designated receiver. The broker assigns all messages with the same index to the same receiver.

In the client application, extract the index to determine the message group and process the message based on your requirements.

Composite Destinations <-

A composite destination is a virtual message destination that represents a collection of physical message destinations.

You can create a source that sends messages to a virtual queue destination in the broker. In the broker configuration, you associate the virtual queue destination to multiple physical queue destinations. You can then create a receiver to receive messages from one of the physical queue destinations.

The following image shows an example of a broker configured with a composite destination:

In this example, the broker has a virtual queue and three physical queues. The source sends messages to the virtual queue. The broker forwards the message to the physical queues. The receivers get messages from the physical queues.

By default, if the broker is configured to use a composite destination, the source cannot send messages directly to one of the associated physical destinations. When you create a source, you specify the name of the virtual queue destination for the messages sent by the source.

Inversely, the receiver cannot consume messages from a virtual message destination. When you create a receiver, you must specify the name of a physical queue destination from which to receive messages sent by the source.

For example, to add auditing capabilities to the queueing system, you can use a composite destination to write all messages to a log. You can set up the source to send messages to a virtual queue named VirtQueue. In the broker, forward the messages in VirtQueue to physical message queues named ProcessQueue and LogQueue. Create a receiver to get messages from ProcessQueue and process the messages. Create another receiver to get messages from LogQueue and write the messages to a log file.

You can also use a composite destination to distribute messages from the source to different receivers based on specific selection criteria. Configure the broker to apply the selection criteria to each message in the virtual queue before it forwards the message to a physical queue.

Composite Destination Configuration <-

You configure the composite destination in the Ultra Messaging message broker. Use the broker configuration file to set up the logical and physical queues for the composite destination.

The following XML code shows an example of a configuration for a composite destination:

<destinationInterceptors>
    <virtualDestinationInterceptor>
        <virtualDestinations>
            <compositeQueue name="VirtQueue0">
                <forwardTo>
                    <queue physicalName="QueueA" />
                    <queue physicalName="QueueB" />
                    <queue physicalName="QueueC" />
                </forwardTo>
            </compositeQueue>
        </virtualDestinations>
    </virtualDestinationInterceptor>
</destinationInterceptors>

Message Lifetime <-

The message lifetime is the length of time within which the broker must assign a message to a receiver before the message expires. The message lifetime starts when the broker receives the message and expires after a length of time that you can configure. When the lifetime of a message expires, you can configure the broker to discard the message or send the message to the dead letter queue (DLQ).

You can set the message lifetime when you send a message or you can set the message lifetime as a configuration option. By default, if you do not set a message lifetime, a message does not expire and can be redelivered to a receiver indefinitely.

Dead Letter Queue <-

The dead letter queue (DLQ) refers to a message destination for messages that cannot be delivered to a receiver.

The Ultra Messaging broker contains a default DLQ message destination for all undeliverable messages. When the lifetime of a message expires or the maximum number of redelivery attempts is reached, the broker sends the message to the DLQ.

You can set up a separate DLQ for each message destination. In the broker configuration file, you can set up a dead letter strategy for a destination and create a DLQ associated with the destination. The broker sends the undeliverable messages to the DLQ associated with a message destination.

You can create a receiver to process the messages in the DLQ. Review the messages to look for common patterns and possible causes for delivery failure.