The operating parameters for umestored come from an XML configuration file that must be supplied on the command line. umestored contains a UM context and receivers that may be configured with default values through a UM configuration file referenced in the XML configuration file. Default UM options my be overridden for each configured store using the XML configuration file.
You configure umestored to instantiate stores or queues with the umestored XML configuration file which UM reads at start up. This umestored XML configuration file has the following sections.
Daemon section - holds administrative parameters for such things as the location of log files, the UM Configuration File, etc.
Stores section - holds parameters for any persistent stores and also the topics to be persisted.
Queues section - holds parameters for the queues to be instantiated along with the topics it should accept and the application set definitions which contain sets of receivers.
High Level Stores Configuration File.
<ume-store version="1.2">
<daemon>
Daemon configuration options
</daemon>
<stores>
<store attributes>
<topics>
<topic attributes>
<ume-attributes>
<option attributes/>
</ume-attributes>
</topic>
</topics>
</store>
</stores>
</ume-store>
High Level Stores and Queues Configuration File. The stores and queues sections are both optional, although, you must specify at least one or the other.
<ume-store version="1.2">
<daemon>
Daemon configuration options
</daemon>
<stores>
<store attributes>
<topics>
<topic attributes>
<ume-attributes>
<option attributes/>
</ume-attributes>
</topic>
</topics>
</store>
</stores>
<queues>
<queue attributes>
<queue-groups>
<queue-group attributes/>
</queue-groups>
<ume-attributes>
<option attributes/>
</ume-attributes>
<indices>
<index attributes>
<ume-attributes>
<option attributes/>
</ume-attributes>
</index>
</indices>
<application-sets>
<application-set attributes>
<ume-attributes>
<option attributes/>
</ume-attributes>
<receiver-types>
<receiver-type attributes>
<ume-attributes>
<option attributes/>
</ume-attributes>
<index-rules attributes>
<rules>
</index-rules>
</receiver-type>
</receiver-types>
</application-set>
</application-sets>
<topics>
<topic attributes>
<ume-attributes>
<option attributes/>
</ume-attributes>
<application-sets>
</application-set attributes
</application-sets>
</topic>
</topics>
</queue>
</queues>
</ume-store>
This section discusses the following topics.
The following table presents child elements allowed in the daemon configuration section.
| Tag | Description | Default Value |
|---|---|---|
| log | Required. Pathname for log file. | None--this is a required element. |
| pidfile | Pathname for daemon process ID (pid) file | No pidfile |
| uid | User ID (uid) for daemon process (if started as root) | Daemon retains starting uid |
| gid | Group ID (gid) for daemon process (if started as root) | Daemon retains starting gid |
| lbm-config | Pathname for UM configuration file | No config file; use UM defaults |
| lbm-license-file | Pathname for UM license file | License read from environment |
| web-monitor | Address:port where web monitor listens. Address of * listens on all interfaces. Also has a single attribute, permission, allowable values are read-only and read-write. Using read-only disables the text fields and buttons on a Web Monitor "debug page" that can only be enabled by Informatica Support. Example: <:web-monitor permission="read-only">*:15304<:/web-monitor> | No web monitor |
| lbm-password-file | Pathname for Queue Browser authentication file. | rel-id/platform-id/bin/password.xml |
The Stores Element is a container for individual store elements which define specific store instances. The below is an example of a Stores Element.
<stores>
<store name="test-store-1" port="14567">
<ume-attributes> ... </ume-attributes>
<topics>
<topic pattern="quote*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
<topic pattern="subject*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
</topics>
</store>
<store name="test-store-2" port="14568">
<ume-attributes> ... </ume-attributes>
<topics>
<topic pattern="issue*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
<topic pattern="topic*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
</topics>
</store>
</stores>
The Store Element contains information about an individual UMP store and has attributes, options and topics. See the example below.
<store name="test-store-1" port="14567">
<ume-attributes> ... </ume-attributes>
<topics>
<topic pattern="quote*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
<topic pattern="subject*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
</topics>
</store>
The following table gives attributes for store
elements.
| Attribute | Description | Default Value |
|---|---|---|
| name | Specifies the name for the store. | None--this is a required attribute |
| port | TCP port where umestored should listen for this store. | None--this is a required attribute and a valid (non-zero) port must be specified. |
| interface | Specifies the network interface over which umestored accepts connection requests for this store. | 0.0.0.0 (INADDR_ANY) |
The following table gives the child elements allowed in the store configuration section.
| Child Element | Description | Default Value |
|---|---|---|
| topics | A container for topic elements. See Topics Element for more information. | None |
| ume-attributes | A container for option elements. See Options for a Store's ume-attributes Element for more information. | None |
You can configure context (scope) options with a type
attribute of lbm-context. UM passes such options through
the normal receiver and context configuration option setting mechanisms. See the UM Configuration Guide for details. Store
options without a type attribute or those explicitly given
a type attribute of store
simply configure the store itself.
The following table gives options allowed for a store
element. Use the store Option Type for these options. A Store's
ume-attributes Element can also accept the lbm-context Option
Type. See Option Types for ume-attributes
Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| disk-cache-directory | Pathname for disk store message cache directory | umestored-cache |
| disk-state-directory | Pathname for disk store state directory | umestored-state |
| allow-proxy-source | Allows the store to act as a proxy source in case a registered source terminates. | 0 (Disable) |
| context-name | Name of the store that can be used by sources to refer to the store instead of the address:request port. A store runs in its own context, therefore a name can used to represent the address:request port. This name facilitates UMP operation across the UM Gateway. Restricted to 128 characters in length, and may contain only alphanumeric characters, hyphens, and underscores. | None. |
| retransmission-request-processing-rate | Specifies the number of retransmission requests processed by a store per second across all topics. The store drops all retransmission requests that exceed this value. | 262144 |
The Topics element is a container element for all the topics persisted by the UMP store. It is one of the two child elements of the Store Element. See the example below.
<topics>
<topic pattern="issue*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
<topic pattern="topic*" type="PCRE">
<ume-attributes> ... </ume-attributes>
</topic>
</topics>
The Topic Element defines an individual topic persisted on the UMP store. The following table gives attributes for the topic element.
| Attribute | Description | Default Value |
|---|---|---|
| pattern | Specifies a pattern used to select topics for which a store provides persistence services. | None--this is a required attribute |
| type | Specifies the type of matching to be performed on the pattern attribute. A value of direct selects an exact string match. A value of PCRE selects a Perl Compatible Regular Expression match. A value of regexp selects a POSIX extended regular expression. PCRE, or regexp. | direct |
The Topic Element has one child element, ume-attributes, the options for which appear in Options for a Topic's
ume-attributes Element.
The following table gives options allowed for a topic
element. Use the store Option Type for these options. You can
also configure receiver (scope) options and source (scope) options in a Topic's
ume-attributes Element by using the Option Types lbm-receiver
and lbm-source, respectively. See Option Types for ume-attributes
Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| retransmission-request-forwarding | If enabled (value = 1), the store always forwards retransmission requests to sources and does not service any requests itself. If disabled (value = 0), the store services retransmission requests for data it has, and forwards requests to sources for data it does not have. | 0 (store services retransmission requests) |
| repository-type | Specifies how messages should be retained by the store. A value of no-cache does not retain messages, only state information. A
value of memory retains messages only in the (presumably
volatile) main memory of the store. A value of disk
retains messages to (presumably non-volatile) disk storage as quickly as possible. In
addition, messages are cached in main memory for a time as well. A value of reduced-fd retains messages in disk storage using significantly
fewer File Descriptors. Use of this repository type may impact performance. (See Persistent Store Architecture.) The reduced-fd disk storage option is not available on Microsoft® Windows®. |
no-cache |
| repository-size-threshold | For topics with a repository-type of memory, disk or reduced-fd, specifies the minimum number of message bytes
(includes payload, headers, and store structure overhead) retained for a topic. For RPP
repositories, this value only includes message payload. For the disk or reduced-fd repository
type, this value configures the size threshold of the memory cache. (units: bytes) |
25165824 (24 MB) |
| repository-size-limit | For topics with a repository-type of memory, disk or reduced-fd, specifies the maximum number of message bytes
(includes payload, headers, and store structure overhead) retained for each source. For
RPP repositories, this value only includes message payload. For the disk or reduced-fd repository
type, this value configures the size of the memory cache. (units: bytes) |
50331648 (48 MB) |
| repository-age-threshold | For topics with a repository-type of memory, disk or reduced-fd, specifies a message age threshold. Younger messages
will be retained. Space used to store older messages may be reclaimed. A value of 0 means
message age is not considered in retention decisions. (units: seconds) |
0 |
| repository-disk-max-async-cbs | For topics with a repository-type of disk or reduced-fd, specifies the
maximum number of outstanding async I/O callbacks for reading and writing messages to
disk. (units: async callbacks) |
16 callbacks |
| repository-disk-max-write-async-cbs | For topics with a repository-type of disk or reduced-fd, specifies the
maximum number of outstanding async I/O callbacks for writing messages to disk. Reducing
this option can improve throughput by batching more fragments into a single write.
(units: async callbacks) |
16 callbacks |
| repository-disk-max-read-async-cbs | For topics with a repository-type of disk or reduced-fd, specifies the
maximum number of outstanding async I/O callbacks for reading messages from disk. Raising
this value can improve recovery rates. For topics with a repository-type of reduced-fd, Informatica recommendeds a value of 200 times the
number of expected receivers per topic. (units: async callbacks) |
16 callbacks |
| repository-disk-file-size-limit | For topics with a repository-type of disk or reduced-fd, specifies the
maximum amount of disk space that will be used to store retained messages. A minimum
value of 196992 is enforced. (units: bytes) |
104857600 (100 MB) |
| repository-disk-file-preallocate | For topics with a repository-type of disk, If set to 1, UMP pre-allocates
a store's cache files to match their maximum size on disk (as configured by repository-disk-file-size-limit) upon creation, as opposed to
growing to that size as the store receives new messages. For ext3/4 and NTFS file
systems, this options creates a sparse file, which does not allocate all of the
underlying data blocks. Advantages of pre-allocation include better performance on
rotating disks due to less file fragmentation, and knowing that enough disk space exists
for any new source that registers. Disadvantage is the time to create the cache files,
especially if many sources register at once. |
0 (zero) - do not pre-allocate |
| repository-disk-async-buffer-length | For topics with a repository-type of disk or reduced-fd, specifies the
size of the buffers that will be used in async I/O operations for reading and writing
messages to disk. A minimum value of 65664 is enforced. (units: bytes) |
65664 (64 KB + 128) |
| repository-disk-message-checksum | For topics with a repository-type of disk or reduced-fd, specifies
whether the messages saved to disk should include a checksum field or not for validation
if the store is restarted. (units: flag) |
0 (disabled) |
| source-activity-timeout | Establishes the period of time from a source's last activity to the release of the source's RegID. Stores return an error to any new source requesting the source's RegID during this period. If proxy sources are enabled ( ume_proxy_source) the store does not release the source's RegID and UMP elects a proxy source. If neither proxy sources nor ume_state_lifetime are configured, the store also deletes the source's state and cache. Can be overridden by ume_activity_timeout. See also Proxy Sources. (units: milliseconds) | 30000 (30 seconds) |
| source-state-lifetime | Establishes the period of time from a source's last activity to the deletion of the source's state and cache by the store, regardless of whether a proxy source has been created or not. You can also configure ume_state_lifetime for the source. The store uses whichever is shorter. See also Proxy Sources. (units: milliseconds) | 0 (zero) |
| receiver-activity-timeout | Establishes the period of time from a receiver's last activity to the release of the receiver's RegID. Stores return an error to any new request for the receiver's RegID during this period. Can be overridden by ume_activity_timeout. See also Proxy Sources. (units: milliseconds) | 30000 (30 seconds) |
| receiver-state-lifetime | Establishes the period of time from a receiver's last activity to the deletion of the receiver's state and cache by the store. You can also configure ume_state_lifetime for the receiver. The store uses whichever is shorter. See also Proxy Sources. (units: milliseconds) | 0 (zero) |
| source-check-interval | Specifies how often a store will check for activity of sources and receivers. (units: milliseconds) | 100 (100 milliseconds) |
| keepalive-interval | Specifies how often a store will generate keepalive traffic to sources and receivers if there has been no traffic required in the normal course of operation. (units: milliseconds) | 500 (500 milliseconds) |
| receiver-new-registration-rollback | Specifies the number of stabilized messages that a newly registered receiver should consume. For example, setting this to 10, "rolls back" the new receiver's starting message to the 10th most recent message. This value must be positive and less than 2147483648. The recommended value of 2147483647 indicates that the rollback should begin at the start of the stream. A value of 0 indicates the store should instruct the receivers to start with the next new message from the source known by the store. (units: messages) | 0 (no recovery requested) |
| proxy-election-interval | Specifies the interval, in milliseconds, used when electing a proxy source. When a source, which requested that a proxy source be provided for it, has been detected as no longer active, each store eligible to provide a proxy source for it waits for an amount of time which is randomized in the range [0.5*proxy-election_interval .. 1.5*proxy-election_interval]. If no other store has been elected to serve as the proxy source, the store declares itself as the proxy source. (units: milliseconds) | 5000 (5 seconds) |
| stability-ack-interval | Specifies the maximum amount of time that stability acknowledgments will be batched before being sent to a source. Batching stability ACKs can increase throughput of UMP stores (especially memory stores) significantly, but introduces a delay between when a message is actually stable in the UMP store and when the source is notified of message stability. (units: milliseconds) | 200 (200 milliseconds) |
| stability-ack-minimum-number | Specifies the minimum number of message stability acknowledgments that must accumulate before a stability ACK is sent to a source. With the default value of 1, stability ACKs are sent immediately as soon as messages are stable. Increasing this value causes stability ACKs to be batched, which can increase throughput of UMP stores (especially memory stores) significantly, but introduces a delay between when a message is actually stable in the UMP store and when the source is notified of message stability. If using a stability ACK-based flight size on a UMP source in combination with this option, it is advisable to make sure stability-ack-minimum-number is set less than or equal to the source's flight size. Otherwise, stability ACKs will only be sent upon expiration of the stability-ack-interval timer, resulting in bursty stop-and-go sending. (units: number of message fragments) | 1 (1 fragment) |
| repository-allow-receiver-paced-persistence | Specifies if the repository allows receiver-paced persistence (1) or a source-paced persistence (0). The source must enable or disable this option with ume_receiver_paced_persistence, but cannot enable it if this value is set to 0. | 0 (source-paced persistence) |
| repository-allow-ack-on-reception | Specifies if the repository should acknowledge to the source that it has persisted a message as soon as the repository receives it instead of when it writes the message to disk. Set this option to 1 to allow this behavior. The source must also set this option with ume_repository_ack_on_reception to enable this behavior, but cannot enable it if this value is set to 0. For memory stores, this option has no effect. This option only applies to RPP repositories (repository-allow-receiver-paced-persistence = 1). | 0 (store acknowledges persistence when the store writes the message to disk) |
| repository-disk-write-delay | For topics with a repository-type of disk or reduced-fd, specifies the
maximum delay in milliseconds before the repository persists a message to disk. If the
repository sets this option a value other than the default, the source can reconfigure
this option with ume_write_delay to a lower or equal value, but
cannot increase it. (units: milliseconds) |
0 milliseconds |
| source-flight-size-bytes-maximum | Specifies the maximum message payload in bytes allowed to be inflight (un-stabilized at a store and without delivery confirmation) before a new message send either blocks or triggers a notification source event. UMP monitors both this option and ume_flight_size. If either threshold is met, the configured blocking or notification behavior executes. See ume_flight_size_behavior. A source can reconfigure this option to a value of less than or equal to this value. This option only applies to RPP repositories (repository-allow-receiver-paced-persistence = 1). (units: bytes) | 4194304 bytes (4MB) |
The Queues Element is a container element for all queues. An abbreviated queues section appears below.
<queues>
<queue name="Queue 1" interface=10.29.3.24" port="20555" group-index="0">
<queue-groups>
<queue-group index="0" size="5"/>
<queue-group index="1" size="1"/>
</queue-groups>
<ume-attributes> ... </ume-attributes>
<application-sets> ... </application-sets>
<topics> ... </topics>
</queue>
<queue name="Queue 2" interface=10.29.3.25" port="20555" group-index="0">
<queue-groups>
<queue-group index="0" size="5"/>
<queue-group index="1" size="1"/>
</queue-groups>
<ume-attributes> ... </ume-attributes>
<application-sets> ... </application-sets>
<topics> ... </topics>
</queue>
</queues>
The Queue Element defines a single queue. Each queue must have a unique name and a port. An interface, group-index and group-size are optional. The following table gives attributes for a
queue element.
| Attribute | Description | Default Value |
|---|---|---|
| name | Specifies the name for the queue. The queue name is the prime method for identifying the queue and must be unique. UMQ uses it during queue resolution, etc. | None--this is a required attribute |
| port | TCP port where umestored should listen for this queue. As with UMP stores, UMQ access the queue during registration and other operations with the port. | None--this is a required attribute |
| interface | Specifies the network interface over which umestored accepts connection requests for this queue. | 0.0.0.0 (INADDR_ANY) |
| group-index | A number that identifies this queue instance's queue group. See Queue Redundancy for more information. | 0 - Valid values range from 0 to 65535. |
| group-size | The number of queue instances in this queue's group. You can use this attribute to specify the number of queue instances instead of the Queue Groups Element if the queue consists of only one group. | 1 - Valid values range from 1 to 65535. |
The Queue Element can be configured with the Queue Groups Element, Application Sets
Element and QueueTopics Element.
The options for a Queue's ume-attributes appear in the
following sections.
Message Storage Options for a Queue's ume-attributes Element
Queue Management Options for a Queue's ume-attributes Element
Queue Slave Instance (QSI) Options for a Queue's ume-attributes Element
The queue-groups element contains queue-group elements that define all the queue groups that make
up the queue. In the abbreviated Queues section shown in Queues Element, the
queue element,
<queue name="Queue 1" interface=10.29.3.24" port="20555" group-index="0">
specifies Queue 1 as a Queue Instance in Group 0. The queue-groups element in the same sample specifies that Queue 1 comprises two groups, Group 0, which has 5 queue instances
and Group 1, which has 1 queue instance.
<queue-groups>
<queue-group index="0" size="5"/>
<queue-group index="1" size="1"/>
</queue-groups>
To completely configure Queue 1, you must specify 6 queue
instances in either one umestored.xml file as individual queue
elements within the queues element or in separate umestored.xml
files, one for each instance. The Queue Element for all 6 queue instances would be the
same except for the interface, port and group-index because
every instance of Queue 1 must have the same name. And all 6 queue instances would also have the same queue-groups configured.
| Attribute | Description | Default Value |
|---|---|---|
| index | The queue's redundancy group. See Queue Redundancy for more information. | 0 - Valid values range from 0 to 65535. |
| size | The number of queue instances in this queue's redundancy group. | 1 - Valid values range from 1 to 65535. |
The table below displays the general options available for a Queue Element. Use the queue Option Type for these options. A Queue's ume-attributes Element can also accept Option Types lbm-receiver, lbm-context and lbm-source. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| control-topic-name | The name of the queue's control topic. The queue sends all control information for Parallel Queue Dissemination (PQD) and Source Dissemination (SD) to this topic. | UMQ-cntl:qname where qname is the name of the queue. |
| queue-advertisement-interval | The interval in milliseconds between advertisements of the queue for queue resolution. | 1000 (1.0 seconds). Value must be greater than 0. |
| forwarding-behavior | How UMQ forwards messages in the Serial Queue Dissemination (SQD) and Parallel Queue Dissemination (PQD) dissemination models. Valid values are store-while-forward and store-then-forward. | store-while-forward |
| dissemination-model | The dissemination model used by the queue. Valid values are: serial-queue-dissemination, SQD, parallel-queue-dissemination, PQD, source-dissemination, and SD. See also Message Paths | SQD |
| sending-threads | The number of threads used to send control and data messages from the queue. | 1 (This must be greater than 0.) |
| context-check-interval | The interval in milliseconds between activity checks by the queue of a source or receiver context. Each check looks at the context as well as any associated source and receiver activity. | 1000 (1.0) seconds This must be greater than 0. |
| context-activity-timeout | The length of time a source or receiver context may be inactive before the queue disengages from the context. | 30000 (30.0 seconds) This must be greater than 0. |
| context-keepalive-interval | The interval in milliseconds between keepalives messages sent by the queue to determine whether a source or receiver context is alive or not. | 1000 (1.0 seconds) This must be greater than 0. |
| source-activity-timeout | The length of time a source may be inactive before the queue disengages from the source. | 30000 (30.0 seconds) This must be greater than 0. |
| source-keepalive-interval | The interval in milliseconds between keepalives messages sent by the queue to determine whether a source is alive or not. | 1000 (1.0 seconds) This must be greater than 0. |
| rcr-save-timeout | The maximum time in milliseconds that the queue holds RCR information for retransmission to receivers or other queue instances. | 30000 (30.0 seconds) This must be greater than 0. |
| dead-letter-topic-prefix | Prefix to be used for Dead Letter topic name. UMQ creates a Dead Letter topic name composed of this prefix, the unconsumed message's topic name or string and the Application Set name. See also Dead Letter Queue. | None. |
| dead-letter-topic-separator | The separator UMQ uses between the dead-letter-topic-prefix, topic name or string and the Application Set name when it constructs the Dead Letter topic name. See also Dead Letter Queue. | / (forward slash) |
| allow-browsing | Allow observer receivers to retrieve lists of messages in the queue and to retrieve specific messages. Allowing browsing introduces some CPU and memory overhead within the queue, so if browsing support is not needed, setting this to 0 may increase performance. | 1 (browsing enabled). This must be 0 or 1. |
| require-client-authentication | Certain actions (currently topic list, message list, and message retrieve actions by default) require greater access permissions than a default queue user is given. Normally, the queue requires the client that is requesting the queue to perform any of those actions to pass a once-per-session authentication challenge by providing the username and password credentials of a user authorized to perform the requested action. If the client fails the authentication challenge, an error is returned to the client by the queue and the action is not performed. Turning this option off causes the queue to respond to such requests even if the user fails the authentication challenge. It is therefore NOT recommended to turn this option off in production environments, as it will make the queue less secure, but it is provided for convenience in functional test or development environments where security is not needed. | 1 (authentication required). This must be 0 or 1. |
UMQ provides 3 basic modes or operation for message storage.
Memory Storage: UMQ stores messages and message state only in memory. A restarted umestored does not resume any previous operation. In this configuration, sinc-data-filename, sinc-queue-swap-filename, and sinc-log-filename are not set.
Disk Storage, no Persistence: UMQ swaps messages and message state from memory to disk as needed to maintain low memory overhead. A restarted umestored does not resume any previous operation. In this configuration, sinc-data-filename and sinc-queue-swap-filename are set. The sinc-log-filename is not set.
Disk Storage, Persistence: UMQ swaps messages and message state from memory to disk as needed to maintain low memory overhead. In addition, UMQ maintains a separate log file that contains all operations needed to reconstruct state when UMQ restores umestored. In this configuration, sinc-data-filename, sinc-queue-swap-filename, and sinc-log-filename are set.
Use the queue Option Type for these options. A Queue's ume-attributes Element can also accept Option Types lbm-receiver, lbm-context and lbm-source. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| sinc-data-filename | The filename used to store message data on disk. | None. |
| sinc-queue-swap-filename | The filename used to store message state for the queue. | None. |
| sinc-log-filename | The filename used to contain the queue log used to reconstruct state upon restarts. You can use the umqsltool utility to manage this file. | None. |
| sinc-max-size | The storage area's maximum size in bytes. When exceeded, the queue drops new messages until more space room is available. For file-based storage areas, the value of this option is bytes on disk. For memory-based storage, the value of this option is bytes of memory. | 104857600 bytes |
| sinc-data-swap-threshold | The threshold, in bytes, where enough message data causes the oldest and least referred to messages to be swapped to disk and removed from memory. | 104857600 bytes. This option must be greater than 0. |
| sinc-block-swap-threshold | The threshold, in blocks, where enough message state information causes the oldest and least referred to information to be swapped to disk and removed from memory. A block is state for roughly 120 messages for the same topic and Application Set and is roughly 4KB in size. | 128 blocks This option must be greater than 0. |
| sinc-data-readahead | The number of messages that are read in ahead of the current point in an Application Set's queue position. This helps to initiate asynchronous reads ahead of the time when they are needed for message assignment. | 16 |
The Queue Management options control how queue instances communicate regarding elections and how they monitor each other.
Use the queue Option Type for these options. A Queue's ume-attributes Element can also accept Option Types lbm-receiver, lbm-context and lbm-source. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| queue-management-election-backoff-timeout | The period of time, in milliseconds, before the Queue Election Master (QEM) recalls an election. This backoff is initiated only if an error occurs during the election operation or multiple election calls appear. To avoid livelock, the actual timeout value used is in the range [0.5 * timeout, 1.5 * timeout]. | 5000 (5.0 seconds) This option must be greater than 0. |
| queue-management-election-bias | The bias used during elections to determine the Post Election Master (PEM). A higher value biases the election to that queue instance. | 0 - This option must be less than or equal to 1000. |
| queue-management-election-call-interval | The interval, in milliseconds, between Election Calls by the Queue Election Master (QEM) once an election is called due to an unresponsive master. | 500 (0.5 seconds) This option must be greater than 0. |
| queue-management-election-call-timeout | The period of time, in milliseconds, that an election call can take. If the Queue Election Master (QEM) can not end the election early, then this timeout signals the end of the election and causes the QEM to generate a New Instance List (NIL). | 5000 (5.0 seconds) This option must be greater than 0. |
| queue-management-join-request-interval | The interval, in milliseconds, between Join Requests sent by queues that start up. | 500 (0.5 seconds) This option must be greater than 0. |
| queue-management-join-request-timeout | The period of time, in milliseconds, after which a starting queue instance gives up on joining an existing queue and starts its own and elects itself master. | 5000 (5.0 seconds) This option must be greater than 0. |
| queue-management-master-activity-timeout | The period of time, in milliseconds, that a master must be active (by sending Queue Management Instance Lists) or be declared dead. A new Queue Election Master (QEM) calls an election after the expiration of this period. | 5000 (5.0 seconds) This option must be greater than 0. |
| queue-management-master-check-interval | The interval, in milliseconds, between checks on the master by the slaves. | 1000 (1.0 second) This option must be greater than 0. |
| queue-management-new-instance-list-interval | The interval, in milliseconds, at which the Queue Election Master (QEM) retransmits the New Instance List (NIL) after an election call has been completed. | 200 (0.2 seconds) This option must be greater than 0. |
| queue-management-new-instance-list-timeout | The period, in milliseconds, by which a Queue Election Master (QEM) must receive confirmations of the New Instance List (NIL) that it has sent out. If insufficient confirmations come back with this period expires, the QEM calls a new election and resets the timeout. | 2000 (2.0 seconds) This option must be greater than 0. |
| queue-management-resume-operation-interval | The interval, in milliseconds, at which a Queue Election Master (QEM) sends a resume operation message to the new master after the New Instance List (NIL) has been confirmed. | 200 (0.2 seconds)This option must be greater than 0. |
| queue-management-resume-operation-timeout | The timeout, in milliseconds, by which a Queue Election Master (QEM) must detect the resumption of operation after the New Instance List (NIL) has been confirmed. If resumption has not occurred when this timeout expires, the QEM calls a new election and resets the timeout. | 2000 (2.0 seconds) This option must be greater than 0. |
| queue-management-slave-activity-timeout | The period of time, in milliseconds, that a slave must be unresponsive before the master removes the slave from the Queue Management Instance List. | 5000 (5.0 seconds) This option must be greater than 0. |
| queue-management-slave-check-interval | The interval, in milliseconds, between checks of the slaves by the master. This also controls how often the master sends Queue Management Instance List messages out to all the slaves. | 1000 (1.0 second) This option must be greater than 0. |
Queue Slave Instances (QSIs) receive assignment information from the master queue using the control topic. These messages are called Receiver Control Records (RCR). If a queue instance misses an RCR, it can request it. The options below control the operation of requesting RCRs.
The acronym, qrcrr, refers to Queue Receiver Control Records Request.
Use the queue Option Type for these options. A Queue's ume-attributes Element can also accept Option Types lbm-receiver, lbm-context and lbm-source. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| slave-qrcrr-generation-interval | The timeout, in milliseconds, before an RCR must be retransmitted before UMQ abandons the request. | 10000 (10.0 seconds) This option must be greater than 0. |
| slave-qrcrr-interval | The interval, in milliseconds, between sending RCR retransmit requests. | 200 (0.2 seconds) This option must be greater than 0. |
| slave-qrcrr-outstanding-maximum | The maximum number of outstanding Receiver Control Records (RCR) to request at a single time. | 100 This option must be greater than 0. |
| source-missing-message-timeout | Controls the period of time that UMQ instance tracks source messages that are missed before abandoning them. | 5000 (5.0 seconds) This option must be greater than 0. |
| source-save-timeout | The timeout, in milliseconds, that the queue instance maintains historical state for a source that has had all its messages consumed. | 600000 (10.0 minutes) This option must be greater than 0. |
The Indices element is a container element for a queue's indices. (See Indexed Queuing.) Configuration of indices is optional, however, specifically configured indices can have rules (allow or deny) applied to them on a per receiver type basis. See Index Rules Element. Messages sent to the queue with a specifically configured index can be treated with different attributes than a normal message or a message sent on a non specifically-configured index. In the abbreviated sample below, Red Messages is an example of a named index, and Brown Messages is an example of a ranged numeric index.
<indices>
<index name="Red Messages" type="named" value="red">
<ume-attributes> ... </ume-attributes>
</index>
<index name="Brown Messages" type="ranged" value="13, 44, 80-90">
<ume-attributes> ... </ume-attributes>
</index>
</indices>
The Index element defines a single index and has three attributes, a name, type and value. See the following examples:
<index name="Red Messages" type="named" value="red"> ... </index> <index name="Brown Messages" type="ranged" value="4"> ... </index> <index name="Yellow Messages" type="ranged" value="-10, 30-40, 90, 400-"> ... </index>
| Attribute | Description | Default Value |
|---|---|---|
| name | Name of the index that also accompanies the message ( lbm_umq_index_info_t). Can be a string or an unsigned 64-bit integer. | None. |
| type | An index type can be either the name of a single index or a range of indices. A named index is simply a string literal no longer than 216 characters. A ranged index can specify several ranges of unsigned 64-bit integers at once, or individual integers, separated by commas. A dash (-) can be used to indicate "and lower" or "and up". | None. |
| value | Either the index's string value (if a named index) or the index's range(s) of 64-bit unsigned numbers (if a ranged index). | None. |
Note that negative ranged indices are not allowed; the "-10" value in the example Yellow Messages index is shorthand for "0-10". The notation "400-" indicates "400 and up", "up" meaning up to the maximum 64-bit unsigned integer, so it is shorthand for "400-18446744073709551615".
Options for an index appear in the following table. Use the queue Option Type for these options. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| log-audit-trail | Flag indicates whether or not the Queue logs receiver activity (assignments, consumption reports, reassignments, etc.) to the umestored log. Valid values are 0 (no logging) or 1. | 0 (zero) |
| message-lifetime | The maximum lifetime of a queue message in milliseconds. A value of 0 (zero) configures no lifetime. | 0 (zero) |
| message-reassignment-timeout | The maximum amount of time, starting from assignment, that a message may remain unconsumed by its assigned receiver before the Queue reassigns it to another receiver. A value of 0 (zero) configures the queue to never reassign. | 10000 (10.0 seconds) |
| message-max-reassignments | The maximum number of re-assignments allowed per message. UMQ applies the initial assignment to this maximum. The queue discards messages that exceed this maximum. Setting this option to 1 means that the message will never be reassigned. A value of 0 (zero) configures no maximum. | 0 (zero) |
| receiver-activity-timeout | The length of time a receiver may be inactive before the Queue disengages from the receiver. | 30000 (30.0 seconds) This must be greater than 0. |
| receiver-keepalive-interval | The interval in milliseconds between keepalive messages sent by the Queue to determine whether a receiver is alive or not. | 1000 (1.0 seconds) This must be greater than 0. |
The Application Sets element is a container element for every application set serviced by the queue. In the abbreviated sample below, Set 1 is one application set and Set 2 is another application set. Receiver Type ID of 100 identifies receivers in Set 1 and Receiver Type ID of 200 identifies receivers in Set 2.
<application-sets>
<application-set name="Set 1">
<ume-attributes> ... </ume-attributes>
<receiver-types>
<receiver-type id="100">
<ume-attributes> ... </ume-attributes>
</receiver-type>
</receiver-types>
</application-set>
<application-set name="Set 2">
<receiver-types>
<receiver-type id="200"/>
</receiver-types>
</application-set>
</application-sets>
The Application Set element defines an individual application set and has only one attribute, name, which identifies the application set. Each application set also requires a Receiver Type. See Options for an Application Set's ume-attributes Element for application set options.
Options for an application set appear in the following table. Use the queue Option Type for these options. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| log-audit-trail | Flag indicates whether or not receiver activity (assignments, consumption reports, reassignments, etc.) are logged to the umestored log. Valid values are 0 (no logging) or 1. | 0 (zero) |
| message-lifetime | The maximum lifetime of a queue message in milliseconds. A value of 0 (zero) configures no lifetime. | 0 (zero) |
| message-reassignment-timeout | The maximum amount of time, starting from assignment, that a message may remain unconsumed by its assigned receiver before the queue reassigns it to another receiver. | 10000 (10.0 seconds) |
| message-max-reassignments | The maximum number of re-assignments allowed per message. UMQ applies the initial assignment to this maximum. The queue discards messages that exceed this maximum. Setting this option to 1 means that the message will never be reassigned. A value of 0 (zero) configures no maximum. | 0 (zero) |
| receiver-activity-timeout | The length of time a receiver may be inactive before the queue disengages from the receiver. | 30000 (30.0 seconds) This must be greater than 0. |
| receiver-keepalive-interval | The interval in milliseconds between keepalive messages sent by the queue to determine whether a receiver is alive or not. | 1000 (1.0 seconds) This must be greater than 0. |
| discard-behavior | Queue behavior for unconsumed messages of the Application Set that have exceeded their message-lifetime. If set to dead-letter, UMQ places messages that exceed their message-lifetime on the Dead Letter Queue. If set to drop, UMQ discards such unconsumed messages. See also Dead Letter Queue. | drop |
The Receiver Types element is a container element for all the receiver types within the Application Set.
The Receiver Type element defines an individual receiver type and has only one attribute, id, which identifies the receiver type. A receiver type id is a 32-bit integer. See Options for a Receiver Type's ume-attributes Element for receiver type options.
Options for a Receiver Type appear in the following table. Use the queue Option Type for these options. See Option Types for ume-attributes Elements for more information.
The Index Rules Element is a container element for index rule elements that can allow or deny receivers of the Receiver Type ID to process messages sent on specific configured indices. It contains one attribute, order, which specifies the order in which the Queue applies the index rules. Additionally, the order attribute specifies the default rule (allow or deny) to apply to an index for a particular receiver type if it is not specifically configured. Valid orders are either "allow, deny" or "deny, allow".
The Queue first applies rules of the first type specified in the order attribute.
The Queue then applies rules of the second type specified and overrides any overlapping rules of the first type for the indices in which they overlap.
The Queue then applies the default rule to any indices not specifically configured.
The example below establishes the following rules for receivers with Receiver Type ID 100.
Allowed to process messages with any of the Red Messages indices.
Explicitly prohibited from being assigned messages with any of the Brown Messages indices.
Prohibited (via the default rule) from being assigned messages sent with any other indices that fall outside those defined in either Red Messages or Brown Messages.
<receiver-type id="100">
<ume-attributes> ... </ume-attributes>
<index-rules order="allow, deny">
<index-allow name="Red Messages" />
<index-deny name="Brown Messages" />
</index-rules>
</receiver-type>
Note: The Index Rules Element is not available to ULB receivers.
The Topics Element is a container element for all topics handled by the queue. In the abbreviated example below, application set Set 1 is associated with the configured topic. Since the topic is a PCRE regular expression, any topic message submitted to the queue that matches the pattern is sent to Set 1.
<topics>
<topic pattern="." type="PCRE">
<ume-attributes> ... </ume-attributes>
<application-sets>
<application-set name="Set 1"/>
</application-sets>
</topic>
</topics>
The Topic Element for a queue uses the same attributes for store topics listed in Topic Element and must also specify an Application Set that is configured in the Application Sets section. The options listed in Options for a Queue Topic's ume-attributes Element are also available.
Options for a queue topic appear in the following table. Use the queue Option Type for these options. A Queue's ume-attributes Element can also accept Option Types lbm-receiver, lbm-context and lbm-source. See Option Types for ume-attributes Elements for more information.
| Option | Description | Default Value |
|---|---|---|
| message-management-check-interval | The interval in milliseconds between status checks of the outstanding assignments for all assigned messages on the topic. These checks are done for lifetime and re-assignment. | 1000 (1.0 seconds) This value must not be 0. |
| topic-check-interval | The interval in milliseconds between checks for any sources and receivers of the topic. If none exist, the queue disengages from the topic. | 5000 (5.0 seconds) This value must not be 0. |
| rcr-save-timeout | The maximum time in milliseconds that the queue holds RCR information for retransmission to receivers or other queue instances. | 30000 (30.0 seconds) This must be greater than 0. |
| message-dissemination-hold-interval | The interval in milliseconds between dissemination of the same message to multiple receivers. | 10000 (10.0 seconds) This must be greater than 0. |
| dead-letter-topic | Setting this option to 1 specifies this topic as a Dead Letter Topic to be used by UMQ to receive unconsumed messages of the Queue. (The pattern for this topic should match the dead-letter-topic-prefix.) This topic can only be configured with one Application Set that has its discard-behavior set to drop to prevent the chaining of Dead Letter Topics. See also Dead Letter Queue. | 0 (zero) |
| message-total-lifetime | Establishes the period of time from when a queue enqueues a message until the time the message cannot be assigned or reassigned to a receiver. The queue deletes the message upon expiration of the lifetime. See also Message Lifetimes. | 0 (zero) |
All options configured for ume-attributes require an
Option Type. The following table describes the five Option Types.
| Option Type | Description | Default Value |
|---|---|---|
| lbm-receiver | Allows you to configure receiver (scope) options that you usually specify in a UM Configuration File or set using lbm_*_attr_setopt(). This Option Type is appropriate for a Topic
Element's ume-attributes child element. For example, you
could turn off delivery of NAKs for a particular Topic by including the following within
the Topic's ume-attributes element: <option type="lbm-receiver" name="transport_lbtrm_send_naks"
value="0"> . |
None - this is a required attribute. |
| lbm-context | Allows you to configure context (scope) options that you usually specify in a UM Configuration File or set using lbm_*_attr_setopt(). This Option Type is appropriate for a Store
Element's ume-attributes child element. For example, you
could increase the receiver socket buffer by including the following within the ume-attributes element: <option
type="lbm-context" name="transport_lbtrm_receiver_socket_buffer"
value="1048576"> . |
None - this is a required attribute. |
| lbm-source | Allows you to configure source (scope) options that you usually specify in a UM Configuration File or set using lbm_*_attr_setopt(). This Option Type is appropriate for a Topic
Element's ume-attributes child element. For example, you
could change the transport by including the following within the ume-attributes element: <option
type="lbm-source" name="transport" value="lbtru"> . |
None - this is a required attribute. |
| store | Use this option type for all options that directly configure a store or repository
and appear in either a store or topic ume-attributes element. For
example, <option type="store" name="context-name"
value="remote-store"/> or <option type="store"
name="repository-type" value="disk"/>. |
None - this is a required attribute. |
| queue | Option type used for all ume-attributes configured for
the queue element and its application-set, receiver-type
and topic elements. |
None - this is a required attribute. |
The DTD for version 1.2 of umestored appears below. See also the DTD revision table below.
| DTD Version | Release Date | Product Version | Supported Features |
|---|---|---|---|
| 1.0 | Feb. 2007 | UME 1.0 | Persistent Stores |
| 1.1 | April 2010 | UME 3.0.1 / UMQ 1.0 | Persistent Stores, Queues and Ultra Load Balancing (ULB) |
| 1.2 | March 2011 | UME 3.2 / UMQ 2.1 | Persistent Stores, Queues, Ultra Load Balancing (ULB), Dead Letter Queue, Indexed Queuing and Indexed ULB |
<!ELEMENT ume-store (daemon, stores?, queues?)> <!ATTLIST ume-store version CDATA #REQUIRED> <!ELEMENT daemon (log | uid | pidfile | gid | lbm-config | lbm-license-file | web-monitor)*> <!ELEMENT log ( #PCDATA )> <!ATTLIST log type CDATA #IMPLIED> <!ATTLIST log xml:space (default | preserve) \"default\"> <!ELEMENT pidfile ( #PCDATA )> <!ATTLIST pidfile xml:space (default | preserve) \"default\"> <!ELEMENT uid ( #PCDATA )> <!ATTLIST uid xml:space (default | preserve) \"default\"> <!ELEMENT gid ( #PCDATA )> <!ATTLIST gid xml:space (default | preserve) \"default\"> <!ELEMENT lbm-config ( #PCDATA )> <!ATTLIST lbm-config xml:space (default | preserve) \"default\"> <!ELEMENT lbm-license-file ( #PCDATA )> <!ATTLIST lbm-license-file xml:space (default | preserve) \"default\"> <!ELEMENT web-monitor ( #PCDATA )> <!ATTLIST web-monitor xml:space (default | preserve) \"default\"> <!ATTLIST web-monitor permission CDATA #IMPLIED> <!ELEMENT stores (store*)> <!ELEMENT store (ume-attributes | topics)+> <!ATTLIST store name CDATA #REQUIRED> <!ATTLIST store interface CDATA #IMPLIED> <!ATTLIST store port CDATA #REQUIRED> <!ELEMENT topics (topic+)> <!ELEMENT topic (ume-attributes | application-sets)*> <!ATTLIST topic pattern CDATA #REQUIRED> <!ATTLIST topic type (direct | PCRE | regexp) #IMPLIED> <!ELEMENT ume-attributes (option+)> <!ELEMENT option EMPTY> <!ATTLIST option type (lbm-receiver | lbm-context | lbm-source | store | queue) #IMPLIED> <!ATTLIST option name CDATA #REQUIRED> <!ATTLIST option value CDATA #REQUIRED> <!ELEMENT queues (queue*)> <!ELEMENT queue (indices | application-sets | ume-attributes | topics | queue-groups)+> <!ATTLIST queue name CDATA #REQUIRED> <!ATTLIST queue interface CDATA #IMPLIED> <!ATTLIST queue port CDATA #REQUIRED> <!ATTLIST queue group-index CDATA #IMPLIED> <!ATTLIST queue group-size CDATA #IMPLIED> <!ELEMENT receiver-types (receiver-type+)> <!ELEMENT indices (index+)> <!ELEMENT index (ume-attributes)*> <!ATTLIST index name CDATA #REQUIRED> <!ATTLIST index type CDATA #IMPLIED> <!ATTLIST index value CDATA #REQUIRED> <!ELEMENT application-sets (application-set+)> <!ELEMENT application-set (ume-attributes | receiver-types)*> <!ATTLIST application-set name CDATA #REQUIRED> <!ELEMENT receiver-type (ume-attributes*, index-rules?)> <!ATTLIST receiver-type id CDATA #REQUIRED> <!ELEMENT index-rules (index-allow | index-deny)*> <!ATTLIST index-rules order CDATA #IMPLIED> <!ELEMENT index-allow EMPTY> <!ATTLIST index-allow name CDATA #REQUIRED> <!ELEMENT index-deny EMPTY> <!ATTLIST index-deny name CDATA #REQUIRED> <!ELEMENT queue-groups (queue-group+)> <!ELEMENT queue-group EMPTY> <!ATTLIST queue-group index CDATA #REQUIRED> <!ATTLIST queue-group size CDATA #REQUIRED>
<?xml version="1.0"?>
<ume-store version="1.2">
<daemon>
<log>stored.log</log>
<pidfile>stored.pid</pidfile>
<web-monitor>*:15304</web-monitor>
</daemon>
<stores>
<store name="test-store" port="14567">
<ume-attributes>
<option type="store" name="disk-cache-directory" value="cache"/>
<option type="store" name="disk-state-directory" value="state"/>
<option type="store" name="context-name" value="remote-store"/>
</ume-attributes>
<topics>
<topic pattern="test*" type="PCRE">
<ume-attributes>
<option type="store" name="repository-type" value="disk"/>
<option type="store" name="repository-size-threshold" value="104857600"/>
<option type="store" name="repository-size-limit" value="209715200"/>
<option type="store" name="repository-disk-file-size-limit" value="1073741824"/>
<option type="store" name="source-activity-timeout" value="120000"/>
<option type="store" name="receiver-activity-timeout" value="120000"/>
<option type="store" name="retransmission-request-forwarding" value="0"/>
</ume-attributes>
</topic>
</topics>
</store>
</stores>
</ume-store>
<?xml version="1.0" encoding="UTF-8"?>
<ume-store version="1.2">
<daemon>
<log>/tmp/qlog.txt</log>
<lbm-license-file>/home/.ume_license</lbm-license-file>
<lbm-config>/home/queues/lbm-config.cfg</lbm-config>
<web-monitor>*:20395</web-monitor>
</daemon>
<queues>
<queue name="sample_queue" port="20555" group-index="0">
<queue-groups>
<queue-group index="0" size="5"/>
<queue-group index="1" size="1"/>
</queue-groups>
<ume-attributes>
<option type="queue" name="dissemination-model" value="SQD"/>
<option type="queue" name="sinc-data-filename" value="/tmp/sample_queue_1-sd"/>
<option type="queue" name="sinc-queue-swap-filename" value="/tmp/sample_queue_1-sqs"/>
<option type="queue" name="sinc-log-filename" value="/tmp/sample_queue_1-sl"/>
<option type="queue" name="sinc-max-size" value="25000000"/>
<option type="queue" name="sending-threads" value="1"/>
<option type="queue" name="sinc-block-swap-threshold" value="100000"/>
<option type="queue" name="source-missing-message-timeout" value="5000"/>
<option type="lbm-source" name="transport" value="lbtrm"/>
</ume-attributes>
<application-sets>
<application-set name="Set 1">
<ume-attributes>
<option type="queue" name="log-audit-trail" value="0"/>
<option type="queue" name="message-lifetime" value="6000"/>
<option type="queue" name="message-reassignment-timeout" value="2000"/>
<option type="queue" name="message-max-reassignments" value="10"/>
</ume-attributes>
<receiver-types>
<receiver-type id="100">
<ume-attributes>
<option type="queue" name="priority" value="1"/>
<option type="queue" name="portion" value="10"/>
</ume-attributes>
</receiver-type>
<receiver-type id="101">
<ume-attributes>
<option type="queue" name="priority" value="100"/>
<option type="queue" name="portion" value="10"/>
</ume-attributes>
</receiver-type>
</receiver-types>
</application-set>
<application-set name="Set 2">
<ume-attributes>
<option type="queue" name="log-audit-trail" value="0"/>
<option type="queue" name="message-lifetime" value="6000"/>
<option type="queue" name="message-reassignment-timeout" value="2000"/>
<option type="queue" name="message-max-reassignments" value="10"/>
</ume-attributes>
<receiver-types>
<receiver-type id="200">
<ume-attributes>
<option type="queue" name="priority" value="1"/>
<option type="queue" name="portion" value="10"/>
</ume-attributes>
</receiver-type>
<receiver-type id="201">
<ume-attributes>
<option type="queue" name="priority" value="100"/>
<option type="queue" name="portion" value="10"/>
</ume-attributes>
</receiver-type>
</receiver-types>
</application-set>
</application-sets>
<topics>
<topic pattern="testA" type="PCRE">
<application-sets>
<application-set name="Set 1"/>
</application-sets>
</topic>
<topic pattern="testB" type="PCRE">
<application-sets>
<application-set name="Set 2"/>
</application-sets>
</topic>
<topic pattern="testC" type="PCRE">
<application-sets>
<application-set name="Set 1"/>
<application-set name="Set 2"/>
</application-sets>
</topic>
</topics>
</queue>
</queues>
</ume-store>
Copyright 2007 - 2014 Informatica Corporation.