Guide for Persistence
|
As shown in the diagram, UM provides messaging functionality as well as persistent operation.
The highlights of this architecture are:
Note that the store is not supported on all platforms. For example, while OpenVMS supports persistent clients (source and receiver), you cannot run a store on an OpenVMS system. However, an OpenVMS-based client can interoperate with a store running an any other supported platform.
The umestored daemon runs the persistent store feature. You can configure multiple stores per daemon using the '<store>
' element in the umestored XML configuration file. See Configuration Reference for Umestored. Individual stores can use separate disk cache and disk state directories and be configured to persist messages for multiple sources (topics), which are referred to as, source repositories. UM provides each umestored daemon with a Web Monitor for statistics monitoring. See Store Web Monitor.
Within a store, you configure repositories for individual topics and each can have their own set of '<topic>
' level options that affect the repository's type, size, liveness behavior and much more. If you have multiple sources sending on the same topic, UM creates a separate repository for each source. UM uses the repository options configured for the topic to apply to each source's repository. If you specify 48MB for the size of the repository and have 10 sources sending on the topic, the persistent store requires 480MB of storage for that topic.
A repository can be configured as one of the following types:
You can configure any combination of repository types within a single store configuration.
Repositories are designed as circular buffers. When age or size thresholds are met for a topic, the repository removes or overwrites messages in order to prevent reaching its configured limit, which keeps space available for new messages. UM provides UM configuration options and store configuration options to control threshold and limit behavior.
UM configuration options control source repositories for all the sources sending within the context. The default for these options, listed below, are 0 (zero) which makes the like-name option for the repository in the umestored XML configuration file active.
See Ultra Messaging Persistence Options.
Note: The above configuration options' default values can be altered for individual sources and receivers by calling lbm_src_topic_attr_setopt() before you allocate the topic.
The umestored configuration options for source/topic repositories explained below can also be used to control threshold and limit behavior. See Options for a Topic's ume-attributes Element for complete information about the following repository options.
Memory Repository
A memory type source repository has three configuration options that manage its size relative to its capacity.
repository-age-threshold - This value determines how long the repository retains messages. The repository deletes any message older than this configured value.
repository-size-threshold - The size in bytes that a repository can reach before it begins to delete the oldest retained messages. If the repository size falls below the threshold, it stops deleting old messages.
repository-size-limit - The maximum size in bytes for the repository. Once this limit is reached, the repository stops accepting new messages. The age and size thresholds should be set at levels that guarantee the size limit is never met. You should consider how fast the source sends messages, the size of the messages and the reliability of the receivers. For example, more reliable receivers mean less recovery instances, which could mean a younger age threshold.
Disk or Reduced-fd Repositories
A disk or reduced-fd type source repository maintains a memory cache in addition to the actual disk storage. It continually persists messages from the memory cache to the disk, and uses the memory cache for receiver recovery first before performing disk reads to access needed messages. It has four configuration options that manage its size relative to its capacity.
repository-age-threshold - This value determines how long the disk repository retains messages in its memory cache. The repository deletes any message from memory cache older than this configured value. These messages could have been persisted to disk and may be available for recovery.
repository-size-threshold - The size in bytes that a repository can reach before it begins to delete the oldest retained messages. These messages could have been persisted to disk and may be available for recovery. If the disk repository memory cache size falls below the threshold, it stops deleting old messages.
repository-size-limit - The maximum size in bytes for the disk repository's memory cache. Once this limit is reached, the repository stops accepting new messages. The age and size thresholds should be set at levels that guarantee the size limit is never met. You should consider how fast the source sends messages, the size of the messages and the reliability of the receivers. For example, more reliable receivers mean less recovery instances, which could mean a younger age threshold.
Sources and receivers register with a store and use individual repositories within the store. Sources can use redundant repositories configured in multiple stores in Quorum/Consensus arrangement for fault tolerance. Be aware that the arrangement of stores into Quorum/Consensus groups is a function of the source. I.e. the individual stores of a Quorum/Consensus group are not aware of each other and do not coordinate their activities.
You can identify stores with either a domainID:interface:port, interface:port or a name. Using only interface:port is more feasible in smaller implementations where the smaller number of possible IP addresses is easier to manage. Larger implementations, especially those that span topic resolution domains using UM Routers, are better served with stores identified by a name or domainID:interface:port.
UM automatically resolves and maintains a mapping between a store name and a single topic resolution domain, IP address and port. UM also automatically resolves store names if the store is located across one or more UM Routers in a different topic resolution domain.
The following lists other specifics of store identification.
Using a Single Interface and Port
Configure store for a single interface and port.
Identify the store with only the interface:port, specified in umestored configuration file.
To run the store on a different machine for any reason, you must change both the umestored XML configuration file and the UM configuration file.
Using a Range of Interfaces
Configure a store with a range of IP addresses.
Identify the store with a range of interfaces specified in the umestored configuration file.
To run the store on a different machine, you must only change the interface specified in the ume_store (source) UM configuration option, provided you use one of the interfaces in the range specified in in the umestored configuration file.
Using a Store (context) Name
Configure a store with a name instead of just IP:port. '0.0.0.0
' (INADDR_ANY
) or no value is the default for the store's interface attribute.
Identify the store with a context-name option that resolves to the interface and port - or range of interfaces and port - specified in the umestored configuration file:
OR
OR
Add the store's context name to ume_store_name (source) so sources can find and register with the store.
You do not have to make any configuration changes to run NEWYORK-1 on another machine, provided the new interface matches one of those specified in the umestored configuration file. This includes running the store in a different topic resolution domain.