Guide for Persistence
Persistence Concepts

In discussing Persistence, we refer to specific recovery from the failures of sources, receivers, and persistent stores. Failed sources can restart and resume sending data from the point at which they stopped. Receivers can recover from failure and begin receiving data from the point immediately prior to failure. This process is sometimes called durable subscription. Persistent stores can also be restarted and continue providing persistence to the sources and receivers that they serve. Persistence is not designed to address ongoing, corrupting agents. Rather, if one of its components fails, the design of Persistence enables it to continue supporting its ongoing operations at some level.

UM offers persistence in the following two modes:

  • Source-paced Persistence (SPP) - default mode - the rate of message consumption by receivers does not constrain the rate a source can send. Persistent Stores write all messages to non-volatile storage, and messages are retained until they are overwritten when the allocated storage is filled. See Persistence Normal Operation.
  • Receiver-paced Persistence (RPP) - optional mode - the rate of message consumption by receivers does constrain the rate a source can send. Persistent Stores only write message to non-volatile storage if one or more necessary receiver is slow in consuming the messages, and messages are deleted from the Store once all necessary receivers have consumed the message. See RPP Normal Operation.


Persistent Store Concept  <-

UM uses a daemon to persist source and receiver state outside the actual sources and receivers themselves. This daemon is the Persistent Store. The store can persist state in memory as well as on disk. State is persisted on a per-topic, per-source basis by the store. Persistent stores need not be a single entity. For fault tolerance purposes, it is possible to configure multiple stores in various ways.

For more information, see:


Registration Identifier Concept  <-

UM persistence identifies sources and receivers with Registration Identifiers, also called Registration IDs or RegIDs. A RegID is a 32-bit number that uniquely identifies a source or a receiver to a store. This means that RegIDs are also specific to a store and can be reused between individual stores, if needed. No two active sources or receivers can share a RegID or use the same RegID at the same time. This point is critical: since UM enables your application to use and handle RegIDs very freely, you must use RegIDs carefully to avoid destructive results.

For more information, see:

Delivery Confirmation Concept  <-

A persistent receiver provides confirmation (acknowledgement) to the persistent store as it consumes (processes) messages. This is fundamental to the design of UM persistence.

The receiver can optionally provide this confirmation (acknowledgment) to the persistent source. These confirmations are turned off by default, but can be requested through either or both two configuration options:

These two options are unrelated to each other, except that they both request the receiver to send delivery confirmations. Note that when either or both of the options are set, the persistent source requests that the persistent receiver supply delivery confirmations. The persistent receiver has the option to decline the request by setting the option ume_allow_confirmed_delivery (receiver) to 0.

Note
Smart Sources do not support either form of delivery confirmation.

The latter option, ume_retention_unique_confirmations (source), can provide a form of receiver-pacing; the source will not be allowed to exceed Persistence Flight Size beyond receiving applications. For more information, see: Confirmed Delivery


Release Policy  <-

Sources and persistent stores retain data according to a release policy, which is a set of rules that specifies when a message can be reclaimed. Each rule would allow any message that complies with the rule to be reclaimed. However, a message must comply with all rules before it can be reclaimed. Conversely, any message not complying with all rules will not be reclaimed. A source or store retains messages until its retention policy dictates the message may be removed. Sources and stores use slightly different retention policies based on their individual roles.

For more information, see Source Message Retention and Release.


Message Stability Concept  <-

Sources send messages to both receivers and to stores. Messages become stable once the message has been persisted at the store or a set of stores, and those stores acknowledge stability to the sources. Since it takes time to write messages to disk and signal stability, the source is allowed to continue sending messages while waiting for stability acknowledgements. Any messages sent but not yet acknowledged are said to be "<i>in flight</i>". The number of in-flight messages is normally limited. For more information, see Persistence Flight Size.

In addition, UM informs the application when messages are stabilized. Until that stability acknowledgement is received, the source can not assume the messages will be successfully delivered. The message stability acknowledgement is vital to ensuring that messages will not be lost. For more information, see Source Message Retention and Release.


Quorum/Consensus Store Failover  <-

Typically, multiple stores are deployed for simultaneous redundant operation. In this configuration, one or more stores (or the hosts they run on) can fail without impacting the message flow from sources to receivers, as long as a quorum of the configured stores is operational. UM defines a quorum as a majority of the configured stores. E.g. if 3 stores are configured, messaging can continue as long as at least 2 are operational. If 5 stores are configured, messaging can continue if at least 3 are operational. (Quorum/Consensus requires that an odd number of stores be configured.)