Guide for Persistence

Demonstrating Persistence

The following files are used in this section:

Filename | Content
--- | ---
ume-example-src-3.c | Source Application 3
ume-example-rcv-3.c | Receiver Application 3
ume-example-config.xml | Persistent Store Configuration File

Perform the following tasks first:

1. Build ume-example-rcv-3.c and ume-example-src-3.c. Instructions for building them are at the beginning of the source files.

2. Create default directories, umestored-cache and umestored-state in the /doc/UME directory where the other ume-example files are located. Our sample Store configuration file, ume-example-config.xml, doesn't specify directories for the Store's cache and state files, so those will be placed in the default directories.

3. Start the Store.

       $ umestored ume-example-config.xml

You should see no output if the Store started successfully. However, you should find a new log file, ume-example-stored.log, in the directory you ran the Store in. The first couple lines should look similar to below.

    Sun Oct 29 14:15:01 yyyy [INFO]: Store-5688-5273: Latency Busters Persistent Store version 6.17.1
    Sun Oct 29 14:15:01 yyyy [INFO]: Store-5688-5274: UMP 6.17.1 [UMP-6.17.1] [64-bit] Build: mmm dd yyyy, hh:mm:ss ( DEBUG license LBT-RM LBT-RU LBT-IPC LBT-SMX ) WC[PCRE 7.4 2007-09-21, regex, appcb] HRT[gettimeofday()]

You'll also be able to view the Store's web monitor. Open a web browser and go to: `http://127.0.0.1:15304/`

You should see the Store's web monitor page, which is a diagnostic and monitoring tool for the Store. See Store Web Monitor.

# Running Persistent Example Applications <-

With the Store running, let's try our example source and receiver applications.

1. Start the Receiver.

       $ ume-example-rcv-3.exe

2. Start the Source.

       $ ume-example-src-3.exe

You should see output for the source similar to the following:

    saving RegID info to "UME-example-src-RegID" - 127.0.0.1:14567:2795623327

You should see output for the receiver similar to the following:

    UME Store 0: 127.0.0.1:14567 [TCP:169.254.97.160:14371][2795623327] Requesting RegID: 0
    saving RegID info to "UME-example-rcv-RegID" - 127.0.0.1:14567:2795623327:2795623328
    Received 15 bytes on topic UME Example (sequence number 0) 'UME Message 01'
    Received 15 bytes on topic UME Example (sequence number 1) 'UME Message 02'
    Received 15 bytes on topic UME Example (sequence number 2) 'UME Message 03'
    Received 15 bytes on topic UME Example (sequence number 3) 'UME Message 04'
    ...

The example source sends 20 messages. After the 20th messages, both the source and receiver exit and print:`<br>`removing saved RegID file...`
So what just happened? Let's walk through the output line by line.

Source

    saving RegID info to "UME-example-src-RegID" - 127.0.0.1:14567:2795623327

The source successfully registered with the Store using its pre-configured Store address and port of 127.0.0.1:14567. It didn't ask for a specific RegID from the Store, so the Store automatically assigned one to it. In this case, the Store assigned the ID, 2795623327. Your source's ID will likely be different because Stores assign random RegIDs.

If you run the test again, you'll notice the source application has written a file named '`UME-example-src-RegID`' that contains the same information the source printed on startup, namely the IP address and port of the Store it registered with, along with its RegID assigned by the Store.

Receiver

    UME Store 0: 127.0.0.1:14567 [TCP:169.254.97.160:14371][2795623327] Requesting RegID: 0
    saving RegID info to "UME-example-rcv-RegID" - 127.0.0.1:14567:2795623327:2795623328

The receiver has been informed of how to connect to the Store by the source, and it also successfully registered with the Store. The Store's IP address and port are shown, followed by the source's unique identifier string (in this case, it's a TCP source on port 14371), and the source's RegID. The receiver then requests RegID 0 from the Store, which is a special value that means pick an ID for me (Although not displayed, the source requested ID 0 when it started up as well).

In parallel with the source application, the receiver application writes its RegID with this Store to the file, `UME-example-rcv-RegID`.

After sending 20 messages under normal, stable conditions, the source and receiver applications exit and remove their RegID files.

# Single Receiver Fails and Recovers <-

Perform the following procedure with the Store running to see what happens when a receiver fails and recovers:

1. Start the Receiver.

       $ ume-example-rcv-3.exe

2. Start the source. Let it run for a few seconds so the receiver gets a few messages.

       $ ume-example-src-3.exe
       UME Store 0: 127.0.0.1:14567 [TCP:169.254.97.160:14371][3735579353] Requesting RegID: 0
       saving RegID info to "UME-example-rcv-RegID" - 127.0.0.1:14567:3735579353:3735579354
       Received 15 bytes on topic UME Example (sequence number 0) 'UME Message 01'
       Received 15 bytes on topic UME Example (sequence number 1) 'UME Message 02'
       Received 15 bytes on topic UME Example (sequence number 2) 'UME Message 03'

3. Stop the receiver (Ctrl/C) and leave the source running. Wait a few more seconds so that the source sends some messages while the receiver was down.

4. Restart the Receiver and let it run to completion.

       $ ume-example-rcv-3.exe
       read in saved RegID info from "UME-example-rcv-RegID" - 127.0.0.1:14567 RegIDs
       source 3735579353, receiver 3735579354
       UME Store 0: 127.0.0.1:14567 [TCP:169.254.97.160:14371][3735579353]
       Requesting RegID: 3735579354
       Received 15 bytes on topic UME Example (sequence number 3) 'UME Message 04'
       Received 15 bytes on topic UME Example (sequence number 4) 'UME Message 05'
       Received 15 bytes on topic UME Example (sequence number 5) 'UME Message 06'
       Received 15 bytes on topic UME Example (sequence number 6) 'UME Message 07'
       Received 15 bytes on topic UME Example (sequence number 7) 'UME Message 08'
       Received 15 bytes on topic UME Example (sequence number 8) 'UME Message 09'
       Received 15 bytes on topic UME Example (sequence number 9) 'UME Message 10'
       Received 15 bytes on topic UME Example (sequence number 10) 'UME Message 11'

Notice that the receiver picked up the message stream right where it had left off - after message 3. The first few messages (which the source had sent while the receiver was down) appear to come in much faster than the source's normal rate of one per second. That's because they are being served to the receiver from the Store. The remaining messages continue to come in at the normal one-per-second rate because they're being received from the source's live message stream. This is durable subscription at work.

# Single Source Fails and Recovers <-

Perform the following procedure with the Store running to see what happens when a source fails and recovers.

1. Start the Receiver.

       $ ume-example-rcv-3.exe

2. Start the source.

       $ ume-example-src-3.exe

   Let it run for a few seconds so the receiver gets a few messages.

3. Stop the Source (Ctrl/C).

4. Restart the Source and let it run to completion.

       $ ume-example-rcv-3.exe

Source

You should see output similar to the following on the second run of the source:

    read in saved RegID info from "UME-example-src-RegID" - 127.0.0.1:14567:2118965523
    will start with message number 5
    removing saved RegID file "UME-example-src-RegID"

Receiver

The receiver's output looks like the following:

    UME Store 0: 127.0.0.1:14567 [TCP:169.254.97.160:14371][2118965523] Requesting RegID: 0
    saving RegID info to "UME-example-rcv-RegID" - 127.0.0.1:14567:2118965523:2118965524
    Received 15 bytes on topic UME Example (sequence number 0) 'UME Message 01'
    Received 15 bytes on topic UME Example (sequence number 1) 'UME Message 02'
    Received 15 bytes on topic UME Example (sequence number 2) 'UME Message 03'
    Received 15 bytes on topic UME Example (sequence number 3) 'UME Message 04'
    UME Store 0: 127.0.0.1:14567 [TCP:169.254.97.160:14371][2118965523] Requesting RegID: 2118965524
    saving RegID info to "UME-example-rcv-RegID" - 127.0.0.1:14567:2118965523:2118965524
    Received 15 bytes on topic UME Example (sequence number 4) 'UME Message 05'
    Received 15 bytes on topic UME Example (sequence number 5) 'UME Message 06'
    Received 15 bytes on topic UME Example (sequence number 6) 'UME Message 07'
    Received 15 bytes on topic UME Example (sequence number 7) 'UME Message 08'
    ...

When the source was restarted, it read in its previously saved RegID and requested the same ID when registering with the Store. The Store informed the source that it had left off at sequence number 3 (UME Message 04), and the next sequence number it should send is 4 (UME Message 05). Bringing the source back up also caused the receiver to re-register with the Store. Receivers can only find out about Stores from sources they are listening to. Once the receiver re-registered with the Store, it continued receiving messages from the source where it had left off.

# Single Store Fails <-

Perform the following procedure with the Store running to see what happens when the Store itself fails.

1. Start the Receiver.

       $ ume-example-rcv-3.exe

2. Start the source.

       $ ume-example-src-3.exe

   Let it run for a few seconds so the receiver gets a few messages.

3. Stop the Store (Ctrl/C).

Notice that with this simple example program, the source simply prints the following and exits.

    saving RegID info to "UME-example-src-RegID" - 127.0.0.1:14567:4095035673
    Store unresponsive: store 0 [127.0.0.1:14567] unresponsive
    Store unresponsive: store 0 [127.0.0.1:14567] unresponsive - no registration response.
    line 318: not currently registered with enough UMP stores

When a source application tries to send a message without being registered with a Store, the send call returns an error. Messages sent while not registered with a Store cannot be persisted. See Designing Persistent Stores for information about using multiple Stores.

Your source application(s) should assume an unresponsive Store is a temporary problem and wait before sending the message again. See `umesrc.c`, `umesrc.java`, or `umesrc.cs` for examples of this behavior.