.. index:: Out of focus, OOF, Command; oof,

.. _#ug-oof:

.. _oof:

Out-of-Focus Messages (OOF)
===========================

One of the more difficult problems in messaging is knowing when a record
that previously matched a subscription has been updated so that the
record no longer matches the subscription. AMPS solves this problem by
providing an out-of-focus, or *OOF*, message to let subscribers know
that a record they have previously received no longer matches the
subscription. The OOF messages help subscribers easily maintain state
and remove records that are no longer relevant.

OOF notification is optional. A subscriber must explicitly request that
AMPS provide out-of-focus messages for a subscription.

When OOF notification has been requested, AMPS produces an ``oof``
message for any record that has previously been received by the
subscription at the point at which:

-  The record is deleted

-  The record expires

-  The record no longer matches the filter criteria, *or*

-  The subscriber is no longer entitled to view the new state of the
   record.

AMPS produces an ``oof`` message for each record that no longer matches
the subscription. The ``oof`` message is sent as part of processing the
update that caused the record to no longer match. Each ``oof`` message
contains information the subscriber can use to identify the record that
has gone out of focus and the reason that the record is now out of
focus.

Since AMPS must maintain the current state of a record to know when to
produce an ``oof`` message, these messages are only supported for SOW
topics, conflated topics, and views. The ``oof`` option is not supported
for subscriptions that do not include a SOW query or bookmark replays.

When AMPS returns an OOF message, the data contained in the body of the
message represents the updated state of the message (except as
described below). This will allow the client to make a determination as
to how to handle the data, be it to remove the data from the client view
or to change their query to broaden the filter thresholds. This enables
a client to take a different action depending on why the message no
longer matches. For example, an application may present a different icon
for an order that moves to a status of ``completed`` than it would
present for an order that moves to a status of ``cancelled``.

When a ``delta_publish`` message causes the SOW record to go out of
focus, AMPS returns the merged record.

When there is no updated message to send, AMPS sends the state of the
record before the change that produced the ``oof``. This can occur when the
message had been deleted, when the message has expired, or when an
update causes the client to no longer have permission to receive the
record.

.. index:: oof; conflation, conflation; oof

For a conflated view or a subscription that uses conflation, the data included
in the ``oof`` message will be the last data that the subscriber received. When
both an update to a message and a change that would cause the message to go
out of focus happen in the same conflation interval, the subscriber receives
an ``oof`` notification with the previously-received state of the message.
Likewise, if a change that causes a message to go out of focus and a change
that causes the message to come back into focus occur within the same
conflation interval, the subscriber receives the state of the message at the
end of the conflation interval, and does not receive an indication that the
message had gone out of focus during the conflation interval. 

.. _#oof-usage:

Usage
------

Consider the following scenario where AMPS is configured with the
following SOW key for the buyer topic:

.. _#oof-configuration-example:

.. code-block:: XML

    <SOW>
        <Topic>
            <Name>buyer</Name>
            <MessageType>xml</MessageType>
            <Key>/buyer/id</Key>
        </Topic>
    </SOW>            

*Topic Configuration*

When the following message is published, it is persisted in the SOW
topic:

.. _#oof-example-1:

.. code-block:: XML

    <buyer>
        <id>100</id>
        <loc>NY</loc>
    </buyer>

*First Publish Message*

A client issues a ``sow_and_subscribe`` request for the topic ``buyer``
with the filter ``/buyer/loc="NY"`` and the ``oof`` option set on the
request. The client will be sent the message as part of the SOW query
result.

Subsequently, the following message is published to update the ``loc``
tag to LN:

.. _#oof-example-2:

.. code-block:: XML

    <buyer>
        <id>100</id>
        <loc>LN</loc>
    </buyer>

*Second Publish Message*

The original message in the SOW cache is updated. The client does not
receive the second publish message, because that message does not match
the filter (``/buyer/loc="NY"``). This is problematic. The client has a
message that is no longer in the SOW cache and that no longer matches
the current state of the record. Since the ``oof`` option was set on
the subscription, however, the AMPS engine sends an ``oof`` message to
let these clients know that the message that they hold is no longer in
the SOW cache. The following is an example of what's returned:

.. code-block:: XML

    <?xml version="1.0" encoding="iso-8859-1"?>
    <SOAP-ENV:Envelope>
        <SOAP-ENV:Header>
            <Reason>match</Reason>
            <Tpc>buyer</Tpc>
            <Cmd>oof</Cmd>
            <MsgTyp>xml</MsgTyp>
            <SowKey>6387219447538349146</SowKey>
            <SubIds>SAMPS-1214725701_1</SubIds>
        </SOAP-ENV:Header>
        <SOAP-ENV:Body>
            <client>
                <id>100</id>
                <loc>LN</loc>
            </client>
        </SOAP-ENV:Body>
    </SOAP-ENV:Envelope>

*oof XML Example Message*

An easy way to think about the situations where AMPS sends an OOF
notification is to consider what would happen if the client re-issued
the original ``sow`` request after the above message was published. The
``/client/loc="NY"`` expression no longer matches the message in the SOW
cache and as a result, this message would not be returned.


.. _#oof-ex1:

Example
--------

To help reinforce the concept of OOF messages, and how OOF messaging can
be used in AMPS, consider a scenario where there is a GUI application
whose requirement is to display all open orders of a client. There are
several possible solutions to ensure that the GUI client data is
constantly updated as information changes, some of which are examined
below; however, the goal of this section is to build up a
``sow_and_subscribe`` message to demonstrate the power that OOF
notifications add to AMPS.

Client-Side Filtering in a sow_and_subscribe Command
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

First, consider an approach that sends a ``sow_and_subscribe`` message
on the topic ``orders`` using the filter ``/Client="Adam"``:

AMPS completes the ``sow`` portion of this call by sending all matching
messages from the ``orders`` SOW topic. AMPS then places a subscription
whereby all future messages that match the filter get sent to the
subscribing GUI client.

.. _#oof-figure-OOF1:

.. figure:: ../svg/oof/OOF5.png
   :alt: ``sow_and_subscribe`` example
   :width: 75.0%

*sow_and_subscribe example*

As the messages come in, the GUI client will be responsible for
determining the state of the order. It does this by examining the
``State`` field and determining if the state is equal to “Open” or not,
and then updating the GUI based on the information returned.

This approach puts the burden of work on the GUI and, in a high volume
environment, has the potential to make the client GUI unresponsive due
to the potential load that this filtering can place on a CPU. If a
client GUI becomes unresponsive, AMPS will queue the messages to ensure
that the client is given the opportunity to catch up. The specifics of
how AMPS handles slow clients is covered in the section called
:ref:`Slow Clients<#operations-and-deployment-best-practices-slow-clients>`.

AMPS Filtering in a sow_and_subscribe Command
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The next step is to add an additional ’AND’ clause to the filter. In
this scenario we can let AMPS do the filtering work that was previously
handled on the client. This is accomplished by modifying our original
``sow_and_subscribe`` to use the following filter:

.. code:: XML

    /Client = "Adam" AND /State = "Open"       

Similar to the above case, this ``sow_and_subscribe`` will first send
all messages from the ``orders`` SOW topic that have a ``Client`` field
matching "Adam" and a ``State`` field matching "Open". Once all of the
SOW topic messages have been sent to the client, the subscription will
ensure that all future messages matching the filter will be sent to the
client.

.. _#off-figure-OOF2:

.. figure:: ../svg/oof/OOF6.png
   :alt: State Filter in a sow_and_subscribe
   :width: 75.0%

*State Filter in a sow_and_subscribe*

There is a less obvious issue with this approach to maintaining the
client state. The problem with this solution is that, while it initially
will yield all open orders for client "Adam", this scenario is unable to
stay in sync with the server. For example, when the order for Adam is
filled, the ``State`` changes to ``State=Filled``. This means that,
inside AMPS, the order on the client will no longer match the initial
filter criteria. The client will continue to display and maintain these
out-of-sync records. Since the client is not subscribed to messages with
a ``State`` of “Filled,” the GUI client would never be updated to
reflect this change.

OOF Processing in a sow_and_subscribe Command
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The final solution is to implement the same ``sow_and_subscribe`` query
which was used in the first scenario. This time, we use the filter
requests only for the ``State`` that we're interested in, but we add the ``oof``
option to the command so the subscriber receives OOF messages.

.. code:: bash

    /Client = "Adam" AND /State = "Open"
    options: oof            

AMPS will respond immediately with the query results, exactly as it does
with a ``sow_and_subscribe`` command that does not use the ``oof`` option.

.. _#oof-figure-OOF3:

.. figure:: ../svg/oof/OOF7.png
   :alt: sow_and_subscribe with oof enabled
   :width: 75.0%

*sow_and_subscribe with* ``oof`` *enabled*

This approach provides the following advantage: for all future messages
in which the same ``Open`` order is updated, such that its status is no
longer ``Open``, AMPS will send the client an ``OOF`` message specifying
that the record which previously matched the filter criteria has fallen
out of focus. AMPS will not send any further information about the
message unless another incoming AMPS message causes that message to come
back into focus.

In the following diagram, the Publisher publishes a message stating that
Adam’s order for MSFT has been fulfilled. When AMPS processes this
message, it will notify the GUI client with an ``OOF`` message that the
original record no longer matches the filter criteria. The ``OOF``
message will include a ``Reason`` field with it in the message header,
defining the reason for the message to lose focus. In this case the
``Reason`` field will state ``match`` since the record no longer matches
the filter.

.. _#oof-figure-OOF4:

.. figure:: ../svg/oof/OOF8.png
   :alt: OOF message
   :width: 75.0%

*OOF message*

AMPS will also send ``OOF`` messages when a message is deleted or has
expired from the SOW topic.

We see the power of the ``OOF`` message when a client application wants
to have a local cache that is a subset of the SOW. This is best managed
by first issuing a query filter ``sow_and_subscribe`` which populates
the GUI, and enabling the ``oof`` option. AMPS informs our application
when those records which originally matched no longer do, at which time
the program can remove them.
