docs/rt_tables.rst - oransc/ric-plt/lib/rmr - Gitiles

 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. SPDX-License-Identifier: CC-BY-4.0
 .. CAUTION: this document is generated from source in doc/src/rtd.
 .. To make changes edit the source and recompile the document.
 .. Do NOT make changes directly to .rst or .md files.

 ============================================================================================
 Route Table Guide
 ============================================================================================
 --------------------------------------------------------------------------------------------
 RIC Message Router -- RMR
 --------------------------------------------------------------------------------------------


 Overview
 ========

 Messages sent via the RIC Message Router (RMR) are routed to
 an endpoint (another application) based on a combination of
 the *message type* (MT) and *subscription ID* (SID) supplied
 in the message. RMR determines the endpoint by matching the
 MT and SID combination to an entry in a route table which has
 been supplied dynamically by a *Route Manager* service, or as
 a static table loaded during RMR initialisation. It is also
 possible to route messages directly to an endpoint which is
 the *managed entity* "owner," using the *managed entity ID*
 (MEID).

 For most xAPP developers the format of the RMR route table is
 not important beyond understanding how to create a static
 table for local testing. For developers of a *Route Manager*
 service, the need is certainly a requirement. This document
 describes the overall syntax of a route table and the
 interface between the *Route Manager* service and RMR.


 Contents of a Route Table
 =========================

 The table consists of a start record, one or more entry
 records, and an end record. Each entry record defines one
 message type, with an optional sender application, and the
 endpoint(s) which accept the indicated message type. All
 table records contain fields separated with vertical bars
 (|), and allow for trailing comments with the standard shell
 comment symbol (hash, #) provided that the start of the
 comment is separated from the last token on the record by one
 or more spaces. Leading and trailing white space in each
 field is ignored. Figure 1 illustrates a very basic route
 table with two message types, 1000 and 2000, and two
 subscription IDs for message type 1000.


 ::

     newrt | start | rt-0928
     rte   | 2000  | logger:30311
     mse   | 1000  | 10 | forwarder:43086
     mse   | 1000  | 21 | app0:43086,app1:43086
     newrt | end   | 3

 Figure 1: A basic route table.


 Entry record syntax
 -------------------

 Two types of table entries are supported for compatibility
 with the original RMR implementation, but only the *mse*
 entry type is needed and that should be the entry used when
 creating new tables. The following shows the syntax for both
 entry types:


 ::

    rte | <msg-type>[,<sender-endpoint>] | <endpoint-group>[;<endpoint-group>;...]
    mse | <msg-type>[,<sender-endpoint>] | <sub-id> | <endpoint-group>[;<endpoint-group>;...]


 Where:


     .. list-table::
       :widths: 25,70
       :header-rows: 0
       :class: borderless

       * - **mse, rte**
         -
           is the table entry type

       * - **<msg-type>**
         -
           is the integer message type

       * - **<sender-endpoint>**
         -
           is the endpoint description of the message sender; only that
           sender will read the entry from the table, so a single table
           may be used for all senders when a common message type is
           delivered to varying endpoints based on senders. If the
           sender endpoint is omitted from the entry, then the entry
           will be used by all applications.

       * - **<sub-id>**
         -
           is the subscription id (integer) for subscription-based
           messages, or -1 if the message type is not
           subscription-based. An *mse* entry with a sub-id of -1 is the
           **same** as an *rte* entry with the same message type.

       * - **<endpoint-group>**
         -
           is one or more, comma separated, endpoint descriptions.


 When an application sends a message with the indicated type,
 the message will be sent to one endpoint in the group in a
 round-robin ordering. If multiple endpoint groups are given,
 then the message is sent to a member selected from each
 group; 3 groups, then three messages will be sent. The first
 group is required.


 Line separation
 ---------------

 Table entries **must** end with a record termination sequence
 which may be one of the following three sequences:


 * a single newline (\\n)
 * a DOS style CRLF pair (\\r\\n)
 * a single carriage return (\\r)


 Care must be taken when manually editing a static table; some
 editors do **not** add a final record termination sequence to
 the last line of a file. RMR expects the final record to have
 a termination sequence to ensure that the record was not
 truncated (especially important when receiving dynamic
 tables).


 Table framing
 -------------

 The route table parser within RMR assumes that route table
 entries are sent via RMR messages as a stream. To ensure
 synchronisation and prevent malformed tables because of
 broken sessions or lost packets, each table must begin and
 end with an *newrt* record. Each *newrt* record has one of
 two possible syntax layouts as described below.


 ::

     newrt | begin [| table-id-string]
     newrt | end  [| record-count]

 Figure 2: Illustration of the newrt records in the table.

 The *table-id-string* is an optional string which is used by
 RMR when sending an acknowledgement back to the *Route
 Manager* service (see the *Route Manager Interface* section
 for more details). If a *record-count* is given as the final
 field on the *end* record, RMR will verify that the number of
 *mse* and *rte* entries in the table matches the count; if
 there is a mismatch in values the table is not used.


 Comments, spaces, and blank lines
 ---------------------------------

 Comments may be placed to the right of any table entry line
 using the standard shell comment symbol (#). The start of a
 comment must be separated from any previous record content by
 at least one space or tab. Complete lines are treated as
 comments when the first non-whitespace character of a line is
 a comment symbol. Blank lines are also ignored.

 Fields on table records are separated using the vertical bar
 (|) character. Any white space (tabs or spaces) which appear
 immediately before or after a separator are ignored.


 Endpoint Description
 --------------------

 The endpoint description is either the hostname or IP address
 followed by a port number; the two are separated by a single
 colon. The illustration below assumes that host names (e.g.
 forwarder and app1) are defined; they also make the tables
 easier to read. The port number given is the port number that
 the user application provides to RMR when the RMR
 initialisation function is invoked (and thus is the port that
 RMR is listening on).


 Table Mechanics
 ===============

 Creating a table from the two entry types is fairly simple,
 however there are some subtleties which should be pointed out
 to avoid unexpected behaviour. For this discussion the
 following complete table will be used.

 .. list-table::
   :widths: 75,10
   :header-rows: 0
   :class: borderless


   * -

        ::

            newrt | start | rt-0928
            rte | 2000 | logger:30311
            mse | 1000 | 10 | forwarder:43086
            mse | 1000,forwarder:43086 | 10 | app2:43086
            mse | 1000 | -1 | app0:43086,app1:43086; logger:20311
            newrt | end | 4

     -

        ::

          (1)
          (2)
          (3)
          (4)
          (5)
          (6)


 Figure 3: A complete RMR routing table (line numbers to the
 right for reference).


 Table Entry Ordering
 --------------------

 Whether a table is read from a file on disk, or is received
 from a *Route Manager* service, RMR parses the records to
 build an internal route table keeping only the relevant
 information. Entries are read in the order they appear (from
 the file or in messages received), and RMR will use only one
 entry for each MT/SID pair.

 For most tables, the ordering of entries is not important,
 but when there are entries which duplicate the MT/SID pair
 ordering becomes significant. RMR will use the **last** valid
 entry for a MT/SID pair that it encounters. An entry is
 considered valid if there is no sender identified with the
 message type (line 3), and when the sender (host and port)
 match the the applications' location and the port provided to
 RMR for listening.

 Using the table in figure 3 as an example, there are two
 entries which match the MT/SID pair of 1000/10. When this
 table is parsed on any host, RMR will recognise and add the
 first entry (line 3) to the internal representation; this
 entry is valid for all applications. The second 1000/10 entry
 (line 4) is valid when the table is parsed on the *forwarder*
 host, and only by the application which is listening on port
 43086. For this application the entry will override the more
 generic entry for the MT/SID combination.

 As a rule, the ordering of entries for a given MT/SID pair
 should be from most generic to most specific.


 Route Manager Communications
 ============================

 During initialisation RMR will use the value of the
 ``RMR_RTG_SVC`` environment variable to connect to the *Route
 Manager* service in order to request a route table. The
 connection between RMR and the *Route Manager* is also an RMR
 session and thus RMR messages will be used to exchange
 requests and responses.


 Table Request
 -------------

 During initialisation, RMR establishes a wormhole connection
 to the *Route Manager* and sends a message type of 21 to
 request a new table. RMR will continue to send table requests
 until a table is received and accepted; in other words it is
 fine for the *Route Manager* to ignore the requests if it is
 not ready to respond.


 Sending Tables To RMR
 ---------------------

 Table entry data is expected to arrive via RMR message with a
 message type of 20. The message may contain one or more
 entries provided that the entries are newline separated.
 Current versions of RMR support very large messages, however
 to ensure compatibility with an xAPP built using an older
 version of RMR (pre 3.8), messages should be limited to 4
 KiB.


 Table Acceptance and Acknowledgement
 ------------------------------------

 When RMR receives the table end entry (newrt|end), it will
 send a state message back to the *Route Manager* to indicate
 the state of the received table. The message type is 22 and
 the payload will contain UTF-8 tokens which indicate the
 state. The second token will be the *table ID* supplied on
 the start record, or the string "<id-missing>." When the
 state is an error state, RMR might add a final set of tokens
 which contain the reason for the failure.

 Upon receipt of a status message which indicates an "OK"
 response, the *Route Manager* can assume that the table has
 been installed and is in use. Any other response indicates
 that RMR did not use the table and has dropped it; the
 previous table is still in use.


 Using A Static Route Table
 --------------------------

 A static route table can be provided to assist with testing,
 or to provide a bootstrap set of route information until a
 dynamic table is received from a routing manager. The
 environment variable ``RMR_SEED_RT`` is checked during RMR
 initialisation and if set is expected to reference a file
 containing a route table. This table will be loaded and used
 until overlaid by a table sent by the *Route Manager*.

 For testing, the static table will be reloaded periodically
 if the ``RMR_RTG_SVC`` environment variable is set to -1.
 When this testing feature is enabled RMR will not listen for
 *Route Manager* connections, nor will it attempt to request a
 dynamic table.


 Routing Using MEID
 ==================

 Starting with version 1.13.0, RMR provides the ability to
 select the endpoint for a message based on the MEID (managed
 entity ID) in the message, rather than selecting the endpoint
 from the round-robin list for the matching route table entry.
 When the MEID is used, the message is sent to the endpoint
 which *owns,* or is responsible for the managed entity.
 Should the *owner* change messages will be routed to the new
 owner when the route table is updated. To make use of MEID
 routing, there must be one or more route table entries which
 list the special endpoint name ``%meid`` instead of providing
 a round robin list. As an example, consider the following
 route table entry:


 ::

    mse| 1000,forwarder:43086 | 10 | %meid

 Figure 4: Sample route entry with the meid flag.

 The final field of the entry doesn't specify a round-robin
 group which means that when an application attempts to send a
 message with type 1000, and the subscription ID of 10, the
 MEID in the message will be used to select the endpoint.


 MEID endpoint selection
 -----------------------

 To select an endpoint for the message based on the MEID in a
 message, RMR must know which endpoint owns the MEID. This
 information, known as an MEID map, is provided by the *Route
 Manager* over the same communication path as the route table
 is supplied. The following is the syntax for an MEID map.


 ::

    meid_map | start | <table-id>
    mme_ar | <owner-endpoint> | <meid> [<meid>...]
    mme_del | <meid> [<meid>...]
    meid_map | end | <count> [| <md5sum> ]

 Figure 5: Meid map table.

 The mme_ar records are add/update records and allow for the
 list of MEIDs to be associated with (owned by) the indicated
 endpoint. The <owner-endpoint> is the hostname:port, or IP
 address and port, of the application which owns the MEID and
 thus should receive any messages which are routed based on a
 route table entry with %meid as the round-robin group. The
 mme_del records allow for MEIDs to be deleted from RMR's
 view. Finally, the <count> is the number of add/replace and
 delete records which were sent; if RMR does not match the
 <count> value to the number of records, then it will not add
 the data to the table. Updates only need to list the
 ownership changes that are necessary; in other words, the
 *Route Manager* does not need to supply all of the MEID
 relationships with each update.

 The optional <md5sum> field on the end record should be the
 MD5 hash of all of the records between the start and end
 records. This allows for a precise verification that the
 transmitted data was correctly received.

 If a static seed file is being used for the route table, a
 second section can be given which supplies the MEID map. The
 following is a small example of a seed file:


 ::

   newrt|start | id-64306
   mse|0|-1| %meid
   mse|1|-1|172.19.0.2:4560
   mse|2|-1|172.19.0.2:4560
   mse|3|-1|172.19.0.2:4560
   mse|4|-1|172.19.0.2:4560
   mse|5|-1|172.19.0.2:4560
   newrt|end

   meid_map | start | id-028919
   mme_ar| 172.19.0.2:4560 | meid000 meid001 meid002 meid003 meid004 meid005
   mme_ar| 172.19.0.42:4560 | meid100 meid101 meid102 meid103
   mme_del | meid1000
   meid_map | end | 1

 Figure 6: Illustration of both a route table and meid map in
 the same file.

 The tables above will route all messages with a message type
 of 0 based on the MEID. There are 10 meids which are owned by
 two different endpoints. The table also deletes the MEID
 meid1000 from RMR's view.


 Reserved Message Types
 ======================

 RMR is currently reserving message types in the range of 0
 through 99 (inclusive) for its own use. Please do not use
 these types in any production or test environment as the
 results may be undesired.


 Appendix A -- Glossary
 ======================

 Many terms in networking can be interpreted with multiple
 meanings, and several terms used in various RMR documentation
 are RMR specific. The following definitions are the meanings
 of terms used within RMR documentation and should help the
 reader to understand the intent of meaning.

    .. list-table::
      :widths: 25,70
      :header-rows: 0
      :class: borderless

      * - **application**
        -
          A programme which uses RMR to send and/or receive messages
          to/from another RMR based application.

      * - **Critical error**
        -
          An error that RMR has encountered which will prevent further
          successful processing by RMR. Critical errors usually
          indicate that the application should abort.

      * - **Endpoint**
        -
          An RMR based application that is defined as being capable of
          receiving one or more types of messages (as defined by a
          *routing key.*)

      * - **Environment variable**
        -
          A key/value pair which is set externally to the application,
          but which is available to the application (and referenced
          libraries) through the ``getenv`` system call. Environment
          variables are the main method of communicating information
          such as port numbers to RMR.

      * - **Error**
        -
          An abnormal condition that RMR has encountered, but will not
          affect the overall processing by RMR, but may impact certain
          aspects such as the ability to communicate with a specific
          endpoint. Errors generally indicate that something, usually
          external to RMR, must be addressed.

      * - **Host name**
        -
          The name of the host as returned by the ``gethostbyname``
          system call. In a containerised environment this might be the
          container or service name depending on how the container is
          started. From RMR's point of view, a host name can be used to
          resolve an *endpoint* definition in a *route* table.)

      * - **IP**
        -
          Internet protocol. A low level transmission protocol which
          governs the transmission of datagrams across network
          boundaries.

      * - **Listen socket**
        -
          A *TCP* socket used to await incoming connection requests.
          Listen sockets are defined by an interface and port number
          combination where the port number is unique for the
          interface.

      * - **Message**
        -
          A series of bytes transmitted from the application to another
          RMR based application. A message is comprised of RMR specific
          data (a header), and application data (a payload).

      * - **Message buffer**
        -
          A data structure used to describe a message which is to be
          sent or has been received. The message buffer includes the
          payload length, message type, message source, and other
          information.

      * - **Message type**
        -
          A signed integer (0-32000) which identifies the type of
          message being transmitted, and is one of the two components
          of a *routing key.* See *Subscription ID.*

      * - **Payload**
        -
          The portion of a message which holds the user data to be
          transmitted to the remote *endpoint.* The payload contents
          are completely application defined.

      * - **RMR context**
        -
          A set of information which defines the current state of the
          underlying transport connections that RMR is managing. The
          application will be give a context reference (pointer) that
          is supplied to most RMR functions as the first parameter.

      * - **Round robin**
        -
          The method of selecting an *endpoint* from a list such that
          all *endpoints* are selected before starting at the head of
          the list.

      * - **Route table**
        -
          A series of "rules" which define the possible *endpoints* for
          each *routing key.*

      * - **Route table manager**
        -
          An application responsible for building a *route table* and
          then distributing it to all applicable RMR based
          applications.

      * - **Routing**
        -
          The process of selecting an *endpoint* which will be the
          recipient of a message.

      * - **Routing key**
        -
          A combination of *message type* and *subscription ID* which
          RMR uses to select the destination *endpoint* when sending a
          message.

      * - **Source**
        -
          The sender of a message.

      * - **Subscription ID**
        -
          A signed integer value (0-32000) which identifies the
          subscription characteristic of a message. It is used in
          conjunction with the *message type* to determine the *routing
          key.*

      * - **Target**
        -
          The *endpoint* selected to receive a message.

      * - **TCP**
        -
          Transmission Control Protocol. A connection based internet
          protocol which provides for lossless packet transportation,
          usually over IP.

      * - **Thread**
        -
          Also called a *process thread, or pthread.* This is a
          lightweight process which executes in concurrently with the
          application and shares the same address space. RMR uses
          threads to manage asynchronous functions such as route table
          updates.

      * - **Trace information**
        -
          An optional portion of the message buffer that the
          application may populate with data that allows for tracing
          the progress of the transaction or application activity
          across components. RMR makes no use of this data.

      * - **Transaction ID**
        -
          A fixed number of bytes in the *message* buffer) which the
          application may populate with information related to the
          transaction. RMR makes use of the transaction ID for matching
          response messages with the &c function is used to send a
          message.

      * - **Transient failure**
        -
          An error state that is believed to be short lived and that
          the operation, if retried by the application, might be
          successful. C programmers will recognise this as
          ``EAGAIN.``

      * - **Warning**
        -
          A warning occurs when RMR has encountered something that it
          believes isn't correct, but has a defined work round.

      * - **Wormhole**
        -
          A direct connection managed by RMR between the user
          application and a remote, RMR based, application.
	.. This work is licensed under a Creative Commons Attribution 4.0 International License.
	.. SPDX-License-Identifier: CC-BY-4.0
	.. CAUTION: this document is generated from source in doc/src/rtd.
	.. To make changes edit the source and recompile the document.
	.. Do NOT make changes directly to .rst or .md files.

	============================================================================================
	Route Table Guide
	============================================================================================
	--------------------------------------------------------------------------------------------
	RIC Message Router -- RMR
	--------------------------------------------------------------------------------------------


	Overview
	========

	Messages sent via the RIC Message Router (RMR) are routed to
	an endpoint (another application) based on a combination of
	the message type (MT) and subscription ID (SID) supplied
	in the message. RMR determines the endpoint by matching the
	MT and SID combination to an entry in a route table which has
	been supplied dynamically by a Route Manager service, or as
	a static table loaded during RMR initialisation. It is also
	possible to route messages directly to an endpoint which is
	the managed entity "owner," using the managed entity ID
	(MEID).

	For most xAPP developers the format of the RMR route table is
	not important beyond understanding how to create a static
	table for local testing. For developers of a Route Manager
	service, the need is certainly a requirement. This document
	describes the overall syntax of a route table and the
	interface between the Route Manager service and RMR.


	Contents of a Route Table
	=========================

	The table consists of a start record, one or more entry
	records, and an end record. Each entry record defines one
	message type, with an optional sender application, and the
	endpoint(s) which accept the indicated message type. All
	table records contain fields separated with vertical bars
	(\|), and allow for trailing comments with the standard shell
	comment symbol (hash, #) provided that the start of the
	comment is separated from the last token on the record by one
	or more spaces. Leading and trailing white space in each
	field is ignored. Figure 1 illustrates a very basic route
	table with two message types, 1000 and 2000, and two
	subscription IDs for message type 1000.


	::

	newrt \| start \| rt-0928
	rte \| 2000 \| logger:30311
	mse \| 1000 \| 10 \| forwarder:43086
	mse \| 1000 \| 21 \| app0:43086,app1:43086
	newrt \| end \| 3

	Figure 1: A basic route table.


	Entry record syntax
	-------------------

	Two types of table entries are supported for compatibility
	with the original RMR implementation, but only the mse
	entry type is needed and that should be the entry used when
	creating new tables. The following shows the syntax for both
	entry types:


	::

	rte \| <msg-type>[,<sender-endpoint>] \| <endpoint-group>[;<endpoint-group>;...]
	mse \| <msg-type>[,<sender-endpoint>] \| <sub-id> \| <endpoint-group>[;<endpoint-group>;...]


	Where:


	.. list-table::
	:widths: 25,70
	:header-rows: 0
	:class: borderless

	* - mse, rte
	-
	is the table entry type

	* - <msg-type>
	-
	is the integer message type

	* - <sender-endpoint>
	-
	is the endpoint description of the message sender; only that
	sender will read the entry from the table, so a single table
	may be used for all senders when a common message type is
	delivered to varying endpoints based on senders. If the
	sender endpoint is omitted from the entry, then the entry
	will be used by all applications.

	* - <sub-id>
	-
	is the subscription id (integer) for subscription-based
	messages, or -1 if the message type is not
	subscription-based. An mse entry with a sub-id of -1 is the
	same as an rte entry with the same message type.

	* - <endpoint-group>
	-
	is one or more, comma separated, endpoint descriptions.



	When an application sends a message with the indicated type,
	the message will be sent to one endpoint in the group in a
	round-robin ordering. If multiple endpoint groups are given,
	then the message is sent to a member selected from each
	group; 3 groups, then three messages will be sent. The first
	group is required.


	Line separation
	---------------

	Table entries must end with a record termination sequence
	which may be one of the following three sequences:


	* a single newline (\\n)
	* a DOS style CRLF pair (\\r\\n)
	* a single carriage return (\\r)


	Care must be taken when manually editing a static table; some
	editors do not add a final record termination sequence to
	the last line of a file. RMR expects the final record to have
	a termination sequence to ensure that the record was not
	truncated (especially important when receiving dynamic
	tables).


	Table framing
	-------------

	The route table parser within RMR assumes that route table
	entries are sent via RMR messages as a stream. To ensure
	synchronisation and prevent malformed tables because of
	broken sessions or lost packets, each table must begin and
	end with an newrt record. Each newrt record has one of
	two possible syntax layouts as described below.


	::

	newrt \| begin [\| table-id-string]
	newrt \| end [\| record-count]

	Figure 2: Illustration of the newrt records in the table.

	The table-id-string is an optional string which is used by
	RMR when sending an acknowledgement back to the *Route
	Manager* service (see the Route Manager Interface section
	for more details). If a record-count is given as the final
	field on the end record, RMR will verify that the number of
	mse and rte entries in the table matches the count; if
	there is a mismatch in values the table is not used.


	Comments, spaces, and blank lines
	---------------------------------

	Comments may be placed to the right of any table entry line
	using the standard shell comment symbol (#). The start of a
	comment must be separated from any previous record content by
	at least one space or tab. Complete lines are treated as
	comments when the first non-whitespace character of a line is
	a comment symbol. Blank lines are also ignored.

	Fields on table records are separated using the vertical bar
	(\|) character. Any white space (tabs or spaces) which appear
	immediately before or after a separator are ignored.


	Endpoint Description
	--------------------

	The endpoint description is either the hostname or IP address
	followed by a port number; the two are separated by a single
	colon. The illustration below assumes that host names (e.g.
	forwarder and app1) are defined; they also make the tables
	easier to read. The port number given is the port number that
	the user application provides to RMR when the RMR
	initialisation function is invoked (and thus is the port that
	RMR is listening on).


	Table Mechanics
	===============

	Creating a table from the two entry types is fairly simple,
	however there are some subtleties which should be pointed out
	to avoid unexpected behaviour. For this discussion the
	following complete table will be used.

	.. list-table::
	:widths: 75,10
	:header-rows: 0
	:class: borderless


	* -

	::

	newrt \| start \| rt-0928
	rte \| 2000 \| logger:30311
	mse \| 1000 \| 10 \| forwarder:43086
	mse \| 1000,forwarder:43086 \| 10 \| app2:43086
	mse \| 1000 \| -1 \| app0:43086,app1:43086; logger:20311
	newrt \| end \| 4

	-

	::

	(1)
	(2)
	(3)
	(4)
	(5)
	(6)


	Figure 3: A complete RMR routing table (line numbers to the
	right for reference).


	Table Entry Ordering
	--------------------

	Whether a table is read from a file on disk, or is received
	from a Route Manager service, RMR parses the records to
	build an internal route table keeping only the relevant
	information. Entries are read in the order they appear (from
	the file or in messages received), and RMR will use only one
	entry for each MT/SID pair.

	For most tables, the ordering of entries is not important,
	but when there are entries which duplicate the MT/SID pair
	ordering becomes significant. RMR will use the last valid
	entry for a MT/SID pair that it encounters. An entry is
	considered valid if there is no sender identified with the
	message type (line 3), and when the sender (host and port)
	match the the applications' location and the port provided to
	RMR for listening.

	Using the table in figure 3 as an example, there are two
	entries which match the MT/SID pair of 1000/10. When this
	table is parsed on any host, RMR will recognise and add the
	first entry (line 3) to the internal representation; this
	entry is valid for all applications. The second 1000/10 entry
	(line 4) is valid when the table is parsed on the forwarder
	host, and only by the application which is listening on port
	43086. For this application the entry will override the more
	generic entry for the MT/SID combination.

	As a rule, the ordering of entries for a given MT/SID pair
	should be from most generic to most specific.


	Route Manager Communications
	============================

	During initialisation RMR will use the value of the
	``RMR_RTG_SVC`` environment variable to connect to the *Route
	Manager* service in order to request a route table. The
	connection between RMR and the Route Manager is also an RMR
	session and thus RMR messages will be used to exchange
	requests and responses.


	Table Request
	-------------

	During initialisation, RMR establishes a wormhole connection
	to the Route Manager and sends a message type of 21 to
	request a new table. RMR will continue to send table requests
	until a table is received and accepted; in other words it is
	fine for the Route Manager to ignore the requests if it is
	not ready to respond.


	Sending Tables To RMR
	---------------------

	Table entry data is expected to arrive via RMR message with a
	message type of 20. The message may contain one or more
	entries provided that the entries are newline separated.
	Current versions of RMR support very large messages, however
	to ensure compatibility with an xAPP built using an older
	version of RMR (pre 3.8), messages should be limited to 4
	KiB.


	Table Acceptance and Acknowledgement
	------------------------------------

	When RMR receives the table end entry (newrt\|end), it will
	send a state message back to the Route Manager to indicate
	the state of the received table. The message type is 22 and
	the payload will contain UTF-8 tokens which indicate the
	state. The second token will be the table ID supplied on
	the start record, or the string "<id-missing>." When the
	state is an error state, RMR might add a final set of tokens
	which contain the reason for the failure.

	Upon receipt of a status message which indicates an "OK"
	response, the Route Manager can assume that the table has
	been installed and is in use. Any other response indicates
	that RMR did not use the table and has dropped it; the
	previous table is still in use.


	Using A Static Route Table
	--------------------------

	A static route table can be provided to assist with testing,
	or to provide a bootstrap set of route information until a
	dynamic table is received from a routing manager. The
	environment variable ``RMR_SEED_RT`` is checked during RMR
	initialisation and if set is expected to reference a file
	containing a route table. This table will be loaded and used
	until overlaid by a table sent by the Route Manager.

	For testing, the static table will be reloaded periodically
	if the ``RMR_RTG_SVC`` environment variable is set to -1.
	When this testing feature is enabled RMR will not listen for
	Route Manager connections, nor will it attempt to request a
	dynamic table.


	Routing Using MEID
	==================

	Starting with version 1.13.0, RMR provides the ability to
	select the endpoint for a message based on the MEID (managed
	entity ID) in the message, rather than selecting the endpoint
	from the round-robin list for the matching route table entry.
	When the MEID is used, the message is sent to the endpoint
	which owns, or is responsible for the managed entity.
	Should the owner change messages will be routed to the new
	owner when the route table is updated. To make use of MEID
	routing, there must be one or more route table entries which
	list the special endpoint name ``%meid`` instead of providing
	a round robin list. As an example, consider the following
	route table entry:


	::

	mse\| 1000,forwarder:43086 \| 10 \| %meid

	Figure 4: Sample route entry with the meid flag.

	The final field of the entry doesn't specify a round-robin
	group which means that when an application attempts to send a
	message with type 1000, and the subscription ID of 10, the
	MEID in the message will be used to select the endpoint.


	MEID endpoint selection
	-----------------------

	To select an endpoint for the message based on the MEID in a
	message, RMR must know which endpoint owns the MEID. This
	information, known as an MEID map, is provided by the *Route
	Manager* over the same communication path as the route table
	is supplied. The following is the syntax for an MEID map.


	::

	meid_map \| start \| <table-id>
	mme_ar \| <owner-endpoint> \| <meid> [<meid>...]
	mme_del \| <meid> [<meid>...]
	meid_map \| end \| <count> [\| <md5sum> ]

	Figure 5: Meid map table.

	The mme_ar records are add/update records and allow for the
	list of MEIDs to be associated with (owned by) the indicated
	endpoint. The <owner-endpoint> is the hostname:port, or IP
	address and port, of the application which owns the MEID and
	thus should receive any messages which are routed based on a
	route table entry with %meid as the round-robin group. The
	mme_del records allow for MEIDs to be deleted from RMR's
	view. Finally, the <count> is the number of add/replace and
	delete records which were sent; if RMR does not match the
	<count> value to the number of records, then it will not add
	the data to the table. Updates only need to list the
	ownership changes that are necessary; in other words, the
	Route Manager does not need to supply all of the MEID
	relationships with each update.

	The optional <md5sum> field on the end record should be the
	MD5 hash of all of the records between the start and end
	records. This allows for a precise verification that the
	transmitted data was correctly received.

	If a static seed file is being used for the route table, a
	second section can be given which supplies the MEID map. The
	following is a small example of a seed file:


	::

	newrt\|start \| id-64306
	mse\|0\|-1\| %meid
	mse\|1\|-1\|172.19.0.2:4560
	mse\|2\|-1\|172.19.0.2:4560
	mse\|3\|-1\|172.19.0.2:4560
	mse\|4\|-1\|172.19.0.2:4560
	mse\|5\|-1\|172.19.0.2:4560
	newrt\|end

	meid_map \| start \| id-028919
	mme_ar\| 172.19.0.2:4560 \| meid000 meid001 meid002 meid003 meid004 meid005
	mme_ar\| 172.19.0.42:4560 \| meid100 meid101 meid102 meid103
	mme_del \| meid1000
	meid_map \| end \| 1

	Figure 6: Illustration of both a route table and meid map in
	the same file.

	The tables above will route all messages with a message type
	of 0 based on the MEID. There are 10 meids which are owned by
	two different endpoints. The table also deletes the MEID
	meid1000 from RMR's view.


	Reserved Message Types
	======================

	RMR is currently reserving message types in the range of 0
	through 99 (inclusive) for its own use. Please do not use
	these types in any production or test environment as the
	results may be undesired.



	Appendix A -- Glossary
	======================

	Many terms in networking can be interpreted with multiple
	meanings, and several terms used in various RMR documentation
	are RMR specific. The following definitions are the meanings
	of terms used within RMR documentation and should help the
	reader to understand the intent of meaning.

	.. list-table::
	:widths: 25,70
	:header-rows: 0
	:class: borderless

	* - application
	-
	A programme which uses RMR to send and/or receive messages
	to/from another RMR based application.

	* - Critical error
	-
	An error that RMR has encountered which will prevent further
	successful processing by RMR. Critical errors usually
	indicate that the application should abort.

	* - Endpoint
	-
	An RMR based application that is defined as being capable of
	receiving one or more types of messages (as defined by a
	routing key.)

	* - Environment variable
	-
	A key/value pair which is set externally to the application,
	but which is available to the application (and referenced
	libraries) through the ``getenv`` system call. Environment
	variables are the main method of communicating information
	such as port numbers to RMR.

	* - Error
	-
	An abnormal condition that RMR has encountered, but will not
	affect the overall processing by RMR, but may impact certain
	aspects such as the ability to communicate with a specific
	endpoint. Errors generally indicate that something, usually
	external to RMR, must be addressed.

	* - Host name
	-
	The name of the host as returned by the ``gethostbyname``
	system call. In a containerised environment this might be the
	container or service name depending on how the container is
	started. From RMR's point of view, a host name can be used to
	resolve an endpoint definition in a route table.)

	* - IP
	-
	Internet protocol. A low level transmission protocol which
	governs the transmission of datagrams across network
	boundaries.

	* - Listen socket
	-
	A TCP socket used to await incoming connection requests.
	Listen sockets are defined by an interface and port number
	combination where the port number is unique for the
	interface.

	* - Message
	-
	A series of bytes transmitted from the application to another
	RMR based application. A message is comprised of RMR specific
	data (a header), and application data (a payload).

	* - Message buffer
	-
	A data structure used to describe a message which is to be
	sent or has been received. The message buffer includes the
	payload length, message type, message source, and other
	information.

	* - Message type
	-
	A signed integer (0-32000) which identifies the type of
	message being transmitted, and is one of the two components
	of a routing key. See Subscription ID.

	* - Payload
	-
	The portion of a message which holds the user data to be
	transmitted to the remote endpoint. The payload contents
	are completely application defined.

	* - RMR context
	-
	A set of information which defines the current state of the
	underlying transport connections that RMR is managing. The
	application will be give a context reference (pointer) that
	is supplied to most RMR functions as the first parameter.

	* - Round robin
	-
	The method of selecting an endpoint from a list such that
	all endpoints are selected before starting at the head of
	the list.

	* - Route table
	-
	A series of "rules" which define the possible endpoints for
	each routing key.

	* - Route table manager
	-
	An application responsible for building a route table and
	then distributing it to all applicable RMR based
	applications.

	* - Routing
	-
	The process of selecting an endpoint which will be the
	recipient of a message.

	* - Routing key
	-
	A combination of message type and subscription ID which
	RMR uses to select the destination endpoint when sending a
	message.

	* - Source
	-
	The sender of a message.

	* - Subscription ID
	-
	A signed integer value (0-32000) which identifies the
	subscription characteristic of a message. It is used in
	conjunction with the message type to determine the *routing
	key.*

	* - Target
	-
	The endpoint selected to receive a message.

	* - TCP
	-
	Transmission Control Protocol. A connection based internet
	protocol which provides for lossless packet transportation,
	usually over IP.

	* - Thread
	-
	Also called a process thread, or pthread. This is a
	lightweight process which executes in concurrently with the
	application and shares the same address space. RMR uses
	threads to manage asynchronous functions such as route table
	updates.

	* - Trace information
	-
	An optional portion of the message buffer that the
	application may populate with data that allows for tracing
	the progress of the transaction or application activity
	across components. RMR makes no use of this data.

	* - Transaction ID
	-
	A fixed number of bytes in the message buffer) which the
	application may populate with information related to the
	transaction. RMR makes use of the transaction ID for matching
	response messages with the &c function is used to send a
	message.

	* - Transient failure
	-
	An error state that is believed to be short lived and that
	the operation, if retried by the application, might be
	successful. C programmers will recognise this as
	``EAGAIN.``

	* - Warning
	-
	A warning occurs when RMR has encountered something that it
	believes isn't correct, but has a defined work round.

	* - Wormhole
	-
	A direct connection managed by RMR between the user
	application and a remote, RMR based, application.