blob: 5ed3de18ac43e661a9145fdea0abc9e9487dfcac [file] [log] [blame]
Saryu Shah97ff5342018-05-15 18:07:54 +00001
2.. This work is licensed under a Creative Commons Attribution 4.0 International License.
3.. http://creativecommons.org/licenses/by/4.0
4
5****************
6Feature: Pooling
7****************
8
9Summary
10^^^^^^^
11
12The Pooling feature provides the ability to load-balance work across a pool of active-active Drools-PDP hosts. This particular implementation uses a DMaaP topic for communication between the hosts within the pool.
13
14The pool is adjusted automatically, with no manual intervention when:
15 * a new host is brought online
16 * a host goes offline, whether gracefully or due to a failure in the host or in the network
17
18Assumptions and Limitations
19^^^^^^^^^^^^^^^^^^^^^^^^^^^
20 * Session persistence is not required
21 * Data may be lost when processing is moved from one host to another
22 * The entire pool may shut down if the inter-host DMaaP topic becomes inaccessible
23
24 .. image:: poolingDesign.png
25
26
27Key Points
28^^^^^^^^^^
29 * Requests are received on a common DMaaP topic
30 * DMaaP distributes the requests randomly to the hosts
31 * The request topic should have at least as many partitions as there are hosts
32 * Uses a single, internal DMaaP topic for all inter-host communication
33 * Allocates buckets to each host
34 * Requests are assigned to buckets based on their respective request IDs
35 * No session persistence
36 * No objects copied between hosts
37 * Requires feature(s): distributed-locking
38 * Precludes feature(s): session-persistence, active-standby, state-management
39
40Example Scenario
41^^^^^^^^^^^^^^^^
42 1. Incoming DMaaP message is received on a topic all hosts are listening, but only one random host receives the message
43 2. Decode message to determine request ID key (message-specific operation)
44 3. Hash request ID to determine the bucket number
45 4. Look up host associated with hash bucket (most likely remote)
46 5. Publish forward message to internal DMaaP topic, including remote host, bucket number, DMaaP topic information, and message body
47 6. Remote host verifies ownership of bucket, and routes the DMaaP message to its own rule engine for processing
48
49 The figure below shows several different hosts in a pool. Each host as a copy of the bucket assignments, which specifies which buckets are assigned to which hosts. Incoming requests are mapped to a bucket, and a bucket is mapped to a host, to which the request is routed. The host table includes an entry for each active host in the pool, to which one or more buckets are mapped.
50
51 .. image:: poolingPdps.png
52
53Bucket Reassignment
54^^^^^^^^^^^^^^^^^^^
55 * When a host goes up or down, buckets are rebalanced
56 * Attempts to maintain an even distribution
57 * Leaves buckets with their current owner, where possible
58 * Takes a few buckets from each host to assign to new hosts
59
60 For example, in the diagram below, the left side shows how 32 buckets might be assigned among four different hosts. When the first host fails, the buckets from host 1 would be reassigned among the remaining hosts, similar to what is shown on the right side of the diagram. Any requests that were being processed by host 1 will be lost and must be restarted. However, the buckets that had already been assigned to the remaining hosts are unchanged, thus requests associated with those buckets are not impacted by the loss of host 1.
61
62 .. image:: poolingBuckets.png
63
64Usage
65^^^^^
66
67For pooling to be enabled, the distributed-locking feature must be also be enabled.
68
69 .. code-block:: bash
70 :caption: Enable Feature Pooling
71
72 policy stop
73
74 features enable distributed-locking
75 features enable pooling-dmaap
76
77 The configuration is located at:
78
79 * $POLICY_HOME/config/feature-pooling-dmaap.properties
80
81
82 .. code-block:: bash
83 :caption: Start the PDP-D using pooling
84
85 policy start
86
87
88 .. code-block:: bash
89 :caption: Disable the pooling feature
90
91 policy stop
92 features disable pooling-dmaap
93 policy start
94
95
96End of Document
97
98.. SSNote: Wiki page ref. https://wiki.onap.org/display/DW/Feature+Pooling
99
100