| Filip Krzywka | 4bb79b7 | 2018-09-25 10:57:17 +0200 | [diff] [blame] | 1 | .. This work is licensed under a Creative Commons Attribution 4.0 International License. |
| 2 | .. http://creativecommons.org/licenses/by/4.0 |
| 3 | |
| 4 | Design |
| 5 | ====== |
| 6 | |
| 7 | |
| 8 | Compatibility aspects (VES-JSON) |
| 9 | -------------------------------- |
| 10 | |
| 11 | HV-VES Collector has been designed as a high-volume variant of the existing VES(JSON) collector, and not a completely new collector. |
| 12 | HV-VES follows the VES-JSON schema - as much as possible |
| 13 | |
| 14 | - It uses a Google Protocol Buffers ( GPB/PROTO ) representation of the VES Common Header |
| 15 | - The PROTO files tend to use most encoding effective types defined by GPB to cover Common Header fields. |
| 16 | - It makes routing decisions based mostly on the content of the "Domain" parameter |
| 17 | - It allows to embed Payload of different types (by default PERF3GPP domain is included) |
| 18 | |
| 19 | Analytics applications impacts |
| 20 | |
| 21 | - An analytics application operating on high-volume data needs to be prepared to read directly from Kafka |
| 22 | - An analytics application need to operate on GPB encoded data in order to benefit from GPB encoding efficiencies |
| 23 | - It is assumed, that due to the nature of high volume data, there would have to be dedicated applications provided, |
| 24 | able to operate on such volumes of data. |
| 25 | |
| 26 | Extendability |
| 27 | ------------- |
| 28 | |
| 29 | HV-VES was designed to allow for extendability - by adding new domain-specific PROTO files. |
| 30 | |
| 31 | The PROTO file, which contains the VES CommonHeader, comes with a binary-type Payload parameter, where domain-specific data shall be placed. |
| 32 | Domain-specific data are encoded as well with GPB, and they do require a domain-specific PROTO file to decode the data. |
| 33 | This domain-specific PROTO needs to be shared with analytics applications - HV-VES is not analyzing domain-specific data. |
| 34 | |
| 35 | In order to support the RT-PM use-case, HV-VES includes a "PERF3GPP" domain PROTO file, as within this domain, |
| 36 | the high volume data is expected to be reported to HV-VES collector. |
| 37 | Still, there are no limitations to define additional domains, based on existing VES domains (like Fault, Heartbeat) |
| 38 | or completely new domains. New domains can be added "when needed". |
| 39 | |
| 40 | GPB PROTO files are backwards compatible, and such a new domain could be added without affecting existing systems. |
| 41 | |
| 42 | Analytics applications will have to be as well equipped with this new domain-specific PROTO file. |
| 43 | Currently, these additional, domain specific proto files could be simply added to respective repos of HV-VES collector. |
| 44 | |
| 45 | Implementation details |
| 46 | ---------------------- |
| 47 | |
| 48 | - Project Reactor is used as a backbone of the internal architecture. |
| 49 | - Netty is used by means of reactor-netty library. |
| 50 | - We are using Kotlin so we can write very concise code with great interoperability with existing Java libraries. |
| 51 | - Types defined in Λrrow library are also used when it improves readability or general cleanness of the code. |