[HV-VES-KAFKA] Add doc updates for hv-ves kafka
Signed-off-by: efiacor <fiachra.corcoran@est.tech>
Change-Id: I3b377f39fd2b0b764f88990b6a8b97fa63ed158d
Issue-ID: DCAEGEN2-3083
diff --git a/docs/sections/services/ves-hv/architecture.rst b/docs/sections/services/ves-hv/architecture.rst
index 8720767..e71bf6a 100644
--- a/docs/sections/services/ves-hv/architecture.rst
+++ b/docs/sections/services/ves-hv/architecture.rst
@@ -6,13 +6,13 @@
High-level architecture of HV-VES
=================================
-HV-VES Collector is a part of DCAEGEN2. Its goal is to collect data from xNF (PNF/VNF) and publish it in DMaaP's Kafka.
+HV-VES Collector is a part of DCAEGEN2. Its goal is to collect data from xNF (PNF/VNF) and put the data on a given Kafka topic.
High Volume Collector is deployed with DCAEGEN2 via OOM Helm charts and Cloudify blueprints.
Input messages come from TCP interface and Wire Transfer Protocol. Each frame includes Google Protocol Buffers (GPB) encoded payload.
-Based on information provided in CommonEventHeader, domain messages are validated and published to specific Kafka topic in DMaaP.
+Based on information provided in CommonEventHeader, domain messages are validated and published to specific Kafka topic.
.. image:: resources/ONAP_VES_HV_Architecture.png
-Messages published in DMaaP's Kafka topic will be consumed by DCAE analytics application or other ONAP component that consumes messages from DMaaP/Kafka.
+Messages published on the given Kafka topic will be consumed by DCAE analytics application or other ONAP component that require these message.
DMaaP serves direct access to Kafka allowing other analytics applications to utilize its data.
diff --git a/docs/sections/services/ves-hv/deployment.rst b/docs/sections/services/ves-hv/deployment.rst
index e764a9a..72daa26 100644
--- a/docs/sections/services/ves-hv/deployment.rst
+++ b/docs/sections/services/ves-hv/deployment.rst
@@ -8,7 +8,11 @@
To run HV-VES Collector container you need to specify required command line options and environment variables.
-Command line parameters can be specified either by using long form (--long-form) or short form (-s) followed by argument if needed (see `Arg` column in table below). These parameters can be omitted if corresponding environment variables are set. These variables are named after command line option name rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix, for example `VESHV_CONFIGURATION_FILE`.
+Command line parameters can be specified either by using long form (--long-form) or short form (-s)
+followed by an argument if needed (see `Arg` column in table below).
+These parameters can be omitted if the corresponding environment variables are set.
+These variables are named after the command line option name rewritten using `UPPER_SNAKE_CASE` and prepended
+with `VESHV_` prefix, for example `VESHV_CONFIGURATION_FILE`.
Command line options have precedence over environment variables in cases when both are present.
@@ -21,7 +25,8 @@
configuration-file ; c ; yes ; VESHV_CONFIGURATION_FILE ; Path to JSON file containing HV-VES configuration
-Environment variables that are required by HV-VES are used by collector for provisioning of run-time configuration and are provided by DCAE platform.
+Environment variables that are required by HV-VES are used by collector for provisioning of run-time configuration
+and are provided by DCAE platform.
.. csv-table::
:widths: auto
@@ -32,7 +37,8 @@
CONFIG_BINDING_SERVICE ; Hostname under which Config Binding Service is available
HOSTNAME ; Configuration key of HV-VES as seen by CBS, usually *dcae-hv-ves-collector*
-There is also optional command line parameter which configures container-internal port for Healthcheck Server API (see :ref:`healthcheck_and_monitoring`).
+There is also optional command line parameter which configures container-internal port
+for Healthcheck Server API (see :ref:`healthcheck_and_monitoring`).
.. csv-table::
:widths: auto
@@ -48,9 +54,11 @@
File must provide base configuration for HV-VES Collector in JSON format.
-Some entries in configuration can also be obtained from Config Binding Service (see :ref:`run_time_configuration`). **Every entry defined in configuration file will be OVERRIDEN if it is also present in run-time configuration.**
+Some entries in configuration can also be obtained from Config Binding Service (see :ref:`run_time_configuration`).
+**Every entry defined in configuration file will be OVERRIDEN if it is also present in run-time configuration.**
-Following JSON shows every possible configuration option. Default file shipped with HV-VES container can be found in the collector's repository (see :ref:`repositories`).
+Following JSON shows every possible configuration option. Default file shipped with HV-VES container
+can be found in the collector's repository (see :ref:`repositories`).
.. literalinclude:: resources/base-configuration.json
:language: json
@@ -92,7 +100,8 @@
security.keys.trustStoreFile ; String ; Path to file with trusted certificates bundle used in HV-VES incoming connections
security.keys.trustStorePasswordFile ; String ; Trust store password file used in HV-VES incoming connections
-All security entries are mandatory with `security.sslDisable` set to `false`. Otherwise only `security.sslDisable` needs to be specified. If `security.sslDisable` flag is missing, then it is interpreted as it was set to `false`.
+All security entries are mandatory with `security.sslDisable` set to `false`.
+Otherwise only `security.sslDisable` needs to be specified. If `security.sslDisable` flag is missing, then it is interpreted as it was set to `false`.
.. csv-table::
:widths: auto
@@ -110,7 +119,8 @@
Kubernetes command line tool (`kubectl`) is recommended for manual horizontal scaling of HV-VES Collector.
To scale HV-VES deployment you need to determine its name and namespace in which it is deployed.
-For default OOM deployment, HV-VES full deployment name is `deployment/dep-dcae-hv-ves-collector` and it is installed under `onap` namespace.
+For default OOM deployment, HV-VES full deployment name is `deployment/dep-dcae-hv-ves-collector`
+and it is installed under `onap` namespace.
1. If the namespace is unknown, execute the following command to determine possible namespaces.
diff --git a/docs/sections/services/ves-hv/index.rst b/docs/sections/services/ves-hv/index.rst
index f6209cf..ccd8730 100644
--- a/docs/sections/services/ves-hv/index.rst
+++ b/docs/sections/services/ves-hv/index.rst
@@ -17,7 +17,7 @@
The goal of the collector is to support high volume data. It uses plain TCP connections.
Connections are stream-based (as opposed to request-based) and long running.
Payload is binary-encoded (currently using Google Protocol Buffers).
-HV-VES uses direct connection to DMaaP's Kafka.
+HV-VES uses direct connection to Kafka.
All these decisions were made in order to support high-volume data with minimal latency.
diff --git a/docs/sections/services/ves-hv/installation.rst b/docs/sections/services/ves-hv/installation.rst
index ada7b7f..4b7eb23 100644
--- a/docs/sections/services/ves-hv/installation.rst
+++ b/docs/sections/services/ves-hv/installation.rst
@@ -13,7 +13,6 @@
---------------------------------
HV-VES application is configured by default to use TLS/SSL encryption on TCP connection.
-However it is posible to turn off TLS/SSL authorization by overriding Cloudify blueprint inputs.
Accessing bootstrap container with Kubernetes command line tool
---------------------------------------------------------------
diff --git a/docs/sections/services/ves-hv/resources/dynamic-configuration.json b/docs/sections/services/ves-hv/resources/dynamic-configuration.json
index 40521ab..2bdeb04 100644
--- a/docs/sections/services/ves-hv/resources/dynamic-configuration.json
+++ b/docs/sections/services/ves-hv/resources/dynamic-configuration.json
@@ -1,42 +1,48 @@
{
- "logLevel": "INFO",
- "server.listenPort": 6061,
- "server.idleTimeoutSec": 60,
- "cbs.requestIntervalSec": 5,
- "security.sslDisable": false,
- "security.keys.keyStoreFile": "/etc/ves-hv/ssl/cert.jks",
- "security.keys.keyStorePasswordFile": "/etc/ves-hv/ssl/jks.pass",
- "security.keys.trustStoreFile": "/etc/ves-hv/ssl/trust.jks",
- "security.keys.trustStorePasswordFile": "/etc/ves-hv/ssl/trust.pass",
- "streams_publishes": {
- "perf3gpp": {
- "type": "kafka",
- "kafka_info": {
- "bootstrap_servers": "message-router-kafka:9092",
- "topic_name": "HV_VES_PERF3GPP"
+ "logLevel":"INFO",
+ "server.listenPort":6061,
+ "server.idleTimeoutSec":60,
+ "cbs.requestIntervalSec":5,
+ "security.sslDisable":false,
+ "security.keys.keyStoreFile":"/etc/ves-hv/ssl/cert.jks",
+ "security.keys.keyStorePasswordFile":"/etc/ves-hv/ssl/jks.pass",
+ "security.keys.trustStoreFile":"/etc/ves-hv/ssl/trust.jks",
+ "security.keys.trustStorePasswordFile":"/etc/ves-hv/ssl/trust.pass",
+ "streams_publishes":{
+ "ves-3gpp-fault-supervision":{
+ "type":"kafka",
+ "kafka_info":{
+ "bootstrap_servers":"kafka:9092",
+ "topic_name":"SEC_3GPP_FAULTSUPERVISION_OUTPUT"
+ }
+ },
+ "ves-3gpp-provisioning":{
+ "type":"kafka",
+ "kafka_info":{
+ "bootstrap_servers":"kafka:9092",
+ "topic_name":"SEC_3GPP_PROVISIONING_OUTPUT"
+ }
+ },
+ "ves-3gpp-heartbeat":{
+ "type":"kafka",
+ "kafka_info":{
+ "bootstrap_servers":"kafka:9092",
+ "topic_name":"SEC_3GPP_HEARTBEAT_OUTPUT"
+ }
+ },
+ "ves-3gpp-performance-assurance":{
+ "type":"kafka",
+ "kafka_info":{
+ "bootstrap_servers":"kafka:9092",
+ "topic_name":"SEC_3GPP_PERFORMANCEASSURANCE_OUTPUT"
+ }
+ },
+ "perf3gpp":{
+ "type":"kafka",
+ "kafka_info":{
+ "bootstrap_servers":"kafka:9092",
+ "topic_name":"HV_VES_PERF3GPP"
+ }
}
- },
- "heartbeat": {
- "type": "kafka",
- "kafka_info": {
- "bootstrap_servers": "message-router-kafka:9092",
- "topic_name": "HV_VES_HEARTBEAT"
- }
- },
- "ves-3gpp-fault-supervision": {
- "type": "kafka",
- "kafka_info": {
- "bootstrap_servers": "message-router-kafka:9092",
- "topic_name": "SEC_3GPP_FAULTSUPERVISION_OUTPUT"
- }
- },
- "ves-3gpp-heartbeat": {
- "type": "kafka",
- "kafka_info": {
- "bootstrap_servers": "message-router-kafka:9092",
- "topic_name": "SEC_3GPP_HEARTBEAT_OUTPUT"
- }
- }
- }
+ }
}
-
diff --git a/docs/sections/services/ves-hv/run-time-configuration.rst b/docs/sections/services/ves-hv/run-time-configuration.rst
index c2352c7..830ed66 100644
--- a/docs/sections/services/ves-hv/run-time-configuration.rst
+++ b/docs/sections/services/ves-hv/run-time-configuration.rst
@@ -7,7 +7,13 @@
======================
HV-VES dynamic configuration is primarily meant to provide DMaaP Connection Objects (see :ref:`dmaap-connection-objects`).
-These objects contain information necessary to route received VES Events to correct Kafka topic. This metadata will be later referred to as Routing definition.
+
+.. note:: Kafka config info.
+ In the case of HV-VES, this configuration method is purely used as a generic reference.
+ The underlying kafka connection config is managed as part of the dcea helm configuration.
+
+These objects contain information necessary to route received VES Events to correct Kafka topic.
+This metadata will be later referred to as Routing definition.
Collector uses DCAE-SDK internally, to fetch configuration from Config Binding Service.
@@ -45,4 +51,4 @@
The configuration is created from HV-VES Cloudify blueprint by specifying **application_config** node during ONAP OOM/Kubernetes deployment. Example of the node specification:
.. literalinclude:: resources/blueprint-snippet.yaml
- :language: yaml
\ No newline at end of file
+ :language: yaml
diff --git a/docs/sections/services/ves-hv/troubleshooting.rst b/docs/sections/services/ves-hv/troubleshooting.rst
index ebb80d9..3651035 100644
--- a/docs/sections/services/ves-hv/troubleshooting.rst
+++ b/docs/sections/services/ves-hv/troubleshooting.rst
@@ -162,7 +162,8 @@
| org.onap.dcae.collectors.veshv.impl.socket.NettyTcpServer | INFO | Connection has been closed
-The above log is printed when the message payload size is too big. **HV-VES** does not handle messages that exceed maximum payload size specified under streams_publishes configuration (see :ref:`dmaap-connection-objects`)
+The above log is printed when the message payload size is too big.
+**HV-VES** does not handle messages that exceed maximum payload size specified under streams_publishes configuration (see :ref:`dmaap-connection-objects`)
====