Update HV-VES documentation
- include changes to collector configuration, especially moving from
command line parameters to configuration file
- include changes to run-time configuration i.e. moving from direct
Consul requests to CBS usage
- changes in design.rst cover extraction of protofiles from hv-ves
repository to sdk
- also swapped rstTable to rstCSVTable in deployment.rst as they
are easier to maintain
Change-Id: I5cdee7170d03edaf31615e40f6cb6ad14a6a2331
Issue-ID: DCAEGEN2-1352
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
diff --git a/docs/sections/services/ves-hv/architecture.rst b/docs/sections/services/ves-hv/architecture.rst
index 986e8bb..1812f4e 100644
--- a/docs/sections/services/ves-hv/architecture.rst
+++ b/docs/sections/services/ves-hv/architecture.rst
@@ -12,7 +12,7 @@
Input messages come from TCP interface and Wire Transfer Protocol. Each frame includes Google Protocol Buffers (GPB) encoded payload.
Based on information provided in CommonEventHeader, domain messages are validated and published to specific Kafka topic in DMaaP.
-.. image:: ONAP_VES_HV_Architecture.png
+.. image:: resources/ONAP_VES_HV_Architecture.png
Messages published in DMaaP's Kafka topic will be consumed by DCAE analytics application or other ONAP component that consumes messages from DMaaP/Kafka.
DMaaP serves direct access to Kafka allowing other analytics applications to utilize its data.
diff --git a/docs/sections/services/ves-hv/deployment.rst b/docs/sections/services/ves-hv/deployment.rst
index 07d26b9..3a14eb1 100644
--- a/docs/sections/services/ves-hv/deployment.rst
+++ b/docs/sections/services/ves-hv/deployment.rst
@@ -7,83 +7,106 @@
Deployment
============
-To run HV-VES Collector container, you need to specify required parameters by passing them as command
-line arguments either by using long form (--long-form) or short form (-s) followed by argument if needed.
+To run HV-VES Collector container you need to specify required command line options and environment variables.
-All parameters can also be configured by specifying environment variables. These variables have to be named after command line option name
-rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix, for example `VESHV_LISTEN_PORT`.
+Command line parameters can be specified either by using long form (--long-form) or short form (-s) followed by argument if needed (see `Arg` column in table below). These parameters can be omitted if corresponding environment variables are set. These variables are named after command line option name rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix, for example `VESHV_CONFIGURATION_FILE`.
-Command line options have precedence over environment variables.
+Command line options have precedence over environment variables in cases when both are present.
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
-| Long form | Short form | Env form | Required | Arg | Description |
-+=============+============+===================+==========+=====+=================================================+
-| listen-port | p | VESHV_LISTEN_PORT | yes | yes | Port on which HV-VES listens internally |
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
-| config-url | c | VESHV_CONFIG_URL | yes | yes | URL of HV-VES configuration on Consul service |
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
+Currently HV-VES requires single command line parameter which points to base configuration file.
-HV-VES requires also to specify if SSL should be used when handling incoming TCP connections.
-This can be done by passing the flag below to the command line.
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header: Long form , Short form , Arg , Env form , Description
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
-| Long form | Short form | Env form | Required | Arg | Description |
-+=============+============+===================+==========+=====+=================================================+
-| ssl-disable | l | VESHV_SSL_DISABLE | no | no | Disables SSL encryption |
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
+ configuration-file ; c ; yes ; VESHV_CONFIGURATION_FILE ; Path to JSON file containing HV-VES configuration
+
+Environment variables that are required by HV-VES are used by collector for provisioning of run-time configuration and are usually provided by DCAE platform.
+
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header: Environment variable name , Description
+
+ CONSUL_HOST ; Hostname under which Consul service is available
+ CONFIG_BINDING_SERVICE ; Hostname under which Config Binding Service is available
+ HOSTNAME ; Configuration key of HV-VES as seen by CBS, usually *dcae-hv-ves-collector*
+
+There is also optional command line parameter which configures container-internal port for Healthcheck Server API (see :ref:`healthcheck_and_monitoring`).
+
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header: Long form , Short form , Arg , Env form , Description
+
+ health-check-api-port ; H ; yes ; VESHV_HEALTH_CHECK_API_PORT ; Health check rest api listen port
+
+.. _configuration_file:
+
+Configuration file
+------------------
+
+File must provide base configuration for HV-VES Collector in JSON format.
+
+Some entries in configuration can also be obtained from Config Binding Service (see :ref:`run_time_configuration`). **Every entry defined in configuration file will be OVERRIDEN if it is also present in run-time configuration.**
+
+Following JSON shows every possible configuration option. Default file shipped with HV-VES container can be found in the collector's repository (see :ref:`repositories`).
+
+.. literalinclude:: resources/base-configuration.json
+ :language: json
-Minimal command for running the container:
+The configuration is split into smaller sections.
+Tables show restrictions on fields in file configuration and short description.
-.. code-block:: bash
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header-rows: 2
- docker run nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main --listen-port 6061 --config-url http://consul:8500/v1/kv/dcae-hv-ves-collector --ssl-disable
+ Server
+ Key ; Value type ; Description
+ server.listenPort ; number ; Port on which HV-VES listens internally
+ server.idleTimeoutSec ; number ; Idle timeout for remote hosts. After given time without any data exchange, the connection might be closed
-Optional configuration parameters:
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header-rows: 2
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| Long form | Short form | Env form | Required | Arg | Default | Description |
-+=======================+============+============================+==========+=====+=================+=======================================================+
-| health-check-api-port | H | VESHV_HEALTHCHECK_API_PORT | no | yes | 6060 | Health check REST API listen port |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| first-request-delay | d | VESHV_FIRST_REQUEST_DELAY | no | yes | 10 | Delay of first request to Consul service in seconds |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| request-interval | I | VESHV_REQUEST_INTERVAL | no | yes | 5 | Interval of Consul configuration requests in seconds |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| idle-timeout-sec | i | VESHV_IDLE_TIMEOUT_SEC | no | yes | 60 | Idle timeout for remote hosts. After given time |
-| | | | | | | without any data exchange, the connection |
-| | | | | | | might be closed. |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| max-payload-size | m | VESHV_MAX_PAYLOAD_SIZE | no | yes | 1048576 (1 MiB) | Maximum supported payload size in bytes |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| log-level | ll | VESHV_LOG_LEVEL | no | yes | INFO | Log level on which HV-VES publishes all log messages |
-| | | | | | | Valid argument values are (case insensitive): ERROR, |
-| | | | | | | WARN, INFO, DEBUG, TRACE. |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
+ Config Binding Service
+ Key ; Value type ; Description
+ cbs.firstRequestDelaySec ; number ; Delay of first request to Config Binding Service in seconds
+ cbs.requestIntervalSec ; number ; Interval of configuration requests in seconds
-As part of experimental API if you do not specify `ssl-disable` flag, there is need to specify additional
-parameters for security configuration.
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header-rows: 2
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| Long form | Short form | Env form | Required | Arg | Default | Description |
-+=======================+============+============================+==========+=====+========================+==============================================================+
-| key-store | k | VESHV_KEY_STORE | no | yes | /etc/ves-hv/server.p12 | Key store in PKCS12 format path |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| key-store-password | kp | VESHV_KEY_STORE_PASSWORD | no | yes | | Key store password |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| trust-store | t | VESHV_TRUST_STORE | no | yes | /etc/ves-hv/trust.p12 | File with trusted certificate bundle in PKCS12 format path |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| trust-store-password | tp | VESHV_TRUST_STORE_PASSWORD | no | yes | | Trust store password |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
+ Security
+ Key ; Value type ; Description
+ security.sslDisable ; boolean ; Disables SSL encryption
+ security.keys.keyStoreFile ; String ; Key store path used in HV-VES incoming connections
+ security.keys.keyStorePasswordFile ; String ; Key store password file used in HV-VES incoming connections
+ security.keys.trustStoreFile ; String ; Path to file with trusted certificates bundle used in HV-VES incoming connections
+ security.keys.trustStorePasswordFile ; String ; Trust store password file used in HV-VES incoming connections
-Passwords are mandatory without ssl-disable flag. If key-store or trust-store location is not specified, HV-VES will try to read them from default locations.
+All security entries are mandatory with `security.sslDisable` set to `false`. Otherwise only `security.sslDisable` needs to be specified. If `security.sslDisable` flag is missing, then it is interpreted same as it would be set to `false`.
-These parameters can be configured either by passing command line option during `docker run` call or
-by specifying environment variables named after command line option name
-rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix e.g. `VESHV_LISTEN_PORT`.
+.. csv-table::
+ :widths: auto
+ :delim: ;
+ :header-rows: 2
+
+ Uncategorized
+ Key ; Value type ; Description
+ logLevel ; String ; Log level on which HV-VES publishes all log messages. Valid argument values are (case insensitive): ERROR, WARN, INFO, DEBUG, TRACE.
+
Horizontal Scaling
-==================
+------------------
Kubernetes command line tool (`kubectl`) is recommended for manual horizontal scaling of HV-VES Collector.
diff --git a/docs/sections/services/ves-hv/design.rst b/docs/sections/services/ves-hv/design.rst
index a6c2b86..fb4fa2c 100644
--- a/docs/sections/services/ves-hv/design.rst
+++ b/docs/sections/services/ves-hv/design.rst
@@ -33,13 +33,13 @@
Domain-specific data are encoded as well with GPB. A domain-specific proto file is required to decode the data.
This domain-specific proto has to be shared with analytics applications - HV-VES does not analyze domain-specific data.
-In order to support the RT-PM use-case, HV-VES includes a **perf3gpp** domain proto file. Within this domain, high volume data are expected to be reported to HV-VES collector.
+In order to support the RT-PM use-case, HV-VES uses a **perf3gpp** domain proto file. Within this domain, high volume data are expected to be reported to HV-VES collector.
Additional domains can be defined based on existing VES domains (like Fault, Heartbeat) or completely new domains. New domains can be added when needed.
GPB proto files are backwards compatible, and a new domain can be added without affecting existing systems.
Analytics applications have to be equipped with the new domain-specific proto file as well.
-Currently, these additional, domain specific proto files can be added to respective repos of HV-VES collector.
+Currently, these additional, domain specific proto files can be added to hv-ves-client protobuf library repository (artifactId: hvvesclient-protobuf).
Implementation details
----------------------
diff --git a/docs/sections/services/ves-hv/example-event.rst b/docs/sections/services/ves-hv/example-event.rst
index 3a33539..a413d40 100644
--- a/docs/sections/services/ves-hv/example-event.rst
+++ b/docs/sections/services/ves-hv/example-event.rst
@@ -11,5 +11,5 @@
Values of fields can be changed according to types specified in noted definition files.
-.. literalinclude:: WTP.yaml
+.. literalinclude:: resources/WTP.yaml
:language: yaml
diff --git a/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst b/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst
index 1833377..9d35e1e 100644
--- a/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst
+++ b/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst
@@ -9,7 +9,7 @@
Healthcheck
-----------
Inside HV-VES docker container runs a small HTTP service for healthcheck. Port for healthchecks can be configured
-at deployment using ``--health-check-api-port`` command line option or via `VESHV_HEALTHCHECK_API_PORT` environment variable (for details see :ref:`deployment`).
+at deployment using command line (for details see :ref:`deployment`).
This service exposes endpoint **GET /health/ready** which returns a **HTTP 200 OK** when HV-VES is healthy
and ready for connections. Otherwise it returns a **HTTP 503 Service Unavailable** message with a short reason of unhealthiness.
@@ -90,4 +90,4 @@
Sample response for **GET monitoring/prometheus**:
-.. literalinclude:: metrics_sample_response.txt
+.. literalinclude:: resources/metrics_sample_response.txt
diff --git a/docs/sections/services/ves-hv/index.rst b/docs/sections/services/ves-hv/index.rst
index 5bb83dd..144f557 100644
--- a/docs/sections/services/ves-hv/index.rst
+++ b/docs/sections/services/ves-hv/index.rst
@@ -29,11 +29,11 @@
architecture
design
- run-time-configuration
repositories
deployment
- troubleshooting
+ run-time-configuration
HV-VES Offered APIs <../../apis/ves-hv/index>
authorization
example-event
healthcheck-and-monitoring
+ troubleshooting
diff --git a/docs/sections/services/ves-hv/ONAP_VES_HV_Architecture.png b/docs/sections/services/ves-hv/resources/ONAP_VES_HV_Architecture.png
similarity index 100%
rename from docs/sections/services/ves-hv/ONAP_VES_HV_Architecture.png
rename to docs/sections/services/ves-hv/resources/ONAP_VES_HV_Architecture.png
Binary files differ
diff --git a/docs/sections/services/ves-hv/WTP.yaml b/docs/sections/services/ves-hv/resources/WTP.yaml
similarity index 100%
rename from docs/sections/services/ves-hv/WTP.yaml
rename to docs/sections/services/ves-hv/resources/WTP.yaml
diff --git a/docs/sections/services/ves-hv/resources/base-configuration.json b/docs/sections/services/ves-hv/resources/base-configuration.json
new file mode 100644
index 0000000..6580287
--- /dev/null
+++ b/docs/sections/services/ves-hv/resources/base-configuration.json
@@ -0,0 +1,12 @@
+{
+ "logLevel": "INFO",
+ "server.listenPort": 6061,
+ "server.idleTimeoutSec": 60,
+ "cbs.firstRequestDelaySec": 10,
+ "cbs.requestIntervalSec": 5,
+ "security.sslDisable": false,
+ "security.keys.keyStoreFile": "/etc/ves-hv/ssl/server.p12",
+ "security.keys.keyStorePasswordFile": "/etc/ves-hv/ssl/server.pass",
+ "security.keys.trustStoreFile": "/etc/ves-hv/ssl/trust.p12",
+ "security.keys.trustStorePasswordFile": "/etc/ves-hv/ssl/trust.pass"
+}
\ No newline at end of file
diff --git a/docs/sections/services/ves-hv/metrics_sample_response.txt b/docs/sections/services/ves-hv/resources/metrics_sample_response.txt
similarity index 100%
rename from docs/sections/services/ves-hv/metrics_sample_response.txt
rename to docs/sections/services/ves-hv/resources/metrics_sample_response.txt
diff --git a/docs/sections/services/ves-hv/run-time-configuration.rst b/docs/sections/services/ves-hv/run-time-configuration.rst
index 76d622c..5efc6e3 100644
--- a/docs/sections/services/ves-hv/run-time-configuration.rst
+++ b/docs/sections/services/ves-hv/run-time-configuration.rst
@@ -8,7 +8,7 @@
(see :ref:`deployment`)
-HV-VES can fetch configuration directly from Consul service in the following JSON format:
+HV-VES fetches configuration from Config Binding Service in the following JSON format:
.. code-block:: json
@@ -50,10 +50,8 @@
fromDomain: perf3gpp
toTopic: HV_VES_PERF3GPP
-Endpoint on which HV-VES seeks configuration can be set during deployment as described in :ref:`deployment`.
+HV-VES waits 10 seconds (default, configurable during deployment with **firstRequestDelay** option, see :ref:`configuration_file`) before the first attempt to retrieve configuration from CBS. This is to prevent possible synchronization issues. During that time HV-VES declines any connection attempts from xNF (VNF/PNF).
-HV-VES waits 10 seconds (default, configurable during deplyoment with **first-request-delay** option, see :ref:`deployment`) before the first attempt to retrieve configuration from Consul. This is to prevent possible synchronization issues. During that time HV-VES declines any connection attempts from xNF (VNF/PNF).
+After first request, HV-VES asks for configuration in fixed intervals, configurable from file configuration (**requestInterval**). By default interval is set to 5 seconds.
-After first request, HV-VES asks for configuration in fixed intervals, configurable from command line (**request-interval**). By defualt interval is set to 5 seconds.
-
-In case of failing to retrieve configuration, collector temporarily extends this interval and retries. After five unsuccessfull attempts, container becomes unhealthy and cannot recover. HV-VES in this state is unusable and the container should be restarted.
+In case of failing to retrieve configuration, collector retries the action. After five unsuccessful attempts, container becomes unhealthy and cannot recover. HV-VES in this state is unusable and the container should be restarted.
