Documentation for 3GPP PM Mapper
Change-Id: Ifbc4928630a329eed019c1c79f129234dfa1fdd3
Issue-ID: DCAEGEN2-567
Signed-off-by: Rajendra Jaiswal <rajendra.jaiswal@ericsson.com>
diff --git a/docs/sections/services/pm-mapper/architecture.rst b/docs/sections/services/pm-mapper/architecture.rst
new file mode 100644
index 0000000..78011f7
--- /dev/null
+++ b/docs/sections/services/pm-mapper/architecture.rst
@@ -0,0 +1,29 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+Architecture
+============
+
+Introduction
+""""""""""""
+3GPP PM Mapper is a part of DCAEGEN2. Some information about PM Mapper can be found here: `5G bulk PM wiki page`_.
+
+.. _5G bulk PM wiki page: https://wiki.onap.org/display/DW/5G+-+Bulk+PM
+
+3GPP PM Mapper will process 3GPP PM XML files to produce perf3gpp VES PM Events.
+
+.. image:: ../../images/pm-mapper.png
+
+
+.. _Delivery: ./delivery.html
+
+Functionality
+"""""""""""""
+The 3GPP PM Mapper micro-service will extract selected measurements from a 3GPP XML file and publish them as VES events on a DMaaP Message Router topic for consumers that prefer such data in VES format.
+The mapper receives the files by subscribing to a Data Router feed.
+
+.. image:: ../../images/pmmapper-flow.png
+
+Interaction
+"""""""""""
+PM Mapper will interact with the DMaaP Bus Controller for Subscribing to Data Router feeds and publishing to Message Router. PM Mapper also intracts with the Config Binding Service to get configuration information.
diff --git a/docs/sections/services/pm-mapper/configuration.rst b/docs/sections/services/pm-mapper/configuration.rst
new file mode 100644
index 0000000..b014f6f
--- /dev/null
+++ b/docs/sections/services/pm-mapper/configuration.rst
@@ -0,0 +1,7 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+Configuration
+=============
+
+< To-Do >
\ No newline at end of file
diff --git a/docs/sections/services/pm-mapper/delivery.rst b/docs/sections/services/pm-mapper/delivery.rst
new file mode 100644
index 0000000..0a03237
--- /dev/null
+++ b/docs/sections/services/pm-mapper/delivery.rst
@@ -0,0 +1,12 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+Delivery
+========
+
+Docker Container
+----------------
+
+**PM Mapper** is delivered as a docker image that can be downloaded from ONAP docker registry:
+
+ ``docker run -d --name pmmapper -e CONFIG_BINDING_SERVICE_SERVICE_HOST=<IP Required> -e CONFIG_BINDING_SERVICE_SERVICE_PORT=<Port Required> -e HOSTNAME=<HOSTNAME> nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pm-mapper:1.0-SNAPSHOT``
diff --git a/docs/sections/services/pm-mapper/index.rst b/docs/sections/services/pm-mapper/index.rst
new file mode 100644
index 0000000..8a53753
--- /dev/null
+++ b/docs/sections/services/pm-mapper/index.rst
@@ -0,0 +1,18 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+3GPP PM Mapper Service
+=============================
+
+.. Add or remove sections below as appropriate for the platform component.
+
+.. toctree::
+ :maxdepth: 1
+
+ ./architecture.rst
+ ./delivery.rst
+ ./logging.rst
+ ./installation.rst
+ ./configuration.rst
+ ./troubleshooting.rst
diff --git a/docs/sections/services/pm-mapper/installation.rst b/docs/sections/services/pm-mapper/installation.rst
new file mode 100644
index 0000000..5672e4d
--- /dev/null
+++ b/docs/sections/services/pm-mapper/installation.rst
@@ -0,0 +1,7 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+Installation
+============
+
+< To - Do >
diff --git a/docs/sections/services/pm-mapper/logging.rst b/docs/sections/services/pm-mapper/logging.rst
new file mode 100644
index 0000000..cc56f41
--- /dev/null
+++ b/docs/sections/services/pm-mapper/logging.rst
@@ -0,0 +1,7 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+Logging
+=======
+
+The log file is located under /var/log/ONAP/dcaegen2/services/pm-mapper/pm-mapper_output.log.
diff --git a/docs/sections/services/pm-mapper/troubleshooting.rst b/docs/sections/services/pm-mapper/troubleshooting.rst
new file mode 100644
index 0000000..ed0b398
--- /dev/null
+++ b/docs/sections/services/pm-mapper/troubleshooting.rst
@@ -0,0 +1,159 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _troubleshooting:
+
+Troubleshooting
+===============
+
+**NOTE**
+
+According to **ONAP** logging policy, **PM Mapper** logs contain all required markers as well as service and client specific Mapped Diagnostic Context (later referred as MDC)
+
+Default console log pattern:
+
+::
+
+ | %date{"yyyy-MM-dd'T'HH:mm:ss.SSSXXX", UTC}\t| %thread\t| %highlight(%-5level)\t| %msg\t| %marker\t| %rootException\t| %mdc\t| %thread
+
+A sample, fully qualified message implementing this pattern:
+
+::
+
+ | 2018-12-18T13:12:44.369Z | p.dcae | DEBUG | Client connection request received | ENTRY | | RequestID=d7762b18-854c-4b8c-84aa-95762c6f8e62, InstanceID=9b9799ca-33a5-4f61-ba33-5c7bf7e72d07, InvocationID=b13d34ba-e1cd-4816-acda-706415308107, PartnerName=C=PL, ST=DL, L=Wroclaw, O=Nokia, OU=MANO, CN=dcaegen2-hvves-client, StatusCode=INPROGRESS, ClientIPAddress=192.168.0.9, ServerFQDN=a4ca8f96c7e5 | reactor-tcp-nio-2
+
+
+For simplicity, all log messages in this section are shortened to contain only:
+ * logger name
+ * log level
+ * message
+
+Error and warning logs contain also:
+ * exception message
+ * stack trace
+
+**Do not rely on exact log messages or their presence, as they are often subject to change.**
+
+Configuration errors
+--------------------
+
+**Config binding service not responding**
+
+::
+
+
+ 2019-02-19T17:25:17.499Z main INFO org.onap.dcaegen2.services.pmmapper.config.ConfigHandler Fetching pm-mapper configuration from Configbinding Service ENTRY
+ 2019-02-19T17:25:17.502Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender ee5ff670-accd-4c30-8689-0a1d12491b51 INVOKE [ SYNCHRONOUS ]
+ 2019-02-19T17:25:17.509Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Sending:\n{X-ONAP-PartnerName=[pm-mapper], X-ONAP-InvocationID=[ee5ff670-accd-4c30-8689-0a1d12491b51], X-ONAP-RequestID=[2778e346-590a-4ade-8f45-358d1adf048b]}
+ 2019-02-19T17:25:18.515Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Sending:\n{X-ONAP-PartnerName=[pm-mapper], X-ONAP-InvocationID=[ee5ff670-accd-4c30-8689-0a1d12491b51], X-ONAP-RequestID=[2778e346-590a-4ade-8f45-358d1adf048b]}
+ 2019-02-19T17:25:19.516Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Sending:\n{X-ONAP-PartnerName=[pm-mapper], X-ONAP-InvocationID=[ee5ff670-accd-4c30-8689-0a1d12491b51], X-ONAP-RequestID=[2778e346-590a-4ade-8f45-358d1adf048b]}
+ 2019-02-19T17:25:20.518Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Sending:\n{X-ONAP-PartnerName=[pm-mapper], X-ONAP-InvocationID=[ee5ff670-accd-4c30-8689-0a1d12491b51], X-ONAP-RequestID=[2778e346-590a-4ade-8f45-358d1adf048b]}
+ 2019-02-19T17:25:21.519Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Sending:\n{X-ONAP-PartnerName=[pm-mapper], X-ONAP-InvocationID=[ee5ff670-accd-4c30-8689-0a1d12491b51], X-ONAP-RequestID=[2778e346-590a-4ade-8f45-358d1adf048b]}
+ 2019-02-19T17:25:21.520Z main INFO org.onap.dcaegen2.services.pmmapper.config.ConfigHandler Received pm-mapper configuration from ConfigBinding Service:\n EXIT
+ Exception in thread "main" org.onap.dcaegen2.services.pmmapper.exceptions.CBSServerError: Error connecting to Configbinding Service:
+ at org.onap.dcaegen2.services.pmmapper.config.ConfigHandler.getMapperConfig(ConfigHandler.java:78)
+ at org.onap.dcaegen2.services.pmmapper.App.main(App.java:58)
+ caused by: java.net.ConnectException: Connection refused (Connection refused)
+ at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
+ at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
+ at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
+ at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
+ at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1944)
+ at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1939)
+ at java.security.AccessController.doPrivileged(Native Method)
+ at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1938)
+ at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1508)
+ at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
+ at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
+ at java.net.HttpURLConnection.getResponseMessage(HttpURLConnection.java:546)
+ at org.onap.dcaegen2.services.pmmapper.utils.RequestSender.send(RequestSender.java:80)
+ at org.onap.dcaegen2.services.pmmapper.config.ConfigHandler.getMapperConfig(ConfigHandler.java:76)
+ ... 1 more
+ Caused by: java.net.ConnectException: Connection refused (Connection refused)
+ at java.net.PlainSocketImpl.socketConnect(Native Method)
+ at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
+ at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
+ at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
+ at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
+ at java.net.Socket.connect(Socket.java:589)
+ t java.net.Socket.connect(Socket.java:538)
+ at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
+ at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
+ at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
+ at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
+ at sun.net.www.http.HttpClient.New(HttpClient.java:339)
+ at sun.net.www.http.HttpClient.New(HttpClient.java:357)
+ at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
+ at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
+ at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
+ at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
+ at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
+ at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
+ at org.onap.dcaegen2.services.pmmapper.utils.RequestSender.send(RequestSender.java:66)
+
+
+Make sure Config Binding Service is up and running and the **ip + port** combination is correct.
+
+====
+
+**Missing configuration on Consul**
+
+::
+
+
+ 2019-02-19T17:36:32.664Z main INFO org.onap.dcaegen2.services.pmmapper.config.ConfigHandler Fetching pm-mapper configuration from Configbinding Service ENTRY
+ 2019-02-19T17:36:32.666Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender 9fa1b84f-05ce-4e27-bba9-4ea477c1baa7 INVOKE [ SYNCHRONOUS ]
+ 2019-02-19T17:36:32.671Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Sending:\n{X-ONAP-PartnerName=[pm-mapper], X-ONAP-InvocationID=[9fa1b84f-05ce-4e27-bba9-4ea477c1baa7], X-ONAP-RequestID=[6e861d17-3f4b-4a2e-9ea8-a31bb9dbb7e8]}
+ 2019-02-19T17:36:32.696Z main INFO org.onap.dcaegen2.services.pmmapper.utils.RequestSender Received:\n{"pm-mapper-filter": "{ \"filters\":[]}", "3GPP.schema.file": "{\"3GPP_Schema\":\"./etc/3GPP_relaxed_schema.xsd\"}", "streams_subscribes": {"dmaap_subscriber": {"type": "data_router", "aaf_username": null, "aaf_password": null, "dmaap_infooooo": {"location": "csit-pmmapper", "delivery_url": "3gpppmmapper", "username": "username", "password": "password", "subscriber_id": "subsriber_id"}}}, "streams_publishes": {"pm_mapper_handle_out": {"type": "message_router", "aaf_password": null, "dmaap_info": {"topic_url": "https://message-router:3904/events/org.onap.dmaap.onapCSIT.pm_mapper", "client_role": "org.onap.dmaap.client.pub", "location": "csit-pmmapper", "client_id": null}, "aaf_username": null}}, "buscontroller_feed_subscription_endpoint": "http://dmaap-bc:8080/webapi/dr_subs", "services_calls": {}}
+ 2019-02-19T17:36:32.696Z main INFO org.onap.dcaegen2.services.pmmapper.config.ConfigHandler Received pm-mapper configuration from ConfigBinding Service:\n{"pm-mapper-filter": "{ \"filters\":[]}", "3GPP.schema.file": "{\"3GPP_Schema\":\"./etc/3GPP_relaxed_schema.xsd\"}", "streams_subscribes": {"dmaap_subscriber": {"type": "data_router", "aaf_username": null, "aaf_password": null, "dmaap_infooooo": {"location": "csit-pmmapper", "delivery_url": "3gpppmmapper", "username": "username", "password": "password", "subscriber_id": "subsriber_id"}}}, "streams_publishes": {"pm_mapper_handle_out": {"type": "message_router", "aaf_password": null, "dmaap_info": {"topic_url": "https://message-router:3904/events/org.onap.dmaap.onapCSIT.pm_mapper", "client_role": "org.onap.dmaap.client.pub", "location": "csit-pmmapper", "client_id": null}, "aaf_username": null}}, "buscontroller_feed_subscription_endpoint": "http://dmaap-bc:8080/webapi/dr_subs", "services_calls": {}} EXIT
+ Exception in thread "main" org.onap.dcaegen2.services.pmmapper.exceptions.MapperConfigException: Error parsing mapper configuration:
+ {}{"pm-mapper-filter": "{ \"filters\":[]}", "3GPP.schema.file": "{\"3GPP_Schema\":\"./etc/3GPP_relaxed_schema.xsd\"}", "streams_subscribes": {"dmaap_subscriber": {"type": "data_router", "aaf_username": null, "aaf_password": null, "dmaap_infooooo": {"location": "csit-pmmapper", "delivery_url": "3gpppmmapper", "username": "username", "password": "password", "subscriber_id": "subsriber_id"}}}, "streams_publishes": {"pm_mapper_handle_out": {"type": "message_router", "aaf_password": null, "dmaap_info": {"topic_url": "https://message-router:3904/events/org.onap.dmaap.onapCSIT.pm_mapper", "client_role": "org.onap.dmaap.client.pub", "location": "csit-pmmapper", "client_id": null}, "aaf_username": null}}, "buscontroller_feed_subscription_endpoint": "http://dmaap-bc:8080/webapi/dr_subs", "services_calls": {}}
+ at org.onap.dcaegen2.services.pmmapper.config.ConfigHandler.convertMapperConfigToObject(ConfigHandler.java:94)
+ at org.onap.dcaegen2.services.pmmapper.config.ConfigHandler.getMapperConfig(ConfigHandler.java:83)
+ at org.onap.dcaegen2.services.pmmapper.App.main(App.java:58)
+ Caused by: com.google.gson.JsonParseException: Failed to check fields.
+ at org.onap.dcaegen2.services.pmmapper.utils.RequiredFieldDeserializer.deserialize(RequiredFieldDeserializer.java:49)
+ at com.google.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:69)
+ at com.google.gson.Gson.fromJson(Gson.java:927)
+ at com.google.gson.Gson.fromJson(Gson.java:892)
+ at com.google.gson.Gson.fromJson(Gson.java:841)
+ at com.google.gson.Gson.fromJson(Gson.java:813)
+ at org.onap.dcaegen2.services.pmmapper.config.ConfigHandler.convertMapperConfigToObject(ConfigHandler.java:92)
+ ... 2 more
+ Caused by: com.google.gson.JsonParseException: Field: 'busControllerFeedId', is required but not found.
+ at org.onap.dcaegen2.services.pmmapper.utils.RequiredFieldDeserializer.deserialize(RequiredFieldDeserializer.java:46)
+
+
+
+**PM Mapper** logs this information when connected to Consul, but cannot find a valid JSON configuration.
+
+====
+
+**Unable to Subscriber to Data Router Feed**
+
+::
+
+ 2019-02-11T16:44:37.012Z main DEBUG org.onap.dcaegen2.services.pmmapper.config.ConfigHandler Mapper configuration:\nMapperConfig(messageRouterTopicName=unauthenticated.PM_VES_OUTPUT, busControllerConfig=BusControllerConfig(dcaeLocation=dcaeLocation, deliveryURL=deliveryURL, feedId=2, lastMod=lastMod, username=username, password=password, dataRouterSubscribeEndpoint=null), streamsSubscribes=MapperConfig.StreamsSubscribes(dmaapSubscriber=MapperConfig.DmaapSubscriber(dmaapInfo=MapperConfig.DmaapInfo(location=location, username=username, password=password, deliveryUrl=delivery_url, subscriberId=subsriber_id))))
+ 2019-02-11T16:44:37.013Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber Starting subscription to DataRouter ENTRY
+ 2019-02-11T16:44:37.013Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber e04d9fb4-5bc3-405b-89ea-6b824de8f137 INVOKE [ SYNCHRONOUS ]
+ 2019-02-11T16:44:37.061Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber Request to bus controller executed with Response Code: '404' and Response Event: 'Not Found'.
+ 2019-02-11T16:44:39.062Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber d4982d98-574d-42e4-9922-640610ffa34d INVOKE [ SYNCHRONOUS ]
+ 2019-02-11T16:44:39.071Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber Request to bus controller executed with Response Code: '404' and Response Event: 'Not Found'.
+ 2019-02-11T16:44:43.079Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber 05c73228-d04c-47c2-b897-8e936de5c8fd INVOKE [ SYNCHRONOUS ]
+ 2019-02-11T16:44:43.089Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber Request to bus controller executed with Response Code: '404' and Response Event: 'Not Found'.
+ 2019-02-11T16:44:51.129Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber c898b5c7-a347-496a-af3b-aeac661d4f1b INVOKE [ SYNCHRONOUS ]
+ 2019-02-11T16:44:51.138Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber Request to bus controller executed with Response Code: '404' and Response Event: 'Not Found'.
+ 2019-02-11T16:45:07.220Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber b190fbc1-ab91-44da-b393-727810925374 INVOKE [ SYNCHRONOUS ]
+ 2019-02-11T16:45:07.234Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber Request to bus controller executed with Response Code: '404' and Response Event: 'Not Found'.
+ 2019-02-11T16:45:07.235Z main INFO org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber EXIT
+ Exception in thread "main" org.onap.dcaegen2.services.pmmapper.exceptions.TooManyTriesException: Failed to subscribe within appropriate amount of attempts
+ at org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber.subscribe(DataRouterSubscriber.java:152)
+ at org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber.subscribe(DataRouterSubscriber.java:150)
+ at org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber.subscribe(DataRouterSubscriber.java:150)
+ at org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber.subscribe(DataRouterSubscriber.java:150)
+ at org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber.subscribe(DataRouterSubscriber.java:150)
+ at org.onap.dcaegen2.services.pmmapper.datarouter.DataRouterSubscriber.start(DataRouterSubscriber.java:97)
+ at org.onap.dcaegen2.services.pmmapper.App.main(App.java:53)
+
+This log is printed when pm mapper couldn't subscribe to the data router.
+