datafilecollector/docs/overview.rst

.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. SPDX-License-Identifier: CC-BY-4.0
.. Copyright (C) 2023 Nordix


Data File Collector
~~~~~~~~~~~~~~~~~~~

************
Introduction
************

The task of the Data File Collector is to collect files from traffical nodes in the RAN.
The main use case is:

* The DFC received a File Ready VES event from a Kafka topic. This contains a list of all available files.

* The DFC fetches files that are not already fectched from the traffical node. This is done using one of the supported file transfer protocols.

* Each file is stored in an S3 Object Store bucket or in the file system (in a persistent volume).

* For each stored file, a file publish message is sent to a Kafka topic for further processing.

Supported file transfer protocols are:

* SFTP

* FTPES

* HTTP

* HTTPS


The service is implemented in Java Spring Boot.

.. image:: ./Architecture.png
   :width: 1000pt

This product is a part of :doc:`NONRTRIC <nonrtric:index>`.

**************************
Input File Ready VES Event
**************************

Here follows an example of the expected input object:

.. code-block:: javascript

  {
   "event":{
      "commonEventHeader":{
         "sequence":0,
         "eventName":"Noti_RnNode-Ericsson_FileReady",
         "sourceName":"5GRAN_DU",
         "lastEpochMicrosec":151983,
         "startEpochMicrosec":15198378,
         "timeZoneOffset":"UTC+05:00",
         "changeIdentifier":"PM_MEAS_FILES"
      },
      "notificationFields":{
         "notificationFieldsVersion":"notificationFieldsVersion",
         "changeType":"FileReady",
         "changeIdentifier":"PM_MEAS_FILES",
         "arrayOfNamedHashMap":[
            {
               "name":"A20220418.1900-1915_seliitdus00487.xml",
               "hashMap":{
                  "fileFormatType":"org.3GPP.32.435#measCollec",
                  "location":"https://launchpad.net/ubuntu/+source/perf-tools-unstable/1.0+git7ffb3fd-1ubuntu1/+build/13630748/+files/perf-tools-unstable_1.0+git7ffb3fd-1ubuntu1_all.deb",
                  "fileFormatVersion":"V10",
                  "compression":"gzip"
               }
            }
         ]
      }
   }
 }


***************************
Output File Publish Message
***************************


Below follows an example of an output File Publish Message. .

.. code-block:: javascript

   {
     "productName":"RnNode",
     "vendorName":"Ericsson",
     "lastEpochMicrosec":151983,
     "sourceName":"5GRAN_DU",
     "startEpochMicrosec":15198378,
     "timeZoneOffset":"UTC+05:00",
     "compression":"gzip",
     "fileFormatType":"org.3GPP.32.435#measCollec",
     "fileFormatVersion":"V10",
     "name":"5GRAN_DU/A20220418.1900-1915_seliitdus00487.xml",
     "changeIdentifier":"PM_MEAS_FILES",
     "objectStoreBucket":"ropfiles"
  }


*************
Configuration
*************

The DFC is configured via its application.yaml


An example application.yaml configuration file: ":download:`link <../config/application.yaml>`"