Update DFC tests to use file based conifg
To remove Consul/CBS dependency in DCAE DFC was updated
to use file based configuration
CSIT updates:
- file based config use in DFC
- removal of Consul/CBS from DFC tests
Issue-ID: DCAEGEN2-2692
Signed-off-by: Piotr Marcinkiewicz <piotr.marcinkiewicz@nokia.com>
Change-Id: I9d84559f3650fc8554c317ae8eab09c76662de09
diff --git a/plans/dcaegen2-collectors-datafile/Functional-suite/setup.sh b/plans/dcaegen2-collectors-datafile/Functional-suite/setup.sh
index e85c0ee..2dc7e94 100644
--- a/plans/dcaegen2-collectors-datafile/Functional-suite/setup.sh
+++ b/plans/dcaegen2-collectors-datafile/Functional-suite/setup.sh
@@ -22,6 +22,9 @@
echo "Using SIM_ROOT from environmental variable: " $SIM_ROOT
fi
+#Prepare DFC config volume
+mkdir $SIM_ROOT/simulator-group/dfc_config_volume || true
+
#Location of the above simulators when run as a group. For start+config and stop.
SIMGROUP_ROOT=$SIM_ROOT/simulator-group
diff --git a/plans/dcaegen2-collectors-datafile/Functional-suite/teardown.sh b/plans/dcaegen2-collectors-datafile/Functional-suite/teardown.sh
index 81d85fe..c05f356 100644
--- a/plans/dcaegen2-collectors-datafile/Functional-suite/teardown.sh
+++ b/plans/dcaegen2-collectors-datafile/Functional-suite/teardown.sh
@@ -5,6 +5,8 @@
docker exec dfc_app0 cat /var/log/ONAP/application.log >> $WORKSPACE/archives/dfc_app0_application.log
docker logs dfc_mr-sim >> $WORKSPACE/archives/dfc_mr-sim.log
+rm -rf $SIM_ROOT/simulator-group/dfc_config_volume || true
+
if [ -z "$running_containers" ]
then
echo "No container requires termination"
diff --git a/scripts/dcaegen2-collectors-datafile/dfc-management/docker-compose.yml b/scripts/dcaegen2-collectors-datafile/dfc-management/docker-compose.yml
index 2ea2882..14bfad0 100644
--- a/scripts/dcaegen2-collectors-datafile/dfc-management/docker-compose.yml
+++ b/scripts/dcaegen2-collectors-datafile/dfc-management/docker-compose.yml
@@ -17,6 +17,7 @@
dfcnet: null
volumes:
- $SIMGROUP_ROOT/tls/:/opt/app/datafile/etc/cert/
+ - $SIMGROUP_ROOT/dfc_config_volume:/app-config
environment:
CONSUL_HOST: "consul-server"
CONSUL_PORT: 8500
diff --git a/tests/dcaegen2-collectors-datafile/resources/common-keywords.robot b/tests/dcaegen2-collectors-datafile/resources/common-keywords.robot
index c425fe2..b872806 100644
--- a/tests/dcaegen2-collectors-datafile/resources/common-keywords.robot
+++ b/tests/dcaegen2-collectors-datafile/resources/common-keywords.robot
@@ -13,6 +13,7 @@
${CLI_DRSIM_CTR_PUBLISHED_FILES} curl --connect-timeout 10 -X GET http://${SIM_IP}:3906/ctr_published_files
${CLI_DR_REDIR_SIM_DOWNLOADED_VOLUME} curl --connect-timeout 10 -X GET http://${SIM_IP}:3908/dwl_volume
+${DFC_CONFIG_VOLUME_FILE} ${SIMGROUP_ROOT}/dfc_config_volume/application_config.yaml
*** Keywords ***
MR Sim Emitted Files Equal
@@ -45,6 +46,13 @@
Log To Console Dfc-start:
Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
+Set DFC config
+ [Documentation] Set DFC configuration
+ [Arguments] ${dfc_config_file}
+ Copy File ${dfc_config_file} ${DFC_CONFIG_VOLUME_FILE}
+ ${dfc_config_file_content}= Get File ${DFC_CONFIG_VOLUME_FILE}
+ Log To Console APP configuration:
+ Log To Console ${dfc_config_file_content}
Test Teardown
[Documentation] Cleanup containers
diff --git a/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Ftp-suite/FuncSingleFileFtp.robot b/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Ftp-suite/FuncSingleFileFtp.robot
index bb70b50..35bacd9 100755
--- a/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Ftp-suite/FuncSingleFileFtp.robot
+++ b/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Ftp-suite/FuncSingleFileFtp.robot
@@ -6,9 +6,7 @@
Resource ../../resources/common-keywords.robot
*** Variables ***
-${CONSUL_UPL_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?dc=dc1 -X PUT -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'X-Requested-With: XMLHttpRequest' --data-binary @${SIMGROUP_ROOT}/consul/c12_feed2_PM_MEAS.json
-${CONSUL_GET_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?raw
-${CBS_GET_MERGED_CONFIG} /usr/bin/curl -v http://127.0.0.1:10000/service_component_all/dfc_app0
+${DFC_CONFIG_FILE} ${SIMGROUP_ROOT}/dfc_configs/c12_feed2_PM_MEAS.yaml
*** Test Cases ***
@@ -64,19 +62,7 @@
MR Sim Emitted Files Equal 0 #Verify 0 file emitted from MR sim
DR Sim Published Files Equal 0 #Verify 0 file published to DR sim
- ${cli_cmd_output}= Run Process ${CONSUL_UPL_APP} shell=yes
- Log To Console Consul APP write:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CONSUL_GET_APP} shell=yes
- Log To Console Consul APP read:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CBS_GET_MERGED_CONFIG} shell=yes
- Log To Console CBS merged configuration:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- Sleep 10
+ Set DFC config ${DFC_CONFIG_FILE}
Start DFC
diff --git a/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Http-suite/FuncSingleFileHttp.robot b/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Http-suite/FuncSingleFileHttp.robot
index f96c2f9..26dcef4 100755
--- a/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Http-suite/FuncSingleFileHttp.robot
+++ b/tests/dcaegen2-collectors-datafile/testsuites/Functional-Single-File-Http-suite/FuncSingleFileHttp.robot
@@ -6,9 +6,7 @@
Resource ../../resources/common-keywords.robot
*** Variables ***
-${CONSUL_UPL_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?dc=dc1 -X PUT -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'X-Requested-With: XMLHttpRequest' --data-binary @${SIMGROUP_ROOT}/consul/c12_feed2_PM_HTTPS.json
-${CONSUL_GET_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?raw
-${CBS_GET_MERGED_CONFIG} /usr/bin/curl -v http://127.0.0.1:10000/service_component_all/dfc_app0
+${DFC_CONFIG_FILE} ${SIMGROUP_ROOT}/dfc_configs/c12_feed2_PM_HTTPS.yaml
*** Test Cases ***
@@ -67,19 +65,7 @@
MR Sim Emitted Files Equal 0 #Verify 0 file emitted from MR sim
DR Sim Published Files Equal 0 #Verify 0 file published to DR sim
- ${cli_cmd_output}= Run Process ${CONSUL_UPL_APP} shell=yes
- Log To Console Consul APP write:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CONSUL_GET_APP} shell=yes
- Log To Console Consul APP read:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CBS_GET_MERGED_CONFIG} shell=yes
- Log To Console CBS merged configuration:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- Sleep 10
+ Set DFC config ${DFC_CONFIG_FILE}
Start DFC
diff --git a/tests/dcaegen2-collectors-datafile/testsuites/HTTP-Various-Connection-Types-suite/HttpVariousConnectionTypes.robot b/tests/dcaegen2-collectors-datafile/testsuites/HTTP-Various-Connection-Types-suite/HttpVariousConnectionTypes.robot
index 9e69e53..d30e736 100755
--- a/tests/dcaegen2-collectors-datafile/testsuites/HTTP-Various-Connection-Types-suite/HttpVariousConnectionTypes.robot
+++ b/tests/dcaegen2-collectors-datafile/testsuites/HTTP-Various-Connection-Types-suite/HttpVariousConnectionTypes.robot
@@ -6,9 +6,7 @@
Resource ../../resources/common-keywords.robot
*** Variables ***
-${CONSUL_UPL_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?dc=dc1 -X PUT -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'X-Requested-With: XMLHttpRequest' --data-binary @${SIMGROUP_ROOT}/consul/c12_feed2_PM_HTTPS.json
-${CONSUL_GET_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?raw
-${CBS_GET_MERGED_CONFIG} /usr/bin/curl -v http://127.0.0.1:10000/service_component_all/dfc_app0
+${DFC_CONFIG_FILE} ${SIMGROUP_ROOT}/dfc_configs/c12_feed2_PM_HTTPS.yaml
*** Test Cases ***
@@ -61,19 +59,7 @@
MR Sim Emitted Files Equal 0 #Verify 0 file emitted from MR sim
DR Sim Published Files Equal 0 #Verify 0 file published to DR sim
- ${cli_cmd_output}= Run Process ${CONSUL_UPL_APP} shell=yes
- Log To Console Consul APP write:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CONSUL_GET_APP} shell=yes
- Log To Console Consul APP read:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CBS_GET_MERGED_CONFIG} shell=yes
- Log To Console CBS merged configuration:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- Sleep 10
+ Set DFC config ${DFC_CONFIG_FILE}
Start DFC
diff --git a/tests/dcaegen2-collectors-datafile/testsuites/Strict-Host-Checking-suite/StrictHostChecking.robot b/tests/dcaegen2-collectors-datafile/testsuites/Strict-Host-Checking-suite/StrictHostChecking.robot
index 61f689d..e7a5585 100755
--- a/tests/dcaegen2-collectors-datafile/testsuites/Strict-Host-Checking-suite/StrictHostChecking.robot
+++ b/tests/dcaegen2-collectors-datafile/testsuites/Strict-Host-Checking-suite/StrictHostChecking.robot
@@ -8,10 +8,8 @@
Test Teardown
*** Variables ***
-${CONSUL_UPL_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?dc=dc1 -X PUT -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'X-Requested-With: XMLHttpRequest' --data-binary @${SIMGROUP_ROOT}/consul/c12_feed2_PM_MEAS.json
-${CONSUL_UPL_APP_INSECURE_SFTP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?dc=dc1 -X PUT -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'X-Requested-With: XMLHttpRequest' --data-binary @${SIMGROUP_ROOT}/consul/c12_feed2_PM_MEAS_no_strict_host_key_checking.json
-${CONSUL_GET_APP} /usr/bin/curl -v http://127.0.0.1:8500/v1/kv/dfc_app0?raw
-${CBS_GET_MERGED_CONFIG} /usr/bin/curl -v http://127.0.0.1:10000/service_component_all/dfc_app0
+${DFC_CONFIG_FILE} ${SIMGROUP_ROOT}/dfc_configs/c12_feed2_PM_MEAS.yaml
+${DFC_CONFIG_FILE_INSECURE_SFTP} ${SIMGROUP_ROOT}/dfc_configs/c12_feed2_PM_MEAS_no_strict_host_key_checking.yaml
*** Test Cases ***
@@ -20,7 +18,7 @@
Verify single event with SFTP file, when host known and strict host key checking enabled. From event poll to published file
[TAGS] DFC_STRICT_HOST_KEY_CHECKING_1
[Documentation] Verify single event with SFTP file, when host known and strict host key checking enabled. From event poll to published file.
- [Setup] Setup Strict Host Key Checking Test ${CONSUL_UPL_APP} all_hosts_keys
+ [Setup] Setup Strict Host Key Checking Test ${DFC_CONFIG_FILE} all_hosts_keys
Wait Until Keyword Succeeds 1 minute 10 sec MR Sim Emitted Files Equal 1 #Verify 1 file emitted from MR sim
Wait Until Keyword Succeeds 1 minute 10 sec DR Sim Query Not Published Equal 1 #Verify 1 query response for not published files
@@ -32,7 +30,7 @@
Verify single event with SFTP file, when host unknown and strict host key checking disabled. From event poll to published file
[TAGS] DFC_STRICT_HOST_KEY_CHECKING_2
[Documentation] Verify single event with SFTP file, when host unknown and strict host key checking disabled. From event poll to published file.
- [Setup] Setup Strict Host Key Checking Test ${CONSUL_UPL_APP_INSECURE_SFTP} known_hosts_empty
+ [Setup] Setup Strict Host Key Checking Test ${DFC_CONFIG_FILE_INSECURE_SFTP} known_hosts_empty
Wait Until Keyword Succeeds 1 minute 10 sec MR Sim Emitted Files Equal 1 #Verify 1 file emitted from MR sim
Wait Until Keyword Succeeds 1 minute 10 sec DR Sim Query Not Published Equal 1 #Verify 1 query response for not published files
@@ -44,7 +42,7 @@
Verify single event with SFTP file, when no known hosts file and strict host key checking enabled. From event poll to published file
[TAGS] DFC_STRICT_HOST_KEY_CHECKING_3
[Documentation] Verify single event with SFTP file, when host unknown and strict host key checking enabled. File not published.
- [Setup] Setup Strict Host Key Checking Test ${CONSUL_UPL_APP} no_known_hosts_file
+ [Setup] Setup Strict Host Key Checking Test ${DFC_CONFIG_FILE} no_known_hosts_file
Wait Until Keyword Succeeds 1 minute 10 sec MR Sim Emitted Files Equal 1 #Verify 1 file emitted from MR sim
Wait Until Keyword Succeeds 1 minute 10 sec DR Sim Query Not Published Equal 1 #Verify 1 query response for not published files
@@ -58,7 +56,7 @@
Verify single event with SFTP file, when host unknown and strict host key checking enabled. File not published
[TAGS] DFC_STRICT_HOST_KEY_CHECKING_4
[Documentation] Verify single event with SFTP file, when host unknown and strict host key checking enabled. File not published.
- [Setup] Setup Strict Host Key Checking Test ${CONSUL_UPL_APP} known_hosts_empty
+ [Setup] Setup Strict Host Key Checking Test ${DFC_CONFIG_FILE} known_hosts_empty
Wait Until Keyword Succeeds 1 minute 10 sec MR Sim Emitted Files Equal 1 #Verify 1 file emitted from MR sim
Wait Until Keyword Succeeds 1 minute 10 sec DR Sim Query Not Published Equal 1 #Verify 1 query response for not published files
Sleep 60
@@ -69,7 +67,7 @@
Setup Strict Host Key Checking Test
[Documentation] Sets up strict host key checking test with single 1MB file
- [Arguments] ${consul_config_request} ${known_hosts_file}
+ [Arguments] ${dfc_config_path} ${known_hosts_file}
Set Environment Variable MR_TC --tc100
Set Environment Variable DR_TC --tc normal
Set Environment Variable DR_REDIR_TC --tc normal
@@ -102,19 +100,7 @@
MR Sim Emitted Files Equal 0 #Verify 0 file emitted from MR sim
DR Sim Published Files Equal 0 #Verify 0 file published to DR sim
- ${cli_cmd_output}= Run Process ${consul_config_request} shell=yes
- Log To Console Consul APP write:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CONSUL_GET_APP} shell=yes
- Log To Console Consul APP read:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- ${cli_cmd_output}= Run Process ${CBS_GET_MERGED_CONFIG} shell=yes
- Log To Console CBS merged configuration:
- Log To Console ${cli_cmd_output.stdout} ${cli_cmd_output.stderr}
-
- Sleep 10
+ Set DFC config ${dfc_config_path}
${cli_cmd_output}= Run Process ${DFC_ROOT}/dfc-start.sh cwd=${DFC_ROOT} env:KNOWN_HOSTS=${known_hosts_file} env:SIMGROUP_ROOT=${SIMGROUP_ROOT}
Log To Console Dfc-start: