Improve auto test for policy agent
Auto test for policy agent remote/local image
Return basic health check in the printout
Add test description files.
Issue-ID: NONRTRIC-79
Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
Change-Id: I1dbdfbcc2568a1b46dae5d42ebcb8329c818f3ac
diff --git a/near-rt-ric-simulator/auto-test/FTC1.sh b/near-rt-ric-simulator/auto-test/FTC1.sh
index 180682b..f49a179 100755
--- a/near-rt-ric-simulator/auto-test/FTC1.sh
+++ b/near-rt-ric-simulator/auto-test/FTC1.sh
@@ -1,9 +1,11 @@
#!/usr/bin/env bash
-TC_ONELINE_DESCR="Auto test for policy agent with simulator"
+TC_ONELINE_DESCR="Auto test for policy agent refreshing configurations from consul/cbs"
. ../common/testcase_common.sh $1 $2
+#### TEST BEGIN ####
+
clean_containers
start_simulators
diff --git a/near-rt-ric-simulator/auto-test/README.md b/near-rt-ric-simulator/auto-test/README.md
new file mode 100644
index 0000000..676f079
--- /dev/null
+++ b/near-rt-ric-simulator/auto-test/README.md
@@ -0,0 +1,69 @@
+## Automated test Description
+This auto-test repo stores test script for automated test cases for policy agent.
+Each of the testcase script will bring up a containerized test enviroment for Policy Agent,
+CBS, consul, and simulator(TBD)
+
+### Overveiw
+
+Right now, test cases are written in bash scripts. \
+Each test case script(ex. `FTC1.sh)` will call functions defined in `../common`. \
+The environment vriables are set in`test_env.sh`. \
+The automated test support both local build policy agent image testing and remote image stored in Nexus.
+```
+# Lcal image
+export POLICY_AGENT_LOCAL_IMAGE=o-ran-sc/nonrtric-policy-agent
+# Remote image
+export POLICY_AGENT_REMOTE_IMAGE=nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-policy-agent
+```
+### Test Cases Description(more TBD)
+`FTC1.sh`: Test policy-agent can refresh configurations from consul
+
+### Logs
+All log files are stored at `logs/<testcase id>`. \
+The logs include the application.log and the container log from policy agent, the container logs from each simulator and the
+test case log (same as the screen output). \
+In the test cases the logs are stored with a prefix so the logs can be stored at different steps during the test.
+All test cases contains an entry to save all logs with prefix 'END' at the end of each test case.
+
+### Manual
+Test case command:
+```
+./<testcase-id>.sh local | remote
+
+Discription:
+local: test image: POLICY_AGENT_LOCAL_IMAGE=o-ran-sc/nonrtric-policy-agent
+remote: test image: nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-policy-agent
+```
+
+### Test case file
+A test case file contains a number of steps to verify a certain functionality.
+A description of the test case should be given to the ``TC_ONELINE_DESCR`` var. The description will be printed in
+the test result.
+
+The empty template for a test case files looks like this:
+
+(Only the parts noted with < and > shall be changed.)
+
+-----------------------------------------------------------
+```
+#!/usr/bin/env bash
+
+TC_ONELINE_DESCR="<test case description>"
+
+. ../common/testcase_common.sh $1
+
+#### TEST BEGIN ####
+
+
+<tests here>
+
+
+#### TEST COMPLETE ####
+
+store_logs END
+
+```
+-----------------------------------------------------------
+
+The ../common/testcase_common.sh contains all functions needed for the test case file. See the README.md file in
+the ../common dir for a description of all available functions.
\ No newline at end of file
diff --git a/near-rt-ric-simulator/common/README.md b/near-rt-ric-simulator/common/README.md
new file mode 100644
index 0000000..37d81ec
--- /dev/null
+++ b/near-rt-ric-simulator/common/README.md
@@ -0,0 +1,32 @@
+## Description for common test scripts
+
+`test_env.sh` \
+Common env variables for test in the auto-test dir.
+Used by the auto test cases/suites but could be used for other test script as well.
+
+`testcase_common.sh` \
+Common functions for auto test cases in the auto-test dir.
+A subset of the functions could be used in other test scripts as well.
+
+###Descriptions of functions in testcase_common.sh
+
+`clean_containers` \
+Stop and remove all containers including policy agent apps and simulators
+
+`start_simulators` \
+Start all simulators in the simulator group
+
+`consul_config_app ` \
+Configure consul with json file with app config for a policy agent instance using the policy agent
+instance id and the json file.
+
+`start_policy_agent` \
+Start the policy agent application.
+
+`check_policy_agent_logs`
+Check the Policy agent application log for WARN and ERR messages and print the count.
+`store_logs`
+Store all Policy agent app and simulators log to the test case log dir. All logs get a prefix to
+separate logs stored at different steps in the test script.
+If logs need to be stored in several locations, use different prefix to easily identify the location
+when the logs where taken.
diff --git a/near-rt-ric-simulator/common/test_env.sh b/near-rt-ric-simulator/common/test_env.sh
index 363f879..b307a69 100755
--- a/near-rt-ric-simulator/common/test_env.sh
+++ b/near-rt-ric-simulator/common/test_env.sh
@@ -1,15 +1,18 @@
#!/usr/bin/env bash
+# Set the images for the Policy agent app to use for the auto tests. Do not add the image tag.
+#
# Local image and tag, shall point to locally built image (non-nexus path)
export POLICY_AGENT_LOCAL_IMAGE=o-ran-sc/nonrtric-policy-agent
-
+# Remote image
+export POLICY_AGENT_REMOTE_IMAGE=nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-policy-agent
# Common env var for auto-test.
POLICY_AGENT_PORT=8081
POLICY_AGENT_LOGPATH="/var/log/policy-agent/application.log" #Path the application log in the policy agent container
-DOCKER_SIM_NWNAME="nonrtric-docker-net" #Name of docker private network
-CONSUL_HOST="consul-server" #Host name of consul
-CONSUL_PORT=8500 #Port number of consul
-CONFIG_BINDING_SERVICE="config-binding-service" #Host name of CBS
-PA_APP_BASE="policy-agent"
+DOCKER_SIM_NWNAME="nonrtric-docker-net" #Name of docker private network
+CONSUL_HOST="consul-server" #Host name of consul
+CONSUL_PORT=8500 #Port number of consul
+CONFIG_BINDING_SERVICE="config-binding-service" #Host name of CBS
+PA_APP_BASE="policy-agent" #Base name for policy agent container
diff --git a/near-rt-ric-simulator/common/testcase_common.sh b/near-rt-ric-simulator/common/testcase_common.sh
index 6f2097e..7a8d8dc 100755
--- a/near-rt-ric-simulator/common/testcase_common.sh
+++ b/near-rt-ric-simulator/common/testcase_common.sh
@@ -4,12 +4,16 @@
echo "Test case started as: ${BASH_SOURCE[$i+1]} "$1 $2
+# This is a script that contains all the functions needed for auto test
+# Arg: local | remote
+
STARTED_POLICY_AGENT="" #Policy agent app names added to this var to keep track of started container in the script
START_ARG=$1
IMAGE_TAG="1.0.0-SNAPSHOT"
+IMAGE_TAG_REMOTE="1.0.0"
if [ $# -lt 1 ] || [ $# -gt 2 ]; then
- echo "Expected arg: local [<image-tag>] ]| remote [<image-tag>] ]| remote-remove [<image-tag>]] | manual-container | manual-app"
+ echo "Expected arg: local [<image-tag>] | remote [<image-tag>] "
exit 1
elif [ $1 == "local" ]; then
if [ -z $POLICY_AGENT_LOCAL_IMAGE ]; then
@@ -17,6 +21,12 @@
exit 1
fi
POLICY_AGENT_IMAGE=$POLICY_AGENT_LOCAL_IMAGE":"$IMAGE_TAG
+elif [ $1 == "remote" ]; then
+ if [ -z $POLICY_AGENT_REMOTE_IMAGE ]; then
+ echo "POLICY_AGENT_REMOTE_IMAGE not set in test_env"
+ exit 1
+ fi
+ POLICY_AGENT_IMAGE=$POLICY_AGENT_REMOTE_IMAGE":"$IMAGE_TAG_REMOTE
fi
# Set a description string for the test case
@@ -72,17 +82,31 @@
echo ""
if [ $1 != "manual-container" ] && [ $1 != "manual-app" ]; then
- echo -e "Policy agent image tag set to: \033[1m" $IMAGE_TAG"\033[0m"
- echo "Configured image for policy agent app(s) (${1}): "$POLICY_AGENT_LOCAL_IMAGE
- tmp_im=$(docker images ${POLICY_AGENT_LOCAL_IMAGE} | grep -v REPOSITORY)
+ #echo -e "Policy agent image tag set to: \033[1m" $IMAGE_TAG"\033[0m"
+ echo "Configured image for policy agent app(s) (${1}): "$POLICY_AGENT_IMAGE
+ tmp_im=$(docker images ${POLICY_AGENT_IMAGE} | grep -v REPOSITORY)
if [ $1 == "local" ]; then
if [ -z "$tmp_im" ]; then
- echo "Local image (non nexus) "$POLICY_AGENT_LOCAL_IMAGE" does not exist in local registry, need to be built"
+ echo "Local image (non nexus) "$POLICY_AGENT_IMAGE" does not exist in local registry, need to be built"
exit 1
else
echo -e "Policy agent local image: \033[1m"$tmp_im"\033[0m"
- echo "If the policy agen image seem outdated, rebuild the image and run the test again."
+ echo "If the policy agent image seem outdated, rebuild the image and run the test again."
+ fi
+ elif [ $1 == "remote" ]; then
+ if [ -z "$tmp_im" ]; then
+ echo "Pulling policy agent image from nexus: "$POLICY_AGENT_IMAGE
+ docker pull $POLICY_AGENT_IMAGE > /dev/null
+ tmp_im=$(docker images ${POLICY_AGENT_IMAGE} | grep -v REPOSITORY)
+ if [ -z "$tmp_im" ]; then
+ echo "Image could not be pulled"
+ exit 1
+ fi
+ echo -e "Policy Agent image: \033[1m"$tmp_im"\033[0m"
+ else
+ echo -e "Policy Agent image: \033[1m"$tmp_im"\033[0m"
+ echo "!! If the Policy agent image seem outdated, consider removing it from your docker registry and run the test again."
fi
fi
fi
@@ -129,7 +153,7 @@
appname=$PA_APP_BASE
- if [ $START_ARG == "local" ] ; then
+ if [ $START_ARG == "local" ] || [ $START_ARG == "remote" ]; then
__start_policy_agent_image $appname
fi
}
@@ -171,6 +195,7 @@
result="$(__do_curl http://127.0.0.1:${localport}/status)"
if [ $? -eq 0 ]; then
echo "Policy Agent ${appname} responds to service status: " $result
+ echo "Policy Agent is alive."
pa_st=true
break
else
diff --git a/near-rt-ric-simulator/simulator-group/consul_cbs/config.json b/near-rt-ric-simulator/simulator-group/consul_cbs/config.json
index f75b2c5..d63d860 100644
--- a/near-rt-ric-simulator/simulator-group/consul_cbs/config.json
+++ b/near-rt-ric-simulator/simulator-group/consul_cbs/config.json
@@ -1,13 +1,21 @@
- {
- "//description": "Application configuration",
- "ric": [
- {
- "name": "ric1",
- "baseUrl": "http://localhost:8080/",
- "managedElementIds": [
- "kista_1",
- "kista_2"
- ]
- }
- ]
- }
+{
+ "//description": "Application configuration",
+ "ric": [
+ {
+ "name": "ric1",
+ "baseUrl": "http://localhost:8080/",
+ "managedElementIds": [
+ "kista_1",
+ "kista_2"
+ ]
+ },
+ {
+ "name": "ric2",
+ "baseUrl": "http://localhost:8081/",
+ "managedElementIds": [
+ "kista_3",
+ "kista_4"
+ ]
+ }
+ ]
+}